High load, frequent updates, low latency requirement use case
I'm trying to set up a Solr Cloud cluster to support a system with the
following characteristics:
It will be writing documents at a rate of approximately 500 docs/second,
and running search queries at about the same rate.
The documents are fairly small, with about 10 fields, most of which range
in size from a simple int to a string that holds a UUID. There's a date
field, and then three text fields that typically hold in the range of 350
to 500 chars.
Documents should be available for searching within 30 seconds of being
added.
We need an average search latency of 50 ms or faster.
We've been using DataStax Enterprise with decent results, but trying to
determine if we can get more out of the latest version of Solr Cloud, as we
originally chose DSE ~4 years ago *I believe* because its Cassandra-backed
Solr provided redundancy/high availability features that weren't currently
available with straight Solr (not even sure if Solr Cloud was available
then).
We have 24 fairly beefy servers (96 CPU cores, 256 GB RAM, SSDs) for the
task, and I'm trying to figure out the best way to distribute the documents
into collections, cores, and shards.
If I can categorize a document into one of 8 "types", should I create 8
collections? Is that going to provide better performance than putting them
all into one collection and then using a filter query with the type field
when doing a search?
What are the options/things to consider when deciding on the number of
shards for each collection? As far as I know, I don't choose the number of
Solr cores, that is just determined base on the replication factor (and
shard count?).
Some of the settings I'm using in my solrconfig that seem important:
${solr.lock.type:native}
${solr.autoCommit.maxTime:3}
false
${solr.autoSoftCommit.maxTime:1000}
true
8
I've got the updateLog/transaction log enabled, as I think I read it's
required for Solr Cloud.
Are there any settings I should look at that affect performance
significantly, especially outside of the solrconfig.xml for each collection
(like jetty configs, logging properties, etc)?
How much impact do the directives in the solrconfig have on
performance? Do they only take effect if I have something configured that
requires them, and therefore if I'm missing one that I need, I'd get an
error if it's not defined?
Any help will be greatly appreciated. Thanks!
-Brent
SolrJ Solr Cloud timeout
Is there any way to set a timeout with a CloudSolrClient?
Mailing list subscriptions
Is there a way to subscribe to just responses to a question I ask on the mailing list, without getting emails for all activity on the mailing list? You'd think it'd be designed in a way that when someone submits a question, they automatically get emails for any responses.
Adding request parameters when using SolrCloudClient.add()
Using SolrJ, I'm trying to figure out how to include request parameters
when adding a document with SolrCloudClient.add().
Here is what I was doing when using HttpSolrClient instead of
SolrCloudClient:
HttpSolrClient client = new HttpSolrClient.Builder("
http://hostname.com:8983/solr/corename";)
.withHttpClient(HttpClientBuilder.create().build())
.build();
client.getInvariantParams().set("param1_name", "param1_value");
client.getInvariantParams().set("param2_name", "param2_value");
SolrInputDocument sDoc = new SolrInputDocument();
// Populate sDoc fields.
UpdateResponse response = client.add(sDoc);
SolrCloudClient doesn't appear to have any methods for adding parameters,
so I'm not sure how I can do it. When doing reads, I add parameters to the
SolrQuery object, but when adding a SolrInputDocument object, I don't have
that option.
The only thing I can think of is that maybe adding them as fields in the
SolrInputDocument object would do the trick, but would that actually work
the same?
leader/replica update error + client "expected mime type/stale state" error
I'm running Solr Cloud 6.1.0, with a Java client using SolrJ 5.4.1. Every once in awhile, during a query, I get a pair of messages logged in the client from CloudSolrClient -- an error about a request failing, then a warning saying that it's retrying after a stale state error. For this test, the collection (test_collection) has one shard, with RF=2. There are two machines, 10.112.7.2 (replica) and 10.112.7.4 (leader). The client is on 10.112.7.4. Note that the system time on 10.112.7.4 is about 1 minute, 5-6 seconds ahead of the other machine. --- Leader (10.112.7.4) Solr log: --- 19:27:16.583 ERROR [c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica2] o.a.s.u.StreamingSolrClients error org.apache.http.NoHttpResponseException: 10.112.7.2:8983 failed to respond at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251) at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:311) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:185) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$22(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 19:27:16.587 WARN [c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica2] o.a.s.u.p.DistributedUpdateProcessor Error sending update to http://10.112.7.2:8983/solr org.apache.http.NoHttpResponseException: 10.112.7.2:8983 failed to respond at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:143) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251) at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:311) at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:185) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambd
