Inconsistent query times

[drmangrum]

-Running a 2 shard (each with a replica) setup with 100k records on JBoss7.
-Using SolrJ and CloudSolrServer to interface with SOLR.

Executing queries through the admin screen query tool is amazingly fast,
usually less than 40ms.  However, when I execute the queries through my java
application I get wildly inconsistent response times, anywhere from 190ms to
21 seconds.

I've tried tweaking the searcher autowarming and cache settings within
solrconfig, but nothing seems to make a dent.

Any help is appreciated.






--
View this message in context: 
http://lucene.472066.n3.nabble.com/Inconsistent-query-times-tp4138956.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Inconsistent query times

[drmangrum]

I don't think that's the case.  Ignoring the results from the admin screen,
I'm still left with inconsistencies within the SolrJ calls.  

I ran the exact same query (a simple integer equality returning 9 documents)
5 times every 10 seconds through solrJ and I get the following results (3
separate executions):

Run 1
1) 259ms
2) 21030ms
3) 21030ms
4) 27ms
5) 24ms

Run 2
1) 42292ms
2) 26ms
3) 23ms
4) 24ms
5) 22ms

Run 3
1) 252ms
2) 21030ms
3) 21028ms
4) 27ms
5) 24ms

No indexing is occurring, so searchers and caches shouldn't be invalidated,
right?  I'm inclined to believe the fast results are from the cache and the
~250ms results from a fresh query, but what could cause the spikes of 21 and
42 seconds?  Is it possible I'm missing a setting within the construction of
the client-side CloudSolrServer object?

Thanks,
Ryan



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Inconsistent-query-times-tp4138956p4138966.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Inconsistent query times

[drmangrum]

No, I'm not indexing.  My index has been static for most than 2 weeks.

I'm running the exact same query over and over.  The CloudSolrServer is
constructed through spring, build my query using the SolrQuery class, and
then execute a the query a few times in a loop.

Query: atp:20
Start: 0
Rows: 20

Nothing crazy here. 

I'm using SOLR 4.8



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Inconsistent-query-times-tp4138956p4138976.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Inconsistent query times

[drmangrum]

Bumping with a little more information...

I think I've localized the issue to Zookeeper.  

When I query SolrCloud nodes individually, they all run well and perform as
expected; I can execute 1800 queries in 3 seconds.  It's only when I run a
query through zookeeper that problems arise.  I've looked through the
documentation, and done everything suggested (set a heap size, secondary
device for transaction logging), but nothing seems to help.

The really odd thing here is that when I get the pauses, they're always in
multiples of 21 seconds.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Inconsistent-query-times-tp4138956p4142083.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Inconsistent query times

[drmangrum]

I'm using SolrJ's  CloudSolrServer

  
that accepts the address of your zookeeper instances and does the grunt work
for you.


I guess the problem could be with that class.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Inconsistent-query-times-tp4138956p4142091.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Inconsistent query times

[drmangrum]

I've found the issue and I feel rather foolish.

One of my servers wasn't responding.  For some reason it didn't like to be
on port 12000.  As soon as I changed the port number, everything ran
swimmingly.

Thanks for your time. 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Inconsistent-query-times-tp4138956p4142125.html
Sent from the Solr - User mailing list archive at Nabble.com.