Re: Very slow first query

2009-09-11 Thread Jonathan Ariel
The index is 8GB and I'm giving it 1,5 GB of RAM On Fri, Sep 11, 2009 at 5:09 PM, Lance Norskog wrote: > This sounds like a memory-handling problem. The JVM could be too > small, forcing a lot of garbage collections during the first search. > It could be too big and choke off the OS disk cache.

Re: Very slow first query

2009-09-11 Thread Lance Norskog
This sounds like a memory-handling problem. The JVM could be too small, forcing a lot of garbage collections during the first search. It could be too big and choke off the OS disk cache. It could be too big and cause paging. Does this search query include a sort command? Sorting creates a large da

Re: Very slow first query

2009-09-11 Thread Jonathan Ariel
Ok thanks, if it's the IO OS Disk cache, which would be my options? changing the disk to a faster one? On Fri, Sep 11, 2009 at 1:32 PM, Yonik Seeley < yonik.see...@lucidimagination.com> wrote: > At the Lucene level there is the term index and the norms too: > > http://search.lucidimagination.com/

Re: Very slow first query

2009-09-11 Thread Yonik Seeley
At the Lucene level there is the term index and the norms too: http://search.lucidimagination.com/search/document/b5eee1fc75cc454c/caching_in_lucene But 50s? That would seem to indicate it's the OS disk cache and you're waiting for IO. You should be able to confirm if you're IO bound by simply lo

Re: Very slow first query

2009-09-11 Thread Jonathan Ariel
yes of course. but in my case I'm not using filter queries nor facets. it is a really simple query. actually the query params are like this: ?q=location_country:1 AND category:377 AND location_state:"CA" and location_city:"Sacramento" location_country is an integer category is an integer location_

Re: Very slow first query

2009-09-11 Thread Uri Boness
"Not having any facet" and "Not using a filter cache" are two different things. If you're not using query filters, you can still have facet calculated and returned as part of the search result. The facet component uses lucene's field cache to retrieve values for the facet field. Jonathan Ariel

Re: Very slow first query

2009-09-10 Thread Jonathan Ariel
Yes, but in this case the query that I'm executing doesn't have any facet. I mean for this query I'm not using any filter cache.What does it means "operating system cache can be significant"? That my first query uploads a big chunk on the index into memory (maybe even the entire index)? On Thu, Se

Re: Very slow first query

2009-09-10 Thread Yonik Seeley
At 12M documents, operating system cache can be significant. Also, the first time you sort or facet on a field, a field cache instance is populated which can take a lot of time. You can prevent slow first queries by configuring a static warming query in solrconfig.xml that includes the common sort

Very slow first query

2009-09-10 Thread Jonathan Ariel
Hi!Why would it take for the first query that I execute almost 60 seconds to run and after that no more than 50ms? I disabled all my caching to check if it is the reason for the subsequent fast responses, but the same happens. I'm using solr 1.3. Something really strange is that it doesn't happen w