Re: Need help with Solr Performance

2008-10-09 Thread Chris Hostetter
: considering you're doing faceting on quite a few fields, the filterCache : is somewhat important. Sorry ... i overlooked the bit where QueryComponent was taking 6.x seconds ... in general knowing what the cache hit rates are looking like is crucial to understanding the performance, but as Ry

Re: Need help with Solr Performance

2008-10-09 Thread Chris Hostetter
Maybe i missed it, but skimming this thread i haven't seen any indication of how your configured the various caches in solrconfig.xml ... or any indication of what kinds of cache hit/miss/expullsion stats you see from stats.jsp after running any tests. considering you're doing faceting on quit

Re: Need help with Solr Performance

2008-10-08 Thread Ryan McKinley
On Oct 8, 2008, at 6:11 PM, Rajiv2 wrote: w/ faceting qtime is around +200ms. if your target time is 250, this will need some work... but lets ignore that for now... qtime for a standard query on the default search field is less than 100ms. Usually around 60ms. qtime for id: i

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
w/ faceting qtime is around +200ms. qtime for a standard query on the default search field is less than 100ms. Usually around 60ms. qtime for id: is around 16ms. ryantxu wrote: > >> - >> >> 6727.0 >> - >> >> 6457.0 >> >> - >> >> 0.0 >> >> - > > So I take it, this is with faceting t

Re: Need help with Solr Performance

2008-10-08 Thread Ryan McKinley
- 6727.0 - 6457.0 - 0.0 - So I take it, this is with faceting turned off... what are your timing results for a simple (not dismax query), perhaps: "id:" ryan

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
yes I'm using 1.3 here are the contents of debug.. I'm only pasting the first explain since the whole debug section is very long. cleaning services cleaning services - +((DisjunctionMaxQuery((about_us:cleaning^0.7 | zip:cleaning^0.7 | (insurances:cleaners insurances:cleaning insurances:clean)

Re: Need help with Solr Performance

2008-10-08 Thread Ryan McKinley
On Oct 8, 2008, at 4:03 PM, Rajiv2 wrote: and query times without faceting are... ? solr's built in faceting is "simple" and has its limits. 15M is higher than i've seen good faceting performance out of, particularly multivalued fields. Erik Hi, My facet fields are multi valued

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
>and query times without faceting are... ? >solr's built in faceting is "simple" and has its limits. 15M is >higher than i've seen good faceting performance out of, particularly >multivalued fields. > > Erik Hi, My facet fields are multi valued and w/o faceting the query time is about

Re: Need help with Solr Performance

2008-10-08 Thread Erik Hatcher
On Oct 8, 2008, at 3:30 PM, Rajiv2 wrote: what is your actual query? Are you doing faceting / highlighting / or anything else? I am doing faceting on 5 fields, no highlighting or anything else, debugging is also off. A basic query that I'm doing using dismax is 'cleaning services' over

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
>what is your actual query? >Are you doing faceting / highlighting / or anything else? I am doing faceting on 5 fields, no highlighting or anything else, debugging is also off. A basic query that I'm doing using dismax is 'cleaning services' over 15 million local business records. -- View thi

Re: Need help with Solr Performance

2008-10-08 Thread Ryan McKinley
what is your actual query? Are you doing faceting / highlighting / or anything else? On Oct 8, 2008, at 2:17 PM, Rajiv2 wrote: Hi, thanks for responding so quickly, 6-12 seconds seems really long and 15 million docs is nothing on a machine like this. Are you sure the issue is in Solr? How

Re: Need help with Solr Performance

2008-10-08 Thread Walter Underwood
One other question: are you using real query logs or a set of unique queries? With real query logs, the caches will warm up after a while (tens of minutes) and performance will improve. With a set of unique queries, you are mostly measuring Solr cache misses. For us, that is about 4X slower, and w

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
Hi, thanks for responding so quickly, >6-12 seconds seems really long and 15 million docs is nothing on a >machine like this. Are you sure the issue is in Solr? How are you >measuring the 6-12 seconds? I'm looking at the value in the Solr response. >Assuming it is Solr... >How often are

Re: Need help with Solr Performance

2008-10-08 Thread Grant Ingersoll
On Oct 8, 2008, at 11:56 AM, Rajiv2 wrote: Hi, I need some recommendations w/ some issues I'm having w/ solr search performance. Here is my index/hardware config: - CentOS on 8 quad core xeon processors @ 3.16 Ghz - 32 GB RAM - Tomcat and JAVA 1.6 - Solr 1.3 ~15 million documents . - Index

Re: Need help with Solr Performance

2008-10-08 Thread Mark Miller
I don't think you can search a 15 million doc index with any kind of query complexity beyond a low freq query term in under .25 seconds unless its a cached hit (in which case it still might not *quite* make it under .25 every time either I'd think). Would love to be proven wrong though You hav