Re: really slow performance when trying to get facet.field

2012-01-18 Thread Dmitry Kan
Sounds good! So the take away lesson here is to remember cache pre-warming. And of course keep track of RAM allocation :) On Tue, Jan 17, 2012 at 11:23 PM, Daniel Bruegge < daniel.brue...@googlemail.com> wrote: > Ok, I have now changed the static warming in the solrconfig.xml using > first- and n

Re: really slow performance when trying to get facet.field

2012-01-17 Thread Daniel Bruegge
Ok, I have now changed the static warming in the solrconfig.xml using first- and newSearcher. "Content" is my field to facet on. Now the commits take longer, which is OK for me, but the searches are really faster right now. I also reduced the number of documents on my shards to 15mio/shard. So the

Re: really slow performance when trying to get facet.field

2012-01-17 Thread Daniel Bruegge
Evictions are 0 for all cache types. Your server max heap space with 12G is pretty huge. Which is good I think. The CPU on my server is a 8-Core Intel i7 965. Commit frequency is low, because shards are added and old shards exist for historical reasons. Old shards will be then cleaned after coupl

Re: really slow performance when trying to get facet.field

2012-01-17 Thread Daniel Bruegge
Evictions are 0 for all cache types. Your server max heap space with 12G is pretty huge. Which is good I think. The CPU on my server is a 8-Core Intel i7 965. Commit frequency is low, because shards are added and old shards exist for historical reasons. Old shards will be then cleaned after coupl

Re: really slow performance when trying to get facet.field

2012-01-17 Thread Dmitry Kan
Hi Daniel, My index is 6,5G. I'm sure it can be bigger. facet.limit we ask for is beyond 100 thousand. It is sub-second speed. I run it with -Xms1024m -Xmx12000m under tomcat, it currently takes 5,4G of RAM. Amount of docs is over 6,5 million. Do you see any evictions in your caches? What kind of

Re: really slow performance when trying to get facet.field

2012-01-17 Thread Daniel Bruegge
Hi Dmitry, I had everything on one Solr Instance before, but this got to heavy and I had the same issue here, that the 1st facet.query was really slow. When querying the facet: - facet.limit = 100 Cache settings are like this: How big was your index? Did it fit into the RAM wh

Re: really slow performance when trying to get facet.field

2012-01-17 Thread Dmitry Kan
I had a similar problem for a similar task. And in my case merging the results from two shards turned out to be a culprit. If you can logically store your data just in one shard, your faceting should become faster. Size wise it should not be a problem for SOLR. Also, you didn't say anything about

really slow performance when trying to get facet.field

2012-01-17 Thread Daniel Bruegge
Hi, I have 2 Solr-shards. One is filled with approx. 25mio documents (local index 6GB), the other with 10mio documents (2.7GB size). I am trying to create some kind of 'word cloud' to see the frequency of words for a *text_general *field. For this I am currently using a facet over this field and I