Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

2013-06-05 Thread SandeepM
/So we see the jagged edge waveform which keeps climbing (GC cycles don't completely collect memory over time). Our test has a short capture from real traffic and we are replaying that via solrmeter./ Any idea why the memory climbs over time. The GC should cleanup after data is shipped back. Co

Re: Solr 4.2.1 higher memory footprint vs Solr 3.5

2013-06-04 Thread SandeepM
Thanks Eric and Shawn, Your explanations help understand where SOLR may be spending its time. Sounds like compression can be a CPU and heap hog. (I'll try to confirm this with the heapdumps) Initially we tried to keep the JVM heap sizes the same on both Solr 3.5 and 4.2.1, which was around 3GB ,

Solr 4.2.1 higher memory footprint vs Solr 3.5

2013-06-03 Thread SandeepM
Hi, Using the same schema for both Solr 3.5 and Solr 4.2.1 and posting the same data to both these server, and the memory requirements seem to have gone up sharply during request handling. . Requests come in at around 200QPS. . Document sizes are very large but that did not seem to be a problem w

RE: DirectSolrSpellChecker : vastly varying spellcheck QTime times.

2013-04-24 Thread SandeepM
One of our main concerns is the solr returns the best match based on what it thinks is the best. It uses Levenshtein's distance metrics to determine the best suggestions. Can we tune this to put more weightage on the number of frequency/hits vs the number of edits ? If we can tune this, sugge

RE: DirectSolrSpellChecker : vastly varying spellcheck QTime times.

2013-04-23 Thread SandeepM
James, Is there a way to determine how many times the collations were tried? Is there a parameter that can be issued that can return this in debug information? This would be very helpful. Appreciate your help with this. Thanks. -- Sandeep -- View this message in context: http://lucene.47206

Re: spellcheck: change in behavior and QTime

2013-04-23 Thread SandeepM
I apologize for the length of the previous message. I do see a problem with spellcheck becoming faster (notice QTime). I also see an increase in the number of cache hits if spellcheck=false is run one time followed by the original spellcheck query. Seems like spellcheck=false alters the behavior

RE: DirectSolrSpellChecker : vastly varying spellcheck QTime times.

2013-04-22 Thread SandeepM
Chocolat Factry 0 77 1 0 8 615 chocolate 6544 5 9 15 6 factory 23614 factor 5128

RE: DirectSolrSpellChecker : vastly varying spellcheck QTime times.

2013-04-22 Thread SandeepM
James, Thanks. That was very helpful. That helped me understand count and alternativeTermCount a bit more. I also have the following case as pointed out earlier... My query: http://host/solr/select?q=&spellcheck.q=chocolat%20factry&spellcheck=true&df=spell&fl=&indent=on&wt=xml&rows=10&version=2

spellcheck: change in behavior and QTime

2013-04-22 Thread SandeepM
I am using the same setup (solrconfig.xml and schema.xml) as stated in my prior message: http://lucene.472066.n3.nabble.com/DirectSolrSpellChecker-vastly-varying-spellcheck-QTime-times-tt4057176.html#a4057389 I am using SOLR 4.2.1 . Just wanted to report something wierd that I am seeing and would l

RE: DirectSolrSpellChecker : vastly varying spellcheck QTime times.

2013-04-19 Thread SandeepM
James, Thanks for the reply. I see your point and sure enough, reducing maxCollationTries does reduce time, however may not produce results. It seems like the time is taken for the collations re-runs. Is there any way we can activate caching for collations. The same query repeatedly takes the sa

DirectSolrSpellChecker : vastly varying spellcheck QTime times.

2013-04-18 Thread SandeepM
Hi! I am using SOLR 4.2.1. My solrconfig.xml contains the following: text_spell MySpellchecker spell solr.DirectSolrSpellChecker internal 0.5 2 1 5 3 0.01 10 id MySpell