Re: Solr custom document routing

2016-12-04 Thread Erick Erickson
_Why_ do you need to do this? Frankly this sounds like an XY problem, you think the proper solution is partitioning the docs as you're asking about because it'll solve some problem. But you haven't stated what the problem you're trying to solve is. If you're just trying to optimize querying, I _st

Re: Memory leak in Solr

2016-12-04 Thread Erick Erickson
All of this is consistent with not having a properly tuned Solr instance wrt # documents, usage pattern, memory allocated to the JVM, GC settings and the like. Your leader issues can be explained by long GC pauses too. Zookeeper periodically pings each replica it knows about and if the response ti

Re: Memory leak in Solr

2016-12-04 Thread S G
Thank you Eric. Our Solr version is 4.10 and we are not doing any sorting or faceting. I am trying to find some ways of investigating this problem. Hence asking a few more questions to see what are the normal steps taken in such situations. (I did search a few of them on the Internet but could not

Re: Memory leak in Solr

2016-12-04 Thread Walter Underwood
That is a huge heap. Once you have enough heap memory to hold a Java program’s working set, more memory doesn’t make it faster. I just makes the GC take longer. If you have GC monitoring, look at how much memory is in use after a full GC. Add the space for new generation (eden, whatever), then a

field length within BM25 score calculation in Solr 6.3

2016-12-04 Thread Sascha Szott
Hi folks, my Solr index consists of one document with a single valued field "title" of type "text_general". The title field was index with the content: 1 2 3 4 5 6 7 8 9. The field type text_general uses a StandardTokenizer which should result in 9 tokens. The corresponding length of field titl

Re: Memory leak in Solr

2016-12-04 Thread Shawn Heisey
On 12/3/2016 9:46 PM, S G wrote: > The symptom we see is that the java clients querying Solr see response > times in 10s of seconds (not milliseconds). > Some numbers for the Solr Cloud: > > *Overall infrastructure:* > - Only one collection > - 16 VMs used > - 8 shards (1 leader and 1 replica per

Re: Memory leak in Solr

2016-12-04 Thread Jeff Wartes
Here’s an earlier post where I mentioned some GC investigation tools: https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201604.mbox/%3c8f8fa32d-ec0e-4352-86f7-4b2d8a906...@whitepages.com%3E In my experience, there are many aspects of the Solr/Lucene memory allocation model that scale wi

Query on SuggestComponent in SolrCloud(6.2.1).

2016-12-04 Thread Pavan Kumar VVR
Hi, We are in process of migrating our solr environment from standalone Solr(4.9) to SolrCloud(6.2.1). During this process we facing issues with solr.SuggestComponent, which is working fine earlier in 4.9 with solr.TermsComponent PFB detailed description for same and kindly assistance. Let us k