Re: Heap space problem with mlt query

2013-06-06 Thread Erick Erickson
Your cache sizes are still much too large. I wouldn't expect the changes you outlined to change anything. And your autowarm sizes are still far too big. The default sizes are 512 and 0 for size and autowarm counts. Try those. In fact, Solr will happily function (admittedly with slower queries) if t

Re: Heap space problem with mlt query

2013-06-06 Thread Varsha Rani
Hi Stavros, I checked it with batchSize=-1, But still the same issue. As my single mlt query is : http://machine_ip:8983/solr/News/mlt?q=field1:34358471&qt=/mlt&mlt.match.include=true&mlt=true&mlt.mindf=1&mlt.mintf=1&mlt.minwl=3&mlt.boost=true&fq=cat:"News"; AND date:[136644000 TO 1362827

Re: Heap space problem with mlt query

2013-06-06 Thread Stavros Delisavas
I recently had the same issue which could be fixed very easily. Add the property batchSize="-1" to your -tag. Tell me if that helped. Am 06.06.2013 11:30, schrieb Varsha Rani: Hi, As per suggestions , changed in my config file as : reduced document cache size from 31067 to 16384 and aut

Re: Heap space problem with mlt query

2013-06-06 Thread Varsha Rani
Hi, As per suggestions , changed in my config file as : reduced document cache size from 31067 to 16384 and autowarmcount from 2046 to 1024. My machine RAM size is 16GB , 1 GB RAM used as index of 85GB started. my config file as : 128 I am running 20-25 mlt queries in 1 sec .

Re: Heap space problem with mlt query

2013-06-05 Thread Erick Erickson
To add some numbers to adityab's comment. Each entry in your filter cache will probably consist of maxDocs/8 bytes plus some overhead. Or about 16G. This will only grow as you fire queries at Solr, so it's no surprise you're running out of memory as you process queries. Your documentCache is prob

Re: Heap space problem with mlt query

2013-06-05 Thread Shawn Heisey
On 6/5/2013 3:07 AM, Varsha Rani wrote: > Hi , > > I am having solr index of 80GB with 1 million documents .Each document of > aprx. 500KB . I have a machine with 16GB ram. > > I am running mlt query on 3-5 fields of theses document . > > I am getting solr out of memory problem . This wiki pag

Re: Heap space problem with mlt query

2013-06-05 Thread adityab
Did you try reducing filter and query cache. They are fairly large too unless you really need them to be cached for your use cache. Do you have that many distinct filter queries hitting solr for the size you have defined for filterCache? Are you doing any sorting? as this will chew up a lot of memo

Re: Heap space problem with mlt query

2013-06-05 Thread Varsha Rani
Hi yriveiro, When i was using document cache size=" 131072", i got exception in 5000-6000 mlt queries. But once i done document cache size="16384", i got same problem in 1500-2000 mlt queries. -- View this message in context: http://lucene.472066.n3.nabble.com/Heap-space-problem-with-mlt-

Re: Heap space problem with mlt query

2013-06-05 Thread Yago Riveiro
Varsha, How is the size of your jvm heap? Other question is the document cache. The documentCache does cache of document objects fetched from the disk (http://wiki.apache.org/solr/SolrCaching#documentCache), if each document has 500KB aprx. and you configure a cache of 131072 size, you are

Re: Heap space problem with mlt query

2013-06-05 Thread Varsha Rani
Hi yriveiro, I am using Solr version3.6. My cache config is below : -- View this message in context: http://lucene.472066.n3.nabble.com/Heap-space-problem-with-mlt-query-tp4068278p4068282.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Heap space problem with mlt query

2013-06-05 Thread Yago Riveiro
Varsha, Unless I'm mistaken, the ramBufferSizeMB param is used to do buffering of document before write them to disk. Can you post the cache config that you have in the solrconfig.xml, what version are you using? -- Yago Riveiro Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On We

Re: Heap space problem with mlt query

2013-06-05 Thread Raheel Hasan
and I just asked a similar question just 1 sec ago On Wed, Jun 5, 2013 at 2:07 PM, Varsha Rani wrote: > Hi , > > I am having solr index of 80GB with 1 million documents .Each document of > aprx. 500KB . I have a machine with 16GB ram. > > I am running mlt query on 3-5 fields of theses docu