Re: Field collapsing memory usage

2015-01-23 Thread Toke Eskildsen
On Thu, 2015-01-22 at 22:52 +0100, Erick Erickson wrote: > What do you think about folding this into the Solr (or Lucene?) code > base? Or is it to specialized? (writing under the assumption that DVEnabler actually works as it should for everyone and not just us) Right now it is an explicit tool.

Re: Field collapsing memory usage

2015-01-22 Thread Erick Erickson
Toke: What do you think about folding this into the Solr (or Lucene?) code base? Or is it to specialized? Not sure one way or the other, just askin' Erick On Thu, Jan 22, 2015 at 3:47 AM, Toke Eskildsen wrote: > Norgorn [lsunnyd...@mail.ru] wrote: >> Is there any way to make 'docValues="t

RE: Field collapsing memory usage

2015-01-22 Thread Toke Eskildsen
Norgorn [lsunnyd...@mail.ru] wrote: > Nice, thanks! > If u'd like to, I'll write our results with that amazing util. By all means, please do. Good as well as bad. Independent testing is needed to ensure proper working tools. - Toke Eskildsen

RE: Field collapsing memory usage

2015-01-22 Thread Norgorn
Nice, thanks! If u'd like to, I'll write our results with that amazing util. -- View this message in context: http://lucene.472066.n3.nabble.com/Field-collapsing-memory-usage-tp4181092p4181159.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Field collapsing memory usage

2015-01-22 Thread Toke Eskildsen
Norgorn [lsunnyd...@mail.ru] wrote: > Is there any way to make 'docValues="true"' without reindexing? Depends on how brave you are :-) We recently had the same need and made https://github.com/netarchivesuite/dvenabler To my knowledge that is the only existing tool for that task an as we are the

RE: Field collapsing memory usage

2015-01-22 Thread Norgorn
Thank you for your answer. We've found out that the problem was in our SOLR spec (heliosearch 0.08). There are no crushes, after changing to 4.10.3 (although, there are lot of OOMs while handling query, it's not really strange for 1.1 bil of documents ). Now we are going to try latest Heliosearch.

RE: Field collapsing memory usage

2015-01-21 Thread Toke Eskildsen
Norgorn [lsunnyd...@mail.ru] wrote: > So, as we see, memory, used by first shard to group, wasn't released. > Caches are already nearly zero. It should be one or the other: Either the memory is released or there is something in the caches. Anyway, DocValues is the way to go, so ensure that it tu