Tom: I thin my comments were that grouping on a field where there was a unique value _per document_ chewed up a lot of resources. Conceptually, there's a bucket for each unique group value. And grouping on a file path is just asking for trouble.
But the memory used for grouping should max as a function of the unique values in the grouped field. Best Erick On Wed, Aug 22, 2012 at 11:32 PM, Lance Norskog <goks...@gmail.com> wrote: > Yes, distributed grouping works, but grouping takes a lot of > resources. If you can avoid in distributed mode, so much the better. > > On Wed, Aug 22, 2012 at 3:35 PM, Tom Burton-West <tburt...@umich.edu> wrote: >> Thanks Tirthankar, >> >> So the issue in memory use for sorting. I'm not sure I understand how >> sorting of grouping fields is involved with the defaults and field >> collapsing, since the default sorts by relevance not grouping field. On >> the other hand I don't know much about how field collapsing is implemented. >> >> So far the few tests I've made haven't revealed any memory problems. We >> are using very small string fields for grouping and I think that we >> probably only have a couple of cases where we are grouping more than a few >> thousand docs. I will try to find a query with a lot of docs per group >> and take a look at the memory use using JConsole. >> >> Tom >> >> >> On Wed, Aug 22, 2012 at 4:02 PM, Tirthankar Chatterjee < >> tchatter...@commvault.com> wrote: >> >>> Hi Tom,**** >>> >>> We had an issue where we are keeping millions of docs in a single node and >>> we were trying to group them on a string field which is nothing but full >>> file path… that caused SOLR to go out of memory…**** >>> >>> ** ** >>> >>> Erick has explained nicely in the thread as to why it won’t work and I had >>> to find another way of architecting it. **** >>> >>> ** ** >>> >>> How do you think this is different in your case. If you want to group by a >>> string field with thousands of similar entries I am guessing you will face >>> the same issue. **** >>> >>> ** ** >>> >>> Thanks,**** >>> >>> Tirthankar**** >>> ***************************Legal Disclaimer*************************** >>> "This communication may contain confidential and privileged material for >>> the >>> sole use of the intended recipient. Any unauthorized review, use or >>> distribution >>> by others is strictly prohibited. If you have received the message in >>> error, >>> please advise the sender by reply email and delete the message. Thank you." >>> ********************************************************************** >>> > > > > -- > Lance Norskog > goks...@gmail.com
