RE: Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation...

2017-02-13 Thread Bryant, Michael
solr-user@lucene.apache.org Subject: Re: Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation... FYI, I just opened https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSOLR-10122&data=01%7C01%7Cmichael.brya

Re: Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation...

2017-02-10 Thread Yonik Seeley
FYI, I just opened https://issues.apache.org/jira/browse/SOLR-10122 for this -Yonik On Fri, Feb 10, 2017 at 4:32 PM, Yonik Seeley wrote: > On Thu, Feb 9, 2017 at 6:58 AM, Bryant, Michael > wrote: >> Hi all, >> >> I'm converting my legacy facets to JSON facets and am seeing much better >> perfor

Re: Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation...

2017-02-10 Thread Yonik Seeley
On Thu, Feb 9, 2017 at 6:58 AM, Bryant, Michael wrote: > Hi all, > > I'm converting my legacy facets to JSON facets and am seeing much better > performance, especially with high cardinality facet fields. However, the one > issue I can't seem to resolve is excessive memory usage (and OOM errors)

Re: Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation...

2017-02-10 Thread Bryant, Michael
Darn, spoke too soon. Field collapsing throws off my facet counts where facet fields differ within groups. Back to the drawing board. FWIW, I tried hyperloglog for JSON facet aggregate counts and it has the same issue as unique() when used as the facet sort parameter - while reasonably fast it

Re: Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation...

2017-02-10 Thread Bryant, Michael
Hi Tom, Well the collapsing query parser is… a much better solution to my problems! Thanks for cluing me in to this, I love it when you can delete a load of hacks for something both simpler and faster. Best, ~Mike -- Mike Bryant Research Associate Department of Digital Humanities King’s

Re: Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation...

2017-02-10 Thread Tom Evans
Hi Mike Looks like you are trying to get a list of the distinct item ids in a result set, ordered by the most frequent item ids? Can you use collapsing qparser for this instead? Should be much quicker. https://cwiki.apache.org/confluence/display/solr/Collapse+and+Expand+Results Every document w

Simulating group.facet for JSON facets, high mem usage w/ sorting on aggregation...

2017-02-09 Thread Bryant, Michael
Hi all, I'm converting my legacy facets to JSON facets and am seeing much better performance, especially with high cardinality facet fields. However, the one issue I can't seem to resolve is excessive memory usage (and OOM errors) when trying to simulate the effect of "group.facet" to sort face