Re: Field collapsing, facets, and qtime: caching issue?

2017-02-13 Thread Joel Bernstein
The > consensus across various other solr tickets regarding grouped search seems > to be to prefer the collapse/expand approach to grouping. I'm using > non-grouped search now but would like to switch to grouped and > collapse/expand could work for my use case, but the effective defeat

Re: Field collapsing, facets, and qtime: caching issue?

2017-02-13 Thread ronbraun
proach to grouping. I'm using non-grouped search now but would like to switch to grouped and collapse/expand could work for my use case, but the effective defeat of query caching for any faceted application seems pretty problematic and I'd be hesitant to switch over if I'm effectively l

Re: Field collapsing, facets, and qtime: caching issue?

2017-02-10 Thread Joel Bernstein
experimenting with field collapsing in solrcloud 6.2.1 and have this > set of request parameters against a collection: > > /default?indent=on&q=*:*&wt=json&fq={!collapse+field=groupid} > > My default handler is just defaults: > > > > expli

Field collapsing, facets, and qtime: caching issue?

2017-02-10 Thread Ronald K. Braun
I'm experimenting with field collapsing in solrcloud 6.2.1 and have this set of request parameters against a collection: /default?indent=on&q=*:*&wt=json&fq={!collapse+field=groupid} My default handler is just defaults: explicit The

Re: How to speed up field collapsing on large number of groups

2016-07-14 Thread Joel Bernstein
>> rises with the cardinality of the collapse field. So you'll want to be >> sure >> there is enough memory to comfortably perform the collapse. >> >> > >> >> > >> > >> >> > >> >> > >> > >>

Re: How to speed up field collapsing on large number of groups

2016-07-13 Thread Jichi Guo
re about 2 million groups. > > I found the search with collapsing on groupId significantly slower comparing to without collapsing, especially when combined with facet queries. > > > > > I am wondering what would be the general approach to speedup field collapsing by 2~4

Re: How to speed up field collapsing on large number of groups

2016-07-04 Thread Alessandro Benedetti
t; > > > > > > > > > > > > On Tue, Jun 28, 2016 at 4:08 PM, jichi > <[jichi...@gmail.com](mailto:jichi...@gmail.com)> wrote: > > > > > > > >> Hi everyone, > > > > I am using Solr 4.10 to index 20 million documents

Re: How to speed up field collapsing on large number of groups

2016-06-28 Thread Jichi Guo
> > >> Hi everyone, > > I am using Solr 4.10 to index 20 million documents without sharding. > Each document has a groupId field, and there are about 2 million groups. > I found the search with collapsing on groupId significantly slower > comparing to without

Re: How to speed up field collapsing on large number of groups

2016-06-28 Thread Joel Bernstein
paring to without collapsing, especially when combined with facet > queries. > > I am wondering what would be the general approach to speedup field > collapsing by 2~4 times? > Would sharding the index help? > Is it possible to optimize collapsing without sharding? > > The filter par

How to speed up field collapsing on large number of groups

2016-06-28 Thread jichi
queries. I am wondering what would be the general approach to speedup field collapsing by 2~4 times? Would sharding the index help? Is it possible to optimize collapsing without sharding? The filter parameter for collapsing is like this: q=*:*&fq={!collapse field=groupId max=sum(...a long for

Field collapsing on parent document

2015-07-13 Thread StrW_dev
grouped or collapsed at the intermediate level. Can I do that? So basically a query that combines a parent child query with a collapse field query. Something like this: {!child of=type:path}{!collapse field=_root_} Gr -- View this message in context: http://lucene.472066.n3.nabble.com/Field

Re: Field collapsing memory usage

2015-01-23 Thread Toke Eskildsen
On Thu, 2015-01-22 at 22:52 +0100, Erick Erickson wrote: > What do you think about folding this into the Solr (or Lucene?) code > base? Or is it to specialized? (writing under the assumption that DVEnabler actually works as it should for everyone and not just us) Right now it is an explicit tool.

Re: Field collapsing memory usage

2015-01-22 Thread Erick Erickson
Toke: What do you think about folding this into the Solr (or Lucene?) code base? Or is it to specialized? Not sure one way or the other, just askin' Erick On Thu, Jan 22, 2015 at 3:47 AM, Toke Eskildsen wrote: > Norgorn [lsunnyd...@mail.ru] wrote: >> Is there any way to make 'docValues="t

RE: Field collapsing memory usage

2015-01-22 Thread Toke Eskildsen
Norgorn [lsunnyd...@mail.ru] wrote: > Nice, thanks! > If u'd like to, I'll write our results with that amazing util. By all means, please do. Good as well as bad. Independent testing is needed to ensure proper working tools. - Toke Eskildsen

RE: Field collapsing memory usage

2015-01-22 Thread Norgorn
Nice, thanks! If u'd like to, I'll write our results with that amazing util. -- View this message in context: http://lucene.472066.n3.nabble.com/Field-collapsing-memory-usage-tp4181092p4181159.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Field collapsing memory usage

2015-01-22 Thread Toke Eskildsen
Norgorn [lsunnyd...@mail.ru] wrote: > Is there any way to make 'docValues="true"' without reindexing? Depends on how brave you are :-) We recently had the same need and made https://github.com/netarchivesuite/dvenabler To my knowledge that is the only existing tool for that task an as we are the

RE: Field collapsing memory usage

2015-01-22 Thread Norgorn
eliosearch. Is there any way to make 'docValues="true"' without reindexing? -- View this message in context: http://lucene.472066.n3.nabble.com/Field-collapsing-memory-usage-tp4181092p4181108.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Field collapsing memory usage

2015-01-21 Thread Toke Eskildsen
Norgorn [lsunnyd...@mail.ru] wrote: > So, as we see, memory, used by first shard to group, wasn't released. > Caches are already nearly zero. It should be one or the other: Either the memory is released or there is something in the caches. Anyway, DocValues is the way to go, so ensure that it tu

Field collapsing memory usage

2015-01-21 Thread Norgorn
We are trying to run SOLR with big index, using as little RAM as possible. Simple search for our cases works nice, but field collapsing (group=true) queries fall with OOM. Our setup is several shards per SOLR entity, each shard on it's own HDD. We've tried same queries, but to one spec

field collapsing related problem

2014-09-19 Thread sankalp gupta
Hi I have about 15 fields in my solr schema but there are two fields lets say them field1 and field2 in my schema. For most searches I feel I have a perfect schema but for one use case it is not apt: *problem*: I have to group by column using field1 and then I have to search a particular value "a"

need help in field collapsing

2014-08-18 Thread Sankalp Gupta
Hi I have about 15 fields in my solr schema but there are two fields lets say them field1 and field2 in my schema. For most searches I feel I have a perfect schema but for one use case it is not apt: *problem*: I have to group by column using field1 and then I have to search a particular value "a"

Re: Getting min and max of a solr field for each group while doing field collapsing/result grouping

2014-05-01 Thread Varun Gupta
Hi Ahmet, Thanks for the information! But as per Solr documentation, group.truncate is not supported in distributed searches and I am looking for a solution that can work on SolrCloud. -- Varun Gupta On Thu, May 1, 2014 at 4:12 PM, Ahmet Arslan wrote: > Hi Varun, > > I think you can use group.

Re: Getting min and max of a solr field for each group while doing field collapsing/result grouping

2014-05-01 Thread Ahmet Arslan
Hi Varun, I think you can use group.truncate=true with stats component http://wiki.apache.org/solr/StatsComponent     If true, facet counts are based on the most relevant document of each group matching the query. Same applies for StatsComponent. Default is false. Solr3.4 Supported from Solr

Getting min and max of a solr field for each group while doing field collapsing/result grouping

2014-05-01 Thread Varun Gupta
Hi, I am using SolrCloud for getting results grouped by a particular field. Now, I also want to get min and max value for a particular field for each group. For example, if I am grouping results by city, then I also want to get the minimum and maximum price for each city. Is this possible to do w

Sort order in field collapsing (group.field=xxxx)

2014-02-24 Thread Sesha Sendhil Subramanian
Hey, I have a solr cloud setup with 2 shards. I am trying to use solr's field grouping feature My query looks like q=*:*&fq=field:value&group=true&group.field=otherValue The *ordering* of the groups *differs based on which shard the query is fired from* and it seems that docs located in that sam

Re: field collapsing performance in sharded environment

2013-11-19 Thread Otis Gospodnetic
iano < dtroi...@basistech.com> wrote: > Hello, > > I'm hitting a performance issue when using field collapsing in a > distributed Solr setup and I'm wondering if others have seen it and if > anyone has an idea to work around. it. > > I'm using field collapsi

Re: field collapsing performance in sharded environment

2013-11-15 Thread Paul Masurel
e question of why you're doing this. What purpose is served > by > grouping on documents that probably only have 1 member? > > Best, > Erick > > > On Wed, Nov 13, 2013 at 2:46 PM, David Anthony Troiano < > dtroi...@basistech.com> wrote: > > > Hello, &g

Group and Field Collapsing in SOLR "More like this"

2013-11-14 Thread balaji
Hi I have two types of profile : Shadow and DO and I am trying to use MLT to bring related recommendation of a userID In the result I get both the types but I want to restrict the results of document through a field (type) I pass it on. Currently grouping and field collapsing does not seem to

Re: field collapsing performance in sharded environment

2013-11-14 Thread Erick Erickson
urpose is served by grouping on documents that probably only have 1 member? Best, Erick On Wed, Nov 13, 2013 at 2:46 PM, David Anthony Troiano < dtroi...@basistech.com> wrote: > Hello, > > I'm hitting a performance issue when using field collapsing in a > distributed Solr

field collapsing performance in sharded environment

2013-11-13 Thread David Anthony Troiano
Hello, I'm hitting a performance issue when using field collapsing in a distributed Solr setup and I'm wondering if others have seen it and if anyone has an idea to work around. it. I'm using field collapsing to deduplicate documents that have the same near duplicate

Carrot2 Clustering with Field Collapsing

2013-10-23 Thread Thanigai Vellore
an error: /clustering? q=text:tiger&rows=500&carrot.title=title&group=true&group.field=id&group.main=true I would like to know if clustering is supported with field collapsing or I'm doing something incorrect? The stack trace for the error is given below: java

Issue with Group By / Field Collapsing

2013-09-30 Thread Shamik Bandopadhyay
Hi, I'm trying to use group by option to remove duplicates from my search result. I'm applying Group By option on a field called TopicId. I'm simply appending this at the end of my query. group=true&group.field=TopicId Initially, the result looked great as I was able to see the duplicates gett

Re: How to sort over all documents by score after Result Grouping / Field Collapsing

2013-09-25 Thread Erick Erickson
olr other than my application, because the search result are very > large. > > Thanks! > Jun > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/How-to-sort-over-all-documents-by-score-after-Result-Grouping-Field-Collapsing-tp4091593p4091784.html > Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to sort over all documents by score after Result Grouping / Field Collapsing

2013-09-24 Thread go2jun
/How-to-sort-over-all-documents-by-score-after-Result-Grouping-Field-Collapsing-tp4091593p4091784.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to sort over all documents by score after Result Grouping / Field Collapsing

2013-09-24 Thread Erick Erickson
ed="true"/> > indexed="true"/> > indexed="true"/> > > I know I can use solr Result Grouping / Field Collapsing to get the top 2 > result by grouping by source_id. Within each groups, documents sorted by > scroe by query like this: > http://localho

How to sort over all documents by score after Result Grouping / Field Collapsing

2013-09-23 Thread go2jun
Hi, I have solr documents like this: I know I can use solr Result Grouping / Field Collapsing to get the top 2 result by grouping by source_id. Within each groups, documents sorted by scroe by query like this: http://localhost:8983/solr/select?q=bank&group.field=source_id&gr

Solr Out Of Memory with Field Collapsing

2013-07-29 Thread tushar_k47
Hi, We are using Field collapsing feature with multiple shards. We ran into into Out of Memory errors on one of the shards. We use filed collapsing on a particular field which has only one specific value on the shard that goes out of memory. Interestingly the Out of Memory error recurred multiple

Re: Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-06 Thread Erick Erickson
Well, you've just restated the problem. I'm asking what use-case this is supporting? You've said: "he/she wants to know the occurrence of a specific term in the result set of that 'X1' search criteria" OK, _why_? Idle curiosity? Ranking the docs? Choosing the most relevant? I don't think you can

Re: Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-03 Thread Tony Mullins
Ok let me explain the problem. I have lots of crawled data, indexed in my Solr (4.3.0) and lets say user creates a search criteria 'X1' and he/she wants to know the occurrence of a specific term in the result set of that 'X1' search criteria. And then again he/she creates another search criteria '

Re: Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-03 Thread Erick Erickson
I don't think you can, thus the silence. But why do you want to do this thing? Smells like an XY problem, you've asked how to do a specific thing without stating the problem. Perhaps there's a better approach that _is_ do-able. Best Erick On Wed, Jul 3, 2013 at 2:14 AM, Tony Mullins wrote: > An

Re: Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-02 Thread Tony Mullins
Any suggestions please ! On Tue, Jul 2, 2013 at 3:24 PM, Tony Mullins wrote: > Hi, > > Is it possible to perform aggregated termfreq(field,term) on Result > Grouping ? > > I am trying to get total count of term's appearance in a document and then > want to aggregate that count by grouping the do

Aggregate TermFrequency on Result Grouping / Field Collapsing

2013-07-02 Thread Tony Mullins
Hi, Is it possible to perform aggregated termfreq(field,term) on Result Grouping ? I am trying to get total count of term's appearance in a document and then want to aggregate that count by grouping the document on one of my field. Like this http://localhost:8080/solr/collection1/select?q=iphon

Re: Field collapsing bad performances, schema redesign

2013-02-05 Thread Mikhail Khludnev
t;Converse all star - blue - 8", > "attribute_size":"8", "attribute_color":"blue", "price":49}, > {"product_id":1, "offer_id":4,"title":"Converse all star - blue - 9", > "attribute_size&qu

Re: Field collapsing bad performances, schema redesign

2013-02-05 Thread Mickael Magniez
uct_id":1, "offer_id":4,"title":"Converse all star - blue - 9", "attribute_size":"9", "attribute_color":"blue", "price":49} ] How can i index/query for using join in order to have only one shoe per model (the best price for example)? -- View this message in context: http://lucene.472066.n3.nabble.com/Field-collapsing-bad-performances-schema-redesign-tp4038359p4038500.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Field collapsing bad performances, schema redesign

2013-02-04 Thread Mikhail Khludnev
gt; to > retrieve the total number of groups > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Field-collapsing-bad-performances-schema-redesign-tp4038359p4038377.html > Sent from the Solr - User mailing list archive at Nabble.com. > --

Re: Field collapsing bad performances, schema redesign

2013-02-04 Thread Yonik Seeley
On Mon, Feb 4, 2013 at 10:34 AM, Mickael Magniez wrote: > group.ngroups=true This is currently very inefficient - if you can live without retrieving the total number of groups, performance should be much better. -Yonik http://lucidworks.com

Re: Solr 4 - Missing Field Collapsing Components

2013-02-04 Thread Jan Høydahl
Hi, Field collapsing is built-in and is also called Result Grouping: http://wiki.apache.org/solr/FieldCollapsing You simply enable it with ...&group=true&group.field=myfield If that does not work for you, please respond with detailed error messages so we can help you further. -- Jan

RE: Field Collapsing - Anything in the works for multi-valued fields?

2013-01-18 Thread David Parks
27;t quite follow the last point. -Original Message- From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com] Sent: Friday, January 18, 2013 9:34 AM To: solr-user@lucene.apache.org Subject: Re: Field Collapsing - Anything in the works for multi-valued fields? Hi, Instead of the mul

Re: Field Collapsing - Anything in the works for multi-valued fields?

2013-01-17 Thread Otis Gospodnetic
ludnev [mailto:mkhlud...@griddynamics.com] > Sent: Friday, January 18, 2013 2:32 AM > To: solr-user > Subject: Re: Field Collapsing - Anything in the works for multi-valued > fields? > > David, > > What's the documents and the field? It can help to suggest workaround.

RE: Field Collapsing - Anything in the works for multi-valued fields?

2013-01-17 Thread David Parks
18, 2013 2:32 AM To: solr-user Subject: Re: Field Collapsing - Anything in the works for multi-valued fields? David, What's the documents and the field? It can help to suggest workaround. On Thu, Jan 17, 2013 at 5:51 PM, David Parks wrote: > I want to configure Field Collapsing, but m

Re: Field Collapsing - Anything in the works for multi-valued fields?

2013-01-17 Thread Mikhail Khludnev
David, What's the documents and the field? It can help to suggest workaround. On Thu, Jan 17, 2013 at 5:51 PM, David Parks wrote: > I want to configure Field Collapsing, but my target field is multi-valued > (e.g. the field I want to group on has a variable # of entries per > d

Field Collapsing - Anything in the works for multi-valued fields?

2013-01-17 Thread David Parks
I want to configure Field Collapsing, but my target field is multi-valued (e.g. the field I want to group on has a variable # of entries per document, 1-N entries). I read on the wiki (http://wiki.apache.org/solr/FieldCollapsing) that grouping doesn't support multi-valued fields yet. Anythi

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-11-02 Thread ilay
/Scalability-of-Solr-Result-Grouping-Field-Collapsing-Millions-Billions-of-documents-tp4002524p4017945.html Sent from the Solr - User mailing list archive at Nabble.com.

Field collapsing / Grouping - How to make SOLR return resultset of 2 field groups (intersection)?

2012-08-30 Thread bbarani
message in context: http://lucene.472066.n3.nabble.com/Field-collapsing-Grouping-How-to-make-SOLR-return-resultset-of-2-field-groups-intersection-tp4004414.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-23 Thread Mikhail Khludnev
g 22, 2012 at 3:35 PM, Tom Burton-West > wrote: > >> Thanks Tirthankar, > >> > >> So the issue in memory use for sorting. I'm not sure I understand how > >> sorting of grouping fields is involved with the defaults and field > >> collapsing, sinc

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-23 Thread Erick Erickson
2 at 3:35 PM, Tom Burton-West wrote: >> Thanks Tirthankar, >> >> So the issue in memory use for sorting. I'm not sure I understand how >> sorting of grouping fields is involved with the defaults and field >> collapsing, since the default sorts by relevance not gro

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Lance Norskog
w > sorting of grouping fields is involved with the defaults and field > collapsing, since the default sorts by relevance not grouping field. On > the other hand I don't know much about how field collapsing is implemented. > > So far the few tests I've made haven't r

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tom Burton-West
Thanks Tirthankar, So the issue in memory use for sorting. I'm not sure I understand how sorting of grouping fields is involved with the defaults and field collapsing, since the default sorts by relevance not grouping field. On the other hand I don't know much about how field col

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tom Burton-West
Hi Lance and Tirthankar, We are currently using Solr 3.6. I tried a search across our current 12 shards grouping by book id (record_no in our schema) and it seems to work fine (the query with the actual urls for the shards changed is appended below.) I then searched for the record_no of the seco

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tom Burton-West
r...@commvault.com> wrote: > This wont work, see my thread on Solr3.6 Field collapsing > Thanks, > Tirthankar > >

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tom Burton-West
Hi Lance, I don't understand enough of how the field collapsing is implemented, but I thought it worked with distributed search. Are you saying it only works if everything that needs collapsing is on the same shard? Tom On Wed, Aug 22, 2012 at 2:41 AM, Lance Norskog wrote: > Ho

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-22 Thread Tirthankar Chatterjee
at 4:10 PM, Tirthankar Chatterjee wrote: > This wont work, see my thread on Solr3.6 Field collapsing > Thanks, > Tirthankar > > -Original Message- > From: Tom Burton-West > Date: Tue, 21 Aug 2012 18:39:25 > To: solr-user@lucene.apache.org > Reply-To: "s

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-21 Thread Lance Norskog
Solr3.6 Field collapsing > Thanks, > Tirthankar > > -Original Message- > From: Tom Burton-West > Date: Tue, 21 Aug 2012 18:39:25 > To: solr-user@lucene.apache.org > Reply-To: "solr-user@lucene.apache.org" > Cc: William Dueber; Phillip Farber > Subject:

Re: Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-21 Thread Tirthankar Chatterjee
This wont work, see my thread on Solr3.6 Field collapsing Thanks, Tirthankar -Original Message- From: Tom Burton-West Date: Tue, 21 Aug 2012 18:39:25 To: solr-user@lucene.apache.org Reply-To: "solr-user@lucene.apache.org" Cc: William Dueber; Phillip Farber Subject: Scalabili

Scalability of Solr Result Grouping/Field Collapsing: Millions/Billions of documents?

2012-08-21 Thread Tom Burton-West
Hello all, We are thinking about using Solr Field Collapsing on a rather large scale and wonder if anyone has experience with performance when doing Field Collapsing on millions of or billions of documents (details below. ) Are there performance issues with grouping large result sets? Details

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-18 Thread Erick Erickson
EFFs work fine with millions of objects. Under the covers, the float values are just read into memory so looking them up in order to use in a function is very fast. Best Erick On Thu, Aug 16, 2012 at 9:52 PM, Tirthankar Chatterjee wrote: > Awesome, you rock !!!. Thanks to Eric too for coming

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-16 Thread Tirthankar Chatterjee
Awesome, you rock !!!. Thanks to Eric too for coming up with the ideas. But honestly I am more interested to know this EFF concept. I will do my reading and then throw up questions if I don't get it. External File Field and function queries are something which i have not evaluated yet, but I a

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-16 Thread prosens
ucene.472066.n3.nabble.com/SOLR3-6-Field-Collapsing-Grouping-throws-OOM-tp4001055p4001640.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-15 Thread Tirthankar Chatterjee
Awesome thanks a lot, I am already on it with option 1. We need to track delete to flip the previous one as the current. Erick Erickson wrote: No, sharding into multiple cores on the same machine still is limited by the physical memory available. It's still lots of stuf on a limited box. But.

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-15 Thread Chris Hostetter
: 2> Use external file fields (EFF) for the same purpose, that : won't require you to re-index the doc. The trick : here is you use the value in the EFF as a multiplier : for the score (that's what function queries do). So older : versions of the doc have scores of 0 and just d

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-15 Thread Erick Erickson
No, sharding into multiple cores on the same machine still is limited by the physical memory available. It's still lots of stuf on a limited box. But try backing up and re-thinking the problem a bit. Some possibilities off the top of my head: 1> have a new field "current". when you update a d

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-15 Thread Tirthankar Chatterjee
Hi Erick, You are so right on the memory calculations. I am happy that I know now that I was doing something wrong. Yes I am getting confused with SQL. I will back up and let you know the use case. I am tracking file versions. And I want to give an option to browse your system for the latest fil

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-14 Thread Erick Erickson
You'r putting a lot of data on a single box, then asking to group on what I presume is a string field. That's just going to eat up a _bunch_ of memory. let's say your average file name is 16 bytes long. Each unique value will take up 58 + 32 bytes (58 bytes of overhead, I'm presuming Solr 3.X and

Re: SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-14 Thread Tirthankar Chatterjee
Editing the query...remove I don't know where it came from while I did copy/paste Tirthankar Chatterjee wrote: Hi, I have a beefy box with 24Gb RAM (12GB for Tomcat7 which houses SOLR3.6) 2 Processors Intel Xeon 64 bit Server, 30TB HDD. JDK 1.7.0_03 x64 bit Data Index Dir Size: 400GB

SOLR3.6:Field Collapsing/Grouping throws OOM

2012-08-13 Thread Tirthankar Chatterjee
Hi, I have a beefy box with 24Gb RAM (12GB for Tomcat7 which houses SOLR3.6) 2 Processors Intel Xeon 64 bit Server, 30TB HDD. JDK 1.7.0_03 x64 bit Data Index Dir Size: 400GB Metadata of files is stored in it. I have around 15 schema fields. Total number of items:150million approx. I have a sce

RE: Exception using distributed field-collapsing

2012-06-21 Thread Bryan Loofbourrow
n the list of fields whose values I am requesting with fl=. So I guess this is likely to be the date that is somehow turning up in the ClassCastException. Great suggestion. Thanks, Cody. Now I'm wondering if anyone familiar with the Field Collapsing code can see a possible vector for a bug, given

RE: Exception using distributed field-collapsing

2012-06-21 Thread Young, Cody
@lucene.apache.org Subject: RE: Exception using distributed field-collapsing Cody, > Does it work in the non distributed case? Yes. > > Is the field you're grouping on stored? What is the type on the uniqueKey > field? Is it stored and indexed? The field I'm grouping on is a stri

RE: Exception using distributed field-collapsing

2012-06-21 Thread Bryan Loofbourrow
: Wednesday, June 20, 2012 1:54 PM > To: solr-user@lucene.apache.org > Subject: RE: Exception using distributed field-collapsing > > > Hi Bryan, > > > > What is the fieldtype of the groupField? You can only group by field > > that is of type string as is descri

RE: Exception using distributed field-collapsing

2012-06-21 Thread Young, Cody
Exception using distributed field-collapsing > Hi Bryan, > > What is the fieldtype of the groupField? You can only group by field > that is of type string as is described in the wiki: > http://wiki.apache.org/solr/FieldCollapsing#Request_Parameters > > When you group by another

RE: Exception using distributed field-collapsing

2012-06-20 Thread Bryan Loofbourrow
> Hi Bryan, > > What is the fieldtype of the groupField? You can only group by field > that is of type string as is described in the wiki: > http://wiki.apache.org/solr/FieldCollapsing#Request_Parameters > > When you group by another field type a http 400 should be returned > instead if this error.

Re: Exception using distributed field-collapsing

2012-06-20 Thread Martijn v Groningen
Hi Bryan, What is the fieldtype of the groupField? You can only group by field that is of type string as is described in the wiki: http://wiki.apache.org/solr/FieldCollapsing#Request_Parameters When you group by another field type a http 400 should be returned instead if this error. At least that

Exception using distributed field-collapsing

2012-06-20 Thread Bryan Loofbourrow
I am doing a search on three shards with identical schemas (I double-checked!), using the group feature, and Solr/Lucene 3.5. Solr is giving me back the exception listed at the bottom of this email: Other information: My schema uses the following field types: StrField, DateField, TrieDateFiel

Re: Issue with field collapsing in solr 4 while performing distributed search

2012-06-11 Thread roz dev
o do > > some manual document partitioning. > > > > Martijn > > > > On 11 June 2012 14:29, Nitesh Nandy wrote: > > > Version: Solr 4.0 (svn build 30th may, 2012) with Solr Cloud (2 slices > > and > > > 2 shards) > > > > > > The

Re: Issue with field collapsing in solr 4 while performing distributed search

2012-06-11 Thread Nitesh Nandy
ent partitioning. > > Martijn > > On 11 June 2012 14:29, Nitesh Nandy wrote: > > Version: Solr 4.0 (svn build 30th may, 2012) with Solr Cloud (2 slices > and > > 2 shards) > > > > The setup was done as per the wiki: > http://wiki.apache.org/solr/SolrClou

Re: Issue with field collapsing in solr 4 while performing distributed search

2012-06-11 Thread Jack Krupansky
---Original Message- From: Martijn v Groningen Sent: Monday, June 11, 2012 8:53 AM To: solr-user@lucene.apache.org Subject: Re: Issue with field collapsing in solr 4 while performing distributed search The ngroups returns the number of groups that have matched with the query. However if you w

Re: Issue with field collapsing in solr 4 while performing distributed search

2012-06-11 Thread Martijn v Groningen
While querying, we use field collapsing > with "ngroups" set as true as we need the number of search results. > > However, there is a difference in the number of "result list" returned and > the "ngroups" value returned. > > Ex: > http://localhost:8

Issue with field collapsing in solr 4 while performing distributed search

2012-06-11 Thread Nitesh Nandy
Version: Solr 4.0 (svn build 30th may, 2012) with Solr Cloud (2 slices and 2 shards) The setup was done as per the wiki: http://wiki.apache.org/solr/SolrCloud We are doing distributed search. While querying, we use field collapsing with "ngroups" set as true as we need the number

Field Collapsing / Result Grouping

2012-01-19 Thread cmathur
group.limit, then all results under the group are returned. Why is the default 1 instead of ALL? Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Field-Collapsing-Result-Grouping-tp3674000p3674000.html Sent from the Solr - User mailing list archive at Nabble.com.

Using sort_values (fsv=true parameter) and Field Collapsing (group=true) at the same time

2011-12-27 Thread Jose Aguilar
Hi all, I am using Solr 4.0 trunk with the Field Collapsing feature (http://wiki.apache.org/solr/FieldCollapsing) and I notice that when used at the same time as the fsv=true parameter, the sort_values in the response is gone. I haven't found much information about the fsv parameter,

Re: Field collapsing results caching

2011-12-19 Thread astubbs
Why is this? And what happened to http://lucene.472066.n3.nabble.com/Re-Field-Collapsing-disable-cache-td481783.html ? I don't see why basic caching of request -> result shouldn't be the same? I know I could put a layer on top, but I'd like to use a built in cache if possi

Re: Difference between field collapsing and result grouping

2011-12-13 Thread Chris Hostetter
: Nope, they're the same. The original name was Field Collapsing, : but it was changed to "Grouping" later. Specificly: "Field Collapsing" is one type of usecase for the more general concept of "Result Grouping" (other types of result grouping are "gr

Re: Field collapsing results caching

2011-12-09 Thread Martijn v Groningen
There is no cross query cache for result grouping. The only caching option out there is the group.cache.percent option: http://wiki.apache.org/solr/FieldCollapsing#Request_Parameters Martijn On 8 December 2011 14:29, Kissue Kissue wrote: > Hi, > > I was just testing field collapsing i

Re: Difference between field collapsing and result grouping

2011-12-08 Thread Erick Erickson
Nope, they're the same. The original name was Field Collapsing, but it was changed to "Grouping" later. But note that the functionality has changed over time, so you might be seeing documents from different incarnations of the code. Best Erick On Wed, Dec 7, 2011 at 10:41 AM

Field collapsing results caching

2011-12-08 Thread Kissue Kissue
Hi, I was just testing field collapsing in my solr admin on solr 3.5.0. I have observed that the results of field collapsing are not being cached unlike other solr query results. Am doing the same query multiple times and the time taken still remains approximately the same. Is there something i

Difference between field collapsing and result grouping

2011-12-07 Thread Kissue Kissue
Sorry if this question sounds stupid but i am really really confused about this. Is there actually a difference between field collapsing and result grouping in SOLR? I have come across articles that have talked about setting up field collapsing with commands that look different from the grouping

Re: facet with group by (or field collapsing)

2011-11-03 Thread Martijn v Groningen
in the group. Is > this possible? I tried adding the parameter collapse.facet=after, but that > seemed to have no effect. > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/facet-with-group-by-or-field-collapsing-tp497252p3478515.html > Sent fro

Re: facet with group by (or field collapsing)

2011-11-03 Thread erj35
r, but that seemed to have no effect. -- View this message in context: http://lucene.472066.n3.nabble.com/facet-with-group-by-or-field-collapsing-tp497252p3478515.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Field Collapsing and Record Filtering

2011-10-16 Thread Michael Sokolov
ent_at" is interesting; could you expand on what you mean by "group sorting" though? Would that provide a means to get only the most recent version? Say I have access to versions 1,3,4 of some document and the current version is 5. I'd like to get version 4 as the result. Would you

Re: Field Collapsing and Record Filtering

2011-10-13 Thread lee carroll
the result from rendering or execute >>> a second grouped search to get more >>> groups. The downsides are that pagination will never be correct and >>> overall search time will take more time. >>> >>> Martijn >>> >>> On 5 October 2011 21

Re: Field Collapsing and Record Filtering

2011-10-13 Thread lee carroll
ination will never be correct and >> overall search time will take more time. >> >> Martijn >> >> On 5 October 2011 21:55, Daniel Skiles >>  wrote: >> >>> >>> A while back I sent a question to the list about only returning the most >>> recent

  1   2   3   4   >