Re: [CAUTION] Re: Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-22 Thread Shawn Heisey
On 3/22/2019 2:02 AM, Hubert-Price, Neil wrote: One other question Is there a system level configuration that can change the default for the sow= parameter? Can it be flipped to have the default set to true? Any parameter can be put into the query handler definition. In defaults, inva

Re: [CAUTION] Re: Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-22 Thread Hubert-Price, Neil
One other question Is there a system level configuration that can change the default for the sow= parameter? Can it be flipped to have the default set to true? Many Thanks, Neil On 22/03/2019, 08:36, "Hubert-Price, Neil" wrote: Thanks Erick, that makes sense. However it do

Re: Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-22 Thread Hubert-Price, Neil
Thanks Erick, that makes sense. However it does lead me to another conclusion: in Solr prior to 6.0, or with sow=true on Solr 6.0+ that would mean that the ShingleFilter is totally ineffective within query analysers. It would be logically equivalent to not having the ShingleFilter configur

Re: Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-21 Thread Erick Erickson
sow was introduced in Solr 6, so it’s just ignored in 4x. bq. Surely the tokenizer splits on white space anyway, or it wouldn't work? I didn’t work on that code, so I don’t have the details off the top of my head, but I’ll take a stab at it as far as my understanding goes. The result is in your

Re: Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-21 Thread Hubert-Price, Neil
2HzyvHsdbIW_n0190ofjPS3hAeN > > The LargeQueryStructure image shows as single thread with retained set of 4.8GB, with the biggest items being a BooleanWeight object of just over 1.8GB and a BooleanQuery object of just under 1.8GB > > The LargeNumberClauses image shows

Re: Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-21 Thread Erick Erickson
age shows as single thread with retained set of > 4.8GB, with the biggest items being a BooleanWeight object of just over 1.8GB > and a BooleanQuery object of just under 1.8GB > > The LargeNumberClauses image shows a drilldown into the BooleanQuery object, > where a subquery is

Re: Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-21 Thread Hubert-Price, Neil
leanWeight object of just over 1.8GB and a BooleanQuery object of just under 1.8GB The LargeNumberClauses image shows a drilldown into the BooleanQuery object, where a subquery is taking around 0.9GB and contains a BooleanClause[524288] array of clauses (not shown: each of these 524288 is

Re: Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-20 Thread Erick Erickson
The Apache mail server aggressively strips attachments, so yours didn’t come through. People often provide links to images stored somewhere else As to why this is behaving this way, I’m pretty clueless. A _complete_ shot in the dark is the query parsing changed its default for split on white

Use of ShingleFilter causing very large BooleanQuery structures in Solr 7.1

2019-03-20 Thread Hubert-Price, Neil
Hello All, We have a recently upgraded system that went from Solr 4.6 to Solr 7.1 (used as part of an ecommerce application). In the upgraded version we are seeing frequent issues with very high Solr memory usage for certain types of query, but the older 4.6 version does not produce the same r

RE: BooleanQuery and WordDelimiterFilter

2017-05-02 Thread Avi Steiner
shuntman SFhuntman'shuntman LCFhuntman'shuntman KSFhuntman'shuntman BTW, I use Solr 5.3.1 -Original Message- From: Rick Leir [mailto:rl...@leirtech.com] Sent: Monday, May 1, 2017 2:19 PM To: solr-user@lucene.apache.org Subject: Re: BooleanQuery and WordDelimiterFilter Avi,

Re: BooleanQuery and WordDelimiterFilter

2017-05-01 Thread Rick Leir
Avi, Tell us the relevant field types you have in schema.xml. You can also solve this all for yourself in the Solr Admin Analysis panel. Cheers -- Rick On May 1, 2017 2:34:31 AM EDT, Avi Steiner wrote: >Hi > >I have a question regarding the use of query parser and BooleanQuery. &g

BooleanQuery and WordDelimiterFilter

2017-04-30 Thread Avi Steiner
Hi I have a question regarding the use of query parser and BooleanQuery. I have 3 documents indexed. Doc1 contains the words huntman's and huntman Doc2 contains the word huntman's Doc3 contains the word huntman When I search for huntman's I get Doc1 and Doc2 When I search for +

BooleanQuery Migration from Solr 4 to SOlr 6

2016-07-18 Thread Max Bridgewater
HI Folks, I am tasked with migrating a Solr app from Solr 4 to Solr 6. This solr app is in essence a bunch of solr components/handlers. One part that challenges me is BooleanQuery immutability in Solr 6. Here is the challenge: In our old code base, we had classes that implemented custom

Re: MultiPhraseQuery:Rewrite to BooleanQuery

2015-01-21 Thread ku3ia
Tomoko Uchida wrote > Hi, > > Strictly speaking, MultiPhraseQuery and BooleanQuery wrapping PhraseQuerys > are not equal. > > For each query, Query.rewrite() returns different object. (with Lucene > 4.10.3) > q1.rewrite(reader).toString() returns: > body:&qu

Re: MultiPhraseQuery:Rewrite to BooleanQuery

2015-01-21 Thread Tomoko Uchida
Hi, Strictly speaking, MultiPhraseQuery and BooleanQuery wrapping PhraseQuerys are not equal. For each query, Query.rewrite() returns different object. (with Lucene 4.10.3) q1.rewrite(reader).toString() returns: body:"blueberry chocolate (pie tart)", where q1 is your first multi ph

Re: MultiPhraseQuery:Rewrite to BooleanQuery

2015-01-21 Thread ku3ia
Any ideas? -- View this message in context: http://lucene.472066.n3.nabble.com/MultiPhraseQuery-Rewrite-to-BooleanQuery-tp4180638p4180820.html Sent from the Solr - User mailing list archive at Nabble.com.

MultiPhraseQuery:Rewrite to BooleanQuery

2015-01-20 Thread ku3ia
ot;pie"), new Term("body", "tart")}); assertEquals(2, searcher.search(q, 1).totalHits); r.close(); indexStore.close(); I need to know on which phrase query will be match. Explanation doesn't return exact information, only that is match by this query. So can I rewrit

Re: Solr GC issues - Too many BooleanQuery & BooleanClause objects in heap

2012-11-13 Thread Otis Gospodnetic
; >8: 6876651 495118872 > > > org.apache.lucene.index.FieldsReader$LazyField > > >9: 11354214 363334848 > org.apache.lucene.search.PrefixQuery > > > 10: 4281624 137011968 java.util.HashMap$Entry > > > 11: 3466680 832003

Re: Solr GC issues - Too many BooleanQuery & BooleanClause objects in heap

2012-11-13 Thread Prasanna R
java.util.HashMap$Entry > > 11: 3466680 83200320 org.apache.lucene.search.TermQuery > > 12: 1987450 79498000 org.apache.lucene.search.PhraseQuery > > 13:631994 70148624 [Ljava.util.HashMap$Entry; > > . > > > > I have look

Re: Solr GC issues - Too many BooleanQuery & BooleanClause objects in heap

2012-11-12 Thread Otis Gospodnetic
1987450 79498000 org.apache.lucene.search.PhraseQuery > 13:631994 70148624 [Ljava.util.HashMap$Entry; > . > > I have looked at the Solr cache settings multiple times but am not able to > figure out how/why the high number of BooleanQuery and BooleanClause object &

RE: BooleanQuery

2009-05-01 Thread Ankush Goyal
Hi Otis, I was asking about all the reviews for matched hotels. Thanks for your input on that... -Ankush -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: Thursday, April 30, 2009 9:50 PM To: solr-user@lucene.apache.org Subject: Re: BooleanQuery All

RE: BooleanQuery

2009-05-01 Thread Ankush Goyal
Thanks Avlesh! -Original Message- From: Avlesh Singh [mailto:avl...@gmail.com] Sent: Thursday, April 30, 2009 7:46 PM To: solr-user@lucene.apache.org Subject: Re: BooleanQuery > > (+(rev.headline:beach^2.0) | rev.comments:beach^2.0)~0.01 ()) (+ > hot.id:5823 +hot.id:5847

Re: BooleanQuery

2009-04-30 Thread Otis Gospodnetic
http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Ankush Goyal > To: "solr-user@lucene.apache.org" > Sent: Thursday, April 30, 2009 6:56:44 PM > Subject: BooleanQuery > > Hey Guys, > > I have 2 indexes with one having hotel cont

Re: BooleanQuery

2009-04-30 Thread Avlesh Singh
> > (+(rev.headline:beach^2.0) | rev.comments:beach^2.0)~0.01 ()) (+ > hot.id:5823 +hot.id:5847) > Your hot.id's are not OR'd. They are AND'ed. (+hot.id:5823 +hot.id:5847) should have been (hot.id:5823 hot.id:5847). Cheers Avlesh On Fri, May 1, 2009 at 4:26 AM, Ankush Goyal wrote: > Hey Guys,

BooleanQuery

2009-04-30 Thread Ankush Goyal
Hey Guys, I have 2 indexes with one having hotel content and other containing reviews for hotels. When a user queries for a location the logic first calls the hotel index to get hotels for the location, then it needs to call review index to ask for all the reviews for all the hotels. So, I need

Re: BooleanQuery exception

2007-11-23 Thread Chris Hostetter
there was an earlier exception you aren't showing us, that suggests that the version of the BooleanQuery class loaded by Solr at runtime isn't the same version it was compiled against. have you odn't anything funky with your classpath? like swap he lucene jar in the solr.war, or pu

BooleanQuery exception

2007-11-20 Thread Cody Caughlan
I am trying to run a very simple query via the Admin interface and receive the exception below. The query is: description_t:guard AND title_t:help I am using dynamic fields (hence the underscored suffix). Any ideas? Thanks in advance /cody Nov 19, 2007 3:01:31 PM org.apache.solr.core.SolrE