Re: Performance issue with Solr 8.6.1 Unified Highlighter does not occur on Solr 6.

Hi David, Thanks for filing this issue. The classic non-weightMatcher mode works well for us right now. Yes, we are using the POSTINGS mode for most of the fields although explicitly mentioning it gives an error since not all fields are indexed with offsets. So I guess the highlighter is picking

Re: Performance issue with Solr 8.6.1 Unified Highlighter does not occur on Solr 6.

https://issues.apache.org/jira/browse/SOLR-10321 -- near the end my opinion is we should just omit the field if there is no highlight, which would address your need to do this work-around. Glob or no glob. PR welcome! It's satisfying seeing that the Unified Highlighter is so much faster than the

Re: Performance issue with Solr 8.6.1 Unified Highlighter does not occur on Solr 6.

On another note, since response time is in question, I have been using a customhighlighter to just override the method encodeSnippets() in the UnifiedSolrHighlighter class since solr 6 since Solr sends back blank array (ZERO_LEN_STR_ARRAY) in the response payload for fields that do not match. Here

Re: Performance issue with Solr 8.6.1 Unified Highlighter does not occur on Solr 6.

Hi David, Thanks so much for your reply. hl.weightMatches was indeed the culprit. After setting it to false, I am now getting the same sub-second response as Solr 6. I am using Solr 8.6.1 (8.6.1) Here are the tests I carried out: hl.requireFieldMatch=true&hl.weightMatches=true (2458 ms) hl.requi

Re: Performance issue with Solr 8.6.1 Unified Highlighter does not occur on Solr 6.

Hello Kerwin, Firstly, hopefully you've seen the upgrade notes: https://lucene.apache.org/solr/guide/8_7/solr-upgrade-notes.html 8.6 fixes a performance regression found in 8.5; perhaps you are using 8.5? Missing from the upgrade notes but found in the CHANGES.txt for 8.0 is hl.weightMatches=true

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Any one is looking my issue? Due to this issue I can not upgrade Solr 8.3.0. regards, Vishal Patel From: vishal patel Sent: Sunday, May 17, 2020 11:49 AM To: solr-user Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1 Solr 6.1.0 : 1881

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

From: vishal patel Sent: Sunday, May 17, 2020 11:04 AM To: solr-user Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1 Thanks for reply. I know Query field value is large. But same thing is working fine in Solr 6.1.0 and query executed within 300 milliseconds

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Patel From: Mikhail Khludnev Sent: Saturday, May 16, 2020 6:55 PM To: solr-user Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1 It seems this thread is doing heavy work, mind the bottom line. 202.8013ms 124.8008ms qtp153245266-156 (156

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

t; Vishal Patel > ____________ > From: Mikhail Khludnev > Sent: Saturday, May 16, 2020 2:23 PM > To: solr-user > Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1 > > Can you check Thread Dump in Solr Admin while Solr 8.3 crunches

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

. Regards, Vishal Patel From: Mikhail Khludnev Sent: Saturday, May 16, 2020 2:23 PM To: solr-user Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1 Can you check Thread Dump in Solr Admin while Solr 8.3 crunches query for 34 seconds? Please

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

t from Outlook<http://aka.ms/weboutlook> > > From: vishal patel > Sent: Friday, May 15, 2020 3:06 PM > To: solr-user@lucene.apache.org > Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1 > > I have result of query

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Any one is looking my issue? Please help me. Sent from Outlook<http://aka.ms/weboutlook> From: vishal patel Sent: Friday, May 15, 2020 3:06 PM To: solr-user@lucene.apache.org Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

I have result of query debug for both version so It will helpful. Solr 6.1 query debug URL https://drive.google.com/file/d/1ixqpgAXsVLDZA-aUobJLrMOOefZX2NL1/view Solr 8.3.1 query debug URL https://drive.google.com/file/d/1MOKVE-iPZFuzRnDZhY9V6OsAKFT38U5r/view I indexed same data in both version.

Re: Performance Issue since Solr 7.7 with wt=javabin

kef> From: Karl Stoney Sent: Thursday, January 30, 2020 3:56:31 PM To: solr-user@lucene.apache.org Subject: Re: Performance Issue since Solr 7.7 with wt=javabin I don’t have confidence in my ability to do that, I was hoping someone could help out as moving to 8.4 is too muc

Re: Performance Issue since Solr 7.7 with wt=javabin

ary 30, 2020 2:23:40 PM To: solr-user Subject: Re: Performance Issue since Solr 7.7 with wt=javabin No further releases are planned for 7.x, so your best bet is to patch branch_7_7 yourself and build a custom Solr version. Jan > 29. jan. 2020 kl. 20:54 skrev Karl Stoney > : > > Coul

Re: Performance Issue since Solr 7.7 with wt=javabin

Sithi > Sent: 29 January 2020 14:34 > To: solr-user@lucene.apache.org > Subject: Re: Performance Issue since Solr 7.7 with wt=javabin > > yes thanks so much, fixed in 8.4.0 > > > > -- > Sent from: > https://eur03.safelinks.protection.outlook.com/?url=ht

Re: Performance Issue since Solr 7.7 with wt=javabin

Could anyone produce a patch for 7.7 please? From: Florent Sithi Sent: 29 January 2020 14:34 To: solr-user@lucene.apache.org Subject: Re: Performance Issue since Solr 7.7 with wt=javabin yes thanks so much, fixed in 8.4.0 -- Sent from: https://eur03

Re: Performance Issue since Solr 7.7 with wt=javabin

yes thanks so much, fixed in 8.4.0 -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Performance Issue since Solr 7.7 with wt=javabin

Check out SOLR-14013 which I believe is what you are looking for Jan > 29. jan. 2020 kl. 11:46 skrev Florent Sithi : > > Hi Paras, > > Thanks for your answer and your ideas ;) > > I have the exact same issue than Andy "wt=javabin&version=2"

Re: Performance Issue since Solr 7.7 with wt=javabin

Hi Paras, Thanks for your answer and your ideas ;) I have the exact same issue than Andy "wt=javabin&version=2" have really poor performances comprared to wt=json I'm using : - solr 7.7.2 - OpenJDK8U-jdk_x64_linux_hotspot_8u222b10 or jdk-8u241-linux-x64 (same behaviour) The server have much R

Re: Performance Issue since Solr 7.7 with wt=javabin

Hi Andy, Have you run performance benchmarking for sometime and made sure that the Solr Caching and GC doesn't impact the performance? I recommend that you should rebuild the performance matrix after few warmups and requests. Have you invalidated this? On Fri, 18 Oct 2019 at 12:35, Jan Høydahl w

Re: Performance Issue since Solr 7.7 with wt=javabin

Hi, Did you find a solution to your performance problem? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 17. jun. 2019 kl. 17:17 skrev Andy Reek : > > Hi Solr team, > > we are using Solr in version 7.1 as search engine in our online shop (SAP > Hybris). And as a t

Re: Performance Issue since Solr 7.7 with wt=javabin

How are you consuming the output? Are you using solrj? On Tue, Jun 18, 2019, 1:27 AM Andy Reek wrote: > Hi Solr team, > > > we are using Solr in version 7.1 as search engine in our online shop (SAP > Hybris). And as a task I needed to migrate to the most recent Solr in > version 7 (7.7). Doing t

Re: Performance Issue in Streaming Expressions

Once you've scaled up the export from collection4 you can test the performance of the join by moving the NullStream around the join. parallel(null(innerJoin(collection 3, collection4))) Again you'll want to test with different numbers of workers and replicas to see where you max out performance o

Re: Performance Issue in Streaming Expressions

innerJoin(intersect(innerJoin(collection1, collection2), innerJoin(collection 3, collection4)), collection5) Let's focus on: innerJoin(collection 3, collection4)) The first thing to focus on is how fast is the export from collection4. You can test t

Re: Performance Issue in Streaming Expressions

Hi, Which version of solr are you on? Increasing memory may not be useful as streaming API does not keep stuff in memory (except may be hash joins). Increasing replicas (not sharding) and pushing the join computation on worker solr cluster with #workers > 1 would definitely make things faster. Are

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

Yes, swap will switch which core the name points to. For non Cloud setup. Just remember that your directory name does not get renamed, when you are deleting the old one. Just the core name in core.properties file. Regards, Alex On 24 Sep 2016 10:28 AM, "slee" wrote: Erick / Alex, I want to

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

Erick / Alex, I want to thank you both. Your hints got me understand SOLR a bit better. I ended up with reversewildcard, and it speeds up performance a lot. That's what I'm expecting from SOLR... I also no longer experience the huge memory hog. The only down-side I can think of is, you need to r

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

But if "SEF" and "OFF" are known to be searched for and especially if they are well-delimited, they could just be pulled-out into a separate field and just checked with an FQ. In the end, there may be no need for either EdgeNGram or wildcards. Just twisting the data during _indexing_ to represent

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

If you can break these up into tokens somehow, that's clearly best. But from the patterns you show it's not likely. WordDelimiterFactory won't quite work since it wouldn't be able to separate ASEF into the token SEF. You'll have a _lot_ fewer terms if you don't use edgengram. Try just using bi

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

Not fully clear still, but perhaps you need several fields, at least one of which just contains your SEF and OFF values serving effectively as binary switches (FQ matches). And then maybe you strip the leading IDs that you are not matching on. Remember your Solr data shape does not need to match y

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

Alex, You do have a point with EdgeNGramFilterFactory. As requested, I've attached a sample screenshotfor your review. Erick, Here's my use-case. Assume I have the following term stored in global_Value as such: - executionvenuetype#*

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

I totally missed EdgeNGram. Good catch Alex! Yeah, that's a killer. My shot in the dark here is that your analysis chain isn't the best choice to support your use-case and you're shooting yourself in the foot. So let's back up and talk about your use-case and maybe re-define your analysis chain fo

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

Well, I am guessing this is the line that's causing the problem: Run your real sample for that field against your indexing definition in Admin UI and see how many tokens you end up with. You may have 50 tokens, but if each of them generates up to 47 representations.. Regards, Alex.

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

Wait: Are you really doing leading wildcard queries? If so, that's likely the root of the problem. Unless you add ReverseWildcardFilterFactory to your analysis chain, Lucene has to enumerate your entire set of terms to find likely candidates, which takes a lot of resources. What happens if you use

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

Here's what I have define in my schema: This is what I send in the query (2 values): q=global_Value:*mas+AND+global_Value:*sef&df=text&rows=5&version=2.2&echoParams=explicit&fl=global_Value In addition, memory is taking wa

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

if multivalue fields cannot exceed certain terms? I only have 54 > to 60 terms. > > > Original Message > From: arafa...@gmail.com > Sent: September 21, 2016 7:40 PM > To: solr-user@lucene.apache.org > Reply-to: solr-user@lucene.apache.org > Subject: Re: Performance Issue wh

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

1, 2016 7:40 PM To: solr-user@lucene.apache.org Reply-to: solr-user@lucene.apache.org Subject: Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0] Do you _return_ the same set of fields in both queries? Is the difference truly just which field you search against? Regards,     Alex O

Re: Performance Issue when querying Multivalued fields [SOLR 6.1.0]

Do you _return_ the same set of fields in both queries? Is the difference truly just which field you search against? Regards, Alex On 22 Sep 2016 3:03 AM, "slee" wrote: > I've been doing a lot of reading on this forum with regards to performance > on > multivalued fields, and nothing helps.

RE: Performance issue with FILTER QUERY

[mailto:erickerick...@gmail.com] Sent: 20 August 2015 00:44 To: solr-user@lucene.apache.org Subject: Re: Performance issue with FILTER QUERY If you're committing that rapidly then you're correct, filter caching may not be a good fit. The entire _point_ of filter caching is to increase perfo

Re: Performance issue with FILTER QUERY

fectively >> uses cache and hence it slow every time in our case. >> >> -Original Message- >> From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] >> Sent: 19 August 2015 12:16 >> To: solr-user@lucene.apache.org >> Subject: Re: Performance issue with FILTER Q

Re: Performance issue with FILTER QUERY

gt; added document available for search). Due to which it is not effectively > uses cache and hence it slow every time in our case. > > -Original Message- > From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] > Sent: 19 August 2015 12:16 > To: solr-user@lucene.apache.org >

Re: Performance issue with FILTER QUERY

Hello, try to experiment with fq={!cache=false}... or fq={!cache=false cost=100}... see https://cwiki.apache.org/confluence/display/solr/Common+Query+Parameters On Wed, Aug 19, 2015 at 8:55 AM, Maulin Rathod wrote: > > Hi, > > http://stackoverflow.com/questions/11627427/solr-query-q-or-filter-q

RE: Performance issue with FILTER QUERY

- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: 19 August 2015 12:16 To: solr-user@lucene.apache.org Subject: Re: Performance issue with FILTER QUERY On Wed, 2015-08-19 at 05:55 +, Maulin Rathod wrote: > SLOW WITH FILTER QUERY (takes more than 1 sec

Re: Performance issue with FILTER QUERY

On Wed, 2015-08-19 at 05:55 +, Maulin Rathod wrote: > SLOW WITH FILTER QUERY (takes more than 1 second) > > > q=+recipient_id:(4042) AND project_id:(332) AND resource_id:(13332247 > 13332245 13332243 13332241 13332239) AND entity_type:(2) AND -acti

Re: Performance issue with group.ngroups=true

Mickael, I just wonder you have considered BlockJoin? it performs much better than query time approaches http://blog.griddynamics.com/2012/08/block-join-query-performs.html ,but faceting hasn't been implemented for it yet. On Tue, Jan 15, 2013 at 2:01 PM, Mickael Magniez wrote: > Hi, > > Retry

Re: Performance issue with group.ngroups=true

Hi, Retry on a better machine (2CPU, 8GB RAM, 1.5GB for java half used according admin interface) still have the same issue. It seems to grow with matches count : with a search matching 100k documents, it takes 700ms, vs 70ms without ngroup (CPU is 100% during request) For information, my index

Re: Performance issue with group.ngroups=true

group.ngroups=true is always going to be somewhat expensive, but in your case it seems more expensive than I would expect. You should check to see that you have enough Java JVM heap to hold more of the index and to avoid any excessive GCs. -- Jack Krupansky -Original Message- From: M

Re: Performance issue: Frange with geodist()

Hi Yonik, I have used your suggestion to implement a better radius searcher: &facet.query={!geofilt d=10 key=d10} &facet.query={!geofilt d=20 key=d20} &facet.query={!geofilt d=50 key=d50} It is a little bit faster than with geodist() but still a bottleneck i think. -- View this message in cont

Re: Performance issue: Frange with geodist()

I added a Jira issue for this: https://issues.apache.org/jira/browse/SOLR-2840 On 10/13/11 8:15 AM, "Yonik Seeley" wrote: >On Thu, Oct 13, 2011 at 9:55 AM, Mikhail Khludnev > wrote: >> is it possible with geofilt and facet.query? >> >> facet.query={!geofilt pt=45.15,-93.85 sfield=store d=5} >

Re: Performance issue: Frange with geodist()

On Thu, Oct 13, 2011 at 9:55 AM, Mikhail Khludnev wrote: > is it possible with geofilt and facet.query? > > facet.query={!geofilt pt=45.15,-93.85 sfield=store d=5} Yes, that should be both possible and faster... something along the lines of: &sfield=store&pt=45.15,-93.85 &facet.query={!geofilt d=

Re: Performance issue: Frange with geodist()

is it possible with geofilt and facet.query? facet.query={!geofilt pt=45.15,-93.85 sfield=store d=5} On Thu, Oct 13, 2011 at 4:20 PM, roySolr wrote: > I don't want to use some basic facets. When the user doesn't get any > results > i want > to search in the radius of his search location. Exampl

Re: Performance issue: Frange with geodist()

I don't want to use some basic facets. When the user doesn't get any results i want to search in the radius of his search location. Example: apple store in Manchester gives no result. I want this: Click here to see 2 results in a radius of 10km. Click here to see 11 results in a radius of 50km. C

Re: Performance issue: Frange with geodist()

Ok. I've found explicit caveat for you http://wiki.apache.org/solr/SpatialSearch#How_to_facet_by_distance I don't think I'm able to help. Just for curious why geofilt is not enough for you? My concern is that the functions and queries score documents, but for facet queries the only filtering is re

Re: Performance issue: Frange with geodist()

Hello Mikhail, Thanks for your answer.. I think my cache is enabled for Geodist(). First time request takes 1440ms and second time only 2ms. In the statistics i see it's hits the cache. The problem is every request had another location with other distances and results. So almost every request tak

Re: Performance issue: Frange with geodist()

Hello, Could check that every request doesn't trigger loading values into the cache? You can see it in log. Recently I have similar issue when caching for geodist() was disabled. Regards On Thu, Oct 13, 2011 at 11:31 AM, roySolr wrote: > Hello, > > I use the facet.query to search documents nea

Re: Performance issue in indexing the data with DIH when using subqueries

Thanks a lot Shalin.. This resolve my issue :). Thanks, Barani Shalin Shekhar Mangar wrote: > > On Tue, Feb 23, 2010 at 1:01 AM, JavaGuy84 wrote: > >> >> Hi, >> >> I am facing a performace issue when I am trying to index the data using >> DIH.. I have a model as below >> >> Tables >> ---

Re: Performance issue in indexing the data with DIH when using subqueries

On Tue, Feb 23, 2010 at 1:01 AM, JavaGuy84 wrote: > > Hi, > > I am facing a performace issue when I am trying to index the data using > DIH.. I have a model as below > > Tables > > Object > ObjectProperty > ObjectRelationship > > > Object --> ObjectProperty one to Many Relationship

Re: performance issue

Can you run 32-bit Java there? Will use less memory! :) Otis -- Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch - Original Message > From: Matthieu Labour > To: solr-user@lucene.apache.org > Sent: Fri, January 22, 2010 11:07:45 AM > Subject: Re: performance i

Re: performance issue

You likely are running into problems by having 1000 cores and simply don't have enough memory to handle that much. You should look at what you are filtering, sorting, faceting on and see if there are ways you can tune these. Take a look at your cache configurations as well. What are your inde

Re: performance issue

Hi Thank you for your reponse Which version of solr? I inherited the project so not exactly sure ... in CHANGES.txt it says Apache Solr Version 1.4-dev $Id: CHANGES.txt 793090 2009-07-10 19:40:33Z yonik $ What garbage collection parameters? ulimit -n 10 ; nohup java -server -XX:+UseConcMarkS

Re: performance issue

Which version of Solr? Java? What garbage collection parameters? On Thu, Jan 21, 2010 at 1:03 PM, Matthieu Labour wrote: > Hi > > I have been requested to look at a solr instance that has been patched with > our own home grown patch to be able to handle 1000 cores on a solr instance > > The solr

Re: Performance issue

2009/1/27 mahendra mahendra > > Is there anyway I can get the results without restarting the server or > reloading cores from search machine. > A commit on the slave (searcher) will be needed to re-open the IndexReader and show the latest updates. Ofcourse, that should be done after the commit o

Re: Performance issue.

have been multi-valued? Do you see anything wrong with this? . Thanks... - Original Message - From: "Yonik Seeley" <[EMAIL PROTECTED]> To: Sent: Wednesday, December 06, 2006 9:55 PM Subject: Re: Performance issue. > It is using the cache, but the number of

Re: Performance issue.

AIL PROTECTED]> To: Sent: Wednesday, December 06, 2006 9:55 PM Subject: Re: Performance issue. It is using the cache, but the number of items is larger than the size of the cache. If you want to continue to use the filter method then you need to increase the size of the filter cache to someth

Re: Performance issue.

AIL PROTECTED]> To: Sent: Tuesday, December 05, 2006 8:46 PM Subject: Re: Performance issue. > On 12/5/06, Gmail Account <[EMAIL PROTECTED]> wrote: >> > There's nothing wrong with CPU jumping to 100% each query, that just >> > means you aren't IO bou

Re: Performance issue.

ns : 32338 size : 512 cumulative_lookups : 32849 cumulative_hits : 0 cumulative_hitratio : 0.00 cumulative_inserts : 32850 cumulative_evictions : 32338 Thanks, Mike - Original Message - From: "Yonik Seeley" <[EMAIL PROTECTED]> To: Sent: Tuesday, December 05, 2006 8:46 P

Re: Performance issue.

On 12/5/06, Gmail Account <[EMAIL PROTECTED]> wrote: > There's nothing wrong with CPU jumping to 100% each query, that just > means you aren't IO bound :-) What do you mean not IO bound? There is always going to be a bottleneck somewhere. In very large indicies, the bottleneck may be waiting f

Re: Performance issue.

There's nothing wrong with CPU jumping to 100% each query, that just means you aren't IO bound :-) What do you mean not IO bound? >- I did an optimize index through Luke with compound format and > noticed > in the solrconfig file that useCompoundFile is set to false. Don't do this unle

Re: Performance issue.

On 12/5/06, Gmail Account <[EMAIL PROTECTED]> wrote: Sorry.. I put the wrong subject on my message. I also wanted to mention that my cpu jumps to to almost 100% each query. There's nothing wrong with CPU jumping to 100% each query, that just means you aren't IO bound :-) It's the 3 seconds tha