Solr 8 - Sort Issue

When I sort desc on posting_id sort=posting_id%20desc, I get the following result "posting_id":"313" "posting_id":"312" "posting_id":"310" When I sort asc on posting_id sort=posting_id%20asc, I get the following result "posting_id":"10005343" "posting_id":"10005349" "posting_id":"10005

Re: [Q] Faster Atomic Updates - use docValues?

Hi Erick, I believed optimizing explicitly merges segments and that's why I was expecting it to give performance boost. I know that optimizations should not be done very frequently. For a full indexing, optimizations occurred 30 times between batches. I take your suggestion to undo all the changes

Re: Solr indexing performance

On 12/5/2019 10:42 PM, Paras Lehana wrote: Can ulimit settings impact this? Review once. If the OS limits prevent Solr from opening a file or starting a thread, it is far more likely

Re: From solr to solr cloud

On 12/5/2019 12:28 PM, Vignan Malyala wrote: I currently have 500 collections in my stand alone solr. Bcoz of day by day increase in Data, I want to convert it into solr cloud. Can you suggest me how to do it successfully. How many shards should be there? How many nodes should be there? Are so ca

Re: xms/xmx choices

On 12/5/2019 12:57 PM, David Hastings wrote: That probably isnt enough data, so if youre interested: https://gofile.io/?c=rZQ2y4 The previous one was less than 4 minutes, so it doesn't reveal anything useful. This one is a little bit less than two hours. That's more useful, but still pret

Re: xms/xmx choices

Hi David, Your Xmx seems to be an overkill though without usage stats, this cannot be factified. I think you should analyze long GC pauses given that you have so much difference between the min and max. I prefer making the min/max same before stressing on the values. You can start with 20G but wha

Re: FlattenGraphFilter Eliminates Tokens - Can't match "Can't"

Hi Michael, I think you only want to use FlattenGraphFilter *once* in the indexing > analysis chain I had been doing this for a long time before I finally shifted to use FGF after every GraphFilterFactory. Although I don't know much about it on the code level, are you sure that all the following

Re: From solr to solr cloud

Do you mean 500 cores? Tell us about the data more. How many documents per core do you have or what performance issues are you facing? On Fri, 6 Dec 2019 at 01:01, David Hastings wrote: > are you noticing performance decreases in stand alone solr as of now? > > On Thu, Dec 5, 2019 at 2:29 PM Vig

Re: Solr indexing performance

Can ulimit settings impact this? Review once. On Thu, 5 Dec 2019 at 23:31, Shawn Heisey wrote: > On 12/5/2019 10:28 AM, Rahul Goswami wrote: > > We have a Solr 7.2.1 Solr Cloud setup w

Re: [ANNOUNCE] Apache Solr 8.3.1 released

Yup, now reflected. :) On Thu, 5 Dec, 2019, 19:43 Erick Erickson, wrote: > It’s there for me when I click on your link. > > > On Dec 5, 2019, at 1:08 AM, Paras Lehana > wrote: > > > > Hey Ishan, > > > > Cannot find 8.3.1 here: https://lucene.apache.org/solr/downloads.html > (8.3.0 > > is listed

Re: Re:Learning to rank - Bad Request

I am using SOLR 8.2.0 Cloud mode... but when i start with *-Dsolr.ltr.enabled=true* it shows me the error *techproducts_shard1_replica_n2: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Failed to create new ManagedResource /schema/model-store of type org.apache.solr.lt

Re: xms/xmx choices

and if this may be of use: https://imgur.com/a/qXBuSxG just been more or less winging the options since solr 1.3 On Thu, Dec 5, 2019 at 2:41 PM Shawn Heisey wrote: > On 12/5/2019 11:58 AM, David Hastings wrote: > > as of now we do an xms of 8gb and xmx of 60gb, generally through the > > dashbo

Re: xms/xmx choices

That probably isnt enough data, so if youre interested: https://gofile.io/?c=rZQ2y4 On Thu, Dec 5, 2019 at 2:52 PM David Hastings wrote: > I know theres no hard answer, and I know the Xms and Xmx should be the > same, but it was a set it and forget it sort of thing from years ago. I > will def

Re: xms/xmx choices

I know theres no hard answer, and I know the Xms and Xmx should be the same, but it was a set it and forget it sort of thing from years ago. I will definitely be changing it but figured I may as well figure out as much as possible from this user group resource. as far as the raw GC data goes: http

Re: FlattenGraphFilter Eliminates Tokens - Can't match "Can't"

Thanks for the reply, I wouldn't be surprised if the issue you linked is related, I also found another similar issue: https://issues.apache.org/jira/browse/LUCENE-8723 You are absolutely right that the FlattenGraphFilter should only be used once, but as you noted the issue I am experiencing se

Re: xms/xmx choices

On 12/5/2019 11:58 AM, David Hastings wrote: as of now we do an xms of 8gb and xmx of 60gb, generally through the dashboard the JVM hangs around 16gb. I know Xms and Xmx are supposed to be the same so thats the change #1 on my end, I am just concerned of dropping it from 60 as thus far over the

Re: From solr to solr cloud

are you noticing performance decreases in stand alone solr as of now? On Thu, Dec 5, 2019 at 2:29 PM Vignan Malyala wrote: > Hi > I currently have 500 collections in my stand alone solr. Bcoz of day by day > increase in Data, I want to convert it into solr cloud. > Can you suggest me how to do i

From solr to solr cloud

Hi I currently have 500 collections in my stand alone solr. Bcoz of day by day increase in Data, I want to convert it into solr cloud. Can you suggest me how to do it successfully. How many shards should be there? How many nodes should be there? Are so called nodes different machines i should take?

xms/xmx choices

Hey all, over time ive adjusted and changed the solr Xms/Xmx various times with not too much thought aside from more is better, but ive noticed in many of the emails the recommended values are much lower than the numbers ive historically put in. i never really bothered to change them as the perfor

Re: FlattenGraphFilter Eliminates Tokens - Can't match "Can't"

I wonder if this might be similar/related to the underlying problem that is intended to be addressed by https://issues.apache.org/jira/browse/LUCENE-8985? btw, I think you only want to use FlattenGraphFilter *once* in the indexing analysis chain, towards the end (after all components that emit gra

Re: Solr indexing performance

On 12/5/2019 10:28 AM, Rahul Goswami wrote: We have a Solr 7.2.1 Solr Cloud setup where the client is indexing in 5 parallel threads with 5000 docs per batch. This is a test setup and all documents are indexed on the same node. We are seeing connection timeout issues thereafter some time into ind

Re: Solr indexing performance

Hi, the clients are reusing their SolrClient? Ciao, Vincenzo -- mobile: 3498513251 skype: free.dev > On 5 Dec 2019, at 18:28, Rahul Goswami wrote: > > Hello, > > We have a Solr 7.2.1 Solr Cloud setup where the client is indexing in 5 > parallel threads with 5000 docs per batch. This is a te

Solr indexing performance

Hello, We have a Solr 7.2.1 Solr Cloud setup where the client is indexing in 5 parallel threads with 5000 docs per batch. This is a test setup and all documents are indexed on the same node. We are seeing connection timeout issues thereafter some time into indexing. I am yet to analyze GC pauses a

Re: [Q] Faster Atomic Updates - use docValues?

> I think I should have also done optimize between batches, no? No, no, no, no. Absolutely not. Never. Never, never, never between batches. I don’t recommend optimizing at _all_ unless there are demonstrable improvements. Please don’t take this the wrong way, the whole merge process is really h

Re: [ANNOUNCE] Apache Solr 8.3.1 released

It’s there for me when I click on your link. > On Dec 5, 2019, at 1:08 AM, Paras Lehana wrote: > > Hey Ishan, > > Cannot find 8.3.1 here: https://lucene.apache.org/solr/downloads.html (8.3.0 > is listed here). > > Anyways, I'm downloading it from here: > https://archive.apache.org/dist/lucene/

Re: shard.preference for single shard queries

Look at SOLR-12217, it explains the limitation and has a patch for SolrJ cases. Should be merged soon. Note that the combination of replica types you are describing is not recommended. See https://lucene.apache.org/solr/guide/8_1/shards-and-indexing-data-in-solrcloud.html#combining-replica-types-i

Enabling LTR in SOLRCloud (solr version8.2)

I am trying to work with solr-cloud and I have to use learning to rank models and features for my project. But I am facing this issue of *SolrCore Initialization Failures* *techproducts_shard1_replica_n2: org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Failed to create n

Re: Search Performance and omitNorms

Hi Erick, thanks for the reply. Just to follow up, I'm using "unified" highlighter (fastVector does not work for my purposes). I search and highlight on a multivalued string string field which contains small strings (usually less than 200 chars). This multivalued field is subject to various process

shard.preference for single shard queries

Hi all, Thanks to SOLR-11982 we can now give solr parameter to sort replicas while giving results but ONLY for distributed queries as per documentation. May I know why this limitation? As my setup, I have 3 replicas(2 NRT, 1 PULL) of a single s