LTR - FieldValueFeature Question

2020-04-23 Thread Ashwin Ramesh
Hi everybody, Do we need to have 'indexed=true' to be able to retrieve the value of a field via FieldValueFeature or is having docValue=true enough? Currently, we have some dynamic fields as [dynamicField=true, stored=false, indexed=false, docValue=true]. However when we noticing that the value e

Re: using S3 as the Directory for Solr

2020-04-23 Thread Walter Underwood
It will be a lot more than 2X or 3X slower. Years ago, I accidentally put Solr indexes on an NFS mounted filesystem and it was 100X slower. S3 would be a lot slower than that. Are you doing relevance-ranked searches on all that data? That is the only reason to use Solr instead of some other sol

Re: using S3 as the Directory for Solr

2020-04-23 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Rahul, On 4/23/20 21:49, dhurandar S wrote: > Thank you for your reply. The reason we are looking for S3 is since > the volume is close to 10 Petabytes. We are okay to have higher > latency of say twice or thrice that of placing data on the local >

Re: using S3 as the Directory for Solr

2020-04-23 Thread dhurandar S
Hi Jan, Thank you for your reply. The reason we are looking for S3 is since the volume is close to 10 Petabytes. We are okay to have higher latency of say twice or thrice that of placing data on the local disk. But we have a requirement to have long-range data and providing Seach capability on tha

Solr 8.2 Cloud Replication Locked

2020-04-23 Thread Justin Sweeney
Hi all, We are running Solr 8.2 Cloud in a cluster where we have a single TLOG replica per shard and multiple PULL replicas for each shard. We have noticed an issue recently where some of the PULL replicas stop replicating from the masters. The will have a replication which outputs: o.a.s.h.Index

Dynamic reload of TLS configuration

2020-04-23 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 All, Does anyone know if it is possible to reconfigure Solr's TLS configuration (specifically, the server key and certificate) without a restart? I'm looking for a zero-downtime situation with a single-server and an updated TLS certificate. Thanks

Failure to distribute update after 25 retries

2020-04-23 Thread Beikov Christian
Hello, I have a few very strange problems and hope anyone can help me with that. I'm trying to index something with Solr 8.4.1 but after a few documents I get the following exceptions: 2020-04-23 13:00:43.484 INFO (qtp1635378213-21) [c:cc5363_dm_documentversion s:shard1 r:core_node3 x:cc5363_

failed collection‘s metadata remains in ZK

2020-04-23 Thread YangLiu
Hello everyone, I am using Solr 7.7.2, I create a collection with shards=10 more than my nodes, then the service returned the following error: "Cannot create collection solrdemo. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 1.

Automatic reset of zxid on client for CloudSolrClient

2020-04-23 Thread Beikov Christian
Hey there! I am curious if there is an option to automatically reset the zxid on a CloudSolrClient if a mismatch is detected. After a reset of my cluster I currently have to also restart the application because of the wrong zxid, although it doesn't participate in the cluster. I have a feeling

Re: Defaults Merge Policy

2020-04-23 Thread Erick Erickson
Glad those articles helped, I remember them well ;) Do note that 30 (well, actually 33%) is usually the ceiling. But as I mentioned, it’s soft, not absolute. So your index might have a higher percentage temporarily. Best, Erick > On Apr 23, 2020, at 4:01 AM, Kayak28 wrote: > > Hello, Erick Eri

Re: Cause of java.io.IOException: No space left on device Error

2020-04-23 Thread Erick Erickson
In addition to what Dario mentioned, background merges happen all the time, optimize is just a special case (and very expensive). You say “one of my Solr cores has 47G”, but segment merging can easily occur on multiple cores at once, so that’s not definitive. We usually recommend that people hav

Re: SegmentsInfoRequestHandler does not release IndexWriter

2020-04-23 Thread Andrzej Białecki
Hi Tiziano, Indeed, this looks like a bug - good catch! Please file a Jira issue, I’ll get to it soon. > On 23 Apr 2020, at 00:19, Tiziano Degaetano > wrote: > > Hello, > > I’m digging in an issue getting timeouts doing a managed schema change using > the schema api. > The call hangs reloa

Re: using S3 as the Directory for Solr

2020-04-23 Thread Jan Høydahl
Hi, Is your data so partitioned that it makes sense to consider splitting up in multiple collections and make some arrangement that will keep only a few collections live at a time, loading index files from S3 on demand? I cannot see how an S3 directory would be able to effectively cache files in

Re: Cause of java.io.IOException: No space left on device Error

2020-04-23 Thread Dario Rigolin
When solr starts an optimization of the index you have to have free at least same size (I don't know if 3 times is correct) of the core you are optimizing. Maybe your free space isn't enough to handle the optimization process. Sometimes you have to restart the Solr process to have released more spa

Re: FuzzyQuery causing Out of Memory Errors in 8.5.x

2020-04-23 Thread Colvin Cowie
https://issues.apache.org/jira/browse/SOLR-14428 On Thu, 23 Apr 2020 at 08:45, Colvin Cowie wrote: > I created a little test that fires off fuzzy queries from random UUID > strings for 5 minutes > *FIELD_NAME + ":" + UUID.randomUUID().toString().replace("-", "") + "~2"* > > The change in heap

Re: Solr indexing with Tika DIH - ZeroByteFileException

2020-04-23 Thread Charlie Hull
If users can upload any PDF, including broken or huge ones, and some cause a Tika error, you should decouple Tika from Solr and run it as a separate process to extract text before indexing with Solr. Otherwise some of what is uploaded *will* break Solr. https://lucidworks.com/post/indexing-with

Cause of java.io.IOException: No space left on device Error

2020-04-23 Thread Kayak28
Hello, Community: I am currently using Solr 5.3.1. on CentOS. The other day, I faced an error message that shows " java.io.IOException: No space left on device" My disk for Solr has empty space about 35GB and the total amount of the disk is 581GB. I doubted there was no enough space for Linux in

Re: Defaults Merge Policy

2020-04-23 Thread Kayak28
Hello, Erick Erickson: Thank you for answering my questions. Deleted docs in Solr 8.3.0 has not reached to 30% of the entire index, so I will monitor it for now. Again thank you for your response. Actually, the articles below helped me a lot. https://lucidworks.com/post/segment-merging-deleted-d

gzip compression solr 8.4.1

2020-04-23 Thread Johannes Siegert
Hi, we want to use gzip-compression between our application and the solr server. We use a standalone solr server version 8.4.1 and the prepackaged jetty as application server. We have enabled the jetty gzip module by adding these two files: {path_to_solr}/server/modules/gzip.mod (see below the

Re: FuzzyQuery causing Out of Memory Errors in 8.5.x

2020-04-23 Thread Colvin Cowie
I created a little test that fires off fuzzy queries from random UUID strings for 5 minutes *FIELD_NAME + ":" + UUID.randomUUID().toString().replace("-", "") + "~2"* The change in heap usage is really severe. On 8.5.1 Solr went OOM almost immediately on a 512mb heap, and with a 4GB heap it only j

RE: How upgrade to Solr 8 impact performance

2020-04-23 Thread Srinivas Kashyap
Can you share with details, what performance was degraded? Thanks, srinivas From: Natarajan, Rajeswari Sent: 23 April 2020 12:41 To: solr-user@lucene.apache.org Subject: Re: How upgrade to Solr 8 impact performance With the same hardware and configuration we also saw performance degradation fro

Re: How upgrade to Solr 8 impact performance

2020-04-23 Thread Natarajan, Rajeswari
With the same hardware and configuration we also saw performance degradation from 7.6 to 8.4.1 as this is why we are checking here to see if anyone else saw this behavior. -Rajeswari On 4/22/20, 7:16 AM, "Paras Lehana" wrote: Hi Rajeswari, I can only share my experience of moving fr

Re: Potential bug with optimistic concurrency

2020-04-23 Thread Sachin Divekar
Missed an important detail. It works correctly for single shard collections. -- Sachin On Wed, Apr 22, 2020 at 10:03 PM Sachin Divekar wrote: > Hi all, > > I am facing the exact same issue reported > https://issues.apache.org/jira/browse/SOLR-8733 and > https://issues.apache.org/jira/browse/SOL