Re: Solr 7.7.1 indexing failing with analysis error: startOffset must be non-negative, and endOffset must be >= startOffset, and offsets must not go backwards

How are you indexing the message, or what is the command that you used to index the message? Also, the attachment might not make it to the server, so you likely need to upload the file to a file sharing / storage site and share the link here. Regards, Edwin On Mon, 27 May 2019 at 15:24, SAM wro

open searchers memory leaks

hey , i have a memory leak in my solr cluster. sometimes searchers aren't closed after commit, and they slowly consume all the jvm of my solr instances. this is similar to a problem to i found here . but in my plugins i

Using BinaryDocValues with CustomScoreProvider

Hi, We implemented our own CustomScoreProvider which used BinaryDocValues to get data on each document and calculate a score for it, the CustomScoreProvider gets a docId and could easily be used with the BinaryDocValues methods when use lucene < 7. We want to move to a newer version and see the B

Re: Solr-8.1.0 uses much more memory

    An interesting note on the memory returning issue for the G1 collector.     https://openjdk.java.net/jeps/346 Entitled "JEP 346: Promptly Return Unused Committed Memory from G1" with a summary saying "Enhance the G1 garbage collector to automatically return Java heap memory to the operating

Re: Solr-8.1.0 uses much more memory

    My comments are inserted in-line this time. Thanks for the amplifications Shawn. On 27/05/2019 17:39, Shawn Heisey wrote: On 5/27/2019 9:49 AM, Joe Doupnik wrote: A few more numbers to contemplate. An experiment here, adding 80 PDF and PPTX files into an empty index. Solr v8.0 regul

Re: Solr-8.1.0 uses much more memory

On 5/27/2019 9:49 AM, Joe Doupnik wrote:     A few more numbers to contemplate. An experiment here, adding 80 PDF and PPTX files into an empty index. Solr v8.0 regular settings, 1.7GB quiesent memory consumption, 1.9GB while indexing, 2.92 minutes to do the job. Solr v8.0, using GC_TUNE from

Re: Solr-8.1.0 uses much more memory

    A few more numbers to contemplate. An experiment here, adding 80 PDF and PPTX files into an empty index. Solr v8.0 regular settings, 1.7GB quiesent memory consumption, 1.9GB while indexing, 2.92 minutes to do the job. Solr v8.0, using GC_TUNE from v8.1 solr.in.sh, 1.1GB quiesent, 1.3GB whi

Re: My problem with T-shirts and nested documents

If it really is a search problem, without transactional updates, I’d flatten and denormalize. Make a non-nested document for each kind of search response you want. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On May 27, 2019, at 3:55 AM, Gian Marco

Re: Solr-8.1.0 uses much more memory

Solr really should use a limited pool for handling external requests. We’ve driven it into OOM a few times with too much traffic, just creating a useless number of threads. But that requires separate pools for external requests and cluster-internal requests, which would probably require separat

Re: My problem with T-shirts and nested documents

Hi Walter, It was just an example, I thought it was simpler to explain than my real problem. thanks, GM On Fri, May 24, 2019 at 4:47 PM Walter Underwood wrote: > If you are really keeping inventory, use a relational database. Solr is a > really poor choice for this kind of application. > > wund

Re: Solr-8.1.0 uses much more memory

    You are certainly correct about using external load balancers when appropriate. However, a basic problem with servers, that of accepting more incoming items than can be handled gracefully is as we know an age-old one and solved by back pressure methods (particularly hard limits). My experie

Re: Solr-8.1.0 uses much more memory

I think it is not fair blaiming Solr not also having a load balancer. It is up to you and your needs to set up the required infrastucture including load balancing. The are many products available on the market. If your current system can't handle all requests then install more replicas. Regards B

Re: Solr-8.1.0 uses much more memory

    While on the topic of resource consumption and locks etc, there is one other aspect to which Solr has been vulnerable. It is failing to fend off too many requests at one time. The standard approach is, of course, named back pressure, such as not replying to a query until resources permit an

How to migrate the queries having core-across join and json.facet to SolrCloud

Hi, community. We are trying to migrate from single Solr instance to SolrCloud with Solr 7.4.0 due to the increase of documents. We have some join query running on current Solr, and need to migrate these because join queries has some restrictions when running on SolrCloud. (We cannot use custom do

Re: Solr-8.1.0 uses much more memory

    Generalizations tend to fail when confronted with conflicting evidence. The simple  evidence is asking how much real memory the Solr owned process has been allocated (top, or ps aux or similar) and that yields two very different values (the ~1.6GB of Solr v8.0 and 4.5+GB of Solr v8.1). I ha

Solr 7.7.1 indexing failing with analysis error: startOffset must be non-negative, and endOffset must be >= startOffset, and offsets must not go backwards

indexing a message on solr7.7.1 is failing with the following error. any help is appreciated. attaching schema files. 2019-05-24 19:32:42.010 ERROR (qtp1115201599-17) [c:bn_sample s:shard1 r:core_node2 x:bn_sample_shard1_replica_n1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: