Re: Read timed out exception during Solr Full-import

2019-10-03 Thread amruth
I have changed settings in solrconfig.xml for autocommit thinking it's an issue with autocommit which didn't fix the issue, ${solr.autoCommit.maxTime:3} ${solr.autoCommit.maxDocs:15000} false Can someone please help me solve the issue? -- Sent from: https://lucene.47206

Re: Read timed out exception during Solr Full-import

2019-10-03 Thread amruth
I have changed settings in solrconfig.xml for autocommit thinking it's an issue with autocommit which didn't fix the issue, ${solr.autoCommit.maxTime:3} ${solr.autoCommit.maxDocs:15000} false amruth wrote > > Can someone please help me solve the issue? > > Hello Mikhail,

Re: Solr standalone timeouts after upgrading to SOLR 7

2019-10-03 Thread Walter Underwood
Just set Xms and Xmx the same. The server will be running for weeks, so allocate the memory and get on with it. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Oct 3, 2019, at 11:38 AM, ndra wrote: > >> I don’t think having the initial heap larger t

Re: Solr standalone timeouts after upgrading to SOLR 7

2019-10-03 Thread ndra
> I don’t think having the initial heap larger than the max heap is a legal configuration. > I have no idea what that would do. Sorry my wording was poor. I meant if, instead of my initial HEAP of 512MB, it was closer to say 6 or 8GB or equal to my Max allowed of 10GB. Appreciate the info though.

Re: Solrcloud export all results sorted by score

2019-10-03 Thread Chris Hostetter
: We show a table of search results ordered by score (relevancy) that was : obtained from sending a query to the standard /select handler. We're : working in the life-sciences domain and it is common for our result sets to : contain many millions of results (unfortunately). After users browse the

Re: Solr standalone timeouts after upgrading to SOLR 7

2019-10-03 Thread Walter Underwood
I don’t think having the initial heap larger than the max heap is a legal configuration. I have no idea what that would do. Modern GCs have a separate area for short-lived allocations. When that fills up, a minor GC happens. As allocations survive several minor GCs, they are moved to the long-li

Re: Solr standalone timeouts after upgrading to SOLR 7

2019-10-03 Thread ndra
> When the heap is out of free space that >can be recovered with minor GC, the JVM will increase the size if possible. >Once it is at max, it will do a major GC. Thanks Walter, One more quick question about the above, so if the initial HEAP was larger (or equal to the max as you suggested earlier

Re: Solr standalone timeouts after upgrading to SOLR 7

2019-10-03 Thread Walter Underwood
> On Oct 3, 2019, at 9:31 AM, ndra wrote: > > I was under the impression that by allocating a smaller initial HEAP I > could avoid having a larger GCs but if I am understanding what you all are > suggesting, the smaller initial HEAP is requiring more full GCs to maintian > a HEAP closer to 512MB

Re: Solr standalone timeouts after upgrading to SOLR 7

2019-10-03 Thread ndra
Thank you all for the responses. I do have a question about moving to G1. Is there any plans for SOLR to make this the default JAVA_OPT for SOLR? We have been running CMS without any issue prior to 7. The primary change we made between 6->7 was to increase the max HEAP size and increase the Xms a

Re: Solr standalone timeouts after upgrading to SOLR 7

2019-10-03 Thread Jörn Franke
As the others said: move to the newer GC. I would also use this opportunity to work with the default Java options of Solr 7 and then tune again. If you change major versions you should always review the GC settings if they still make sense. > Am 02.10.2019 um 23:14 schrieb Solr User : > > Hel

Re: Solr standalone timeouts after upgrading to SOLR 7

2019-10-03 Thread Walter Underwood
Always make Xmx and Xms the same. The heap will increase to the max before a major GC, so avoid the pauses to grow it. Use the G1 collector. CMS is really obsolete. We’ve had G1 in prod for at least three years. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my

Re: solr.log explanations for update handler

2019-10-03 Thread Dominique Bejean
Thank you Le mer. 2 oct. 2019 à 14:30, Mikhail Khludnev a écrit : > Bonjour, Dominique. > > Turns out it's zero and elapsed millis > > https://github.com/apache/lucene-solr/blob/302cd09b4ce7bd3049f8480287b3dd03bb838b0d/solr/core/src/java/org/apache/solr/update/processor/LogUpdateProcessorFactory

Re: how to improve indexing using autocommit

2019-10-03 Thread Erick Erickson
First of all, taking that long to index that few documents is surprising. My guess is that the data acquisition is the problem here, Solr is just idling along. But the best way I know of is to alias. You have two collections, call them “query” and “index”, that originally point to collection1 a

how to improve indexing using autocommit

2019-10-03 Thread babloorawat
Hi, We are performing solr indexing on a daily basis full import: once in a day delta import: after every 3 hours. We have around 4 docs for indexing. Time taken to do full import indexing is around 1 hour 45 minutes and we need to optimize it. I am wondering if anyone helps me figure out how

Re: Question regarding subqueries

2019-10-03 Thread Bram Biesbrouck
Hi Mikhail, You're right, I'm probably over-complicating things. I was stuck trying to combine a function in a regular query using a local variable, but Solr doesn't seem to bend the way my mind did ;-) Anyway, I worked around it using your suggestion and/or a slightly modified prefix parser plugi

Re: Solrcloud export all results sorted by score

2019-10-03 Thread Edward Turner
Hi Walter, Thank you also for your reply. Good to know of your experience. Roughly how many documents were you fetching? Unfortunately, it's possible that some of our users could attempt to "download" many records, meaning we'd need to make a request to Solr where rows >= 150M. A key challenge for

Re: Solrcloud export all results sorted by score

2019-10-03 Thread Jörn Franke
Maybe you can sort later using Spark or similar. For that you don’t need a full blown cluster - it runs also on localhost. > Am 03.10.2019 um 09:49 schrieb Edward Turner : > > Hi Erick, > > Many thanks for your detailed reply. It's really good information for us to > know, and although not exa

Re: Solrcloud export all results sorted by score

2019-10-03 Thread Edward Turner
Hi Erick, Many thanks for your detailed reply. It's really good information for us to know, and although not exactly what we wanted to hear (that /export wasn't designed to handle ranking), it's much better for us to definitively know one way or the other -- and this allows us to move forward. We'