Re: Solr index slow response

2019-03-19 Thread Aaron Yingcai Sun
>> It could be an issue with jdk 8 that may not be suitable for such large >> heaps. Have more nodes with smaller heaps (eg 31 gb) >>> Am 18.03.2019 um 11:47 schrieb Aaron Yingcai Sun : >>> >>> Hello, Solr! >>> >>> >>> We are having

Re: Solr index slow response

2019-03-19 Thread Aaron Yingcai Sun
r index slow response If you start indexing with just a single thread/client, do you still see slow bulks? Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 19 Mar 2019, at 12:54, Aaron Yi

Re: Solr index slow response

2019-03-19 Thread Aaron Yingcai Sun
rting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 19 Mar 2019, at 09:17, Aaron Yingcai Sun wrote: > > We have around 80 million documents to index, total index size around 3TB, I > guess I'm not the first one to work with this big a

Re: Solr index slow response

2019-03-19 Thread Aaron Yingcai Sun
is. If you must have consistent response times for update requests, you’ll need to do that outside of Solr. But if you need high data import rates, we can probably help. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Mar 18, 2019, at 8:31 AM, A

Re: Solr index slow response

2019-03-18 Thread Aaron Yingcai Sun
doesn't result in the large bursts of IO due to the infrequent hard commits. For more info, Erick Erickson has a great write up: https://lucidworks.com/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ Best, Chris On Mon, Mar 18, 2019 at 9:36 AM Aaron Yingcai Sun

Re: Solr index slow response

2019-03-18 Thread Aaron Yingcai Sun
ion: https://sematext.com/spm HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 18 Mar 2019, at 13:14, Aaron Yingcai Sun wrote: > > Hello, Emir, > > Thanks for the reply, this i

Re: Solr index slow response

2019-03-18 Thread Aaron Yingcai Sun
olr-user@lucene.apache.org Subject: Re: Solr index slow response On Mon, 2019-03-18 at 10:47 +, Aaron Yingcai Sun wrote: > Solr server is running on a quit powerful server, 32 cpus, 400GB RAM, > while 300 GB is reserved for solr, [...] 300GB for Solr sounds excessive. > Our applica

Re: Solr index slow response

2019-03-18 Thread Aaron Yingcai Sun
3257436126790475, "processCpuTime":9386945000, "systemCpuLoad":0.3279781055816521, "totalPhysicalMemorySize":406480175104, "totalSwapSpaceSize":4302303232, "maxFileDescriptorCount":32768, "openFileDescriptorCount&quo

Solr index slow response

2019-03-18 Thread Aaron Yingcai Sun
Hello, Solr! We are having some performance issue when try to send documents for solr to index. The repose time is very slow and unpredictable some time. Solr server is running on a quit powerful server, 32 cpus, 400GB RAM, while 300 GB is reserved for solr, while this happening, cpu usage is