Hi, It's simply expensive. You are rewriting your whole index.
Why are you running optimize? Are you seeing performance problems you are trying to fix with optimize? Otis -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ On Thu, Mar 2, 2017 at 10:39 AM, Caruana, Matthew <mcaru...@icij.org> wrote: > Thank you. The question remains however, if this is such a hefty operation > then why is it walking to the destination instead of running, so to speak? > > Is the process throttled in some way? > > > On 2 Mar 2017, at 16:20, David Hastings <hastings.recurs...@gmail.com> > wrote: > > > > Agreed, and since it takes three times the space is part of the reason it > > takes so long, so that 190gb index ends up writing another 380 gb until > it > > compresses down and deletes the two left over files. its a pretty hefty > > operation > > > > On Thu, Mar 2, 2017 at 10:13 AM, Alexandre Rafalovitch < > arafa...@gmail.com> > > wrote: > > > >> Optimize operation is no longer recommended for Solr, as the > >> background merges got a lot smarter. > >> > >> It is an extremely expensive operation that can require up to 3-times > >> amount of disk during the processing. > >> > >> This is not to say yours is a valid question, which I am leaving to > >> others to respond. > >> > >> Regards, > >> Alex. > >> ---- > >> http://www.solr-start.com/ - Resources for Solr users, new and > experienced > >> > >> > >> On 2 March 2017 at 10:04, Caruana, Matthew <mcaru...@icij.org> wrote: > >>> I’m currently performing an optimise operation on a ~190GB index with > >> about 4 million documents. The process has been running for hours. > >>> > >>> This is surprising, because the machine is an EC2 r4.xlarge with four > >> cores and 30GB of RAM, 24GB of which is allocated to the JVM. > >>> > >>> The load average has been steady at about 1.3. Memory usage is 25% or > >> less the whole time. iostat reports ~6% util. > >>> > >>> What gives? > >>> > >>> Running Solr 6.4.1. > >> > >
