Re: Very Slow Commits After Solr Index Optimization

2016-09-22 Thread Shawn Heisey
On 9/22/2016 3:27 PM, vsolakhian wrote: > This is not the cause of the problem though. The disk cache is > important for queries and overall performance during optimization, but > once it is done, everything should go back to "normal" (whatever that > normal is). In our case it is the SOFT COMMIT (

Re: Very Slow Commits After Solr Index Optimization

2016-09-22 Thread vsolakhian
Thanks again, Shawn. You are completely right about the use of disk cache and the special note regarding the optimize operation in Solr wiki. This is not the cause of the problem though. The disk cache is important for queries and overall performance during optimization, but once it is done, ever

Re: Very Slow Commits After Solr Index Optimization

2016-09-22 Thread Shawn Heisey
On 9/22/2016 1:01 PM, vsolakhian wrote: > Our index is in HDFS, but we did not change any configuration after we > deleted 35% of records and optimized. > > The relatively slow commit (soft commit and warming up took 1.5 minutes) is > OK for our use case (adding hundreds of thousands and even milli

Re: Very Slow Commits After Solr Index Optimization

2016-09-22 Thread vsolakhian
Hi Shawn, Thank you for response. Everything you said is correct in general. Our index is in HDFS, but we did not change any configuration after we deleted 35% of records and optimized. The relatively slow commit (soft commit and warming up took 1.5 minutes) is OK for our use case (adding hundre

Re: Very Slow Commits After Solr Index Optimization

2016-09-21 Thread Shawn Heisey
On 9/20/2016 4:13 PM, vsolakhian wrote: > We knew that optimization is not a good idea and it was discussed in forums > that it should be completely removed from API and Solr Admin, but discussing > is one thing and doing it is another. To make the story short, we tried to > optimize through Solr