Re: Auto-optimization for Solr indexes

2015-12-20 Thread Rahul Ramesh
Hi Erick, We index around several million documents/ day and we optimize everyday when the relative load is low. The reason we optimize is, we dont want the index sizes to grow too large and auto optimzie to kick in. When auto optimize kicks in, it results in unpredictable performance as it is CPU

Re: Auto-optimization for Solr indexes

2015-12-20 Thread Rahul Ramesh
> Here's a wonderful video about the process. you want the third one > down (TieredMergePolicy) as that's the default. > > > http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html > > Best, > Erick > > On Sun, Dec 20, 2015 at 8:26 PM,

Re: Pro and cons of using Solr Cloud vs standard Master Slave Replica

2016-01-11 Thread Rahul Ramesh
Please have a look at this post https://support.lucidworks.com/hc/en-us/articles/201298317-What-is-SolrCloud-And-how-does-it-compare-to-master-slave- We dont use Master slave architecture, however we use solr cloud and standalone solr for our documents. Indexing is a bit slow in cloud when compa

Understanding solr commit

2016-01-25 Thread Rahul Ramesh
We are facing some issue and we are finding it difficult to debug the problem. We wanted to understand how solr commit works. A background on our setup: We have 3 Node Solr Cluster running in version 5.3.1. Its a index heavy use case. In peak load, we index 400-500 documents/second. We also want t

Re: Understanding solr commit

2016-01-25 Thread Rahul Ramesh
We are using Solr Cloud with replication factor of 2 and no of shards as either 2 or 3. Thanks, Rahul On Mon, Jan 25, 2016 at 4:43 PM, Alessandro Benedetti wrote: > Let me answer in line : > > On 25 January 2016 at 11:02, Rahul Ramesh wrote: > > > We are facing so

Re: Understanding solr commit

2016-01-25 Thread Rahul Ramesh
ls about Solr heap parameters. Running Solr > on the edge of OOM is always risk of starting snowball effect and crashing > entire cluster. Also can you give us info about auto commit (both hard and > soft) you used when experienced OOM. > > Thanks, > Emir > > On 25.01.2016

Re: Understanding solr commit

2016-01-25 Thread Rahul Ramesh
e such tool that can give you info how > you Solr, JVM and host handle different load. One such tool can give you > enough info to tune your Solr. > > Regards, > Emir > > > On 25.01.2016 13:42, Rahul Ramesh wrote: > >> Can you give us bit more details about Solr heap

Re: Increasing Solr5 time out from 30 seconds while starting solr

2015-12-08 Thread Rahul Ramesh
Hi Debraj, I dont think increasing the timeout will help. Are you sure solr/ any other program is not running on 8789? Please check the output of lsof -i :8789 . Regards, Rahul On Tue, Dec 8, 2015 at 11:58 PM, Debraj Manna wrote: > Can someone help me on this? > On Dec 7, 2015 7:55 PM, "D" wro

Re: Moving to SolrCloud, specifying dataDir correctly

2015-12-14 Thread Rahul Ramesh
We currently moved data from magnetic drive to SSD. We run Solr in cloud mode. Only data is stored in the drive configuration is stored in ZK. We start solr using the -s option specifying the data dir Command to start solr ./bin/solr start -c -h -p -z -s We followed the following steps to migr

Re: solrcloud used a lot of memory and memory keep increasing during long time run

2015-12-15 Thread Rahul Ramesh
You should actually decrease solr heap size. Let me explain a bit. Solr requires very less heap memory for its operation and more memory for storing data in main memory. This is because solr uses mmap for storing the index files. Please check the link http://blog.thetaphi.de/2012/07/use-lucenes-mm