Lots of questions indeed :)
1. Total virtual machines: 3
2. Replication factor: 0 (don't have any replicas yet)
3. Each machine has 1 shard which has 20GB of data. So data for a
collection is spread across 3 machines totalling to 60GB
4. Start solr:
java -Xmx10000m
-javaagent:newrelic/newrelic.jar
-Dsolr.clustering.enabled=true
-Dsolr.solr.home=multicore
-Djetty.class.path=lib/ext/* "
-Dbootstrap_conf=true
-DnumShards=3
-DzkHost=localhost:2181 -jar start.jar"
5. Yes, all machines have 24GB RAM and 9GB heap. Separate process of ZK is
running on these machine.
6. top screenshot: http://i.imgur.com/g6w9Bim.png
Thanks!
On Tue, Apr 8, 2014 at 4:48 PM, Shawn Heisey <s...@elyograg.org> wrote:
> On 4/8/2014 5:30 PM, Utkarsh Sengar wrote:
> > I see sudden drop in throughput once every 3-4 days. The "downtime" is
> for
> > about 2-6minutes and things stabilize after that.
> >
> > But I am not sure what is causing it the problem.
> >
> > I have 3 shards with 20GB of data on each shard.
> > Solr dashboard: http://i.imgur.com/6RWT2Dj.png
> > Newrelic graphs when during the downtime of about 4hours:
> > http://i.imgur.com/9vhKiB2.png
> > JVM memory graph says its normal: http://i.imgur.com/pAycgdC.png
> >
> > I thought it was GC pauses but it should be in the newrelic logs.
> >
> > How can I go about investigating this problem? I am running solr 4.4.0, I
> > don't see a strong reason to upgrade yet.
>
> Lots of questions:
>
> How many total machines? What is your replicationFactor? Does each
> machine have one shard replica and therefore 20GB of total index data,
> or if you add up all the index directories for the cores on each
> machine, is there more than 20GB of data?
>
> What options are you passing to your JVM when you start the servlet
> container that runs Solr?
>
> The dashboard says that this machine has 24GB of RAM and a 9GB heap. Is
> this the case for all machines? Is there any software other than Solr
> on the machine?
>
> If it's a linux/unix machine, can you run top, press shift-M to sort by
> memory, and grab a screenshot? If it's a Windows machine, a similar
> list should be available in the task manager, but it must include all
> processes for all users on the whole machine, and it would be best if it
> showed virtual memory as well as private.
>
> Thanks,
> Shawn
>
>
--
Thanks,
-Utkarsh
