We also noticed that disk IO shoots up to 100% on 1 of the nodes. Do all updates get sent to one machine or something?
On Mon, Jan 20, 2014 at 2:42 PM, Software Dev <static.void....@gmail.com>wrote: > We commit have a soft commit every 5 seconds and hard commit every 30. As > far as docs/second it would guess around 200/sec which doesn't seem that > high. > > > On Mon, Jan 20, 2014 at 2:26 PM, Erick Erickson > <erickerick...@gmail.com>wrote: > >> Questions: How often do you commit your updates? What is your >> indexing rate in docs/second? >> >> In a SolrCloud setup, you should be using a CloudSolrServer. If the >> server is having trouble keeping up with updates, switching to CUSS >> probably wouldn't help. >> >> So I suspect there's something not optimal about your setup that's >> the culprit. >> >> Best, >> Erick >> >> On Mon, Jan 20, 2014 at 4:00 PM, Software Dev <static.void....@gmail.com> >> wrote: >> > We are testing our shiny new Solr Cloud architecture but we are >> > experiencing some issues when doing bulk indexing. >> > >> > We have 5 solr cloud machines running and 3 indexing machines (separate >> > from the cloud servers). The indexing machines pull off ids from a queue >> > then they index and ship over a document via a CloudSolrServer. It >> appears >> > that the indexers are too fast because the load (particularly disk io) >> on >> > the solr cloud machines spikes through the roof making the entire >> cluster >> > unusable. It's kind of odd because the total index size is not even >> > large..ie, < 10GB. Are there any optimization/enhancements I could try >> to >> > help alleviate these problems? >> > >> > I should note that for the above collection we have only have 1 shard >> thats >> > replicated across all machines so all machines have the full index. >> > >> > Would we benefit from switching to a ConcurrentUpdateSolrServer where >> all >> > updates get sent to 1 machine and 1 machine only? We could then remove >> this >> > machine from our cluster than that handles user requests. >> > >> > Thanks for any input. >> > >
