I am using Solr Cloud 4.4. It is pretty much a base configuration. We have 2 servers and 3 collections. Collection1 is 1 shard and the Collection2 and Collection3 both have 2 shards. Both servers are identical.
So, here is my process, I do a lot of queries on Collection1 and Collection2. I then do a bunch of inserts into Collection3. I am doing CSV uploads. I am also doing custom shard routing. All the products in a single upload will have the same shard key. All Solr interaction is through SolrJ with full Zookeeper awareness. My uploads are also using soft commits. I tried this on a record set of 936 products. Everything worked fine. I then sent over a record set of 300k products. The upload into Collection3 is chunked. I tried both 1000 and 200,000 with similar results. The first upload to Solr would just hang. There would simply be no response from Solr. A few of the products from this request would make it into the index, but not many. In this state, queries continued to work, but deletes did not. My only solution was to kill each Solr process. As an experiment, I did the large catalog first. First, I reset everything. With A chunk size of 1000, about 110,000 out of 300,000 records made it into Solr before the process hung. Again, queries worked, but deletes did not and I had to kill Solr. It hung after about 30 seconds. Timing-wise, this is at about the second autocommit cycle, given the default autocommit of 15 seconds. I am not sure if this is related or not. As an additional experiment, I ran the entire test with just a single node in the cluster. This time, everything ran fine. Does anyone have any ideas? Everything is pretty default. These servers are Azure VMs, although I have seen similar behavior running two Solr instances on a single internal server as well. I had also noticed similar behavior before with Solr 4.3. It definitely has something do with the clustering, but I am not sure what. And I don't see any error message (or really anything else) in the Solr logs. Thanks. -- *KEVIN OSBORN* LEAD SOFTWARE ENGINEER CNET Content Solutions OFFICE 949.399.8714 CELL 949.310.4677 SKYPE osbornk 5 Park Plaza, Suite 600, Irvine, CA 92614 [image: CNET Content Solutions]
