But this discussion says that if you don't specify numshards, you can still have shards and you can have custom hashing http://lucene.472066.n3.nabble.com/Solr-4-1-Custom-Hashing-DIH-td4036316.html, and I'm pretty sure I've read that elsewhere as well. Does anyone have more information on that?
That JIRA looks interesting, I don't have the bandwidth, but I'll keep my eye on it. On Mon, Aug 19, 2013 at 4:51 PM, Erick Erickson <erickerick...@gmail.com>wrote: > Right, you can't just tell Solr to create a single shard (i.e. by not > specifying numshards) then expect to be able to do anything > except index to a single shard. All the nodes will be replicas > of the single shard. From there it really doesn't matter what you do, > the documents will be routed to all the replicas since they're all > the same shard. > > bq: It seems silly to add them across all the shards and then have > each shard forward them to a potentially different shard based on the hash. > > This is a sticky wicket to do from scratch. Consider that > you'll have to create the collection with the number of shards you > expect to need (or you can split later). And then you'll have to keep > track of which node is the leader for each shard for the most > efficient routing. Then you'll have to accommodate nodes that go > down. Then > > Are you actually seeing a problem? Or is this theoretical? > Because I'd just forget about it if you weren't seeing actual problems. > Especially if you're using SolrJ because of this JIRA: > https://issues.apache.org/jira/browse/SOLR-4816 which is designed > to exactly route documents to the correct leader automagically. > > Of course if you have the bandwidth to help on that JIRA I'm sure Joel > & Mark would appreciate it. > > Best > Erick > > > > On Mon, Aug 19, 2013 at 6:26 PM, Katie McCorkell <kmcc...@uw.edu> wrote: > > > Hey All, > > > > If you don't specify numShards at the start, then you can do custom > > hashing, because Solr will just write the document to whatever shard you > > send it to. > > > > However, when I don't specify numshards, I'm having trouble creating more > > than one shard. It makes one shard and the others I add are simply > > replicas. Here's the params I'm using to start > > http://pastebin.com/818SguiA . Am I missing something? > > > > I only want custom hashing because before when I had it do the automatic > > hashing, I was posting all documents to one shard, and then it would > > forward them to the right place. I want to add them to different shards > to > > distribute the load, so that it's not just one of them handling the > > forwarding. It seems silly to add them across all the shards and then > have > > each shard forward them to a potentially different shard based on the > hash. > > > > Thanks! > > >
