Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread Yonik Seeley
On Fri, Jan 25, 2013 at 4:09 PM, davers wrote: > I'm not sure I understand. I thought ID had to be unique. Right - the group becomes part of the ID (the prefix), not the whole ID. > for example > I have the following > > [ > { "id" : 1, "groupid" : 1 }, > { "id" : 2, "groupid" : 1}, > { "id" : 3

Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread davers
I'm not sure I understand. I thought ID had to be unique. for example I have the following [ { "id" : 1, "groupid" : 1 }, { "id" : 2, "groupid" : 1}, { "id" : 3 "groupid : 2 } ] How would I currently ensure that id 1 & 2 end up on the same shard using the ID ? -- View this message in context:

Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread Yonik Seeley
On Fri, Jan 25, 2013 at 3:59 PM, davers wrote: > I want to shard on groupid instead of id but it doesn't seem to be working. That's not yet implemented. Currently you need to put the group in the ID. From the release notes: * Simple multi-tenancy through enhanced document routing: - The "comp

Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread davers
Ok that worked thank you! Now it seems like it is still using the default hashing function. I put this in my solrconfig.xml under the tag groupid false I want to shard on groupid instead of id but it doesn't seem to be working. -- View this message in context: http://lucene.472

Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread Mark Miller
On Jan 25, 2013, at 3:15 PM, davers wrote: > I am using the coreadmin to assign nodes to the cluster. How can I tell the > collection how many shards it has without using the collections api or > boostrapping > On the first CoreAdmin API call that creates a core, pass the numShards param. The

Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread davers
I am using the coreadmin to assign nodes to the cluster. How can I tell the collection how many shards it has without using the collections api or boostrapping -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-1-Custom-Hashing-DIH-tp4036316p4036340.html Sent from the So

Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread Yonik Seeley
On Fri, Jan 25, 2013 at 1:56 PM, davers wrote: > When I used 4.0 I could use my DIH on any shard and the documents would be > distributed based on the internal hashing algorithm and end up distributed > evenly across my three shards. > > I have just upgraded to Solr 4.1 and I have noticed that my

Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread davers
Also this doesn't seem like a problem with DIH specifically. I tried doing JSON updates with the same result. Documents are only added to the shard that I send the updates to. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-1-Custom-Hashing-DIH-tp4036316p4036332.html

Re: Solr 4.1 Custom Hashing DIH

2013-01-25 Thread davers
I found this: https://issues.apache.org/jira/browse/SOLR-2592 And tried adding the following to my solrconfig.xml groupid false However I am still getting all documents added to the shard that I run the DIH on. According to this comment on the tracker the default sharding should b

Solr 4.1 Custom Hashing DIH

2013-01-25 Thread davers
When I used 4.0 I could use my DIH on any shard and the documents would be distributed based on the internal hashing algorithm and end up distributed evenly across my three shards. I have just upgraded to Solr 4.1 and I have noticed that my documents always end up on the shard that I run the DIH o