im looking at the clusterstate.json t see why it is doing this I really
dont understand it though...
{"collection1":{
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node1":{
"state":"active",
"base_url":"http://10.135.2.153:8981/solr";,
"core":"collection1",
"node_name":"10.135.2.153:8981_solr",
"leader":"true"},
"core_node10":{
"state":"active",
"base_url":"http://10.135.2.153:8982/solr";,
"core":"collection1",
"node_name":"10.135.2.153:8982_solr"}}},
"shard2":{
"range":"0-7fffffff",
"state":"inactive",
"replicas":{
"core_node9":{
"state":"active",
"base_url":"http://10.135.2.153:8984/solr";,
"core":"collection1",
"node_name":"10.135.2.153:8984_solr",
"leader":"true"},
"core_node11":{
"state":"active",
"base_url":"http://10.135.2.153:8983/solr";,
"core":"collection1",
"node_name":"10.135.2.153:8983_solr"}}},
"shard1_1":{
"range":null,
"state":"active",
"parent":null,
"replicas":{
"core_node6":{
"state":"active",
"base_url":"http://10.135.2.153:8981/solr";,
"core":"collection1_shard1_1_replica1",
"node_name":"10.135.2.153:8981_solr",
"leader":"true"},
"core_node8":{
"state":"active",
"base_url":"http://10.135.2.153:8984/solr";,
"core":"collection1_shard1_1_replica2",
"node_name":"10.135.2.153:8984_solr"}}},
"shard1_0":{
"range":null,
"state":"active",
"parent":null,
"replicas":{
"core_node5":{
"state":"active",
"base_url":"http://10.135.2.153:8981/solr";,
"core":"collection1_shard1_0_replica1",
"node_name":"10.135.2.153:8981_solr",
"leader":"true"},
"core_node7":{
"state":"active",
"base_url":"http://10.135.2.153:8983/solr";,
"core":"collection1_shard1_0_replica2",
"node_name":"10.135.2.153:8983_solr"}}},
"shard2_0":{
"range":"0-3fffffff",
"state":"active",
"replicas":{
"core_node13":{
"state":"active",
"base_url":"http://10.135.2.153:8984/solr";,
"core":"collection1_shard2_0_replica1",
"node_name":"10.135.2.153:8984_solr",
"leader":"true"},
"core_node14":{
"state":"active",
"base_url":"http://10.135.2.153:8982/solr";,
"core":"collection1_shard2_0_replica2",
"node_name":"10.135.2.153:8982_solr"}}},
"shard2_1":{
"range":"40000000-7fffffff",
"state":"active",
"replicas":{
"core_node12":{
"state":"active",
"base_url":"http://10.135.2.153:8984/solr";,
"core":"collection1_shard2_1_replica1",
"node_name":"10.135.2.153:8984_solr",
"leader":"true"},
"core_node15":{
"state":"active",
"base_url":"http://10.135.2.153:8981/solr";,
"core":"collection1_shard2_1_replica2",
"node_name":"10.135.2.153:8981_solr"}}}},
"maxShardsPerNode":"1",
"router":{"name":"compositeId"},
"replicationFactor":"1",
"autoCreated":"true"}}
--
Thanks,
Jeff Courtade
M: 240.507.6116
On Wed, Aug 26, 2015 at 8:44 AM, Jeff Courtade <courtadej...@gmail.com>
wrote:
> Hi,
>
>
> So i got the shards too split. But they are very unbalanced.
>
>
> 7204922 total docs on the original collection
>
> shard1_0 numdocs 3661699
>
> shard1_1 numdocs 3543132
>
> shard2_0 numdocs 0
>
> shard2_1 numdcs 0
>
> Any ideas?
>
> This is what i had to do to get this to split with the custom libs
>
> I got shard1 to split successfully and it created replicas on the other
> servers in the cloud for the new shard/shards.
>
>
> This is the jist of it.
>
>
> When you split a shard solr creates a 2 new cores.
>
> When creating a core it uses the solr/solr.xml settings for classpath
> etc....
>
> This is why searches etc work fine and can find the opa plugins but when
> we called shardsplit it could not.
>
>
> I had to move the custom jars outside of the collection directory and add
> this to solr/solr.xml on the 4 nodes.
>
>
> info here .... https://wiki.apache.org/solr/Solr.xml%204.4%20and%20beyond
>
>
>
> <solr>
>
>
> <str name="sharedLib">${sharedLib:../lib}</str>
>
>
> when you restart you can see it in the log loading the jars form the new
> location.
>
>
>
> INFO - 2015-08-25 23:40:52.297; org.apache.solr.core.CoreContainer;
> loading shared library: /opt/solr/solr-4.7.2/solr01/solr/../lib
>
> INFO - 2015-08-25 23:40:52.298; org.apache.solr.core.SolrResourceLoader;
> Adding 'file:/opt/solr/solr-4.7.2/solr01/lib/commons-pool-1.6.jar' to
> classloader
>
> INFO - 2015-08-25 23:40:52.298; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/opt/solr/solr-4.7.2/solr01/lib/query-processing-language-0.2-SNAPSHOT.jar'
> to classloader
>
> INFO - 2015-08-25 23:40:52.299; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/opt/solr/solr-4.7.2/solr01/lib/jetty-continuation-8.1.10.v20130312.jar'
> to classloader
>
> INFO - 2015-08-25 23:40:52.301; org.apache.solr.core.SolrResourceLoader;
> Adding 'file:/opt/solr/solr-4.7.2/solr01/lib/groovy-all-2.0.4.jar' to
> classloader
>
> INFO - 2015-08-25 23:40:52.302; org.apache.solr.core.SolrResourceLoader;
> Adding 'file:/opt/solr/solr-4.7.2/solr01/lib/qpl-solr472-0.2-SNAPSHOT.jar'
> to classloader
>
> INFO - 2015-08-25 23:40:52.302; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/opt/solr/solr-4.7.2/solr01/lib/jetty-jmx-8.1.10.v20130312.jar' to
> classloader
>
> INFO - 2015-08-25 23:40:52.303; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/opt/solr/solr-4.7.2/solr01/lib/jetty-deploy-8.1.10.v20130312.jar' to
> classloader
>
> INFO - 2015-08-25 23:40:52.303; org.apache.solr.core.SolrResourceLoader;
> Adding 'file:/opt/solr/solr-4.7.2/solr01/lib/ext/' to classloader
>
> INFO - 2015-08-25 23:40:52.303; org.apache.solr.core.SolrResourceLoader;
> Adding
> 'file:/opt/solr/solr-4.7.2/solr01/lib/jetty-xml-8.1.10.v20130312.jar' to
> classloader
>
> so I then ran the split and checked on it in the morning
>
>
> http://dj01.aws.narasearch.us:8981/solr/admin/collections?action=SPLITSHARD&collection=collection1&shard=shard1
>
>
> it succeeded and created replicas.
>
> ls /opt/solr/solr-4.7.2/solr0*/solr/
>
> /opt/solr/solr-4.7.2/solr01/solr/:
> bin collection1_shard1_0_replica1 README.txt zoo.cfg
> collection1 collection1_shard1_1_replica1 solr.xml
>
> /opt/solr/solr-4.7.2/solr02/solr/:
> bin collection1 README.txt solr.xml zoo.cfg
>
> /opt/solr/solr-4.7.2/solr03/solr/:
> bin collection1 collection1_shard1_0_replica2 README.txt solr.xml
> zoo.cfg
>
> /opt/solr/solr-4.7.2/solr04/solr/:
> bin collection1 collection1_shard1_1_replica2 README.txt solr.xml
> zoo.cfg
>
>
> and it actually distributed it
>
> [root@dj01 solr]# du -sh *
> 4.0K bin
> 41G collection1
> 18G collection1_shard1_0_replica1
> 16G collection1_shard1_1_replica1
> 4.0K README.txt
> 4.0K solr.xml
> 4.0K zoo.cfg
> [root@dj01 solr]# du -sh
> /opt/solr/solr-4.7.2/solr04/solr/collection1_shard1_1_replica2
> 16G /opt/solr/solr-4.7.2/solr04/solr/collection1_shard1_1_replica2
> [root@dj01 solr]# du -sh
> /opt/solr/solr-4.7.2/solr03/solr/collection1_shard1_0_replica2
> 18G /opt/solr/solr-4.7.2/solr03/solr/collection1_shard1_0_replica2
>
>
> Jeff Courtade
> M: 240.507.6116
> On Aug 25, 2015 11:09 PM, "Anshum Gupta" <ans...@anshumgupta.net> wrote:
>
>> Can you elaborate a bit more on the setup, what do the custom plugins do,
>> what error do you get ? It seems like a classloader/classpath issue to me
>> which doesn't really relate to Shard splitting.
>>
>>
>> On Tue, Aug 25, 2015 at 7:59 PM, Jeff Courtade <courtadej...@gmail.com>
>> wrote:
>>
>> > I am getting failures when trying too split shards on solr 4.2.7 with
>> > custom plugins.
>> >
>> > It fails regularily it cannot find the jar files for plugins when
>> creating
>> > the new cores/shards.
>> >
>> > Ideas?
>> >
>> > --
>> > Thanks,
>> >
>> > Jeff Courtade
>> > M: 240.507.6116
>> >
>>
>>
>>
>> --
>> Anshum Gupta
>>
>
