2014-05-30 12:24 GMT-04:00 Erick Erickson <erickerick...@gmail.com>: > Let's back up a bit here. Why are you copying your indexes around? > SolrCloud does all this for you. I suspect you've somehow made a mis-step. >
I started by copying the index around because my 4.5.1 instance is not setup as Cloud and I wanted to avoid reindexing all my data when migrating to my new 4.8.1 SolrCloud setup. I've now put that aside and I'm just trying to get replication happening when I populate an empty collection. > So here's what I'd do by preference; Just set up a new collection and > re-index. Make sure all of the nodes are up and then just go ahead and > index to any of them. If you're using SolrJ, CloudSolrServer will be a bit > more efficient than sending the docs to random nodes, but that's not > necessary. > I've been trying that this morning. Stop the instances, deleted the contents of /data on all my 4.8.1 instances then started them again... they all show up in a 1 shard cluster as 4 replicas and one is the leader... they're still shown as down in clusterstate. Then I sent a document to be added to one of the nodes specifically. Only that node now contains the document. It hasn't been replicated to the other instances. When I issue queries to the collection for that document through my load balancer it works roughtly 1/4 times, in accordance with the fact that it's only on the instance where it was added. Must I use the CLI API for collections to create this new collection or can I just do it old style by creating subfolder in /solr directory with my confs? Here's the log of these operations LOG of Instance where document was added: -------------------- 2758138 [qtp1781256139-14] INFO org.apache.solr.update.processor.LogUpdateProcessor – [mycollection] webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json} {add=[Listing_3446279]} 0 271 2769177 [qtp1781256139-12] INFO org.apache.solr.core.SolrCore – [mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=1 [... More Pings ... ] 2773138 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false} 2773377 [commitScheduler-7-thread-1] INFO org.apache.solr.search.SolrIndexSearcher – Opening Searcher@175816a5[mycollection] main 2773389 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener sending requests to Searcher@175816a5[mycollection] main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)} 2773389 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – QuerySenderListener done. 2773390 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore – [mycollection] Registered new searcher Searcher@175816a5[mycollection] main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)} 2773390 [commitScheduler-7-thread-1] INFO org.apache.solr.update.UpdateHandler – end_commit_flush [... More Pings ... ] 2799792 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} 2799883 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore – SolrDeletionPolicy.onCommit: commits: num=2 commit{dir=NRTCachingDirectory(MMapDirectory@/opt/solr-4.8.0/example/solr/mycollection/data/index lockFactory=NativeFSLockFactory@/opt/solr-4.8.0/example/solr/mycollection/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1} commit{dir=NRTCachingDirectory(MMapDirectory@/opt/solr-4.8.0/example/solr/mycollection/data/index lockFactory=NativeFSLockFactory@/opt/solr-4.8.0/example/solr/mycollection/data/index; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2} 2799884 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore – newest commit generation = 2 2799887 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore – SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher 2799887 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – end_commit_flush 2799888 [qtp1781256139-18] INFO org.apache.solr.update.processor.LogUpdateProcessor – [mycollection] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from= http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false} {commit=} 0 96 2800051 [qtp1781256139-14] INFO org.apache.solr.update.processor.LogUpdateProcessor – [mycollection] webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json} {commit=} 0 611 2802085 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} 2802085 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – No uncommitted changes. Skipping IW.commit. 2802086 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore – SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher 2802086 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – end_commit_flush 2802087 [qtp1781256139-18] INFO org.apache.solr.update.processor.LogUpdateProcessor – [mycollection] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from= http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false} {commit=} 0 2 2802126 [qtp1781256139-14] INFO org.apache.solr.update.processor.LogUpdateProcessor – [mycollection] webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json} {commit=} 0 44 [... More Pings ... ] ---------------------- LOG of 3 other instances : ---------------------- 2773666 [qtp1781256139-11] INFO org.apache.solr.core.SolrCore – [mycollection] webapp=/solr path=/admin/ping params={} status=0 QTime=2 2774531 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} 2774531 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – No uncommitted changes. Skipping IW.commit. 2774532 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore – SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher 2774533 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – end_commit_flush 2774533 [qtp1781256139-18] INFO org.apache.solr.update.processor.LogUpdateProcessor – [mycollection] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from= http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false} {commit=} 0 23 2776603 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false} 2776603 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – No uncommitted changes. Skipping IW.commit. 2776604 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore – SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher 2776604 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – end_commit_flush 2776604 [qtp1781256139-18] INFO org.apache.solr.update.processor.LogUpdateProcessor – [mycollection] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from= http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false} {commit=} 0 2 2777428 [qtp1781256139-12] INFO org.apache.solr.core.SolrCore – [mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0 QTime=0 ---------------------- > If that isn't feasible, set up a _one_ node "cloud" and get that running > and showing as up with your current index. Then use the Collections > "ADDREPLICA" command to bring up the other three nodes. All the index > synching should then "just happen". > Will try that. > You're also confusing master/slave replication with SolrCloud. In the > normal state, there is no use of the older-style replication, except when a > downed or new node comes online. If "peer synch" isn't possible, then old > style replication happens, but that should be the only time it's used. > Outside those edge cases, updates go to all nodes when sent to any node, > otherwise NRT wouldn't work. > That's what I understood from the research I made. I just think that I missed some config or something that will make replication happen. Thanks for your time, Marc