Marc,
Fundamentally it’s a good solution design to always be capable of reposting
(reindexing) your data to Solr. You are demonstrating a classic use case of
this, which is upgrade. Is there a critical reason why you are avoiding this
step?
Jason
On May 30, 2014, at 10:38 AM, Marc Campeau <cam...@gmail.com> wrote:
> 2014-05-30 12:24 GMT-04:00 Erick Erickson <erickerick...@gmail.com>:
>
>> Let's back up a bit here. Why are you copying your indexes around?
>> SolrCloud does all this for you. I suspect you've somehow made a mis-step.
>>
>
> I started by copying the index around because my 4.5.1 instance is not
> setup as Cloud and I wanted to avoid reindexing all my data when migrating
> to my new 4.8.1 SolrCloud setup. I've now put that aside and I'm just
> trying to get replication happening when I populate an empty collection.
>
>
>> So here's what I'd do by preference; Just set up a new collection and
>> re-index. Make sure all of the nodes are up and then just go ahead and
>> index to any of them. If you're using SolrJ, CloudSolrServer will be a bit
>> more efficient than sending the docs to random nodes, but that's not
>> necessary.
>>
>
> I've been trying that this morning. Stop the instances, deleted the
> contents of /data on all my 4.8.1 instances then started them again...
> they all show up in a 1 shard cluster as 4 replicas and one is the
> leader... they're still shown as down in clusterstate. Then I sent a
> document to be added to one of the nodes specifically. Only that node now
> contains the document. It hasn't been replicated to the other instances.
>
> When I issue queries to the collection for that document through my load
> balancer it works roughtly 1/4 times, in accordance with the fact that it's
> only on the instance where it was added.
>
> Must I use the CLI API for collections to create this new collection or can
> I just do it old style by creating subfolder in /solr directory with my
> confs?
>
>
> Here's the log of these operations
>
> LOG of Instance where document was added:
> --------------------
> 2758138 [qtp1781256139-14] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json}
> {add=[Listing_3446279]} 0 271
> 2769177 [qtp1781256139-12] INFO org.apache.solr.core.SolrCore –
> [mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0
> QTime=1
>
> [... More Pings ... ]
>
> 2773138 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> 2773377 [commitScheduler-7-thread-1] INFO
> org.apache.solr.search.SolrIndexSearcher – Opening
> Searcher@175816a5[mycollection]
> main
> 2773389 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore
> – QuerySenderListener sending requests to Searcher@175816a5[mycollection]
> main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)}
> 2773389 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore
> – QuerySenderListener done.
> 2773390 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore
> – [mycollection] Registered new searcher Searcher@175816a5[mycollection]
> main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)}
> 2773390 [commitScheduler-7-thread-1] INFO
> org.apache.solr.update.UpdateHandler – end_commit_flush
>
> [... More Pings ... ]
>
> 2799792 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 2799883 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> SolrDeletionPolicy.onCommit: commits: num=2
> commit{dir=NRTCachingDirectory(MMapDirectory@/opt/solr-4.8.0/example/solr/mycollection/data/index
> lockFactory=NativeFSLockFactory@/opt/solr-4.8.0/example/solr/mycollection/data/index;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
> commit{dir=NRTCachingDirectory(MMapDirectory@/opt/solr-4.8.0/example/solr/mycollection/data/index
> lockFactory=NativeFSLockFactory@/opt/solr-4.8.0/example/solr/mycollection/data/index;
> maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> 2799884 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore – newest
> commit generation = 2
> 2799887 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> SolrIndexSearcher has not changed - not re-opening:
> org.apache.solr.search.SolrIndexSearcher
> 2799887 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> end_commit_flush
> 2799888 [qtp1781256139-18] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> webapp=/solr path=/update
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
> {commit=} 0 96
> 2800051 [qtp1781256139-14] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json} {commit=}
> 0 611
> 2802085 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 2802085 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – No
> uncommitted changes. Skipping IW.commit.
> 2802086 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> SolrIndexSearcher has not changed - not re-opening:
> org.apache.solr.search.SolrIndexSearcher
> 2802086 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> end_commit_flush
> 2802087 [qtp1781256139-18] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> webapp=/solr path=/update
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
> {commit=} 0 2
> 2802126 [qtp1781256139-14] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json} {commit=}
> 0 44
>
> [... More Pings ... ]
> ----------------------
>
> LOG of 3 other instances :
> ----------------------
> 2773666 [qtp1781256139-11] INFO org.apache.solr.core.SolrCore –
> [mycollection] webapp=/solr path=/admin/ping params={} status=0 QTime=2
> 2774531 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 2774531 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – No
> uncommitted changes. Skipping IW.commit.
> 2774532 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> SolrIndexSearcher has not changed - not re-opening:
> org.apache.solr.search.SolrIndexSearcher
> 2774533 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> end_commit_flush
> 2774533 [qtp1781256139-18] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> webapp=/solr path=/update
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
> {commit=} 0 23
> 2776603 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> 2776603 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler – No
> uncommitted changes. Skipping IW.commit.
> 2776604 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> SolrIndexSearcher has not changed - not re-opening:
> org.apache.solr.search.SolrIndexSearcher
> 2776604 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> end_commit_flush
> 2776604 [qtp1781256139-18] INFO
> org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> webapp=/solr path=/update
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}
> {commit=} 0 2
> 2777428 [qtp1781256139-12] INFO org.apache.solr.core.SolrCore –
> [mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0
> QTime=0
> ----------------------
>
>
>> If that isn't feasible, set up a _one_ node "cloud" and get that running
>> and showing as up with your current index. Then use the Collections
>> "ADDREPLICA" command to bring up the other three nodes. All the index
>> synching should then "just happen".
>>
>
> Will try that.
>
>
>> You're also confusing master/slave replication with SolrCloud. In the
>> normal state, there is no use of the older-style replication, except when a
>> downed or new node comes online. If "peer synch" isn't possible, then old
>> style replication happens, but that should be the only time it's used.
>> Outside those edge cases, updates go to all nodes when sent to any node,
>> otherwise NRT wouldn't work.
>>
>
> That's what I understood from the research I made. I just think that I
> missed some config or something that will make replication happen.
>
> Thanks for your time,
>
> Marc
