As of now I'm trying reindexing everything. Basically I have now an empty
collection and when I add a document it's not getting replicated. Not
trying anymore to load my old index anymore.
Marc
2014-05-30 16:44 GMT-04:00 Jason Hellman <jhell...@innoventsolutions.com>:
> Marc,
>
> Fundamentally it’s a good solution design to always be capable of
> reposting (reindexing) your data to Solr. You are demonstrating a classic
> use case of this, which is upgrade. Is there a critical reason why you are
> avoiding this step?
>
> Jason
>
> On May 30, 2014, at 10:38 AM, Marc Campeau <cam...@gmail.com> wrote:
>
> > 2014-05-30 12:24 GMT-04:00 Erick Erickson <erickerick...@gmail.com>:
> >
> >> Let's back up a bit here. Why are you copying your indexes around?
> >> SolrCloud does all this for you. I suspect you've somehow made a
> mis-step.
> >>
> >
> > I started by copying the index around because my 4.5.1 instance is not
> > setup as Cloud and I wanted to avoid reindexing all my data when
> migrating
> > to my new 4.8.1 SolrCloud setup. I've now put that aside and I'm just
> > trying to get replication happening when I populate an empty collection.
> >
> >
> >> So here's what I'd do by preference; Just set up a new collection and
> >> re-index. Make sure all of the nodes are up and then just go ahead and
> >> index to any of them. If you're using SolrJ, CloudSolrServer will be a
> bit
> >> more efficient than sending the docs to random nodes, but that's not
> >> necessary.
> >>
> >
> > I've been trying that this morning. Stop the instances, deleted the
> > contents of /data on all my 4.8.1 instances then started them again...
> > they all show up in a 1 shard cluster as 4 replicas and one is the
> > leader... they're still shown as down in clusterstate. Then I sent a
> > document to be added to one of the nodes specifically. Only that node now
> > contains the document. It hasn't been replicated to the other instances.
> >
> > When I issue queries to the collection for that document through my load
> > balancer it works roughtly 1/4 times, in accordance with the fact that
> it's
> > only on the instance where it was added.
> >
> > Must I use the CLI API for collections to create this new collection or
> can
> > I just do it old style by creating subfolder in /solr directory with my
> > confs?
> >
> >
> > Here's the log of these operations
> >
> > LOG of Instance where document was added:
> > --------------------
> > 2758138 [qtp1781256139-14] INFO
> > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json}
> > {add=[Listing_3446279]} 0 271
> > 2769177 [qtp1781256139-12] INFO org.apache.solr.core.SolrCore –
> > [mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0
> > QTime=1
> >
> > [... More Pings ... ]
> >
> > 2773138 [commitScheduler-7-thread-1] INFO
> > org.apache.solr.update.UpdateHandler – start
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> > 2773377 [commitScheduler-7-thread-1] INFO
> > org.apache.solr.search.SolrIndexSearcher – Opening
> > Searcher@175816a5[mycollection]
> > main
> > 2773389 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore
> > – QuerySenderListener sending requests to Searcher@175816a5
> [mycollection]
> > main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)}
> > 2773389 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore
> > – QuerySenderListener done.
> > 2773390 [searcherExecutor-5-thread-1] INFO org.apache.solr.core.SolrCore
> > – [mycollection] Registered new searcher Searcher@175816a5[mycollection]
> > main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)}
> > 2773390 [commitScheduler-7-thread-1] INFO
> > org.apache.solr.update.UpdateHandler – end_commit_flush
> >
> > [... More Pings ... ]
> >
> > 2799792 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> > start
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > 2799883 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > SolrDeletionPolicy.onCommit: commits: num=2
> > commit{dir=NRTCachingDirectory(MMapDirectory@
> /opt/solr-4.8.0/example/solr/mycollection/data/index
> > lockFactory=NativeFSLockFactory@
> /opt/solr-4.8.0/example/solr/mycollection/data/index;
> > maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
> > commit{dir=NRTCachingDirectory(MMapDirectory@
> /opt/solr-4.8.0/example/solr/mycollection/data/index
> > lockFactory=NativeFSLockFactory@
> /opt/solr-4.8.0/example/solr/mycollection/data/index;
> > maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> > 2799884 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore – newest
> > commit generation = 2
> > 2799887 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > SolrIndexSearcher has not changed - not re-opening:
> > org.apache.solr.search.SolrIndexSearcher
> > 2799887 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> > end_commit_flush
> > 2799888 [qtp1781256139-18] INFO
> > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > webapp=/solr path=/update
> >
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> >
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false
> }
> > {commit=} 0 96
> > 2800051 [qtp1781256139-14] INFO
> > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json}
> {commit=}
> > 0 611
> > 2802085 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> > start
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > 2802085 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> No
> > uncommitted changes. Skipping IW.commit.
> > 2802086 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > SolrIndexSearcher has not changed - not re-opening:
> > org.apache.solr.search.SolrIndexSearcher
> > 2802086 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> > end_commit_flush
> > 2802087 [qtp1781256139-18] INFO
> > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > webapp=/solr path=/update
> >
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> >
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false
> }
> > {commit=} 0 2
> > 2802126 [qtp1781256139-14] INFO
> > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json}
> {commit=}
> > 0 44
> >
> > [... More Pings ... ]
> > ----------------------
> >
> > LOG of 3 other instances :
> > ----------------------
> > 2773666 [qtp1781256139-11] INFO org.apache.solr.core.SolrCore –
> > [mycollection] webapp=/solr path=/admin/ping params={} status=0 QTime=2
> > 2774531 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> > start
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > 2774531 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> No
> > uncommitted changes. Skipping IW.commit.
> > 2774532 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > SolrIndexSearcher has not changed - not re-opening:
> > org.apache.solr.search.SolrIndexSearcher
> > 2774533 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> > end_commit_flush
> > 2774533 [qtp1781256139-18] INFO
> > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > webapp=/solr path=/update
> >
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> >
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false
> }
> > {commit=} 0 23
> > 2776603 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> > start
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > 2776603 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> No
> > uncommitted changes. Skipping IW.commit.
> > 2776604 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > SolrIndexSearcher has not changed - not re-opening:
> > org.apache.solr.search.SolrIndexSearcher
> > 2776604 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler –
> > end_commit_flush
> > 2776604 [qtp1781256139-18] INFO
> > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > webapp=/solr path=/update
> >
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> >
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false
> }
> > {commit=} 0 2
> > 2777428 [qtp1781256139-12] INFO org.apache.solr.core.SolrCore –
> > [mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0
> > QTime=0
> > ----------------------
> >
> >
> >> If that isn't feasible, set up a _one_ node "cloud" and get that running
> >> and showing as up with your current index. Then use the Collections
> >> "ADDREPLICA" command to bring up the other three nodes. All the index
> >> synching should then "just happen".
> >>
> >
> > Will try that.
> >
> >
> >> You're also confusing master/slave replication with SolrCloud. In the
> >> normal state, there is no use of the older-style replication, except
> when a
> >> downed or new node comes online. If "peer synch" isn't possible, then
> old
> >> style replication happens, but that should be the only time it's used.
> >> Outside those edge cases, updates go to all nodes when sent to any node,
> >> otherwise NRT wouldn't work.
> >>
> >
> > That's what I understood from the research I made. I just think that I
> > missed some config or something that will make replication happen.
> >
> > Thanks for your time,
> >
> > Marc
>
>
