bq: it's not getting replicated
This is definitely not what's expected. Are you by chance _configuring_
replication while at the same time using SolrCloud? Posting your
solrconfig.xml would help answer that.
This should be all that's in your solrconfig.xml file when running under
SolrCloud:
<requestHandler name="/replication" class="solr.ReplicationHandler" />
No parameters. No polling intervals. No URLs for the master (there isn't
one anyway). Nothing but what's above.
You also need to insure that your soft commit interval (or hard commit if
you have openSearcher set to true) is exceeded before you can search
documents that are being indexed.
If none of that is a problem, I'd really consider just doing the example
code first. Get _that_ set up without doing _any_ changes and get it
running, _then_ try your stuff. It really sounds like you've got a
fundamental mis-configuration and are beating your head against the wall to
no good purpose.
Best
Erick
On Fri, May 30, 2014 at 1:58 PM, Marc Campeau <cam...@gmail.com> wrote:
> As of now I'm trying reindexing everything. Basically I have now an empty
> collection and when I add a document it's not getting replicated. Not
> trying anymore to load my old index anymore.
>
> Marc
>
>
> 2014-05-30 16:44 GMT-04:00 Jason Hellman <jhell...@innoventsolutions.com>:
>
> > Marc,
> >
> > Fundamentally it’s a good solution design to always be capable of
> > reposting (reindexing) your data to Solr. You are demonstrating a
> classic
> > use case of this, which is upgrade. Is there a critical reason why you
> are
> > avoiding this step?
> >
> > Jason
> >
> > On May 30, 2014, at 10:38 AM, Marc Campeau <cam...@gmail.com> wrote:
> >
> > > 2014-05-30 12:24 GMT-04:00 Erick Erickson <erickerick...@gmail.com>:
> > >
> > >> Let's back up a bit here. Why are you copying your indexes around?
> > >> SolrCloud does all this for you. I suspect you've somehow made a
> > mis-step.
> > >>
> > >
> > > I started by copying the index around because my 4.5.1 instance is not
> > > setup as Cloud and I wanted to avoid reindexing all my data when
> > migrating
> > > to my new 4.8.1 SolrCloud setup. I've now put that aside and I'm just
> > > trying to get replication happening when I populate an empty
> collection.
> > >
> > >
> > >> So here's what I'd do by preference; Just set up a new collection and
> > >> re-index. Make sure all of the nodes are up and then just go ahead and
> > >> index to any of them. If you're using SolrJ, CloudSolrServer will be a
> > bit
> > >> more efficient than sending the docs to random nodes, but that's not
> > >> necessary.
> > >>
> > >
> > > I've been trying that this morning. Stop the instances, deleted the
> > > contents of /data on all my 4.8.1 instances then started them again...
> > > they all show up in a 1 shard cluster as 4 replicas and one is the
> > > leader... they're still shown as down in clusterstate. Then I sent a
> > > document to be added to one of the nodes specifically. Only that node
> now
> > > contains the document. It hasn't been replicated to the other
> instances.
> > >
> > > When I issue queries to the collection for that document through my
> load
> > > balancer it works roughtly 1/4 times, in accordance with the fact that
> > it's
> > > only on the instance where it was added.
> > >
> > > Must I use the CLI API for collections to create this new collection or
> > can
> > > I just do it old style by creating subfolder in /solr directory with my
> > > confs?
> > >
> > >
> > > Here's the log of these operations
> > >
> > > LOG of Instance where document was added:
> > > --------------------
> > > 2758138 [qtp1781256139-14] INFO
> > > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > > webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json}
> > > {add=[Listing_3446279]} 0 271
> > > 2769177 [qtp1781256139-12] INFO org.apache.solr.core.SolrCore –
> > > [mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0
> > > QTime=1
> > >
> > > [... More Pings ... ]
> > >
> > > 2773138 [commitScheduler-7-thread-1] INFO
> > > org.apache.solr.update.UpdateHandler – start
> > >
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> > > 2773377 [commitScheduler-7-thread-1] INFO
> > > org.apache.solr.search.SolrIndexSearcher – Opening
> > > Searcher@175816a5[mycollection]
> > > main
> > > 2773389 [searcherExecutor-5-thread-1] INFO
> org.apache.solr.core.SolrCore
> > > – QuerySenderListener sending requests to Searcher@175816a5
> > [mycollection]
> > > main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)}
> > > 2773389 [searcherExecutor-5-thread-1] INFO
> org.apache.solr.core.SolrCore
> > > – QuerySenderListener done.
> > > 2773390 [searcherExecutor-5-thread-1] INFO
> org.apache.solr.core.SolrCore
> > > – [mycollection] Registered new searcher Searcher@175816a5
> [mycollection]
> > > main{StandardDirectoryReader(segments_1:3:nrt _0(4.8):C1)}
> > > 2773390 [commitScheduler-7-thread-1] INFO
> > > org.apache.solr.update.UpdateHandler – end_commit_flush
> > >
> > > [... More Pings ... ]
> > >
> > > 2799792 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > > start
> > >
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > > 2799883 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > > SolrDeletionPolicy.onCommit: commits: num=2
> > > commit{dir=NRTCachingDirectory(MMapDirectory@
> > /opt/solr-4.8.0/example/solr/mycollection/data/index
> > > lockFactory=NativeFSLockFactory@
> > /opt/solr-4.8.0/example/solr/mycollection/data/index;
> > > maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
> > > commit{dir=NRTCachingDirectory(MMapDirectory@
> > /opt/solr-4.8.0/example/solr/mycollection/data/index
> > > lockFactory=NativeFSLockFactory@
> > /opt/solr-4.8.0/example/solr/mycollection/data/index;
> > > maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
> > > 2799884 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> newest
> > > commit generation = 2
> > > 2799887 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > > SolrIndexSearcher has not changed - not re-opening:
> > > org.apache.solr.search.SolrIndexSearcher
> > > 2799887 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > > end_commit_flush
> > > 2799888 [qtp1781256139-18] INFO
> > > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > > webapp=/solr path=/update
> > >
> >
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> > >
> >
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false
> > }
> > > {commit=} 0 96
> > > 2800051 [qtp1781256139-14] INFO
> > > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > > webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json}
> > {commit=}
> > > 0 611
> > > 2802085 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > > start
> > >
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > > 2802085 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > No
> > > uncommitted changes. Skipping IW.commit.
> > > 2802086 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > > SolrIndexSearcher has not changed - not re-opening:
> > > org.apache.solr.search.SolrIndexSearcher
> > > 2802086 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > > end_commit_flush
> > > 2802087 [qtp1781256139-18] INFO
> > > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > > webapp=/solr path=/update
> > >
> >
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> > >
> >
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false
> > }
> > > {commit=} 0 2
> > > 2802126 [qtp1781256139-14] INFO
> > > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > > webapp=/solr path=/update/ params={indent=on&version=2.2&wt=json}
> > {commit=}
> > > 0 44
> > >
> > > [... More Pings ... ]
> > > ----------------------
> > >
> > > LOG of 3 other instances :
> > > ----------------------
> > > 2773666 [qtp1781256139-11] INFO org.apache.solr.core.SolrCore –
> > > [mycollection] webapp=/solr path=/admin/ping params={} status=0 QTime=2
> > > 2774531 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > > start
> > >
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > > 2774531 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > No
> > > uncommitted changes. Skipping IW.commit.
> > > 2774532 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > > SolrIndexSearcher has not changed - not re-opening:
> > > org.apache.solr.search.SolrIndexSearcher
> > > 2774533 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > > end_commit_flush
> > > 2774533 [qtp1781256139-18] INFO
> > > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > > webapp=/solr path=/update
> > >
> >
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> > >
> >
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false
> > }
> > > {commit=} 0 23
> > > 2776603 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > > start
> > >
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> > > 2776603 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > No
> > > uncommitted changes. Skipping IW.commit.
> > > 2776604 [qtp1781256139-18] INFO org.apache.solr.core.SolrCore –
> > > SolrIndexSearcher has not changed - not re-opening:
> > > org.apache.solr.search.SolrIndexSearcher
> > > 2776604 [qtp1781256139-18] INFO org.apache.solr.update.UpdateHandler
> –
> > > end_commit_flush
> > > 2776604 [qtp1781256139-18] INFO
> > > org.apache.solr.update.processor.LogUpdateProcessor – [mycollection]
> > > webapp=/solr path=/update
> > >
> >
> params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=
> > >
> >
> http://192.168.150.90:8983/solr/mycollection/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false
> > }
> > > {commit=} 0 2
> > > 2777428 [qtp1781256139-12] INFO org.apache.solr.core.SolrCore –
> > > [mycollection] webapp=/solr path=/admin/ping params={} hits=0 status=0
> > > QTime=0
> > > ----------------------
> > >
> > >
> > >> If that isn't feasible, set up a _one_ node "cloud" and get that
> running
> > >> and showing as up with your current index. Then use the Collections
> > >> "ADDREPLICA" command to bring up the other three nodes. All the index
> > >> synching should then "just happen".
> > >>
> > >
> > > Will try that.
> > >
> > >
> > >> You're also confusing master/slave replication with SolrCloud. In the
> > >> normal state, there is no use of the older-style replication, except
> > when a
> > >> downed or new node comes online. If "peer synch" isn't possible, then
> > old
> > >> style replication happens, but that should be the only time it's used.
> > >> Outside those edge cases, updates go to all nodes when sent to any
> node,
> > >> otherwise NRT wouldn't work.
> > >>
> > >
> > > That's what I understood from the research I made. I just think that I
> > > missed some config or something that will make replication happen.
> > >
> > > Thanks for your time,
> > >
> > > Marc
> >
> >
>
