So just a hunch... but when the slave downloads the data from the master, doesn't it do a commit to force solr to recognize the changes? In so doing, wouldn't that increase the generation number? In theory it shouldn't matter because the replication looks for files that are different to determine whether or not to do a full download or a partial replication. In the event of a full replication (an optimize would cause this), I think the replication handler considers this a "corruption" and forces a full download into this index.<timestamp> folder with the index.properties pointing at this folder to tell solr this is the new index directory. Since you mentioned you rebuild the index from scratch once per week I'd expect to see this behavior you are mentioning.
I remember debugging the code to find out how replication works in 4.0 because of a bug that was fixed in 4.1 but I haven't read through the 4.1 code to see how much (if any) has changed from this logic. In short, I don't know why you'd have the old "index/" directory there.. that seems either like a bug or something was locking that directory in the filesystem preventing it from being removed. What OS are you using and is the index/ directory stored on a local file system vs NFS? HTH Amit On Tue, Feb 12, 2013 at 2:26 AM, Bernd Fehling < bernd.fehl...@uni-bielefeld.de> wrote: > > Now this is strange, the index generation and index version > is changing with replication. > > e.g. master has index generation 118 index version 136059533234 > and slave has index generation 118 index version 136059533234 > are both same. > > Now add one doc to master with commit. > master has index generation 119 index version 1360595446556 > > Next replicate master to slave. The result is: > master has index generation 119 index version 1360595446556 > slave has index generation 120 index version 1360595564333 > > I have not seen this before. > I thought replication is just taking over the index from master to slave, > more like a sync? > > > > > Am 11.02.2013 09:29, schrieb Bernd Fehling: > > Hi list, > > > > after upgrading from solr4.0 to solr4.1 and running it for two weeks now > > it turns out that replication has problems and unpredictable results. > > My installation is single index 41 mio. docs / 115 GB index size / 1 > master / 3 slaves. > > - the master builds a new index from scratch once a week > > - a replication is started manually with Solr admin GUI > > > > What I see is one of these cases: > > - after a replication a new searcher is opened on index.xxxxxxx > directory and > > the old data/index/ directory is never deleted and besides the file > > replication.properties there is also a file index.properties > > OR > > - the replication takes place everything looks fine but when opening the > admin GUI > > the statistics report > > Last Modified: a day ago > > Num Docs: 42262349 > > Max Doc: 42262349 > > Deleted Docs: 0 > > Version: 45174 > > Segment Count: 1 > > > > Version Gen Size > > Master: 1360483635404 112 116.5 GB > > Slave: 1360483806741 113 116.5 GB > > > > > > In the first case, why is the replication doing that??? > > It is an offline slave, no search activity, just there fore backup! > > > > > > In the second case, why is the version and generation different right > after > > full replication? > > > > > > Any thoughts on this? > > > > > > - Bernd > > > > -- > ************************************************************* > Bernd Fehling Bielefeld University Library > Dipl.-Inform. (FH) LibTec - Library Technology > Universitätsstr. 25 and Knowledge Management > 33615 Bielefeld > Tel. +49 521 106-4060 bernd.fehling(at)uni-bielefeld.de > > BASE - Bielefeld Academic Search Engine - www.base-search.net > ************************************************************* >
