He tried to upgrade lucene version from current 6.6.4 to 8.2. There're some challenges. One challenge is the codec changed, which caused the format of index is also changed.
That's why we did not implement it. If he resolved the coding challenges, then rolling upgrade will probably need option-2 to workaround it. Regards Gester On Wed, Nov 6, 2019 at 11:47 AM Jacob Barrett <jbarr...@pivotal.io> wrote: > What about “versioning” the region that backs the indexes? Old servers > with old license would continue to read/write to old region. New servers > would start re-indexing with the new version. Given the async nature of the > indexing would the mismatch in indexing for some period of time have an > impact? > > Not an ideal solution but it’s something. > > In my previous life we just deleted the indexes and rebuilt them on > upgrade but that was specific to our application. > > -Jake > > > > On Nov 6, 2019, at 11:18 AM, Jason Huynh <jhu...@pivotal.io> wrote: > > > > Hi Mario, > > > > I think there are a few ways to accomplish what Dan was suggesting...Dan > or > > other's, please chime in with more options/solutions. > > > > 1.) We add some product code/lucene listener to detect whether we have > old > > versions of geode and if so, do not write to lucene on the newly updated > > node until all versions are up to date. > > > > 2.) We document it and provide instructions (and a way) to pause lucene > > indexing before someone attempts to do a rolling upgrade. > > > > I'd prefer option 1 or some other robust solution, because I think > option 2 > > has many possible issues. > > > > > > -Jason > > > > > >> On Wed, Nov 6, 2019 at 1:03 AM Mario Kevo <mario.k...@est.tech> wrote: > >> > >> Hi Dan, > >> > >> thanks for suggestions. > >> I didn't found a way to write lucene in older format. They only support > >> reading old format indexes with newer version by using lucene-backward- > >> codec. > >> > >> Regarding to freeze writes to the lucene index, that means that we need > >> to start locators and servers, create lucene index on the server, roll > >> it to current and then do puts. In this case tests passed. Is it ok? > >> > >> > >> BR, > >> Mario > >> > >> > >>> On Mon, 2019-11-04 at 17:07 -0800, Dan Smith wrote: > >>> I think the issue probably has to do with doing a rolling upgrade > >>> from an > >>> old version of geode (with an old version of lucene) to the new > >>> version of > >>> geode. > >>> > >>> Geode's lucene integration works by writing the lucene index to a > >>> colocated > >>> region. So lucene index data that was generated on one server can be > >>> replicated or rebalanced to other servers. > >>> > >>> I think what may be happening is that data written by a geode member > >>> with a > >>> newer version is being read by a geode member with an old version. > >>> Because > >>> this is a rolling upgrade test, members with multiple versions will > >>> be > >>> running as part of the same cluster. > >>> > >>> I think to really fix this rolling upgrade issue we would need to > >>> somehow > >>> configure the new version of lucene to write data in the old format, > >>> at > >>> least until the rolling upgrade is complete. I'm not sure if that is > >>> possible with lucene or not - but perhaps? Another option might be to > >>> freeze writes to the lucene index during the rolling upgrade process. > >>> Lucene indexes are asynchronous, so this wouldn't necessarily require > >>> blocking all puts. But it would require queueing up a lot of updates. > >>> > >>> -Dan > >>> > >>> On Mon, Nov 4, 2019 at 12:05 AM Mario Kevo <mario.k...@est.tech> > >>> wrote: > >>> > >>>> Hi geode dev, > >>>> > >>>> I'm working on upgrade lucene to a newer version. ( > >>>> https://issues.apache.org/jira/browse/GEODE-7309) > >>>> > >>>> I followed instruction from > >>>> > >> > https://cwiki.apache.org/confluence/display/GEODE/Upgrading+to+Lucene+7.1.0 > >>>> Also add some other changes that is needed for lucene 8.2.0. > >>>> > >>>> I found some problems with tests: > >>>> * geode- > >>>> lucene/src/test/java/org/apache/geode/cache/lucene/internal/dist > >>>> ribu > >>>> ted/DistributedScoringJUnitTest.java: > >>>> > >>>> > >>>> * > >>>> geode- > >>>> lucene/src/upgradeTest/java/org/apache/geode/cache/lucene/RollingUp > >>>> gradeQueryReturnsCorrectResultsAfterClientAndServersAreRolledOver.j > >>>> ava: > >>>> * > >>>> geode- > >>>> lucene/src/upgradeTest/java/org/apache/geode/cache/lucene/RollingUp > >>>> gradeQueryReturnsCorrectResultAfterTwoLocatorsWithTwoServersAreRoll > >>>> ed.java: > >>>> * > >>>> ./geode- > >>>> lucene/src/upgradeTest/java/org/apache/geode/cache/lucene/RollingUp > >>>> gradeQueryReturnsCorrectResultsAfterServersRollOverOnPartitionRegio > >>>> n.java: > >>>> * > >>>> ./geode- > >>>> lucene/src/upgradeTest/java/org/apache/geode/cache/lucene/RollingUp > >>>> gradeQueryReturnsCorrectResultsAfterServersRollOverOnPersistentPart > >>>> itionRegion.java: > >>>> > >>>> -> failed due to > >>>> Caused by: org.apache.lucene.index.IndexFormatTooOldException: > >>>> Format > >>>> version is not supported (resource > >>>> BufferedChecksumIndexInput(segments_1)): 6 (needs to be between 7 > >>>> and > >>>> 9). This version of Lucene only supports indexes created with > >>>> release > >>>> 6.0 and later. > >>>> at > >>>> org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.jav > >>>> a:21 > >>>> 3) > >>>> at > >>>> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:3 > >>>> 05) > >>>> at > >>>> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:2 > >>>> 89) > >>>> at > >>>> org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:846) > >>>> at > >>>> org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.finis > >>>> hCom > >>>> putingRepository(IndexRepositoryFactory.java:123) > >>>> at > >>>> org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.compu > >>>> teIn > >>>> dexRepository(IndexRepositoryFactory.java:66) > >>>> at > >>>> org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager > >>>> .com > >>>> puteRepository(PartitionedRepositoryManager.java:151) > >>>> at > >>>> org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager > >>>> .lam > >>>> bda$computeRepository$1(PartitionedRepositoryManager.java:170) > >>>> ... 16 more > >>>> > >>>> > >>>> * > >>>> geode- > >>>> lucene/src/upgradeTest/java/org/apache/geode/cache/lucene/RollingUp > >>>> gradeQueryReturnsCorrectResultsAfterClientAndServersAreRolledOverAl > >>>> lBucketsCreated.java: > >>>> > >>>> -> failed with the same exception as previous tests > >>>> > >>>> > >>>> I found this on web > >>>> > >>>> > >> > >> > https://stackoverflow.com/questions/47454434/solr-indexing-issue-after-upgrading-from-4-7-to-7-1 > >>>> , but not have an idea how to proceed with that. > >>>> > >>>> Does anyone has any idea how to fix it? > >>>> > >>>> BR, > >>>> Mario > >>>> > >>>> > >>>> > >>>> > >>>> > >> >
