In simple words, if Lucene indexes were created by a new version (7.1.0), then
replicated to others that are still in the older version, they won't understand
the index, and the event processors start throwing exceptions.
This can be simply seen by just re-enabling the query execution in the DUnit
tests and commenting out the check blocks: [develop SHA:
68629356f561a932f5dfbace70b01d9971a42473]
In LuceneEventListener
if (cache.hasMemberOlderThan(KnownVersion.GEODE_1_15_0)) {
logger.info("Some members are older than " +
KnownVersion.GEODE_1_15_0.getName());
return false;
}
In IndexRepositoryFactory:
if (userRegion.getCache() != null
&& userRegion.getCache().hasMemberOlderThan(KnownVersion.GEODE_1_15_0))
{
logger.info("Some members are older than " +
KnownVersion.GEODE_1_15_0.getName());
return null;
}
This is the exception that will be encountered:
[Exception]
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.251 PDT <Event Processor for
GatewaySender_AsyncEventQueue_index#_aRegion_5> tid=102] An Exception occurred.
The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index
repository
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at
java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at
org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at
org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at
org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at
org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException:
Format version is not supported (resource
BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at
org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at
org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at
org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
Also:
[vm2_v1.2.0] [warn 2021/09/27 14:24:42.134 PDT <Event Processor for
GatewaySender_AsyncEventQueue_index#_aRegion_7> tid=106] An Exception occurred.
The dispatcher will continue.
[vm2_v1.2.0] org.apache.geode.InternalGemFireError: Unable to create index
repository
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:118)
[vm2_v1.2.0] at
java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1853)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.computeRepository(AbstractPartitionedRepositoryManager.java:108)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:137)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.getRepository(AbstractPartitionedRepositoryManager.java:76)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.LuceneEventListener.process(LuceneEventListener.java:87)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.LuceneEventListener.processEvents(LuceneEventListener.java:64)
[vm2_v1.2.0] at
org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:154)
[vm2_v1.2.0] at
org.apache.geode.internal.cache.wan.GatewaySenderEventCallbackDispatcher.dispatchBatch(GatewaySenderEventCallbackDispatcher.java:80)
[vm2_v1.2.0] at
org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.processQueue(AbstractGatewaySenderEventProcessor.java:609)
[vm2_v1.2.0] at
org.apache.geode.internal.cache.wan.AbstractGatewaySenderEventProcessor.run(AbstractGatewaySenderEventProcessor.java:1051)
[vm2_v1.2.0] Caused by: org.apache.lucene.index.IndexFormatTooNewException:
Format version is not supported (resource
BufferedChecksumIndexInput(segments_2)): 7 (needs to be between 4 and 6)
[vm2_v1.2.0] at
org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:216)
[vm2_v1.2.0] at
org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:302)
[vm2_v1.2.0] at
org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:286)
[vm2_v1.2.0] at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.IndexRepositoryFactory.computeIndexRepository(IndexRepositoryFactory.java:84)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.PartitionedRepositoryManager.computeRepository(PartitionedRepositoryManager.java:42)
[vm2_v1.2.0] at
org.apache.geode.cache.lucene.internal.AbstractPartitionedRepositoryManager.lambda$computeRepository$0(AbstractPartitionedRepositoryManager.java:116)
[vm2_v1.2.0] ... 10 more
________________________________
From: Jacob Barrett <[email protected]>
Sent: Monday, September 27, 2021 2:08 PM
To: [email protected] <[email protected]>
Subject: Re: [DISCUSS] Upgrading to Lucene 7.1.0
> On Sep 27, 2021, at 11:48 AM, nabarun nag <[email protected]> wrote:
>
> Recently, a commit was pushed to develop which upgraded the Lucene
> version used in Apache Geode to 7.1.0. These new Lucene indexes are
> not compatible with the previous versions hence it breaks the rolling
> upgrade contract. We are no longer able to execute Lucene queries when
> there are severs of mixed versions in the cluster.
Can you describe the problem with a little more detail? Does this mean that
while there is a mix the execution throws an exception on all servers or is
there a subset for which it works? If there is a subset for which it works, are
those instances sufficient to provide accurate results if the instances that
fail are ignored?
-Jake
