[ https://issues.apache.org/jira/browse/GEODE-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15767845#comment-15767845 ]
Darrel Schneider commented on GEODE-2240: ----------------------------------------- I think this bug might be caused by the code that adds to "expiredTombstones", TombstoneService.ReplicateTombstoneSweeper.expireTombstone, not syncing on getBlockGCLock() as the code that removes from "expiredTomstones", TombstoneService.ReplicateTombstoneSweeper.removeExpiredIf, does. Since these are the only two methods that modify the "expiredTombstones" collection they need a common sync to prevent unsafe concurrent modification of the ArrayList. I think the only time multiple threads would do this concurrently is when a region is being destroyed. > unexpected NullPointerException from Tombstone service > ------------------------------------------------------ > > Key: GEODE-2240 > URL: https://issues.apache.org/jira/browse/GEODE-2240 > Project: Geode > Issue Type: Bug > Components: regions > Reporter: Darrel Schneider > > A test failed and the logs were found to be full of NPEs from the tombstone > service:[severe 2016/12/20 02:04:35.605 UTC > dataStoregemfire7_rs-StorageBTTest-2016-12-19-23-35-42-client-14_19508 > <Replicate/Partition Region Garbage Collector> tid=0x44] GemFire garbage > collection service encountered an unexpected exception > java.lang.NullPointerException > at > org.apache.geode.internal.cache.TombstoneService$TombstoneSweeper.lambda$purgeObsoleteTombstones$1(TombstoneService.java:938) > at > org.apache.geode.internal.cache.TombstoneService$ReplicateTombstoneSweeper.removeExpiredIf(TombstoneService.java:479) > at > org.apache.geode.internal.cache.TombstoneService$TombstoneSweeper.removeIf(TombstoneService.java:823) > at > org.apache.geode.internal.cache.TombstoneService$TombstoneSweeper.purgeObsoleteTombstones(TombstoneService.java:937) > at > org.apache.geode.internal.cache.TombstoneService$TombstoneSweeper.run(TombstoneService.java:880) > at java.lang.Thread.run(Thread.java:745) > [severe 2016/12/20 02:05:45.987 UTC > dataStoregemfire7_rs-StorageBTTest-2016-12-19-23-35-42-client-14_19508 > <Replicate/Partition Region Garbage Collector> tid=0x44] GemFire garbage > collection service encountered an unexpected exception > java.lang.NullPointerException > at > org.apache.geode.internal.cache.TombstoneService$ReplicateTombstoneSweeper.expireBatch(TombstoneService.java:524) > at > org.apache.geode.internal.cache.TombstoneService$ReplicateTombstoneSweeper.checkExpiredTombstoneGC(TombstoneService.java:594) > at > org.apache.geode.internal.cache.TombstoneService$TombstoneSweeper.run(TombstoneService.java:878) > at java.lang.Thread.run(Thread.java:745) > Both of these stacks indicate that the "expiredTombstones" ArrayList somehow > has nulls in it. It is an ArrayList of Tombstone instances and the only code > that adds to it first tests that the item it is adding is not null. The only > other modify operation done on it is to remove an item. > Perhaps unsafe concurrent access is happening causing this code to see nulls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)