Hi Mario, Thanks for discovering this difference in behavior between the geode-native clients (C++ and .NET) and geode Java client regarding the sync'ing of local caches after a restore operation.
It looks like geode-native has no tests for the exception type you're seeing: GF_CACHE_CONCURRENT_MODIFICATION_EXCEPTION We definitely should write a test around this functionality using our new testing framework and then fix the new failing test. If you're not familiar with our new test framework, I'd be happy to walk you through it. It's much nicer! Thanks, Mike. ________________________________ From: Mario Salazar de Torres <mario.salazar.de.tor...@est.tech> Sent: Monday, March 1, 2021 6:38 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: Cached regions are not synchronized after restore Hi everyone, After replicating the test with both clients being the Java implementation, I could verify that what @Dan Smith<mailto:dasm...@vmware.com> pointed out about the documentation is happening. Every time subscription redundancy is lost, cached entries are erased. This is clearly not happening in geode-native. So, I will further investigate to bring this functionality into the native client. Thanks for the help 🙂 BR/ Mario ________________________________ From: Dan Smith <dasm...@vmware.com> Sent: Monday, March 1, 2021 5:27 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: Cached regions are not synchronized after restore The java client at least should automatically drop its cache when it loses and restores connectivity to all the servers. See https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgeode.apache.org%2Fdocs%2Fguide%2F12%2Fdeveloping%2Fevents%2Fhow_client_server_distribution_works.html%23how_client_server_distribution_works__section_928BB60066414BEB9FAA7FB3120334A3&data=04%7C01%7Cmartellm%40vmware.com%7Cf889a418de304df8b09708d8dd245484%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C1%7C0%7C637502495409048002%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GIbtpxg%2F4iaUC0dqu9JwUtUu9TaDK%2B4YjRAKiSgekGI%3D&reserved=0 -Dan ________________________________ From: Jacob Barrett <jabarr...@vmware.com> Sent: Friday, February 26, 2021 9:29 AM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: Cached regions are not synchronized after restore For clarification, does the a Java client show the same behavior in missing the events after restore or is this something you are saying is unique to the native clients? > On Feb 26, 2021, at 4:48 AM, Mario Salazar de Torres > <mario.salazar.de.tor...@est.tech> wrote: > > Hi, > These months I have been tackling a series of issues having to do with > several inconsistencies in the geode-native client after a cluster restore. > The later one has to do with subscription notification and cached regions. > The scenario is as follows: > > 1. Start the cluster and the clients. For clarification purposes let's say > we have a NC and a Java client. Being the NC the local client, and Java > client the external one. > Also note that NC client has subscription notification enabled for its pool > and the region has caching enabled. > 2. Register interest for all the region entries. > 3. Write some entries into the region from the Java client. Notifications > are received in the NC client and entries cached. > 4. Take a backup of the disk-stores. > 5. Write/modify some entries with the Java client. Notifications are > received in the NC client and entries cached. > 6. Restore the previous backup. > 7. Write/modify some entries with the Java client. Some of the > notifications are discarded and some others not. > Note that all the entries which notifications were ignored did not exist in > the step 4 of the scenario. > > The reason why notifications mentioned in step 7 are ignored is due to the > following log: > "Region::localUpdate: updateNoThrow<Region::put> for key [<redacted>] failed > because the cache already contains an entry with higher version. The cache > listener will not be invoked" > > So, first of, I wanted to ask: > > * Any of you have encountered this issue before? How did you tackle it? > * Is there any mechanism in the Java client to avoid this kind of issues > with caching de-sync? Note that I did not found any > * Maybe we should add an option to clear local cached regions after > connection is lost towards the cluster in the same way is done with > PdxTypeRegistry? > * Maybe any other solution having to do with cluster versioning? > > BR, > Mario.
