Re: Question on how Geode handles data on Disk
Hi there Amit, On Sep 7, 2020, 4:25 AM +1000, Amit Pandey , wrote: What I meant here "Also if I request data for ID 1 will it bring it only from disk or " was that if I request a tuple which is not in memory and is in disk, 1) How does Geode know its in DISK 2) Does it bring only that tuple (value) to Memory or brings the whole page Regards On Sun, Sep 6, 2020 at 11:36 PM Amit Pandey wrote: Hi Geode Devs, How does Geode handle data on disk ? How does Geode mark that some data of a region is in disk , is it basically what we call as the Tombstone approach ? Also if I request data for ID 1 will it bring it only from disk or Secondly, how does Geode handle updates of data on disk ? SO does it discard the data on disk when it brings a tuple in memory and writes a new file to disk ? Regards
Re: Question on how Geode handles data on Disk
Hi there Amit, (Attempt 2), So, create a Region, you configure if the Region is to persist the data to disk or not. There is no selective algorithm that would select which data is written to disk and which is not. If it is configured to be persistent or eviction overflowed, it will be written to disk. Persistence allows you to shutdown your server and restart it with the saved data, and overflowed will be that data is written to disk on LRU eviction when memory pressures are experienced. Overflowed will only allow for the reading of the value from disk whilst the server is running. If you shutdown the server, any overflowed regions (with non-persistence) will loose all region data for that server. You can configure persistence and overflow on the same region. When Geode starts up, it reads all keys from disk into memory. At that time, no values are read into memory. There is a special object that we use internally to describe that value, in order for Geode to know where to load that value from disk when required. When you do a ‘get’ operation, and the data is still on disk, Geode will know how (and where) to load the value (single) from disk, into memory. Geode does not disregard the data on disk, just because it is in memory, but it will interact with the object in memory and not from disk. If you make any CUD operations on the object, it will automatically write the data back down to disk. This is ONLY true for regions configured with Persistence. It is important to note, that any updates to the Object need to be written back to the region using the ‘put’ operation. Any changes made to the object without being written back to the region will lead to data being lost. Note: It is possible to make in-memory changes on an object, when you use the `copy-on-read=false` setting. In this case, the region could return you the direct memory referenced object, and making changes to the object without committing it back to the region using the ‘put’ method will and can result in data being lost. As the whole replication and persistence cycle is activated on the ‘put’ operation. Can write a for much longer here.. But let me know if this does not explain the behavior well enough. —Udo On Sep 7, 2020, 4:25 AM +1000, Amit Pandey , wrote: What I meant here "Also if I request data for ID 1 will it bring it only from disk or " was that if I request a tuple which is not in memory and is in disk, 1) How does Geode know its in DISK 2) Does it bring only that tuple (value) to Memory or brings the whole page Regards On Sun, Sep 6, 2020 at 11:36 PM Amit Pandey wrote: Hi Geode Devs, How does Geode handle data on disk ? How does Geode mark that some data of a region is in disk , is it basically what we call as the Tombstone approach ? Also if I request data for ID 1 will it bring it only from disk or Secondly, how does Geode handle updates of data on disk ? SO does it discard the data on disk when it brings a tuple in memory and writes a new file to disk ? Regards
Errored: apache/geode-examples#455 (rel/v1.13.0.RC1 - 9779f8f)
Build Update for apache/geode-examples - Build: #455 Status: Errored Duration: 1 min and 6 secs Commit: 9779f8f (rel/v1.13.0.RC1) Author: Dave Barnes Message: Bumping version to 1.13.0 View the changeset: https://github.com/apache/geode-examples/compare/rel/v1.13.0.RC1 View the full build log and details: https://travis-ci.org/github/apache/geode-examples/builds/725060336?utm_medium=notification&utm_source=email -- You can unsubscribe from build emails from the apache/geode-examples repository going to https://travis-ci.org/account/preferences/unsubscribe?repository=11483039&utm_medium=notification&utm_source=email. Or unsubscribe from *all* email updating your settings at https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notification&utm_source=email. Or configure specific recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications.
Passed: apache/geode-native#2612 (rel/v1.13.0.RC1 - 5c5b320)
Build Update for apache/geode-native - Build: #2612 Status: Passed Duration: 1 hr, 25 mins, and 24 secs Commit: 5c5b320 (rel/v1.13.0.RC1) Author: Dave Barnes Message: GEODE-8388: Geode Native Client User Guide - delete unused source files (#631) View the changeset: https://github.com/apache/geode-native/compare/rel/v1.13.0.RC1 View the full build log and details: https://travis-ci.org/github/apache/geode-native/builds/725060430?utm_medium=notification&utm_source=email -- You can unsubscribe from build emails from the apache/geode-native repository going to https://travis-ci.org/account/preferences/unsubscribe?repository=11948127&utm_medium=notification&utm_source=email. Or unsubscribe from *all* email updating your settings at https://travis-ci.org/account/preferences/unsubscribe?utm_medium=notification&utm_source=email. Or configure specific recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications.
[VOTE] Apache Geode 1.13.0.RC1
Hello Geode Dev Community, This is a release candidate for Apache Geode version 1.13.0.RC1. Thanks to all the community members for their contributions to this release! Please do a review and give your feedback, including the checks you performed. Voting deadline: 3PM PDT Wed, September 9, 2020. Please note that we are voting upon the source tag: rel/v1.13.0.RC1 Release notes: https://cwiki.apache.org/confluence/display/GEODE/Release+Notes#ReleaseNotes-1.13.0 Source and binary distributions: https://dist.apache.org/repos/dist/dev/geode/1.13.0.RC1/ Maven staging repo: https://repository.apache.org/content/repositories/orgapachegeode-1069 GitHub: https://github.com/apache/geode/tree/rel/v1.13.0.RC1 https://github.com/apache/geode-examples/tree/rel/v1.13.0.RC1 https://github.com/apache/geode-native/tree/rel/v1.13.0.RC1 https://github.com/apache/geode-benchmarks/tree/rel/v1.13.0.RC1 Pipelines: https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-13-main https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-support-1-13-rc Geode's KEYS file containing PGP keys we use to sign the release: https://github.com/apache/geode/blob/develop/KEYS Command to run geode-examples: ./gradlew -PgeodeReleaseUrl= https://dist.apache.org/repos/dist/dev/geode/1.13.0.RC1 -PgeodeRepositoryUrl= https://repository.apache.org/content/repositories/orgapachegeode-1069 build runAll Regards Dave Barnes
Re: Question on how Geode handles data on Disk
I agree its kind of academic but any answers will be appreciated, I just wanted to understand the performance characteristics when using disk persistence On Sun, Sep 6, 2020 at 11:54 PM Amit Pandey wrote: > What I meant here "Also if I request data for ID 1 will it bring it only > from disk or " was that if I request a tuple which is not in memory and is > in disk, > 1) How does Geode know its in DISK > 2) Does it bring only that tuple (value) to Memory or brings the whole > page > > Regards > > > On Sun, Sep 6, 2020 at 11:36 PM Amit Pandey > wrote: > >> Hi Geode Devs, >> >> How does Geode handle data on disk ? How does Geode mark that some data >> of a region is in disk , is it basically what we call as the Tombstone >> approach ? Also if I request data for ID 1 will it bring it only from disk >> or >> >> Secondly, how does Geode handle updates of data on disk ? SO does it >> discard the data on disk when it brings a tuple in memory and writes a new >> file to disk ? >> >> Regards >> >>
