Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

2019-03-20 Thread Sai Boorlagadda
I would like to resolve the issue around NOTICE and LICENSE files related to new/removed dependencies on develop, which I have a PR[1] open and would need some guidance. There is some feedback provided by Dick earlier this week and I would like to see if I can get some help. [1] https://github.com

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Anthony Baker
An important use case for this is session caching. Obviously it’s pointless to replicate an expired session—the user has already gone away. Copying the bits to the remote cluster is just creating unnecessary work. > On Mar 20, 2019, at 11:22 AM, Bruce Schuchardt wrote: > > I don't know why t

Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

2019-03-20 Thread Dale Emery
> On Mar 20, 2019, at 11:25 AM, Alexander Murmann wrote: > > Dale, is there any downside to making these changes in 1.10 other than > plainly having them later? I don’t think so, but I’d want to hear Dan’s opinion on that, given that his approval of our PR was contingent on our promise to do it

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Bruce Schuchardt
I don't know why the users didn't use conflation but I suspect they're generating events with new keys.  Conflation only applies if you're operating on the same keys all the time.  If you're generating new keys it doesn't help at all. On 3/20/19 11:10 AM, Udo Kohlmeyer wrote: If all that the c

Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

2019-03-20 Thread Alexander Murmann
Dale, is there any downside to making these changes in 1.10 other than plainly having them later? On Wed, Mar 20, 2019 at 11:15 AM Dave Barnes wrote: > geode-native is ready to into the 1.9 release candidate build. > > On Wed, Mar 20, 2019 at 10:42 AM Dave Barnes wrote: > > > The geode-native P

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Dan Smith
> 2) The developer wants to replicate _state_. This means that implicit > state changes (expiration or eviction w/ destroy) could allow us to > optimize the queue size. This is very similar to conflation, just a > different kind of optimization. > > For this second case, does it make sense to all

Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

2019-03-20 Thread Dave Barnes
geode-native is ready to into the 1.9 release candidate build. On Wed, Mar 20, 2019 at 10:42 AM Dave Barnes wrote: > The geode-native PR will be ready to check in momentarily. Just waiting > for Travis to do its diligence. > > On Wed, Mar 20, 2019 at 9:47 AM Alexander Murmann > wrote: > >> Dale

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Udo Kohlmeyer
If all that the customer is concerned about is that the receiving side gets the "latest" data, conflation is definitely the best bet. How do we classify old? The only classification that I have of old (in this context) is that there is a newer version of a data entry. This classification is not

Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

2019-03-20 Thread Dale Emery
Hi Alexander, > On Mar 20, 2019, at 9:47 AM, Alexander Murmann wrote: > > Dale, do I understand correctly that the only concern around the Micrometer > work right now it that it's not useful yet, however it's not harmful either? It’s useful, but somewhat less usable than we want it to be. Curr

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Anthony Baker
I think there are two modes: 1) The developer wants to replicate _events_. This means all changes need to be sent to the remote site regardless of the state in the local cluster. Most likely in order :-) 2) The developer wants to replicate _state_. This means that implicit state changes (ex

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Bruce Schuchardt
Udo, in the cases I've looked at the user is okay with inconsistency because they don't really care about the old data. They're most interested in getting the newest data and keeping the sending site from going down.  I guess the docs for TTL should make it very clear that it will cause inconsi

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Udo Kohlmeyer
-1, I don't believe this is a feature that we should support. IF a client is experiencing slow WAN replication and users only care about the "latest" data, then maybe the user should use "conflation". With a TTL model, we are messing with our consistency tenet. I'm am NOT in support of a setti

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Bruce Schuchardt
Dan, I agree - there shouldn't be a choice of action here.  We can change it to just take an int and specify in the docs that it is in seconds and will cause timed-out events to not be transmitted to the other site. I think we want to guarantee that time-out happens in queue order. On 3/20/19

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Bruce Schuchardt
IDK Anil, we'll figure that out in the implementation.  I was thinking it would be in the dispatch threads, so if distribution is need that will happen as it does now.  I'm hopeful that this won't perturb the code too much. One thing that was brought up to me in person was the Dead Letter Queu

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Dan Smith
Sounds like a good feature. I'm interested to see what ordering guarantees we want to implement - if we can expire things in the order they were added to the queue that seems like a good way to go. As Anil pointed out - do you really want to let the user pass in an ExpirationAttributes object? Tha

Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

2019-03-20 Thread Dave Barnes
The geode-native PR will be ready to check in momentarily. Just waiting for Travis to do its diligence. On Wed, Mar 20, 2019 at 9:47 AM Alexander Murmann wrote: > Dale, do I understand correctly that the only concern around the Micrometer > work right now it that it's not useful yet, however it'

Re: [DISCUSS] TTL setting on WAN

2019-03-20 Thread Anilkumar Gingade
+1. Will the expiration (destroy) be applied on local queues or the expiration will be replicated (for both serial and parallel)? -Anil. On Wed, Mar 20, 2019 at 8:59 AM Bruce Schuchardt wrote: > We've seen situations where the receiving side of a WAN gateway is slow > to accept data or is not

Re: [DISCUSS] Proposal to re-cut Geode 1.9.0 release branch

2019-03-20 Thread Alexander Murmann
Dale, do I understand correctly that the only concern around the Micrometer work right now it that it's not useful yet, however it's not harmful either? Dave, is it correct that if that PR doesn't make it into the newly cut branch, we'd be shipping with a older version of geode-native? What are th

[DISCUSS] TTL setting on WAN

2019-03-20 Thread Bruce Schuchardt
We've seen situations where the receiving side of a WAN gateway is slow to accept data or is not accepting any data.  This can cause queues to fill up on the sending side.  If disk-overflow is being used this can even lead to an outage.  Some users are concerned more with the latest data and do