Hi, Alberto:
I was the original author who introduced the tmpDroppedEvents. Due to other
work, I only got chance to read the issue on Thursday, which is your deadline.
Can you hold on a little bit longer to Monday?
I have been thinking of history of the code changes and issues you encountered.
I will try to find a light-weight solution with minimum impact to current code.
Regards
Xiaojian Zhou
On 7/8/20, 1:05 PM, "Eric Shu" <e...@vmware.com> wrote:
I think the only case the memory issue occurred is when all gateway senders
are stopped in the wan-site. Otherwise another member would assume to be the
primary queue. No more events will be enqueued in tmpDroppedEvents on the
member with original primary queue. (For parallel wan queue, I do not think
stop one gateway queue is a valid case to support.)
For all gateway senders are stopped case, no need to notify any other
members in the wan site if the limit is reached. The tmpDroppedEvents is only
used for remove events on the secondary queue. If no events are enqueued in the
secondary queue, there is no need to add into tmpDroppedEvents at all. To me,
it should be only used for limited events to be queued.
Regards,
Eric
________________________________
From: Alberto Gomez <alberto.go...@est.tech>
Sent: Wednesday, July 8, 2020 12:02 PM
To: dev@geode.apache.org <dev@geode.apache.org>
Subject: Re: [DISCUSS] RFC - Avoid the queueing of dropped events by the
primary gateway sender when the gateway sender is stopped
Thanks for your comments, Eric.
Limiting the size of the queue would be a simple solution but I think it
would pose several problems on the the one configuring and operating Geode:
* How big should the queue be? Probably not easy to dimension. Should
the limit by on the memory occupied by the elements or on the number of
elements in the queue (in which case, depending on the size of the elements,
the memory used could vary a lot)?
* What to do when the limit has been reached? how do we notify that it
was reached, what to do afterwards, how would we know what dropped events did
not make it to the queue but should have been removed from the secondary's
queue...
I think the solution proposed in the RFC is simple enough and also
addresses a possible confusion with the semantics of the gateway sender stop
command.
Stopping a gateway sender currently makes that all events received while
the sender is stopped are dropped; but at the same time, unlimited memory may
be consumed by the dropped events. We could put a limit on the amount of memory
used by the queued dropped events but what would be the point in the first
place to store them if those events will not be sent to the remote site anyway?
I would expect that after stopping a gateway sender no resources (or at
least a minimal part) would be consumed by it. Otherwise we may as well not
stop it or use the pause command depending on what we want to achieve.
From what I have seen, queuing dropped events has its place while the
gateway sender is starting and while it is stopping but if it is done in a
sender to be started manually or in a manually stopped server it could provoke
an unexpected memory exhaustion.
I really think the solution proposed makes the behavior of the gateway
sender command more logical.
Best regards,
Alberto
________________________________
From: Eric Shu <e...@vmware.com>
Sent: Wednesday, July 8, 2020 7:32 PM
To: dev@geode.apache.org <dev@geode.apache.org>
Subject: Re: [DISCUSS] RFC - Avoid the queueing of dropped events by the
primary gateway sender when the gateway sender is stopped
It seems that I was not able to comment on the RFC in the wiki yet.
Just try to find out if we have a simple solution for the issue you raised
-- can we have a up-limit for the tmpDroppedEvents queue in question?
Always check the limit before adding to the queue -- so that the tmp queue
is not unbound?
Regards,
Eric
________________________________
From: Alberto Gomez <alberto.go...@est.tech>
Sent: Monday, July 6, 2020 8:24 AM
To: geode <dev@geode.apache.org>
Subject: [DISCUSS] RFC - Avoid the queueing of dropped events by the
primary gateway sender when the gateway sender is stopped
Hi,
I have published a new RFC in the Apache Geode wiki with the following
title: "Avoid the queueing of dropped events by the primary gateway sender when
the gateway sender is stopped".
https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FGEODE%2FAvoid%2Bthe%2Bqueuing%2Bof%2Bdropped%2Bevents%2Bby%2Bthe%2Bprimary%2Bgateway%2Bsender%2Bwhen%2Bthe%2Bgateway%2Bsender%2Bis%2Bstopped&data=02%7C01%7Czhouxh%40vmware.com%7C368e4337126e4e4ca05508d8237a31c6%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637298355008581269&sdata=8R2rlTCFR1sVOk77ZDwjG5IVAnCBnuXHWTc2lluB2do%3D&reserved=0
Could you please give comments by Thursday, July 9th, 2020?
Thanks in advance,
Alberto G.
