I don't know gRPC well enough to really answer your query, but I can
comment on the design... that I would be very cautious and wary about
removing back-pressure
for a system, like you are thinking of doing. Because e.g. without
back-pressure,
I would expect the system to fill up the local disk with cached writes,
when under load.
On Wednesday, March 26, 2025 at 8:43:44 PM UTC Bushnell, Thomas wrote:
I would like a gRPC server, written in Go, to be able to implement
something like the following. My question is whether the “good thing”
happens enough to provide a benefit. It is fine if sometimes the “good
thing” does not happen.
The server is receiving updates from a large number of clients by an RPC,
say “Update”, and then forward each on one to a third agent. The
forwarding-on is normally speedy, but might fail or be slow because it’s
just another RPC to a different system. Each update has an id which
guarantees idempotency, so duplicated updates are not an issue, but lost
ones must be prevented.
I would like to have the return from “Update” imply “I have taken
possession of the update; it is or will be forwarded on”, resulting in a
trivial server stub which does this:
return msg.Forward(ctx)
This is fine and clearly correct; the if the Update RPC is canceled the
error will indicate to the client that it cannot assume the update is or
will be forwarded, and can retry appropriately.
But it would be nice to avoid that retry in the commonest case, which is
simply that forwarding on took too long for some reason, but everything
else is good. In that situation, I would like my server to notice that the
Forward call was canceled, persist the update to disk, and then return *success
*from the Update stub. I want that successful return to be propagated back
to the client. (This is the “good thing.”) This would be:
err := msg.Forward(ctx)
if errors.Is(err, context.Canceled) || errors.Is(err,
context.DeadlineExceeded) {
msg.PersistToDisk()
return true
}
return err
If the success is occasionally not delivered to the client, that’s fine; at
least we tried to avoid the unnecessary retry. But if it’s never delivered
back, then there’s no point in doing the persistence – the client will just
always retry and so it doesn’t matter. And we don’t want to just write
every update to disk, because in the normal case Forward proceeds quickly
and there’s no need to stick another I/O bottleneck in the way on the most
common fast path.
So will the “good thing” happen, at least often? That is: if a Go gRPC
server stub is canceled, and the stub implementation handles the context
cancelation by returning success, will that (at least often) be reflected
back to the RPC client?
[Note that in a non-gRPC context, in which these are just all local
function calls, the answer is defined by the Go Language Specification –
the return value is of course returned no matter what the state of the
context may be.]
Thomas
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/golang-nuts/01ee9a4d-70cb-465c-a21f-0b10c9f4f8a3n%40googlegroups.com.
