Hi, On Wed, Mar 18, 2026 at 8:08 AM Michael Paquier <[email protected]> wrote: > > On Tue, Mar 17, 2026 at 10:55:51AM +0530, Ashutosh Bapat wrote: > > On Tue, Mar 17, 2026 at 10:32 AM Amit Kapila <[email protected]> > > wrote: > >> Yesterday, I raised a point related to this which is whether we want > >> to count messages like keep_alive, copy_done, or > >> PqReplMsg_PrimaryStatusUpdate? These don't contain the user decoded > >> data from WAL but some additional information between > >> publisher-subscriber required to ensure data is being applied. > > > > I think sent_bytes should count the logically decoded data - the data > > that the output plugin produces in logical form. This is the amount of > > data that the downstream has to process for data replication. So, the > > protocol related data shouldn't be counted here, IMO. Compare this > > with the total_bytes which does not account for the memory used by the > > reorder buffer metadata itself - it just accounts for the size of the > > changes. If we describe the sent_bytes as below, we are explicitly > > mentioning what sent_bytes is. So anything which is not data in the > > output format is excluded including the protocol data. If you think we > > need to clarify that we are excluding protocol data, we can add a > > line. But that may confuse readers who are unaware of the protocol > > data. > > As we aim to count how many bytes are sent over the wire, including > the amount of bytes related to the messages sent as part of the > protocol exchanged, sounds like an important property that we should > have in the implementation because these bytes are required for the > exchanges to work. If an output plugin decodes few changes and has a > lot of messages (unlikely in most cases, still imagine it's the case), > sent_bytes could be misleading if we only include the transactional > changes, couldn't it?
Here is my understanding and perspective: 1) For transactional changes, logical decoding accumulates the changes and then sends them downstream either at commit time or while streaming an in-progress transaction. The sent bytes are accounted for when the formatted output is actually written by the decoding output write path, and the slot stats are then reported through the normal decoding stats update path. 2) For non-transactional logical messages, the behavior is different in that they are not queued as reorder buffer changes. They are processed immediately when read from WAL. However, they still go through logical decoding and can still produce output that is written downstream, so their sent bytes should be accounted for as well. For non-transactional messages, we are currently (with the patch) updating the sent bytes but not reporting it. 3) Messages such as keepalives are sent on the replication connection, but they are not logical decoding output. Since no logical decoding payload is being produced there, they should not be included in sent_bytes. So the right boundary for sent_bytes is according to me would be: a) include logical decoding payload sent downstream (or via SQL decoding path), whether it came from transactional changes or non-transactional logical messages b) exclude auxiliary replication protocol traffic such as keepalives If this makes sense then the revised documentation shared by Ashutosh Bapat in his last email looks good to me, which is: Amount of transaction changes and non-transactional messages sent downstream in the output plugin format for this slot. The output plugin may filter the changes it receives. Hence the amount of data that it converts to the output plugin format is less than the <structfield>total_bytes</structfield>. But the format of data before and after the conversion is different. Hence the value of <structfield>sent_bytes</structfield> is not directly related to the value of <structfield>total_bytes</structfield>. If you all agree with this, I will first prepare a patch to fix the missing part in #2 which is about reporting stats for non-transactional messages that goes via logical decoding process and then on top of that share the ongoing 0001 patch with above doc change. Let me know your thoughts. thanks -- With Regards, Ashutosh Sharma.
