yjhjstz commented on PR #1942:
URL: https://github.com/apache/cloudberry/pull/1942#issuecomment-5543272654
Alternative: reuse the dispatch connection instead of a backward libpq
connection
The QD↔QE dispatch connection is already bidirectional mid-execution.
nextval() on a QE does exactly this: it sends a NOTIFY to the QD over its own
frontend connection and waits for the reply on the same socket
(src/backend/commands/sequence.c:2117-2176). The QD handles it inside the
dispatcher's receive loop and writes the answer back with a custom message type
(cdbdisp_async.c:1160-1195, send_sequence_response at :972). Bloom filter
exchange is the same pattern with N producers and M consumers.
Sketch:
1. Producer QE finishes its scan and sends NotifyMyFrontEnd("anser_filter",
payload). Payload has to be a C string, so hex/base64 encode the filter.
2. QD merges per channel in processResults. Producer count is known from the
gang size. When complete, write the merged filter to every consumer connection
via pqPutMsgStart with a custom type. No libpq change is needed for QD→QE since
the QE reads raw with pq_getmessage.
3. Consumer QE waits for that message on first ExecProcNode, using
WaitLatchOrSocket on MyProcPort->sock rather than the busy loop nextval uses.
What this removes: the segment→QD connection, the pg_hba bypass hooks, the
token hash, QD shared memory and background workers, the three catalog
functions and the catversion bump. Cancellation and error propagation come for
free: any QE error makes the QD cancel all gangs, and the consumer's wait loop
exits via CHECK_FOR_INTERRUPTS. That also resolves open question 3 about the
consumer having no timeout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
