GitHub user my-ship-it added a comment to the discussion: [Ideas] Build some like greenplum_fdw for GPDB or ADB-to-ADB connector for Arenadata, for Cloudberry to run queries between one and more CBDB clusters
Thanks for the discussion. This feature consists of two parts. - The first part is to implement parallel cursor / retriever on the database kernel side, which is currently supported. - The second part is, transform postgres_fdw into cloudberry_fdw (or greenplum_fdw), allowing for direct parallel tuple fetching between segments. This will bring some benefits, including: - Data does not need to be copied multiple times, and one piece of data can be queried between different clusters through cloudberry_fdw - Data is read from segment to segment instead of through the master, and query efficiency can be improved by utilizing postgres_fdw aggregation push-down and join push-down capabilities - Clusters use parallel retrivers as interfaces to avoid binary incompatibility issues to some extent Possible limitations: - Required to configure pg_hba conf to enable direct connection between segments - The first step is to support queries, and writing requires support for the Two-Phase Commit Protocol, which will be more complex and can be implemented in subsequent stages GitHub link: https://github.com/apache/cloudberry/discussions/1069#discussioncomment-12964242 ---- This is an automatically sent email for dev@cloudberry.apache.org. To unsubscribe, please send an email to: dev-unsubscr...@cloudberry.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
