Hi all, I'd like to share a handbook on how Cloudberry works internally. It's a piece of community work: it was built with a lot of support from the community, and it draws on what many contributors have shared over time in discussions, code reviews, design notes and answers on the mailing list. It's now available here:
- Read online: https://cloudberry.apache.org/core-internals/ - Source: https://github.com/apache/cloudberry/tree/main/doc/core-internals What's inside It has 21 chapters on how Cloudberry builds on PostgreSQL and where an MPP database has to work differently: - Storage: heap, AO/AOCO, PAX, and data distribution - Transactions: distributed snapshots, two-phase commit, and deadlock detection - Query processing: the planner, ORCA, the executor, and Motion - High availability and resource management It explains things with diagrams and points to the relevant functions in the code. I hope it's useful to new contributors, and to anyone who wants some background before reading a part of the code they don't usually work in. This is a starting point, not a finished book. I'm sure it still has mistakes, places that are oversimplified, and topics that are missing, and many of you know these subsystems far better than I do. If you notice anything wrong or unclear, please open an issue or send a PR against doc/core-internals, or reply on this thread. If there's a subsystem you know well, you're welcome to add or expand a chapter on it. This is the community's handbook, and it reflects many people's contributions. Special thanks to Dianjin and Leonard for taking the time to review it and keep the quality up. Dianjin also did all the website work, building the pages and getting the handbook published on cloudberry.apache.org. Thanks as well to everyone who looked at the early drafts and shared suggestions. Best regards, Max Yang
