my-ship-it opened a new pull request, #1971:
URL: https://github.com/apache/cloudberry/pull/1971
### Summary
Adds a developer-oriented handbook for the Cloudberry kernel under
`docs/core-internals/` — 21 chapters, 104 sections, organised into five
parts after the structure of Rogov's *PostgreSQL 14 Internals*.
Coverage: storage and data layout (heap, AO/AOCS, PAX, indexes, the
catalog), transactions and MVCC (isolation, snapshots, vacuum, locks and
the global deadlock detector, distributed transactions and 2PC), query
processing (stages, statistics, the PostgreSQL planner, distributed
planning, ORCA, the executor, Motion and the interconnect), high
availability and recovery, and resource management.
**A PDF of the whole handbook is attached below** so reviewers can read it
without building the site — 652 pages, all diagrams rendered.
### What the chapters contain
| | |
|---|---|
| Code excerpts | 372, each fenced with `title="file:line"` pointing at the
source it was read from |
| Captured `psql` sessions | 487, split into an input block and an `output`
block — transcripts from a running cluster, not illustrations |
| Diagrams | 113 mermaid + 64 SVG figures |
| Cross-references | internal `§N.M` links that resolve to section anchors
and are checked at build time |
### Why here, and how the site uses it
The directory is written for Docusaurus so that `apache/cloudberry-site`
can pull it in as a git submodule with a sparse checkout limited to
`docs/core-internals`, served by its own docs plugin instance with an
autogenerated sidebar — the same arrangement already used for the PXF
documentation (apache/cloudberry-site#372). Keeping the source next to the
code it describes means a change to the engine and the change to its
description can travel in one commit.
A follow-up PR against `cloudberry-site` will add the submodule and the
three config touch points. It has to come second, since the submodule
needs a commit on this side to point at.
Nothing here depends on site-level configuration except one small MDX
component: 37 figures are CSS-laid-out HTML rather than SVG, and are handed
to that component as a string so MDX does not try to parse them as JSX. The
component and its CSS are a few lines and are described in the README.
### Verification
- **Docusaurus 3 / MDX v3, built locally**: no MDX compilation errors, no
broken links, no broken anchors, 23 pages, all 64 figures resolved.
- **Apache RAT**: every one of the 92 files carries the ASF license header,
so no `pom.xml` exclusions are needed.
- **No environment-specific detail**: host names, addresses and install
paths from the machine the sessions were captured on are replaced with
documentation-safe equivalents *of identical width*, so the column
alignment of captured output is preserved. Verified across the markdown,
the HTML and the PDF.
### Testing
<!-- Documentation-only change; no code paths are touched. -->
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] Passed `make installcheck`
- [ ] Passed `make -C src/test installcheck-cbdb-parallel`
### Impact
**Performance:** None. No code is touched.
**User-facing changes:** New documentation. Nothing changes for existing
users until `cloudberry-site` links it in.
**Dependencies:** None added to this repository. A site that renders the
diagrams needs `@docusaurus/theme-mermaid`; without it the mermaid blocks
degrade to code blocks and everything else renders unchanged.
### Checklist
- [x] Followed [contribution
guide](https://cloudberry.apache.org/contribute/code)
- [x] Added/updated documentation
- [x] Reviewed code for security implications
- [x] This PR contains AI-assisted code generation
- [ ] Requested review from [cloudberry
committers](https://github.com/orgs/apache/teams/cloudberry-committers)
### Additional Context
**On maintenance.** The chapters describe a specific point in the tree's
history. Prose is the durable part; line numbers are the perishable part.
The `title="file:line"` convention is machine-readable on purpose — a CI
job can verify that every cited path still exists, which catches the worst
class of drift (a file that moved or was deleted) cheaply. I would rather
agree on that check in review than add it unilaterally here.
**On the size of the review.** This is a large diff and I do not expect
anyone to read 29k lines in a PR. Reading order that costs the least: the
attached PDF for the content, `docs/core-internals/README.md` for the
layout and conventions, and one chapter of your choice for the texture —
ch02 is the shortest, ch18 the most demanding. Chapter-level corrections
are welcome as ordinary follow-up PRs.
**On the branch base.** This branch is cut from an April snapshot of `main`
rather than today's tip, so GitHub will report it as behind. That is an
artifact of how it was pushed, not of the content: all 93 files are new
paths under `docs/core-internals/`, nothing else is touched, and the diff
GitHub computes is exactly those 93 additions. Happy to rebase onto current
`main` if you would rather review it that way.
**On AI assistance.** Disclosed above per `AI_GUIDELINE.md`. The generation
was AI-assisted; the technical claims, the code citations and the captured
sessions are mine to answer for, and I will answer review questions
directly rather than through a tool.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
