tuhaihe commented on code in PR #1971: URL: https://github.com/apache/cloudberry/pull/1971#discussion_r3985260462
########## docs/core-internals/part-1-introduction/ch01.md: ########## @@ -0,0 +1,711 @@ +--- +id: ch01 +title: "1. Introduction: The Big Picture" +sidebar_position: 1 +--- + +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# 1. Introduction: The Big Picture + +```mermaid +flowchart TB + CL["Client"]:::ext + subgraph QD["QD — coordinator · Gp_role = GP_ROLE_DISPATCH"] + direction LR + PL["Parser → Planner / ORCA<br/>cuts the plan into slices at each Motion · §15–§17"]:::q + DP["Dispatcher — CdbDispatchPlan<br/>allocates gangs, ships plan + snapshot, coordinates 2PC · §13, §12"]:::q + PL --> DP + end + IC["INTERCONNECT — UDPIFC · §19<br/>Motion between gangs, and the final Gather to the QD"]:::net + subgraph SEG0["Segment 0 — forks one QE per slice, all sharing one snapshot"] + direction TB + Q0a["QE · Executor · §18"]:::q + Q0b["QE · Executor"]:::q + SS0["SharedLocalSnapshot<br/>one consistent view for the segment's QEs"]:::t + D0[("data shard 0")]:::s + Q0a --- SS0 + Q0b --- SS0 + Q0a -->|"local scan / write"| D0 + Q0b -->|"local"| D0 + end + subgraph SEG1["Segment 1 — its own QEs + shard"] + direction TB + Q1a["QE · Executor"]:::q + Q1b["QE · Executor"]:::q + SS1["SharedLocalSnapshot"]:::t + D1[("data shard 1")]:::s + Q1a --- SS1 + Q1b --- SS1 + Q1a -->|"local"| D1 + Q1b -->|"local"| D1 + end + LK["Locks · §11<br/>each QE locks the objects it touches"]:::t + WALN["WAL · §20<br/>each segment logs its own changes"]:::h + CL ==>|"SQL"| PL + DP ==>|"dispatch plan + snapshot — over libpq"| SEG0 + DP ==>|"libpq"| SEG1 + Q0a -. Motion .- IC + Q1a -. Motion .- IC + IC ==>|"Gather results"| DP + DP ==>|"rows"| CL + LK -.-> SEG0 + LK -.-> SEG1 + SEG0 -.->|"changes logged"| WALN + SEG1 -.->|"changes logged"| WALN + classDef ext fill:#ffffff,stroke:#1f2328,color:#1f2328 + classDef q fill:#ddf4ff,stroke:#0969da,color:#0a3069 + classDef s fill:#dafbe1,stroke:#1a7f37,color:#04260f + classDef net fill:#fff8c5,stroke:#bf8700,stroke-width:2px,color:#3b2300 + classDef t fill:#fff1e5,stroke:#bc4c00,color:#5e2700 + classDef h fill:#ffebe9,stroke:#cf222e,color:#5c0011 + style QD fill:#f3f8ff,stroke:#0969da + style SEG0 fill:#f3fbf5,stroke:#1a7f37 + style SEG1 fill:#f3fbf5,stroke:#1a7f37 +``` + +*How a query really runs. On the coordinator the single **QD** (`Gp_role = GP_ROLE_DISPATCH`) parses and plans the query, cutting it into **slices** at each Motion; its **Dispatcher** (`CdbDispatchPlan`) allocates **gangs** of QEs and ships the plan with the **distributed snapshot** to them over **libpq**, then coordinates the 2PC commit (§13, §12). A segment may **fork several QEs** — one per slice — which share one **SharedLocalSnapshot** so they see the same data; each **Executor** scans and writes its segment's shard *locally* (§18), takes the **locks** it needs (§11), and logs its changes to that segment's **WAL** (§20). Reaching another segment's data — and the final **Gather** to the QD — travels over the **Interconnect** (UDPIFC) as **Motion**, a path separate from the libpq dispatch (§19).* + +## 1.1 The MPP Cluster: Coordinator, Segments & Distribution + +**本节要点:** `shared-nothing` · `distribution policies` · `cdbhash → segment` · `cluster catalogs` Review Comment: `本节要点:` needs to be translated into English here. FYI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
