starocean999 opened a new pull request, #59872: URL: https://github.com/apache/doris/pull/59872
Refactor recursive-CTE fe part. Related PR: https://github.com/apache/doris/pull/58916 **Summary** - **Description:** Replace the legacy RecursiveCTE model with a clearer RecursiveUnion model (anchor / producer / work-table reference), update planner/physical nodes, runtime Thrift wiring, stats, visitors and tests to match. - **Motivation:** Separate anchor (initial rows) and producer (recursive body) responsibilities; make work-table explicit; deliver BE control/reset info via Thrift. **High-Level Design Changes** - **Model Change:** Introduced RecursiveUnion concept (anchor / producer / work-table reference) replacing old RecursiveCte/RecursiveCteScan model. - **Stats:** New estimation flow for recursive nodes and an expansion ratio to estimate recursion growth. - **Types:** Added [equalsForRecursiveCte](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) hooks for type-equality checks used during recursive compatibility/coercion. **Logical / Physical Nodes Added** - **LogicalRecursiveUnion:** [LogicalRecursiveUnion.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) — new main logical node representing recursive union. - **LogicalRecursiveUnionAnchor:** [LogicalRecursiveUnionAnchor.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) — anchor sentinel (initial side). - **LogicalRecursiveUnionProducer:** [LogicalRecursiveUnionProducer.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) — producer sentinel (recursive side). - **LogicalWorkTableReference:** [LogicalWorkTableReference.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) — logical ref to recursive work table. - **Physical counterparts:** - PhysicalRecursiveUnion: [PhysicalRecursiveUnion.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - PhysicalRecursiveUnionAnchor: [PhysicalRecursiveUnionAnchor.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - PhysicalRecursiveUnionProducer: [PhysicalRecursiveUnionProducer.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - PhysicalWorkTableReference: [PhysicalWorkTableReference.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) **Removed / Renamed** - **Removed old scan/job classes:** `LogicalRecursiveCteScan`, `PhysicalRecursiveCteScan`, `UnassignedRecursiveCteScanJob` were deleted. - Deleted: fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalRecursiveCteScan.java - Deleted: fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalRecursiveCteScan.java - Deleted: fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/distribute/worker/job/UnassignedRecursiveCteScanJob.java **Visitor / Copier / Planner Updates** - **PlanVisitor / RelationVisitor:** Added visitors for the new nodes and removed/renamed the old recursive-cte visit methods. - Files: [PlanVisitor.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html), [RelationVisitor.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - **Deep copy:** [LogicalPlanDeepCopier](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) support for new RecursiveUnion nodes. - File: [LogicalPlanDeepCopier.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) **Rewrite / Optimization Rules** - **CTEInline / ColumnPruning / AdjustNullable:** Adjusted to work with new node types; legacy per-recursive-node logic removed or migrated. - Files: - [CTEInline.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - [ColumnPruning.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - [AdjustNullable.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) **Distribution / Job Builder** - **UnassignedJobBuilder:** Removed special-case for recursive-CTE scanning; new sentinel nodes drive shuffle/assignment decisions. - File: [UnassignedJobBuilder.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) **Statistics / Cost** - **StatsCalculator:** Reworked to support new nodes and added [RECURSIVE_CTE_EXPAND_RATION](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) to estimate recursion expansion; anchor stores producer stats by [CTEId](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html). - File: [StatsCalculator.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) **Type Equality** - **DataType equality hooks:** Added [equalsForRecursiveCte](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) and specific implementations for complex types to relax/adjust equality semantics in recursive contexts. - Files: [DataType.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html), [StructType.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html), [VariantType.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) **Analyzer / CTE handling** - **LogicalCTE:** field renamed to [isRecursive](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) and usage updated. - File: [LogicalCTE.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) - **LogicalSubQueryAlias:** enhanced logic to detect recursive CTE by relation name (case handling) to differentiate analysis paths. - File: [LogicalSubQueryAlias.java](vscode-file://vscode-app/usr/share/code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: https://github.com/apache/doris-website/pull/1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
