starocean999 opened a new pull request, #67256:
URL: https://github.com/apache/doris/pull/67256
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
A prepared statement lives as long as its connection. The
`PreparedStatementContext` kept in `ConnectContext.preparedStatementContextMap`
retains a single `StatementContext` and reuses the same object across every
`EXECUTE` for the whole connection lifetime.
Because one object is reused across executions, its per-statement state
keeps accumulating: bound tables (`tables`, `oneLevelTables`,
`mtmvRelatedTables`, `insertTargetTables`, `viewInfos`), CTE maps, statistics
(`relationIdToStatisticsMap`, `tableIdMapping`), MV/partition rewrite state
(`mvCanRewritePartitionsMap`, `tmpPlanForMvRewrite`,
`materializationRewrittenSuccessSet`), MVCC snapshots, connector write schemas,
placeholder bindings (`idToPlaceholderRealExpr`), etc. On long-lived
connections with a high number of `EXECUTE`s, these maps only grow and are
never released until the connection closes, which can OOM the FE.
**Root cause:** the `StatementContext` stored in `PreparedStatementContext`
was treated as a permanent per-prepared-statement object and reused, so state
that should be per-execution lived as long as the connection.
**Fix:** instead of reusing (and clearing in place) the same
`StatementContext`, allocate a brand-new context on every `EXECUTE` and carry
over only the state that must survive between executions:
- **ID generator positions** — so ids generated during this execution never
collide with ids already present in the cached analyzed plan from `PREPARE`;
- **placeholder real expressions** bound by the protocol layer for this
`EXECUTE` (`idToPlaceholderRealExpr`) — this is the piece that prevents the
#63920 parameter-mismatch regression;
- the **placeholder → comparison-slot registry** (`idToComparisonSlot`) used
by the short-circuit fast path;
- the **placeholder list**;
- the **short-circuit / nondeterministic flags** that gate the short-circuit
fast path before any re-planning.
After the swap, the previous context becomes unreachable and is promptly
GC'd, so memory no longer grows with the number of executions. The cached
analyzed plan and the point-query (short-circuit) cache live on
`PrepareCommand` and `PreparedStatementContext` respectively, so they keep
being reused across executions.
**Changes:**
- `IdGenerator`: add `getCurrentId()` so a fresh context can continue the id
generators from the previous one.
- `StatementContext`: add `createNextExecuteContext()` which allocates the
fresh context and copies over the cross-execution state above.
- `PreparedStatementContext`: add `nextStatementContext()` which swaps in
the fresh context so the old one is released.
- `ExecuteCommand`: `run()` now uses the fresh per-execution context (and
the now-redundant in-place
`resetConnectorStatementScope()`/`resetMvccSnapshots()` calls are removed since
a fresh context starts empty by construction).
- Unit tests updated to assert the fresh-context behavior
(`ExecuteCommandTest`, `ConnectorStatementScopeTest`).
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
