zy-kkk opened a new pull request, #68095:
URL: https://github.com/apache/doris/pull/68095
### What problem does this PR solve?
Issue Number: close #68035
Related feature: #68036
Doris reconstructs loaded Iceberg tables as BaseTable snapshot views.
BaseTable implements SupportsDistributedScanPlanning with a default true
result, so wrapping RESTTable loses its false override. The existing scan-entry
guard then allows unsupported local planning. With restricted metadata
credentials this surfaces as an S3 error; a separately documented simulated
server residual demonstrates why ignoring the server plan can also change
results.
Check the original table before statement snapshot conversion, tracked-table
publication, and cached TableOwner metadata serialization/weight estimation.
Reject unsupported tables before local manifest access and release acquired
table/catalog ownership on failure. Preserve the existing scan-entry guard and
normal client-planned snapshot semantics.
**Rejection includes metadata acquisition, not only SELECT scans.** DESC,
SHOW CREATE TABLE, and INSERT target-column binding also fail for
server-planned tables. Row-count and partition metadata paths using the same
read acquisition are subject to this guard. The sharedWritableTable method
itself is unchanged; this does not imply SQL INSERT remains supported.
Add a dedicated native Iceberg REST fixture, pinned by multi-platform
digest, to the existing Iceberg Compose stack. Keep the existing Spark/Postgres
REST baseline. MinIO initialization creates a private test bucket and data-only
credentials. The enabled regression requires a working /plan endpoint: an old
or missing fixture fails instead of skipping.
The fixture digest fixes the contents of an upstream `main` snapshot, built
on 2026-04-29 from `b0df3ca01d61b2f7ae7143ac660c6b16e33b6e46` (recorded in the
JAR build properties). This commit precedes the `apache-iceberg-1.11.0` tag by
105 commits; the image is not a release build. Moving `latest` cannot change
the pinned contents. The retention risk is that the old manifest can become
untagged and later be removed upstream, breaking fresh pulls.
**Request to CI maintainers:** build a linux/amd64 + linux/arm64 fixture
from the upstream `apache-iceberg-1.11.0` tag using the upstream shadowJar task
and Dockerfile, and publish it as
`doristhirdpartydocker/iceberg-rest-fixture:1.11.0`. Keep the current digest
until that artifact exists; verify source/architectures and rerun this
regression, then switch to the versioned image with its verified digest.
Mirroring the current image into the Doris namespace is a retention fallback,
but remains a snapshot. We do not assume the upstream release-image workflow
will be repaired in time for this PR.
The regression invokes the real /plan endpoint, verifies restricted
data-file reads succeed and manifest access returns 403, and checks explicit
Doris rejection for SELECT, metadata statements, INSERT, empty tables, and
$files with cache enabled/disabled. Client-planned reads remain four rows
before and after. No policy proxy is part of this Docker regression.
Server-forced planning mode is additionally covered by HTTP RESTCatalog unit
tests.
### Release note
Doris now rejects unsupported Iceberg server-side scan planning before
snapshot wrapping and local metadata-file access. Operations that acquire table
metadata, including DESC, SHOW CREATE TABLE and INSERT target binding, are also
rejected. Use client-side planning until server-side planning support is
implemented separately.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test (details below)
- Behavior changed:
- [ ] No.
- [x] Yes: server-planned table read/metadata acquisition, including
INSERT target binding, is rejected before local planning.
- Does this need documentation?
- [ ] No.
- [x] Yes: the included Iceberg Docker README documents setup, mandatory
coverage and image maintenance.
### Validation
- 275 focused connector tests passed, including 6 real HTTP RESTCatalog
tests; Checkstyle reported 0 violations. Connector module packaging succeeded.
The focused tests and Checkstyle were rerun after rebasing onto master at
`19ebe13d3ba`.
- Standard regression runner against the repository's native REST/MinIO
services and updated FE plugin: 1 suite passed, 0 skipped. Actual `/plan`,
restricted data reads, manifest 403, SELECT/DESC/SHOW CREATE/INSERT rejection,
and catalog cleanup were verified. Normal reads return the same four rows
before and after. No policy proxy is included in this regression.
- Old 1.10.0 fixture negative control: exit 1, 1 failed suite, 0 skipped,
with the explicit missing scan-planning-capability assertion. Repeat storage
initialization, shell syntax and Compose validation passed.
- The `.out` file was generated by the standard runner and replayed
unchanged. Full-diff whitespace checking only reports its final blank line,
which follows the regression framework's output convention.
**Validation limits:** Local E2E used macOS arm64 FE/BE and Linux arm64
containers before the rebase; the rebase leaves the patch unchanged. Linux
amd64 host execution and remote CI remain to be verified. The full existing
Spark/Postgres suite was not rerun. Full `build.sh --fe` is blocked by the
local x86_64 grpc-java generator; connector packaging and deployment succeeded.
**Existing baseline failure:** A broader Iceberg run had 1 failure out of
1254 tests, with 4 skipped:
`IcebergWritePlanProviderTest.planMergePreservesExplicitlyEmptyReadAcrossConcurrentFirstAppend`
expects null but receives -1. The same failure was reproduced on the
unmodified original baseline `28577df9f6d`; it is not suppressed or changed
here.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
