github-actions[bot] commented on code in PR #68135:
URL: https://github.com/apache/doris/pull/68135#discussion_r4043412875
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonCatalogOps.java:
##########
@@ -422,13 +423,44 @@ public boolean branchExists(Table table, String
branchName) {
@Override
public long rowCount(Table table) {
- // Legacy PaimonExternalTable.fetchRowCount /
PaimonSysExternalTable.fetchRowCount: sum
- // the planned-split record counts.
- long rowCount = 0;
- for (Split split :
table.newReadBuilder().newScan().plan().splits()) {
- rowCount += split.rowCount();
+ // System/format tables have no data snapshot count. A fallback
pair combines two
+ // branches, so its main snapshot alone cannot estimate the
relation either.
+ if (!(table instanceof FileStoreTable)
+ ||
PaimonTableDecorators.unwrapToFallbackOrBase((FileStoreTable) table)
+ instanceof FallbackReadFileStoreTable) {
+ return -1;
}
- return rowCount;
+ FileStoreTable fileStoreTable = (FileStoreTable) table;
+ CoreOptions options = fileStoreTable.coreOptions();
+ // Batch scans can exclude level-0 files or postponed buckets even
in full-snapshot mode.
+ // The snapshot counter includes those files; do not enumerate
manifests to correct it.
+ if ((!fileStoreTable.primaryKeys().isEmpty() &&
options.batchScanSkipLevel0()
+ &&
options.toConfiguration().get(CoreOptions.BATCH_SCAN_MODE) ==
CoreOptions.BatchScanMode.NONE)
+ || options.bucket() == BucketMode.POSTPONE_BUCKET) {
+ return -1;
+ }
+ switch (options.startupMode()) {
+ case LATEST:
+ case LATEST_FULL:
+ case FROM_TIMESTAMP:
+ case FROM_SNAPSHOT:
+ case FROM_SNAPSHOT_FULL:
+ break;
+ default:
+ // Incremental/file-creation-time scans and unresolved
compacted-full scans
+ // do not read the full snapshot selected by
TimeTravelUtil.
+ return -1;
+ }
+ if (fileStoreTable instanceof PrivilegedFileStoreTable) {
+ // Match the old scan's SELECT check without planning any
splits. TimeTravelUtil
+ // eagerly calls tagManager(), which requires INSERT on the
privilege wrapper.
+ fileStoreTable.newScan();
Review Comment:
[P2] Preserve catalog query authorization
With `query-auth.enabled=true`, the removed
`newReadBuilder().newScan().plan()` reached `DataTableBatchScan.plan() ->
authQuery() -> CatalogEnvironment.tableQueryAuth(...).auth(null)`, so a REST
catalog that denied this identity produced `TableNoPermissionException` and
statistics degraded to UNKNOWN. The replacement only calls `newScan()` for
`PrivilegedFileStoreTable` (and creating a scan does not run `authQuery`); an
ordinary REST-loaded `FileStoreTable` now reaches `TimeTravelUtil` and can
publish `totalRecordCount` despite the catalog denial. This is separate from
the wrapper's SELECT/INSERT check. Please run the catalog query authorization
before reading snapshot metadata (the existing `authorizeDeferredScan` path
shows the no-manifest form) and add a denying `query-auth.enabled` catalog test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
