github-actions[bot] commented on code in PR #70216:
URL: https://github.com/apache/doris/pull/70216#discussion_r4475958229
##########
fe/be-java-extensions/fluss-scanner/src/main/java/org/apache/doris/fluss/FlussJniScanner.java:
##########
@@ -241,6 +247,38 @@ protected void openInternal() throws IOException {
}
}
+ /**
+ * The table, built from the info {@link FlussTableInfoCache} keeps for it
rather than asked of the
+ * cluster by every range. A TTL of zero asks the way every range used to,
metadata refresh included.
+ */
+ private Table openTable(Map<String, String> config) {
+ TablePath tablePath = TablePath.of(required(DB_NAME),
required(TABLE_NAME));
Review Comment:
[P1] Preserve the planned Fluss identity when metadata caching is disabled
When `fluss.metadata.cache.ttl.sec=0`, this early return skips the
`matchesPlannedIdentity` check below. If the table is dropped and recreated
after FE planned the range, `connection.getTable()` returns the replacement
table and the scanner then uses its table ID with the old bucket/offset range,
allowing rows from the wrong table/schema to be read. Load the live table info
through the same identity-check path (or check the returned table before
returning) even when TTL is zero.
##########
fe/fe-connector/fe-connector-fluss/src/main/java/org/apache/doris/connector/fluss/FlussConnectorMetadata.java:
##########
@@ -492,7 +492,9 @@ public List<ConnectorPartitionInfo>
listPartitions(ConnectorSession session,
// name makes every partition of this table unreadable, whether it has
any yet or not.
rejectUnreadablePartitionColumns(flussHandle, partitionKeys);
- List<PartitionInfo> flussPartitions =
adminOps.listPartitionInfos(flussHandle.toTablePath());
+ // From a short-lived cache: the engine lists on every statement,
point lookups included, and
+ // FlussPartitionListCache says why that is worth avoiding and what it
costs.
+ List<PartitionInfo> flussPartitions =
adminOps.listPartitionInfosForPruning(flussHandle.toTablePath());
Review Comment:
[P1] Do not use the stale partition cache for pruning
This call now supplies the partition set used to build Nereids' pruning map,
but `listPartitionInfosForPruning` may return a list cached for up to five
seconds. A partition created during that window is absent from the map; a
predicate that targets it is reduced to an empty selection before `planScan`,
whose later live listing cannot add it back. Use a fresh/versioned list for
correctness-critical pruning, or detect and refresh when the predicate names a
partition absent from the cached set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
