924060929 commented on code in PR #66187:
URL: https://github.com/apache/doris/pull/66187#discussion_r4408445371
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java:
##########
@@ -1539,6 +1539,67 @@ public Partition dropPartitionForTruncate(long dbId,
boolean isForceDrop,
*
*/
+ @Override
+ public Partition getMaxVisiblePartition() {
+ PartitionInfo partitionInfo = getPartitionInfo();
+ PartitionType type = partitionInfo.getType();
+ // Cloud mode: prefetch all visible versions in one batch RPC so the
loop below hits
+ // cache instead of firing N sequential meta-service RPCs under
metadata locks.
+ if (Config.isCloudMode()) {
+ try {
+ getVersionInBatchForCloudMode(nameToPartition.values().stream()
+ .map(Partition::getId).collect(Collectors.toList()));
+ } catch (RpcException e) {
+ LOG.warn("batch prefetch visible version failed, fallback to
per-partition lookup", e);
+ }
+ }
+ if (type == PartitionType.UNPARTITIONED) {
+ for (Partition partition : nameToPartition.values()) {
+ if (partition.getVisibleVersion() >
Partition.PARTITION_INIT_VERSION) {
+ return partition;
+ }
+ }
+ return null;
+ }
+ // RANGE/LIST: pick the visible partition with the greatest key (RANGE
by upper bound,
+ // LIST by max discrete key). A LIST default partition is only a
last-resort fallback.
+ Partition result = null;
+ PartitionKey maxKey = null;
+ Partition defaultFallback = null;
+ for (Partition partition : nameToPartition.values()) {
+ if (partition.getVisibleVersion() <=
Partition.PARTITION_INIT_VERSION) {
Review Comment:
[P2] Consume batch versions instead of rereading each partition
In cloud mode, set `cloud_partition_version_cache_ttl_ms=0` and query
`SELECT * FROM t max_visible_partition()` on a table with N partitions. The
preceding `getVersionInBatchForCloudMode()` performs a batch lookup through
`CloudPartition.getSnapshotVisibleVersion()`, but discards its returned
versions. This loop then calls `getVisibleVersion()` for every partition. With
TTL <= 0, `isCachedVersionExpired()` always returns true, so each call invokes
`getVisibleVersionFromMs()`: one batch lookup is followed by N sequential RPCs.
Responses marked non-cacheable because of pending transactions can also cause
repeated lookups. This runs during binding while planner table locks are held,
increasing planning latency, meta-service load, and DDL lock waits for large
partition counts. Please select the partition directly from the batch API's
returned partition/version mapping rather than assuming the prefetched cache is
usable, and cover the TTL=0 cloud case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
