github-actions[bot] commented on code in PR #67186:
URL: https://github.com/apache/doris/pull/67186#discussion_r3891609704
##########
fe/fe-connector/fe-connector-hive/src/main/java/org/apache/doris/connector/hive/HiveScanPlanProvider.java:
##########
@@ -134,6 +139,7 @@ public List<ConnectorScanRange> planScan(ConnectorSession
session, ConnectorScan
HiveTableHandle hiveHandle = (HiveTableHandle)
request.getTableHandle();
String dbName = hiveHandle.getDbName();
String tableName = hiveHandle.getTableName();
+ recordPruningProfile(hiveHandle);
Review Comment:
[P2] Publish pruning stats on the prune-to-zero path. `applyFilter` can
already fetch a nonempty subset for a supported equality and attach its batch
stats, while Nereids evaluates another partition predicate and reduces
`SelectedPartitions` to empty. `PluginDrivenScanNode#getSplits` then returns
before `planScan` and `collectScanProfiles`, so this new `recordPruningProfile`
call is never reached and the successful zero-row query loses the HMS request
entirely. Drain the handle's stats before that early return (or publish them at
the request boundary), and cover a mixed predicate that performs the fetch but
ultimately prunes to zero.
##########
fe/fe-connector/fe-connector-hms/src/main/java/org/apache/doris/connector/hms/HmsClient.java:
##########
@@ -158,12 +158,37 @@ default List<String> listPartitionNamesFresh(String
dbName, String tableName, in
* @param dbName database name
* @param tableName table name
* @param partNames partition name strings
- * @return list of partition info
+ * @return exactly one partition per requested name, in request order
* @throws HmsClientException if the operation fails
*/
List<HmsPartitionInfo> getPartitions(String dbName, String tableName,
List<String> partNames);
+ /**
+ * Returns the requested partitions that still exist, in request order.
Missing partitions are omitted;
+ * duplicate, unexpected and malformed remote results remain errors. This
is used only by freshness probes
+ * where a partition may disappear after its name was pinned.
+ */
+ default List<HmsPartitionInfo> getExistingPartitions(String dbName, String
tableName,
Review Comment:
[P1] Keep whole-table freshness tolerant of vanished partitions. The new
exact contract also reaches `HiveConnectorMetadata#getTableFreshness`, which
reads a cached partition-name list and calls `getPartitions`. If an external
partition is dropped after that list was cached (the default TTL is 24h), the
object fetch omits it and strict validation now throws before the method can
compute the max over surviving partitions; the same race was omission-tolerant
before this change and can repeatedly abort MTMV freshness checks. Route this
whole-table probe through `getExistingPartitions` as well, preserving
duplicate/unexpected/malformed validation, and cover a
cached-name/list-then-drop case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
