Copilot commented on code in PR #51192:
URL: https://github.com/apache/doris/pull/51192#discussion_r2108122170
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java:
##########
@@ -658,17 +660,21 @@ public static long getHiveRowCount(HMSExternalTable
table) {
// Table parameters contains row count, simply get and return it.
if (parameters.containsKey(NUM_ROWS)) {
long rows = Long.parseLong(parameters.get(NUM_ROWS));
+ if (rows <= 0 && parameters.containsKey(SPARK_NUM_ROWS)) {
+ rows = Long.parseLong(parameters.get(SPARK_NUM_ROWS));
+ }
// Sometimes, the NUM_ROWS in hms is 0 but actually is not. Need
to check TOTAL_SIZE if NUM_ROWS is 0.
if (rows > 0) {
LOG.info("Get row count {} for hive table {} in table
parameters.", rows, table.getName());
return rows;
}
Review Comment:
[nitpick] The logic for reading NUM_ROWS and SPARK_NUM_ROWS could be folded
into a helper to reduce duplication and improve clarity.
```suggestion
// Use helper method to get row count from parameters.
long rows = getRowCountFromParameters(parameters);
if (rows > 0) {
LOG.info("Get row count {} for hive table {} in table
parameters.", rows, table.getName());
return rows;
```
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalTable.java:
##########
@@ -444,17 +444,21 @@ public long getCreateTime() {
private long getRowCountFromExternalSource() {
long rowCount = UNKNOWN_ROW_COUNT;
- switch (dlaType) {
- case HIVE:
- rowCount = StatisticsUtil.getHiveRowCount(this);
- break;
- case ICEBERG:
- rowCount = IcebergUtils.getIcebergRowCount(getCatalog(),
getDbName(), getName());
- break;
- default:
- if (LOG.isDebugEnabled()) {
- LOG.debug("getRowCount for dlaType {} is not supported.",
dlaType);
- }
+ try {
+ switch (dlaType) {
+ case HIVE:
+ rowCount = StatisticsUtil.getHiveRowCount(this);
+ break;
+ case ICEBERG:
+ rowCount = IcebergUtils.getIcebergRowCount(getCatalog(),
getDbName(), getName());
+ break;
+ default:
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("getRowCount for dlaType {} is not
supported.", dlaType);
+ }
+ }
+ } catch (Exception e) {
+ LOG.info("Failed to get row count for table {}.{}.{}",
getCatalog().getName(), getDbName(), getName(), e);
Review Comment:
[nitpick] Catching Exception and logging at INFO may hide real issues;
consider using WARN or DEBUG levels and narrowing the exception types.
```suggestion
} catch (IOException | UserException e) {
LOG.warn("Failed to get row count for table {}.{}.{} due to {}:
{}",
getCatalog().getName(), getDbName(), getName(),
e.getClass().getSimpleName(), e.getMessage(), e);
```
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalRowCountCache.java:
##########
@@ -110,10 +111,16 @@ public long getCachedRowCount(long catalogId, long dbId,
long tableId) {
RowCountKey key = new RowCountKey(catalogId, dbId, tableId);
try {
CompletableFuture<Optional<Long>> f = rowCountCache.get(key);
- if (f.isDone()) {
+ // Get row count synchronously by default.
+ if (ConnectContext.get() == null
+ ||
ConnectContext.get().getSessionVariable().fetchHiveRowCountSynchronous) {
return f.get().orElse(TableIf.UNKNOWN_ROW_COUNT);
+ } else {
+ if (f.isDone()) {
+ return f.get().orElse(TableIf.UNKNOWN_ROW_COUNT);
+ }
+ LOG.info("Row count for table {}.{}.{} is still processing.",
catalogId, dbId, tableId);
Review Comment:
[nitpick] The synchronous vs asynchronous branch is a bit nested; consider
refactoring into clearly separated methods or adding comments to improve
readability.
```suggestion
if (isSynchronousFetch()) {
return getRowCountSynchronously(f);
} else {
return getRowCountAsynchronously(f, catalogId, dbId,
tableId);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
