924060929 commented on code in PR #69629:
URL: https://github.com/apache/doris/pull/69629#discussion_r4384532986
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalTable.java:
##########
@@ -393,6 +394,31 @@ public Optional<ColumnStatistic> getColumnStatistic(String
colName) {
return Optional.empty();
}
+ /** Only table-level remote reads use this seam; partition misses and
arbitrary failures do not repair names. */
+ protected <R> R withMissingTableRepair(Supplier<R> read) {
+ // Metadata validators can enter a schema read under this monitor.
Repair only after
+ // an outer unlocked entry point has unwound it, preserving catalog ->
table lock order.
+ if (Thread.holdsLock(this)) {
+ return read.get();
+ }
+ Optional<Runnable> observation = db.observeTableLookup(this);
Review Comment:
Fixed in b2bf7140487. ShowCreateTableCommand now delegates the live DDL read
to HMSExternalTable.getCreateTableSql(), which uses guarded repair. The
real-HMS regression warms a listed table, drops it externally, executes SHOW
CREATE first, verifies membership becomes empty, and verifies a subsequent
lookup reports FE table absence.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalTable.java:
##########
@@ -1246,14 +1258,17 @@ public void beforeMTMVRefresh(MTMV mtmv) throws
DdlException {
@Override
public MvccSnapshot loadSnapshot(Optional<TableSnapshot> tableSnapshot,
Optional<TableScanParams> scanParams) {
- if (getDlaType() == DLAType.HUDI) {
- return HudiUtils.getHudiMvccSnapshot(tableSnapshot, this);
- } else if (getDlaType() == DLAType.ICEBERG) {
- return new IcebergMvccSnapshot(
- IcebergUtils.getSnapshotCacheValue(tableSnapshot, this,
scanParams));
- } else {
- return new EmptyMvccSnapshot();
- }
+ return withMissingTableRepair(() -> {
Review Comment:
Fixed in b2bf7140487. HMS initialization now enters the repair boundary
before a private synchronized initializer and runs cleanup only after that
monitor unwinds. A supportsLatestSnapshotPreload()-first missing-table test
verifies repair and asserts cleanup does not hold the table monitor.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalDatabase.java:
##########
@@ -680,6 +683,64 @@ public boolean unregisterTableForReplay(String tableName) {
return true;
}
+ public void invalidateTableLookup(String tableName) {
+ if (metaCache != null) {
+ String localName =
getTableForReplay(tableName).map(ExternalTable::getName).orElse(tableName);
+ metaCache.invalidateLookup(localName,
extCatalog.getLowerCaseTableNames() != 0);
+ }
+ }
+
+ /** Historical spellings require typed engine invalidation and an
independent DB row-count fence. */
+ public void invalidateTableLookupDependencies() {
+ ExternalMetaCacheMgr cacheMgr =
Env.getCurrentEnv().getExtMetaCacheMgr();
+ long catalogId = getCatalog().getId();
+ cacheMgr.invalidateRowCountCache(catalogId, getId());
+ try {
+ cacheMgr.invalidateDb(this);
+ } finally {
+ cacheMgr.invalidateRowCountCache(catalogId, getId());
+ }
+ }
+
+ public Optional<Runnable> beginTableMetadataRefresh() {
+ return metaCache == null ? Optional.empty() :
Optional.of(metaCache.beginMetadataRefresh());
+ }
+
+ /** A missing result is authorized only against the retained DB, names and
object generations. */
+ @SuppressWarnings("unchecked")
+ public Optional<Runnable> observeTableLookup(ExternalTable table) {
+ MetaCache<T> cache = metaCache;
+ if (!initialized || cache == null) {
+ return Optional.empty();
+ }
+ long epoch = acquireTableMetadataLoadEpoch();
Review Comment:
Fixed in b2bf7140487. Observation captures only the existing database epoch
and compares the existing parent epoch; it never acquires/initializes a remote
catalog. The retired-parent pinned-read test makes acquireMetadataLoadEpoch()
fail if invoked and verifies the retained schema read still succeeds without
calling it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
