924060929 commented on code in PR #68196:
URL: https://github.com/apache/doris/pull/68196#discussion_r4129770827
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalMetaCacheMgr.java:
##########
@@ -520,9 +597,44 @@ private void invalidateLanceTableAccess(long catalogId) {
public void invalidatePartitions(long catalogId,
String dbName, String tableName, List<String> partitions) {
- routeCatalogEngines(catalogId, cache -> safeInvalidate(
- cache, catalogId, "invalidatePartitions",
- () -> cache.invalidatePartitions(catalogId, dbName, tableName,
partitions)));
+ Optional<ExternalDatabase<? extends ExternalTable>> db =
Optional.empty();
+ try {
+ db = getCachedDb(catalogId, dbName);
+ invalidateTableRowCount(catalogId, db, tableName);
+ routeCatalogEngines(catalogId, cache -> safeInvalidate(
+ cache, catalogId, "invalidatePartitions",
+ () -> cache.invalidatePartitions(catalogId, dbName,
tableName, partitions)));
+ } finally {
+ invalidateTableRowCount(catalogId, db, tableName);
+ }
+ }
+
+ private void invalidateTableRowCount(long catalogId,
+ Optional<ExternalDatabase<? extends ExternalTable>> db, String
tableName) {
+ if (db.isPresent()) {
+ Optional<? extends ExternalTable> table =
db.get().getTableForReplay(tableName);
+ if (table.isPresent()) {
+ invalidateRowCountCache(table.get());
+ } else {
+ rowCountCache.invalidateDb(catalogId, db.get().getId());
+ }
+ } else {
+ rowCountCache.invalidateCatalog(catalogId);
Review Comment:
Fixed in 3d33dba7ab26. The table/partition invalidation routes now resolve
and capture one row-count fence before engine invalidation. If the DB object is
cold but its name-to-ID mapping remains, the fence uses that DB ID (before and
after the route), not the whole catalog. ExternalMetaCacheRouteResolverTest
covers all three routes and verifies DB-scoped invalidation with no
catalog-wide fence. Post-rebase FE build and 165 focused tests pass.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/PaimonMetadataOps.java:
##########
@@ -191,25 +191,10 @@ private boolean performDropDb(String dbName, boolean
ifExists, boolean force) th
@Override
public void afterDropDb(String dbName) {
- Optional<ExternalDatabase<? extends ExternalTable>> db =
dorisCatalog.getDbForReplay(dbName);
try {
- if (db.isPresent()) {
- // getDbForReplay normalizes case-insensitive database names
(lower_case_database_names
- // mode 1/2), so an alternate-case DROP DATABASE can resolve
the cached database while
- // an exact-key eviction with the caller's spelling would miss
it. Evict by the resolved
- // canonical local key so the removal listener still performs
the one typed SDK
- // invalidation; do not add a second typed scan under the
catalog write fence.
- dorisCatalog.unregisterDatabase(db.get().getFullName());
- return;
- }
- // The cached database could not be resolved (for example a mode-2
case mapping was removed
- // by a names refresh before replay). Exact-key eviction can miss
the canonical local key,
- // so also retire the remaining legacy database objects; otherwise
a same-name recreation
- // could reuse the stale object and its nested table-name cache.
The catalog-wide engine
- // flush below covers the SDK side, so the per-database engine
callbacks are suppressed.
+ // The DROP owns historical-name resolution. Ordinary replay
lookup also serves CREATE
Review Comment:
Fixed in 3d33dba7ab26. The metadata-ops drop contract now returns the
database name resolved by the connector; afterDropDb and the edit log use that
same name. Paimon returns the resolved ExternalDatabase full name after its
authenticated remote drop, so leader cleanup cannot switch back to the caller
alias. PaimonMetadataOpsTest verifies Foo resolves to FOO, remote drop
succeeds, and only FOO is unregistered; CatalogMgrTest covers new-log and
legacy-log replay histories. Post-rebase FE build and 165 focused tests pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
