github-actions[bot] commented on code in PR #69632:
URL: https://github.com/apache/doris/pull/69632#discussion_r4385876465
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/MetaCache.java:
##########
@@ -662,6 +687,103 @@ public void invalidate(String localName, long id) {
}
}
+ /** Fences old objects while retaining unrelated numeric routes for fresh
reload after wide retirement. */
+ public void invalidateLookup(String localName, boolean allObjects) {
+ metaObjLifecycleLock.writeLock().lock();
+ try {
+ synchronized (namesMutationLock) {
+ invalidateNames();
+ metaObjGeneration.incrementAndGet();
+ lookupObservationGeneration++;
+ if (allObjects) {
+ metaObjCache.invalidateAll();
+ } else {
+ metaObjCache.invalidate(localName);
+ }
+ // Only the target's routes retire. Surviving siblings must
reload by ID without
+ // requiring a foreground named lookup to register their
identities again.
+ String foldedName = allObjects ?
localName.toLowerCase(Locale.ROOT) : localName;
+ idToName.entrySet().removeIf(entry -> allObjects
+ ?
entry.getValue().toLowerCase(Locale.ROOT).equals(foldedName)
+ : entry.getValue().equals(localName));
+ }
+ } finally {
+ metaObjLifecycleLock.writeLock().unlock();
+ }
+ }
+
+ /** Blocks missing-result repair during a metadata-only refresh without
evicting names or objects. */
+ public Runnable beginMetadataRefresh() {
+ metaObjLifecycleLock.writeLock().lock();
+ try {
+ metadataRefreshes++;
+ lookupObservationGeneration++;
+ } finally {
+ metaObjLifecycleLock.writeLock().unlock();
+ }
+ return () -> {
+ metaObjLifecycleLock.writeLock().lock();
+ try {
+ metadataRefreshes--;
+ lookupObservationGeneration++;
+ } finally {
+ metaObjLifecycleLock.writeLock().unlock();
+ }
+ };
+ }
+
+ /** Captures a complete names snapshot and matching resident object; never
loads remote metadata. */
+ public Optional<Runnable> observeLookup(String localName, String
remoteName, T expected,
+ boolean allObjects, Runnable invalidateMetadata) {
+ metaObjLifecycleLock.readLock().lock();
+ try {
+ synchronized (namesMutationLock) {
+ NamesCacheValue names = namesCache.getIfPresent("");
+ Optional<T> value = metaObjCache.getIfPresent(localName);
+ Pair<String, String> pair = names == null ? null :
names.names.get(localName);
+ // A complete refresh may already omit a remotely dropped
table while its object/ID
+ // remains hot. The retained object still authorizes repair; a
conflicting live name does not.
+ if (metadataRefreshes != 0 || names == null || !names.complete
+ || (pair != null && !pair.key().equals(remoteName)) ||
value == null || !value.isPresent()
+ || value.get() != expected) {
+ return Optional.empty();
+ }
+ long objectGeneration = metaObjGeneration.get();
+ long observationGeneration = lookupObservationGeneration;
+ return Optional.of(() -> {
+ metaObjLifecycleLock.writeLock().lock();
+ try {
+ synchronized (namesMutationLock) {
+ if (metadataRefreshes != 0 || objectGeneration !=
metaObjGeneration.get()
+ || observationGeneration !=
lookupObservationGeneration
+ || namesCache.getIfPresent("") != names
+ || metaObjCache.getIfPresent(localName) !=
value) {
+ return;
+ }
+ // Case-folded catalogs can retain Foo's old
object after a names refresh
+ // has published a new FOO identity. Do not widen
that old miss to the DB.
+ if (pair == null && allObjects &&
names.names.values().stream().anyMatch(current ->
+ current.value().equalsIgnoreCase(localName)
Review Comment:
[P2] Use the lookup identity's folding rule for replacement checks
With `lower_case_table_names=2` on an English/ROOT-casing FE, `I` and
dotless `ı` have distinct lowercase lookup keys (as the new
`testWideInvalidationUsesRootLocaleForIdentityMatching` also expects). Warm
both, drop `I` externally, and let a complete names-only refresh publish just
`ı` while the old `I` object/ID remains hot. An ID-only statistics lookup can
still reach `getColumn()` -> guarded Iceberg `getFullSchema()`; with
`meta.cache.iceberg.table.ttl-second=0`, it confirms `NoSuchTableException`.
Here `pair == null`, but `ı.equalsIgnoreCase("I")` is true, so repair returns
before either metadata fencing or `invalidateLookup()`. The dropped object's
numeric route survives, and hot ID readers can keep repeating the stale-object
failure. Compare with the same ROOT-lowercase identity rule used by
`invalidateLookup()` and cover this omitted-name observation case. This differs
from the existing sibling-route-loss thread: it incorrectly retains the dropped
target before retireme
nt begins.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
