XJDKC commented on code in PR #14398:
URL: https://github.com/apache/iceberg/pull/14398#discussion_r2483800221
##########
core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java:
##########
@@ -413,14 +521,34 @@ public Table loadTable(SessionContext context,
TableIdentifier identifier) {
// attempt to load a metadata table using the identifier's namespace
as the base table
TableIdentifier baseIdent =
TableIdentifier.of(identifier.namespace().levels());
try {
- response = loadInternal(context, baseIdent, snapshotMode);
+ responseHeaders.clear();
+ cachedTable =
+ tableCache.getIfPresent(SessionIDTableID.of(context.sessionId(),
baseIdent));
+
+ response =
+ loadInternal(
+ context,
+ baseIdent,
+ snapshotMode,
+ headersForLoadTable(cachedTable),
Review Comment:
I fully understand your points, they make sense
But just to add some context from real-world use cases: There are scenarios
where the query engine tries to optimize loadTable performance by persisting
the ETag somewhere. That way, when the engine restarts, it doesn't have to
perform a full loadTable again and can instead reuse the previously fetched
version.
It would be great if we could provide some flexibility or hooks for
customization when implementing this!
##########
core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java:
##########
@@ -413,14 +521,34 @@ public Table loadTable(SessionContext context,
TableIdentifier identifier) {
// attempt to load a metadata table using the identifier's namespace
as the base table
TableIdentifier baseIdent =
TableIdentifier.of(identifier.namespace().levels());
try {
- response = loadInternal(context, baseIdent, snapshotMode);
+ responseHeaders.clear();
+ cachedTable =
+ tableCache.getIfPresent(SessionIDTableID.of(context.sessionId(),
baseIdent));
+
+ response =
+ loadInternal(
+ context,
+ baseIdent,
+ snapshotMode,
+ headersForLoadTable(cachedTable),
Review Comment:
I fully understand your points, they make sense!
But just to add some context from real-world use cases: There are scenarios
where the query engine tries to optimize loadTable performance by persisting
the ETag somewhere. That way, when the engine restarts, it doesn't have to
perform a full loadTable again and can instead reuse the previously fetched
version.
It would be great if we could provide some flexibility or hooks for
customization when implementing this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
