eric-maynard commented on code in PR #1508:
URL: https://github.com/apache/polaris/pull/1508#discussion_r2072467650
##########
service/common/src/main/java/org/apache/polaris/service/task/TableCleanupTaskHandler.java:
##########
@@ -243,12 +243,13 @@ private Stream<TaskEntity> getMetadataTaskStream(
private List<List<String>> getMetadataFileBatches(TableMetadata
tableMetadata, int batchSize) {
List<List<String>> result = new ArrayList<>();
List<String> metadataFiles =
Review Comment:
I see. Separating out the data files makes sense to me.
I think this came up on one of the previous PRs, but the real solution here
needs to eventually involve moving this purge work out of the catalog server
and into the maintenance service where we handle compaction etc. That's the
only way to really achieve scalability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
