github-actions[bot] commented on code in PR #68998:
URL: https://github.com/apache/doris/pull/68998#discussion_r4257799798
##########
fe/fe-connector/fe-connector-hive/src/main/java/org/apache/doris/connector/hive/HiveConnectorTransaction.java:
##########
@@ -1142,7 +1143,17 @@ private void deleteTargetPathContents(String targetPath,
String excludedChildPat
}
deleteDir(dir);
}
- List<FileEntry> files =
getFileSystem().listFiles(Location.of(targetPath));
+ // Storage implementations may treat listFiles arguments as globs.
Overwrite must enumerate
+ // the literal target, or a table name such as table_{prod} can
leave old rows behind.
+ List<FileEntry> files = new ArrayList<>();
+ try (FileIterator iterator =
getFileSystem().list(Location.of(targetPath))) {
+ while (iterator.hasNext()) {
+ FileEntry file = iterator.next();
+ if (!file.isDirectory()) {
Review Comment:
[P1] Keep staged files out of recursive overwrite cleanup
For an unpartitioned Hive table at
`abfss://[email protected]/lake.lakehouse/Files/table_*`
with `hive.staging_dir=.doris_staging`, `LocationPath` selects `FILE_HDFS`, so
new files are staged beneath the target. `AzureFileSystem.list()` recursively
emits those staged blobs as non-directories. The exclusion in the directory
loop never protects them: this loop collects and deletes both staged input and
old data before `prepareAlterTable` schedules the move, leaving the copy
without its source and no restoration task for old rows. Previously the
wildcard `listFiles()` branch rejected descendant keys, leaving the staging
source intact even for an empty table. Exclude every file beneath
`excludedChildPath`, or use literal nonrecursive enumeration, and cover this
OneLake branch. This is distinct from the fixed DFS glob-matching issue in
thread 4256817233. The literal-directory applicability follows [ADLS directory
naming](https://learn.microsoft.com/en-us/rest/a
pi/storageservices/naming-and-referencing-containers--blobs--and-metadata#directory-names)
and the documented [OneLake API
parity](https://learn.microsoft.com/en-us/fabric/onelake/onelake-api-parity);
the endpoint behavior is inferred from that parity and the traced code, not a
live service test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
