xylaaaaa opened a new issue, #69664: URL: https://github.com/apache/doris/issues/69664
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master (a626232f5cc) ### What's Wrong? With spill to object storage in cloud mode (#68032) on an Azure storage vault whose account has the hierarchical namespace (HNS, ADLS Gen2) enabled, every spilling query leaves empty directories behind after its spill files are cleaned up. On such an account directories are real entries (0-byte blobs with metadata `hdi_isfolder=true`), not just key prefixes. In one run, a sort query spilled 7 files; the query result was correct and all 7 spill file deletions logged OK, and no data file was left, but 9 directory entries remained: - 7 spill file directories, - 1 query directory, - 1 BE spill root directory (expected to stay). The 8 directories that belong to the query are never removed, so they accumulate with every spilling query. They hold no data, but they add objects to the vault and slow down listing. On S3 and on Azure accounts without HNS the prefix is empty after cleanup. ### What You Expected? After a query releases its spill files, the spill file directories and the query directory are removed from the vault, leaving only the BE spill root. ### How to Reproduce? 1. Cloud mode, an Azure storage vault on an account with hierarchical namespace enabled, spill to object storage enabled. 2. Run a query that spills, e.g. a sort over 50,000 wide rows with a small memory limit. 3. After the query finishes, list the spill prefix of the BE in the container (e.g. with the Azure SDK or `az storage fs file list`): the spill file directories and the query directory are still there as empty directories. ### Anything Else? Two places cause it: 1. `S3FileSystem::delete_directory_impl` appends `/` to the directory and deletes every object under that prefix (`delete_objects_recursively`). On an HNS account the directory itself is an entry without the trailing `/`, so it is not deleted. This is the path a spill file takes when it is destroyed. 2. `SpillFileManager::delete_query_spill_directory` returns early for remote spill (\"nothing is deleted by directory on object storage\"), so the query directory is never deleted on object storage. That is enough for flat namespaces, where the query directory disappears with its last object, but not with HNS. Possible fix: for Azure, also delete the directory entry itself after deleting its objects (an empty HNS directory can be deleted through the Blob API), and delete the remote query directory once all spill files of the query are released, keeping the existing retry on failure. A check with the Azure SDK on an HNS account confirmed that deleting the child objects leaves the directory, that the Blob API deletes an empty directory, and that sibling prefixes are unaffected. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
