davsclaus opened a new pull request, #30440: URL: https://github.com/apache/camel/pull/30440
## Description [CAMEL-27181](https://issues.apache.org/jira/browse/CAMEL-27181): the Data Lake consumer reads every file of the file system (or of `path`) at every poll. It had no option to delete a file once its exchange completed: the on-completion only logged at TRACE. camel-azure-storage-blob has `deleteAfterRead`; `DataLakeConfiguration` had nothing. - New consumer option `deleteAfterRead` (default `false`, so nothing changes for existing routes). - The file name is taken when the exchange is created, so a route that changes the `CamelAzureStorageDataLakeFileName` header still deletes the file it consumed. - In `onComplete`, the file is deleted with `DataLakeFileClientWrapper.delete` (`deleteWithResponse`). - A 404 (already deleted, for example by another consumer) is logged at DEBUG. - Other failures go to the exception handler of the consumer. - A failed exchange keeps the file, so it is read again at the next poll. - New paragraph and example in the component doc. The catalog, its doc copy and the DSLs are regenerated. `moveAfterRead`, which the JIRA mentions as a possible follow-up, is not part of this PR. Needed for a delete-after-read option on the `azure-storage-datalake-source` Kamelet (apache/camel-kamelets#3280). ## Tests `DataLakeConsumerDeleteAfterReadTest` uses a mocked service client, as `DataLakeConsumerFileThatCannotBeGotTest` does: - files are deleted after they are consumed, and each is consumed once - the original file is deleted when the route changes the header - a failed exchange keeps the file, which is read again - without the option, nothing is deleted - a failed delete reaches the exception handler All 37 unit tests of the module pass. The new test passed 3 runs in a row. _Claude Code on behalf of Claus Ibsen_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
