allthingssecurity opened a new pull request, #27645: URL: https://github.com/apache/camel/pull/27645
# Description [CAMEL-25512](https://issues.apache.org/jira/browse/CAMEL-25512) The minio consumer hands each exchange to the route with an empty callback and does not wait; the object is deleted (`deleteAfterRead`, the default) or moved by the exchange's on completion. With an asynchronous route (`seda`, which hands the on completion over, kafka or another async producer, `threads()`), the next poll lists the same object again and routes it a second time. If the first exchange deletes the object in between, the stat or get of the next poll fails and that poll ends with an exception. The same happens with `objectName` set. This is the defect fixed for aws2-s3 in CAMEL-17110. The consumer now keeps the names of the objects whose exchanges are in progress: a poll skips them (listing and `objectName` mode), and a name is released when its exchange completes or fails (after the delete/move), or when the poll does not process it. No new option: aws2-s3 has a pluggable `inProgressRepository`, but camel-minio is deprecated in 4.23, so a plain set in the consumer keeps the change small and backportable to 4.14.x/4.18.x/4.22.x. With `deleteAfterRead=false` an object is now consumed again only after its previous exchange completed. As in aws2-s3, a poll can still fail if an exchange completes between the list request and the stat of its object in the next poll. Tests: - `MinioConsumerInProgressTest` with a small in-process fake of the S3 calls the consumer makes (JDK `HttpServer`, no container): the seda route holds the exchange until the consumer has completed a second poll (counted with a `pollStrategy`); listing and `objectName` mode. - `MinioConsumerInProgressFailureTest`: a failed object is consumed again by a later poll (passes without the change too; it guards the release on failure). - Without the change: `a.txt was consumed again by a later poll while its first exchange was still in flight ==> expected: <1> but was: <2>` (both modes). - With the change, the camel-minio unit tests pass (6). The MinIO ITs are disabled on main. Found with a TLA+ model of the poll and the asynchronous completion: "with deleteAfterRead every object is routed once" is violated in 10 steps (poll, dispatch, poll again, dispatch again). I then reproduced it with the real consumer. Not changed here: the google-storage consumer has the same shape (no in-progress set, delete in the on completion, asynchronous hand-off). # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built and tested camel-minio, including the formatter and import-sort plugins; the build did not change any generated file. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
