allthingssecurity commented on PR #28133: URL: https://github.com/apache/camel/pull/28133#issuecomment-6285625421
Thanks for the review. **1. Continuation token.** Confirmed and fixed in 040a7cd4e963. `createExchanges` now records the last listed object name and how many objects were listed. The token comes from those values, as in camel-alibaba-oss: null when fewer than `maxMessagesPerPoll` were listed, otherwise the last listed name. It is set in a `finally`, so it also moves when the poll fails because none of the listed objects could be got. Those objects are listed again when the listing starts over. If the listing itself fails, the token stays where it was. The test `refusedLastListedObjectDoesNotHoldTheListingBack` uses `maxMessagesPerPoll=2` with a.txt, b.txt and c.txt (c.txt cannot be stated). After a and b are consumed, it adds a2.txt, which sorts before the old token. Without the change a2 is never consumed (`Expected: <1> but was: <0>`); with it, a2 is consumed. The upgrade guide entry mentions the new continuation. **2. Batch properties.** Agreed, but I left this one out of this PR. `processBatch` sets `CamelBatchSize` and `CamelBatchComplete` before it gets each body. Correct values would need every body of the poll first, which keeps the whole batch in memory, or a look-ahead get. Either one changes how the batch is processed, and the problem predates this PR (the CAMEL-25512 NoSuchKey skip). I can take it as a follow-up if you want. **Open question.** Yes, that is what the code does. When the stat of X fails and the stat of another listed object succeeds, `createExchanges` reports X to the exception handler as "skipped for this poll". If every get in `processBatch` then fails, the poll fails with the first get error. So X is reported as skipped, and the poll fails because of the other objects: X has no exchange in that batch. Nothing is consumed or deleted, and the next poll lists all of them again, as on main. Both messages are accurate, so I think it is acceptable. If you'd rather report X only when the batch then succeeds, I can hold the stat failures back until `processBatch` finishes. The camel-minio tests pass: 16 run, 0 failures (5 of them are the new pooled-exchange class, which runs the same tests with pooled exchanges). _Claude Code on behalf of allthingssecurity_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
