allthingssecurity opened a new pull request, #28136: URL: https://github.com/apache/camel/pull/28136
# Description [CAMEL-25785](https://issues.apache.org/jira/browse/CAMEL-25785) With `streamingUploadMode=true` and `timestampGroupingEnabled=true`, the aws2-s3 stream upload producer names the object of a time window only after the window (`grouped-messages_20240101_0800_0800-0805.txt`). The upload of a window is completed when it reaches `batchMessageNumber` messages (10 by default) or `batchSize` bytes, or when `streamingUploadTimeout` runs, and the window is then forgotten. The next message of the same window started a new multipart upload with the same key, and S3 replaced the first object with it. The messages of the earlier uploads were lost (unless the bucket is versioned), and every exchange had completed normally. The example in the component documentation (a message every 10 seconds, 5-minute windows, default `batchMessageNumber`) has 30 messages per window: the window is uploaded three times and the bucket keeps only the last 10 messages. Change (in `AWS2S3StreamUploadProducer`): the producer counts the completed uploads of each window, under its existing lock. The count lives in an access-ordered map that holds the last 10000 windows. A later upload of a window gets that count after the time range: `grouped-messages_20240101_0800_0800-0805-1.txt`, then `-2`, and so on. The first upload of a window keeps the name it had, so a window that is uploaded once is unchanged. What users can notice: the second and later objects of a window have a new key. Upgrade guide (4.24): a note under its own `camel-aws2-s3` heading. The naming is also described in `aws2-s3-streaming.adoc` (and its catalog copy). Tests: - New `AWS2S3StreamUploadTimestampWindowOverwriteTest` (mocked `S3Client` that keeps completed objects by key, like a bucket). `everyMessageOfAWindowIsStoredWhenTheWindowIsUploadedMoreThanOnce` (the documentation example) and `everyMessageOfAWindowIsStoredWhenTheTimeoutUploadsTheWindowBeforeItEnds` fail without the change (`expected: <3> but was: <1>`, `expected: <2> but was: <1>`: the bucket holds one object for the window). `theFirstUploadOfAWindowKeepsItsName` passes with and without the change. - `AWS2S3StreamUploadProducerTimestampTest` calls the name method through reflection; it now uses the new signature and checks the suffix. - With the change the camel-aws2-s3 tests pass: 107 run, 0 failures, 0 errors, 0 skipped. What is not changed: after a restart the count starts again, so a message for a window that was already uploaded before the restart still uses the name of the first upload. A failed timeout flush still drops its window with a `WARN` (as decided in CAMEL-24195). What was not run: the full root build; no test against S3. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-25785](https://issues.apache.org/jira/browse/CAMEL-25785). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built and tested camel-aws2-s3, including the formatter and import-sort plugins. No option changed; the catalog copy of the edited doc page is updated by hand. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
