allthingssecurity opened a new pull request, #27039: URL: https://github.com/apache/camel/pull/27039
# Description [CAMEL-25130](https://issues.apache.org/jira/browse/CAMEL-25130) With optimistic locking, `JdbcAggregationRepository` can send the messages of a completed group a second time, and can overwrite a new group so that its messages are lost. Optimistic locking is the documented way to share the aggregation table between several Camel instances. `AggregateProcessor` reads the group with `get` (which puts the row version in `CamelOptimisticLockVersion`), aggregates, and calls `add(camelContext, key, oldExchange, newExchange)`, which should be a compare-and-set against `oldExchange`. With optimistic locking the aggregator does not lock the key, so another thread or node can complete the group between `get` and `add`. `JdbcAggregationRepository.add(camelContext, key, oldExchange, newExchange)` ignores `oldExchange`: - the group was completed meanwhile: `remove` deleted the row, so `add` finds no row and inserts the old group plus the new message as a new group. The next completion sends `[x, a, ...]` after `[x, b]`: x twice. - every group starts at version 1, so if a new group was started for the key in between, the stale `UPDATE ... WHERE version = 1` matches the new group and replaces it: its messages are lost. A stale `remove` can delete the new group in the same way. This change: - `add(camelContext, key, oldExchange, newExchange)` takes the expected version from `oldExchange`, not from `newExchange`. If `oldExchange` is not null and the row is gone, it throws `OptimisticLockingException`, so the aggregator retries and starts a new group with its own message. `oldExchange == null` with an existing row is an `OptimisticLockingException`, as before. - A new group starts with a random positive version below `Long.MAX_VALUE / 2` instead of 1 (`newGroupVersion()`, protected), so a version read from an earlier group of the same key never matches. This also stops a stale `remove` from deleting a new group. - The schema is unchanged: the documented column is `version BIGINT NOT NULL`, and groups stored before the upgrade keep their version. The non-optimistic `add(camelContext, key, exchange)` does the same statements and checks as before; only the version of a new group changes. Recovery does not compare versions. `ClusteredJdbcAggregationRepository` and the two Postgres repositories inherit `add`. - A side effect of taking the version from the old exchange: an aggregation strategy that returns the new exchange no longer fails every optimistic `add` for lack of a version. - Upgrade guide note for 4.23. Tests: - New `JdbcAggregateOptimisticCompletedGroupTest`: the Aggregate EIP with `optimisticLocking()` on H2, an aggregation strategy that holds message `a` on a latch after `get` read the group. The group is completed by another message, completed and followed by a new group, or completed by the completion timeout, and then `a` is released. Without the change the aggregated bodies are `[x,b, x,a,z]` (x twice), `[x,b, x,a,z]` (c lost) and `[x, x,a]`. With the change: `[x,b, a,z]`, `[x,b, c,a,z]` and `[x, a]`. - New `JdbcAggregationRepositoryOptimisticAddTest`: repository-level cases, add after the group was removed, add after a new group was started, a stale remove of a new group, an update with the new exchange, and controls (new group, group already exists, update and stale update). - Without the change 7 of the 10 new tests fail; the 3 controls pass. - With the change, the camel-sql unit tests: 299 tests, 0 failures, 2 errors. The 2 errors are `SqlFunctionDataSourceTest`, which cannot start its embedded MariaDB on my machine (the `mariadbd` binary needs a Homebrew `pcre2` library that is not installed). They are not related to this change. Found with a TLA+ model of the repository and the aggregator (get, add, remove and the completion timeout as atomic transactions), which finds both cases in 4 and 6 steps and holds with the fix for three threads plus the timeout checker. I then reproduced both with the real classes. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built and tested the affected module, including the formatter and import-sort plugins. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
