allthingssecurity opened a new pull request, #29589: URL: https://github.com/apache/camel/pull/29589
# Description [CAMEL-26799](https://issues.apache.org/jira/browse/CAMEL-26799) Builds on #PR-A (CAMEL-26754, `ClusteredRoutePolicy` initial delay for routes added after start); opens after it merges (this branch is one commit on top of it, rebased onto main before it opens). `ClusteredRoutePolicy` sets its leader flag when it takes the leadership and then starts its routes; only a later leadership change starts or stops them again. When that start did nothing, the leader never ran the route: - **Leadership taken before the policy started.** A cluster service started on its own (created by the application and passed to `ClusteredRoutePolicy.forNamespace(service, namespace)`, as in `ClusteredRoutePolicyUnmanagedClusterServiceTest`, CAMEL-17065), whose view is already the leader when the routes are created: the policy takes the leadership when it gets the view (since CAMEL-25062 in `retainClusterView`), while it is not started yet. The start of the routes then returned at once on `!isRunAllowed()`; unlike the `isStarting()` case just above it, it did not set `startManagedRoutesEarly`, so nothing started the routes when the CamelContext started. - **Leadership taken while a route is added at runtime.** A route created while the node is not the leader is marked `autoStartup=false`. The route is created (and `onInit` runs) before the CamelContext knows it, without the route lock. If the node became the leader in between, the policy thread called `startRoute` for an id the CamelContext did not know yet, which does nothing, and recorded the route as started; the route was then added with `autoStartup=false` and stayed stopped. - **Route removed while the policy starts it.** `startRoute` on the policy thread waits for the route lock that `removeRoute` holds and then does nothing; the removed route was recorded as started again, so a later route with the same id that the policy does not manage was stopped when the node lost the leadership. Change, all in `doStartManagedRoutes` and the CamelContext listener of the policy: - when the policy is not started yet (new, built or initialized), the start is deferred with `startManagedRoutesEarly`, as while the CamelContext starts; a policy that is stopping or stopped still returns; - a route whose id the route controller does not know yet (`getRouteStatus` is null) is skipped and stays with the stopped routes. The listener now also takes the `RouteAddedEvent` of a stopped route of the policy once the CamelContext has started, and hands it to the policy thread (it only sets a flag on the adding thread, which holds the route lock), which starts the route if the node is the leader; its `startRoute` waits for the route lock, so it runs once the route is added. The route service is registered before the event, so a route skipped by the policy thread always gets that event later; - after `startRoute`, the route is recorded as started and then dropped again when `camelContext.getRoute(id)` is no longer that route or when the route is shut down (a removed route is shut down before `onRemove` forgets it, so either `onRemove` drops it after the add or the check sees it shut down). What stays as it was: routes added while the node is the leader are still started by the route controller; the policy lock and the policy thread are unchanged: leadership changes and the new route-added case are applied on the policy thread, never on the thread of the view or on the adding thread (the deferred start of the first case runs where the existing deferral already runs, on the thread that starts the CamelContext or on the initial-delay timer). Not changed here (follow-ups): the mirror case, a leadership lost after a route was created on the leader and before it was added (the route keeps `autoStartup` and starts on a node that is no longer the leader); the same stale-route check on the stop side (`doStopManagedRoutes`); a route reload can still start the replacement route once, as before. Tests, in `ClusteredEdgeCasesTest` (in-JVM test cluster view): - `testLeadershipTakenBeforeTheCamelContextStarted`: a cluster service started on its own with a leader view, then the CamelContext starts: the route must start; - `testLeadershipTakenWhileARouteIsAdded`: a second route policy, listed after the clustered one, makes the node the leader in its `onInit`, and waits until the policy thread has tried to start the route (the old code did; the fixed policy does not, so this wait ends after its timeout of one second): the route must start once added; - `testRouteRemovedWhileThePolicyStartsIt`: the start by the policy thread waits until the route is removed; a route with the same id that is not clustered is added: it must stay started when the node loses the leadership. On the code of #PR-A the three fail (the route stays stopped); with only the deferral of the first case reverted, the first one fails. With the change the camel-core cluster tests pass (54, 0 failures, `org.apache.camel.cluster`; with the 9 of `ClusteredEdgeCasesTest`, so the tests of #PR-A too), the camel-file cluster tests too (16, 0 failures), and `ClusteredEdgeCasesTest` passes 180/180 over 20 runs in one JVM. A backport would take #PR-A first (same method). Found with Specula (TLA+ model checked against traces of the real code) and confirmed with the tests above. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it): [CAMEL-26799](https://issues.apache.org/jira/browse/CAMEL-26799). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built the core modules with the formatter and import-sort plugins and ran the camel-core cluster tests; no generated file changes. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. This PR was prepared with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
