This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new a5667912eff8 CAMEL-26773: camel-chaos - the journal of an exploration:
a budget it enforces, the baseline, and the rules tried (#29466)
a5667912eff8 is described below
commit a5667912eff88cdefb1f92362804740fbd62d62d
Author: Claus Ibsen <[email protected]>
AuthorDate: Mon Nov 9 14:04:41 2026 +0100
CAMEL-26773: camel-chaos - the journal of an exploration: a budget it
enforces, the baseline, and the rules tried (#29466)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.../org/apache/camel/catalog/docs/chaos.adoc | 31 ++++
components/camel-chaos/src/main/docs/chaos.adoc | 31 ++++
.../camel/component/chaos/ChaosExploration.java | 187 +++++++++++++++++++
.../apache/camel/component/chaos/ChaosManager.java | 143 ++++++++++++++-
.../apache/camel/component/chaos/ChaosReport.java | 15 ++
.../component/chaos/ChaosExplorationTest.java | 204 +++++++++++++++++++++
.../apache/camel/cli/connector/ChaosAction.java | 13 +-
.../LocalCliConnectorChaosActionTest.java | 26 +++
.../dsl/jbang/core/commands/process/ListChaos.java | 44 +++++
.../jbang/core/commands/process/ListChaosTest.java | 25 +++
10 files changed, 715 insertions(+), 4 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
index 4db9c739828f..4d7c00bb6874 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc
@@ -328,6 +328,37 @@ adds or changes a rule, or loads a scenario. The
development MCP tools that `cam
serves over HTTP have only the list and the report (not the check either: the
integration parses the rule, and
`exception=` loads a class; nor what to try or a scenario).
+=== An exploration
+
+An agent that tries chaos rules on the integration by itself, for a time it
was given, runs an exploration. camel-chaos
+keeps its journal, so the report, the CLI and the TUI read it, and it survives
the agent losing its context. And
+camel-chaos enforces the time, whatever the agent does:
+
+* One exploration runs at a time, with a budget of 10 minutes by default, and
at most 2 hours.
+* While it runs, a rule ends at the deadline at the latest, and a rule that
would start after it (`from=`) is refused.
+* At the deadline the rules are removed and the exploration is finished. This
is checked whenever chaos is used: a
+rule added, or the report read, which the status of the application does every
second.
+* Chaos counts the exchanges from the start of the exploration, and its first
rule takes the baseline: the counts per
+route of the run without faults before it, so the failures that were there
already are not blamed on chaos.
+
+The `chaos` action of the CLI connector starts it (`{"action": "chaos",
"command": "explore-start", "budget": "10m",
+"focus": "orders"}`) and finishes it before the deadline (`explore-finish`),
as `ChaosManager.startExploration(budget,
+focus)` and `finishExploration()` do in Java. The report has its journal as
`exploration`: `running`, `startedAt`,
+`budget`, `deadline`, `timeLeft` while it runs, `focus`, the `rules` added
while it ran, the `baseline` (per `route` its
+`completed`, `failed`, `failedKeptByConsumer`, `handled` and `redeliveries`)
and `baselineAt`, and once finished
+`finishedAt`, `finishedBy` (`finished` or `deadline`) and the number of rules
`removed`. It stays in the report after it
+finished, until the next one starts or the report is reset. A scenario loaded
while it runs starts the counts again,
+and the baseline is zero then (`baselineReset`).
+
+[source,text]
+----
+$ camel get chaos --report
+...
+ exploration: running, 6m12s left of 10m0s, focus: orders
+ rules tried: r1, r2
+ baseline (before the first rule): orders 30 completed, 0 failed
+----
+
== Scenarios
A scenario is the rules of an experiment in a file of the project, so it can
be run again, shared and kept with the
diff --git a/components/camel-chaos/src/main/docs/chaos.adoc
b/components/camel-chaos/src/main/docs/chaos.adoc
index 4db9c739828f..4d7c00bb6874 100644
--- a/components/camel-chaos/src/main/docs/chaos.adoc
+++ b/components/camel-chaos/src/main/docs/chaos.adoc
@@ -328,6 +328,37 @@ adds or changes a rule, or loads a scenario. The
development MCP tools that `cam
serves over HTTP have only the list and the report (not the check either: the
integration parses the rule, and
`exception=` loads a class; nor what to try or a scenario).
+=== An exploration
+
+An agent that tries chaos rules on the integration by itself, for a time it
was given, runs an exploration. camel-chaos
+keeps its journal, so the report, the CLI and the TUI read it, and it survives
the agent losing its context. And
+camel-chaos enforces the time, whatever the agent does:
+
+* One exploration runs at a time, with a budget of 10 minutes by default, and
at most 2 hours.
+* While it runs, a rule ends at the deadline at the latest, and a rule that
would start after it (`from=`) is refused.
+* At the deadline the rules are removed and the exploration is finished. This
is checked whenever chaos is used: a
+rule added, or the report read, which the status of the application does every
second.
+* Chaos counts the exchanges from the start of the exploration, and its first
rule takes the baseline: the counts per
+route of the run without faults before it, so the failures that were there
already are not blamed on chaos.
+
+The `chaos` action of the CLI connector starts it (`{"action": "chaos",
"command": "explore-start", "budget": "10m",
+"focus": "orders"}`) and finishes it before the deadline (`explore-finish`),
as `ChaosManager.startExploration(budget,
+focus)` and `finishExploration()` do in Java. The report has its journal as
`exploration`: `running`, `startedAt`,
+`budget`, `deadline`, `timeLeft` while it runs, `focus`, the `rules` added
while it ran, the `baseline` (per `route` its
+`completed`, `failed`, `failedKeptByConsumer`, `handled` and `redeliveries`)
and `baselineAt`, and once finished
+`finishedAt`, `finishedBy` (`finished` or `deadline`) and the number of rules
`removed`. It stays in the report after it
+finished, until the next one starts or the report is reset. A scenario loaded
while it runs starts the counts again,
+and the baseline is zero then (`baselineReset`).
+
+[source,text]
+----
+$ camel get chaos --report
+...
+ exploration: running, 6m12s left of 10m0s, focus: orders
+ rules tried: r1, r2
+ baseline (before the first rule): orders 30 completed, 0 failed
+----
+
== Scenarios
A scenario is the rules of an experiment in a file of the project, so it can
be run again, shared and kept with the
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExploration.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExploration.java
new file mode 100644
index 000000000000..89a9e3560e9f
--- /dev/null
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosExploration.java
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.chaos;
+
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.StringJoiner;
+
+import org.apache.camel.util.json.JsonArray;
+import org.apache.camel.util.json.JsonObject;
+
+/**
+ * The journal of an exploration (CAMEL-26773): an agent tries chaos rules on
the integration for a time it was given,
+ * and camel-chaos keeps what it did, so the report, the CLI and the TUI read
it, and it survives the agent losing its
+ * context. camel-chaos also enforces the time: a rule added while it runs
ends at the deadline at the latest, and at
+ * the deadline the rules are removed and the exploration is finished,
whatever the agent does.
+ * <p/>
+ * It has the baseline, the counts per route when it started, so the failures
that were there before are not blamed on
+ * chaos, and the rules added while it ran. The manager guards it with its
lock.
+ */
+final class ChaosExploration {
+
+ static final long DEFAULT_BUDGET = 10 * 60_000L;
+ static final long MIN_BUDGET = 10_000L;
+ static final long MAX_BUDGET = 2 * 60 * 60_000L;
+
+ /** How an exploration finished. */
+ enum Finish {
+ // finished before the deadline (finishExploration)
+ FINISHED("finished"),
+ // the time was up
+ DEADLINE("deadline");
+
+ final String key;
+
+ Finish(String key) {
+ this.key = key;
+ }
+ }
+
+ final long startedAt;
+ final long budget;
+ final long deadline;
+ final String focus;
+ // the counts per route before its first rule (the run without faults),
taken when the first rule is added:
+ // completed, failed, kept by the consumer, handled, redeliveries
+ private final Map<String, long[]> baseline = new LinkedHashMap<>();
+ private long baselineAt;
+ // the report was reset while it ran: the counts of the routes started
again from zero
+ private boolean baselineReset;
+ // the ids of the rules added while it ran (a changed rule keeps its id)
+ private final List<String> rules = new ArrayList<>();
+ private long finishedAt;
+ private Finish finishedBy;
+ private int removed;
+
+ ChaosExploration(long startedAt, long budget, String focus) {
+ this.startedAt = startedAt;
+ this.budget = budget;
+ this.deadline = startedAt + budget;
+ this.focus = focus != null && !focus.isBlank() ? focus.strip() : null;
+ }
+
+ boolean isRunning() {
+ return finishedBy == null;
+ }
+
+ /** A rule added while it runs: the first one takes the baseline, the
counts of the run without faults. */
+ void ruleAdded(String id, Map<String, ChaosManager.RouteCounts> counts,
long now) {
+ if (baselineAt == 0) {
+ baselineAt = now;
+ counts.forEach((route, c) -> baseline.put(route, new long[] {
+ c.completed.sum(), c.failed.sum(), c.failedKept.sum(),
c.handled.sum(), c.redeliveries.sum() }));
+ }
+ if (!rules.contains(id)) {
+ rules.add(id);
+ }
+ }
+
+ /** The counts of the routes were reset (a scenario was loaded): the
baseline is zero from now on. */
+ void reportReset() {
+ baseline.clear();
+ baselineReset = true;
+ }
+
+ void finished(long now, Finish by, int removedRules) {
+ finishedAt = now;
+ finishedBy = by;
+ removed = removedRules;
+ }
+
+ JsonObject toJson(long now) {
+ JsonObject jo = new JsonObject();
+ jo.put("running", isRunning());
+ jo.put("startedAt", startedAt);
+ jo.put("budget", budget);
+ jo.put("deadline", deadline);
+ if (isRunning()) {
+ jo.put("timeLeft", Math.max(0, deadline - now));
+ }
+ if (focus != null) {
+ jo.put("focus", focus);
+ }
+ if (baselineAt > 0) {
+ jo.put("baselineAt", baselineAt);
+ }
+ JsonArray list = new JsonArray();
+ baseline.forEach((route, c) -> {
+ JsonObject r = new JsonObject();
+ r.put("route", route);
+ r.put("completed", c[0]);
+ r.put("failed", c[1]);
+ r.put("failedKeptByConsumer", c[2]);
+ r.put("handled", c[3]);
+ r.put("redeliveries", c[4]);
+ list.add(r);
+ });
+ jo.put("baseline", list);
+ if (baselineReset) {
+ jo.put("baselineReset", true);
+ }
+ jo.put("rules", new JsonArray(rules));
+ if (!isRunning()) {
+ jo.put("finishedAt", finishedAt);
+ jo.put("finishedBy", finishedBy.key);
+ jo.put("removed", removed);
+ }
+ return jo;
+ }
+
+ int ruleCount() {
+ return rules.size();
+ }
+
+ String text(long now) {
+ StringBuilder sb = new StringBuilder();
+ appendText(sb, now);
+ return sb.toString();
+ }
+
+ void appendText(StringBuilder sb, long now) {
+ sb.append("\n exploration: ");
+ if (isRunning()) {
+ sb.append("running,
").append(ChaosRuleParser.printTime(Math.max(0, deadline - now))).append(" left
of ")
+ .append(ChaosRuleParser.printTime(budget));
+ } else {
+ sb.append(finishedBy == Finish.DEADLINE ? "finished at the
deadline" : "finished")
+ .append(" after
").append(ChaosRuleParser.printTime(finishedAt - startedAt)).append(" of ")
+ .append(ChaosRuleParser.printTime(budget));
+ if (removed > 0) {
+ sb.append(", ").append(removed).append(removed == 1 ? " rule
removed" : " rules removed");
+ }
+ }
+ if (focus != null) {
+ sb.append(" (focus: ").append(focus).append(')');
+ }
+ sb.append("\n rules tried: ").append(rules.isEmpty() ? "none yet" :
String.join(", ", rules));
+ StringJoiner sj = new StringJoiner(", ");
+ baseline.forEach((route, c) -> sj.add(route + " " + c[0] + "
completed, " + c[1] + " failed"));
+ sb.append("\n baseline: ");
+ if (baselineAt == 0) {
+ sb.append("taken when the first rule is added");
+ } else {
+ sb.append(sj.length() == 0 ? "no exchanges" :
sj.toString()).append(" in the ")
+ .append(ChaosRuleParser.printTime(baselineAt -
startedAt)).append(" before the first rule");
+ }
+ if (baselineReset) {
+ sb.append(" (the report was reset while it ran)");
+ }
+ }
+}
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
index 3f5ae8f18426..55ca97db689a 100644
---
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosManager.java
@@ -43,6 +43,7 @@ import org.apache.camel.component.chaos.ChaosRule.Hook;
import org.apache.camel.component.chaos.ChaosRule.State;
import org.apache.camel.support.ScheduledPollConsumer;
import org.apache.camel.util.TimeSource;
+import org.apache.camel.util.TimeUtils;
import org.apache.camel.util.json.JsonArray;
import org.apache.camel.util.json.JsonObject;
import org.slf4j.Logger;
@@ -106,6 +107,8 @@ public final class ChaosManager {
// lock), so the route of such a rule can be checked
private final ChaosLifecycle lifecycle;
private boolean routesCreated;
+ // the exploration running now or the last one (CAMEL-26773), guarded by
the lock
+ private ChaosExploration exploration;
ChaosManager(CamelContext camelContext, long seed, String seedFrom, long
maxDuration) {
this.camelContext = camelContext;
@@ -271,6 +274,8 @@ public final class ChaosManager {
checkRoute(definition);
}
long now = TimeSource.currentTimeMillis();
+ // an exploration whose time is up removes its rules first
+ checkDeadline(now);
if (rules.size() + definitions.size() > MAX_RULES) {
retireEnded(now);
}
@@ -279,12 +284,22 @@ public final class ChaosManager {
"There are " + rules.size() + " chaos rules already, and
at most " + MAX_RULES
+ "; remove some before adding
more");
}
+ ChaosExploration running = exploration != null &&
exploration.isRunning() ? exploration : null;
+ if (running != null) {
+ for (ChaosRule.Builder definition : definitions) {
+ // checked before any is added: all the rules or none
+ checkStartsBeforeDeadline(new ChaosRule(definition, "check",
seed, now, maxDuration), running);
+ }
+ }
List<ChaosRule> added = new ArrayList<>();
for (ChaosRule.Builder definition : definitions) {
ChaosRule rule = new ChaosRule(definition, "r" + nextId++, seed,
now, maxDuration);
rule.recentFaults = recentFaults;
// a rule added before the context started starts its clock when
it has started
rule.clockStarted = camelContext.isStarted();
+ if (running != null) {
+ endByDeadline(rule, running, now);
+ }
rules.add(rule);
added.add(rule);
}
@@ -315,15 +330,21 @@ public final class ChaosManager {
public int changeRule(String id, String rule) {
ChaosRule.Builder definition = parse(rule);
synchronized (lock) {
+ long now = TimeSource.currentTimeMillis();
+ // an exploration whose time is up removes its rules first, so the
rule may be gone
+ checkDeadline(now);
int index = indexOf(id);
if (index < 0) {
throw new IllegalArgumentException("There is no chaos rule " +
id);
}
checkRoute(definition);
- long now = TimeSource.currentTimeMillis();
ChaosRule changed = new ChaosRule(definition, id,
rules.get(index).version + 1, seed, now, maxDuration);
changed.recentFaults = recentFaults;
changed.clockStarted = camelContext.isStarted();
+ if (exploration != null && exploration.isRunning()) {
+ checkStartsBeforeDeadline(changed, exploration);
+ endByDeadline(changed, exploration, now);
+ }
ChaosRule old = rules.set(index, changed);
old.removed(State.REPLACED, now);
remember(old);
@@ -583,6 +604,12 @@ public final class ChaosManager {
evictedLostAtDlq = 0;
routes.clear();
recentFaults.clear();
+ if (exploration != null && exploration.isRunning()) {
+ // a scenario loaded while it runs: the counts start again
from zero, and so does its baseline
+ exploration.reportReset();
+ } else {
+ exploration = null;
+ }
LOG.info("Chaos report reset");
return count;
}
@@ -600,7 +627,11 @@ public final class ChaosManager {
long lost;
int pending;
long lostAtDlq;
+ ChaosExploration explored;
synchronized (lock) {
+ // the status reads the report every second, which finishes an
exploration whose time is up
+ checkDeadline(TimeSource.currentTimeMillis());
+ explored = exploration;
current = new ArrayList<>(rules);
removed = new ArrayList<>(history);
unmet = evictedUnmet;
@@ -613,9 +644,107 @@ public final class ChaosManager {
List<ChaosRule> all = new ArrayList<>(current);
all.addAll(removed);
List<ChaosAdvisor.RouteTried> tried = ChaosAdvisor.tried(all,
suggestedKinds(now), now);
- return new ChaosReport(
+ ChaosReport report = new ChaosReport(
seed, seedFrom, now, current, removed, counts, unmet, lost,
pending,
lostAtDlq, recentFaults.latestFirst(),
ChaosTargets.reach(camelContext), tried);
+ if (explored != null) {
+ synchronized (lock) {
+ report.exploration(explored.toJson(now), explored.text(now));
+ }
+ }
+ return report;
+ }
+
+ /**
+ * Starts an exploration (CAMEL-26773): an agent tries chaos rules for the
time it was given. While it runs, a rule
+ * ends at the deadline at the latest, and at the deadline the rules are
removed and the exploration is finished.
+ * The report has its journal: the baseline (the counts per route now) and
the rules added while it runs.
+ *
+ * @param budget how long, such as 10m (10 minutes when
empty, at most 2 hours)
+ * @param focus what the agent explores, such as a
route (optional)
+ * @return the exploration, as in the report
+ * @throws IllegalArgumentException when the budget is not a time, or too
short or too long
+ * @throws IllegalStateException when an exploration is running
+ */
+ public Map<String, Object> startExploration(String budget, String focus) {
+ long millis;
+ try {
+ millis = budget == null || budget.isBlank()
+ ? ChaosExploration.DEFAULT_BUDGET :
TimeUtils.toMilliSeconds(budget.strip());
+ } catch (RuntimeException e) {
+ throw new IllegalArgumentException("The budget of the exploration
is not a time: " + budget);
+ }
+ if (millis < ChaosExploration.MIN_BUDGET || millis >
ChaosExploration.MAX_BUDGET) {
+ throw new IllegalArgumentException(
+ "The budget of the exploration must be between 10s and 2h,
was: " + budget);
+ }
+ synchronized (lock) {
+ long now = TimeSource.currentTimeMillis();
+ checkDeadline(now);
+ if (exploration != null && exploration.isRunning()) {
+ throw new IllegalStateException(
+ "An exploration is running, for another " +
ChaosRuleParser.printTime(exploration.deadline - now)
+ + ": finish it first");
+ }
+ exploration = new ChaosExploration(now, millis, focus);
+ update();
+ LOG.info("Chaos exploration started: {}{}",
ChaosRuleParser.printTime(millis),
+ exploration.focus != null ? " (focus: " +
exploration.focus + ")" : "");
+ return exploration.toJson(now);
+ }
+ }
+
+ /**
+ * Finishes the running exploration before its deadline: its rules are
removed, and its journal stays in the report
+ * until the next one starts or the report is reset.
+ *
+ * @return the exploration, as in the report
+ * @throws IllegalStateException when no exploration is running
+ */
+ public Map<String, Object> finishExploration() {
+ synchronized (lock) {
+ long now = TimeSource.currentTimeMillis();
+ checkDeadline(now);
+ if (exploration == null || !exploration.isRunning()) {
+ throw new IllegalStateException("No exploration is running");
+ }
+ finish(now, ChaosExploration.Finish.FINISHED);
+ return exploration.toJson(now);
+ }
+ }
+
+ // finishes the running exploration whose time is up; with the lock
+ private void checkDeadline(long now) {
+ if (exploration != null && exploration.isRunning() && now >=
exploration.deadline) {
+ finish(now, ChaosExploration.Finish.DEADLINE);
+ }
+ }
+
+ private void finish(long now, ChaosExploration.Finish by) {
+ int removedRules = clearRules();
+ exploration.finished(now, by, removedRules);
+ // no rule is left and the exploration is done: the counting stops
+ update();
+ LOG.info("Chaos exploration {}: {} rules tried, {} removed",
+ by == ChaosExploration.Finish.DEADLINE ? "finished at its
deadline" : "finished",
+ exploration.ruleCount(), removedRules);
+ }
+
+ private static void checkStartsBeforeDeadline(ChaosRule rule,
ChaosExploration running) {
+ if (rule.startsAt >= running.deadline) {
+ throw new IllegalArgumentException(
+ "The chaos rule would start after the end of the
exploration, in "
+ +
ChaosRuleParser.printTime(running.deadline - rule.addedAt)
+ + ": " + rule.text);
+ }
+ }
+
+ // a rule added while an exploration runs ends at its deadline at the
latest; its first rule takes the baseline
+ private void endByDeadline(ChaosRule rule, ChaosExploration running, long
now) {
+ if (rule.endsAt > running.deadline) {
+ rule.endsAt = running.deadline;
+ }
+ running.ruleAdded(rule.id, new TreeMap<>(routes), now);
}
// the kinds of fault the suggestions offer per route, read again at most
every 10 seconds (the report is read
@@ -639,6 +768,13 @@ public final class ChaosManager {
// -- for the hooks, the event notifier and the plugin
+ /** The rules there are now. */
+ List<ChaosRule> rules() {
+ synchronized (lock) {
+ return List.copyOf(rules);
+ }
+ }
+
RuleSet ruleSet() {
return ruleSet;
}
@@ -1016,7 +1152,8 @@ public final class ChaosManager {
lifecycle.schedule(rule);
}
}
- boolean any = !rules.isEmpty();
+ // an exploration counts from its start, for the baseline before its
first rule (CAMEL-26773)
+ boolean any = !rules.isEmpty() || exploration != null &&
exploration.isRunning();
if (any && !counting) {
used = true;
counting = true;
diff --git
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosReport.java
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosReport.java
index 5371d6c484b6..d2641bf44ba4 100644
---
a/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosReport.java
+++
b/components/camel-chaos/src/main/java/org/apache/camel/component/chaos/ChaosReport.java
@@ -72,6 +72,9 @@ public final class ChaosReport {
private final List<ChaosRecentFaults.Entry> recentFaults;
// what has been tried per route, and what not yet (CAMEL-26728)
private final List<ChaosAdvisor.RouteTried> tried;
+ // the exploration running now or the last one (CAMEL-26773), as JSON and
text, or null
+ private JsonObject exploration;
+ private String explorationText;
ChaosReport(long seed, String seedFrom, long now, List<ChaosRule> current,
List<ChaosRule> removed,
Map<String, RouteCounts> counts, int evictedUnmet, long
evictedLost, int evictedPending,
@@ -146,6 +149,12 @@ public final class ChaosReport {
this.pending = anyPending || evictedPending > 0;
}
+ /** The journal of the exploration running now or the last one, which the
report then has. */
+ void exploration(JsonObject json, String text) {
+ this.exploration = json;
+ this.explorationText = text;
+ }
+
// the number of an id such as r12, for the order of the rules
private static int idNumber(String id) {
try {
@@ -273,6 +282,9 @@ public final class ChaosReport {
}
sb.append("), handled ").append(r.handled).append(", redeliveries
").append(r.redeliveries);
}
+ if (explorationText != null) {
+ sb.append(explorationText);
+ }
if (!tried.isEmpty()) {
sb.append("\n tried, per route:");
for (ChaosAdvisor.RouteTried r : tried) {
@@ -469,6 +481,9 @@ public final class ChaosReport {
root.put("routes", routeArray);
root.put("recentFaults", recentFaultsJson());
root.put("tried", triedJson());
+ if (exploration != null) {
+ root.put("exploration", exploration);
+ }
return root;
}
diff --git
a/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosExplorationTest.java
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosExplorationTest.java
new file mode 100644
index 000000000000..5a17ba5fe034
--- /dev/null
+++
b/components/camel-chaos/src/test/java/org/apache/camel/component/chaos/ChaosExplorationTest.java
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.chaos;
+
+import java.util.List;
+import java.util.Map;
+
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.util.json.JsonObject;
+import org.junit.jupiter.api.Test;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/**
+ * The journal of an exploration (CAMEL-26773): the baseline and the rules
tried, and a deadline that camel-chaos
+ * enforces, on the clock of TimeSource.
+ */
+class ChaosExplorationTest extends ChaosTestSupport {
+
+ private FakeClock clock;
+
+ @Override
+ protected void setupResources() throws Exception {
+ super.setupResources();
+ clock = FakeClock.install();
+ }
+
+ @Override
+ protected void cleanupResources() throws Exception {
+ clock.close();
+ super.cleanupResources();
+ }
+
+ @Override
+ protected RouteBuilder createRouteBuilder() {
+ return new RouteBuilder() {
+ @Override
+ public void configure() {
+ from("direct:orders").routeId("orders").to("mock:shipped");
+ }
+ };
+ }
+
+ private JsonObject exploration() {
+ return (JsonObject) chaos().report().toJson().get("exploration");
+ }
+
+ private static List<Object> list(JsonObject jo, String key) {
+ return jo.getCollection(key);
+ }
+
+ @Test
+ void theBaselineAndTheRulesTriedUntilTheDeadline() {
+ Map<String, Object> started = chaos().startExploration("5m", "orders");
+ assertThat(started).containsEntry("running",
true).containsEntry("budget", 300_000L)
+ .containsEntry("focus", "orders").containsEntry("timeLeft",
300_000L);
+ assertThat(chaos().report().toText()).contains("baseline: taken when
the first rule is added");
+
+ // the run without faults before the first rule is the baseline, so
what fails there is not blamed on chaos
+ template.sendBody("direct:orders", "before");
+ clock.advance(30_000);
+ String r1 = chaos().addRule("endpoint=mock:shipped error until=8m");
+ String r2 = chaos().addRule("endpoint=mock:shipped delay
duration=10ms");
+ // a rule ends at the deadline at the latest
+ long deadline = (long) started.get("deadline");
+ assertThat(chaos().rules()).allMatch(r -> r.endsAt <= deadline);
+
+ clock.advance(120_000);
+ JsonObject running = exploration();
+ assertThat(running.getLong("timeLeft")).isEqualTo(150_000);
+ assertThat(list(running, "rules")).containsExactly(r1, r2);
+ JsonObject orders = (JsonObject)
running.getCollection("baseline").iterator().next();
+ assertThat(orders.getString("route")).isEqualTo("orders");
+ assertThat(orders.getLong("completed")).isEqualTo(1);
+ assertThat(chaos().report().toText()).contains("exploration: running,
2m30s left of 5m0s (focus: orders)",
+ "rules tried: r1, r2", "baseline: orders 1 completed, 0 failed
in the 30s before the first rule");
+
+ // the time is up: the next look at the report finishes it, and the
rules are removed
+ clock.advance(150_000);
+ JsonObject finished = exploration();
+ assertThat(finished.getBoolean("running")).isFalse();
+ assertThat(finished.getString("finishedBy")).isEqualTo("deadline");
+ assertThat(finished.getLong("removed")).isEqualTo(2);
+ assertThat(finished.containsKey("timeLeft")).isFalse();
+ assertThat(chaos().rules()).isEmpty();
+ assertThat(chaos().report().toText()).contains("exploration: finished
at the deadline after 5m0s of 5m0s,"
+ + " 2 rules removed");
+ }
+
+ @Test
+ void finishedBeforeTheDeadline() {
+ chaos().startExploration(null, null);
+ chaos().addRule("endpoint=mock:shipped error");
+ clock.advance(60_000);
+
+ Map<String, Object> finished = chaos().finishExploration();
+
+ assertThat(finished).containsEntry("running",
false).containsEntry("finishedBy", "finished")
+ .containsEntry("budget",
ChaosExploration.DEFAULT_BUDGET).containsEntry("removed", 1);
+ assertThat(chaos().rules()).isEmpty();
+ // the journal stays in the report, and rules added now are not part
of it
+ chaos().addRule("endpoint=mock:shipped drop");
+ assertThat(list(exploration(), "rules")).containsExactly("r1");
+ assertThatThrownBy(() ->
chaos().finishExploration()).isInstanceOf(IllegalStateException.class)
+ .hasMessage("No exploration is running");
+ }
+
+ @Test
+ void oneAtATimeAndWithinItsTime() {
+ assertThatThrownBy(() -> chaos().startExploration("3h",
null)).isInstanceOf(IllegalArgumentException.class)
+ .hasMessageContaining("between 10s and 2h");
+ assertThatThrownBy(() -> chaos().startExploration("1s",
null)).hasMessageContaining("between 10s and 2h");
+ assertThatThrownBy(() -> chaos().startExploration("soon",
null)).hasMessageContaining("not a time: soon");
+
+ chaos().startExploration("10m", null);
+ assertThatThrownBy(() -> chaos().startExploration("10m",
null)).isInstanceOf(IllegalStateException.class)
+ .hasMessage("An exploration is running, for another 10m0s:
finish it first");
+ // a rule that would only start after the deadline is refused, and
none of the rules is added
+ chaos().finishExploration();
+ chaos().startExploration("1m", null);
+ assertThatThrownBy(() -> chaos().addRules("endpoint=mock:shipped
error; endpoint=mock:shipped drop from=2m"))
+
.isInstanceOf(IllegalArgumentException.class).hasMessageContaining("start after
the end of the exploration");
+ assertThat(chaos().rules()).isEmpty();
+ // a new exploration once the time of the last one is up
+ clock.advance(60_000);
+ assertThat(chaos().startExploration("1m",
null)).containsEntry("running", true);
+ }
+
+ @Test
+ void aChangedRuleEndsAtTheDeadlineToo() {
+ chaos().startExploration("2m", null);
+ String id = chaos().addRule("endpoint=mock:shipped error until=1m");
+ clock.advance(90_000);
+
+ chaos().changeRule(id, "endpoint=mock:shipped error until=5m");
+
+ ChaosRule changed = chaos().rules().get(0);
+
assertThat(changed.endsAt).isEqualTo(exploration().getLong("deadline"));
+ assertThat(list(exploration(), "rules")).containsExactly(id);
+ }
+
+ @Test
+ void aRuleChangedAfterTheDeadlineIsGone() {
+ chaos().startExploration("1m", null);
+ String id = chaos().addRule("endpoint=mock:shipped error");
+ clock.advance(60_000);
+
+ // the change is the first to see the time is up: the exploration
removes its rules before the change looks
+ assertThatThrownBy(() -> chaos().changeRule(id, "endpoint=mock:shipped
drop"))
+
.isInstanceOf(IllegalArgumentException.class).hasMessage("There is no chaos
rule " + id);
+
assertThat(exploration().getString("finishedBy")).isEqualTo("deadline");
+ }
+
+ @Test
+ void theCountingStopsWithTheExploration() {
+ chaos().startExploration("1m", null);
+ assertThat(chaos().isCounting()).isTrue();
+ chaos().addRule("endpoint=mock:shipped error");
+
+ chaos().finishExploration();
+ assertThat(chaos().isCounting()).isFalse();
+
+ // and at the deadline
+ chaos().startExploration("1m", null);
+ chaos().addRule("endpoint=mock:shipped error");
+ clock.advance(60_000);
+ chaos().report();
+ assertThat(chaos().isCounting()).isFalse();
+ }
+
+ @Test
+ void aScenarioLoadedWhileItRunsStartsTheCountsAgain() {
+ template.sendBody("direct:orders", "before");
+ chaos().startExploration("10m", null);
+
+ List<String> ids = chaos().loadScenario("outage",
"endpoint=mock:shipped error until=1m");
+
+ JsonObject running = exploration();
+ assertThat(running.getBoolean("running")).isTrue();
+ assertThat(running.getBoolean("baselineReset")).isTrue();
+ assertThat(list(running, "baseline")).isEmpty();
+ assertThat(list(running, "rules")).containsExactlyElementsOf(ids);
+
+ // a finished exploration goes with a reset of the report
+ chaos().finishExploration();
+ chaos().reset();
+ assertThat(exploration()).isNull();
+ }
+}
diff --git
a/dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/ChaosAction.java
b/dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/ChaosAction.java
index 07e1fc7fcb2b..554ac4415f43 100644
---
a/dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/ChaosAction.java
+++
b/dsl/camel-cli-connector/src/main/java/org/apache/camel/cli/connector/ChaosAction.java
@@ -141,8 +141,19 @@ final class ChaosAction {
if (reset) {
answer.put("reset", true);
}
+ } else if ("explore-start".equals(command)) {
+ // an exploration (CAMEL-26773): camel-chaos keeps its journal
and ends its rules at the deadline
+ @SuppressWarnings("unchecked")
+ Map<String, Object> started = (Map<String, Object>) call(type,
chaos, "startExploration",
+ root.getString("budget"), root.getString("focus"));
+ answer.putAll(started);
+ } else if ("explore-finish".equals(command)) {
+ @SuppressWarnings("unchecked")
+ Map<String, Object> finished = (Map<String, Object>)
call(type, chaos, "finishExploration");
+ answer.putAll(finished);
} else {
- output.error("Unknown chaos command: " + command + " (add,
check, change, remove, clear, load or suggest)");
+ output.error("Unknown chaos command: " + command
+ + " (add, check, change, remove, clear, load,
suggest, explore-start or explore-finish)");
return;
}
} catch (IllegalArgumentException | IllegalStateException e) {
diff --git
a/dsl/camel-cli-connector/src/test/java/org/apache/camel/cli/connector/LocalCliConnectorChaosActionTest.java
b/dsl/camel-cli-connector/src/test/java/org/apache/camel/cli/connector/LocalCliConnectorChaosActionTest.java
index 5e5c827e7a8b..73b090e315a0 100644
---
a/dsl/camel-cli-connector/src/test/java/org/apache/camel/cli/connector/LocalCliConnectorChaosActionTest.java
+++
b/dsl/camel-cli-connector/src/test/java/org/apache/camel/cli/connector/LocalCliConnectorChaosActionTest.java
@@ -222,6 +222,32 @@ class LocalCliConnectorChaosActionTest {
assertThat(reportRules()).isEmpty();
}
+ @Test
+ void anExplorationStartsAndFinishes() throws Exception {
+ start(null, Map.of("camel.chaos.enabled", "true"));
+ JsonObject action = new JsonObject();
+ action.put("action", "chaos");
+ action.put("command", "explore-start");
+ action.put("budget", "5m");
+ action.put("focus", "start");
+
+ CliActionOutcome started = CliActionOutcome.dispatch(connector,
action).get(10, TimeUnit.SECONDS);
+ assertThat(started.error()).isNull();
+ assertThat(started.result().getBoolean("running")).isTrue();
+ assertThat(started.result().getLong("budget")).isEqualTo(300_000L);
+ assertThat(started.result().getString("focus")).isEqualTo("start");
+ // one at a time: the reason goes to the tool
+ assertThat(CliActionOutcome.dispatch(connector, action).get(10,
TimeUnit.SECONDS).error())
+ .startsWith("An exploration is running");
+
+ assertThat(chaos("add", null, "endpoint=mock:target
error").error()).isNull();
+ CliActionOutcome finished = chaos("explore-finish", null, null);
+ assertThat(finished.error()).isNull();
+
assertThat(finished.result().getString("finishedBy")).isEqualTo("finished");
+ assertThat(finished.result().getLong("removed")).isEqualTo(1L);
+ assertThat(chaos("explore-finish", null, null).error()).isEqualTo("No
exploration is running");
+ }
+
@Test
void rulesAreAddedTogetherWithWarningsForTheOnesThatMatchNothing() throws
Exception {
start(null, Map.of("camel.chaos.enabled", "true"));
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaos.java
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaos.java
index 4a95b0894029..34c710feb859 100644
---
a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaos.java
+++
b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaos.java
@@ -304,6 +304,50 @@ public class ListChaos extends ProcessWatchCommand {
.with(r -> r.getStringOrDefault("exchangeId",
"")))));
}
printTried(report);
+ printExploration(report);
+ }
+
+ /**
+ * The exploration running now or the last one (CAMEL-26773): its time,
the rules tried, and the baseline before its
+ * first rule.
+ */
+ private void printExploration(JsonObject report) {
+ JsonObject exploration = report.getMap("exploration");
+ if (exploration == null) {
+ return;
+ }
+ StringBuilder sb = new StringBuilder(" exploration: ");
+ long budget = exploration.getLongOrDefault("budget", 0);
+ if (exploration.getBooleanOrDefault("running", false)) {
+ sb.append("running,
").append(TimeUtils.printDuration(exploration.getLongOrDefault("timeLeft", 0)))
+ .append(" left of
").append(TimeUtils.printDuration(budget));
+ } else {
+ sb.append("deadline".equals(exploration.getString("finishedBy")) ?
"finished at the deadline" : "finished")
+ .append("
(").append(TimeUtils.printDuration(budget)).append(')');
+ }
+ String focus = exploration.getString("focus");
+ if (focus != null) {
+ sb.append(", focus: ").append(focus);
+ }
+ printer().println(sb.toString());
+ List<Object> rules = exploration.getCollection("rules");
+ printer().println(" rules tried: "
+ + (rules == null || rules.isEmpty()
+ ? "none yet" :
rules.stream().map(String::valueOf).collect(Collectors.joining(", "))));
+ if (!exploration.containsKey("baselineAt")) {
+ printer().println(" baseline: taken when the first rule is
added");
+ return;
+ }
+ StringJoiner sj = new StringJoiner(", ");
+ List<Object> baseline = exploration.getCollection("baseline");
+ if (baseline != null) {
+ for (Object o : baseline) {
+ JsonObject route = (JsonObject) o;
+ sj.add(route.getString("route") + " " +
route.getLongOrDefault("completed", 0) + " completed, "
+ + route.getLongOrDefault("failed", 0) + " failed");
+ }
+ }
+ printer().println(" baseline (before the first rule): " +
(sj.length() == 0 ? "no exchanges" : sj));
}
/**
diff --git
a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaosTest.java
b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaosTest.java
index 1bb84ce6b76c..56107e222941 100644
---
a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaosTest.java
+++
b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/process/ListChaosTest.java
@@ -186,6 +186,31 @@ class ListChaosTest extends ProcessCommandTestSupport {
assertThat(out).contains("a stub does not read a kept message again,
as the component it stands in for would");
}
+ @Test
+ void theExplorationInTheReport() throws Exception {
+ JsonObject chaos = chaosOn();
+ JsonObject exploration = new JsonObject();
+ exploration.put("running", true);
+ exploration.put("budget", 600_000L);
+ exploration.put("timeLeft", 372_000L);
+ exploration.put("focus", "orders");
+ exploration.put("rules", new JsonArray(List.of("r1", "r2")));
+ exploration.put("baselineAt", 1_700_000_030_000L);
+ JsonObject orders = new JsonObject();
+ orders.put("route", "orders");
+ orders.put("completed", 30L);
+ orders.put("failed", 0L);
+ exploration.put("baseline", new JsonArray(List.of(orders)));
+ ((JsonObject) chaos.get("report")).put("exploration", exploration);
+ writeStatus(chaos);
+
+ assertThat(run("--report")).isZero();
+
+ String out = printer.getStdout();
+ assertThat(out).contains("exploration: running, 6m12s left of 10m0s,
focus: orders",
+ "rules tried: r1, r2", "baseline (before the first rule):
orders 30 completed, 0 failed");
+ }
+
@Test
void whatHasBeenTriedInTheReport() throws Exception {
JsonObject chaos = chaosOn();
