This is an automated email from the ASF dual-hosted git repository. davsclaus pushed a commit to branch CAMEL-26731-chaos-mcp-load-suggest in repository https://gitbox.apache.org/repos/asf/camel.git
commit 43100a440607ee3b0008d940467d5e916a63b75d Author: Claus Ibsen <[email protected]> AuthorDate: Sun Nov 8 11:30:34 2026 +0100 CAMEL-26731: camel-jbang - MCP and AI tools: load a chaos scenario, and what to try (chaos suggest) Two chaos operations of the CLI connector had no tool: load (camel cmd chaos-load) and suggest (camel cmd chaos-suggest). The shared ToolRegistry gets chaos_suggest (read only: the rules suggested per route from what the routes do, with why and whether the kind was tried) and chaos_load (destructive: the rules are replaced by those of the scenario text, all or none, and the report starts again). camel ask asks before a load and shows the scenario (ToolConfirms.chaosLoad). The camel-jbang MCP server has them as camel_runtime_chaos_suggest and camel_runtime_chaos_load in the chaos tool group (with chaos on), so read-only access offers suggest and only admin offers load. Over the HTTP server of camel run --mcp chaos still only lists and reports: suggest is read only but an action of the connector, not of the console, so the HTTP filter now names list and report. Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Claus Ibsen <[email protected]> --- .../org/apache/camel/catalog/docs/chaos.adoc | 16 +++--- components/camel-chaos/src/main/docs/chaos.adoc | 16 +++--- design/chaos.adoc | 6 +-- .../modules/ROOT/pages/camel-jbang-mcp.adoc | 23 +++++++-- .../camel/dsl/jbang/core/commands/AskTools.java | 6 ++- .../dsl/jbang/core/commands/ai/ToolConfirms.java | 16 ++++++ .../dsl/jbang/core/commands/ai/ToolGroups.java | 10 ++-- .../dsl/jbang/core/commands/ai/ToolRegistry.java | 60 +++++++++++++++++++--- .../dsl/jbang/core/commands/AskToolsTest.java | 8 ++- .../jbang/core/commands/ai/ToolConfirmsTest.java | 14 +++++ .../dsl/jbang/core/commands/ai/ToolGroupsTest.java | 11 ++-- .../jbang/core/commands/ai/ToolRegistryTest.java | 14 +++-- .../core/commands/mcp/JbangDevMcpServerTest.java | 3 +- .../dsl/jbang/core/commands/mcp/RuntimeTools.java | 41 ++++++++++++++- .../mcp/RuntimeToolsChaosSecurityTest.java | 14 +++-- .../core/commands/mcp/RuntimeToolsChaosTest.java | 45 ++++++++++++++++ .../jbang/core/commands/mcp/RuntimeToolsTest.java | 3 +- 17 files changed, 253 insertions(+), 53 deletions(-) diff --git a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc index 3f60d53e8671..2c04c83f4875 100644 --- a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc +++ b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/chaos.adoc @@ -258,19 +258,21 @@ In JSON the report has `tried`: per `route` its `kinds` (`kind`, `rules`, `fired The Camel MCP server (`camel mcp`, see xref:manual::camel-jbang-mcp.adoc[Camel MCP server]) has the same operations as tools, for an AI agent that tries a route against failures: `camel_runtime_chaos_list`, `camel_runtime_chaos_report`, -`camel_runtime_chaos_check` (the dry run), `camel_runtime_chaos_add`, `camel_runtime_chaos_change`, -`camel_runtime_chaos_remove` and `camel_runtime_chaos_clear`. +`camel_runtime_chaos_suggest` (what to try), `camel_runtime_chaos_check` (the dry run), `camel_runtime_chaos_add`, +`camel_runtime_chaos_change`, `camel_runtime_chaos_remove`, `camel_runtime_chaos_clear` and +`camel_runtime_chaos_load` (a scenario, given as the text of a `.chaos` file). Without `nameOrPid` they pick the integration as the commands do: the only one that runs, or else the only one with chaos switched on, and otherwise they fail with the list of the running integrations. They take the same one-line rules: adding, changing, removing and clearing go through the same `chaos` action of the CLI connector, and the list and the report read the console from the status of the application. So chaos must be switched on, and a rule that is not valid is refused with the reason; `camel_runtime_chaos_clear` takes `reset=true` as `chaos-clear --reset` does. `camel_runtime_chaos_add` takes several rules separated by `;`, and the add and change -tools return the warnings of a rule that matches nothing, so an agent can fix a typo with a change. Adding and changing a rule are marked destructive, so a -client that honours `destructiveHint` asks before it runs them, and the `read-write` access level of the MCP security -layer does not offer them; the list, the report and the check are read-only, so an agent can check a rule it made from +tools return the warnings of a rule that matches nothing, so an agent can fix a typo with a change. Adding and changing a rule, and loading a scenario, are marked +destructive, so a client that honours `destructiveHint` asks before it runs them, and the `read-write` access level of the MCP security +layer does not offer them; the list, the report, what to try and the check are read-only, so an agent can check a rule it made from a loose request (what it would hit, in plain words) and show it before it asks to add it. `camel ask` has the same tools, and asks before it -adds or changes a rule. The development MCP tools that `camel run --mcp` (`camel.jbang.mcp=true`) serves over HTTP have -only the list and the report (not the check either: the integration parses the rule, and `exception=` loads a class). +adds or changes a rule, or loads a scenario. The development MCP tools that `camel run --mcp` (`camel.jbang.mcp=true`) +serves over HTTP have only the list and the report (not the check either: the integration parses the rule, and +`exception=` loads a class; nor what to try or a scenario). == Scenarios diff --git a/components/camel-chaos/src/main/docs/chaos.adoc b/components/camel-chaos/src/main/docs/chaos.adoc index 3f60d53e8671..2c04c83f4875 100644 --- a/components/camel-chaos/src/main/docs/chaos.adoc +++ b/components/camel-chaos/src/main/docs/chaos.adoc @@ -258,19 +258,21 @@ In JSON the report has `tried`: per `route` its `kinds` (`kind`, `rules`, `fired The Camel MCP server (`camel mcp`, see xref:manual::camel-jbang-mcp.adoc[Camel MCP server]) has the same operations as tools, for an AI agent that tries a route against failures: `camel_runtime_chaos_list`, `camel_runtime_chaos_report`, -`camel_runtime_chaos_check` (the dry run), `camel_runtime_chaos_add`, `camel_runtime_chaos_change`, -`camel_runtime_chaos_remove` and `camel_runtime_chaos_clear`. +`camel_runtime_chaos_suggest` (what to try), `camel_runtime_chaos_check` (the dry run), `camel_runtime_chaos_add`, +`camel_runtime_chaos_change`, `camel_runtime_chaos_remove`, `camel_runtime_chaos_clear` and +`camel_runtime_chaos_load` (a scenario, given as the text of a `.chaos` file). Without `nameOrPid` they pick the integration as the commands do: the only one that runs, or else the only one with chaos switched on, and otherwise they fail with the list of the running integrations. They take the same one-line rules: adding, changing, removing and clearing go through the same `chaos` action of the CLI connector, and the list and the report read the console from the status of the application. So chaos must be switched on, and a rule that is not valid is refused with the reason; `camel_runtime_chaos_clear` takes `reset=true` as `chaos-clear --reset` does. `camel_runtime_chaos_add` takes several rules separated by `;`, and the add and change -tools return the warnings of a rule that matches nothing, so an agent can fix a typo with a change. Adding and changing a rule are marked destructive, so a -client that honours `destructiveHint` asks before it runs them, and the `read-write` access level of the MCP security -layer does not offer them; the list, the report and the check are read-only, so an agent can check a rule it made from +tools return the warnings of a rule that matches nothing, so an agent can fix a typo with a change. Adding and changing a rule, and loading a scenario, are marked +destructive, so a client that honours `destructiveHint` asks before it runs them, and the `read-write` access level of the MCP security +layer does not offer them; the list, the report, what to try and the check are read-only, so an agent can check a rule it made from a loose request (what it would hit, in plain words) and show it before it asks to add it. `camel ask` has the same tools, and asks before it -adds or changes a rule. The development MCP tools that `camel run --mcp` (`camel.jbang.mcp=true`) serves over HTTP have -only the list and the report (not the check either: the integration parses the rule, and `exception=` loads a class). +adds or changes a rule, or loads a scenario. The development MCP tools that `camel run --mcp` (`camel.jbang.mcp=true`) +serves over HTTP have only the list and the report (not the check either: the integration parses the rule, and +`exception=` loads a class; nor what to try or a scenario). == Scenarios diff --git a/design/chaos.adoc b/design/chaos.adoc index e18a62c8ae77..a65c18d17c4f 100644 --- a/design/chaos.adoc +++ b/design/chaos.adoc @@ -431,10 +431,10 @@ framework, so camel-test does not depend on camel-chaos; the rules come from the |=== |Tool |`readOnlyHint` |`destructiveHint` -|`camel_runtime_chaos_add`, `camel_runtime_chaos_change` |`false` |`true` +|`camel_runtime_chaos_add`, `camel_runtime_chaos_change`, `camel_runtime_chaos_load` (CAMEL-26731) |`false` |`true` |`camel_runtime_chaos_remove`, `camel_runtime_chaos_clear` |`false` |`false` -|`camel_runtime_chaos_list`, `camel_runtime_chaos_report`, `camel_runtime_chaos_check` (stage 2, CAMEL-26720) -|`true` |`false` +|`camel_runtime_chaos_list`, `camel_runtime_chaos_report`, `camel_runtime_chaos_check` (stage 2, CAMEL-26720), +`camel_runtime_chaos_suggest` (CAMEL-26731) |`true` |`false` |=== + With `camel.mcp.security.enabled=true` and `camel.mcp.security.access-level=read-write`, add and change are refused, diff --git a/docs/user-manual/modules/ROOT/pages/camel-jbang-mcp.adoc b/docs/user-manual/modules/ROOT/pages/camel-jbang-mcp.adoc index 9de8f7d3463f..bf85f8735035 100644 --- a/docs/user-manual/modules/ROOT/pages/camel-jbang-mcp.adoc +++ b/docs/user-manual/modules/ROOT/pages/camel-jbang-mcp.adoc @@ -1087,6 +1087,13 @@ that changes rules is refused. The list and the report read the chaos console fr | The report: the verdict and whether it is final, the seed, every version of every rule with its live state, the expectations, the counts per route and the last 20 faults. +| `camel_runtime_chaos_suggest` +| `true` | `false` +| What to try, as `camel cmd chaos-suggest` does: rules suggested per route from what the routes do, the most worth + trying first (such as an HTTP call without a circuit breaker, a dead letter channel or a hand-off over `seda`), each + with the route, the kind of fault, the `rule`, `why`, and `tried=true` when a rule of that kind has already faulted + the route. Nothing is added. + | `camel_runtime_chaos_check` | `true` | `false` | Checks a rule (`rule`), or several separated by `;`, without adding it, as `camel cmd chaos-add --dry-run` does: @@ -1111,16 +1118,22 @@ that changes rules is refused. The list and the report read the chaos console fr | `false` | `false` | Removes all rules, and returns how many. With `reset=true` the report starts again (no rules, route counts or verdict), for a new experiment, as `camel cmd chaos-clear --reset` does. + +| `camel_runtime_chaos_load` +| `false` | `true` +| Loads a scenario (`scenario`, the text of a `.chaos` file, and optionally its name `scenarioName`), as + `camel cmd chaos-load` does: the rules there are now are removed and the report starts again, then the rules of the + scenario are added, all or none. Returns the ids, each rule in plain words, and `warnings`. |=== -Adding and changing a rule are marked destructive: a rule makes messages fail, wait or get lost, and its +Adding and changing a rule, and loading a scenario, are marked destructive: a rule makes messages fail, wait or get lost, and its `exception=` can name any exception class. So a client that honours `destructiveHint` asks before it runs them, and the `read-write` access level of the security layer (`camel.mcp.security.access-level`) does not offer them, as it does -not offer `camel_runtime_route_control` or `camel_runtime_sql`; `read-only` offers the list, the report and the check. Removing +not offer `camel_runtime_route_control` or `camel_runtime_sql`; `read-only` offers the list, the report, what to try and the check. Removing and clearing only take faults away, and the report keeps the removed rules, so they are not destructive. `camel ask` -has the same tools (`chaos_add` and the others), and asks the user before it adds or changes a rule. The development -MCP tools that `camel run --mcp` serves over HTTP have only the list and the report: a rule is never added, changed, -removed or checked over the HTTP server of the application. +has the same tools (`chaos_add` and the others), and asks the user before it adds or changes a rule or loads a scenario (showing the rules of the scenario). The +development MCP tools that `camel run --mcp` serves over HTTP have only the list and the report: a rule is never added, +changed, removed, checked, suggested or loaded over the HTTP server of the application. ==== Tool groups for local models diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/AskTools.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/AskTools.java index 14de64363aa4..36c12a883903 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/AskTools.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/AskTools.java @@ -75,7 +75,9 @@ public class AskTools { Map.entry(ToolGroups.CHAOS_ADD_TOOL, "chaos_add"), Map.entry(ToolGroups.CHAOS_CHANGE_TOOL, "chaos_change"), Map.entry(ToolGroups.CHAOS_REMOVE_TOOL, "chaos_remove"), - Map.entry(ToolGroups.CHAOS_CLEAR_TOOL, "chaos_clear")); + Map.entry(ToolGroups.CHAOS_CLEAR_TOOL, "chaos_clear"), + Map.entry(ToolGroups.CHAOS_SUGGEST_TOOL, "chaos_suggest"), + Map.entry(ToolGroups.CHAOS_LOAD_TOOL, "chaos_load")); private static final String CLI_LIST_COMMANDS = "cli_list_commands"; private static final String CLI_COMMAND_HELP = "cli_command_help"; @@ -354,6 +356,8 @@ public class AskTools { args.get("name") != null ? args.get("name") : processName()); case "chaos_add", "chaos_change" -> ToolConfirms.chaos(name.substring("chaos_".length()), args.get("name") != null ? args.get("name") : processName(), args.get("id"), args.get("rule")); + case "chaos_load" -> ToolConfirms.chaosLoad(args.get("name") != null ? args.get("name") : processName(), + args.get("scenarioName"), args.get("scenario")); default -> null; }; } diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolConfirms.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolConfirms.java index 771b93ebcfd1..11941657f920 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolConfirms.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolConfirms.java @@ -315,6 +315,22 @@ public final class ToolConfirms { }; } + /** + * The call of loading a chaos scenario (chaos_load): the rules there are now are replaced by those of the scenario, + * shown as they are, and the report starts again. Null without a scenario, which the tool refuses. + */ + public static Call chaosLoad(String integration, String scenarioName, String scenario) { + if (scenario == null || scenario.isBlank()) { + return null; + } + String where = integration != null && !integration.isBlank() ? integration : "the selected integration"; + String name = scenarioName != null && !scenarioName.isBlank() ? " " + scenarioName.strip() : ""; + return new Call( + "load chaos scenario" + name + " into " + where, scenario.strip(), + "The rules of the integration are replaced by those of the scenario and the report starts again; while" + + " they are active, the integration's messages fail, wait or get lost"); + } + /** * Splits a text with several rules separated by {@code ;}, as in camel.chaos.rules; a {@code ;} inside quotes does * not split. diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolGroups.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolGroups.java index 8d402a6bb73c..76824c582e0f 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolGroups.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolGroups.java @@ -45,6 +45,8 @@ public final class ToolGroups { public static final String CHAOS_CHANGE_TOOL = "camel_runtime_chaos_change"; public static final String CHAOS_REMOVE_TOOL = "camel_runtime_chaos_remove"; public static final String CHAOS_CLEAR_TOOL = "camel_runtime_chaos_clear"; + public static final String CHAOS_SUGGEST_TOOL = "camel_runtime_chaos_suggest"; + public static final String CHAOS_LOAD_TOOL = "camel_runtime_chaos_load"; /** * A loaded group. @@ -139,8 +141,9 @@ public final class ToolGroups { case CHAOS -> groups.add(new Group( group, f.chaosOn() - ? List.of(CHAOS_LIST_TOOL, CHAOS_REPORT_TOOL, CHAOS_CHECK_TOOL, CHAOS_ADD_TOOL, - CHAOS_CHANGE_TOOL, CHAOS_REMOVE_TOOL, CHAOS_CLEAR_TOOL) + ? List.of(CHAOS_LIST_TOOL, CHAOS_REPORT_TOOL, CHAOS_SUGGEST_TOOL, CHAOS_CHECK_TOOL, + CHAOS_ADD_TOOL, CHAOS_CHANGE_TOOL, CHAOS_REMOVE_TOOL, CHAOS_CLEAR_TOOL, + CHAOS_LOAD_TOOL) : List.of(CHAOS_LIST_TOOL, CHAOS_REPORT_TOOL), describeChaos(f))); } @@ -238,7 +241,8 @@ public final class ToolGroups { return "Chaos: camel-chaos is on, so failures may be injected on purpose; " + CHAOS_LIST_TOOL + " and " + CHAOS_REPORT_TOOL + " show the rules and what they did; " + CHAOS_ADD_TOOL + " takes a rule such as endpoint=kafka:orders* error rate=0.1 until=5m, and " + CHAOS_CHECK_TOOL - + " says what it would hit without adding it."; + + " says what it would hit without adding it; " + CHAOS_SUGGEST_TOOL + + " suggests rules from what the routes do, and " + CHAOS_LOAD_TOOL + " loads a scenario."; } /** {@code Circuit breakers in routes a, b}, or without the routes when they are not known. */ diff --git a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolRegistry.java b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolRegistry.java index fceacc0efc98..ba3e838d2336 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolRegistry.java +++ b/dsl/camel-jbang/camel-jbang-core/src/main/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolRegistry.java @@ -81,9 +81,11 @@ public final class ToolRegistry { * over HTTP chaos only lists and reports, as its developer console does (design/chaos.adoc). */ public static List<ToolDescriptor> httpTools() { - // chaos_check is read-only, but the integration parses the rule, and exception= loads a class: off HTTP too + // chaos_check is read-only, but the integration parses the rule, and exception= loads a class; chaos_suggest + // is read-only, but an action of the CLI connector, not of the console: off HTTP too return TOOLS.stream() - .filter(t -> !t.name().startsWith("chaos_") || t.isReadOnly() && !"chaos_check".equals(t.name())) + .filter(t -> !t.name().startsWith("chaos_") || "chaos_list".equals(t.name()) + || "chaos_report".equals(t.name())) .toList(); } @@ -734,12 +736,13 @@ public final class ToolRegistry { + " with chaos switched on)"; /** - * The chaos tools, the seven operations of camel cmd chaos-add [--dry-run], chaos-change, chaos-remove, chaos-clear - * and camel get chaos [--report]. The list and the report read the chaos console in the status of the integration; - * the others send the chaos action of the CLI connector, which works only when chaos is switched on and answers a - * rule that is not valid with the reason of the parser of camel.chaos.rules. Adding and changing a rule is - * destructive: a rule makes messages fail, wait or get lost, and exception= can name any exception class. Removing - * and clearing only take faults away, and the report keeps the removed rules. + * The chaos tools, the nine operations of camel cmd chaos-add [--dry-run], chaos-change, chaos-remove, chaos-clear, + * chaos-load, chaos-suggest and camel get chaos [--report]. The list and the report read the chaos console in the + * status of the integration; the others send the chaos action of the CLI connector, which works only when chaos is + * switched on and answers a rule that is not valid with the reason of the parser of camel.chaos.rules. Adding and + * changing a rule, and loading a scenario, are destructive: a rule makes messages fail, wait or get lost, and + * exception= can name any exception class. Removing and clearing only take faults away, and the report keeps the + * removed rules. What to try (suggest) only reads the routes. */ private static void registerChaosTools() { register(tool("chaos_list", @@ -802,6 +805,47 @@ public final class ToolRegistry { return chaos(ctx, "check", null, rule); })); + register(tool("chaos_suggest", + "What to try (as camel cmd chaos-suggest): chaos rules suggested per route from what the routes do, " + + "the most worth trying first, such as an HTTP call without a circuit breaker, " + + "a dead letter channel or a hand-off to another route over seda. Each has the " + + "route, the kind of fault, the rule, why to try it, and tried=true when a rule " + + "of that kind has already faulted the route. Nothing is added: check a rule " + + "with chaos_check and add it with chaos_add.") + .param("name", "string", CHAOS_NAME, false) + .readOnly(true).destructive(false) + .executor((ctx, args) -> { + selectChaosProcess(ctx, args); + return chaos(ctx, "suggest", null, null); + })); + + register(tool("chaos_load", + "Load a chaos scenario into the integration (as camel cmd chaos-load): the rules there are now are " + + "removed and the report starts again, then the rules of the scenario are added, " + + "all or none (a line that is not valid is refused with the reason, and nothing " + + "changes). Returns the ids and each rule in plain words, and warnings for a rule " + + "that matches nothing in the integration now.") + .param("scenario", "string", + "The scenario, the text of a .chaos file: one rule per line, # comments, and optionally a " + + "seed=<number> line for the same sequence of faults each run", + true) + .param("scenarioName", "string", + "The name of the scenario in the report, such as outage (default scenario)", false) + .param("name", "string", CHAOS_NAME, false) + .readOnly(false).destructive(true) + .executor((ctx, args) -> { + String scenario = AuthoringTools.required(args, "scenario"); + String scenarioName = args.get("scenarioName"); + selectChaosProcess(ctx, args); + return ctx.executeAction("chaos", root -> { + root.put("command", "load"); + root.put("text", scenario); + if (scenarioName != null && !scenarioName.isBlank()) { + root.put("name", scenarioName); + } + }); + })); + register(tool("chaos_change", "Change a chaos rule of the integration: the rule with the id gets the new one-line rule and one more " + "version; the report keeps the version it replaces. Returns the id, the new " diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/AskToolsTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/AskToolsTest.java index 7c06a083e250..0020ea06cf3e 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/AskToolsTest.java +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/AskToolsTest.java @@ -232,7 +232,13 @@ class AskToolsTest { assertThat(change.title()).isEqualTo("change chaos rule r1 of the selected integration"); assertThat(change.target()).isEqualTo("endpoint=* drop"); // the others only read or take faults away - for (String name : List.of("chaos_check", "chaos_remove", "chaos_clear", "chaos_list", "chaos_report")) { + ToolConfirms.Call load = tools.confirmation("chaos_load", + Map.of("scenario", "# outage\nendpoint=http:* error\n", "scenarioName", "outage")); + assertThat(load.title()).isEqualTo("load chaos scenario outage into the selected integration"); + // the user sees the rules of the scenario + assertThat(load.target()).isEqualTo("# outage\nendpoint=http:* error"); + for (String name : List.of("chaos_check", "chaos_suggest", "chaos_remove", "chaos_clear", "chaos_list", + "chaos_report")) { assertThat(tools.confirmation(name, Map.of("id", "r1"))).as(name).isNull(); } } diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolConfirmsTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolConfirmsTest.java index f868d9b950e4..ebeb41c947ec 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolConfirmsTest.java +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolConfirmsTest.java @@ -285,6 +285,20 @@ class ToolConfirmsTest { .isEqualTo("browse an endpoint the integration does not have"); } + @Test + void loadingAChaosScenarioIsAskedWithItsRules() { + ToolConfirms.Call load = ToolConfirms.chaosLoad("orders", "outage", + "# the payment service is down\nseed=42\nendpoint=http:* error\n"); + assertThat(load.title()).isEqualTo("load chaos scenario outage into orders"); + assertThat(load.target()).isEqualTo("# the payment service is down\nseed=42\nendpoint=http:* error"); + assertThat(load.reason()).contains("replaced", "report starts again", "fail, wait or get lost"); + + assertThat(ToolConfirms.chaosLoad(null, null, "endpoint=* drop").title()) + .isEqualTo("load chaos scenario into the selected integration"); + // without a scenario the tool refuses the call, so there is nothing to ask + assertThat(ToolConfirms.chaosLoad("orders", "outage", " ")).isNull(); + } + @Test void addingOrChangingAChaosRuleIsAskedWithTheRule() { ToolConfirms.Call add = ToolConfirms.chaos("add", "orders", null, "endpoint=* error exception=java.sql.SQLException"); diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolGroupsTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolGroupsTest.java index da9c37b4302d..6b9776a79d95 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolGroupsTest.java +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolGroupsTest.java @@ -144,17 +144,18 @@ class ToolGroupsTest { } @Test - void chaosLoadsItsSevenToolsWhenOnAndOnlyTheReadsWhenOff() { + void chaosLoadsItsNineToolsWhenOnAndOnlyTheReadsWhenOff() { AppFeatures on = AppFeatures.fromStatus(new JsonObject(Map.of("chaos", Map.of("enabled", true)))); ToolGroups.Selection s = ToolGroups.select(on); assertEquals(List.of(ToolGroup.CHAOS), s.toolGroups()); - assertEquals(List.of("camel_runtime_chaos_list", "camel_runtime_chaos_report", "camel_runtime_chaos_check", - "camel_runtime_chaos_add", "camel_runtime_chaos_change", "camel_runtime_chaos_remove", - "camel_runtime_chaos_clear"), s.mcpTools()); + assertEquals(List.of("camel_runtime_chaos_list", "camel_runtime_chaos_report", "camel_runtime_chaos_suggest", + "camel_runtime_chaos_check", "camel_runtime_chaos_add", "camel_runtime_chaos_change", + "camel_runtime_chaos_remove", "camel_runtime_chaos_clear", "camel_runtime_chaos_load"), s.mcpTools()); assertEquals("Chaos: camel-chaos is on, so failures may be injected on purpose; camel_runtime_chaos_list and" + " camel_runtime_chaos_report show the rules and what they did; camel_runtime_chaos_add takes a rule" + " such as endpoint=kafka:orders* error rate=0.1 until=5m, and camel_runtime_chaos_check says what it" - + " would hit without adding it.", + + " would hit without adding it; camel_runtime_chaos_suggest suggests rules from what the routes do," + + " and camel_runtime_chaos_load loads a scenario.", s.guidance().get(0)); assertEquals("chaos|", s.fingerprint()); diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolRegistryTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolRegistryTest.java index 7db1a1e6744d..8fa9bb960510 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolRegistryTest.java +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/ai/ToolRegistryTest.java @@ -316,12 +316,12 @@ class ToolRegistryTest { @Test void theChaosToolsHaveTheMarksOfTheMcpTools() { // the same marks as camel_runtime_chaos_* (design/chaos.adoc, "Control") - for (String name : List.of("chaos_list", "chaos_report", "chaos_check")) { + for (String name : List.of("chaos_list", "chaos_report", "chaos_check", "chaos_suggest")) { ToolDescriptor t = ToolRegistry.findTool(name); assertTrue(t.isReadOnly(), name); assertFalse(t.isDestructive(), name); } - for (String name : List.of("chaos_add", "chaos_change")) { + for (String name : List.of("chaos_add", "chaos_change", "chaos_load")) { ToolDescriptor t = ToolRegistry.findTool(name); assertFalse(t.isReadOnly(), name); assertTrue(t.isDestructive(), name); @@ -346,6 +346,9 @@ class ToolRegistryTest { .getMessage()); assertEquals("id is required", assertThrows(ToolExecutionException.class, () -> ToolRegistry.execute("chaos_remove", new ToolContext(), Map.of())).getMessage()); + assertEquals("scenario is required", assertThrows(ToolExecutionException.class, + () -> ToolRegistry.execute("chaos_load", new ToolContext(), Map.of("scenarioName", "outage"))) + .getMessage()); } @Test @@ -382,9 +385,10 @@ class ToolRegistryTest { List<String> http = ToolRegistry.httpTools().stream().map(ToolDescriptor::name).toList(); List<String> all = ToolRegistry.allTools().stream().map(ToolDescriptor::name).toList(); List<String> left = all.stream().filter(n -> !http.contains(n)).toList(); - // only the chaos tools that change the rules, and the check (the integration parses the rule), stay off HTTP; - // the other tools keep what they offered - assertEquals(List.of("chaos_add", "chaos_check", "chaos_change", "chaos_remove", "chaos_clear"), left); + // only the chaos tools that change the rules, the check (the integration parses the rule) and what to try (an + // action of the CLI connector) stay off HTTP; the other tools keep what they offered + assertEquals(List.of("chaos_add", "chaos_check", "chaos_suggest", "chaos_load", "chaos_change", "chaos_remove", + "chaos_clear"), left); assertTrue(http.containsAll(List.of("chaos_list", "chaos_report", "stop_route", "execute_sql"))); } } diff --git a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/JbangDevMcpServerTest.java b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/JbangDevMcpServerTest.java index 98d0e3e8c750..ddbe695922b0 100644 --- a/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/JbangDevMcpServerTest.java +++ b/dsl/camel-jbang/camel-jbang-core/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/JbangDevMcpServerTest.java @@ -77,7 +77,8 @@ class JbangDevMcpServerTest { .contains(ToolRegistry.allTools().get(0).name()) // over HTTP chaos only lists and reports: a rule is never added, changed or removed over HTTP .contains("chaos_list", "chaos_report") - .doesNotContain("chaos_check", "chaos_add", "chaos_change", "chaos_remove", "chaos_clear"); + .doesNotContain("chaos_check", "chaos_suggest", "chaos_add", "chaos_change", "chaos_remove", + "chaos_clear", "chaos_load"); McpSchema.Tool parameterizedTool = client.listTools().tools().stream() .filter(t -> "select_process".equals(t.name())) diff --git a/dsl/camel-jbang/camel-jbang-mcp/src/main/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeTools.java b/dsl/camel-jbang/camel-jbang-mcp/src/main/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeTools.java index a4553d8ac4df..cadde20be252 100644 --- a/dsl/camel-jbang/camel-jbang-mcp/src/main/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeTools.java +++ b/dsl/camel-jbang/camel-jbang-mcp/src/main/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeTools.java @@ -1106,8 +1106,8 @@ public class RuntimeTools { } // ---- camel-chaos (the chaos tool group): the rules and the report, and the chaos action of the CLI connector ---- - // add and change are destructive (a rule makes messages fail, wait or get lost) and the clients ask first - // (ToolConfirms.chaos); remove and clear change the integration only back towards normal, and the report keeps + // add, change and load are destructive (a rule makes messages fail, wait or get lost) and the clients ask first + // (ToolConfirms.chaos and chaosLoad); suggest only reads the routes; remove and clear change the integration only back towards normal, and the report keeps // the removed rules, so they are not destructive, as trace start and stop are not @ToolGuardrails(input = McpArgumentGuardrail.class, output = McpArgumentGuardrail.class) @@ -1150,6 +1150,43 @@ public class RuntimeTools { return delegateChaos("chaos_check", nameOrPid, Map.of("rule", rule)); } + @ToolGuardrails(input = McpArgumentGuardrail.class, output = McpArgumentGuardrail.class) + @Tool(annotations = @Tool.Annotations(readOnlyHint = true, destructiveHint = false, openWorldHint = false), + description = """ + What to try (as camel cmd chaos-suggest): chaos rules suggested per route of the Camel application \ + from what the routes do, the most worth trying first, such as an HTTP call without a circuit \ + breaker, a dead letter channel or a hand-off to another route over seda. Each has the route, the \ + kind of fault, the rule, why to try it, and tried=true when a rule of that kind has already \ + faulted the route. Nothing is added: check a rule with camel_runtime_chaos_check and add it with \ + camel_runtime_chaos_add.""") + public JsonObject camel_runtime_chaos_suggest( + @ToolArg(description = CHAOS_NAME_OR_PID_DESC, required = false) String nameOrPid) { + return delegateChaos("chaos_suggest", nameOrPid, Map.of()); + } + + @ToolGuardrails(input = McpArgumentGuardrail.class, output = McpArgumentGuardrail.class) + @Tool(annotations = @Tool.Annotations(readOnlyHint = false, destructiveHint = true, openWorldHint = false), + description = """ + Load a chaos scenario into the Camel application (as camel cmd chaos-load): the rules there are \ + now are removed and the report starts again, then the rules of the scenario are added, all or \ + none (a line that is not valid is refused with the reason, and nothing changes). Returns the ids \ + and each rule in plain words, and warnings for a rule that matches nothing in the application \ + now.""") + public JsonObject camel_runtime_chaos_load( + @ToolArg(description = CHAOS_NAME_OR_PID_DESC, required = false) String nameOrPid, + @ToolArg(description = "The scenario, the text of a .chaos file: one rule per line, # comments, and" + + " optionally a seed=<number> line for the same sequence of faults each run") String scenario, + @ToolArg(description = "The name of the scenario in the report, such as outage (default scenario)", + required = false) String scenarioName) { + requireArg("scenario", scenario); + Map<String, String> args = new HashMap<>(); + args.put("scenario", scenario); + if (scenarioName != null) { + args.put("scenarioName", scenarioName); + } + return delegateChaos("chaos_load", nameOrPid, args); + } + @ToolGuardrails(input = McpArgumentGuardrail.class, output = McpArgumentGuardrail.class) @Tool(annotations = @Tool.Annotations(readOnlyHint = false, destructiveHint = true, openWorldHint = false), description = """ diff --git a/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsChaosSecurityTest.java b/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsChaosSecurityTest.java index b240abd3f67b..510069c74919 100644 --- a/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsChaosSecurityTest.java +++ b/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsChaosSecurityTest.java @@ -45,11 +45,13 @@ class RuntimeToolsChaosSecurityTest { static { MARKS.put("camel_runtime_chaos_list", new boolean[] { true, false }); MARKS.put("camel_runtime_chaos_report", new boolean[] { true, false }); + MARKS.put("camel_runtime_chaos_suggest", new boolean[] { true, false }); MARKS.put("camel_runtime_chaos_check", new boolean[] { true, false }); MARKS.put("camel_runtime_chaos_add", new boolean[] { false, true }); MARKS.put("camel_runtime_chaos_change", new boolean[] { false, true }); MARKS.put("camel_runtime_chaos_remove", new boolean[] { false, false }); MARKS.put("camel_runtime_chaos_clear", new boolean[] { false, false }); + MARKS.put("camel_runtime_chaos_load", new boolean[] { false, true }); } private static Tool.Annotations annotations(String name) { @@ -87,10 +89,12 @@ class RuntimeToolsChaosSecurityTest { void exactlyTheDestructiveToolsAreInToolConfirms() { MARKS.forEach((name, marks) -> { String command = name.substring("camel_runtime_chaos_".length()); - ToolConfirms.Call call = ToolConfirms.chaos(command, "shop", "r1", "endpoint=kafka:* error rate=0.1"); + ToolConfirms.Call call = "load".equals(command) + ? ToolConfirms.chaosLoad("shop", "outage", "endpoint=kafka:* error rate=0.1") + : ToolConfirms.chaos(command, "shop", "r1", "endpoint=kafka:* error rate=0.1"); assertThat(call != null).as(name + " is asked first").isEqualTo(marks[1]); if (call != null) { - // the user sees the rule + // the user sees the rule, or the rules of the scenario assertThat(call.target()).isEqualTo("endpoint=kafka:* error rate=0.1"); } }); @@ -99,10 +103,12 @@ class RuntimeToolsChaosSecurityTest { @Test void theAccessLevelsOfferTheToolsByTheirMarks() { assertThat(offered(McpSecurityConfig.AccessLevel.READ_ONLY)) - .containsExactly("camel_runtime_chaos_list", "camel_runtime_chaos_report", "camel_runtime_chaos_check"); + .containsExactly("camel_runtime_chaos_list", "camel_runtime_chaos_report", "camel_runtime_chaos_suggest", + "camel_runtime_chaos_check"); // as for route_control, sql, send and receive: no destructive tool at read-write assertThat(offered(McpSecurityConfig.AccessLevel.READ_WRITE)).containsExactly( - "camel_runtime_chaos_list", "camel_runtime_chaos_report", "camel_runtime_chaos_check", + "camel_runtime_chaos_list", "camel_runtime_chaos_report", "camel_runtime_chaos_suggest", + "camel_runtime_chaos_check", "camel_runtime_chaos_remove", "camel_runtime_chaos_clear"); assertThat(offered(McpSecurityConfig.AccessLevel.ADMIN)).containsExactlyElementsOf(MARKS.keySet()); // with the security layer on, the interceptor of @McpSecured audits each call (and redacts the answer when diff --git a/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsChaosTest.java b/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsChaosTest.java index 913206909627..e811bccfe4c1 100644 --- a/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsChaosTest.java +++ b/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsChaosTest.java @@ -157,6 +157,51 @@ class RuntimeToolsChaosTest { return template.send("direct:start", e -> e.getMessage().setBody("order")).getException(); } + @Test + void whatToTryAndAScenarioAgainstARunningIntegration() throws Exception { + start(null, Map.of("camel.chaos.enabled", "true")); + awaitList(l -> l.getBoolean("enabled")); + + // a hand-off over seda, which what to try suggests failing the route after it for + context.addRoutes(new RouteBuilder() { + @Override + public void configure() { + from("direct:hand").routeId("hand").to("seda:handed"); + from("seda:handed").routeId("handed").log("handed"); + } + }); + + // what to try only reads the routes: each suggestion is a rule the check takes, and nothing is added + JsonObject suggested = tools.camel_runtime_chaos_suggest("shop"); + List<Map<String, Object>> suggestions = suggested.getCollection("suggestions"); + assertThat(suggestions).anySatisfy(s -> { + assertThat(s.get("route")).isEqualTo("hand"); + assertThat((String) s.get("rule")).startsWith("route=handed error"); + }); + for (Map<String, Object> s : suggestions) { + assertThat(s).containsKeys("route", "kind", "rule", "why"); + assertThat(tools.camel_runtime_chaos_check("shop", (String) s.get("rule")).<List<Object>> getCollection( + "rules")).hasSize(1); + } + assertThat(send()).isNull(); + + // a scenario replaces the rules there are + tools.camel_runtime_chaos_add("shop", "step=* delay duration=1ms"); + JsonObject loaded = tools.camel_runtime_chaos_load("shop", + "# the target is down\nendpoint=direct:target error\n", "outage"); + assertThat(loaded.<List<Object>> getCollection("ids")).hasSize(1); + assertThat(loaded.getBoolean("reset")).isTrue(); + assertThat(rules(awaitList(l -> rules(l).size() == 1 && rules(l).get(0).contains("error"))).get(0)) + .endsWith("endpoint=direct:target error"); + assertThat(send()).isNotNull(); + + // a scenario with a line that is not valid changes nothing + assertThatThrownBy(() -> tools.camel_runtime_chaos_load("shop", "endpoint=direct:target explode\n", null)) + .isInstanceOf(ToolCallException.class); + assertThat(rules(tools.camel_runtime_chaos_list("shop"))).hasSize(1); + assertThat(send()).isNotNull(); + } + @Test void everyToolAgainstARunningIntegration() throws Exception { start(null, Map.of("camel.chaos.enabled", "true")); diff --git a/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsTest.java b/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsTest.java index 4283db6239ce..4cdeac541d40 100644 --- a/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsTest.java +++ b/dsl/camel-jbang/camel-jbang-mcp/src/test/java/org/apache/camel/dsl/jbang/core/commands/mcp/RuntimeToolsTest.java @@ -125,7 +125,8 @@ class RuntimeToolsTest { "get_sql_trace", "get_circuit_breakers", "get_metrics", "get_eip_stats", "get_spans", "get_startup_steps", "get_route_analysis", "detect_config_drift", - "chaos_list", "chaos_report", "chaos_check", "chaos_add", "chaos_change", "chaos_remove", "chaos_clear")) { + "chaos_list", "chaos_report", "chaos_check", "chaos_add", "chaos_change", "chaos_remove", "chaos_clear", + "chaos_suggest", "chaos_load")) { assertThat(ToolRegistry.findTool(name)).as(name).isNotNull(); } }
