allthingssecurity opened a new pull request, #27678: URL: https://github.com/apache/camel/pull/27678
# Description [CAMEL-25531](https://issues.apache.org/jira/browse/CAMEL-25531) Part 1 of 4 of the camel-wasm redesign agreed with @davsclaus on Zulip: the `wasm` endpoint now runs WASI Preview 1 command modules instead of calling a function through the custom `alloc`/`dealloc` and JSON message ABI. A Rust program built for `wasm32-wasip1` or a Go program built with `GOOS=wasip1 GOARCH=wasm` can be used as it is. Author: Shashank Mohan Jain (@allthingssecurity), working with Claude Code (Claude Opus 5.5). ## What changes - **URI**: the path is the module, `wasm:classpath:upper.wasm` (any Camel resource: classpath, `file:`, `http:`...). The function name and the `module` option are gone. - **Messages**: the body is standard input; standard output becomes the body (`byte[]`); the exit code is in `CamelWasmExitCode` (Integer) and standard error in `CamelWasmStderr` (UTF-8). A non-zero exit fails the exchange with a new `WasmExitCodeException` (exit code + stderr, body unchanged) unless `failOnNonZeroExit=false`. Both headers are removed before every run, so a value sent with the message, or left by an earlier `wasm` step, never survives (on success, non-zero exit, trap, output limit or timeout). Other headers are kept and are not passed to the module. - **Instances**: the module is parsed once per producer (`doInit`, so a bad module or bad options fail the route start) and every exchange runs on a new Endive instance, dropped afterwards. Exchanges share no guest memory or globals, there is no lock, and a trap or a timeout leaves nothing behind in a shared instance. - **What the module sees**: `args` (fixed by the route, argument zero is the module location), environment variables only from `environment.NAME=value` and the headers listed in `environmentHeaders` (an allow-list of names, looked up like any Camel header; a missing header is left unset; a value with a NUL fails the exchange). Never the JVM environment. No preopened directories, no sockets. Clock and random are the host's. The standard streams are reported as non-terminals (Endive reports terminals unless told otherwise), so a program that checks for a terminal does not add colours or other terminal output to the body. - **Limits** (all off by default): - `maxMemoryPages`: Endive `MemoryLimits` on the module's memory; growing past it fails inside the guest. Lower than the module's initial memory is rejected at start. - `timeout`: the module runs on a cached thread pool of the producer (created in `doStart` only when a timeout is set, `shutdownNow` in `doStop`) and the route thread waits on the future. On expiry the worker is interrupted; Endive checks the interrupt flag on calls and backward branches, so a spinning guest stops at its next iteration. The exchange fails with `ExchangeTimedOutException`. Interrupting a producer thread keeps the interrupt flag away from route threads. No thread hop without a timeout. - `maxOutputSize`: cap on the buffered standard output and, separately, standard error. Exceeding it stops the module (the buffer throws a `RuntimeException`, which Endive does not turn into an `EIO` for the guest) and fails the exchange with a message naming the stream and the option. - **Trap** (for example a Rust `panic`): `CamelExchangeException` caused by Endive's `TrapException`, with the start of stderr in the message (Rust prints the panic message there) and `CamelWasmStderr` set; a hint is added when the memory is at its maximum. New public types: `org.apache.camel.wasm.WasiCommand` (runs a parsed command module once, given stdin, stdout, stderr and the environment: the place where PR 2 adds the compiled machine factory and PR 3 can reuse it for the language), `org.apache.camel.component.wasm.WasmExitCodeException`, the two header constants in `Wasm.Headers`, `WasmEndpoint.getModule()`. Removed: `WasmComponent.get/setModule`, `WasmConfiguration.get/setModule`, the endpoint's `functionName`, `WasmSupport.deserialize` (only the old producer used it; the language still uses `serialize`); `WasmProducer` now takes a `WasmEndpoint`. `WasmConfiguration.copy()` now copies the `environment` map, so endpoints never share it. Not changed: the `wasm` language (it still uses `WasmFunction` and the custom ABI; PR 3 rebases or drops it). The 4.23 upgrade-guide note about a failed call discarding the instance now only mentions the `wasm` expression, since the endpoint no longer keeps an instance. ## Security model Modules stay trusted route code, as in the security model page; the docs say that the limits are robustness controls, not a sandbox. Bodies and headers stay untrusted data: the body only reaches the module as standard input, headers only through the explicit allow-list, arguments are fixed by the route. No option relaxes a default, so none is marked `insecure:*`. ## Docs and generated files - `wasm-component.adoc` rewritten for the WASI model: writing a module (Rust and Go), how a run works, what the module can access, limits (with the security-model sentence), an error table, performance, Java/XML/YAML examples. Catalog copy and `components/wasm.json` regenerated with the `prepare-catalog` mojo (identical to the component files); endpoint DSL (`WasmEndpointBuilderFactory`, `StaticEndpointBuilders`, `EndpointHeaderBuilders`) and component DSL description regenerated with the `generate-endpoint-dsl` / `generate-component-dsl` mojos. - 4.23 upgrade guide: `=== camel-wasm - the endpoint runs WASI command modules (Breaking change)`, next to the existing camel-wasm entries: what changed and how to migrate (turn the function into a program reading stdin and writing stdout, build it for WASI Preview 1). ## Dependencies Adds `run.endive:wasi:${endive-version}` (1.1.0, Apache-2.0, same project as the runtime already used; brings `run.endive:log`). Endive alignment in the parent POM (importing the Endive BOM so camel-quickjs's `wasi` 1.0.0 via quickjs4j 0.1.0 and camel-opa's `runtime`/`compiler` 1.0.0 via opa-java-wasm 1.1.0 move to 1.1.0) is **not** in this PR: it changes the Endive versions camel-opa and camel-quickjs run with and the generated `camel-dependencies`, so it needs their test suites and belongs with PR 2, where the compiler dependency (the one camel-opa already brings at 1.0.0) arrives. This PR does not add a new split: camel-quickjs already runs quickjs4j's `wasi` 1.0.0 on `runtime` 1.1.0, and the current SBOM already lists Endive `runtime` and `wasm` at both 1.0.0 and 1.1.0. @davsclaus if you would rather have the alignment in this PR, I will add it here. ## Tests `WasmComponentTest` is rewritten: 24 tests. The modules are written in WAT in the test and compiled when the test runs with `run.endive:wabt` (already a test dependency), so no binary is added; the existing `functions.wasm` is used to check that a module without `_start` is rejected (and keeps serving `WasmLanguageTest`). - body to stdin, stdout to body, headers set, other headers kept; 1 MiB body; empty body - exit 3: `WasmExitCodeException` with code and stderr, body unchanged, headers set; same with `failOnNonZeroExit=false` (body = stdout); exit 0 through `proc_exit` with stderr - result headers sent with the message are replaced; trap, without and with a timeout (so also when the exception comes back from the producer thread): exception with cause `TrapException` and the stderr text, `CamelWasmStderr` set, inbound `CamelWasmExitCode` removed - environment: exactly `LEVEL=debug` and `TENANT=acme` from `environment.LEVEL` + `environmentHeaders=TENANT,REGION` while the message also carries `SECRET` and `CamelHttpUri` (and no JVM variables); header with NUL fails; a name in both options fails the start - args with a quoted argument; unbalanced quote fails the start - the module sees standard input, output and error as non-terminals (WASI file type 0, not 2) - timeout on a spinning module, twice on the same endpoint (`ExchangeTimedOutException`, no result headers), then awaits (Awaitility, 10 s) until no producer thread runs guest code; timeout endpoint across a route stop/start - `maxMemoryPages=4`: the guest that grows its memory ends at 4 pages (17 without the cap); a cap below the initial memory fails the start - `maxOutputSize` on stdout (1000 bytes pass at 1000, fail at 999) and on stderr - fresh state: a module with a global counter returns 1 on every exchange - 8 threads x 50 exchanges with distinct payloads, all correct - module without `_start`, module importing outside `wasi_snapshot_preview1`: start fails Sensitivity: with the header removal deleted, the interrupt replaced by `cancel(false)`, the memory limit not applied and the output cap disabled, 6 tests fail (`resultHeadersSentWithTheMessageAreReplaced`, `trapFailsTheExchangeWithStandardError`, `timeoutInterruptsASpinningModule` (the spinning guest is still running after 10 s), `maxMemoryPagesLimitsMemoryGrowth`, `maxOutputSizeLimitsStandardOutput`, `maxOutputSizeLimitsStandardError`). With Endive's default terminal streams, `standardStreamsAreNotTerminals` fails (`222` instead of `000`). camel-wasm with `install` (formatter, import sort, generation): 33 tests, 0 failures (`WasmComponentTest` 24 + `WasmLanguageTest` 6 + `WasmFunctionTest` 3). ## Performance Measured through a route (`ProducerTemplate.requestBody` -> `direct:` -> `wasm:`), interpreted, 1 KiB ASCII body, Apple M4 Pro, JDK 21, Endive 1.1.0; local probe, not committed: | Module | Size | Parse (once) | Per exchange, median | p10 / p90 | |---|---|---|---|---| | standard Go 1.26 `GOOS=wasip1` program (`io.ReadAll` + `bytes.ToUpper`) | 2.3 MB | 0.2-0.4 s | **27-28 ms** (n=50 after 5 warm-up, two rounds) | 26.8-27.5 / 29.4-32.4 ms | | tiny `no_std` Rust WASI command (ASCII upper-case) | 3.9 KB | < 10 ms | **270-283 us** (n=2000 after 300 warm-up) | 247-256 / 327-351 us | A fresh instance per exchange is cheap for a small module; a standard Go program spends most of each run initializing the Go runtime again. Per-exchange cost of this design vs the old endpoint: one instantiation (data segments copied, Go's are large) plus a WASI context, no lock. The optional compiler (PR 2) cut the Go case to about 5 ms per exchange in the earlier prototype; TinyGo/Rust/C guests avoid most of it. This is why PR 4 will show both Rust and Go and state the numbers. ## Plan 1. **This PR**: WASI runner, endpoint and limits replacing the custom ABI, docs and upgrade note. 2. Optional runtime compiler (`compile=true`, off by default because of native images), docs for the Endive build-time compiler plugin, Endive version alignment in the parent POM. 3. The `wasm` language: rebase it on `WasiCommand`, or drop it. 4. Rust and Go examples in camel-examples. @davsclaus please review. # Target - [x] I checked that the commit is targeting the correct branch (Camel 4 uses the `main` branch) # Tracking - [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it). # Apache Camel coding standards and style - [x] I checked that each commit in the pull request has a meaningful subject line and body. - [ ] I have run `mvn clean install -DskipTests` locally from root folder and I have committed all auto-generated changes. (I built and tested camel-wasm with `install`, including the formatter and import-sort plugins, and regenerated the catalog copy and the endpoint and component DSL with the camel-package-maven-plugin mojos. I did not run the full root build.) # AI-assisted contributions - [x] If this PR includes AI-generated code, commits have proper co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR description identifies the AI tool used. Prepared by Shashank Mohan Jain (@allthingssecurity) with Claude Code (Claude Opus 5.5). The commit carries a `Co-Authored-By` trailer. ## Notes (for the author, remove before opening) - Go timing: measured locally (Go 1.26.0 darwin/arm64), numbers above; probe source and raw output in `findings/wasm-evidence/wasi-runner-timing/`. Rust `wasm32-wasip1` is not installed locally, so the Rust row is the `no_std` guest of the earlier prototype, not a std Rust program. The Go program was run again through the endpoint after the non-terminal streams change (local probe, 4000-byte body, correct output). - No Zulip thread URL was found in the findings files; add the link to the JIRA and here if wanted. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
