This is an automated email from the ASF dual-hosted git repository. luigidemasi pushed a commit to branch fix/security-model-verification-153-158 in repository https://gitbox.apache.org/repos/asf/camel-kit.git
commit 58fb578cb23d4ca77a14cdcc216e6f2077641a26 Author: Luigi De Masi <[email protected]> AuthorDate: Mon Nov 9 19:15:20 2026 +0100 Fix #158: verify Knowledge boundaries and enforce offline models Fixes #158 Co-authored-by: OpenAI Codex <[email protected]> --- CHANGELOG.md | 2 + docs/modules/ROOT/pages/knowledge.adoc | 18 ++ docs/security-model.md | 65 +++-- docs/security-model.yaml | 9 +- docs/security-verification.md | 45 +++ .../kit/knowledge/embedding/OfflineTokenizer.java | 33 +++ .../knowledge/embedding/OnnxEmbeddingProvider.java | 2 +- .../kit/knowledge/embedding/OnnxReranker.java | 2 +- .../kit/knowledge/mcp/LuceneSearchService.java | 12 +- .../knowledge/mcp/KnowledgeInputBoundaryTest.java | 191 +++++++++++++ .../knowledge/mcp/LuceneFallbackBoundaryTest.java | 191 +++++++++++++ .../camel/kit/knowledge/mcp/OfflineModelsTest.java | 222 +++++++++++++++ tools/FixtureIndex.java | 21 ++ tools/verify_knowledge_runtime.py | 311 +++++++++++++++++++++ 14 files changed, 1082 insertions(+), 42 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index be030dc3..95ba9893 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,6 +22,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- **Knowledge runtime boundaries (#158)** — enforce offline CPU tokenizer initialization, cover query/cap/fallback and transport behavior, document native caches, and accept conventional CVE severity spellings. + - **Generated and cached path confinement (#157)** — refuse descendant symlinks before init/catalog writes and validate catalog versions before extraction or replacement. - **Claude MCP permissions (#155)** — replace server-wide wildcard approvals with explicit manifest-listed tools, with generation and actual-host permission checks. diff --git a/docs/modules/ROOT/pages/knowledge.adoc b/docs/modules/ROOT/pages/knowledge.adoc index 1fe36de3..0bbdae8e 100644 --- a/docs/modules/ROOT/pages/knowledge.adoc +++ b/docs/modules/ROOT/pages/knowledge.adoc @@ -153,3 +153,21 @@ lookup in a read-only research role where the agent has one (Bob 2's reviewer pr reviewer leaf, OpenCode's researcher task) and run it inline on Pi. The role returns a concise answer with references instead of raw search results, and a failed lookup is reported as missing evidence rather than replaced with invented documentation. + +== Local model initialization and host effects + +The monorepo Knowledge runner loads its bundled embedding and reranking models locally. +Tokenizer initialization enforces offline CPU mode, disables DJL telemetry/native downloads and +implicit CUDA subprocess discovery. Remove conflicting `DJL_OFFLINE` values other than `true` +or `RUST_FLAVOR` values other than `cpu`; startup reports these conflicts before loading native code. +This applies to the monorepo source runtime, not the previously released Knowledge 0.0.2 runner. + +Native initialization still writes DJL's cache (normally `~/.djl.ai/tokenizers`, configurable with +`ENGINE_CACHE_DIR` or `DJL_CACHE_DIR`) and ONNX/native temporary files. The embedding model also +uses a temporary directory. These paths are separate from the Knowledge index cache. + +Stdio is the default and disables the HTTP host. If enabling HTTP/SSE, explicitly configure +`quarkus.http.host` and `quarkus.http.port`; the transport has no built-in authentication. +Use loopback or provide an authenticated deployment boundary. A local `knowledge.index.path` +bypasses index downloads. Endpoint-validation tools inspect catalog metadata and do not execute +an endpoint or fetch its URI. diff --git a/docs/security-model.md b/docs/security-model.md index 008ac56c..a33b0ff3 100644 --- a/docs/security-model.md +++ b/docs/security-model.md @@ -236,7 +236,7 @@ notes, CVE advisories and component metadata" that "also works with MCP clients root `README.md`)*. Kit configures it for every harness at a pinned coordinate (§4.8 P4). This section models the server, the index it acquires and the indexer that produces that index *(maintainer, 2026-10-27 — Q11)*; family K in §4.2 is the server with its index acquisition and bundled models, K2 the indexer. Its statements come from the -Knowledge README and code and await maintainer verification (V6). +Knowledge README, code and executable checks recorded in [V6 evidence](security-verification.md#v6-knowledge). **Processes.** @@ -256,11 +256,17 @@ Knowledge README and code and await maintainer verification (V6). with the JDK's default temporary-directory permissions and removed on orderly shutdown; a pinned client receives the error instead. The monorepo runner bundles no index, so the fallback serves legacy uber-jars that do *(documented — `knowledge/README.md`, "Index Distribution" step 3; `LuceneSearchService.resolveIndex`; the - permission and bundling statements are inferred — V6)*. -- *Bundled embedding model and reranker* (`knowledge/embedding`): ONNX models packaged at build time ("The MCP module - downloads the ONNX embedding model from HuggingFace during `generate-resources`") and loaded at runtime from the - classpath into a private `camel-kit-onnx-*` temporary directory *(documented — `knowledge/README.md`; - `OnnxEmbeddingProvider`)*; no model is downloaded at runtime *(inferred — V6)*. + permission and fallback checks are covered by `LuceneFallbackBoundaryTest`; bundling is checked against the packaged runner)*. +- *Bundled embedding model and reranker* (`knowledge/embedding`): model/tokenizer resources are + acquired at build time and loaded locally. The embedding model uses a `camel-kit-onnx-*` temporary + directory; the reranker loads model bytes directly. Native libraries additionally use DJL's engine + cache (normally `~/.djl.ai/tokenizers`, configurable by `ENGINE_CACHE_DIR`/`DJL_CACHE_DIR`), + ONNX Runtime's `onnxruntime-java*` directory and dependency-managed native temporary files. + `OfflineTokenizer` enforces offline CPU initialization, disables DJL's optional CUDA subprocess + and avoids JNA's implicit `ldconfig` lookup. Conflicting `DJL_OFFLINE` or `RUST_FLAVOR` environment + values fail before native initialization. This prevents DJL metadata/telemetry and native downloads + on the supported bundled CPU path; it is not an OS sandbox or permission to install arbitrary + native-library overrides. `OfflineModelsTest` exercises both model initialization orders. - *Indexer* (`knowledge/indexer`, `IndexerMain`): a maintainer-run build tool that "runs during CI/CD to produce the pre-built index artifact", invoked by the `-Prebuild-index` profile and the `Index Release` workflow. It clones Apache repositories at release tags with JGit, converts AsciiDoc, downloads Camel Catalog JARs, fetches Apache JIRA @@ -271,7 +277,7 @@ Knowledge README and code and await maintainer verification (V6). 1. *MCP client → server.* Tool arguments are model output and therefore untrusted (§4.7). The server escapes free-text queries for Lucene (`QueryParser.escape`), clamps result counts to 25, matches identifiers as structured fields and - validates endpoint URIs against the bundled Camel catalog; no argument names a file, URL or command *(documented — + validates endpoint URIs against the bundled Camel catalog; no argument selects a filesystem target, remote fetch, executable command or index source *(documented — `LuceneSearchService`, `KnowledgeMcpServer`)*. 2. *Release origin → cache.* Discovery reads `https://api.github.com/repos/apache/camel-kit/releases` (at most 10 pages of 100 releases, each at most 2 MiB, and pagination must stay on the configured endpoint), selects only @@ -282,7 +288,7 @@ Knowledge README and code and await maintainer verification (V6). version when an update fails; a pinned version never falls back to another *(documented — `knowledge/README.md`, "Index Distribution"; `IndexDownloader`)*. Connections time out after 3 s, manifests after 5 s and archives after 5 min including the body; redirects are followed only after each target passes the same transport rule — `https`, - or `http` to a loopback host — for at most five hops, so no hop can downgrade to cleartext *(documented — + or `http` to a loopback host — for at most five hops, so no hop can use non-loopback cleartext *(documented — `IndexDownloader.send`; apache/camel-kit#143)*. 3. *Index → results.* The index is "data, not code": it is regenerated on a branch, committed with a signed commit, reviewed through a pull request, built unchanged by the release workflow and published as a GitHub release asset @@ -293,16 +299,16 @@ Knowledge README and code and await maintainer verification (V6). *(documented — `knowledge/README.md`, "Data Sources", "CVE enrichment and fallback")*. These are trusted upstream sources; a hostile document would require a compromised Apache repository or mirror (§4.7). -**What the Knowledge server does to its host** (row in the §4.5 inventory): no listening socket in the default -configuration *(documented — README; the running-process claim is inferred — V6)*; no child process *(inferred — -V6)*; network only to the configured release endpoint, manifest and asset URLs *(documented — `IndexDownloader`; "no -other network" inferred — V6)*; writes the cache directory — version directories, the `current` marker, the `etag` -validator, staging files and an `.update.lock` — and the private `camel-kit-onnx-*` temporary directory *(documented — -`IndexDownloader`, `OnnxEmbeddingProvider`)*, plus, only when index resolution fails and no version is pinned, a -`knowledge-index*` temporary directory holding the classpath-extracted fallback index, removed on orderly shutdown -*(documented — `LuceneSearchService`)*; nothing else *(inferred — V6)*; reads `knowledge.index.*` properties and their -`KNOWLEDGE_INDEX_*` -environment equivalents *(documented — README)*; "No Docker. No Docling. No external tokens." *(documented — README)*. +**What the Knowledge server does to its host:** stdio disables the HTTP host; explicit HTTP/SSE +opt-in listens on the configured address without built-in authentication. Set the host explicitly; +framework defaults are not a loopback guarantee. The application fetches index releases, manifests +and archives when acquisition needs them; a local index bypasses acquisition. Bundled models initialize +in process under `OfflineTokenizer`'s offline CPU policy. Writes include index cache/staging/metadata/ +locks, temporary fallback and embedding data, DJL's native cache and ONNX/JNA native temporary files. +It reads `knowledge.index.*`, `KNOWLEDGE_INDEX_*`, Quarkus, JVM home/temp and native-library/cache +configuration. These are scoped application/dependency behaviors, not restrictions on JVM startup +agents, custom native libraries or the operator's OS account. See the §4.5 inventory and +[V6 evidence](security-verification.md#v6-knowledge) for regression and packaged-process checks. **Knobs** (also in §4.5a): `knowledge.index.path`; `knowledge.index.url`, which accepts `https` and `file` anywhere and `http` only to a loopback host (apache/camel-kit#143); `knowledge.index.version`; `knowledge.index.releases-url`; `knowledge.index.cache-dir`; @@ -645,10 +651,9 @@ violation warrants coordinated disclosure and a CVE; **hardening** means it is f 12. **P12 — Knowledge server transport.** "The server uses stdio by default and does not open an HTTP port"; the HTTP and SSE transports exist only when the operator enables them (`quarkus.http.host-enabled=true`) on the - configured host and port *(documented — `knowledge/README.md`)*; the no-listener claim for the running default - configuration is *(inferred — V6)*. *Violation:* a listening socket without the opt-in, or on a host other than + configured host and port *(documented — `knowledge/README.md`)*; the default and opt-in transports have packaged-process probes ([V6 evidence](security-verification.md#v6-knowledge)). *Violation:* a listening socket without the opt-in, or on a host other than the configured one. *Severity:* security-critical. -13. **P13 — Knowledge index acquisition integrity.** A local `knowledge.index.path` is used in place without network. +13. **P13 — Knowledge index acquisition integrity.** A local `knowledge.index.path` is used in place without index-acquisition network access. Otherwise the server fetches a published `knowledge-index-v1-*` release (the newest compatible one, or the exact pinned version) or the configured manifest with bounded pages and deadlines; the manifest must match the release tag, format version 1, the bundled embedding model and 384 dimensions; the archive's SHA-256 must equal the @@ -667,9 +672,12 @@ violation warrants coordinated disclosure and a CVE; **hardening** means it is f sources; the published `0.0.2` server has the standalone repository's download behaviour (§4.4a). 14. **P14 — Knowledge tool arguments are queries, not code.** `camel_docs_*` arguments are used as escaped Lucene query text (`QueryParser.escape`), structured field lookups, version strings and an endpoint URI validated - against the bundled Camel catalog; `max_results` is clamped to 25; no argument names a file path, URL or command - *(documented — `knowledge/README.md`, "MCP Tools"; `LuceneSearchService`; `KnowledgeMcpServer`)*. The server - spawns no process and performs no network access other than index acquisition *(inferred — V6)*. *Violation:* + against the bundled Camel catalog; `max_results` is clamped to 25; no argument selects a filesystem target, remote fetch, executable command or index source + *(documented — `knowledge/README.md`, "MCP Tools"; `LuceneSearchService`; `KnowledgeMcpServer`)*. `KnowledgeInputBoundaryTest` exercises query syntax, structured identifiers, every result cap and + HTTP/XSLT/exec/file endpoint canaries. Bundled model initialization enforces offline CPU behavior + through `OfflineTokenizer` and is tested by `OfflineModelsTest`. These paths do not execute tool + arguments or use them to select an external fetch. JVM local-interface discovery is distinct from + application requests; the runtime is not an OS sandbox. *Violation:* a tool argument reaching the filesystem, the network or a code path, or changing which index or files are read. *Severity:* security-critical. @@ -736,8 +744,7 @@ This is the most important section for a developer adopting Camel-Kit and for an as information, not enforced. - **Knowledge: no authentication or authorization on the MCP transport.** stdio is the invoking user; the opt-in - HTTP/SSE transport carries no authentication beyond its bind address *(documented — `knowledge/README.md`; inferred - for the SSE posture — V6)*. + HTTP/SSE transport carries no authentication beyond its bind address *(documented — `application.properties`; packaged SSE probe in [V6 evidence](security-verification.md#v6-knowledge))*. - **Knowledge: no independent signature on the index.** Integrity rests on HTTPS to the GitHub release origin and the SHA-256 in a manifest served from the same origin; a configured `file://` manifest, or a loopback `http://` one, is trusted as operator input, and remote cleartext is refused before any request *(documented — `knowledge/README.md`; @@ -1050,11 +1057,7 @@ own issue, linked below. `VALID`. - **V5 — Init and download path controls (Q7, Q9; #157): implemented.** Regression tests cover shared init outputs, every agent, generated leaves, cache ancestors, malicious versions and archive paths for Citrus, Forage and Knowledge. [V5 evidence](security-verification.md#v5-generated-and-extracted-paths). Redirected writes remain `VALID`. -- **V6 — Knowledge section verification (Q11; #158) → §4.4a, §4.8 P12–P14.** Maintainer verification of the statements - drafted from code and the README: no listener in the default configuration, no process execution and no network - access other than index acquisition, tool-argument handling, cache permissions, the SSE posture, and the classpath - fallback (temporary-directory permissions; no bundled index in the monorepo runner). Related hardening: #137 - (apache/camel-kit#143), #139 (apache/camel-kit#142), #140. *Interim triage:* a breach of P12–P14 is `VALID`. +- **V6 — Knowledge verification (Q11; #158): implemented.** Tool argument/cap tests, pinned/temp fallback tests, real-model offline probes and packaged stdio/SSE checks are recorded in [V6 evidence](security-verification.md#v6-knowledge). Native caches are disclosed, DJL offline initialization is enforced and ordinary severity spellings work. Existing #143/#142 hardening remains covered. A P12–P14 breach remains `VALID`. --- diff --git a/docs/security-model.yaml b/docs/security-model.yaml index 72acec8f..b307c3d4 100644 --- a/docs/security-model.yaml +++ b/docs/security-model.yaml @@ -339,17 +339,17 @@ claimed_properties: property: Knowledge server transport - stdio by default with no listening socket; the HTTP/SSE transport exists only when the operator enables it (quarkus.http.host-enabled=true) on the configured host and port. violation_symptom: A listening socket without the opt-in, or on a host other than the configured one. severity: security-critical - provenance: documented (knowledge/README.md "Running the MCP Server"); no-listener claim for the default configuration inferred (V6) + provenance: "application.properties and knowledge/README.md; packaged stdio and explicit loopback SSE probe recorded in security-verification.md V6. HTTP/SSE has no built-in authentication; configure bind host explicitly." - id: P13 property: Knowledge index acquisition integrity - a local knowledge.index.path is used in place without network; otherwise a published knowledge-index-v1-* release (newest compatible, or the exact pinned version) or an explicitly configured manifest is fetched with bounded pages and deadlines, the manifest must match the release tag, format version 1, the bundled embedding model and 384 dimensions, the archive SHA-256 must equal the manifest value before unpacking, zip entries are co [...] violation_symptom: An archive accepted on SHA-256 mismatch; an entry written outside the staging or version directory; a version name outside the safe-name rules accepted; a pinned client using another version or the classpath fallback; a manifest from a different release tag accepted; a remote fetch or redirect issued over non-loopback http; activation of a partially unpacked version. severity: security-critical (integrity and path escape) provenance: documented (knowledge/README.md "Index Distribution", IndexDownloader, LuceneSearchService, apache/camel-kit#143) - id: P14 - property: Knowledge tool arguments are queries, not code - camel_docs_* arguments are used as escaped Lucene query text, structured field lookups, version strings and an endpoint URI validated against the bundled Camel catalog; no argument names a file path, URL or command; the server spawns no process and performs no network access other than index acquisition. + property: Knowledge tool arguments are queries, not code - camel_docs_* arguments are used as escaped Lucene query text, structured field lookups, version strings and an endpoint URI validated against the bundled Camel catalog; no argument selects a filesystem target, remote fetch, executable command or index source. OfflineTokenizer enforces offline CPU model initialization and disables implicit DJL/JNA subprocess discovery; native libraries still use caches and temporary files. App [...] violation_symptom: A tool argument reaching the filesystem, the network or a code path, or changing which index or files are read. severity: security-critical - provenance: documented (knowledge/README.md tool table, LuceneSearchService QueryParser.escape, KnowledgeMcpServer MAX_RESULTS_CAP); no-process and no-other-network claims inferred (V6) + provenance: "KnowledgeMcpServer, LuceneSearchService, OfflineTokenizer; KnowledgeInputBoundaryTest, OfflineModelsTest and packaged-process observations (security-verification.md V6)." correctness_only_properties: - Deterministic graph merge order. @@ -624,8 +624,9 @@ verification_items: sections: ["4.4a", "4.8 P12", "4.8 P13", "4.8 P14"] work: "Maintainer verification of the Knowledge statements drafted from code and README - no listener in the default configuration, no process execution and no network access other than index acquisition, tool-argument handling, cache permissions, the SSE posture, and the classpath fallback (temporary-directory permissions, no bundled index in the monorepo runner). Related hardening - #137 (PR #143), #139 (PR #142), #140." interim_triage: A breach of P12-P14 is VALID. - status: open + status: "implemented" + evidence: "KnowledgeInputBoundaryTest, LuceneFallbackBoundaryTest, OfflineModelsTest, existing IndexDownloader/ReleaseDiscovery regressions and packaged stdio/SSE probe; security-verification.md V6" revision_triggers: - New CLI command, parser or input format reading external content; parser moving to later compilation phase or symbol resolution. - Change to the graph parser timeout or the POM parser path, or addition of size, memory or symbolic-link checks in kit/camel-kit-graph (closes V2 items). diff --git a/docs/security-verification.md b/docs/security-verification.md index 494fedc0..6277ca6c 100644 --- a/docs/security-verification.md +++ b/docs/security-verification.md @@ -79,3 +79,48 @@ An explicitly chosen root alias is allowed. Extraction creates ordinary files an materializes ZIP symlink metadata. These checks do not provide descriptor-relative isolation against concurrent same-account directory substitution; existing same-account exclusions apply. Website impact: required; initialization behavior documented in the agent reference. + + +## V6: Knowledge + +`KnowledgeInputBoundaryTest` supplies synthetic Lucene documents to actual service/tool methods. +It checks escaped free text, structured identifiers, upper/default/lower result bounds across all +call sites and catalog-only HTTP/XSLT/exec/file endpoint validation. A conventional HIGH severity +query exposed duplicate elements in `Set.of`; distinct case variants now accept HIGH/high/High. +`LuceneFallbackBoundaryTest` checks pinned failures, readable unpinned temporary fallback, +POSIX temporary-directory privacy where supported, cleanup, persistent-index preservation and +failure when no bundled index exists. Existing downloader tests cover cache permissions, hash, +transport/redirect, pin, staging and activation boundaries. Removing query escaping, increasing +the cap or disabling fallback cleanup each made the respective regression fail. + +`OfflineModelsTest` forks fresh JVMs with isolated caches and real bundled embedding/reranker +resources, in both initialization orders. Both return valid inference results with zero observed +HTTP opens, JFR process starts or socket I/O. Four conflicting-environment cases fail before +native/model extraction. All six cases passed directly on Java 17, 21 and 25. Normal cases start +with deliberately conflicting system properties to check enforcement. Removing either provider's +offline call produced four blocked EC2 metadata requests. No real metadata request was sent. +An earlier probe caught JNA launching `ldconfig`; the shared initializer now suppresses implicit +platform-library discovery while preserving an explicitly configured search path. + +The offline policy is process-wide DJL configuration for the bundled CPU runtime, not an OS +sandbox. DJL cache and ONNX/native temporary writes remain expected. Conflicting environment +values are rejected rather than silently enabling telemetry or GPU-native downloads. +Packaged transport verification uses `tools/verify_knowledge_runtime.py` with a local synthetic +index: stdio initialize/list-tools/first query, and explicit loopback SSE initialize/list-tools/ +query without authorization. It inspects the child process's sockets and syscall trace where +available; JVM local-interface discovery is reported separately from application network. +Website impact: required; Knowledge runtime/reference updated. Published Knowledge0.0.2 is not +retroactively changed by these source fixes. + + +### Packaged V6 observation (2026-11-09) + +The checked-in probe passed on Temurin 21.0.8+9 with runner SHA-256 +`d5e8e7b1a379e75bea6f432d0538d1bc54a75a1143bbb040730f7edb6575e6a6`, with no +baseline/offline/CUDA/JNA command overrides. Both fresh JVMs initialized MCP, listed seven +tools and returned the local fixture from the first query with reranking. Default stdio had +no IP sockets, binds or listeners; explicit SSE had one 127.0.0.1 listener and accepted the +unauthenticated initialization/query. Lifetime syscall traces contained only the initial JVM +exec, no child executable and no application service connection/send. JVM netlink and UDP +port-zero interface discovery was recorded separately. The packaged runner contained all five +model/tokenizer resources and no `knowledge-index/` payload. diff --git a/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OfflineTokenizer.java b/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OfflineTokenizer.java new file mode 100644 index 00000000..a1654eac --- /dev/null +++ b/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OfflineTokenizer.java @@ -0,0 +1,33 @@ +package org.apache.camel.kit.knowledge.embedding; + +import java.io.IOException; +import java.io.InputStream; +import java.util.Map; + +import ai.djl.huggingface.tokenizers.HuggingFaceTokenizer; + +/** Keeps bundled tokenizer initialization local, including DJL native-library discovery. */ +final class OfflineTokenizer { + private OfflineTokenizer() { + } + + static synchronized HuggingFaceTokenizer load(InputStream input, Map<String, String> options) throws IOException { + // DJL gives environment variables precedence over system properties. + String offline = System.getenv("DJL_OFFLINE"); + String flavor = System.getenv("RUST_FLAVOR"); + if (offline != null && !Boolean.parseBoolean(offline)) { + throw new IllegalStateException( + "Camel Kit requires DJL_OFFLINE=true; remove the conflicting environment value"); + } + if (flavor != null && !"cpu".equals(flavor)) { + throw new IllegalStateException( + "Camel Kit requires RUST_FLAVOR=cpu; remove the conflicting environment value"); + } + System.setProperty("ai.djl.offline", "true"); + System.setProperty("RUST_FLAVOR", "cpu"); + System.setProperty("ai.djl.util.cuda.fork", "false"); + // JNA otherwise runs ldconfig while DJL probes CUDA, even for the bundled CPU tokenizer. + System.getProperties().putIfAbsent("jna.platform.library.path", ""); + return HuggingFaceTokenizer.newInstance(input, options); + } +} diff --git a/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OnnxEmbeddingProvider.java b/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OnnxEmbeddingProvider.java index f4a182de..4b56878c 100644 --- a/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OnnxEmbeddingProvider.java +++ b/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OnnxEmbeddingProvider.java @@ -143,7 +143,7 @@ public class OnnxEmbeddingProvider implements EmbeddingProvider { if (is == null) { throw new RuntimeException("tokenizer.json not found on classpath"); } - tokenizer = HuggingFaceTokenizer.newInstance(is, Map.of( + tokenizer = OfflineTokenizer.load(is, Map.of( "padding", "false", "truncation", "true", "maxLength", String.valueOf(MAX_SEQ_LENGTH))); diff --git a/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OnnxReranker.java b/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OnnxReranker.java index f2133180..d9bd02e5 100644 --- a/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OnnxReranker.java +++ b/knowledge/embedding/src/main/java/org/apache/camel/kit/knowledge/embedding/OnnxReranker.java @@ -70,7 +70,7 @@ public class OnnxReranker { if (is == null) { throw new RuntimeException(TOKENIZER_FILE + " not found on classpath"); } - tokenizer = HuggingFaceTokenizer.newInstance(is, Map.of( + tokenizer = OfflineTokenizer.load(is, Map.of( "padding", "false", "truncation", "true", "maxLength", String.valueOf(MAX_SEQ_LENGTH))); diff --git a/knowledge/mcp/src/main/java/org/apache/camel/kit/knowledge/mcp/LuceneSearchService.java b/knowledge/mcp/src/main/java/org/apache/camel/kit/knowledge/mcp/LuceneSearchService.java index 794a02ee..64c2b99e 100644 --- a/knowledge/mcp/src/main/java/org/apache/camel/kit/knowledge/mcp/LuceneSearchService.java +++ b/knowledge/mcp/src/main/java/org/apache/camel/kit/knowledge/mcp/LuceneSearchService.java @@ -6,6 +6,7 @@ import java.nio.charset.StandardCharsets; import java.nio.file.*; import java.nio.file.attribute.BasicFileAttributes; import java.util.*; +import java.util.LinkedHashSet; import jakarta.annotation.PostConstruct; import jakarta.annotation.PreDestroy; @@ -507,11 +508,12 @@ public class LuceneSearchService { /** Case variants for exact StringField matching — indexed severities vary ("HIGH" vs "Important"). */ private static BooleanQuery caseVariantsClause(String field, String value) { BooleanQuery.Builder b = new BooleanQuery.Builder(); - for (String v : Set.of(value, value.toUpperCase(Locale.ROOT), value.toLowerCase(Locale.ROOT), - value.isEmpty() - ? value - : Character.toUpperCase(value.charAt(0)) - + value.substring(1).toLowerCase(Locale.ROOT))) { + for (String v : new LinkedHashSet<>( + List.of(value, value.toUpperCase(Locale.ROOT), value.toLowerCase(Locale.ROOT), + value.isEmpty() + ? value + : Character.toUpperCase(value.charAt(0)) + + value.substring(1).toLowerCase(Locale.ROOT)))) { b.add(new TermQuery(new Term(field, v)), BooleanClause.Occur.SHOULD); } return b.build(); diff --git a/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/KnowledgeInputBoundaryTest.java b/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/KnowledgeInputBoundaryTest.java new file mode 100644 index 00000000..9e3e0e07 --- /dev/null +++ b/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/KnowledgeInputBoundaryTest.java @@ -0,0 +1,191 @@ +package org.apache.camel.kit.knowledge.mcp; + +import java.lang.reflect.Field; +import java.net.InetSocketAddress; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.camel.kit.knowledge.schema.KnowledgeDocument; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.sun.net.httpserver.HttpServer; +import org.apache.lucene.analysis.standard.StandardAnalyzer; +import org.apache.lucene.index.DirectoryReader; +import org.apache.lucene.index.IndexWriter; +import org.apache.lucene.index.IndexWriterConfig; +import org.apache.lucene.search.IndexSearcher; +import org.apache.lucene.store.ByteBuffersDirectory; +import org.apache.lucene.store.Directory; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import static org.junit.jupiter.api.Assertions.*; + +/** Exercises the MCP entry points against a real, deterministic BM25 index without model initialization. */ +class KnowledgeInputBoundaryTest { + + private final ObjectMapper mapper = new ObjectMapper(); + private Directory directory; + private LuceneSearchService service; + private KnowledgeMcpServer server; + + @TempDir + Path temp; + + @BeforeEach + void createIndex() throws Exception { + directory = new ByteBuffersDirectory(); + try (StandardAnalyzer analyzer = new StandardAnalyzer(); + IndexWriter writer = new IndexWriter(directory, new IndexWriterConfig(analyzer))) { + for (int i = 0; i < 40; i++) { + writer.addDocument(new KnowledgeDocument("release-" + i, "apache_camel") + .source("apache-camel").docType("release-notes").sourceVersion("4.22") + .component("safecomponent").jiraId("CAMEL-999") + .sectionTitle("Release notes").content("needle release ordinary documentation").build()); + writer.addDocument(new KnowledgeDocument("advisory-" + i, "apache_camel") + .source("apache-camel").docType("cve").sourceVersion("4.22") + .cveId("CVE-2099-0001").erratumId("advisory-" + i).severity("HIGH") + .sectionTitle("Security advisory").content("CVE security vulnerability advisorytoken").build()); + } + writer.addDocument(new KnowledgeDocument("secretmarker", "apache_camel") + .docType("guide").sectionTitle("Decoy").content("unrelated orchard").build()); + writer.addDocument(new KnowledgeDocument("foreign-domain", "another_domain") + .docType("guide").sectionTitle("Other domain").content("needle release").build()); + } + service = new LuceneSearchService(); + DirectoryReader reader = DirectoryReader.open(directory); + // Install the real Lucene fixture without invoking unrelated ONNX/native startup. + setField(service, "reader", reader); + setField(service, "searcher", new IndexSearcher(reader)); + server = new KnowledgeMcpServer(); + server.searchService = service; + } + + @AfterEach + void closeIndex() throws Exception { + if (service != null) { + service.close(); + } + if (directory != null) { + directory.close(); + } + } + + @Test + void querySyntaxCannotSelectFieldsOrTurnIntoMatchAll() throws Exception { + assertEquals(0, results(server.camel_docs_search("id:secretmarker", null, 25)).size()); + assertEquals(0, results(server.camel_docs_search("*:*", null, 25)).size()); + + for (String query : List.of("needle", "needle OR *:*", "(\"needle")) { + JsonNode matches = results(server.camel_docs_search(query, null, 25)); + assertEquals(25, matches.size(), query); + for (JsonNode match : matches) { + assertTrue(match.path("id").asText().startsWith("release-"), match.toString()); + } + } + assertTrue(service.searchErrata(null, null, null, "*:*", 25).isEmpty()); + assertEquals(25, service.searchErrata(null, null, null, "advisorytoken", 25).size()); + } + + @Test + void identifiersAndVersionFiltersCannotBecomeQueries() throws Exception { + assertEquals(20, results(server.camel_docs_jira_lookup("CAMEL-999")).size()); + assertEquals(0, results(server.camel_docs_jira_lookup("CAMEL-999 OR *:*")).size()); + assertEquals(20, results(server.camel_docs_cve_search("CVE-2099-0001", null, null, null, 100)).size()); + assertEquals(0, results(server.camel_docs_cve_search("CVE-2099-0001 OR *:*", null, null, null, 100)).size()); + assertEquals(10, results(server.camel_docs_component_info("safecomponent", null, null)).size()); + assertEquals(0, results(server.camel_docs_component_info("id:secretmarker", null, null)).size()); + assertEquals(0, results(server.camel_docs_search("needle", "absent OR *:*", 25)).size()); + assertTrue(service.lookupComponent("apache_camel", "safecomponent OR *:*", null, null).isEmpty()); + } + + @Test + void everyConfigurableResultLimitIsBoundedAtTheToolEntryPoint() throws Exception { + assertEquals(25, results(server.camel_docs_search("needle", null, Integer.MAX_VALUE)).size()); + assertEquals(25, results(server.camel_docs_release_info("4.22", Integer.MAX_VALUE)).size()); + assertEquals(25, results(server.camel_docs_cve_search(null, null, null, null, Integer.MAX_VALUE)).size()); + assertEquals(1, results(server.camel_docs_search("needle", null, -1)).size()); + assertEquals(1, results(server.camel_docs_release_info("4.22", 0)).size()); + assertEquals(1, results(server.camel_docs_cve_search(null, null, null, null, -1)).size()); + assertEquals(5, results(server.camel_docs_search("needle", null, null)).size()); + assertEquals(20, results(server.camel_docs_release_info("4.22", null)).size()); + assertEquals(10, results(server.camel_docs_cve_search(null, null, null, null, null)).size()); + } + + @Test + void conventionalSeveritySpellingsReturnTheSameAdvisories() throws Exception { + for (String severity : List.of("hIgH", "HIGH", "high", "High")) { + JsonNode matches = results(server.camel_docs_cve_search(null, null, severity, null, 25)); + assertEquals(25, matches.size(), severity); + for (JsonNode match : matches) { + assertEquals("HIGH", match.path("severity").asText(), match.toString()); + } + } + } + + @Test + void endpointValidationDoesNotContactAnHttpEndpoint() throws Exception { + AtomicInteger requests = new AtomicInteger(); + HttpServer canary = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0); + canary.createContext("/canary", exchange -> { + requests.incrementAndGet(); + exchange.sendResponseHeaders(200, -1); + exchange.close(); + }); + canary.start(); + try { + String uri = "http://127.0.0.1:"; + canary.getAddress().getPort() + "/canary"; + JsonNode validation = mapper.readTree(server.camel_docs_validate_endpoint(uri)); + assertTrue(validation.path("valid").asBoolean(), validation.toString()); + assertEquals(uri, validation.path("uri").asText()); + JsonNode stylesheet = mapper.readTree(server.camel_docs_validate_endpoint("xslt:" + uri)); + assertTrue(stylesheet.path("valid").asBoolean(), stylesheet.toString()); + assertEquals(0, requests.get(), "Catalog validation must not send the endpoint a request"); + } finally { + canary.stop(0); + } + } + + @Test + void endpointValidationDoesNotRunCommandsOrModifyFiles() throws Exception { + Path marker = temp.resolve("executed"); + Path command = temp.resolve("BoundaryCanary.java"); + Files.writeString(command, "class BoundaryCanary { public static void main(String[] args) throws Exception {" + + "java.nio.file.Files.writeString(java.nio.file.Path.of(args[0]), \"executed\"); }}"); + String executable = Path.of(System.getProperty("java.home"), "bin", "java").toString(); + JsonNode exec = mapper.readTree(server.camel_docs_validate_endpoint( + "exec:" + executable + "?args=" + command + " " + marker)); + assertTrue(exec.path("valid").asBoolean(), exec.toString()); + assertFalse(Files.exists(marker), "The exec endpoint is metadata, not a command invocation"); + + Path secret = temp.resolve("private.xml"); + Files.writeString(secret, "<private>content-canary-unique</private>"); + String validation = server.camel_docs_validate_endpoint("xslt:" + secret.toUri()); + assertTrue(mapper.readTree(validation).path("valid").asBoolean(), validation); + assertFalse(validation.contains("content-canary-unique")); + assertEquals("<private>content-canary-unique</private>", Files.readString(secret)); + + Path output = temp.resolve("output"); + JsonNode file = mapper.readTree(server.camel_docs_validate_endpoint("file:" + output + "?fileName=result.txt")); + assertTrue(file.path("valid").asBoolean(), file.toString()); + assertFalse(Files.exists(output), "Catalog validation must not create the endpoint directory"); + } + + private JsonNode results(String response) throws Exception { + JsonNode json = mapper.readTree(response); + assertFalse(json.has("error"), response); + assertTrue(json.path("results").isArray(), response); + return json.path("results"); + } + + private static void setField(Object target, String name, Object value) throws Exception { + Field field = target.getClass().getDeclaredField(name); + field.setAccessible(true); + field.set(target, value); + } +} diff --git a/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/LuceneFallbackBoundaryTest.java b/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/LuceneFallbackBoundaryTest.java new file mode 100644 index 00000000..7b1b400e --- /dev/null +++ b/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/LuceneFallbackBoundaryTest.java @@ -0,0 +1,191 @@ +package org.apache.camel.kit.knowledge.mcp; + +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.lang.reflect.Field; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.attribute.PosixFileAttributeView; +import java.nio.file.attribute.PosixFilePermission; +import java.util.HashMap; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.camel.kit.knowledge.schema.KnowledgeDocument; + +import org.apache.lucene.analysis.standard.StandardAnalyzer; +import org.apache.lucene.index.DirectoryReader; +import org.apache.lucene.index.IndexNotFoundException; +import org.apache.lucene.index.IndexWriter; +import org.apache.lucene.index.IndexWriterConfig; +import org.apache.lucene.store.FSDirectory; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import static org.junit.jupiter.api.Assertions.*; + +class LuceneFallbackBoundaryTest { + + @TempDir + Path temp; + + @Test + void pinnedResolutionFailureNeverUsesAnAvailableClasspathIndex() throws Exception { + AtomicInteger resourceReads = new AtomicInteger(); + ClassLoader original = Thread.currentThread().getContextClassLoader(); + LuceneSearchService service = new LuceneSearchService(); + IOException failure = new IOException("Pinned index is unavailable"); + service.indexDownloader = failingDownloader(true, failure); + Thread.currentThread().setContextClassLoader(classpathIndex(resourceReads)); + try { + RuntimeException thrown = assertThrows(RuntimeException.class, service::init); + assertSame(failure, thrown.getCause()); + assertEquals(0, resourceReads.get(), "A pin must not silently select even a valid bundled index"); + } finally { + Thread.currentThread().setContextClassLoader(original); + service.close(); + } + } + + @Test + void unpinnedFallbackExtractsAReadablePrivateIndexAndRemovesItOnClose() throws Exception { + ClassLoader original = Thread.currentThread().getContextClassLoader(); + LuceneSearchService service = new LuceneSearchService(); + service.indexDownloader = failingDownloader(false, new IOException("Release unavailable")); + Thread.currentThread().setContextClassLoader(classpathIndex(new AtomicInteger())); + Path extracted = null; + try { + IndexDownloader.ResolvedIndex resolved = resolveIndex(service); + extracted = resolved.dir(); + // Keep model initialization out of this fixture; exercise the real resolved-index cleanup policy. + setResolvedIndex(service, resolved); + assertTrue(resolved.temporary()); + assertTrue(extracted.getFileName().toString().startsWith("knowledge-index")); + if (Files.getFileAttributeView(extracted, PosixFileAttributeView.class) != null) { + assertEquals(Set.of(PosixFilePermission.OWNER_READ, PosixFilePermission.OWNER_WRITE, + PosixFilePermission.OWNER_EXECUTE), Files.getPosixFilePermissions(extracted)); + } + try (FSDirectory directory = FSDirectory.open(extracted); + DirectoryReader reader = DirectoryReader.open(directory)) { + assertEquals(1, reader.numDocs()); + assertEquals("fallback-canary", reader.storedFields().document(0).get("id")); + } + } finally { + Thread.currentThread().setContextClassLoader(original); + service.close(); + } + assertNotNull(extracted); + assertFalse(Files.exists(extracted), "Only the extracted temporary index must be deleted on close"); + } + + @Test + void closingTheServicePreservesAnExplicitPersistentIndex() throws Exception { + Path persistent = Files.createDirectory(temp.resolve("persistent")); + Path sentinel = Files.writeString(persistent.resolve("sentinel"), "keep"); + LuceneSearchService service = new LuceneSearchService(); + service.indexDownloader = new IndexDownloader(null) { + @Override + public ResolvedIndex resolve() { + return new ResolvedIndex(persistent, false); + } + }; + IndexDownloader.ResolvedIndex resolved = resolveIndex(service); + assertEquals(persistent, resolved.dir()); + assertFalse(resolved.temporary()); + setResolvedIndex(service, resolved); + service.close(); + assertEquals("keep", Files.readString(sentinel)); + } + + @Test + void missingBundledIndexCannotTurnFailedAcquisitionIntoSuccessfulStartup() throws Exception { + ClassLoader original = Thread.currentThread().getContextClassLoader(); + LuceneSearchService service = new LuceneSearchService(); + service.indexDownloader = failingDownloader(false, new IOException("Release unavailable")); + Thread.currentThread().setContextClassLoader(new ClassLoader(original) { + @Override + public InputStream getResourceAsStream(String name) { + return name.startsWith("knowledge-index/") ? null : super.getResourceAsStream(name); + } + }); + Path extracted; + try { + RuntimeException thrown = assertThrows(RuntimeException.class, service::init); + assertInstanceOf(IndexNotFoundException.class, thrown.getCause()); + Field field = LuceneSearchService.class.getDeclaredField("resolvedIndex"); + field.setAccessible(true); + extracted = ((IndexDownloader.ResolvedIndex) field.get(service)).dir(); + } finally { + Thread.currentThread().setContextClassLoader(original); + service.close(); + } + assertFalse(Files.exists(extracted)); + } + + private ClassLoader classpathIndex(AtomicInteger resourceReads) throws Exception { + Path source = Files.createTempDirectory(temp, "bundled-"); + try (FSDirectory directory = FSDirectory.open(source); + StandardAnalyzer analyzer = new StandardAnalyzer(); + IndexWriter writer = new IndexWriter(directory, new IndexWriterConfig(analyzer))) { + writer.addDocument(new KnowledgeDocument("fallback-canary", "apache_camel") + .docType("guide").content("bundled fallback").build()); + } + Map<String, byte[]> resources = new HashMap<>(); + StringBuilder manifest = new StringBuilder(); + try (var files = Files.list(source)) { + for (Path file : files.filter(path -> !path.getFileName().toString().equals("write.lock")).toList()) { + String name = file.getFileName().toString(); + manifest.append(name).append('\n'); + resources.put("knowledge-index/" + name, Files.readAllBytes(file)); + } + } + resources.put("knowledge-index/INDEX_FILES", manifest.toString().getBytes(StandardCharsets.UTF_8)); + return new ClassLoader(Thread.currentThread().getContextClassLoader()) { + @Override + public InputStream getResourceAsStream(String name) { + if (name.startsWith("knowledge-index/")) { + resourceReads.incrementAndGet(); + byte[] bytes = resources.get(name); + return bytes != null ? new ByteArrayInputStream(bytes) : null; + } + return super.getResourceAsStream(name); + } + }; + } + + private static IndexDownloader failingDownloader(boolean pinned, IOException failure) { + return new IndexDownloader(null) { + @Override + public ResolvedIndex resolve() throws IOException { + throw failure; + } + + @Override + boolean isVersionPinned() { + return pinned; + } + }; + } + + private static IndexDownloader.ResolvedIndex resolveIndex(LuceneSearchService service) throws Exception { + Method method = LuceneSearchService.class.getDeclaredMethod("resolveIndex"); + method.setAccessible(true); + try { + return (IndexDownloader.ResolvedIndex) method.invoke(service); + } catch (InvocationTargetException e) { + throw (Exception) e.getCause(); + } + } + + private static void setResolvedIndex(LuceneSearchService service, IndexDownloader.ResolvedIndex resolved) + throws Exception { + Field field = LuceneSearchService.class.getDeclaredField("resolvedIndex"); + field.setAccessible(true); + field.set(service, resolved); + } +} diff --git a/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/OfflineModelsTest.java b/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/OfflineModelsTest.java new file mode 100644 index 00000000..594a91f0 --- /dev/null +++ b/knowledge/mcp/src/test/java/org/apache/camel/kit/knowledge/mcp/OfflineModelsTest.java @@ -0,0 +1,222 @@ +package org.apache.camel.kit.knowledge.mcp; + +import java.io.IOException; +import java.net.HttpURLConnection; +import java.net.Proxy; +import java.net.URL; +import java.net.URLConnection; +import java.net.URLStreamHandler; +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Duration; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.TimeUnit; + +import org.apache.camel.kit.knowledge.embedding.OnnxEmbeddingProvider; +import org.apache.camel.kit.knowledge.embedding.OnnxReranker; + +import jdk.jfr.Recording; +import jdk.jfr.consumer.RecordingFile; +import org.junit.jupiter.api.io.TempDir; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.CsvSource; +import org.junit.jupiter.params.provider.ValueSource; + +import static org.junit.jupiter.api.Assertions.*; + +class OfflineModelsTest { + + @TempDir + Path tempDir; + + @ParameterizedTest + @ValueSource(strings = {"embedding", "reranker"}) + void bundledModelsInitializeOfflineInEitherOrder(String firstModel) throws Exception { + runProbe(firstModel, "", ""); + } + + @ParameterizedTest + @CsvSource({ + "embedding, DJL_OFFLINE, false", + "reranker, DJL_OFFLINE, false", + "embedding, RUST_FLAVOR, cu122", + "reranker, RUST_FLAVOR, cu122" + }) + void conflictingEnvironmentIsRejectedBeforeNativeInitialization(String firstModel, String name, String value) + throws Exception { + runProbe(firstModel, name, value); + } + + private void runProbe(String firstModel, String environmentName, String environmentValue) throws Exception { + Path home = Files.createDirectory(tempDir.resolve("home")); + Path temporary = Files.createDirectory(tempDir.resolve("tmp")); + Path cache = tempDir.resolve("cache"); + Path output = tempDir.resolve("probe.log"); + ProcessBuilder builder = new ProcessBuilder( + Path.of(System.getProperty("java.home"), "bin", + System.getProperty("os.name").startsWith("Windows") ? "java.exe" : "java").toString(), + "-Duser.home=" + home, + "-Djava.io.tmpdir=" + temporary, + "-Donnx.threads=1", + "-Dai.djl.offline=false", + "-DRUST_FLAVOR=cu122", + "-Dai.djl.util.cuda.fork=true", + "-XX:ActiveProcessorCount=2", + "-cp", System.getProperty("surefire.test.class.path", System.getProperty("java.class.path")), + Probe.class.getName(), firstModel, environmentName, tempDir.toString()); + var environment = builder.environment(); + for (String name : List.of("DJL_OFFLINE", "RUST_FLAVOR", "RUST_LIBRARY_PATH", "DJL_CACHE_DIR", + "ENGINE_CACHE_DIR", "JAVA_TOOL_OPTIONS", "JDK_JAVA_OPTIONS", "_JAVA_OPTIONS")) { + environment.remove(name); + } + environment.put("DJL_CACHE_DIR", cache.toString()); + environment.put("ENGINE_CACHE_DIR", cache.toString()); + environment.put("TMPDIR", temporary.toString()); + environment.put("TMP", temporary.toString()); + environment.put("TEMP", temporary.toString()); + if (!environmentName.isEmpty()) { + environment.put(environmentName, environmentValue); + } + Process child = builder.redirectErrorStream(true).redirectOutput(output.toFile()).start(); + try { + assertTrue(child.waitFor(120, TimeUnit.SECONDS), () -> "Model probe timed out: " + readOutput(output)); + assertEquals(0, child.exitValue(), () -> readOutput(output)); + assertTrue(readOutput(output).contains("OFFLINE_MODELS_PASS"), () -> readOutput(output)); + } finally { + if (child.isAlive()) { + child.destroyForcibly(); + child.waitFor(10, TimeUnit.SECONDS); + } + } + } + + private static String readOutput(Path output) { + try { + return Files.readString(output); + } catch (IOException e) { + return e.toString(); + } + } + + public static class Probe { + public static void main(String[] args) throws Exception { + List<String> networkOpens = new ArrayList<>(); + installBlockingNetworkObserver(networkOpens); + Path directory = Path.of(args[2]); + Path recordingPath = directory.resolve("models.jfr"); + Throwable failure = null; + try (Recording recording = new Recording()) { + recording.enable("jdk.ProcessStart").withThreshold(Duration.ZERO).withStackTrace(); + recording.enable("jdk.SocketRead").withThreshold(Duration.ZERO); + recording.enable("jdk.SocketWrite").withThreshold(Duration.ZERO); + recording.start(); + try { + if (args[1].isEmpty()) { + exercise(args[0]); + exercise(args[0].equals("embedding") ? "reranker" : "embedding"); + assertEquals("true", System.getProperty("ai.djl.offline")); + assertEquals("cpu", System.getProperty("RUST_FLAVOR")); + assertEquals("false", System.getProperty("ai.djl.util.cuda.fork")); + } else { + IllegalStateException rejection + = assertThrows(IllegalStateException.class, () -> exercise(args[0])); + assertTrue(rejection.getMessage().contains(args[1]), rejection::getMessage); + assertFalse(Files.exists(directory.resolve("cache")), + "Rejection must precede native extraction"); + try (var temporaryFiles = Files.list(directory.resolve("tmp"))) { + assertTrue( + temporaryFiles.noneMatch( + path -> path.getFileName().toString().startsWith("camel-kit-onnx-")), + "Rejection must precede model extraction"); + } + } + } catch (Throwable e) { + failure = e; + } finally { + recording.stop(); + recording.dump(recordingPath); + } + } + System.out.println("HTTP opens: " + networkOpens.size() + " " + networkOpens); + assertTrue(networkOpens.isEmpty(), + () -> "Tokenizer initialization attempted network access: " + networkOpens); + var events = RecordingFile.readAllEvents(recordingPath); + var processes = events.stream().filter(event -> event.getEventType().getName().equals("jdk.ProcessStart")) + .toList(); + assertTrue(processes.isEmpty(), () -> "Model initialization started child processes: " + processes); + assertFalse(events.stream().anyMatch(event -> event.getEventType().getName().startsWith("jdk.Socket")), + "Model initialization must not read or write sockets"); + if (failure != null) { + throw new AssertionError("Model initialization contract failed", failure); + } + System.out.println("OFFLINE_MODELS_PASS"); + } + + private static void exercise(String model) { + if (model.equals("embedding")) { + float[] vector = new OnnxEmbeddingProvider().embed("Apache Camel routes messages between endpoints"); + assertEquals(OnnxEmbeddingProvider.DIMENSIONS, vector.length); + double norm = 0; + for (float value : vector) { + assertTrue(Float.isFinite(value)); + norm += value * value; + } + assertEquals(1.0, norm, 0.0001, "Embedding must be a real normalized vector"); + } else { + float score = new OnnxReranker().score("Apache Camel routes", + "Apache Camel routes messages between endpoints"); + assertTrue(Float.isFinite(score) && score > 0 && score < 1, + "Reranker must return a real relevance score"); + } + } + + private static void installBlockingNetworkObserver(List<String> networkOpens) { + URL.setURLStreamHandlerFactory(protocol -> { + if (!protocol.equals("http") && !protocol.equals("https")) { + return null; + } + return new URLStreamHandler() { + @Override + protected URLConnection openConnection(URL url) { + return blocked(url); + } + + @Override + protected URLConnection openConnection(URL url, Proxy proxy) { + return blocked(url); + } + + private URLConnection blocked(URL url) { + networkOpens.add(url.toString()); + return new HttpURLConnection(url) { + @Override + public void disconnect() { + } + + @Override + public boolean usingProxy() { + return false; + } + + @Override + public void connect() throws IOException { + throw new IOException("Network disabled by offline-model test"); + } + + @Override + public int getResponseCode() { + return 404; + } + + @Override + public String getResponseMessage() { + return "Network disabled by offline-model test"; + } + }; + } + }; + }); + } + } +} diff --git a/tools/FixtureIndex.java b/tools/FixtureIndex.java new file mode 100644 index 00000000..1add05d6 --- /dev/null +++ b/tools/FixtureIndex.java @@ -0,0 +1,21 @@ +import java.nio.file.Path; +import org.apache.camel.kit.knowledge.schema.KnowledgeDocument; +import org.apache.lucene.analysis.standard.StandardAnalyzer; +import org.apache.lucene.index.IndexWriter; +import org.apache.lucene.index.IndexWriterConfig; +import org.apache.lucene.store.FSDirectory; + +public class FixtureIndex { + public static void main(String[] args) throws Exception { + try (var directory = FSDirectory.open(Path.of(args[0])); + var writer = new IndexWriter(directory, new IndexWriterConfig(new StandardAnalyzer()))) { + writer.addDocument(new KnowledgeDocument("runtime-canary", "apache_camel") + .source("local-canary").docType("guide").sourceVersion("4.22") + .sectionTitle("Apache Camel timer component") + .content("The Apache Camel timer component generates periodic messages. " + + "Use a timer endpoint to trigger a route at regular intervals. " + + "The period option configures the interval between generated messages.") + .build()); + } + } +} diff --git a/tools/verify_knowledge_runtime.py b/tools/verify_knowledge_runtime.py new file mode 100644 index 00000000..36b6b57c --- /dev/null +++ b/tools/verify_knowledge_runtime.py @@ -0,0 +1,311 @@ +#!/usr/bin/env python3 +"""Linux packaged Knowledge stdio/SSE boundary check with a synthetic local index.""" +import argparse +import hashlib +import http.client +import json +import os +from pathlib import Path +import queue +import re +import shutil +import signal +import socket +import subprocess +import threading +import time + + +def sockets(pid): + inodes = set() + for fd in Path(f"/proc/{pid}/fd").iterdir(): + try: + target = fd.readlink().as_posix() + except FileNotFoundError: + continue + if target.startswith("socket:["): + inodes.add(target[8:-1]) + found = [] + for protocol in ["tcp", "tcp6", "udp", "udp6"]: + for line in Path(f"/proc/{pid}/net/{protocol}").read_text().splitlines()[1:]: + fields = line.split() + if fields[9] in inodes: + found.append({"protocol": protocol, "local": fields[1], "remote": fields[2], + "state": fields[3], "inode": fields[9]}) + return found + + +def syscall_evidence(case, mode, baseline_offline): + """Inspect the whole child lifetime, including transient startup activity.""" + lines = (case / "syscalls.log").read_text().splitlines() + execs = [line for line in lines if " execve(" in line or " execveat(" in line] + connects = [line for line in lines if " connect(" in line] + inet_connects = [line for line in connects if "sa_family=AF_INET" in line] + service_connects = [line for line in inet_connects if not re.search(r"sin6?_port=htons\(0\)", line)] + inet_sends = [line for line in lines if re.search(r" (sendto|sendmsg|sendmmsg)\(", line) + and ("<UDP" in line or "<TCP" in line)] + inet_binds = [line for line in lines if " bind(" in line and "sa_family=AF_INET" in line] + listeners = [line for line in lines if " listen(" in line] + evidence = {"exec_calls": execs, "inet_connect_calls": inet_connects, + "inet_service_connect_calls": service_connects, + "inet_send_calls": inet_sends, "inet_bind_calls": inet_binds, + "listen_calls": listeners, + "note": "UDP port-zero connects inspect local interface routing; they do not send payloads. Netlink interface enumeration is recorded in the full trace."} + (case / "syscall-summary.json").write_text(json.dumps(evidence, indent=2) + "\n") + assert not service_connects and not inet_sends, evidence + if mode == "stdio": + assert not inet_binds and not listeners, evidence + else: + assert len(inet_binds) == 1 and len(listeners) == 1, evidence + if not baseline_offline: + assert len(execs) == 1, evidence # The initial JVM, without any child executable. + return evidence + + +def wait_item(items, description, timeout=60): + try: + result = items.get(timeout=timeout) + except queue.Empty: + raise AssertionError(f"Timed out waiting for {description}") from None + if isinstance(result, Exception): + raise result + return result + + +def request(method, id=None, params=None): + value = {"jsonrpc": "2.0", "method": method} + if id is not None: + value["id"] = id + if params is not None: + value["params"] = params + return value + + +def init(): + return request("initialize", 1, {"protocolVersion": "2024-11-05", "capabilities": {}, + "clientInfo": {"name": "local-boundary-canary", "version": "1"}}) + + +class Runner: + def __init__(self, args, mode, port): + self.case = args.output / mode + self.case.mkdir() + for name in ["tmp", "cache", "djl", "engine", "home"]: + (self.case / name).mkdir() + self.events, self.snapshots = [], [] + self.stdout = queue.Queue() + env = {key: value for key, value in os.environ.items() if key in ["PATH", "LANG", "LC_ALL", "TZ"]} + env.update({"DJL_CACHE_DIR": str(self.case / "djl"), "ENGINE_CACHE_DIR": str(self.case / "engine"), + "OMP_NUM_THREADS": "1"}) + java_cmd = [str(args.java), "-XX:ActiveProcessorCount=2", "-Donnx.threads=1", + "-Duser.home=" + str(self.case / "home"), + "-Djava.io.tmpdir=" + str(self.case / "tmp"), + "-Dknowledge.index.path=" + str(args.output / "index"), + "-Dknowledge.index.cache-dir=" + str(self.case / "cache")] + if args.baseline_offline: + java_cmd.append("-Dai.djl.offline=true") + if mode == "sse": + java_cmd += ["-Dquarkus.http.host-enabled=true", "-Dquarkus.http.host=127.0.0.1", + f"-Dquarkus.http.port={port}"] + java_cmd += ["-jar", str(args.jar)] + self.cmd = ["strace", "-f", "-s", "256", "-yy", "-e", "trace=%network,%process", + "-o", str(self.case / "syscalls.log"), *java_cmd] + self.stderr = (self.case / "stderr.log").open("w") + self.proc = subprocess.Popen(self.cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, + stderr=self.stderr, text=True, env=env, cwd=self.case, + start_new_session=True) + def read(): + for line in self.proc.stdout: + try: + value = json.loads(line) + except json.JSONDecodeError: + value = {"stdout_text": line.rstrip()} + self.stdout.put(value) + threading.Thread(target=read, daemon=True).start() + deadline = time.monotonic() + 10 + while time.monotonic() < deadline: + children = Path(f"/proc/{self.proc.pid}/task/{self.proc.pid}/children").read_text().split() + if children: + self.pid = int(children[0]) + break + time.sleep(0.05) + else: + raise AssertionError("No traced JVM child") + + def capture(self, phase): + self.snapshots.append({"phase": phase, "sockets": sockets(self.pid)}) + + def send(self, message): + self.proc.stdin.write(json.dumps(message) + "\n") + self.proc.stdin.flush() + + def stdio_result(self, id): + deadline = time.monotonic() + 60 + while time.monotonic() < deadline: + item = wait_item(self.stdout, f"stdio response {id}", deadline - time.monotonic()) + self.events.append(item) + if item.get("id") == id: + assert "error" not in item, item + return item["result"] + raise AssertionError(f"Missing stdio result {id}") + + def close(self): + if self.proc.poll() is None: + self.proc.stdin.close() + try: + self.proc.wait(timeout=10) + except subprocess.TimeoutExpired: + os.killpg(self.proc.pid, signal.SIGTERM) + try: + self.proc.wait(timeout=10) + except subprocess.TimeoutExpired: + os.killpg(self.proc.pid, signal.SIGKILL) + self.proc.wait() + self.stderr.close() + (self.case / "events.json").write_text(json.dumps(self.events, indent=2) + "\n") + (self.case / "sockets.json").write_text(json.dumps(self.snapshots, indent=2) + "\n") + (self.case / "command.json").write_text(json.dumps(self.cmd, indent=2) + "\n") + + +def tool_content(result): + assert not result.get("isError"), result + content = result["content"] + assert len(content) == 1 and content[0]["type"] == "text", result + value = json.loads(content[0]["text"]) + assert "error" not in value, value + return value + + +def stdio(args): + runner = Runner(args, "stdio", None) + try: + runner.send(init()) + initialized = runner.stdio_result(1) + runner.capture("initialized") + runner.send(request("notifications/initialized")) + runner.send(request("tools/list", 2)) + tools = runner.stdio_result(2) + assert "camel_docs_search" in [tool["name"] for tool in tools["tools"]] + runner.capture("tools-listed") + runner.send(request("tools/call", 3, {"name": "camel_docs_search", "arguments": { + "query": "Apache Camel timer component generates periodic messages", "max_results": 1}})) + query = tool_content(runner.stdio_result(3)) + assert "runtime-canary" in json.dumps(query), query + runner.capture("first-query-completed") + assert not any(snapshot["sockets"] for snapshot in runner.snapshots), runner.snapshots + return {"pid": runner.pid, "initialization": initialized, "tool_count": len(tools["tools"]), + "query": query, "snapshots": runner.snapshots} + finally: + runner.close() + + +def sse(args): + with socket.socket() as reservation: + reservation.bind(("127.0.0.1", 0)) + port = reservation.getsockname()[1] + runner = Runner(args, "sse", port) + conn = None + try: + deadline = time.monotonic() + 20 + while time.monotonic() < deadline: + try: + with socket.create_connection(("127.0.0.1", port), timeout=0.2): + break + except OSError: + time.sleep(0.1) + else: + raise AssertionError("HTTP listener did not start") + runner.capture("http-started") + listeners = [s for s in runner.snapshots[-1]["sockets"] if s["state"] == "0A"] + expected_port = f"{port:04X}" + assert len(listeners) == 1 and listeners[0]["local"].endswith(":" + expected_port), listeners + assert listeners[0]["local"].split(":")[0] in ["0100007F", "0000000000000000FFFF00000100007F"], listeners + conn = http.client.HTTPConnection("127.0.0.1", port, timeout=60) + conn.request("GET", "/mcp/sse", headers={"Accept": "text/event-stream"}) + response = conn.getresponse() + assert response.status == 200, (response.status, response.read()) + events = queue.Queue() + def read_sse(): + event, data = None, [] + try: + while line := response.readline(): + text = line.decode().rstrip("\r\n") + if not text: + if data: + events.put({"event": event, "data": "\n".join(data)}) + event, data = None, [] + elif text.startswith("event:"): + event = text[6:].strip() + elif text.startswith("data:"): + data.append(text[5:].strip()) + except Exception as e: + events.put(e) + threading.Thread(target=read_sse, daemon=True).start() + endpoint_event = wait_item(events, "SSE endpoint") + assert endpoint_event["event"] == "endpoint", endpoint_event + endpoint = endpoint_event["data"] + assert endpoint.startswith("/") and not endpoint.startswith("//"), endpoint + runner.events.append(endpoint_event) + def post(message): + client = http.client.HTTPConnection("127.0.0.1", port, timeout=60) + client.request("POST", endpoint, body=json.dumps(message), headers={"Content-Type": "application/json"}) + reply = client.getresponse() + status = reply.status + body = reply.read().decode() + client.close() + assert status in [200, 202, 204], (status, body) + if "id" not in message: + return + while True: + event = wait_item(events, "SSE response") + runner.events.append(event) + value = json.loads(event["data"]) + if value.get("id") == message["id"]: + assert "error" not in value, value + return value["result"] + initialized = post(init()) + post(request("notifications/initialized")) + tools = post(request("tools/list", 2)) + query = tool_content(post(request("tools/call", 3, {"name": "camel_docs_search", "arguments": { + "query": "Apache Camel timer component generates periodic messages", "max_results": 1}}))) + assert "runtime-canary" in json.dumps(query), query + runner.capture("unauthenticated-query-completed") + return {"pid": runner.pid, "port": port, "initialization": initialized, "tool_count": len(tools["tools"]), + "query": query, "snapshots": runner.snapshots, "authorization_header_sent": False} + finally: + runner.close() + if conn: + conn.close() + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--jar", type=Path, required=True) + parser.add_argument("--output", type=Path, required=True) + parser.add_argument("--java", type=Path, default=Path(shutil.which("java") or "java")) + parser.add_argument("--baseline-offline", action="store_true") + args = parser.parse_args() + args.jar = args.jar.resolve() + args.java = args.java.resolve() + args.output = args.output.resolve() + args.output.mkdir(parents=True, exist_ok=False) + source = Path(__file__).with_name("FixtureIndex.java") + subprocess.run([str(args.java.with_name("javac")), "-cp", str(args.jar), "-d", str(args.output), str(source)], check=True) + subprocess.run([str(args.java), "-cp", str(args.output) + os.pathsep + str(args.jar), "FixtureIndex", + str(args.output / "index")], check=True) + with args.jar.open("rb") as jar: + jar_hash = hashlib.file_digest(jar, "sha256").hexdigest() + result = {"jar": str(args.jar), "jar_sha256": jar_hash, + "java": subprocess.run([str(args.java), "-version"], capture_output=True, text=True).stderr, + "baseline_offline": args.baseline_offline} + result["stdio"] = stdio(args) + result["stdio"]["syscalls"] = syscall_evidence(args.output / "stdio", "stdio", args.baseline_offline) + print("stdio PASS", flush=True) + result["sse"] = sse(args) + result["sse"]["syscalls"] = syscall_evidence(args.output / "sse", "sse", args.baseline_offline) + print("sse PASS", flush=True) + (args.output / "result.json").write_text(json.dumps(result, indent=2) + "\n") + + +if __name__ == "__main__": + main()
