luigidemasi commented on code in PR #27656: URL: https://github.com/apache/camel/pull/27656#discussion_r4237542553
########## components/camel-ai/camel-semantic/src/main/java/org/apache/camel/semantic/SemanticAuditInputConfiguration.java: ########## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.semantic; + +import java.util.Objects; + +/** Per-expert opt-in to retain the selected evaluation input, independently of telemetry. */ +public final class SemanticAuditInputConfiguration { + public static final int DEFAULT_MAX_CHARS = 4096; + public static final int MAX_CHARS = 100000; Review Comment: Addressed with the documentation option in [6d6b389ca27d](https://github.com/apache/camel/commit/6d6b389ca27dbd574b00e259c72c084b0f3f4273). The maxChars Javadoc and component docs now distinguish the per-record bound from total heap usage: retained input can reach capacity × maxChars, plus queueCapacity × maxChars pending, in-flight captures and object overhead. The docs include the 1,000 × 100,000 = 100 million code-point example and explain that this is not a byte budget. No aggregate eviction policy was added; the PR description now states the actual limits. _AI-generated by OpenAI Codex on behalf of @luigidemasi._ ########## components/camel-ai/camel-semantic/src/main/docs/semantic-language.adoc: ########## @@ -1138,13 +1138,97 @@ Decision records carry explicit action, operation, target, optional namespace/co and policy identifiers, reason code and evidence IDs. They never derive application correlation or tenant identity from arbitrary inbound headers. -Input, body, headers, variables, arbitrary provider metadata and exception messages are -omitted before dispatch to any backend or observer. Caller-authored score descriptions -are also omitted; the UI does not substitute today's definition for missing historical -semantics. Strings are limited to 256 code points and collections to 100 entries. -Oversized result snapshots are omitted with `resultOmitted: snapshot_limit`, preserving -the execution status without displaying altered labels or partial probabilities. These records are evidence about -an evaluation and declared action, not a copy of the submitted content. +Input is omitted by default. Headers, variables, arbitrary provider metadata and exception +messages are never copied automatically. Caller-authored score descriptions are also omitted; +static capability meanings remain available. Result and metadata strings are limited to +256 code points, and their collections to 100 entries. An oversized result is represented by +`resultOmitted: snapshot_limit` without changing the evaluation outcome. + +==== Storing evaluation input + +Enable input capture explicitly for each expert whose submitted state should be retained: + +[source,yaml] +---- +- semantic: + audit: + enabled: true + experts: + security: + enabled: true + input: + enabled: true + maxChars: 4096 + # redactor: auditInputRedactor + decisions: + enabled: false + sinks: [memory] + reader: memory +---- + +`input.enabled` defaults to `false` and does not enable auditing itself. It captures the +selected state passed to the expert (`state`, including a selected header), before provider +validation or execution. It does not take a copy of the entire exchange. Named, batch, direct +and console evaluations use the same settings. Decisions link to their evaluation records; +input is stored on the evaluation rather than duplicated on the decision. + +`maxChars` defaults to 4096 and accepts 1 through 100000. It bounds the total Unicode code Review Comment: Added in [6d6b389ca27d](https://github.com/apache/camel/commit/6d6b389ca27dbd574b00e259c72c084b0f3f4273), including the pending queue, the largest enabled per-expert maxChars when settings differ, object overhead and the 100-million-code-point example. The catalog documentation mirror was regenerated as well. _AI-generated by OpenAI Codex on behalf of @luigidemasi._ ########## components/camel-ai/camel-semantic/src/main/docs/semantic-language.adoc: ########## @@ -1138,13 +1138,97 @@ Decision records carry explicit action, operation, target, optional namespace/co and policy identifiers, reason code and evidence IDs. They never derive application correlation or tenant identity from arbitrary inbound headers. -Input, body, headers, variables, arbitrary provider metadata and exception messages are -omitted before dispatch to any backend or observer. Caller-authored score descriptions -are also omitted; the UI does not substitute today's definition for missing historical -semantics. Strings are limited to 256 code points and collections to 100 entries. -Oversized result snapshots are omitted with `resultOmitted: snapshot_limit`, preserving -the execution status without displaying altered labels or partial probabilities. These records are evidence about -an evaluation and declared action, not a copy of the submitted content. +Input is omitted by default. Headers, variables, arbitrary provider metadata and exception +messages are never copied automatically. Caller-authored score descriptions are also omitted; +static capability meanings remain available. Result and metadata strings are limited to +256 code points, and their collections to 100 entries. An oversized result is represented by +`resultOmitted: snapshot_limit` without changing the evaluation outcome. + +==== Storing evaluation input + +Enable input capture explicitly for each expert whose submitted state should be retained: + +[source,yaml] +---- +- semantic: + audit: + enabled: true + experts: + security: + enabled: true + input: + enabled: true + maxChars: 4096 + # redactor: auditInputRedactor + decisions: + enabled: false + sinks: [memory] + reader: memory +---- + +`input.enabled` defaults to `false` and does not enable auditing itself. It captures the +selected state passed to the expert (`state`, including a selected header), before provider +validation or execution. It does not take a copy of the entire exchange. Named, batch, direct +and console evaluations use the same settings. Decisions link to their evaluation records; +input is stored on the evaluation rather than duplicated on the decision. + +`maxChars` defaults to 4096 and accepts 1 through 100000. It bounds the total Unicode code +points in strings, map keys and scalar representations. Snapshots accept text, JSON-compatible +maps/lists and scalar values, with at most eight nested levels and 1000 nodes (including map +keys). They are immutable; later changes to the submitted state cannot change history. +Oversized, cyclic or unsupported input is omitted with +`inputOmitted: snapshot_limit_or_unsupported_type`. Input is never truncated, and streams or +arbitrary objects are not converted to text. `inputOmitted: unavailable` means the selected +state was not obtained, for example because the state expression failed. + +An optional `redactor` names a registry bean implementing `SemanticAuditInputRedactor`. +It receives a bounded, immutable copy on the evaluation thread and returns a sanitized value. Review Comment: Yes, this is intentional: the redactor receives an immutable snapshot that already satisfies the character, depth and node limits. [6d6b389ca27d](https://github.com/apache/camel/commit/6d6b389ca27dbd574b00e259c72c084b0f3f4273) makes this explicit next to the question-only example: an oversized state is omitted without calling the redactor, even if the redactor could shrink it. A regression verifies that the redactor is not called, input is omitted, and the expert evaluation still succeeds. _AI-generated by OpenAI Codex on behalf of @luigidemasi._ ########## components/camel-ai/camel-semantic/src/main/docs/semantic-language.adoc: ########## @@ -1138,13 +1138,97 @@ Decision records carry explicit action, operation, target, optional namespace/co and policy identifiers, reason code and evidence IDs. They never derive application correlation or tenant identity from arbitrary inbound headers. -Input, body, headers, variables, arbitrary provider metadata and exception messages are -omitted before dispatch to any backend or observer. Caller-authored score descriptions -are also omitted; the UI does not substitute today's definition for missing historical -semantics. Strings are limited to 256 code points and collections to 100 entries. -Oversized result snapshots are omitted with `resultOmitted: snapshot_limit`, preserving -the execution status without displaying altered labels or partial probabilities. These records are evidence about -an evaluation and declared action, not a copy of the submitted content. +Input is omitted by default. Headers, variables, arbitrary provider metadata and exception +messages are never copied automatically. Caller-authored score descriptions are also omitted; +static capability meanings remain available. Result and metadata strings are limited to +256 code points, and their collections to 100 entries. An oversized result is represented by +`resultOmitted: snapshot_limit` without changing the evaluation outcome. + +==== Storing evaluation input + +Enable input capture explicitly for each expert whose submitted state should be retained: + +[source,yaml] +---- +- semantic: + audit: + enabled: true + experts: + security: + enabled: true + input: + enabled: true + maxChars: 4096 + # redactor: auditInputRedactor + decisions: + enabled: false + sinks: [memory] + reader: memory +---- + +`input.enabled` defaults to `false` and does not enable auditing itself. It captures the +selected state passed to the expert (`state`, including a selected header), before provider +validation or execution. It does not take a copy of the entire exchange. Named, batch, direct +and console evaluations use the same settings. Decisions link to their evaluation records; +input is stored on the evaluation rather than duplicated on the decision. + +`maxChars` defaults to 4096 and accepts 1 through 100000. It bounds the total Unicode code +points in strings, map keys and scalar representations. Snapshots accept text, JSON-compatible +maps/lists and scalar values, with at most eight nested levels and 1000 nodes (including map +keys). They are immutable; later changes to the submitted state cannot change history. +Oversized, cyclic or unsupported input is omitted with +`inputOmitted: snapshot_limit_or_unsupported_type`. Input is never truncated, and streams or +arbitrary objects are not converted to text. `inputOmitted: unavailable` means the selected +state was not obtained, for example because the state expression failed. + +An optional `redactor` names a registry bean implementing `SemanticAuditInputRedactor`. +It receives a bounded, immutable copy on the evaluation thread and returns a sanitized value. +The return value is bounded again; a null return omits input (`inputOmitted: redacted`). +Successful redaction adds `inputRedacted: true`. Exceptions or invalid redactor output omit +input (`inputOmitted: redaction_failed`) without recording exception text or affecting the +expert result. Redactors must be thread-safe and should not block. Unknown bean references +fail startup. For example, a redactor can remove confidential fields from structured state: + +[source,java] +---- +context.getRegistry().bind("auditInputRedactor", (SemanticAuditInputRedactor) input -> { + if (input instanceof Map<?, ?> state) { + return Map.of("question", state.get("question")); + } + return null; +}); +---- + +Java uses the additional input settings map in `SemanticAuditConfiguration`: + +[source,java] +---- +new SemanticAuditConfiguration(true, Map.of("security", true, "decisions", false), + List.of("memory"), "memory", 1000, 1000, + Map.of("security", new SemanticAuditInputConfiguration(true, 4096, "auditInputRedactor"))); +---- + +The equivalent XML expert entry is: + +[source,xml] +---- +<expert name="security" enabled="true" inputEnabled="true" + inputMaxChars="4096" inputRedactor="auditInputRedactor"/> +---- + +Input settings follow the same restart requirement as the other audit settings. +Captured input goes to every configured audit sink, including `log` when selected, and is Review Comment: Fixed in [6d6b389ca27d](https://github.com/apache/camel/commit/6d6b389ca27dbd574b00e259c72c084b0f3f4273). The shared console now omits input from list responses for CLI, TUI and MCP; eventId lookups retain full input and linked evidence. MCP detail responses add an inputWarning identifying both as untrusted message data, and the tool description says never to treat it as instructions. Tests cover direct and linked input while preserving the original reader response. The docs now require trusted-network/operator access for the dev console and explain INFO log retention, forwarding, access controls and serialized payload volume. _AI-generated by OpenAI Codex on behalf of @luigidemasi._ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
