davsclaus opened a new pull request, #27998: URL: https://github.com/apache/camel/pull/27998
## What The AI tools that write and edit files (`camel_write_file`, `camel_edit_file`) become more lenient with what a model sends, local models in particular: - **Repairs.** A YAML shape that a model often gets wrong but that has only one reading is repaired before the write is refused for it: an option at the column of its EIP (`steps:` next to `split:`), a plain value where an expression goes, a list where a map goes and the other way round, a list item or key one or two columns off, an unquoted placeholder, an enum value in another case, and the names models use for an option that has another name. Each repair is driven by the validator's message for that shape and changes only the lines it names; a name is never guessed by similarity. The result is validated again and kept only when no new problem is left, and the answer lists the repairs. - **Closest-line match.** Lines to find that are not in the file as given (a model writes them from memory) are matched to the closest lines of the file when exactly one place matches well. The lines the model changed between find and replace are changed there, and the other lines stay as the file has them. Such an edit stands only when its result is valid; otherwise the answer is the one of a miss, with the lines the file has there. The system property `camel.jbang.lenientWrites=false` turns both off. The docs and a 4.24 upgrade note are included. ## Why: the local-model benchmark Measured with qwen3.6 35B-A3B against camel-jbang-mcp, with the harness in apache/camel-performance-tests (`ai-benchmark`): - Overnight A/B runs on the full ladder (21 examples, 71 steps), Oct 10-12: about the same pass rate (333 against 328 of 355, and 261 against 264 of 284), with about 30% fewer refused writes and 38-47% fewer missed finds. Two flaws of the closest-line match showed up there, and the last code commit fixes them: a window that ended one line short left the old line in the file a second time, and after a refused edit the match hid that the model's copy of the file was out of date. - The loop with `steps:` written next to `split:` costs main steps every night: on Oct 16-17 it caused 82 refused writes and two steps at the call limit in the five runs on main. - A targeted A/B on Oct 17, after the fixes, on the five examples where these shapes and snippets occur, 3 runs each: | | main | this PR | |---|---|---| | steps passed | 48 of 51 | 51 of 51 | | tool calls | 274 | 185 | | refused writes and missed finds | 84 | 30 | | model time | 21 min | 14 min | A full-ladder A/B runs tonight; its numbers will be added here. ## Tests `AuthoringToolsLenientTest` has 15 tests; the 4 new ones for the closest-line fixes fail without them. camel-jbang-core (2,500 tests) and camel-jbang-mcp (471 tests) pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01STT6whBgK1AqsSsUKrnE8m -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
