This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch codex/report-review-runtime-effort
in repository https://gitbox.apache.org/repos/asf/doris-skills.git
The following commit(s) were added to
refs/heads/codex/report-review-runtime-effort by this push:
new e16aa6f fix: publish verified local review receipts (#9)
e16aa6f is described below
commit e16aa6fe1d7363d0462c85269e9fc904afef39ed
Author: shuke <[email protected]>
AuthorDate: Tue Sep 1 22:54:45 2026 +0800
fix: publish verified local review receipts (#9)
* fix: publish verified local review receipts
* fix: harden verified review receipts
---
README.md | 11 +-
skills/doris-repo-review/SKILL.md | 88 +++---
.../doris-repo-review/references/doc-templates.md | 2 +-
.../references/pr-comment-format.md | 25 +-
.../references/qualified-runtime.md | 2 +
.../doris-repo-review/scripts/post-pass-comment.sh | 107 +++----
.../scripts/record-review-runtime.sh | 43 +++
skills/doris-repo-review/scripts/verify-anchors.py | 172 ------------
.../scripts/verify-review-docs.py | 312 +++++++++++++++++++++
verify/README.md | 7 +-
verify/repo-review/post-comment.sh | 149 ++++++++++
verify/repo-review/review-documents.sh | 121 ++++++++
verify/repo-review/run.sh | 5 +-
verify/repo-review/runtime-attestation.sh | 50 ++++
14 files changed, 801 insertions(+), 293 deletions(-)
diff --git a/README.md b/README.md
index 0ff7565..1ef36b3 100644
--- a/README.md
+++ b/README.md
@@ -39,7 +39,8 @@ skill from its `description`, so in practice you describe the
problem and the ri
These skills read a Doris **source** checkout and never touch a cluster; the
cluster-side skills
never touch repository source. Nothing here builds Doris or runs tests. The
one GitHub write in
the whole set is `doris-repo-review`'s PASS comment — posted only when the
review finds no
-`Blocker` and no `Major`, and only after the user has seen the exact body and
approved it.
+`Blocker` or `Major`, converges under a qualified reviewer, and still matches
the live PR head.
+Explicitly invoking the skill authorizes that one automatic comment.
## Install
@@ -100,7 +101,8 @@ keep that file and doriscli in sync.
The factual claims in the cluster-side skills are regression-tested — against
a real Apache Doris
cluster, against `doriscli`, and against the skills' own runtime behavior — so
they do not drift
-from reality. See [`verify/`](verify/README.md); it runs in three layers:
+from reality. See [`verify/`](verify/README.md); it runs three
cluster/behavior layers plus a
+repository-review contract suite:
- **L1 — knowledge**: every DDL template and gotcha in `doris-best-practices`
is accepted
or rejected by a live cluster exactly as the skill claims (`mysql` client).
@@ -109,10 +111,11 @@ from reality. See [`verify/`](verify/README.md); it runs
in three layers:
- **L3 — behavior**: the skills *behave* as written — evidence-first and
safety guardrails,
an end-to-end advisor→DDL→live-cluster loopback, and skill-router triggering
— exercised
through a nested `claude -p`.
+- **Repo review**: qualified-runtime policy, document-result validation, and
mocked GitHub comment
+ create/update behavior.
A skill that makes no DDL or doriscli claim (`doris-debug`,
`doris-profile-reader`,
-`doris-repo-review`) has no L1/L2 coverage today. Adding cases for a new skill
is welcome but not
-a merge requirement.
+`doris-repo-review`) has no L1/L2 coverage; its local receipt contract is
covered separately.
## Contributing a skill
diff --git a/skills/doris-repo-review/SKILL.md
b/skills/doris-repo-review/SKILL.md
index 99ff69c..80d1db2 100644
--- a/skills/doris-repo-review/SKILL.md
+++ b/skills/doris-repo-review/SKILL.md
@@ -1,6 +1,6 @@
---
name: doris-repo-review
-description: Review an Apache Doris PR from a local clone with the same
multi-agent, shared-ledger convergence workflow as the CI Code Review Runner.
Use when the user supplies a PR to `/doris-repo-review`, asks to review a Doris
PR locally, or asks for the CI-style review flow. Safely align only the current
worktree to the exact PR head, refuse to disturb tracked local changes, write
equivalent English and Chinese review documents with verified path-line
anchors, and never build, test, [...]
+description: Review an Apache Doris PR from a local clone with the same
multi-agent, shared-ledger convergence workflow as the CI Code Review Runner.
Use when the user supplies a PR to `/doris-repo-review`, asks to review a Doris
PR locally, or asks for the CI-style review flow. Safely align only the current
worktree to the exact PR head, refuse to disturb tracked local changes, write
equivalent English and Chinese review documents with verified path-line
anchors, and never build, test, [...]
---
# Local pipeline-style Doris code review
@@ -34,8 +34,9 @@ Let `$S` be this skill's own `scripts/` directory (with a
default Claude Code in
$S/align-to-pr.sh <PR> --check # diagnosis only, changes
nothing
$S/align-to-pr.sh <PR> --out "$CTX/align.env" # step 1: align the
current directory
$S/prepare-review-context.sh --ctx "$CTX" --align "$CTX/align.env" # step 2:
gather context
-python3 $S/verify-anchors.py --ctx "$CTX" --doc <en> --doc <zh> # step 9:
verify anchors
-$S/post-pass-comment.sh --ctx "$CTX" --model <id> ... --dry-run # step
10: PASS comment
+python3 $S/verify-review-docs.py --ctx "$CTX" --doc <en> --doc <zh> # step 9:
verify documents
+$S/record-review-runtime.sh --ctx "$CTX" --model <id> --effort <effort>
+$S/post-pass-comment.sh --ctx "$CTX" # step
10: auto-post PASS
$S/review-runtime-policy.sh check <model> <effort> #
reviewer eligibility
```
@@ -43,7 +44,8 @@ $S/review-runtime-policy.sh check <model> <effort>
# reviewer
|---|---|
| `scripts/align-to-pr.sh` | Resolve the PR, diagnose how the current
directory relates to it, align it to the PR head |
| `scripts/prepare-review-context.sh` | Produce the authoritative diff,
new-side line ranges, required AGENTS.md list, existing comments, ledger
skeleton |
-| `scripts/verify-anchors.py` | Check that every `path:line` anchor really
exists and that both documents expose the same finding IDs |
+| `scripts/verify-review-docs.py` | Validate commit, anchors, EN/ZH agreement,
verdict, findings, rounds, and convergence |
+| `scripts/record-review-runtime.sh` | Record the qualified reviewer model,
effort, and exact commit |
| `scripts/review-runtime-policy.sh` | The exact model and effort allowlist
for a pipeline-equivalent review |
| `scripts/post-pass-comment.sh` | Render and post the machine-readable PASS
comment; refuses everything that is not a pass |
| `references/prompts.md` | Subagent prompt templates (CI wording, carried
over verbatim) |
@@ -72,9 +74,9 @@ pipeline-equivalent PASS comment.
2. **Read-only review.** Do not build, do not run tests, do not modify any
source file in the
repository. The only local writes allowed are the two documents under
`review-docs/` and the
context directory `$CTX`.
- **Exactly one thing may ever be written to GitHub**: the PASS comment of
step 10, only when the
- verdict is APPROVE, only through `post-pass-comment.sh`, and only after the
user has seen the
- rendered body and said go. A REQUEST_CHANGES review posts nothing. No
inline comments, no
+ **Exactly one thing may ever be written to GitHub**: the PASS comment of
step 10, automatically
+ after an explicitly requested review reaches a converged APPROVE result,
and only through
+ `post-pass-comment.sh`. A REQUEST_CHANGES review posts nothing. No inline
comments, no
review submission, no labels, no edits to the PR body - and never a comment
on any PR other
than the one being reviewed.
3. **The diff has exactly one source**: `$CTX/pr.diff` and
`$CTX/pr_changed_files.txt`. Do not
@@ -90,7 +92,7 @@ pipeline-equivalent PASS comment.
"Considered and Dismissed". Silently dropping a suspicion means the review
is not finished.
7. **There are no inline comments locally**, so an anchor is the reader's only
way in. Every
finding must carry a `path:line` anchor (new-side line numbers) plus a
verbatim snippet, and
- the documents must pass `verify-anchors.py` at the end.
+ the documents must pass `verify-review-docs.py` at the end.
---
@@ -169,6 +171,15 @@ Output (under `$CTX`):
If `BASE_SOURCE` is not `PR base sha (matches CI)`, the baseline differs from
CI's and the
documents must say so.
+Record the qualified runtime selected in step 0 before reading source:
+
+```bash
+$S/record-review-runtime.sh --ctx "$CTX" --model "<exact model>" --effort
"<exact effort>"
+```
+
+If it refuses the runtime, continue with local documents only and do not post
a pipeline-equivalent
+PASS comment.
+
---
## 3. Required reading (in this order)
@@ -318,23 +329,22 @@ When they are written, running the verifier is
**mandatory**; if it fails, fix t
re-run until it passes:
```bash
-python3 $S/verify-anchors.py --ctx "$CTX" \
+python3 $S/verify-review-docs.py --ctx "$CTX" \
--doc review-docs/pr-<N>-review.en.md \
--doc review-docs/pr-<N>-review.zh.md
```
-It checks that anchor paths exist, that line numbers are inside the file, that
every finding has at
-least one anchor, and that the EN and ZH finding-ID sets match; it also flags
anchors pointing at
-unchanged context lines - which is usually where a miscomputed line number
shows up.
+It validates the reviewed head, anchors, EN/ZH finding order and severity,
verdict, rounds, and
+convergence. Fix every error before continuing.
---
## 10. Post the PASS comment to the PR
-**Only when the verdict is APPROVE** (no `Blocker`, no `Major`) and
`verify-anchors.py` has
-passed. A REQUEST_CHANGES review posts nothing at all - say so in the closing
report and stop.
+**Only when the verdict is APPROVE**, the review converged, and step 9 passed.
A
+REQUEST_CHANGES or non-converged review posts nothing - say so in the closing
report and stop.
-Write the notes, dry-run, get a go, post:
+Write optional notes and run the poster once:
```bash
# At most 5 bullets, each anchored where it can be. Skip the file when there
is nothing to say.
@@ -343,32 +353,14 @@ cat > "$CTX/pr-comment-notes.md" <<'EOF'
EOF
$S/post-pass-comment.sh --ctx "$CTX" \
- --model "<exact model id of this session>" --effort
"${CLAUDE_EFFORT:-unknown}" \
- --findings <blocker>,<major>,<minor>,<nit> \
- --rounds <r> --converged <true|false> \
- --notes-file "$CTX/pr-comment-notes.md" \
- --dry-run
+ --notes-file "$CTX/pr-comment-notes.md"
```
-- **`--model` is the exact model id of the session doing the review**
(`claude-opus-5[1m]`,
- `gpt-5.6-sol`, …), taken from what this session was told about itself -
never a guess, never a
- bare family name. `--effort` comes from `$CLAUDE_EFFORT`. The comment is a
public, signed
- statement about who reviewed the code; both fields are what make it
auditable.
-- The dry run runs **every** precondition and prints the exact body. **Show
that body to the user
- and wait for a go**, then re-run the identical command **without
`--dry-run`**. Never post
- without that confirmation.
-- The script refuses to post when: any `Blocker`/`Major` is present; the live
PR head no longer
- equals the reviewed commit; the PR is not open (`--allow-closed` overrides);
the notes are
- malformed or more than five; `converged: false` came without a note. A
refusal is a real signal
- - relay it, do not work around it.
-- **The PR head moved** means the author pushed during the review: the review
is stale, so re-run
- the whole skill instead of posting.
-- An earlier v1 comment from the same account for the **same** commit is
edited in place; a new
- commit gets a new comment, so each push leaves exactly one record.
-- Findings counts, rounds and `converged` must match the documents written in
step 9. The counts
- are of *accepted* findings, not of candidates.
-- `references/pr-comment-format.md` holds the schema, the field meanings, and
the parser snippet.
- **Never hand-write or hand-edit this comment** - the format is a contract
other programs read.
+The poster reads model, effort, and commit from `review-runtime.json`; it
invokes
+`verify-review-docs.py` itself for verdict, findings, rounds, and convergence;
then it rechecks the
+live PR head. A refusal is final. The normal flow posts immediately;
`--dry-run` exists only for
+maintainer testing. Same account plus same commit updates the existing
comment, while a new commit
+creates a new one. Keep the schema in `references/pr-comment-format.md`
machine-generated.
---
@@ -381,7 +373,7 @@ Tell the user:
2. The `branch check` / `commit check` results - especially `ahead:N`
(unpushed commits that were
not reviewed).
3. **What happened to the PASS comment**: the URL when one was posted or
updated, or the reason
- nothing was posted (the verdict was REQUEST_CHANGES, the PR head moved, the
user said no).
+ nothing was posted (REQUEST_CHANGES, non-convergence, unqualified runtime,
or moved PR head).
4. **Where the current directory now stands**: with `ALIGN_MODE=switched` it
is detached on the PR
head, and `git checkout <PREV_REF>` restores it. **Do not switch back
automatically** - the user
may still want to read the code.
@@ -404,7 +396,7 @@ Tell the user:
| Single-file ledger with sections | A `ledger/` directory, one file per owner
|
| Main risk scan → 1-3 full-review subagents + risk-focused → merge → ≤3
rounds | Identical |
| `gh pr review` / Reviews API posting inline comments | **Two `review-docs/`
documents (EN + ZH) with `path:line` anchors** |
-| CI's review verdict is visible on the PR itself | On a pass, one
`doris-repo-review/v1` comment from the local account (commit sha, timestamp,
model, findings, notes); on REQUEST_CHANGES, nothing - the documents stay local
|
+| CI's review verdict is visible on the PR itself | On a qualified converged
pass, one automatic `doris-repo-review/v1` comment bound to the exact commit;
otherwise nothing |
| 60-minute timeout | No hard timeout, but likewise do not let one round turn
into unbounded digging |
---
@@ -433,13 +425,7 @@ Tell the user:
dropped.
- **Do not commit `review-docs/`.** The doris repository does not ignore it,
and an automatic commit
would slip it into the PR.
-- **The PASS comment is public and signed with the user's name.** It goes to a
public Apache PR
- from their GitHub account, so it is posted only after they have seen the
exact body. Treat a
- script refusal as final rather than something to route around, and never
"tidy up" the rendered
- body by hand - a program reads it.
-- **A pass is not a merge approval.** The comment states that a local pipeline
review found no
- Blocker and no Major on one specific commit. It carries no CI signal and no
Apache sign-off, and
- the `<sub>` disclaimer line says exactly that - keep it.
-- **Counts drift between the documents and the comment.** `--findings` must be
the accepted
- findings of step 9, not the candidate count from the ledger; re-count from
the written documents
- before posting.
+- **The PASS comment is public and signed with the user's name.** Invoking
this skill for a PR
+ authorizes one automatic comment after a qualified converged pass. Never
edit the body by hand.
+- **A pass is not a human merge approval.** Repository policy may accept the
receipt for its exact
+ commit, but it carries no Apache sign-off. Keep the `<sub>` disclaimer.
diff --git a/skills/doris-repo-review/references/doc-templates.md
b/skills/doris-repo-review/references/doc-templates.md
index c1a1c1d..9c9c394 100644
--- a/skills/doris-repo-review/references/doc-templates.md
+++ b/skills/doris-repo-review/references/doc-templates.md
@@ -17,7 +17,7 @@ their original form in both documents.
Local review has no inline comments, so an anchor is the only way a reader
finds the code.
- Write anchors inside backticks: ``
`fe/fe-core/src/main/java/org/apache/doris/X.java:412` ``
- or `` `path:412-430` ``. `scripts/verify-anchors.py` only recognises this
form.
+ or `` `path:412-430` ``. `scripts/verify-review-docs.py` only recognises
this form.
- Numbers are NEW-SIDE (post-change) line numbers of the worktree at HEAD,
cross-checked
against `changed_line_ranges.txt` and confirmed by reading the file.
- Every finding carries at least one anchor plus a fenced snippet copied
verbatim from it.
diff --git a/skills/doris-repo-review/references/pr-comment-format.md
b/skills/doris-repo-review/references/pr-comment-format.md
index a26488e..b099fe7 100644
--- a/skills/doris-repo-review/references/pr-comment-format.md
+++ b/skills/doris-repo-review/references/pr-comment-format.md
@@ -2,7 +2,7 @@
When a review passes, the skill posts **one comment** on the PR from the
locally authenticated
`gh` account. The comment is written by `scripts/post-pass-comment.sh`, never
typed by hand: the
-agent supplies the numbers and the notes, the script owns the layout. Anything
that reads the
+agent supplies optional notes, while verified documents supply the structured
result. Anything that reads the
comment back — a script, a dashboard, another agent — depends on that layout
being fixed.
## Layout
@@ -42,8 +42,8 @@ converged: true
- The behaviour change is gated by `ldap_authentication_enabled`, so no
rolling-upgrade path is
required.
-<sub>Reviewed locally with the `doris-repo-review` pipeline (a local port of
-`.github/workflows/code-review-runner.yml`). This is not a CI status
check.</sub>
+<sub>Reviewed locally with the `doris-repo-review` pipeline. Repository policy
may accept this
+receipt for the matching commit; it is not a human Apache approval.</sub>
<!-- doris-repo-review:v1:end -->
````
@@ -58,11 +58,11 @@ converged: true
| `base` | 40-hex | `BASE_SHA` — with `commit` this reproduces the reviewed
diff |
| `reviewed_at` | ISO-8601, minute precision, with offset | when the comment
was rendered |
| `reviewer` | GitHub login | `gh api user`, falling back to GraphQL `viewer`
and `gh auth status` |
-| `model` | exact model id | passed with `--model`, e.g. `claude-opus-5[1m]`,
`gpt-5.6-sol` |
-| `effort` | reasoning effort | `--effort`, default `$CLAUDE_EFFORT` |
-| `findings` | inline map | counts per severity; `blocker` and `major` are
always 0 in a PASS |
-| `rounds` | int | convergence rounds actually run (1-3) |
-| `converged` | bool | `false` means the 3-round cap was hit with candidates
still open |
+| `model` | exact eligible model id | `review-runtime.json`, recorded before
source review |
+| `effort` | `xhigh`, `max`, or `ultra` | `review-runtime.json` |
+| `findings` | inline map | `verify-review-docs.py`; `blocker` and `major` are
always 0 in a PASS |
+| `rounds` | int | `verify-review-docs.py` (1-3) |
+| `converged` | bool | `verify-review-docs.py`; only `true` is posted |
Rules the script enforces, so they cannot drift:
@@ -70,7 +70,11 @@ Rules the script enforces, so they cannot drift:
refuses to post — a failing review leaves no trace on GitHub.
- **The PR head must not have moved.** The live `head.sha` is re-read and must
equal `commit`,
otherwise the comment would sign off a commit nobody reviewed.
-- **`converged: false` requires at least one note** saying what was left
unexamined.
+- **Runtime and documents must name the same commit.** Both must equal
`meta.env` and the live PR
+ head. Model and effort must pass the exact allowlist.
+- **The documents are the result source.** The poster invokes their verifier
immediately before
+ rendering; command-line overrides for verdict, counts, rounds, or
convergence do not exist.
+- **The review must converge.** A non-converged review never posts a PASS
receipt.
- **At most 5 notes**, each a `- ` bullet (2-space indented continuation lines
allowed). Longer
material belongs in the local review documents.
- **Same commit ⇒ update in place.** An earlier v1 comment by the same account
carrying the same
@@ -114,7 +118,8 @@ place only when a maintainer would act differently without
it:
- a residual risk that did not reach `Minor`, with a `path:line` anchor;
- coverage the review could not reach (no build, no cluster, no test run);
- a backport or upgrade consideration the PR itself does not state;
-- with `converged: false`, what was still open when the round cap hit.
+
+A non-converged review records what remained open in the local documents and
posts no receipt.
Not this: restating what the PR does, listing every `Minor`/`Nit` (they live
in the documents),
praise, or anything that reads as an official Apache sign-off. `_None._` is a
perfectly good
diff --git a/skills/doris-repo-review/references/qualified-runtime.md
b/skills/doris-repo-review/references/qualified-runtime.md
index 1054392..ad9cf5c 100644
--- a/skills/doris-repo-review/references/qualified-runtime.md
+++ b/skills/doris-repo-review/references/qualified-runtime.md
@@ -13,6 +13,8 @@ Validate the exact values with:
$S/review-runtime-policy.sh check "<model>" "<effort>"
```
+Keep these exact values for `record-review-runtime.sh` after the PR context is
prepared.
+
Never infer task settings from a global config file. Continue in the current
task only when its
model and effort are exposed explicitly and pass the policy check.
diff --git a/skills/doris-repo-review/scripts/post-pass-comment.sh
b/skills/doris-repo-review/scripts/post-pass-comment.sh
index fcfd6a0..0e769e7 100755
--- a/skills/doris-repo-review/scripts/post-pass-comment.sh
+++ b/skills/doris-repo-review/scripts/post-pass-comment.sh
@@ -2,49 +2,27 @@
# Render and post the machine-readable PASS comment of a doris-repo-review run.
#
# Usage:
-# post-pass-comment.sh --ctx <dir> --model <id> [options]
+# post-pass-comment.sh --ctx <dir> [options]
#
-# --ctx <dir> review context directory (must contain meta.env)
-# --model <id> exact model id of the reviewing agent, e.g.
claude-opus-5[1m]
-# --effort <s> reasoning effort (default: $CLAUDE_EFFORT, else
"unknown")
-# --findings b,m,mi,n blocker,major,minor,nit counts (default 0,0,0,0)
-# --rounds <n> convergence rounds actually run (default 1)
-# --converged true|false (default true)
+# --ctx <dir> review context with meta.env, review-runtime.json,
and EN/ZH docs
# --notes-file <f> markdown bullet list for "Notes for maintainers"
# --dry-run run every precondition, render the body, post
nothing
-# --force-new always create a new comment, never update in place
-# --allow-closed allow posting on a non-open PR
#
-# Only a PASS is ever posted: blocker and major counts must both be 0, which is
-# the same verdict rule the review documents use. The body layout is fixed here
-# on purpose - the agent supplies the notes, never the format.
+# Runtime fields come from review-runtime.json. Review fields come directly
from
+# verify-review-docs.py. The agent supplies notes, never receipt fields or
format.
#
# Rendered body: <ctx>/pr-comment.md Posted URL: <ctx>/pr-comment.url
set -euo pipefail
CTX=""
-MODEL=""
-EFFORT="${CLAUDE_EFFORT:-unknown}"
-FINDINGS="0,0,0,0"
-ROUNDS="1"
-CONVERGED="true"
NOTES_FILE=""
DRY_RUN=0
-FORCE_NEW=0
-ALLOW_CLOSED=0
while [ $# -gt 0 ]; do
case "$1" in
--ctx) CTX="$2"; shift 2 ;;
- --model) MODEL="$2"; shift 2 ;;
- --effort) EFFORT="$2"; shift 2 ;;
- --findings) FINDINGS="$2"; shift 2 ;;
- --rounds) ROUNDS="$2"; shift 2 ;;
- --converged) CONVERGED="$2"; shift 2 ;;
--notes-file) NOTES_FILE="$2"; shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
- --force-new) FORCE_NEW=1; shift ;;
- --allow-closed) ALLOW_CLOSED=1; shift ;;
-h|--help) sed -n '2,25p' "$0"; exit 0 ;;
*) echo "unknown argument: $1" >&2; exit 2 ;;
esac
@@ -58,7 +36,8 @@ SCHEMA='doris-repo-review/v1'
[ -n "$CTX" ] || { echo "ERROR: --ctx is required." >&2; exit 2; }
META="$CTX/meta.env"
[ -f "$META" ] || { echo "ERROR: $META not found - run
prepare-review-context.sh first." >&2; exit 2; }
-[ -n "$MODEL" ] || { echo "ERROR: --model is required; state the exact model
id, never a guess." >&2; exit 2; }
+RUNTIME_FILE="$CTX/review-runtime.json"
+[ -f "$RUNTIME_FILE" ] || { echo "ERROR: $RUNTIME_FILE not found - record the
qualified reviewer first." >&2; exit 2; }
command -v gh >/dev/null 2>&1 || { echo "ERROR: gh CLI is required." >&2; exit
1; }
command -v jq >/dev/null 2>&1 || { echo "ERROR: jq is required." >&2; exit 1; }
command -v python3 >/dev/null 2>&1 || { echo "ERROR: python3 is required."
>&2; exit 1; }
@@ -69,26 +48,48 @@ PR_NUMBER="$(read_meta PR_NUMBER)"
PR_URL="$(read_meta PR_URL)"
BASE_SHA="$(read_meta BASE_SHA)"
HEAD_SHA="$(read_meta HEAD_SHA)"
+NORMALIZED_HEAD_SHA="$(printf '%s' "$HEAD_SHA" | tr '[:upper:]' '[:lower:]')"
REPO_ROOT="$(read_meta REPO_ROOT)"
+DOCS_ROOT="$(read_meta DOCS_ROOT)"
[ -n "$PR_NUMBER" ] || { echo "ERROR: meta.env has no PR_NUMBER - this review
is not attached to a PR." >&2; exit 2; }
[ -n "$UPSTREAM_REPO" ] || { echo "ERROR: meta.env has no UPSTREAM_REPO." >&2;
exit 2; }
[ -n "$PR_URL" ] ||
PR_URL="https://github.com/${UPSTREAM_REPO}/pull/${PR_NUMBER}";
-case "$CONVERGED" in true|false) ;; *) echo "ERROR: --converged takes true or
false." >&2; exit 2 ;; esac
-[[ "$ROUNDS" =~ ^[0-9]+$ ]] || { echo "ERROR: --rounds takes a number." >&2;
exit 2; }
-IFS=',' read -r F_BLOCKER F_MAJOR F_MINOR F_NIT <<<"$FINDINGS"
-for v in "$F_BLOCKER" "$F_MAJOR" "$F_MINOR" "$F_NIT"; do
- [[ "$v" =~ ^[0-9]+$ ]] || { echo "ERROR: --findings takes
blocker,major,minor,nit as four numbers." >&2; exit 2; }
-done
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+MODEL="$(jq -er '.model | strings' "$RUNTIME_FILE")" || { echo "ERROR: invalid
runtime model." >&2; exit 2; }
+EFFORT="$(jq -er '.effort | strings' "$RUNTIME_FILE")" || { echo "ERROR:
invalid runtime effort." >&2; exit 2; }
+RUNTIME_COMMIT="$(jq -er '.commit | strings' "$RUNTIME_FILE")" || { echo
"ERROR: invalid runtime commit." >&2; exit 2; }
+"$SCRIPT_DIR/review-runtime-policy.sh" check "$MODEL" "$EFFORT" >/dev/null
+[ "$RUNTIME_COMMIT" = "$NORMALIZED_HEAD_SHA" ] || {
+ echo "ERROR: qualified reviewer runtime was recorded for $RUNTIME_COMMIT,
not $NORMALIZED_HEAD_SHA." >&2
+ exit 2
+}
-# The verdict rule: any Blocker or Major means REQUEST_CHANGES, and a
-# REQUEST_CHANGES review posts nothing at all.
-if [ "$F_BLOCKER" -gt 0 ] || [ "$F_MAJOR" -gt 0 ]; then
- echo "ERROR: $F_BLOCKER blocker(s) and $F_MAJOR major(s) mean
REQUEST_CHANGES." >&2
- echo " This skill posts a comment only when the review passes.
Nothing was posted." >&2
+[ -n "$DOCS_ROOT" ] || DOCS_ROOT="$REPO_ROOT"
+EN_DOC="$DOCS_ROOT/review-docs/pr-${PR_NUMBER}-review.en.md"
+ZH_DOC="$DOCS_ROOT/review-docs/pr-${PR_NUMBER}-review.zh.md"
+RESULT_JSON="$(python3 "$SCRIPT_DIR/verify-review-docs.py" --json --ctx "$CTX"
\
+ --doc "$EN_DOC" --doc "$ZH_DOC")" || {
+ echo "ERROR: review documents failed verification. Nothing was posted." >&2
+ exit 2
+}
+RESULT_COMMIT="$(jq -er '.commit | strings' <<<"$RESULT_JSON")"
+VERDICT="$(jq -er '.verdict | strings' <<<"$RESULT_JSON")"
+ROUNDS="$(jq -er '.rounds | numbers' <<<"$RESULT_JSON")"
+CONVERGED="$(jq -r '.converged | if . == true then "true" else "false" end'
<<<"$RESULT_JSON")"
+F_BLOCKER="$(jq -er '.findings.blocker | numbers' <<<"$RESULT_JSON")"
+F_MAJOR="$(jq -er '.findings.major | numbers' <<<"$RESULT_JSON")"
+F_MINOR="$(jq -er '.findings.minor | numbers' <<<"$RESULT_JSON")"
+F_NIT="$(jq -er '.findings.nit | numbers' <<<"$RESULT_JSON")"
+
+[ "$RESULT_COMMIT" = "$NORMALIZED_HEAD_SHA" ] || { echo "ERROR: review
documents target another commit." >&2; exit 2; }
+[ "$VERDICT" = "APPROVE" ] || { echo "ERROR: review verdict is $VERDICT.
Nothing was posted." >&2; exit 1; }
+[ "$CONVERGED" = "true" ] || { echo "ERROR: review did not converge. Nothing
was posted." >&2; exit 1; }
+[ "$F_BLOCKER" -eq 0 ] && [ "$F_MAJOR" -eq 0 ] || {
+ echo "ERROR: Blocker or Major findings cannot produce a PASS comment." >&2
exit 1
-fi
+}
# -------------------------------------------------------------------------
notes
NOTES_BODY="_None._"
@@ -100,11 +101,14 @@ if [ -n "$NOTES_FILE" ]; then
echo "ERROR: the notes file contains a v1 marker; that would break
comment detection." >&2
exit 2
fi
- while IFS= read -r line; do
+ while IFS= read -r line || [ -n "$line" ]; do
[ -n "${line//[[:space:]]/}" ] || continue
case "$line" in
"- "*) NOTE_COUNT=$((NOTE_COUNT + 1)) ;;
- " "*) : ;; # continuation of the previous bullet
+ " "*) [ "$NOTE_COUNT" -gt 0 ] || {
+ echo "ERROR: a note continuation must follow a '- '
bullet." >&2
+ exit 2
+ } ;;
*) echo "ERROR: every note line must be a '- ' bullet or a
2-space continuation: $line" >&2; exit 2 ;;
esac
done < "$NOTES_FILE"
@@ -112,11 +116,6 @@ if [ -n "$NOTES_FILE" ]; then
NOTES_BODY="$(cat "$NOTES_FILE")"
fi
fi
-if [ "$CONVERGED" = "false" ] && [ "$NOTE_COUNT" -eq 0 ]; then
- echo "ERROR: converged=false needs at least one note saying what was left
open." >&2
- exit 2
-fi
-
# ------------------------------------------------------------------ live PR
state
PR_TSV="$(gh api "repos/${UPSTREAM_REPO}/pulls/${PR_NUMBER}" --jq '[.head.sha,
.state] | @tsv')" || {
echo "ERROR: cannot read ${UPSTREAM_REPO}#${PR_NUMBER}." >&2; exit 1; }
@@ -129,8 +128,8 @@ if [ "$LIVE_HEAD_SHA" != "$HEAD_SHA" ]; then
echo " Posting would sign off a commit that was never reviewed.
Re-run the review." >&2
exit 1
fi
-if [ "$LIVE_STATE" != "open" ] && [ "$ALLOW_CLOSED" = "0" ]; then
- echo "ERROR: ${UPSTREAM_REPO}#${PR_NUMBER} is '$LIVE_STATE', not open.
Pass --allow-closed to post anyway." >&2
+if [ "$LIVE_STATE" != "open" ]; then
+ echo "ERROR: ${UPSTREAM_REPO}#${PR_NUMBER} is '$LIVE_STATE', not open." >&2
exit 1
fi
@@ -163,7 +162,7 @@ REVIEWED_AT="$(python3 -c "import datetime;
print(datetime.datetime.now().astime
# ------------------------------------------------------------------ render
body
FENCE='```'
-DISCLAIMER='<sub>Reviewed locally with the `doris-repo-review` pipeline (a
local port of `.github/workflows/code-review-runner.yml`). This is not a CI
status check.</sub>'
+DISCLAIMER='<sub>Reviewed locally with the `doris-repo-review` pipeline.
Repository policy may accept this receipt for the matching commit; it is not a
human Apache approval.</sub>'
BODY_FILE="$CTX/pr-comment.md"
cat > "$BODY_FILE" <<EOF
${BEGIN_MARKER}
@@ -199,6 +198,11 @@ EOF
SAME_ID=""
LAST_ID=""
LAST_COMMIT=""
+COMMENTS_TSV="$(gh api "repos/${UPSTREAM_REPO}/issues/${PR_NUMBER}/comments"
--paginate \
+ --jq '.[] | select(.body | test("doris-repo-review:v1:begin")) | [ .id,
.user.login, ([.body | scan("commit: [0-9a-f]{40}")] | .[0] // "") ] | @tsv')"
|| {
+ echo "ERROR: cannot list existing review comments. Nothing was posted." >&2
+ exit 1
+}
while IFS=$'\t' read -r cid clogin ccommit; do
[ -n "$cid" ] || continue
[ "$clogin" = "$REVIEWER" ] || continue
@@ -206,12 +210,11 @@ while IFS=$'\t' read -r cid clogin ccommit; do
LAST_ID="$cid"
LAST_COMMIT="$ccommit"
[ "$ccommit" = "$HEAD_SHA" ] && SAME_ID="$cid"
-done < <(gh api "repos/${UPSTREAM_REPO}/issues/${PR_NUMBER}/comments"
--paginate \
- --jq '.[] | select(.body | test("doris-repo-review:v1:begin")) | [
.id, .user.login, ([.body | scan("commit: [0-9a-f]{40}")] | .[0] // "") ] |
@tsv' 2>/dev/null || true)
+done <<<"$COMMENTS_TSV"
ACTION="create"
EXISTING_ID=""
-if [ -n "$SAME_ID" ] && [ "$FORCE_NEW" = "0" ]; then
+if [ -n "$SAME_ID" ]; then
ACTION="update"
EXISTING_ID="$SAME_ID"
fi
diff --git a/skills/doris-repo-review/scripts/record-review-runtime.sh
b/skills/doris-repo-review/scripts/record-review-runtime.sh
new file mode 100755
index 0000000..a4bedb8
--- /dev/null
+++ b/skills/doris-repo-review/scripts/record-review-runtime.sh
@@ -0,0 +1,43 @@
+#!/usr/bin/env bash
+# Record the qualified reviewer model, effort, and reviewed commit.
+set -euo pipefail
+
+CTX=""
+MODEL=""
+EFFORT=""
+while [ $# -gt 0 ]; do
+ case "$1" in
+ --ctx) CTX="$2"; shift 2 ;;
+ --model) MODEL="$2"; shift 2 ;;
+ --effort) EFFORT="$2"; shift 2 ;;
+ -h|--help) sed -n '2,24p' "$0"; exit 0 ;;
+ *) echo "unknown argument: $1" >&2; exit 2 ;;
+ esac
+done
+
+[ -n "$CTX" ] || { echo "ERROR: --ctx is required." >&2; exit 2; }
+[ -n "$MODEL" ] || { echo "ERROR: --model is required." >&2; exit 2; }
+[ -n "$EFFORT" ] || { echo "ERROR: --effort is required." >&2; exit 2; }
+command -v jq >/dev/null 2>&1 || { echo "ERROR: jq is required." >&2; exit 1; }
+
+META="$CTX/meta.env"
+[ -f "$META" ] || { echo "ERROR: $META not found - run
prepare-review-context.sh first." >&2; exit 2; }
+HEAD_SHA="$(sed -n 's/^HEAD_SHA=//p' "$META" | head -n 1)"
+[[ "$HEAD_SHA" =~ ^[0-9a-fA-F]{40}$ ]] || { echo "ERROR: meta.env HEAD_SHA is
not a full SHA." >&2; exit 2; }
+NORMALIZED_HEAD_SHA="$(printf '%s' "$HEAD_SHA" | tr '[:upper:]' '[:lower:]')"
+
+RUNTIME_FILE="$CTX/review-runtime.json"
+rm -f "$RUNTIME_FILE" "$CTX/pr-comment.md" "$CTX/pr-comment.url"
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
+# shellcheck source=review-runtime-policy.sh
+. "$SCRIPT_DIR/review-runtime-policy.sh"
+check_review_runtime "$MODEL" "$EFFORT"
+
+TMP_FILE="$(mktemp "$CTX/.review-runtime.XXXXXX")"
+trap 'rm -f "$TMP_FILE"' EXIT
+jq -n --arg model "$MODEL" --arg effort "$EFFORT" --arg commit
"$NORMALIZED_HEAD_SHA" \
+ '{model: $model, effort: $effort, commit: $commit}' > "$TMP_FILE"
+mv "$TMP_FILE" "$RUNTIME_FILE"
+trap - EXIT
+
+echo "recorded qualified review runtime: $MODEL ($EFFORT), commit
$NORMALIZED_HEAD_SHA"
diff --git a/skills/doris-repo-review/scripts/verify-anchors.py
b/skills/doris-repo-review/scripts/verify-anchors.py
deleted file mode 100755
index 8eefd3d..0000000
--- a/skills/doris-repo-review/scripts/verify-anchors.py
+++ /dev/null
@@ -1,172 +0,0 @@
-#!/usr/bin/env python3
-"""Verify the `path:line` anchors of the doris-repo-review output documents.
-
-Local review has no inline GitHub comments, so every finding must carry an
-anchor the reader can jump to. This script proves the anchors are real:
-
- * the path exists in the worktree at HEAD;
- * the line number is inside the file;
- * (informational) whether the line falls inside a range this PR touched;
- * the EN and ZH documents expose the same finding IDs, and every finding
- carries at least one anchor.
-
-Anchors must be written inside backticks as `path:line` or `path:start-end`.
-
-Usage:
- verify-anchors.py --ctx <context-dir> --doc <en.md> --doc <zh.md>
-
-Exits non-zero when an anchor is broken or the two documents disagree.
-"""
-
-from __future__ import annotations
-
-import argparse
-import re
-import sys
-from collections import defaultdict
-from pathlib import Path
-
-ANCHOR_RE =
re.compile(r"`([A-Za-z0-9_][A-Za-z0-9_./+-]*\.[A-Za-z0-9_+-]+):(\d+)(?:-(\d+))?`")
-FINDING_RE = re.compile(r"^#{2,4}\s+(?:\[)?(F-\d+)(?:\])?\b")
-CODE_FENCE_RE = re.compile(r"^\s*```")
-
-
-def parse_args() -> argparse.Namespace:
- parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
- parser.add_argument("--ctx", required=True, type=Path, help="Review
context directory (holds meta.env).")
- parser.add_argument("--doc", required=True, action="append", type=Path,
help="Review document; repeat for EN and ZH.")
- parser.add_argument("--repo-root", type=Path, default=None, help="Repo
root; defaults to REPO_ROOT from meta.env.")
- return parser.parse_args()
-
-
-def read_meta(ctx: Path) -> dict[str, str]:
- meta_path = ctx / "meta.env"
- if not meta_path.is_file():
- sys.exit(f"missing {meta_path}; run prepare-review-context.sh first")
- meta = {}
- for line in meta_path.read_text(encoding="utf-8").splitlines():
- if "=" in line:
- key, _, value = line.partition("=")
- meta[key] = value
- return meta
-
-
-def load_changed_ranges(ctx: Path) -> dict[str, list[tuple[int, int]]]:
- ranges: dict[str, list[tuple[int, int]]] = defaultdict(list)
- tsv = ctx / "changed_line_ranges.tsv"
- if tsv.is_file():
- for line in tsv.read_text(encoding="utf-8").splitlines():
- parts = line.split("\t")
- if len(parts) == 3:
- ranges[parts[0]].append((int(parts[1]), int(parts[2])))
- return ranges
-
-
-def scan_document(doc: Path) -> tuple[list[str], dict[str, list[tuple[int,
int, int | None]]], list[tuple[int, int, int | None]]]:
- """Return (finding ids in order, anchors per finding, all anchors)."""
- findings: list[str] = []
- per_finding: dict[str, list[tuple[int, int, int | None]]] =
defaultdict(list)
- all_anchors: list[tuple[str, int, int | None, int]] = []
- current: str | None = None
- in_fence = False
-
- for lineno, line in
enumerate(doc.read_text(encoding="utf-8").splitlines(), start=1):
- if CODE_FENCE_RE.match(line):
- in_fence = not in_fence
- continue
- heading = FINDING_RE.match(line)
- if heading:
- current = heading.group(1)
- if current not in findings:
- findings.append(current)
- if in_fence:
- continue
- for match in ANCHOR_RE.finditer(line):
- path, start, end = match.group(1), int(match.group(2)),
match.group(3)
- anchor = (path, start, int(end) if end else None, lineno)
- all_anchors.append(anchor)
- if current:
- per_finding[current].append(anchor)
- return findings, per_finding, all_anchors
-
-
-def in_changed_range(ranges: list[tuple[int, int]], start: int, end: int |
None) -> bool:
- last = end or start
- return any(not (last < lo or start > hi) for lo, hi in ranges)
-
-
-def main() -> int:
- args = parse_args()
- ctx = args.ctx.resolve()
- meta = read_meta(ctx)
- repo_root = (args.repo_root or Path(meta.get("REPO_ROOT", "."))).resolve()
- changed_ranges = load_changed_ranges(ctx)
-
- errors: list[str] = []
- warnings: list[str] = []
- finding_sets: dict[Path, list[str]] = {}
-
- for doc in args.doc:
- doc = doc.resolve()
- if not doc.is_file():
- errors.append(f"{doc}: document not found")
- continue
- findings, per_finding, anchors = scan_document(doc)
- finding_sets[doc] = findings
-
- print(f"\n=== {doc} ===")
- print(f"findings: {len(findings)} anchors: {len(anchors)}")
-
- for path, start, end, lineno in anchors:
- target = repo_root / path
- if not target.is_file():
- errors.append(f"{doc}:{lineno}: anchor path does not exist ->
{path}")
- continue
- try:
- total = sum(1 for _ in target.open(encoding="utf-8",
errors="replace"))
- except OSError as exc:
- errors.append(f"{doc}:{lineno}: cannot read {path}: {exc}")
- continue
- last = end or start
- if start < 1 or last > total:
- errors.append(f"{doc}:{lineno}: {path}:{start}{'-' + str(end)
if end else ''} out of range (file has {total} lines)")
- continue
- if path in changed_ranges and not
in_changed_range(changed_ranges[path], start, end):
- warnings.append(f"{doc}:{lineno}: {path}:{start}{'-' +
str(end) if end else ''} is context, not a changed line (fine if intentional)")
- elif path not in changed_ranges:
- warnings.append(f"{doc}:{lineno}: {path} is not in this PR's
changed-file set (fine for upstream/downstream evidence)")
-
- for finding in findings:
- if not per_finding.get(finding):
- errors.append(f"{doc}: finding {finding} has no `path:line`
anchor")
-
- docs = list(finding_sets)
- if len(docs) >= 2:
- base_doc, base_ids = docs[0], set(finding_sets[docs[0]])
- for other in docs[1:]:
- other_ids = set(finding_sets[other])
- missing = sorted(base_ids - other_ids)
- extra = sorted(other_ids - base_ids)
- if missing:
- errors.append(f"{other}: missing finding IDs present in
{base_doc.name}: {', '.join(missing)}")
- if extra:
- errors.append(f"{other}: has finding IDs absent from
{base_doc.name}: {', '.join(extra)}")
-
- if warnings:
- print("\n--- notes ---")
- for note in warnings:
- print(f" note: {note}")
-
- if errors:
- print("\n--- errors ---")
- for error in errors:
- print(f" ERROR: {error}")
- print(f"\nFAILED: {len(errors)} anchor problem(s).")
- return 1
-
- print("\nOK: every anchor resolves and both documents expose the same
findings.")
- return 0
-
-
-if __name__ == "__main__":
- sys.exit(main())
diff --git a/skills/doris-repo-review/scripts/verify-review-docs.py
b/skills/doris-repo-review/scripts/verify-review-docs.py
new file mode 100755
index 0000000..ae6a660
--- /dev/null
+++ b/skills/doris-repo-review/scripts/verify-review-docs.py
@@ -0,0 +1,312 @@
+#!/usr/bin/env python3
+"""Validate both doris-repo-review documents and emit their agreed result."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from dataclasses import dataclass, field
+from pathlib import Path
+
+ANCHOR_RE =
re.compile(r"`([A-Za-z0-9_][A-Za-z0-9_./+-]*\.[A-Za-z0-9_+-]+):(\d+)(?:-(\d+))?`")
+FINDING_RE = re.compile(r"^#{2,4}\s+(?:\[)?(F-\d+)(?:\])?\b")
+SEVERITY_RE = re.compile(
+ r"^-\s+\*\*(?:Severity|等级)\*\*\s*[:：]\s*(Blocker|Major|Minor|Nit)\s*$",
+ re.IGNORECASE,
+)
+HEAD_RE = re.compile(r"^\|\s*PR head\s*\|\s*`([0-9a-fA-F]{40})`")
+VERDICT_RE =
re.compile(r"^\|\s*(?:Verdict|结论)\s*\|\s*\*\*(APPROVE|REQUEST_CHANGES)\*\*\s*\|")
+ROUNDS_RE = re.compile(r"^\|\s*(?:Rounds|轮次)\s*\|(.*?)\|\s*$")
+CODE_FENCE_RE = re.compile(r"^\s*```")
+SEVERITIES = ("Blocker", "Major", "Minor", "Nit")
+
+
+@dataclass(frozen=True)
+class Anchor:
+ path: str
+ start: int
+ end: int | None
+ source_line: int
+
+
+@dataclass
+class Document:
+ path: Path
+ head_sha: str | None = None
+ verdict: str | None = None
+ rounds: int | None = None
+ converged: bool | None = None
+ finding_ids: list[str] = field(default_factory=list)
+ severities: dict[str, str] = field(default_factory=dict)
+ anchors: list[Anchor] = field(default_factory=list)
+ finding_anchors: dict[str, list[Anchor]] = field(default_factory=lambda:
defaultdict(list))
+
+
+def parse_args() -> argparse.Namespace:
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("--ctx", required=True, type=Path)
+ parser.add_argument("--doc", required=True, action="append", type=Path)
+ parser.add_argument("--repo-root", type=Path)
+ parser.add_argument("--json", action="store_true", help="Print only the
validated result as JSON.")
+ return parser.parse_args()
+
+
+def read_meta(ctx: Path) -> dict[str, str]:
+ path = ctx / "meta.env"
+ if not path.is_file():
+ raise ValueError(f"missing {path}; run prepare-review-context.sh
first")
+ meta: dict[str, str] = {}
+ for line in path.read_text(encoding="utf-8").splitlines():
+ if "=" in line:
+ key, _, value = line.partition("=")
+ meta[key] = value
+ return meta
+
+
+def load_changed_ranges(ctx: Path) -> dict[str, list[tuple[int, int]]]:
+ result: dict[str, list[tuple[int, int]]] = defaultdict(list)
+ path = ctx / "changed_line_ranges.tsv"
+ if not path.is_file():
+ return result
+ for line in path.read_text(encoding="utf-8").splitlines():
+ parts = line.split("\t")
+ if len(parts) == 3:
+ result[parts[0]].append((int(parts[1]), int(parts[2])))
+ return result
+
+
+def parse_rounds(value: str) -> tuple[int | None, bool | None]:
+ number = re.match(r"\s*(?:共\s*)?(\d+)(?:\s+of max 3|\s*轮)", value)
+ lowered = value.casefold()
+ if "not converged" in lowered or "未收敛" in value:
+ converged = False
+ elif "converged" in lowered or "已收敛" in value:
+ converged = True
+ else:
+ converged = None
+ return (int(number.group(1)) if number else None, converged)
+
+
+def parse_document(path: Path, errors: list[str]) -> Document:
+ document = Document(path=path)
+ current_finding: str | None = None
+ in_fence = False
+
+ for lineno, line in
enumerate(path.read_text(encoding="utf-8").splitlines(), start=1):
+ if CODE_FENCE_RE.match(line):
+ in_fence = not in_fence
+ continue
+ if in_fence:
+ continue
+
+ if match := HEAD_RE.match(line):
+ if document.head_sha is not None:
+ errors.append(f"{path}:{lineno}: duplicate PR head row")
+ document.head_sha = match.group(1).lower()
+
+ if match := VERDICT_RE.match(line):
+ if document.verdict is not None:
+ errors.append(f"{path}:{lineno}: duplicate verdict row")
+ document.verdict = match.group(1)
+
+ if match := ROUNDS_RE.match(line):
+ if document.rounds is not None:
+ errors.append(f"{path}:{lineno}: duplicate rounds row")
+ document.rounds, document.converged = parse_rounds(match.group(1))
+
+ if match := FINDING_RE.match(line):
+ current_finding = match.group(1)
+ if current_finding in document.finding_ids:
+ errors.append(f"{path}:{lineno}: duplicate finding
{current_finding}")
+ else:
+ document.finding_ids.append(current_finding)
+
+ if match := SEVERITY_RE.match(line):
+ if current_finding is None:
+ errors.append(f"{path}:{lineno}: severity is not under a
finding")
+ elif current_finding in document.severities:
+ errors.append(f"{path}:{lineno}: duplicate severity for
{current_finding}")
+ else:
+ document.severities[current_finding] = match.group(1).title()
+
+ for match in ANCHOR_RE.finditer(line):
+ anchor = Anchor(
+ path=match.group(1),
+ start=int(match.group(2)),
+ end=int(match.group(3)) if match.group(3) else None,
+ source_line=lineno,
+ )
+ document.anchors.append(anchor)
+ if current_finding is not None:
+ document.finding_anchors[current_finding].append(anchor)
+
+ if document.head_sha is None:
+ errors.append(f"{path}: missing PR head row")
+ if document.verdict is None:
+ errors.append(f"{path}: missing Verdict/结论 row")
+ if document.rounds is None:
+ errors.append(f"{path}: missing Rounds/轮次 value")
+ elif not 1 <= document.rounds <= 3:
+ errors.append(f"{path}: rounds must be between 1 and 3")
+ if document.converged is None:
+ errors.append(f"{path}: rounds row must state convergence")
+
+ for finding in document.finding_ids:
+ if finding not in document.severities:
+ errors.append(f"{path}: finding {finding} has no severity")
+ if not document.finding_anchors.get(finding):
+ errors.append(f"{path}: finding {finding} has no `path:line`
anchor")
+
+ counts = severity_counts(document)
+ expected = "REQUEST_CHANGES" if counts["blocker"] or counts["major"] else
"APPROVE"
+ if document.verdict is not None and document.verdict != expected:
+ errors.append(f"{path}: verdict {document.verdict} conflicts with
finding severities")
+ return document
+
+
+def severity_counts(document: Document) -> dict[str, int]:
+ return {
+ name.casefold(): sum(1 for value in document.severities.values() if
value == name)
+ for name in SEVERITIES
+ }
+
+
+def finding_anchor_keys(document: Document, finding: str) -> list[tuple[str,
int, int]]:
+ """Return stable semantic anchors, excluding document source-line
metadata."""
+ return sorted(
+ {
+ (anchor.path, anchor.start, anchor.end or anchor.start)
+ for anchor in document.finding_anchors.get(finding, [])
+ }
+ )
+
+
+def verify_anchors(
+ document: Document,
+ repo_root: Path,
+ changed_ranges: dict[str, list[tuple[int, int]]],
+ errors: list[str],
+ warnings: list[str],
+) -> None:
+ line_counts: dict[str, int] = {}
+ for anchor in document.anchors:
+ if anchor.end is not None and anchor.end < anchor.start:
+ errors.append(
+ f"{document.path}:{anchor.source_line}:
{anchor.path}:{anchor.start}-{anchor.end} "
+ "has an end before its start"
+ )
+ continue
+ target = repo_root / anchor.path
+ if not target.is_file():
+ errors.append(f"{document.path}:{anchor.source_line}: anchor path
does not exist -> {anchor.path}")
+ continue
+ if anchor.path not in line_counts:
+ try:
+ line_counts[anchor.path] = sum(
+ 1 for _ in target.open(encoding="utf-8", errors="replace")
+ )
+ except OSError as exc:
+ errors.append(
+ f"{document.path}:{anchor.source_line}: cannot read
{anchor.path}: {exc}"
+ )
+ continue
+ last = anchor.end or anchor.start
+ if anchor.start < 1 or last > line_counts[anchor.path]:
+ errors.append(
+ f"{document.path}:{anchor.source_line}:
{anchor.path}:{anchor.start} out of range "
+ f"(file has {line_counts[anchor.path]} lines)"
+ )
+ continue
+ ranges = changed_ranges.get(anchor.path)
+ if not ranges:
+ warnings.append(f"{document.path}:{anchor.source_line}:
{anchor.path} is outside the changed-file set")
+ elif not any(not (last < start or anchor.start > end) for start, end
in ranges):
+ warnings.append(f"{document.path}:{anchor.source_line}:
{anchor.path}:{anchor.start} is unchanged context")
+
+
+def main() -> int:
+ args = parse_args()
+ errors: list[str] = []
+ warnings: list[str] = []
+ try:
+ meta = read_meta(args.ctx.resolve())
+ except ValueError as exc:
+ print(f"ERROR: {exc}", file=sys.stderr)
+ return 1
+
+ head_sha = meta.get("HEAD_SHA", "").lower()
+ if not re.fullmatch(r"[0-9a-f]{40}", head_sha):
+ errors.append("meta.env HEAD_SHA is not a full SHA")
+
+ repo_root = (args.repo_root or Path(meta.get("REPO_ROOT", "."))).resolve()
+ docs_root = Path(meta.get("DOCS_ROOT", repo_root)).resolve()
+ pr_number = meta.get("PR_NUMBER", "")
+ expected_paths = {
+ (docs_root / "review-docs" / f"pr-{pr_number}-review.en.md").resolve(),
+ (docs_root / "review-docs" / f"pr-{pr_number}-review.zh.md").resolve(),
+ }
+ actual_paths = {path.resolve() for path in args.doc}
+ if len(args.doc) != 2 or (pr_number and actual_paths != expected_paths):
+ errors.append("use exactly the standard PR-specific EN and ZH review
documents")
+
+ documents: list[Document] = []
+ changed_ranges = load_changed_ranges(args.ctx.resolve())
+ for path in args.doc:
+ resolved = path.resolve()
+ if not resolved.is_file():
+ errors.append(f"{resolved}: document not found")
+ continue
+ document = parse_document(resolved, errors)
+ verify_anchors(document, repo_root, changed_ranges, errors, warnings)
+ documents.append(document)
+
+ if len(documents) == 2:
+ left, right = documents
+ if left.finding_ids != right.finding_ids:
+ errors.append("EN and ZH finding IDs or order differ")
+ if left.severities != right.severities:
+ errors.append("EN and ZH finding severities differ")
+ for finding in sorted(set(left.finding_ids) & set(right.finding_ids)):
+ if finding_anchor_keys(left, finding) !=
finding_anchor_keys(right, finding):
+ errors.append(f"EN and ZH anchors differ for {finding}")
+ if (left.head_sha, left.verdict, left.rounds, left.converged) != (
+ right.head_sha,
+ right.verdict,
+ right.rounds,
+ right.converged,
+ ):
+ errors.append("EN and ZH head, verdict, rounds, or convergence
differ")
+ if left.head_sha != head_sha:
+ errors.append(f"review documents target {left.head_sha}, but
context head is {head_sha}")
+
+ output = sys.stderr if args.json else sys.stdout
+ for warning in warnings:
+ print(f"NOTE: {warning}", file=output)
+ if errors:
+ for error in errors:
+ print(f"ERROR: {error}", file=sys.stderr)
+ return 1
+ if len(documents) != 2:
+ print("ERROR: both review documents are required", file=sys.stderr)
+ return 1
+
+ result = {
+ "commit": head_sha,
+ "verdict": documents[0].verdict,
+ "findings": severity_counts(documents[0]),
+ "rounds": documents[0].rounds,
+ "converged": documents[0].converged,
+ }
+ if args.json:
+ print(json.dumps(result, separators=(",", ":")))
+ else:
+ print("OK: both review documents agree and every anchor resolves.")
+ print(json.dumps(result, indent=2))
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/verify/README.md b/verify/README.md
index c3d4530..150bbfb 100644
--- a/verify/README.md
+++ b/verify/README.md
@@ -8,14 +8,17 @@ real Apache Doris cluster and against `doriscli`. Three
layers:
| **L1 — knowledge** | Every DDL template (T1–T5) and DDL gotcha in
`doris-best-practices/SKILL.md` is accepted / rejected exactly as claimed |
`mysql` client | ✅ `run.sh` |
| **L2 — CLI contract** | Every command + JSON field in `CLI-CONTRACT.md`
really exists in `doriscli` | `doriscli --format json` + `jq` | ✅ `cli/run.sh` |
| **L3 — behavior** | Triggering, evidence-first / safety guardrails,
end-to-end DDL that loops back through L1 | nested `claude -p` | ✅ `behavior/`
(`run.sh` + `e2e-advisor-ddl.sh` + `triggering.sh`) |
-| **Repo review** | Exact model and effort policy for pipeline-equivalent
local reviews | shell | ✅ `repo-review/run.sh` |
+| **Repo review** | Runtime policy, verified review result, commit binding,
and automatic comment behavior | shell + Python + mocked `gh` | ✅
`repo-review/run.sh` |
-Run the repository-review policy tests without a cluster or GitHub access:
+Run the repository-review contract tests without a cluster or GitHub access:
```bash
./verify/repo-review/run.sh
```
+The runner keeps policy, runtime attestation, document validation, and mocked
comment posting in
+separate scripts so each contract can be reviewed and run independently.
+
## Run L1
```bash
diff --git a/verify/repo-review/post-comment.sh
b/verify/repo-review/post-comment.sh
new file mode 100755
index 0000000..45b0149
--- /dev/null
+++ b/verify/repo-review/post-comment.sh
@@ -0,0 +1,149 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+S="$ROOT/skills/doris-repo-review/scripts"
+TMP_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/doris-review-post.XXXXXX")"
+trap 'rm -rf "$TMP_ROOT"' EXIT
+REPO="$TMP_ROOT/repo"
+CTX="$TMP_ROOT/ctx"
+HEAD_SHA="1111111111111111111111111111111111111111"
+OTHER_SHA="3333333333333333333333333333333333333333"
+BASE_SHA="2222222222222222222222222222222222222222"
+PASS_COUNT=0
+mkdir -p "$TMP_ROOT/bin" "$REPO/src" "$REPO/review-docs" "$CTX"
+printf 'int answer = 42;\n' > "$REPO/src/Foo.java"
+printf 'src/Foo.java\t1\t1\n' > "$CTX/changed_line_ranges.tsv"
+
+pass() { PASS_COUNT=$((PASS_COUNT + 1)); echo "PASS: $1"; }
+fail() { echo "FAIL: $1" >&2; exit 1; }
+
+expect_failure() {
+ local label="$1" pattern="$2"
+ shift 2
+ if "$@" >"$TMP_ROOT/out" 2>"$TMP_ROOT/err"; then
+ fail "$label unexpectedly succeeded"
+ fi
+ if ! grep -Fq "$pattern" "$TMP_ROOT/err" && ! grep -Fq "$pattern"
"$TMP_ROOT/out"; then
+ fail "$label failed for the wrong reason"
+ fi
+ pass "$label"
+}
+
+write_meta() {
+ cat > "$CTX/meta.env" <<EOF
+UPSTREAM_REPO=apache/doris
+PR_NUMBER=123
+PR_URL=https://github.com/apache/doris/pull/123
+BASE_SHA=$BASE_SHA
+HEAD_SHA=$1
+REPO_ROOT=$REPO
+DOCS_ROOT=$REPO
+EOF
+}
+
+write_docs() {
+ local head="$1" verdict="$2" severity="$3" rounds="$4" convergence="$5"
+ local zh_convergence=已收敛
+ [ "$convergence" = converged ] || zh_convergence=未收敛
+ cat > "$REPO/review-docs/pr-123-review.en.md" <<EOF
+# Code Review — PR #123: fixture
+| | |
+|---|---|
+| PR head | \`$head\` on \`feature\` of \`fork\` |
+| Verdict | **$verdict** |
+| Rounds | $rounds of max 3, $convergence |
+### F-01 · fixture
+- **Severity**: $severity
+- **Where**: \`src/Foo.java:1\`
+EOF
+ cat > "$REPO/review-docs/pr-123-review.zh.md" <<EOF
+# 代码评审 — PR #123：fixture
+| | |
+|---|---|
+| PR head | \`$head\`，来自 \`fork\` 的 \`feature\` |
+| 结论 | **$verdict** |
+| 轮次 | 共 $rounds 轮（上限 3），$zh_convergence |
+### F-01 · fixture
+- **等级**：$severity
+- **位置**：\`src/Foo.java:1\`
+EOF
+}
+
+cat > "$TMP_ROOT/bin/gh" <<'EOF'
+#!/usr/bin/env bash
+set -euo pipefail
+if [ "$1" = api ] && [ "$2" = repos/apache/doris/pulls/123 ]; then
+ printf '%s\topen\n' "${MOCK_LIVE_HEAD:?}"
+elif [ "$1" = api ] && [ "$2" = user ]; then
+ printf 'reviewer-one\n'
+elif [ "$1" = api ] && [ "$2" = repos/apache/doris/issues/123/comments ]; then
+ [ "${MOCK_COMMENT_LIST_FAIL:-0}" = 1 ] && exit 1
+ [ "${MOCK_SAME_COMMENT:-0}" = 1 ] && printf '987\treviewer-one\tcommit:
%s\n' "${MOCK_LIVE_HEAD:?}"
+ exit 0
+elif [ "$1" = api ] && [ "$2" = --method ]; then
+ printf '%s\n' "$*" >> "${MOCK_GH_LOG:?}"
+ cat > "${MOCK_GH_BODY:?}"
+ printf
'{"html_url":"https://github.com/apache/doris/pull/123#issuecomment-test"}\n'
+else
+ echo "unexpected gh invocation: $*" >&2
+ exit 1
+fi
+EOF
+chmod +x "$TMP_ROOT/bin/gh"
+export PATH="$TMP_ROOT/bin:$PATH"
+export MOCK_GH_LOG="$TMP_ROOT/gh.log"
+export MOCK_GH_BODY="$TMP_ROOT/body.json"
+export MOCK_LIVE_HEAD="$HEAD_SHA"
+
+write_meta "$HEAD_SHA"
+write_docs "$HEAD_SHA" APPROVE Minor 2 converged
+expect_failure "poster requires runtime attestation" "review-runtime.json not
found" \
+ "$S/post-pass-comment.sh" --ctx "$CTX" --dry-run
+"$S/record-review-runtime.sh" --ctx "$CTX" --model gpt-5.6-sol --effort xhigh
>/dev/null
+
+"$S/post-pass-comment.sh" --ctx "$CTX" --dry-run > "$TMP_ROOT/dry-run"
+grep -Fq "commit: $HEAD_SHA" "$CTX/pr-comment.md" || fail "receipt commit is
wrong"
+grep -Fq "model: gpt-5.6-sol" "$CTX/pr-comment.md" || fail "receipt model is
wrong"
+grep -Fq "findings: {blocker: 0, major: 0, minor: 1, nit: 0}"
"$CTX/pr-comment.md" \
+ || fail "receipt findings are wrong"
+[ -z "${RECEIPT_OUTPUT:-}" ] || cp "$CTX/pr-comment.md" "$RECEIPT_OUTPUT"
+pass "verified dry run renders a pipeline-compatible receipt"
+
+: > "$MOCK_GH_LOG"
+MOCK_SAME_COMMENT=0 "$S/post-pass-comment.sh" --ctx "$CTX" > "$TMP_ROOT/create"
+grep -Fq -- "--method POST repos/apache/doris/issues/123/comments --input -"
"$MOCK_GH_LOG" \
+ || fail "poster did not create a comment"
+pass "qualified PASS creates a comment"
+
+: > "$MOCK_GH_LOG"
+MOCK_SAME_COMMENT=1 "$S/post-pass-comment.sh" --ctx "$CTX" > "$TMP_ROOT/update"
+grep -Fq -- "--method PATCH repos/apache/doris/issues/comments/987 --input -"
"$MOCK_GH_LOG" \
+ || fail "poster did not update the same-commit comment"
+pass "same-commit rerun updates the comment"
+
+MOCK_COMMENT_LIST_FAIL=1 expect_failure "comment lookup failure stops posting"
\
+ "cannot list existing review comments" "$S/post-pass-comment.sh" --ctx
"$CTX" --dry-run
+
+BAD_NOTES="$TMP_ROOT/bad-notes.md"
+printf 'not a bullet' > "$BAD_NOTES"
+expect_failure "unterminated malformed note is rejected" "every note line" \
+ "$S/post-pass-comment.sh" --ctx "$CTX" --notes-file "$BAD_NOTES" --dry-run
+
+write_docs "$HEAD_SHA" REQUEST_CHANGES Major 2 converged
+expect_failure "REQUEST_CHANGES never posts" "verdict is REQUEST_CHANGES" \
+ "$S/post-pass-comment.sh" --ctx "$CTX" --dry-run
+
+write_docs "$HEAD_SHA" APPROVE Minor 3 'not converged'
+expect_failure "non-converged review never posts" "did not converge" \
+ "$S/post-pass-comment.sh" --ctx "$CTX" --dry-run
+
+write_docs "$HEAD_SHA" APPROVE Minor 2 converged
+MOCK_LIVE_HEAD="$OTHER_SHA" expect_failure "moved PR head invalidates review"
"PR head moved" \
+ "$S/post-pass-comment.sh" --ctx "$CTX" --dry-run
+
+write_meta "$OTHER_SHA"
+MOCK_LIVE_HEAD="$OTHER_SHA" expect_failure "runtime attestation is
commit-bound" "was recorded for" \
+ "$S/post-pass-comment.sh" --ctx "$CTX" --dry-run
+
+echo "$PASS_COUNT post-comment tests passed"
diff --git a/verify/repo-review/review-documents.sh
b/verify/repo-review/review-documents.sh
new file mode 100755
index 0000000..4faf4b8
--- /dev/null
+++ b/verify/repo-review/review-documents.sh
@@ -0,0 +1,121 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+VERIFIER="$ROOT/skills/doris-repo-review/scripts/verify-review-docs.py"
+TMP_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/doris-review-docs.XXXXXX")"
+trap 'rm -rf "$TMP_ROOT"' EXIT
+REPO="$TMP_ROOT/repo"
+CTX="$TMP_ROOT/ctx"
+HEAD_SHA="1111111111111111111111111111111111111111"
+OTHER_SHA="3333333333333333333333333333333333333333"
+PASS_COUNT=0
+mkdir -p "$REPO/src" "$REPO/review-docs" "$CTX"
+printf 'int answer = 42;\nint other = 7;\n' > "$REPO/src/Foo.java"
+printf 'src/Foo.java\t1\t2\n' > "$CTX/changed_line_ranges.tsv"
+
+pass() { PASS_COUNT=$((PASS_COUNT + 1)); echo "PASS: $1"; }
+fail() { echo "FAIL: $1" >&2; exit 1; }
+
+expect_failure() {
+ local label="$1" pattern="$2"
+ shift 2
+ if "$@" >"$TMP_ROOT/out" 2>"$TMP_ROOT/err"; then
+ fail "$label unexpectedly succeeded"
+ fi
+ grep -Fq "$pattern" "$TMP_ROOT/err" || fail "$label failed for the wrong
reason"
+ pass "$label"
+}
+
+write_meta() {
+ cat > "$CTX/meta.env" <<EOF
+PR_NUMBER=123
+HEAD_SHA=$1
+REPO_ROOT=$REPO
+DOCS_ROOT=$REPO
+EOF
+}
+
+write_docs() {
+ local head="$1" verdict="$2" en_severity="$3" zh_severity="$4"
+ local rounds="$5" en_convergence="$6" zh_convergence="$7"
+ local en_anchor="${8:-1}"
+ local zh_anchor="${9:-$en_anchor}"
+ cat > "$REPO/review-docs/pr-123-review.en.md" <<EOF
+# Code Review — PR #123: fixture
+
+| | |
+|---|---|
+| PR head | \`$head\` on \`feature\` of \`fork\` |
+| Verdict | **$verdict** |
+| Rounds | $rounds of max 3, $en_convergence |
+
+## Findings
+
+### F-01 · fixture finding
+
+- **Severity**: $en_severity
+- **Where**: \`src/Foo.java:$en_anchor\`
+EOF
+ cat > "$REPO/review-docs/pr-123-review.zh.md" <<EOF
+# 代码评审 — PR #123：fixture
+
+| | |
+|---|---|
+| PR head | \`$head\`，来自 \`fork\` 的 \`feature\` |
+| 结论 | **$verdict** |
+| 轮次 | 共 $rounds 轮（上限 3），$zh_convergence |
+
+## 问题清单
+
+### F-01 · fixture finding
+
+- **等级**：$zh_severity
+- **位置**：\`src/Foo.java:$zh_anchor\`
+EOF
+}
+
+verify_json() {
+ "$VERIFIER" --json --ctx "$CTX" \
+ --doc "$REPO/review-docs/pr-123-review.en.md" \
+ --doc "$REPO/review-docs/pr-123-review.zh.md"
+}
+
+write_meta "$HEAD_SHA"
+write_docs "$HEAD_SHA" APPROVE Minor Minor 2 converged 已收敛
+RESULT="$(verify_json)"
+jq -e --arg head "$HEAD_SHA" \
+ '.commit == $head and .verdict == "APPROVE" and .findings.minor == 1 and
.rounds == 2 and .converged' \
+ <<<"$RESULT" >/dev/null || fail "verified result JSON is wrong"
+pass "matching documents produce one verified result"
+
+write_docs "$OTHER_SHA" APPROVE Minor Minor 2 converged 已收敛
+expect_failure "document commit must match context" "context head" verify_json
+
+write_docs "$HEAD_SHA" APPROVE Minor Nit 2 converged 已收敛
+expect_failure "EN and ZH severities must agree" "severities differ"
verify_json
+
+write_docs "$HEAD_SHA" APPROVE Minor Minor 2 converged 已收敛 1 2
+expect_failure "EN and ZH anchors must agree" "anchors differ" verify_json
+
+write_docs "$HEAD_SHA" APPROVE Minor Minor 4 converged 已收敛
+expect_failure "rounds must stay within the cap" "between 1 and 3" verify_json
+
+write_docs "$HEAD_SHA" APPROVE Minor Minor 2 converged 已收敛 3
+expect_failure "anchors must resolve" "out of range" verify_json
+
+write_docs "$HEAD_SHA" APPROVE Minor Minor 2 converged 已收敛 2-1
+expect_failure "anchor ranges cannot run backwards" "end before its start"
verify_json
+
+write_docs "$HEAD_SHA" REQUEST_CHANGES Major Major 2 converged 已收敛
+RESULT="$(verify_json)"
+jq -e '.verdict == "REQUEST_CHANGES" and .findings.major == 1' <<<"$RESULT"
>/dev/null \
+ || fail "REQUEST_CHANGES result is wrong"
+pass "Major findings produce REQUEST_CHANGES"
+
+write_docs "$HEAD_SHA" APPROVE Minor Minor 3 'not converged' 未收敛
+RESULT="$(verify_json)"
+jq -e '.converged == false' <<<"$RESULT" >/dev/null || fail "non-convergence
was not preserved"
+pass "non-converged result remains explicit"
+
+echo "$PASS_COUNT review-document tests passed"
diff --git a/verify/repo-review/run.sh b/verify/repo-review/run.sh
index e9f8ab3..9f55b8b 100755
--- a/verify/repo-review/run.sh
+++ b/verify/repo-review/run.sh
@@ -1,4 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
-"$(cd "$(dirname "$0")" && pwd)/runtime-policy.sh"
+DIR="$(cd "$(dirname "$0")" && pwd)"
+for test in runtime-policy runtime-attestation review-documents post-comment;
do
+ "$DIR/$test.sh"
+done
diff --git a/verify/repo-review/runtime-attestation.sh
b/verify/repo-review/runtime-attestation.sh
new file mode 100755
index 0000000..6e0ccd3
--- /dev/null
+++ b/verify/repo-review/runtime-attestation.sh
@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+RECORDER="$ROOT/skills/doris-repo-review/scripts/record-review-runtime.sh"
+TMP_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/doris-review-runtime.XXXXXX")"
+trap 'rm -rf "$TMP_ROOT"' EXIT
+HEAD_SHA="1111111111111111111111111111111111111111"
+PASS_COUNT=0
+
+pass() { PASS_COUNT=$((PASS_COUNT + 1)); echo "PASS: $1"; }
+fail() { echo "FAIL: $1" >&2; exit 1; }
+
+expect_failure() {
+ local label="$1" pattern="$2"
+ shift 2
+ if "$@" >"$TMP_ROOT/out" 2>"$TMP_ROOT/err"; then
+ fail "$label unexpectedly succeeded"
+ fi
+ grep -Fq "$pattern" "$TMP_ROOT/err" || fail "$label failed for the wrong
reason"
+ pass "$label"
+}
+
+write_meta() {
+ mkdir -p "$1"
+ printf 'HEAD_SHA=%s\n' "$2" > "$1/meta.env"
+}
+
+CTX="$TMP_ROOT/ctx"
+write_meta "$CTX" "$HEAD_SHA"
+"$RECORDER" --ctx "$CTX" --model gpt-5.6-sol --effort xhigh >/dev/null
+jq -e --arg head "$HEAD_SHA" \
+ '.model == "gpt-5.6-sol" and .effort == "xhigh" and .commit == $head' \
+ "$CTX/review-runtime.json" >/dev/null || fail "runtime JSON fields are
wrong"
+pass "qualified runtime records model, effort, and commit"
+
+expect_failure "low effort is rejected" "use xhigh or higher" \
+ "$RECORDER" --ctx "$CTX" --model gpt-5.6-sol --effort high
+[ ! -e "$CTX/review-runtime.json" ] || fail "failed replacement left an old
runtime record"
+pass "failed replacement removes the old attestation"
+
+expect_failure "unlisted model is rejected" "is not eligible" \
+ "$RECORDER" --ctx "$CTX" --model gpt-5.6-terra --effort xhigh
+
+BAD_CTX="$TMP_ROOT/bad-ctx"
+write_meta "$BAD_CTX" deadbeef
+expect_failure "short commit is rejected" "not a full SHA" \
+ "$RECORDER" --ctx "$BAD_CTX" --model gpt-5.6-sol --effort xhigh
+
+echo "$PASS_COUNT runtime-attestation tests passed"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
