gate-composition-unverified-exit-zero
Diagnosis complete and the fix is implemented and verified. The solution document is at /workspace/SOLUTION.md, with working artifacts at /workspace/scripts/verify-duckbrain-census.sh and /workspace/scripts/gate-verdict.py.
Problem class: gate-composition-unverified-exit-zero
Root cause (three compounding bugs):
1. The sub-guard's contract is "emit a VERDICT: line; rc=0 unless the harness crashes" — so UNVERIFIED returns exit 0. A composite gate that just forwards the sub-guard's rc silently certifies it.
2. POSIX sh has no portable pipefail, so guard | tee log returns tee's status and masks real failures.
3. make runs each recipe line in a fresh shell, so inline multi-line/pipelined compositions lose state and exit status (|| exit 0, trailing echo, etc.).
Fix:
- scripts/gate-verdict.py — re-classifies all VERDICT lines (VERIFIED/FAILED/UNVERIFIED/SHAPE_CHANGE, plus H3_/DUCKBRAIN_ prefixes and :/=), takes the strongest verdict, and encodes policy: FAILED → 1 always; UNVERIFIED/SHAPE_CHANGE → 1 unless H3_GATE_ALLOW_UNVERIFIED=1 (disclosed); no verdict → 2.
- scripts/verify-duckbrain-census.sh — runs the guard with no pipeline, captures status and transcript, pipes bytes to the classifier, aggregates (hard guard failure is never waived), and emits one canonical VERDICT: line to stdout.
- Makefile becomes one atomic call: @sh scripts/verify-duckbrain-census.sh.
Verification (12/12 passing): VERIFIED→0; UNVERIFIED→1; UNVERIFIED+opt-in→0; FAILED→1 even when the guard lies rc=0 and even with opt-in; SHAPE_CHANGE→1 (0 with opt-in); missing verdict→2; guard hard-fail rc→propagated; multi-verdict/precedence and stderr-only verdicts handled. Also reproduced the original two bugs (naive rc forwarding and guard|tee masking) against the fixed gate.
# Evidence - Problem class: gate-composition-unverified-exit-zero - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-10-04T11:23:00.645Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "gate-composition-unverified-exit-zero", "provider": "openrouter", "solved_at": "2026-10-04T11:23:00.649Z", "version": ""}