e2e-battery-script-exits-zero-on-all-probes-fail
Verified the root cause concretely. Demo of the exact masking behavior:
--- WITHOUT pipefail ---
9 FAIL got=000
pipeline exit=0 (tee masks the exit 1) <-- why battery exited 0
--- WITH pipefail ---
9 FAIL got=000
pipeline exit=1 (pipefail propagates exit 1)
The battery exited 0 despite all 9 probes FAILing because of three compounding defects in km-e2e-full-battery.sh:
{ ... } | tee masked the subshell exit status. In Bash the pipeline's exit status is that of its last command. tee succeeds, so even when the probe block executed exit 1, the pipeline returned tee's 0 — the script signature was 0 and the audit declared server-up.FAILS counter. The probe loop only printed PASS/FAIL; nothing aggregated failures, so no final gating on a non-zero count was possible.set -u only — no set -e, no pipefail, so no error propagated.~/FIX-km-e2e-battery-exits-zero-on-all-probes-fail.md)set -o pipefail so a non-zero exit inside the {...} | tee subshell propagates.FAILS counter incremented on probe mismatch and on literal-501 stubs.PROBE-FAIL server SUSPECT DOWN when FAILS > 0.FAILS > 0 → BATTERY FAILED: N failure(s) - exit 1BATTERY PASSED: 9/9 PASS - exit 0FAIL got=000, summary BATTERY FAILED: 9 failure(s) - exit 1, process exit 1.9/9 PASS got=200, summary BATTERY PASSED: 9/9 PASS - exit 0, process exit 0.The solution file is written at ~/FIX-km-e2e-battery-exits-zero-on-all-probes-fail.md with the full self-contained markdown (root cause, exact header/fix edits against the script's structure, and the verified pass/fail matrix).
# Evidence - Problem class: e2e-battery-script-exits-zero-on-all-probes-fail - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-24T05:06:56.955Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "E2E battery shell script (km-e2e-full-battery.sh) exited 0 even when EVERY HTTP probe failed (connection refused, got=000 x9) because it had only set -u (no set -e, no FAIL-count aggregation) and the { ... } | tee pipeline masked the subshell exit status. A dead server was invisible to the standing audit; tick header kept claiming server up. Fix: FAILS counter on probe mismatch and literal-501 stubs, set -o pipefail so exit 1 inside the tee-d subshell propagates, summary line battery FAILED: N failure(s) - exit 1 / battery PASSED. Verified both directions: server down -> 9 FAIL got=000, exit 1; server up -> 9/9 PASS, exit 0.", "environment": "linux bash, curl probes against local HTTP server", "language": "bash", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "e2e-battery-script-exits-zero-on-all-probes-fail", "provider": "openrouter", "solved_at": "2026-08-24T05:06:56.956Z", "version": ""}Verified the root cause concretely. Demo of the exact masking behavior:
--- WITHOUT pipefail ---
9 FAIL got=000
pipeline exit=0 (tee masks the exit 1) <-- why battery exited 0
--- WITH pipefail ---
9 FAIL got=000
pipeline exit=1 (pipefail propagates exit 1)
The battery exited 0 despite all 9 probes FAILing because of three compounding defects in km-e2e-full-battery.sh:
{ ... } | tee masked the subshell exit status. In Bash the pipeline's exit status is that of its last command. tee succeeds, so even when the probe block executed exit 1, the pipeline returned tee's 0 — the script signature was 0 and the audit declared server-up.FAILS counter. The probe loop only printed PASS/FAIL; nothing aggregated failures, so no final gating on a non-zero count was possible.set -u only — no set -e, no pipefail, so no error propagated.~/FIX-km-e2e-battery-exits-zero-on-all-probes-fail.md)set -o pipefail so a non-zero exit inside the {...} | tee subshell propagates.FAILS counter incremented on probe mismatch and on literal-501 stubs.PROBE-FAIL server SUSPECT DOWN when FAILS > 0.FAILS > 0 → BATTERY FAILED: N failure(s) - exit 1BATTERY PASSED: 9/9 PASS - exit 0FAIL got=000, summary BATTERY FAILED: 9 failure(s) - exit 1, process exit 1.9/9 PASS got=200, summary BATTERY PASSED: 9/9 PASS - exit 0, process exit 0.The solution file is written at ~/FIX-km-e2e-battery-exits-zero-on-all-probes-fail.md with the full self-contained markdown (root cause, exact header/fix edits against the script's structure, and the verified pass/fail matrix).
# Evidence - Problem class: e2e-battery-script-exits-zero-on-all-probes-fail - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-24T05:06:56.955Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "E2E battery shell script (km-e2e-full-battery.sh) exited 0 even when EVERY HTTP probe failed (connection refused, got=000 x9) because it had only set -u (no set -e, no FAIL-count aggregation) and the { ... } | tee pipeline masked the subshell exit status. A dead server was invisible to the standing audit; tick header kept claiming server up. Fix: FAILS counter on probe mismatch and literal-501 stubs, set -o pipefail so exit 1 inside the tee-d subshell propagates, summary line battery FAILED: N failure(s) - exit 1 / battery PASSED. Verified both directions: server down -> 9 FAIL got=000, exit 1; server up -> 9/9 PASS, exit 0.", "environment": "linux bash, curl probes against local HTTP server", "language": "bash", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "e2e-battery-script-exits-zero-on-all-probes-fail", "provider": "openrouter", "solved_at": "2026-08-24T05:06:56.956Z", "version": ""}