echo "$rep" | jq -e '.status == "ok" and .tick == 48' >/dev/null # exit 0 = PASS
Verdict: zero app gaps. All 3 battery FAILs were script bugs. The fix is to correct the battery assertions, not the CLI. The canonical contract is byte-exact — compare bytes, never grep.
Reconstructed repro at ~/foreman-direct/ (main.go, battery_test.go 26-case Go battery, battery.sh, artifact-demo.sh).
1. Coarse grep on JSON → parse fields exactly. Compact JSON {"tick":48,"status":"ok",...} defeats grep '"status": "ok"'. Fixed with jq (or encoding/json):
# BUGGY: echo "$rep" | grep -o '"status": "ok"'
# FIXED: parse, don't pattern-match
echo "$rep" | jq -e '.status == "ok" and .tick == 48' >/dev/null # exit 0 = PASS
[ "$(echo "$rep" | jq 'keys | length')" = "3" ] # schema lock
[ "$rep" = "$("$BIN" report --tick 48)" ] # deterministic bytes
2. Wrong DB filename pattern → glob the real contract. The CLI writes foreman-<tick>-run.db; scripts looked for foreman.db / *.sqlite. Fixed:
"$BIN" init --dir "$TMP" --tick 48
db=$(find "$TMP" -maxdepth 1 -name 'foreman-*-run.db' -printf '%f\n')
printf '%s' "$db" | grep -Eq '^foreman-[0-9]+-run\.db$' # exact pattern
[ "$(find "$TMP" -maxdepth 1 -type f -name '*.db' | wc -l)" -eq 1 ] # no stray artifacts
3. Empty-body MCP expectation → the contract body is non-empty. [ -z "$mcp" ] failed against a correct app. Fixed:
mcp=$("$BIN" mcp)
[ -n "$mcp" ] # non-empty by contract
echo "$mcp" | jq -e '.resource == "flow" and .content == "42\ntrigger=none" and .ok == true'
Canonical flow — byte-exact, verified with cmp (never grep):
"$BIN" source > out.txt; printf '42\ntrigger=none\n' > exp.txt
cmp -s out.txt exp.txt && [ "$(wc -c < out.txt)" -eq 16 ]
- `go test -v ./...` → **26 PASS, 0 FAIL** (`ok musterflow/foreman-direct`); `go vet` + `gofmt -l` clean. - `battery.sh` → **18/18 PASS, rc=0** (canonical byte-exact, report jq schema, db pattern, mcp body). - `artifact-demo.sh` → proves the lesson: each buggy assertion **fails** (grep exit 1, `ls foreman.db` exit 2, `[ -z "$mcp" ]` exit 1) while the fixed assertion and the app contract pass — i.e., the FAILs were script artifacts. - Byte-exact contract: `xxd` of `source` output == `xxd` of `42\ntrigger=none\n` (`3432 0a74 7269 6767 6572 3d6e 6f6e 650a`). - Edge cases: empty `-trigger=` guards to `none`; whitespace trigger `" "` is a *real* trigger (not none); 16-byte exact count, single trailing newline; idempotent double-run; no browser surface (html/script/http absent); report field order + exactly 3 keys; db tick templating (`--tick 7` → `foreman-7-run.db`); `init` writes into an existing dir (pre-creating it is the caller's job — a test assumption, not an app gap); semver `version`. - Two self-inflicted test bugs found and fixed the same way: my `TestInitCustomDir` assumed auto-mkdir (fixed the test), and a `battery.sh` "one file" count included the script's own temp files (scoped to `*.db`). Both were test-side, matching the lesson.
{"model": "deepseek-v4-flash", "problem_class": "go-cli-e2e-battery", "result": "passed", "tests": 26}Verdict: zero app gaps. All 3 battery FAILs were script bugs. The fix is to correct the battery assertions, not the CLI. The canonical contract is byte-exact — compare bytes, never grep.
Reconstructed repro at ~/foreman-direct/ (main.go, battery_test.go 26-case Go battery, battery.sh, artifact-demo.sh).
1. Coarse grep on JSON → parse fields exactly. Compact JSON {"tick":48,"status":"ok",...} defeats grep '"status": "ok"'. Fixed with jq (or encoding/json):
# BUGGY: echo "$rep" | grep -o '"status": "ok"'
# FIXED: parse, don't pattern-match
echo "$rep" | jq -e '.status == "ok" and .tick == 48' >/dev/null # exit 0 = PASS
[ "$(echo "$rep" | jq 'keys | length')" = "3" ] # schema lock
[ "$rep" = "$("$BIN" report --tick 48)" ] # deterministic bytes
2. Wrong DB filename pattern → glob the real contract. The CLI writes foreman-<tick>-run.db; scripts looked for foreman.db / *.sqlite. Fixed:
"$BIN" init --dir "$TMP" --tick 48
db=$(find "$TMP" -maxdepth 1 -name 'foreman-*-run.db' -printf '%f\n')
printf '%s' "$db" | grep -Eq '^foreman-[0-9]+-run\.db$' # exact pattern
[ "$(find "$TMP" -maxdepth 1 -type f -name '*.db' | wc -l)" -eq 1 ] # no stray artifacts
3. Empty-body MCP expectation → the contract body is non-empty. [ -z "$mcp" ] failed against a correct app. Fixed:
mcp=$("$BIN" mcp)
[ -n "$mcp" ] # non-empty by contract
echo "$mcp" | jq -e '.resource == "flow" and .content == "42\ntrigger=none" and .ok == true'
Canonical flow — byte-exact, verified with cmp (never grep):
"$BIN" source > out.txt; printf '42\ntrigger=none\n' > exp.txt
cmp -s out.txt exp.txt && [ "$(wc -c < out.txt)" -eq 16 ]
- `go test -v ./...` → **26 PASS, 0 FAIL** (`ok musterflow/foreman-direct`); `go vet` + `gofmt -l` clean. - `battery.sh` → **18/18 PASS, rc=0** (canonical byte-exact, report jq schema, db pattern, mcp body). - `artifact-demo.sh` → proves the lesson: each buggy assertion **fails** (grep exit 1, `ls foreman.db` exit 2, `[ -z "$mcp" ]` exit 1) while the fixed assertion and the app contract pass — i.e., the FAILs were script artifacts. - Byte-exact contract: `xxd` of `source` output == `xxd` of `42\ntrigger=none\n` (`3432 0a74 7269 6767 6572 3d6e 6f6e 650a`). - Edge cases: empty `-trigger=` guards to `none`; whitespace trigger `" "` is a *real* trigger (not none); 16-byte exact count, single trailing newline; idempotent double-run; no browser surface (html/script/http absent); report field order + exactly 3 keys; db tick templating (`--tick 7` → `foreman-7-run.db`); `init` writes into an existing dir (pre-creating it is the caller's job — a test assumption, not an app gap); semver `version`. - Two self-inflicted test bugs found and fixed the same way: my `TestInitCustomDir` assumed auto-mkdir (fixed the test), and a `battery.sh` "one file" count included the script's own temp files (scoped to `*.db`). Both were test-side, matching the lesson.
{"model": "deepseek-v4-flash", "problem_class": "go-cli-e2e-battery", "result": "passed", "tests": 26}