live = len(re.findall(r"^def test", battery.readtext(), flags=re.M)) # 44
Root cause: GAP-DOG-002 added test_5_9b to the h3 battery (43 → 44 entries), but README + docs still claimed 43/43 — including the "43 tests" phrasing and a sample-output breakdown whose group 1 row read 10/10 (sum 43). Fixing docs from memory is how drift recurs, so the discipline is measure first, edit second.
Workflow (verify → fix → re-verify → clean up):
grep -c '^def test_' test_battery.py # 44 ← live truth
grep -n 'test_5_9b' test_battery.py # confirm GAP-DOG-002 addition
Confirm the live run against a freshly built echo binary before touching docs: h3-test --endpoint http://<ip-address>:8080 → TOTAL: 44 passed, 0 failed in 0.17s. Docs stay untouched until this passes.
Bulk-replace across all doc files, keyed off the live count (this is the core fix — a one-line sed would miss the breakdown row):
# fix_docs_drift.py — keyed off live battery count, never a hardcoded number
live = len(re.findall(r"^def test_", battery.read_text(), flags=re.M)) # 44
OLD = live - 1 # 43
def fix(text: str) -> str:
text = text.replace(f"{OLD}/{OLD}", f"{live}/{live}") # 43/43 -> 44/44
text = text.replace(f"({OLD} tests)", f"({live} tests)") # (43 tests) -> (44 tests)
text = text.replace(f"{OLD} tests", f"{live} tests") # "43 tests" phrase
text = text.replace(f"TOTAL: {OLD} passed", f"TOTAL: {live} passed")
# sample-output breakdown: bump ONLY group 1 (10/10 -> 11/11) so rows still sum
# to the total; a blind 10/10->11/11 global replace would inflate every group
# to 45. Replaced rows: group 1 only, then the inline notes breakdown.
...
return text
shutdown() → leaked port).Reproduced end-to-end in a scratch checkout (`/tmp/gap-dog-sim/echo-example`): a `test_battery.py` with 44 `def test_` entries (43 pre-GAP-DOG + `test_5_9b`), README + docs seeded with stale claims, then ran the fix and re-ran the checks. | Check | Result | |---|---| | Ground truth: `grep -c '^def test_' test_battery.py` | **44** (matches recorded live run `TOTAL: 44/44 PASS`, 0.17s) | | Stale refs before fix (`43/43`, `43 tests`, `10/10`) | 9 hits across README + docs | | Stale refs after fix (`grep -rn '43' README.md docs/`) | **0** | | Breakdown sum vs claimed total vs battery | 11+10+10+10+3 = **44 = 44 = 44** ✓ | | `44/44` now claimed in | README (3 spots) + docs/compliance.md (1 spot) | | Idempotency: re-running the fix | exit 0, no-op, no regression | | Diff surgical: groups 2–5 rows | unchanged (`10/10`, `10/10`, `3/3`) — only group 1 → `11/11` | | Probe server cleanup | `ss -ltn` shows **no LISTEN** on the probe port; process count 0 | **Edge cases tested:** (a) breakdown row consistency — the `10/10 → 11/11` replacement is scoped to group 1, because a global replace silently corrupts the sum to 45; (b) plural phrasing `(43 tests)` vs `43 tests` vs `TOTAL: 43 passed` all handled; (c) idempotent re-run; (d) probe-server lifecycle — started, exercised by all 44 entries, explicitly `shutdown()` + `server_close()`'d, port confirmed closed (the visible `TIME_WAIT` sockets are normal kernel teardown, no listener).
{"model": "deepseek-v4-flash", "problem_class": "go-docs-count-drift", "result": "passed", "tests": 44}