python-jsonl-duplicate-event-id-rekey
Done. I built the exact corruption, wrote the fix, and verified it end-to-end before writing it up.
/workspace/SOLUTION.md — the full self-contained solution (title, root-cause analysis, complete code, verification transcript, rollout order)./workspace/solution/:repair_jsonl.py — one-shot offline repair (first-copy-kept, drop byte-identical, byte-preserving rekey from max_id+1, atomic os.replace + .bak)eventlog.py — canonical appender minting ids inside the shared events-specific flock with O_APPENDaudit_jsonl.py — audit (zero dups) and reconcile (logical-event set preserved) gateshand_appender_red.py — RED witness reproducing the pre-fix private-lock mintmake_fixture.py, test_regression.pyTwo defects combined:
1. Writer race: max(existing_ids) + 1 was computed outside the lock; the private lock only covered the write, so concurrent writers minted identical ids.
2. Two corruption shapes requiring different repairs: byte-identical re-appends (same id + same bytes → drop later copies) vs. distinct-content collisions (same id, different bytes → re-key later copies).
$ python3 -m unittest test_regression
RED witness (8 private-lock appends): duplicate ids {1: 8} -> audit exit 1
GREEN (32 canonical appends): 32 unique ids 1..32 -> audit exit 0
$ python3 repair_jsonl.py events.jsonl
{"dropped_identical": 3, "rekeyed": 5, "lines_in": 23, "lines_out": 20,
"global_max_id": 15, "next_id": 21, "unique_ids_in": 15}
$ python3 audit_jsonl.py audit events.jsonl -> exit 0, 0 duplicates
$ python3 audit_jsonl.py reconcile *.bak events.jsonl -> exit 0, all logical events preserved
The raw diff shows exactly 3 whole-line deletions and 5 lines changed on the id field only, confirming byte preservation and order. All 5 regression tests pass.
# Evidence - Problem class: python-jsonl-duplicate-event-id-rekey - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-23T02:28:45.879Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "JSONL event log keyed by id accumulated duplicate ids because two writers minted max_id+1 outside a shared lock (3 byte-identical re-appended pairs + 5 distinct-content range collisions across ticks). Verified repair pattern: keep FIRST copy in file order; drop byte-identical later copies; re-key distinct colliding copies to fresh ids continuing from max_id+1 changing ONLY the id field, preserving line order and all other bytes; atomic write + .bak. Writer fix: single canonical appender minting ids inside an events-specific flock with O_APPEND; migrate all hand-appenders to it. Regression test: N concurrent appends yield N unique ids; keep a RED witness reproducing the pre-fix private-lock mint. Verify: Counter zero dups, all logical events survive, reconcile/audit tools exit 0. Ref: eduos BOARD-HYGIENE-004 commit f7883c79.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "python-jsonl-duplicate-event-id-rekey", "provider": "openrouter", "solved_at": "2026-09-23T02:28:45.880Z", "version": ""}