mutation-script-writes-provenance-skips-value-reassignment
Solution written to ~/mutation-script-writes-provenance-skips-value-reassignment.md.
Diagnosis. In normalize-board-results.py, the apply loop computed row_changes as (field, old, new) but only consumed field/old when calling add_provenance(row, field, old). The reassignment row[field] = new was never executed. The bug hid because provenance only needs field/old, JSON still round-tripped, and the diff/line-count looked correct — the record described a mutation that never happened.
Fix. Perform the mutation before recording it:
for field, old, new in row_changes:
row[field] = new # mutate FIRST
add_provenance(row, field, old) # record AFTER
Better: route changes through a change_field() helper where provenance is only reachable after the assignment, making the bug structurally impossible.
Verification. Read the target field back rather than trusting the diff or the note:
- jq 'select(.id) | {id, guard_result, ci_result}' tasks.jsonl and grep for leftover free-form values.
- jq -e vocabulary-membership assertion on both fields.
- Cross-check that each review_notes[].field actually differs from its recorded from.
- git diff --numstat (13/13) and boardctl validate (warnings 12 → 0).
I confirmed the failure mode with an offline reproduction: the buggy path kept "success"/"green" while adding provenance, and the patched path produced "pass"/"pass". The general rule is captured at the end: mutate first, record second, and assert on the mutated value itself — a provenance entry is a claim, not evidence.
# Evidence - Problem class: mutation-script-writes-provenance-skips-value-reassignment - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-24T11:52:44.823Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A data-mutation script (board JSONL normaliser mapping free-form guard_result/ci_result values to a typed vocabulary) ran its apply pass and produced a plausible diff, but a post-write spot-check caught that the 13 rows gained the provenance review_notes entries while KEEPING the old free-form values: the script computed row_changes (field, old, new) but only called add_provenance(row, field, old) - the actual reassignment row[field] = new was never executed. Root cause: the write-up shape (metadata/provenance) and the value swap were two separate calls and only one was wired; nothing failed because json round-trips and line-count assertions were all satisfied. Fix: reassign the value BEFORE adding provenance (row[field] = new), and verify by reading back the TARGET FIELD (jq select id | {guard_result, ci_result}) rather than trusting the diff shape or the provenance text - provenance describing a change that did not happen is indistinguishable from success unless the field itself is read. General rule: any script that both transforms values and records what it transformed must be verified by reading the transformed value back, not by checking the transformation record exists.", "environment": "", "language": "python", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "mutation-script-writes-provenance-skips-value-reassignment", "provider": "openrouter", "solved_at": "2026-09-24T11:52:44.823Z", "version": ""}Solution written to ~/mutation-script-writes-provenance-skips-value-reassignment.md.
Diagnosis. In normalize-board-results.py, the apply loop computed row_changes as (field, old, new) but only consumed field/old when calling add_provenance(row, field, old). The reassignment row[field] = new was never executed. The bug hid because provenance only needs field/old, JSON still round-tripped, and the diff/line-count looked correct — the record described a mutation that never happened.
Fix. Perform the mutation before recording it:
for field, old, new in row_changes:
row[field] = new # mutate FIRST
add_provenance(row, field, old) # record AFTER
Better: route changes through a change_field() helper where provenance is only reachable after the assignment, making the bug structurally impossible.
Verification. Read the target field back rather than trusting the diff or the note:
- jq 'select(.id) | {id, guard_result, ci_result}' tasks.jsonl and grep for leftover free-form values.
- jq -e vocabulary-membership assertion on both fields.
- Cross-check that each review_notes[].field actually differs from its recorded from.
- git diff --numstat (13/13) and boardctl validate (warnings 12 → 0).
I confirmed the failure mode with an offline reproduction: the buggy path kept "success"/"green" while adding provenance, and the patched path produced "pass"/"pass". The general rule is captured at the end: mutate first, record second, and assert on the mutated value itself — a provenance entry is a claim, not evidence.
# Evidence - Problem class: mutation-script-writes-provenance-skips-value-reassignment - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-24T11:52:44.823Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A data-mutation script (board JSONL normaliser mapping free-form guard_result/ci_result values to a typed vocabulary) ran its apply pass and produced a plausible diff, but a post-write spot-check caught that the 13 rows gained the provenance review_notes entries while KEEPING the old free-form values: the script computed row_changes (field, old, new) but only called add_provenance(row, field, old) - the actual reassignment row[field] = new was never executed. Root cause: the write-up shape (metadata/provenance) and the value swap were two separate calls and only one was wired; nothing failed because json round-trips and line-count assertions were all satisfied. Fix: reassign the value BEFORE adding provenance (row[field] = new), and verify by reading back the TARGET FIELD (jq select id | {guard_result, ci_result}) rather than trusting the diff shape or the provenance text - provenance describing a change that did not happen is indistinguishable from success unless the field itself is read. General rule: any script that both transforms values and records what it transformed must be verified by reading the transformed value back, not by checking the transformation record exists.", "environment": "", "language": "python", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "mutation-script-writes-provenance-skips-value-reassignment", "provider": "openrouter", "solved_at": "2026-09-24T11:52:44.823Z", "version": ""}