go-reed-solomon-degraded-read-repair
Done. The solution is verified and delivered.
Deliverables
- ~/REED_SOLOMON_DEGRADED_READ_REPAIR.md — self-contained markdown: title, root-cause analysis, full fix (all source embedded, byte-identical to the working module), run instructions, and captured verification output.
- ~/rs-solution/ — the working Go 1.26 module (gf256.go, rs.go, store.go, repair.go + two test files, no external deps).
Root causes found and fixed
1. Silent bitrot propagates — shards were served without verification; now every replica is hashed against a write-time SHA-256 manifest before use, and mismatched copies are marked corrupt, skipped, and never served (read transparently re-fetches the mirror).
2. Non-minimal parity reads — the read path now fetches+verifies the 6 data shards first and pulls parity only until k=6 verified shards are in hand, so parityReads = 6 − goodData, provably minimal.
3. Reconstruction silently no-ops — Reconstruct sized buffers from slot 0 and assumed pre-allocated outputs; with nil slots for missing shards it short-circuited (len(nil)==0) and failed the parity cross-check. It now sizes from present shards and allocates missing outputs itself.
4. Repair not convergent under churn — replaced the one-shot plan with idempotent sweep passes (scrub → re-check targets → rebuild/copy → prune surplus only after targets verify), so node flaps and external migrations are absorbed within a bounded pass window while restoring the exact minimal set: 8 shards × 2 verified copies per object.
Verification (run: gofmt -l . clean, go vet ./... clean, go test -race -v ./... → 17/17 PASS, 42.7s):
- Exhaustive: all 28 six-shard subsets reconstruct the 4 MiB object bit-for-bit; all 28 loss pairs read correctly with the exact minimum parity reads.
- Bitrot: primary-only corruption → mirror re-fetch (corrupt=1, replica=1, parity=0); both copies corrupt → RS reconstruction, still correct output.
- Repair: 2 antipodal node failures → converged to exactly 2 verified copies/shard, then a second different 2-node failure is tolerated and repair re-converges.
- Churn: partial repair + extra node death + node flap-back + externally migrated shard → converges within the pass bound to the exact minimal set.
- Concurrency: 4 goroutines reading during repair under -race — every read returns exact bytes.
One subtle finding worth noting: the naive test expectation "failing antipodal nodes ⇒ 2 parity reads" is wrong when the pair includes a parity shard (e.g. {3,7} loses data 3 + parity 7 → minimum is 1 parity read); the suite asserts the true minimum per scenario.
# Evidence - Problem class: go-reed-solomon-degraded-read-repair - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-09-04T04:25:20.193Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A sharded object store stripes each 4MiB object as 6 data + 2 parity shards (Reed-Solomon over GF(2^8)) across 8 nodes and can answer reads from any k=6 shards. Implement the degraded-read path that reconstructs an object from an arbitrary live-shard set with the minimum number of parity reads, and make it corruption-safe: when a live shard fails checksum verification (silent bitrot) the read must transparently re-fetch from another replica instead of emitting corrupt output. Also implement the repair scheduler that, after 2 node failures, restores the exact minimum shard set per object while shards may migrate mid-repair under a bounded churn window.", "environment": "go1.26", "language": "go", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "go-reed-solomon-degraded-read-repair", "provider": "openrouter", "solved_at": "2026-09-04T04:25:20.194Z", "version": "1.26"}Done. The solution is verified and delivered.
Deliverables
- ~/REED_SOLOMON_DEGRADED_READ_REPAIR.md — self-contained markdown: title, root-cause analysis, full fix (all source embedded, byte-identical to the working module), run instructions, and captured verification output.
- ~/rs-solution/ — the working Go 1.26 module (gf256.go, rs.go, store.go, repair.go + two test files, no external deps).
Root causes found and fixed
1. Silent bitrot propagates — shards were served without verification; now every replica is hashed against a write-time SHA-256 manifest before use, and mismatched copies are marked corrupt, skipped, and never served (read transparently re-fetches the mirror).
2. Non-minimal parity reads — the read path now fetches+verifies the 6 data shards first and pulls parity only until k=6 verified shards are in hand, so parityReads = 6 − goodData, provably minimal.
3. Reconstruction silently no-ops — Reconstruct sized buffers from slot 0 and assumed pre-allocated outputs; with nil slots for missing shards it short-circuited (len(nil)==0) and failed the parity cross-check. It now sizes from present shards and allocates missing outputs itself.
4. Repair not convergent under churn — replaced the one-shot plan with idempotent sweep passes (scrub → re-check targets → rebuild/copy → prune surplus only after targets verify), so node flaps and external migrations are absorbed within a bounded pass window while restoring the exact minimal set: 8 shards × 2 verified copies per object.
Verification (run: gofmt -l . clean, go vet ./... clean, go test -race -v ./... → 17/17 PASS, 42.7s):
- Exhaustive: all 28 six-shard subsets reconstruct the 4 MiB object bit-for-bit; all 28 loss pairs read correctly with the exact minimum parity reads.
- Bitrot: primary-only corruption → mirror re-fetch (corrupt=1, replica=1, parity=0); both copies corrupt → RS reconstruction, still correct output.
- Repair: 2 antipodal node failures → converged to exactly 2 verified copies/shard, then a second different 2-node failure is tolerated and repair re-converges.
- Churn: partial repair + extra node death + node flap-back + externally migrated shard → converges within the pass bound to the exact minimal set.
- Concurrency: 4 goroutines reading during repair under -race — every read returns exact bytes.
One subtle finding worth noting: the naive test expectation "failing antipodal nodes ⇒ 2 parity reads" is wrong when the pair includes a parity shard (e.g. {3,7} loses data 3 + parity 7 → minimum is 1 parity read); the suite asserts the true minimum per scenario.
# Evidence - Problem class: go-reed-solomon-degraded-read-repair - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-09-04T04:25:20.193Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "A sharded object store stripes each 4MiB object as 6 data + 2 parity shards (Reed-Solomon over GF(2^8)) across 8 nodes and can answer reads from any k=6 shards. Implement the degraded-read path that reconstructs an object from an arbitrary live-shard set with the minimum number of parity reads, and make it corruption-safe: when a live shard fails checksum verification (silent bitrot) the read must transparently re-fetch from another replica instead of emitting corrupt output. Also implement the repair scheduler that, after 2 node failures, restores the exact minimum shard set per object while shards may migrate mid-repair under a bounded churn window.", "environment": "go1.26", "language": "go", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "go-reed-solomon-degraded-read-repair", "provider": "openrouter", "solved_at": "2026-09-04T04:25:20.194Z", "version": "1.26"}