20261005-incremental-grammar-repair

Answer

Done. Here's what I built and verified.

Summary

The environment contained no starter code or tests, so I implemented the incremental parser from scratch and produced a complete, self-contained solution document.

Artifacts in ~: - incremental_parser.py — dependency-free implementation (~725 lines) - test_incremental_parser.py — 17-test verification suite - demo.py — evidence run - SOLUTION.md — the requested markdown (title, root-cause analysis, exact fix, verification, full embedded code)

Root causes addressed

1. Ambiguous precedence — all binary ops on one level can't be a single tree; the chart packs multiple alternatives per span.
2. >> vs nested generics — lexing > individually (no >> token) lets List<List<int>> close.
3. Indentation — byte-exact INDENT/DEDENT lexer that ignores blank/comment lines and bracket continuations.
4. Distant invalidation — reuse a chart cell only when its exact token subsequence is unchanged, shifted by token delta; recompute every overlapping cell.
5. Epsilon on binary boundaries — if_rest -> cond_suite else_opt broke plain CYK; fixed with epsilon-aware unary closure.
6. Stable identity — structural hash-consing of forest nodes.
7. Malformed input — longest-prefix parse plus deterministic ERROR wrapper.

Verification results

Ran 17 tests in 0.158s
OK

Including a fuzz test of 120 random byte patches (insertions/deletions/replacements) across three sources, each matching a fresh full reparse structurally. Demo shows the affected interval is exactly the edited bytes (38, 47) with 237 cells reused, 169 recomputed, and all 54 unaffected first-statement node identities preserved. The code is Python 3.11-compatible (uses only __future__, dataclasses, typing).