python-compiler-incremental-macro-hygiene

Answer

Done. The verified solution is written to ~/SOLUTION.md, with the runnable implementation in ~/incremental_macro_cache.py and tests in ~/test_incremental_macro_cache.py.

Summary

Root cause: A cache keyed only on source bytes is wrong for this language. Generated output also depends on (1) transitively-imported macro definitions, (2) the deterministic hygiene salt, (3) the conditional-config environment, and (4) compiler/parser/hygiene versions. Diagnostics and source maps must be cached atomically with the generated bytes, or they drift on reuse.

Fix — Merkle-style invalidation keys + dependency propagation:

• Macro fingerprint folds in the canonical template, params, and the fingerprints of every macro used in the body (fixpoint over local macros), so a change to a root macro automatically changes the recipes of all transitive consumers.
• Module recipe = H(module, versions, source_hash, used_flags+values, used_macros+fingerprints, module name). Only flags/macros actually referenced are included, so unrelated edits/config stays cached.
• Deterministic hygiene salt derived from call_path + macro fingerprint, so cached output is byte-identical across runs.
• Source maps/diagnostics store origin spans: template-introduced nodes point at the macro call site; substituted user arguments keep their own spans.
• Explicit dependency graph gives an O(V+E) transitive dirty set for scheduling/reporting; the Merkle recipes remain the authoritative correctness mechanism.

Verification:

python3 -m unittest -v test_incremental_macro_cache   # 10 passed

All 10 tests pass and cover: unrelated-edit reuse, direct/transitive macro invalidation, used-vs-unused config flags, call-site diagnostic attribution, hygiene renaming, deterministic reuse, graph propagation, and arity diagnostics. The end-to-end demo confirms noise edit → only noise misses (6 hits), root-macro edit → dirty ['base','mid','top'] with noise still cached, and the macro-body error reports at user:2:16 (the call site). The embedded code in SOLUTION.md also compiles cleanly.