docs-cli-flag-reference-drift
Done. I diagnosed the problem class, reproduced it with a live CLI, and verified the fix end-to-end. Solution written to ~/solution.md.
Reproduction (/tmp/drift-demo): built an argparse CLI with real flags --out/--retries and a stale DOC.md claiming --output-dir/--retry-count. Running live --help exposed the drift.
Fix methodology (the verified source of truth is the binary, not memory/source order):
1. Enumerate subcommands and run <cmd> --help + <cmd> <sub> --help for each.
2. Put a "--help wins on drift" header on the page.
3. Copy usage lines, metavars, and help strings verbatim (defaults included).
4. Add a CI guard that diffs --flags in the doc against live --help.
Verification: extracted the exact guard script from the solution and ran it:
- stale page → FAIL exit=1 (lists --output-dir, --retry-count as stale)
- corrected page → PASS exit=0, including auto-discovery of subcommands
I also caught and fixed a real bug during verification: the first version of the guard had doc_path, cmd, *subs = ..., sys.argv[3:], which nested the subcommand list; corrected to subs = sys.argv[3:]. The published script is the tested one.
The document contains the root-cause analysis, exact commands, the reusable verify_cli_docs.py, a CI snippet, and an acceptance checklist.
# Evidence - Problem class: docs-cli-flag-reference-drift - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-10-07T09:53:37.283Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "docs-cli-flag-reference-drift", "provider": "openrouter", "solved_at": "2026-10-07T09:53:37.286Z", "version": ""}