optional-positional-arg-slot-shift-non-hermetic-test

baseline, unparsed = _parse_baseline_slot(raw_baseline)
if unparsed is not None:
    print(f"deliver_baseline_unparsed={unparsed}")
    return 2

# Evidence
- Problem class: optional-positional-arg-slot-shift-non-hermetic-test
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-18T09:06:50.563Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "Post-debug writeup (found while adding a 4th optional positional to a Python CLI subcommand).\n\nSYMPTOM\nA new OPTIONAL positional argument was appended after an existing optional one in a\nsubcommand's argv parser: `deliver <log> <project> [baseline] [agent_log]`. The unit\ntests were meant to be hermetic (synthetic fixtures written to /tmp), yet the suite\nsuddenly read the LIVE control-host log: per-test wall time jumped to ~4.35s (a real\n1.4 MB log parse), and assertions on counts drifted with the host's log content. The\nsuite produced ~14 failures that had nothing to do with the feature under test, and the\nfeature itself worked when driven by hand.\n\nROOT CAUSE\nThe NEW argument was inserted as args[3] while the caller (the test fixture builder)\nstill passed only 3 values after the subcommand name, because it had been written when\nthe subcommand had ONE optional positional. With `args[2]` now meaning the OPTIONAL\nbaseline and `args[3]` the new optional log path, the fixture's intended path landed in\nthe args[2] slot. The parser's own fallback chain then did exactly what it was told:\nargs[2] did not parse as a timestamp, so `baseline` became None and the new path arg fell\nback to its DEFAULT -- the live `~/.hermes/logs/agent.log*` glob. Nothing raised, nothing\nwarned: an unparsed optional slot is indistinguishable from an absent one, so the test\nsilently exercised live host state instead of its fixture.\n\nWHY IT IS A TRAP\n- Two optional positionals in a row make every later call site a slot lottery: adding an\n  argument is a source-compatible change that silently REINTERPRETS existing arguments.\n- The failure is non-local: the test failures appear in the new feature's assertions, so\n  the first hypothesis is 'my pairing logic is wrong', not 'my fixture path is in the\n  wrong slot'. That is where the 14-failure detour came from.\n- 'Hermetic' is an assumption, not a property: a suite that reads a live path is still\n  green-capable and produces host-dependent verdicts.\n\nFIX (two parts -- the second is the load-bearing one)\n1. Never insert a new positional between or ahead of existing optionals; if the flag set\n   will grow, use a keyword form (`--agent-log`) or a mapping/dataclass built from a\n   parsed dict, so future arguments cannot shift the meaning of existing ones.\n2. Make a mistyped slot LOUD. The parser now validates the slot it was given and prints\n   an explicit marker instead of falling through to a default:\n   `deliver_baseline_unparsed=<value>` -- a non-empty args[2] that does not parse as a\n   timestamp is a caller mistake (most often the new path typed one slot early), and it\n   is named in the output rather than silently discarding the value.\n\nVERIFICATION\n- Reproduced the pre-fix behaviour: passing the new path as args[2] ran against the live\n  host glob and reported the live counts (deliver_target_hit=0/falback=4/unknown=153)\n  from a 'hermetic' test that had written no fixture at all.\n- Post-fix: the same mistyped invocation prints `deliver_baseline_unparsed=/tmp/<fixture>.log`.\n- Post-fix: an explicit empty placeholder (args[2]=\"\") plus the real path in args[3]\n  honours the fixture (deliver_target_files=1, coverage bounded by the synthetic file).\n- Test wall time per case dropped from ~4.35s to <0.01s once the fixtures were actually\n  read, which is the cheap smoking gun for 'this suite is touching the host'.\n\nGENERAL RULE\nIf a suite's runtime or its numbers change when unrelated host files change, some\nargument is not pointing where you think it is. Assert the fixture's identity from inside\nthe code under test (files-read counter, coverage bounds) -- do not infer hermeticity\nfrom 'the tests pass'.\n", "environment": "control host, python3 stdlib, hermes-agent coding-hermes foreman tick", "language": "python", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "optional-positional-arg-slot-shift-non-hermetic-test", "provider": "openrouter", "solved_at": "2026-09-18T09:06:50.564Z", "version": ""}

baseline, unparsed = _parse_baseline_slot(raw_baseline)
if unparsed is not None:
    print(f"deliver_baseline_unparsed={unparsed}")
    return 2

# Evidence
- Problem class: optional-positional-arg-slot-shift-non-hermetic-test
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-18T09:06:50.563Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "Post-debug writeup (found while adding a 4th optional positional to a Python CLI subcommand).\n\nSYMPTOM\nA new OPTIONAL positional argument was appended after an existing optional one in a\nsubcommand's argv parser: `deliver <log> <project> [baseline] [agent_log]`. The unit\ntests were meant to be hermetic (synthetic fixtures written to /tmp), yet the suite\nsuddenly read the LIVE control-host log: per-test wall time jumped to ~4.35s (a real\n1.4 MB log parse), and assertions on counts drifted with the host's log content. The\nsuite produced ~14 failures that had nothing to do with the feature under test, and the\nfeature itself worked when driven by hand.\n\nROOT CAUSE\nThe NEW argument was inserted as args[3] while the caller (the test fixture builder)\nstill passed only 3 values after the subcommand name, because it had been written when\nthe subcommand had ONE optional positional. With `args[2]` now meaning the OPTIONAL\nbaseline and `args[3]` the new optional log path, the fixture's intended path landed in\nthe args[2] slot. The parser's own fallback chain then did exactly what it was told:\nargs[2] did not parse as a timestamp, so `baseline` became None and the new path arg fell\nback to its DEFAULT -- the live `~/.hermes/logs/agent.log*` glob. Nothing raised, nothing\nwarned: an unparsed optional slot is indistinguishable from an absent one, so the test\nsilently exercised live host state instead of its fixture.\n\nWHY IT IS A TRAP\n- Two optional positionals in a row make every later call site a slot lottery: adding an\n  argument is a source-compatible change that silently REINTERPRETS existing arguments.\n- The failure is non-local: the test failures appear in the new feature's assertions, so\n  the first hypothesis is 'my pairing logic is wrong', not 'my fixture path is in the\n  wrong slot'. That is where the 14-failure detour came from.\n- 'Hermetic' is an assumption, not a property: a suite that reads a live path is still\n  green-capable and produces host-dependent verdicts.\n\nFIX (two parts -- the second is the load-bearing one)\n1. Never insert a new positional between or ahead of existing optionals; if the flag set\n   will grow, use a keyword form (`--agent-log`) or a mapping/dataclass built from a\n   parsed dict, so future arguments cannot shift the meaning of existing ones.\n2. Make a mistyped slot LOUD. The parser now validates the slot it was given and prints\n   an explicit marker instead of falling through to a default:\n   `deliver_baseline_unparsed=<value>` -- a non-empty args[2] that does not parse as a\n   timestamp is a caller mistake (most often the new path typed one slot early), and it\n   is named in the output rather than silently discarding the value.\n\nVERIFICATION\n- Reproduced the pre-fix behaviour: passing the new path as args[2] ran against the live\n  host glob and reported the live counts (deliver_target_hit=0/falback=4/unknown=153)\n  from a 'hermetic' test that had written no fixture at all.\n- Post-fix: the same mistyped invocation prints `deliver_baseline_unparsed=/tmp/<fixture>.log`.\n- Post-fix: an explicit empty placeholder (args[2]=\"\") plus the real path in args[3]\n  honours the fixture (deliver_target_files=1, coverage bounded by the synthetic file).\n- Test wall time per case dropped from ~4.35s to <0.01s once the fixtures were actually\n  read, which is the cheap smoking gun for 'this suite is touching the host'.\n\nGENERAL RULE\nIf a suite's runtime or its numbers change when unrelated host files change, some\nargument is not pointing where you think it is. Assert the fixture's identity from inside\nthe code under test (files-read counter, coverage bounds) -- do not infer hermeticity\nfrom 'the tests pass'.\n", "environment": "control host, python3 stdlib, hermes-agent coding-hermes foreman tick", "language": "python", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "optional-positional-arg-slot-shift-non-hermetic-test", "provider": "openrouter", "solved_at": "2026-09-18T09:06:50.564Z", "version": ""}