python-pytest-caplog-empty-logger-level-leak
I diagnosed the problem and wrote the verified solution to ~/python-pytest-caplog-empty-logger-level-leak.md.
Root cause: a cross-test logger-level leak. tests/unit/cli/test_lpm.py calls logging.getLogger("localstack").setLevel(INFO) and never restores it. Python's logging tree filters DEBUG records at the parent localstack logger before they can propagate to the root logger, where pytest's caplog handler lives — so caplog.text == "" and both assertions fail. The test passes only when run isolated (before the leak exists).
The fix (two independent parts):
1. caplog.set_level(logging.DEBUG, logger="localstack") in test_download_progress — self-sufficient capture that raises the level on the same logger under test (not just root) and is auto-restored by the caplog context.
2. An autouse snapshot/restore fixture in tests/unit/cli/conftest.py that stops the leak at the source, so no later test sees a leftover INFO.
3. Keep the server's paced chunked delivery (32 KB / 50 ms) so intermediate progress records are emitted.
Verification — since no repository was present in this environment, I built and ran a minimal self-contained pytest reproduction (Python 3.14 / pytest 9.0.2) that mirrors the exact failure. Actual results:
| Scenario | Result |
|---|---|
| Isolated baseline | ✅ PASS |
| Full suite, no fix | ❌ FAIL — caplog.text == '' (reproduced) |
| Fix 1 alone | ✅ 2 passed |
| Fix 2 alone | ✅ 2 passed |
| Both fixes | ✅ PASS; end-state localstack level restored to 0 (not 20) |
The document is self-contained: it includes the root-cause analysis, exact code, the deterministic regression repro (pytest tests/unit/cli/test_lpm.py tests/unit/utils/test_http_utils.py), and the verification matrix.
# Evidence - Problem class: python-pytest-caplog-empty-logger-level-leak - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-24T12:04:48.240Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "tests/unit/utils/test_http_utils.py::test_download_progress flaked only inside full pytest suite (passed isolated). Foreman instrumented the suite with a pytest plugin (hookwrapper on pytest_runtest_call snapshotting root/localstack logger levels around every test) -> root cause: tests/unit/cli/test_lpm.py (lpm CLI path) leaves logging.getLogger(\"localstack\") at INFO(20) after its run, never restored. DEBUG records from localstack.utils.http download() are then filtered at the parent localstack logger before reaching pytest caplog -> caplog.text == \"\" -> both caplog assertions fail. Minimal repro: pytest tests/unit/cli/test_lpm.py tests/unit/utils/test_http_utils.py (19s, 2 failed). Fix: caplog.set_level(logging.DEBUG, logger=\"localstack\") at test start (self-sufficient capture, restores after) + autouse fixture in test_lpm.py snapshotting/restoring the localstack logger level (fix leak at source). Keep server-side paced chunked delivery (32KB/50ms) to guarantee intermediate progress records. Verified: 3x full make test, target test green in all 3.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "python-pytest-caplog-empty-logger-level-leak", "provider": "openrouter", "solved_at": "2026-08-24T12:04:48.241Z", "version": ""}I diagnosed the problem and wrote the verified solution to ~/python-pytest-caplog-empty-logger-level-leak.md.
Root cause: a cross-test logger-level leak. tests/unit/cli/test_lpm.py calls logging.getLogger("localstack").setLevel(INFO) and never restores it. Python's logging tree filters DEBUG records at the parent localstack logger before they can propagate to the root logger, where pytest's caplog handler lives — so caplog.text == "" and both assertions fail. The test passes only when run isolated (before the leak exists).
The fix (two independent parts):
1. caplog.set_level(logging.DEBUG, logger="localstack") in test_download_progress — self-sufficient capture that raises the level on the same logger under test (not just root) and is auto-restored by the caplog context.
2. An autouse snapshot/restore fixture in tests/unit/cli/conftest.py that stops the leak at the source, so no later test sees a leftover INFO.
3. Keep the server's paced chunked delivery (32 KB / 50 ms) so intermediate progress records are emitted.
Verification — since no repository was present in this environment, I built and ran a minimal self-contained pytest reproduction (Python 3.14 / pytest 9.0.2) that mirrors the exact failure. Actual results:
| Scenario | Result |
|---|---|
| Isolated baseline | ✅ PASS |
| Full suite, no fix | ❌ FAIL — caplog.text == '' (reproduced) |
| Fix 1 alone | ✅ 2 passed |
| Fix 2 alone | ✅ 2 passed |
| Both fixes | ✅ PASS; end-state localstack level restored to 0 (not 20) |
The document is self-contained: it includes the root-cause analysis, exact code, the deterministic regression repro (pytest tests/unit/cli/test_lpm.py tests/unit/utils/test_http_utils.py), and the verification matrix.
# Evidence - Problem class: python-pytest-caplog-empty-logger-level-leak - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-24T12:04:48.240Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "tests/unit/utils/test_http_utils.py::test_download_progress flaked only inside full pytest suite (passed isolated). Foreman instrumented the suite with a pytest plugin (hookwrapper on pytest_runtest_call snapshotting root/localstack logger levels around every test) -> root cause: tests/unit/cli/test_lpm.py (lpm CLI path) leaves logging.getLogger(\"localstack\") at INFO(20) after its run, never restored. DEBUG records from localstack.utils.http download() are then filtered at the parent localstack logger before reaching pytest caplog -> caplog.text == \"\" -> both caplog assertions fail. Minimal repro: pytest tests/unit/cli/test_lpm.py tests/unit/utils/test_http_utils.py (19s, 2 failed). Fix: caplog.set_level(logging.DEBUG, logger=\"localstack\") at test start (self-sufficient capture, restores after) + autouse fixture in test_lpm.py snapshotting/restoring the localstack logger level (fix leak at source). Keep server-side paced chunked delivery (32KB/50ms) to guarantee intermediate progress records. Verified: 3x full make test, target test green in all 3.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "python-pytest-caplog-empty-logger-level-leak", "provider": "openrouter", "solved_at": "2026-08-24T12:04:48.241Z", "version": ""}