pytestmark = pytest.mark.benchmark(minrounds=3, maxtime=0.5)
Root cause. pytest-benchmark 5.2.3 registers its options exclusively as CLI flags — pytest_addoption calls group.addoption("--benchmark-…") and there is no addini() for them (the only ini line it registers is the marker description). Flags placed in addopts are parsed at pytest startup; if the plugin isn't installed/loaded, pytest aborts with unrecognized arguments: --benchmark-* before any test is collected.
Fix. (1) Remove every --benchmark-* flag from addopts; (2) gate the benchmark module behind pytest.importorskip and move the tuning parameters into the module-level benchmark marker, which the plugin does support and validates (min_rounds, max_time, min_time, timer, group, …).
pyproject.toml (before — crashes without the plugin):
[tool.pytest.ini_options]
addopts = "--benchmark-min-rounds=3 --benchmark-max-time=0.5"
pyproject.toml (after):
[tool.pytest.ini_options]
addopts = "-q" # no --benchmark-* flags
tests/test_perf.py (after):
import pytest
# No plugin installed -> whole module skips (no crash, no unknown-marker warning).
pytest.importorskip("pytest_benchmark")
# Module-level marker applies to every benchmark test in this module.
pytestmark = pytest.mark.benchmark(min_rounds=3, max_time=0.5)
def test_list_comp(benchmark):
benchmark(lambda: [i * i for i in range(1000)])
Verified in two fresh venvs (`pytest 9.1.1`; one with `pytest-benchmark 5.2.3`, one without), using `~/demo/{before,fixed}`.
| Scenario | Result |
|---|---|
| Before fix, **no plugin**: `pytest` | 💥 `error: unrecognized arguments: --benchmark-min-rounds=3 --benchmark-max-time=0.5` (startup crash) |
| After fix, **no plugin**: `pytest` | ✅ `1 skipped in 0.00s` — `importorskip` skips the module cleanly, exit 0 |
| After fix, **with plugin 5.2.3**: `pytest` | ✅ `1 passed in 1.21s` — benchmark ran, table shows `Rounds: 9016` (≥ `min_rounds=3`, stopped within the 0.5 s `max_time` budget) |
| After fix, marker wiring probe | ✅ Assertions inside the test passed: `benchmark._min_rounds == 3`, `benchmark._max_time == 0.5`, and default `_min_time` untouched |
Edge cases covered:
- **Marker semantics**: I read 5.2.3's source — `pytest_runtest_setup` raises `ValueError` for unknown/positional marker args, and the fixture factory merges marker kwargs over session options (`**dict(bs.options, **options)`), so module-level `pytestmark` legitimately overrides CLI defaults per-module.
- **No unknown-marker noise**: without the plugin, `importorskip` runs before the module body, so `pytestmark` is never registered on items — no `PytestUnknownMarkWarning`, unlike a bare marker without the guard.
- **Config-free addopts**: `-q` alone keeps output tidy without referencing plugin-only flags, so `pytest` stays runnable in CI images that lack the benchmark dependency.{"model": "deepseek-v4-flash", "problem_class": "python-pytest-benchmark-flag-gating", "result": "passed", "tests": 4}