test-tool-outputdir-points-at-a-tracked-directory
Solution written to ~/test-tool-outputdir-solution.md.
Diagnosis (reproduced in a scratch repo): Playwright's default outputDir is ./test-results relative to the config, and it wipes/rewrites that directory each run. Because the repo tracks real audit files in frontend/test-results/, they were deleted/modified. .gitignore cannot protect tracked files, and frontend/playwright-report/ was never actually ignored.
I reproduced the exact failure:
M frontend/test-results/.last-run.json
D frontend/test-results/accessibility-audit.md
D frontend/test-results/run-a11y-audit.mjs
?? frontend/playwright-report/
Fix (verified):
1. outputDir: './.playwright-artifacts' — a dedicated sibling path, not nested in the HTML reporter dir (confirmed Playwright rejects that with HTML reporter output folder clashes with the tests output folder).
2. Add frontend/playwright-report/ and frontend/.playwright-artifacts/ to .gitignore; never ignore frontend/test-results/.
3. Point CI/docs at that config.
Verification passing after the fix: re-ran the tool, git status --porcelain is empty, the tracked frontend/test-results/ files remain, and git check-ignore -v exits 0 with a matching rule for both generated dirs.
The document includes the config, .gitignore, exact verification commands with expected output, and the before/after evidence.
# Evidence - Problem class: test-tool-outputdir-points-at-a-tracked-directory - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-12T19:26:49.763Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Symptom: putting a test command in CI or docs silently damages the repo - tracked files disappear and an untracked report tree appears, so the next commit contains deletions nobody authored.\nRoot cause: Playwright's default outputDir IS ./test-results relative to the config, and it wipes/rewrites that directory on every run; this repo tracked real audit files (accessibility-audit*.json/.md, run-a11y-audit.*, .last-run.json) in it. Separately the HTML reporter's frontend/playwright-report/ was never git-ignored even though the docs claimed it was.\nFix: set outputDir to a dedicated, git-ignored path. Do NOT nest it inside the HTML reporter dir - Playwright rejects that with 'HTML reporter output folder clashes with the tests output folder' because the reporter clears its folder first. Then add the report and artifacts dirs to .gitignore, and never gitignore the tracked test-results dir.\nVerification that closes it: run the tool, then `git status --porcelain` must show no modification/deletion under the tracked directory and no untracked report dir, and `git check-ignore -v <report>/index.html` must exit 0 with a matching rule.\nLesson: a documented command is only fixed when running it leaves the working tree clean.", "environment": "playwright in a git repo that tracks audit artifacts under frontend/test-results/", "language": "typescript", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "test-tool-outputdir-points-at-a-tracked-directory", "provider": "openrouter", "solved_at": "2026-09-12T19:26:49.763Z", "version": ""}Solution written to ~/test-tool-outputdir-solution.md.
Diagnosis (reproduced in a scratch repo): Playwright's default outputDir is ./test-results relative to the config, and it wipes/rewrites that directory each run. Because the repo tracks real audit files in frontend/test-results/, they were deleted/modified. .gitignore cannot protect tracked files, and frontend/playwright-report/ was never actually ignored.
I reproduced the exact failure:
M frontend/test-results/.last-run.json
D frontend/test-results/accessibility-audit.md
D frontend/test-results/run-a11y-audit.mjs
?? frontend/playwright-report/
Fix (verified):
1. outputDir: './.playwright-artifacts' — a dedicated sibling path, not nested in the HTML reporter dir (confirmed Playwright rejects that with HTML reporter output folder clashes with the tests output folder).
2. Add frontend/playwright-report/ and frontend/.playwright-artifacts/ to .gitignore; never ignore frontend/test-results/.
3. Point CI/docs at that config.
Verification passing after the fix: re-ran the tool, git status --porcelain is empty, the tracked frontend/test-results/ files remain, and git check-ignore -v exits 0 with a matching rule for both generated dirs.
The document includes the config, .gitignore, exact verification commands with expected output, and the before/after evidence.
# Evidence - Problem class: test-tool-outputdir-points-at-a-tracked-directory - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-12T19:26:49.763Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Symptom: putting a test command in CI or docs silently damages the repo - tracked files disappear and an untracked report tree appears, so the next commit contains deletions nobody authored.\nRoot cause: Playwright's default outputDir IS ./test-results relative to the config, and it wipes/rewrites that directory on every run; this repo tracked real audit files (accessibility-audit*.json/.md, run-a11y-audit.*, .last-run.json) in it. Separately the HTML reporter's frontend/playwright-report/ was never git-ignored even though the docs claimed it was.\nFix: set outputDir to a dedicated, git-ignored path. Do NOT nest it inside the HTML reporter dir - Playwright rejects that with 'HTML reporter output folder clashes with the tests output folder' because the reporter clears its folder first. Then add the report and artifacts dirs to .gitignore, and never gitignore the tracked test-results dir.\nVerification that closes it: run the tool, then `git status --porcelain` must show no modification/deletion under the tracked directory and no untracked report dir, and `git check-ignore -v <report>/index.html` must exit 0 with a matching rule.\nLesson: a documented command is only fixed when running it leaves the working tree clean.", "environment": "playwright in a git repo that tracks audit artifacts under frontend/test-results/", "language": "typescript", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "test-tool-outputdir-points-at-a-tracked-directory", "provider": "openrouter", "solved_at": "2026-09-12T19:26:49.763Z", "version": ""}