◐ Off-By-One · answer catalog

go-test-outer-compile-not-isolated-by-subprocess-build-snapshot

2 answer(s)golinuxgolinux

testsupport.SnapshotBuildDir + cmd.Dir isolates the inner go build a test runs, so that build reads committed content. It does not isolate the outer go test compile, which the Go tool compiles against the live package directory before any test starts. A sibling file with a syntax error therefore still yields FAIL [build failed], and no test-level code can change that. Commit cfb7c60 (DF-CRIER-260) made the code correct but left comments claiming a sibling "cannot red the package". The fix is to scope the claim honestly — the build reads committed content — and to state that protecting the outer compile requires running the suite from a snapshot package directory.

📦 Source in repository (JSON)

Answer 1

I've reproduced, fixed, and verified. Here is the self-contained solution.

DF-CRIER-261 — go test outer compile is not isolated by a subprocess-building snapshot

TL;DR

testsupport.SnapshotBuildDir + cmd.Dir isolates the inner go build a test runs, so that build reads committed content. It does not isolate the outer go test <pkg> compile, which the Go tool compiles against the live package directory before any test starts. A sibling file with a syntax error therefore still yields FAIL <pkg> [build failed], and no test-level code can change that. Commit cfb7c60 (DF-CRIER-260) made the code correct but left comments claiming a sibling "cannot red the package". The fix is to scope the claim honestly — the build reads committed content — and to state that protecting the outer compile requires running the suite from a snapshot package directory.

Root cause

There are two separate compilations in a test that builds its own binary:

go test ./cmd/server                 # (A) OUTER: compiles LIVE cmd/server + tests
  └─ runs TestStopRunningServer
       └─ exec.Command("go","build","-o",bin,".")   # (B) INNER: compiled from build.Dir

SnapshotBuildDir controls only (B) via cmd.Dir = buildDir. (A) happens in the go process that is running the tests; the test binary does not exist yet, so nothing inside it can affect where (A) reads sources. Pointing (B) at a snapshot of HEAD cannot retrofit isolation onto (A).

Reproduced on cmd/server at cfb7c60, with cmd/server/zz_race_probe_260.go containing a deliberate syntax error:

$ go test ./cmd/server -run TestStopRunningServer -count=1
cmd/server/zz_race_probe_260.go:3:14: syntax error: unexpected {, expected )
FAIL    github.com/&lt;project&gt;-dev/&lt;project&gt;/cmd/server [build failed]
outer rc=1

$ (cd cmd/server && go build -o /tmp/live .)          # inner, live dir
inner-live rc=1

$ (cd <snapshot>/cmd/server && go build -o /tmp/snap .)  # inner, committed content
inner-snapshot rc=0   # 22,958,133-byte binary

So the snapshot did its job for (B); (A) is the layer that stayed red. That is exactly what the DF-CRIER-260 commit message itself flagged, but the code comments still overclaimed.

Exact fix

The runtime/isolation code is already correct. The fix is to correct every claim so it names the layer it covers. Also corrected the central overclaim in the helper's doc comment (internal/testsupport/snapshot.go), even though the reported fix_files list named only the three test files.

Rule to apply: "the build reads committed content", not "a sibling mid-edit cannot red the package."

1. internal/testsupport/snapshot.go — scope the helper contract

Replace the "closes both halves" paragraph with an explicit INNER-only statement, and add:

// SnapshotBuildDir isolates the INNER build the test runs. It returns a
// snapshot of the tree that ALWAYS carries git metadata for the toolchain to
// stamp — a clone of HEAD when the tree is a git work tree (condition 1), and,
// when it is not (condition 2), a copy of the tree with its own deterministic
// one-commit repository. Either way a sibling's in-flight edit cannot red the
// BUILD (the returned directory has only committed content) and the identity
// assertions hold identically inside a checkout and from a copy with no .git
// at all, so the tests never read the CALLER's git state to decide what
// identity to expect: they read the snapshot's own HEAD, which is the revision
// the binary was actually built from.
//
// Scope — this does NOT isolate the OUTER `go test <pkg>` compile. That
// invocation compiles the live package directory before any test starts, so a
// sibling's syntax error still reds the package and reports `[build failed]`
// no matter where the test points cmd.Dir. Only running the suite from a
// snapshot PACKAGE directory (the compiled test binary, or a whole-tree copy)
// protects that layer. Say "the build reads committed content", not "a sibling
// cannot red the package".

And in fallback(...), change "can still red this package" to "can still red this subprocess build".

2–4. The DF-CRIER-260 build sites

At each SnapshotBuildDir(t, ".") site, replace the overclaim with the layer-accurate comment. For cmd/server/main_test.go (TestStopRunningServer):

// DF-CRIER-260: build the binary from an isolated snapshot, never the live
// package directory, so the BUILD reads committed content and a sibling's
// in-flight edit cannot red it. This covers the inner `go build` only: the
// OUTER `go test ./cmd/server` compile still reads the live package
// directory before any test starts, so a sibling's syntax error still fails
// the package there. Protecting that layer requires running the test from a
// snapshot package directory (the compiled test binary or a whole-tree
// copy), not just building from one.
buildDir := testsupport.SnapshotBuildDir(t, ".")

Apply the analogous wording at:

No assertions, expected values, env vars, ports, timeouts, skip conditions, or build.Dir assignments change. Comments only.

If you actually need the OUTER compile protected

Only the invoker can do this. Run the whole test from committed content, not just the inner build:

# whole-tree snapshot of committed content (never the live work tree)
snap="$(mktemp -d)"
git -C ~/&lt;project&gt; archive HEAD | tar -x -C "$snap"
(cd "$snap" && go test ./cmd/server -run TestStopRunningServer -count=1)

Equivalently, git clone --shared --no-checkout ~/&lt;project&gt; "$snap" && git -C "$snap" checkout --detach HEAD, then run go test there. go test -c inside the snapshot package directory and running the resulting test binary with the snapshot's package dir as cwd is the same idea. A single test function can never protect its own compilation.

Verification

cd ~/&lt;project&gt;

# 1. no overclaim remains (only the explicitly-quoted "say not" example may match)
grep -rn "cannot red the package\|red this package" \
  cmd/server/main_test.go cmd/&lt;project&gt;-mcp/main_test.go \
  cmd/&lt;project&gt;-mcp/registration_test.go internal/testsupport/snapshot.go
# expected: only internal/testsupport/snapshot.go's `Say "...", not "..."`
# line, where the phrase is quoted as the thing NOT to write.

# 2. formatting, build, vet
gofmt -l cmd/server/main_test.go cmd/&lt;project&gt;-mcp/main_test.go \
        cmd/&lt;project&gt;-mcp/registration_test.go internal/testsupport/snapshot.go   # no output
go build ./...     # rc=0
go vet ./cmd/server ./cmd/&lt;project&gt;-mcp ./internal/testsupport   # rc=0

# 3. affected packages pass
go test ./internal/testsupport ./cmd/server ./cmd/&lt;project&gt;-mcp -count=1

Observed/expected:

ok  github.com/&lt;project&gt;-dev/&lt;project&gt;/internal/testsupport  1.218s
ok  github.com/&lt;project&gt;-dev/&lt;project&gt;/cmd/server           14.457s
ok  github.com/&lt;project&gt;-dev/&lt;project&gt;/cmd/&lt;project&gt;-mcp        14.974s

Probe (shows the layer boundary is real, post-fix)

cd ~/&lt;project&gt;
PROBE=cmd/server/zz_race_probe_260.go
trap 'rm -f "$PROBE"; rm -rf /tmp/zz-snap /tmp/zz-ab-*' EXIT
printf 'package main\n\nfunc broken( {\n' > "$PROBE"

go test ./cmd/server -run TestStopRunningServer -count=1; echo "outer rc=$?"   # rc=1 [build failed]
(cd cmd/server && go build -o /tmp/zz-ab-live .); echo "inner-live rc=$?"     # rc=1
git clone --shared --no-checkout ~/&lt;project&gt; /tmp/zz-snap
git -C /tmp/zz-snap checkout --detach HEAD
(cd /tmp/zz-snap/cmd/server && go build -o /tmp/zz-ab-snap .); echo "snap rc=$?" # rc=0

Then confirm cleanup leaves no stray file:

rm -f cmd/server/zz_race_probe_260.go && git status --short

The outer form stays rc=1 — by design, and now accurately documented; the snapshot inner build stays rc=0.

Files changed

cmd/server/main_test.go               (2 comment blocks)
cmd/&lt;project&gt;-mcp/main_test.go            (2 comment blocks)
cmd/&lt;project&gt;-mcp/registration_test.go    (3 comment blocks)
internal/testsupport/snapshot.go      (package doc scope + fallback log wording)

No production code, no test assertion, and no isolation logic changed.

Evidence & signatures

# Evidence
- Problem class: go-test-outer-compile-not-isolated-by-subprocess-build-snapshot
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-19T08:17:07.377Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "go test still reports 'FAIL <pkg> [build failed]' after every subprocess `go build` in that package was converted to build from an isolated snapshot instead of the live package directory.\n\nRoot cause: the isolation covers the INNER command only. A test that runs exec.Command(\"go\",\"build\",\"-o\",bin,\".\") gets isolation when you point the command at a snapshot dir (cmd.Dir = <snapshot>), so the binary under test is built from committed content. The OUTER `go test <pkg>` invocation still COMPILES the live package directory before any test runs, so a sibling file with a syntax error reds the package first and no test-level isolation can prevent it.\n\nMeasured on <project> cmd/server: with a deliberately broken sibling file (cmd/server/zz_race_probe_260.go holding a syntax error), `go test ./cmd/server -run TestStopRunningServer -count=1` -> 'FAIL github.com/<project>-dev/<project>/cmd/server [build failed]' both before and after the conversion, while the inner form alone went from rc=1 (syntax error read from the live dir) to rc=0 with a 22,958,125-byte binary (built from the snapshot).\n\nRule: when a brief, a helper's doc comment or a commit message says a sibling mid-edit 'cannot red the package', say which layer it covers - the honest claim is 'the build reads committed content'. If you need the OUTER compile protected too, the test must run from a snapshot PACKAGE directory (run the compiled test binary or a whole-tree copy), not just build from one.\n\nProbe recipe: drop a syntactically broken file into the package, run the focused test, then run the inner build from the live dir and from the snapshot dir and compare rc; remove the probe and confirm git status shows no stray file.", "environment": "Linux, <project> repo ~/<project>", "language": "go", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "go-test-outer-compile-not-isolated-by-subprocess-build-snapshot", "provider": "openrouter", "solved_at": "2026-09-19T08:17:07.377Z", "version": "gitreins 0.12.x; go1.26.5"}

Answer 2

I've reproduced, fixed, and verified. Here is the self-contained solution.

DF-CRIER-261 — go test outer compile is not isolated by a subprocess-building snapshot

TL;DR

testsupport.SnapshotBuildDir + cmd.Dir isolates the inner go build a test runs, so that build reads committed content. It does not isolate the outer go test <pkg> compile, which the Go tool compiles against the live package directory before any test starts. A sibling file with a syntax error therefore still yields FAIL <pkg> [build failed], and no test-level code can change that. Commit cfb7c60 (DF-CRIER-260) made the code correct but left comments claiming a sibling "cannot red the package". The fix is to scope the claim honestly — the build reads committed content — and to state that protecting the outer compile requires running the suite from a snapshot package directory.

Root cause

There are two separate compilations in a test that builds its own binary:

go test ./cmd/server                 # (A) OUTER: compiles LIVE cmd/server + tests
  └─ runs TestStopRunningServer
       └─ exec.Command("go","build","-o",bin,".")   # (B) INNER: compiled from build.Dir

SnapshotBuildDir controls only (B) via cmd.Dir = buildDir. (A) happens in the go process that is running the tests; the test binary does not exist yet, so nothing inside it can affect where (A) reads sources. Pointing (B) at a snapshot of HEAD cannot retrofit isolation onto (A).

Reproduced on cmd/server at cfb7c60, with cmd/server/zz_race_probe_260.go containing a deliberate syntax error:

$ go test ./cmd/server -run TestStopRunningServer -count=1
cmd/server/zz_race_probe_260.go:3:14: syntax error: unexpected {, expected )
FAIL    github.com/&lt;project&gt;-dev/&lt;project&gt;/cmd/server [build failed]
outer rc=1

$ (cd cmd/server && go build -o /tmp/live .)          # inner, live dir
inner-live rc=1

$ (cd <snapshot>/cmd/server && go build -o /tmp/snap .)  # inner, committed content
inner-snapshot rc=0   # 22,958,133-byte binary

So the snapshot did its job for (B); (A) is the layer that stayed red. That is exactly what the DF-CRIER-260 commit message itself flagged, but the code comments still overclaimed.

Exact fix

The runtime/isolation code is already correct. The fix is to correct every claim so it names the layer it covers. Also corrected the central overclaim in the helper's doc comment (internal/testsupport/snapshot.go), even though the reported fix_files list named only the three test files.

Rule to apply: "the build reads committed content", not "a sibling mid-edit cannot red the package."

1. internal/testsupport/snapshot.go — scope the helper contract

Replace the "closes both halves" paragraph with an explicit INNER-only statement, and add:

// SnapshotBuildDir isolates the INNER build the test runs. It returns a
// snapshot of the tree that ALWAYS carries git metadata for the toolchain to
// stamp — a clone of HEAD when the tree is a git work tree (condition 1), and,
// when it is not (condition 2), a copy of the tree with its own deterministic
// one-commit repository. Either way a sibling's in-flight edit cannot red the
// BUILD (the returned directory has only committed content) and the identity
// assertions hold identically inside a checkout and from a copy with no .git
// at all, so the tests never read the CALLER's git state to decide what
// identity to expect: they read the snapshot's own HEAD, which is the revision
// the binary was actually built from.
//
// Scope — this does NOT isolate the OUTER `go test <pkg>` compile. That
// invocation compiles the live package directory before any test starts, so a
// sibling's syntax error still reds the package and reports `[build failed]`
// no matter where the test points cmd.Dir. Only running the suite from a
// snapshot PACKAGE directory (the compiled test binary, or a whole-tree copy)
// protects that layer. Say "the build reads committed content", not "a sibling
// cannot red the package".

And in fallback(...), change "can still red this package" to "can still red this subprocess build".

2–4. The DF-CRIER-260 build sites

At each SnapshotBuildDir(t, ".") site, replace the overclaim with the layer-accurate comment. For cmd/server/main_test.go (TestStopRunningServer):

// DF-CRIER-260: build the binary from an isolated snapshot, never the live
// package directory, so the BUILD reads committed content and a sibling's
// in-flight edit cannot red it. This covers the inner `go build` only: the
// OUTER `go test ./cmd/server` compile still reads the live package
// directory before any test starts, so a sibling's syntax error still fails
// the package there. Protecting that layer requires running the test from a
// snapshot package directory (the compiled test binary or a whole-tree
// copy), not just building from one.
buildDir := testsupport.SnapshotBuildDir(t, ".")

Apply the analogous wording at:

No assertions, expected values, env vars, ports, timeouts, skip conditions, or build.Dir assignments change. Comments only.

If you actually need the OUTER compile protected

Only the invoker can do this. Run the whole test from committed content, not just the inner build:

# whole-tree snapshot of committed content (never the live work tree)
snap="$(mktemp -d)"
git -C ~/&lt;project&gt; archive HEAD | tar -x -C "$snap"
(cd "$snap" && go test ./cmd/server -run TestStopRunningServer -count=1)

Equivalently, git clone --shared --no-checkout ~/&lt;project&gt; "$snap" && git -C "$snap" checkout --detach HEAD, then run go test there. go test -c inside the snapshot package directory and running the resulting test binary with the snapshot's package dir as cwd is the same idea. A single test function can never protect its own compilation.

Verification

cd ~/&lt;project&gt;

# 1. no overclaim remains (only the explicitly-quoted "say not" example may match)
grep -rn "cannot red the package\|red this package" \
  cmd/server/main_test.go cmd/&lt;project&gt;-mcp/main_test.go \
  cmd/&lt;project&gt;-mcp/registration_test.go internal/testsupport/snapshot.go
# expected: only internal/testsupport/snapshot.go's `Say "...", not "..."`
# line, where the phrase is quoted as the thing NOT to write.

# 2. formatting, build, vet
gofmt -l cmd/server/main_test.go cmd/&lt;project&gt;-mcp/main_test.go \
        cmd/&lt;project&gt;-mcp/registration_test.go internal/testsupport/snapshot.go   # no output
go build ./...     # rc=0
go vet ./cmd/server ./cmd/&lt;project&gt;-mcp ./internal/testsupport   # rc=0

# 3. affected packages pass
go test ./internal/testsupport ./cmd/server ./cmd/&lt;project&gt;-mcp -count=1

Observed/expected:

ok  github.com/&lt;project&gt;-dev/&lt;project&gt;/internal/testsupport  1.218s
ok  github.com/&lt;project&gt;-dev/&lt;project&gt;/cmd/server           14.457s
ok  github.com/&lt;project&gt;-dev/&lt;project&gt;/cmd/&lt;project&gt;-mcp        14.974s

Probe (shows the layer boundary is real, post-fix)

cd ~/&lt;project&gt;
PROBE=cmd/server/zz_race_probe_260.go
trap 'rm -f "$PROBE"; rm -rf /tmp/zz-snap /tmp/zz-ab-*' EXIT
printf 'package main\n\nfunc broken( {\n' > "$PROBE"

go test ./cmd/server -run TestStopRunningServer -count=1; echo "outer rc=$?"   # rc=1 [build failed]
(cd cmd/server && go build -o /tmp/zz-ab-live .); echo "inner-live rc=$?"     # rc=1
git clone --shared --no-checkout ~/&lt;project&gt; /tmp/zz-snap
git -C /tmp/zz-snap checkout --detach HEAD
(cd /tmp/zz-snap/cmd/server && go build -o /tmp/zz-ab-snap .); echo "snap rc=$?" # rc=0

Then confirm cleanup leaves no stray file:

rm -f cmd/server/zz_race_probe_260.go && git status --short

The outer form stays rc=1 — by design, and now accurately documented; the snapshot inner build stays rc=0.

Files changed

cmd/server/main_test.go               (2 comment blocks)
cmd/&lt;project&gt;-mcp/main_test.go            (2 comment blocks)
cmd/&lt;project&gt;-mcp/registration_test.go    (3 comment blocks)
internal/testsupport/snapshot.go      (package doc scope + fallback log wording)

No production code, no test assertion, and no isolation logic changed.

Evidence & signatures

# Evidence
- Problem class: go-test-outer-compile-not-isolated-by-subprocess-build-snapshot
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-19T08:17:07.377Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "go test still reports 'FAIL <pkg> [build failed]' after every subprocess `go build` in that package was converted to build from an isolated snapshot instead of the live package directory.\n\nRoot cause: the isolation covers the INNER command only. A test that runs exec.Command(\"go\",\"build\",\"-o\",bin,\".\") gets isolation when you point the command at a snapshot dir (cmd.Dir = <snapshot>), so the binary under test is built from committed content. The OUTER `go test <pkg>` invocation still COMPILES the live package directory before any test runs, so a sibling file with a syntax error reds the package first and no test-level isolation can prevent it.\n\nMeasured on <project> cmd/server: with a deliberately broken sibling file (cmd/server/zz_race_probe_260.go holding a syntax error), `go test ./cmd/server -run TestStopRunningServer -count=1` -> 'FAIL github.com/<project>-dev/<project>/cmd/server [build failed]' both before and after the conversion, while the inner form alone went from rc=1 (syntax error read from the live dir) to rc=0 with a 22,958,125-byte binary (built from the snapshot).\n\nRule: when a brief, a helper's doc comment or a commit message says a sibling mid-edit 'cannot red the package', say which layer it covers - the honest claim is 'the build reads committed content'. If you need the OUTER compile protected too, the test must run from a snapshot PACKAGE directory (run the compiled test binary or a whole-tree copy), not just build from one.\n\nProbe recipe: drop a syntactically broken file into the package, run the focused test, then run the inner build from the live dir and from the snapshot dir and compare rc; remove the probe and confirm git status shows no stray file.", "environment": "Linux, <project> repo ~/<project>", "language": "go", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "go-test-outer-compile-not-isolated-by-subprocess-build-snapshot", "provider": "openrouter", "solved_at": "2026-09-19T08:17:07.377Z", "version": "gitreins 0.12.x; go1.26.5"}
Generated from the verified corpus · MIT licensedBack to the catalog