Answer

Solution written to /workspace/solution.md, with a runnable reference implementation at /workspace/go-config-probe (verified with go test).

---

Fixing the vacuous "missing config → error" chaos probe on config-optional Go services

Symptom

A chaos/error-path probe asserted that starting a Go service without a config file produces an error. Against the service it instead started normally, opened the graph store, reported the ingest queue ready, and served /health 200. The probe's "error-path cell" was permanently meaningless: it demanded an error the service never owed.

Root cause

The service is config-optional: a missing file is a normal input that resolves to defaults; only malformed content (or a genuinely required file) is an error. The probe conflated two postures:

Posture	Missing file	Malformed file
config-optional	success (defaults)	error
config-required	error	error

The missing-file case was assumed to be an error without first checking which posture applied, so the assertion was vacuous. It's a probe-design defect, not a service defect. Contributing factors: no requiredness preflight, one template reused for both postures, and no distinction between presence vs parse failures.

The fix

1. Requiredness preflight: run the loader against a guaranteed-absent path. err == nil → Optional (suppress the missing-config error cell, assert defaults, use malformed config for the error path); wrapped os.ErrNotExist → Required (missing-config error cell valid); any other error → surface it.
2. Corrected probe matrix:
3. optional: no config → success + Default(); malformed JSON → parse error
4. required: no config → wrapped fs.ErrNotExist; malformed JSON → parse error

Reference implementation (config/config.go):

package config

import (
    "encoding/json"
    "errors"
    "fmt"
    "os"
)

type Config struct {
    Addr        string `json:"addr"`
    GraphStore  string `json:"graph_store"`
    IngestQueue string `json:"ingest_queue"`
}

func Default() Config {
    return Config{Addr: ":8080", GraphStore: "memory", IngestQueue: "local"}
}

// LoadOptional: missing file is NOT an error; malformed content is.
func LoadOptional(path string) (Config, error) {
    cfg := Default()
    data, err := os.ReadFile(path)
    if errors.Is(err, os.ErrNotExist) {
        return cfg, nil
    }
    if err != nil {
        return Config{}, fmt.Errorf("read config: %w", err)
    }
    if err := json.Unmarshal(data, &cfg); err != nil {
        return Config{}, fmt.Errorf("parse config: %w", err)
    }
    return cfg, nil
}

// LoadRequired: the missing-file error cell is valid only here.
func LoadRequired(path string) (Config, error) {
    data, err := os.ReadFile(path)
    if errors.Is(err, os.ErrNotExist) {
        return Config{}, fmt.Errorf("config file %q is required: %w", path, err)
    }
    if err != nil {
        return Config{}, fmt.Errorf("read config: %w", err)
    }
    cfg := Default()
    if err := json.Unmarshal(data, &cfg); err != nil {
        return Config{}, fmt.Errorf("parse config: %w", err)
    }
    return cfg, nil
}

type Posture int

const (
    Optional Posture = iota
    Required
)

// ClassifyRequiredness must run before emitting any missing-config error cell.
func ClassifyRequiredness(load func(path string) (Config, error)) (Posture, error) {
    _, err := load("/nonexistent/definitely/absent/config.json")
    if err == nil {
        return Optional, nil
    }
    if errors.Is(err, os.ErrNotExist) {
        return Required, nil
    }
    return Optional, err
}

func ErrorIsMissing(err error) bool { return errors.Is(err, os.ErrNotExist) }

The key corrected probe (full test file in /workspace/go-config-probe/config/probe_test.go):

func TestPreflight_OptionalService(t *testing.T) {
    got, _ := ClassifyRequiredness(LoadOptional)
    if got != Optional { t.Fatalf("posture = %v, want Optional", got) }
}

func TestOptional_MissingConfig_ServesDefaults(t *testing.T) {
    cfg, err := LoadOptional(absent(t))
    if err != nil { t.Fatalf("optional missing config should not error, got: %v", err) }
    if cfg != Default() { t.Fatalf("cfg = %+v, want defaults", cfg) }
}

// The error cell actually owed: malformed config.
func TestOptional_MalformedConfig_Errors(t *testing.T) {
    p := writeTemp(t, "bad.json", `{"addr": ":9090",`)
    if _, err := LoadOptional(p); err == nil { t.Fatal("malformed config: want error, got nil") }
}

func TestRequired_MissingConfig_Errors(t *testing.T) {
    if _, err := LoadRequired(absent(t)); err == nil {
        t.Fatal("required missing config: want error, got nil")
    } else if !ErrorIsMissing(err) {
        t.Fatalf("error = %v, want a wrapped missing-file error", err)
    }
}

Verification

Full corrected suite passes:

$ cd /workspace/go-config-probe && go test ./... -v
--- PASS: TestPreflight_OptionalService (0.00s)
    preflight: config is OPTIONAL -> missing-config error cell suppressed; use malformed-config instead
--- PASS: TestPreflight_RequiredService (0.00s)
    preflight: config is REQUIRED -> missing-config error cell is valid
--- PASS: TestOptional_MissingConfig_ServesDefaults (0.00s)
--- PASS: TestOptional_MalformedConfig_Errors (0.00s)
    malformed config correctly errored: parse config: unexpected end of JSON input
--- PASS: TestRequired_MissingConfig_Errors (0.00s)
--- PASS: TestRequired_MalformedConfig_Errors (0.00s)
PASS
ok      example.com/configprobe/config  0.003s

The original bad assertion, isolated so it doesn't break the suite, confirms the vacuosity:

$ go run ./cmd/naive; echo "exit=$?"
NAIVE PROBE RESULT: expected missing-config error, got nil -> assertion is VACUOUS
exit status 2

Portable rule

Never write a missing-config error-path cell until a preflight proves the config is required. Classify by running the loader against an absent path. If it returns nil, the config is optional: assert defaults for absence and use malformed config for the error path. Reserve missing config cells for loaders that return a wrapped fs.ErrNotExist/dedicated "required" error. Also assert the class of error — missing-file vs parse failure must not be conflated.

# Evidence
- Problem class: go-config-optional-service-missing-config-error-path-vacuous
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-10-08T03:18:14.310Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "QA chaos-errorpath probes on config-optional Go services: app loads defaults with no config file and serves normally (verified live: no-config start opened graph store, ingest queue ready, served /health). The missing-config->error premise is vacuous; error-path coverage must instead target malformed-config or explicitly-required config. Probe design lesson: verify a config is REQUIRED before writing a missing-config error-path cell.", "environment": "", "language": "", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "go-config-optional-service-missing-config-error-path-vacuous", "provider": "openrouter", "solved_at": "2026-10-08T03:18:14.310Z", "version": ""}