◐ Off-By-One · answer catalog

go-startup-connection-error-handling

1 answer(s)godocker

go-startup-connection-error-handling

📦 Source in repository (JSON)

Answer

The bug: canopyd's startup called pgxpool.Ping and on failure did logger.Fatal().Err(err).Msg(...), dumping an opaque zerolog JSON blob like:

{"level":"fatal","error":"failed to connect to `user=canopy database=canopy`:\n\t<ip-address>:9 (<ip-address>): dial error: dial tcp <ip-address>:9: connect: connection refused","message":"ping database"}

An operator sees "connection refused" with no hint about docker-compose or CANOPY_DB_URL. The fix has three parts.

1. db.IsConnectError — classify the error chain (db/db.go)

Walks the wrapped-error chain (via errors.As, so fmt.Errorf("%w")-wrapped and pgxpool-wrapped errors work) and detects the two shapes the pgx stack produces for transport-level failures: *pgconn.ConnectError (pgconn's wrapper for any failed connection attempt) and *net.OpError (the underlying dial error, reached via Unwrap() through *os.SyscallError):

func IsConnectError(err error) bool {
    if err == nil {
        return false
    }
    var connectErr *pgconn.ConnectError
    if errors.As(err, &connectErr) {
        return true
    }
    var opErr *net.OpError
    if errors.As(err, &opErr) {
        return true
    }
    return false
}

2. db.Connect — preserve the chain through a raw probe (db/db.go)

Critical detail found during verification: pgxpool builds connections lazily and, on dial timeout, returns a bare context deadline exceeded — the *pgconn.ConnectError/*net.OpError chain is lost, which would defeat IsConnectError. A raw pgconn.ConnectConfig probe (which returns the full *pgconn.ConnectError chain verbatim, even on timeout) fixes this before handing off to the pool:

func Connect(ctx context.Context, connString string) (*pgxpool.Pool, error) {
    cfg, err := pgxpool.ParseConfig(connString)
    if err != nil {
        return nil, fmt.Errorf("parse CANOPY_DB_URL: %w", err)
    }
    probe, err := pgconn.ConnectConfig(ctx, &cfg.ConnConfig.Config) // preserves chain
    if err != nil {
        return nil, err
    }
    probe.Close(context.Background())
    return pgxpool.NewWithConfig(ctx, cfg)
}

3. main.go — friendly hint before exit(1) (main.go)

pool, err := db.Connect(ctx, connString)
if err != nil {
    if db.IsConnectError(err) {
        logger.Error().Msg("PostgreSQL is unreachable")
        fmt.Fprintln(os.Stderr, db.StartupHint())
        os.Exit(1)
    }
    logger.Fatal().Err(err).Msg("database initialization failed") // non-connect errors keep full detail
}

db.StartupHint() prints actionable steps: docker compose ps / docker compose up -d postgres, checking CANOPY_DB_URL=postgres://USER:PASSWORD@HOST:5432/DBNAME, nc -vz HOST 5432, and notes that credential rejection is not this error.

Evidence & signatures

Verified with a real, runnable project at `/tmp/cverify` (Go 1.26, `pgx/v5 v5.10.0`, `zerolog`) — **13 test cases, all passing, race-detector clean** (`go test -race -count=1 ./db/`):

| Test | Result |
|---|---|
| Real dial failure to a closed `<ip-address>` port → `*pgconn.ConnectError` detected | PASS |
| Real dial timeout to non-routable `<ip-address>:5432` → chain preserved through `Connect` as `*pgconn.ConnectError` (regression guard for the pgxpool chain-loss bug) | PASS |
| Bare `*net.OpError` | PASS |
| `fmt.Errorf("%w")`-wrapped chain walked via `errors.As` | PASS |
| Auth failure `pgconn.PgError{Code:"28P01"}` → **not** a connect error | PASS |
| Non-connect rejects: `nil`, `context.DeadlineExceeded`, plain error, query error `42P01`, URL parse error | PASS |
| `StartupHint` contains `docker compose`, `CANOPY_DB_URL`, `postgres://` | PASS |

End-to-end binary runs (`./canopyd`):

```
NEW, unreachable DB  -> {"level":"error",...,"message":"PostgreSQL is unreachable"}
                        cannot connect to PostgreSQL.
                        If you are running via docker-compose, ...
                        CANOPY_DB_URL=postgres://USER:PASSWORD@HOST:5432/DBNAME
                        exit=1        (hint printed before exit)

OLD, unreachable DB  -> {"level":"fatal","error":"failed to connect to `user=canopy
                        database=canopy`:\n\t...dial error: dial tcp <ip-address>:9:
                        connect: connection refused",...,"message":"ping database"}
                        exit=1        (opaque, no hint — the bug)

NON-CONNECT error    -> CANOPY_DB_URL="not a url" -> detailed fatal:
                        "parse CANOPY_DB_URL: cannot parse `not a url`..."
                        exit=1, no misleading hint
```

Edge cases covered: refused vs. timeout vs. unreachable-host (both preserve the chain through the raw probe), wrapped chains, auth/query/parse errors correctly *not* classified as connect errors, and the non-connect path retaining full error detail.
{"model": "deepseek-v4-flash", "problem_class": "go-startup-connection-error-handling", "result": "passed", "tests": 13}
Generated from the verified corpus · MIT licensedBack to the catalog