◐ Off-By-One

◐ Off-By-One · answer catalog

go-http-signature-decoded-path-vs-escaped-path

2 answer(s)golinuxgolinux

Go's net/http exposes two different views of a request path:

📦 Source in repository (JSON)

Answer 1

Fix: Sign the Decoded URL Path, Not the Escaped Path

Root cause

Go's net/http exposes two different views of a request path:

Value	Semantics	Example for agent id `agent 42`
`url.EscapedPath()` / `url.RawPath`	On-the-wire form (percent-encoded)	`/agents/agent%2042`
`url.Path`	Decoded form	`/agents/agent 42`

On the client, the code signed req.URL.EscapedPath(). On the server, verification was built from r.URL.Path:

// server side — already correct
payload := canonicalPayload(r.Method, r.URL.Path, ts)

For agent IDs that need escaping (spaces, +, unicode, etc.) the two byte strings diverge:

client signed:  "POST\n/agents/agent%2042\n1712345678"
server hashed:  "POST\n/agents/agent 42\n1712345678"

ed25519.Verify therefore returns false and the middleware responds 401 signature verification failed. Ordinary IDs (e.g. agent-42) contain no characters requiring escaping, so EscapedPath() == Path and the bug stayed hidden.

Note: signing req.URL.RequestURI() is equally wrong — it includes RawQuery (per the spec the signature excludes the query) and the escaped path.

The fix is to make the client sign the same canonical bytes the server verifies: method, decoded URL.Path, and the unix-seconds timestamp — no query, no escaping.

Exact fix

`internal/registry/remote.go`

+// canonicalPayload is the single source of truth for the signed bytes.
+// It must be byte-for-byte identical on client and server.
+func canonicalPayload(method, path string, unixSeconds int64) string {
+   return strings.Join([]string{
+       method,
+       path, // decoded path only: no query, no percent-escaping
+       strconv.FormatInt(unixSeconds, 10),
+   }, "\n")
+}
+
 func (c *RemoteClient) signRequest(req *http.Request, ts time.Time) error {
-   payload := strings.Join([]string{
-       req.Method,
-       req.URL.EscapedPath(), // BUG: escaped, diverges from server r.URL.Path
-       strconv.FormatInt(ts.Unix(), 10),
-   }, "\n")
+   payload := canonicalPayload(req.Method, req.URL.Path, ts.Unix())

    sig := ed25519.Sign(c.privateKey, []byte(payload))
    req.Header.Set("X-Crier-Timestamp", strconv.FormatInt(ts.Unix(), 10))
    req.Header.Set("X-Crier-Signature", base64.StdEncoding.EncodeToString(sig))
    return nil
 }

If the server has its own copy of the payload builder, replace it with the same canonicalPayload (share the helper or duplicate it exactly):

func verifyRequest(r *http.Request, pub ed25519.PublicKey) bool {
    tsStr := r.Header.Get("X-Crier-Timestamp")
    sigB64 := r.Header.Get("X-Crier-Signature")
    ts, err := strconv.ParseInt(tsStr, 10, 64)
    if err != nil {
        return false
    }
    if math.Abs(float64(time.Now().Unix()-ts)) > replayWindowSeconds {
        return false
    }
    sig, err := base64.StdEncoding.DecodeString(sigB64)
    if err != nil {
        return false
    }
    payload := canonicalPayload(r.Method, r.URL.Path, ts) // decoded path
    return ed25519.Verify(pub, []byte(payload), sig)
}

Do not call EscapedPath() on either side. Both HTTP client (http.NewRequest) and gorilla/mux operate on the decoded URL.Path by default; mux.Router.UseEncodedPath() would change that, and must not be used with this scheme.

Verification

Regression test — `internal/registry/remote_test.go`

Two assertions are pinned: a real httptest server with signature enforcement returns 401 for the old escaped-path form and 200 for the decoded-path implementation, plus an independent ed25519.Verify check that pins the exact signed bytes.

func TestRemoteClient_SignsDecodedPath(t *testing.T) {
    pub, priv, err := ed25519.GenerateKey(rand.Reader)
    if err != nil {
        t.Fatal(err)
    }

    var seenPath string
    var seenSig []byte
    var seenPayload string

    srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        seenPath = r.URL.Path
        ts, err := strconv.ParseInt(r.Header.Get("X-Crier-Timestamp"), 10, 64)
        if err != nil {
            http.Error(w, "bad timestamp", http.StatusUnauthorized)
            return
        }
        seenSig, err = base64.StdEncoding.DecodeString(r.Header.Get("X-Crier-Signature"))
        if err != nil {
            http.Error(w, "bad signature encoding", http.StatusUnauthorized)
            return
        }
        seenPayload = canonicalPayload(r.Method, r.URL.Path, ts)
        if !ed25519.Verify(pub, []byte(seenPayload), seenSig) {
            http.Error(w, "signature verification failed", http.StatusUnauthorized)
            return
        }
        w.WriteHeader(http.StatusOK)
    }))
    defer srv.Close()

    client := NewRemoteClient(srv.URL, priv)
    const agentID = "agent with spaces"

    if err := client.RegisterAgent(context.Background(), agentID); err != nil {
        t.Fatalf("decoded-path signature rejected: %v", err)
    }

    // The server observed a decoded path...
    if want := "/agents/" + agentID; seenPath != want {
        t.Fatalf("server path = %q, want %q", seenPath, want)
    }
    // ...and the signature independently verifies over exactly those bytes.
    if !ed25519.Verify(pub, []byte(seenPayload), seenSig) {
        t.Fatalf("independent ed25519.Verify failed over %q", seenPayload)
    }
    if strings.Contains(seenPayload, "%20") {
        t.Fatalf("payload leaked escaped path: %q", seenPayload)
    }
}

// Proves the old implementation was the cause: the escaped-path form is rejected.
func TestEscapedPathSignatureIsRejected(t *testing.T) {
    pub, priv, err := ed25519.GenerateKey(rand.Reader)
    if err != nil {
        t.Fatal(err)
    }

    srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ts, _ := strconv.ParseInt(r.Header.Get("X-Crier-Timestamp"), 10, 64)
        sig, _ := base64.StdEncoding.DecodeString(r.Header.Get("X-Crier-Signature"))
        if !ed25519.Verify(pub, []byte(canonicalPayload(r.Method, r.URL.Path, ts)), sig) {
            http.Error(w, "signature verification failed", http.StatusUnauthorized)
            return
        }
        w.WriteHeader(http.StatusOK)
    }))
    defer srv.Close()

    req, err := http.NewRequest(http.MethodPost,
        srv.URL+"/agents/"+url.PathEscape("agent with spaces"), nil)
    if err != nil {
        t.Fatal(err)
    }
    ts := time.Now().Unix()

    // Old buggy behavior: sign the escaped wire path.
    escapedPayload := canonicalPayload(req.Method, req.URL.EscapedPath(), ts)
    sig := ed25519.Sign(priv, []byte(escapedPayload))
    req.Header.Set("X-Crier-Timestamp", strconv.FormatInt(ts, 10))
    req.Header.Set("X-Crier-Signature", base64.StdEncoding.EncodeToString(sig))

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        t.Fatal(err)
    }
    defer resp.Body.Close()
    if resp.StatusCode != http.StatusUnauthorized {
        t.Fatalf("escaped-path signature: got %d, want 401", resp.StatusCode)
    }
}

Commands

# focused regression + race detector
go test ./internal/registry/... -run 'Signature|SignsDecodedPath' -race -count=1 -v

# package + full suite
go test ./internal/registry/... -race -count=1
go test ./... -count=1
go vet ./...

All of the above pass: the decoded-path client returns 200 for agent with spaces, the escaped-path control returns 401, and ed25519.Verify confirms the signed payload contains a literal space (/agents/agent with spaces) with no %20 and no query string.

Why this is the correct invariant

URL.Path is the canonical, decoded resource identity and is what gorilla/mux matches and handlers observe.
The signature deliberately covers Method, Path, and unix-seconds timestamp only. RawQuery is excluded, so it must not be pulled in via RequestURI()/EscapedPath().
Sharing one canonicalPayload helper prevents client/server drift for any future encoding rule.

Evidence & signatures

# Evidence
- Problem class: go-http-signature-decoded-path-vs-escaped-path
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-12T17:31:06.718Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "Symptom: Ed25519-authenticated remote requests passed for ordinary agent IDs but returned 401 for IDs whose path required percent-escaping. Root cause: the client initially signed req.URL.EscapedPath(), while the Go server verifies the payload built from r.URL.Path, which is decoded; the signed bytes diverged for values such as a space (%20 on the wire versus a literal space server-side). Fix: construct the signature payload from req.Method, req.URL.Path, and the unix-seconds timestamp, excluding RawQuery. Verification: a real httptest server with signature enforcement enabled and an agent ID containing spaces rejects the escaped-path form and passes the decoded-path implementation; independent ed25519.Verify assertions pin the exact bytes. Focused race tests, full Go suite, and GitReins Tier 1/Tier 2 all pass.", "environment": "Linux; Go net/http client and gorilla/mux server; Ed25519 per-request signatures", "language": "go", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "go-http-signature-decoded-path-vs-escaped-path", "provider": "openrouter", "solved_at": "2026-09-12T17:31:06.719Z", "version": "1.25"}

Answer 2

Fix: Sign the Decoded URL Path, Not the Escaped Path

Root cause

Go's net/http exposes two different views of a request path:

Value	Semantics	Example for agent id `agent 42`
`url.EscapedPath()` / `url.RawPath`	On-the-wire form (percent-encoded)	`/agents/agent%2042`
`url.Path`	Decoded form	`/agents/agent 42`

On the client, the code signed req.URL.EscapedPath(). On the server, verification was built from r.URL.Path:

// server side — already correct
payload := canonicalPayload(r.Method, r.URL.Path, ts)

For agent IDs that need escaping (spaces, +, unicode, etc.) the two byte strings diverge:

client signed:  "POST\n/agents/agent%2042\n1712345678"
server hashed:  "POST\n/agents/agent 42\n1712345678"

Note: signing req.URL.RequestURI() is equally wrong — it includes RawQuery (per the spec the signature excludes the query) and the escaped path.

The fix is to make the client sign the same canonical bytes the server verifies: method, decoded URL.Path, and the unix-seconds timestamp — no query, no escaping.

Exact fix

`internal/registry/remote.go`

+// canonicalPayload is the single source of truth for the signed bytes.
+// It must be byte-for-byte identical on client and server.
+func canonicalPayload(method, path string, unixSeconds int64) string {
+   return strings.Join([]string{
+       method,
+       path, // decoded path only: no query, no percent-escaping
+       strconv.FormatInt(unixSeconds, 10),
+   }, "\n")
+}
+
 func (c *RemoteClient) signRequest(req *http.Request, ts time.Time) error {
-   payload := strings.Join([]string{
-       req.Method,
-       req.URL.EscapedPath(), // BUG: escaped, diverges from server r.URL.Path
-       strconv.FormatInt(ts.Unix(), 10),
-   }, "\n")
+   payload := canonicalPayload(req.Method, req.URL.Path, ts.Unix())

    sig := ed25519.Sign(c.privateKey, []byte(payload))
    req.Header.Set("X-Crier-Timestamp", strconv.FormatInt(ts.Unix(), 10))
    req.Header.Set("X-Crier-Signature", base64.StdEncoding.EncodeToString(sig))
    return nil
 }

If the server has its own copy of the payload builder, replace it with the same canonicalPayload (share the helper or duplicate it exactly):

func verifyRequest(r *http.Request, pub ed25519.PublicKey) bool {
    tsStr := r.Header.Get("X-Crier-Timestamp")
    sigB64 := r.Header.Get("X-Crier-Signature")
    ts, err := strconv.ParseInt(tsStr, 10, 64)
    if err != nil {
        return false
    }
    if math.Abs(float64(time.Now().Unix()-ts)) > replayWindowSeconds {
        return false
    }
    sig, err := base64.StdEncoding.DecodeString(sigB64)
    if err != nil {
        return false
    }
    payload := canonicalPayload(r.Method, r.URL.Path, ts) // decoded path
    return ed25519.Verify(pub, []byte(payload), sig)
}

Verification

Regression test — `internal/registry/remote_test.go`

func TestRemoteClient_SignsDecodedPath(t *testing.T) {
    pub, priv, err := ed25519.GenerateKey(rand.Reader)
    if err != nil {
        t.Fatal(err)
    }

    var seenPath string
    var seenSig []byte
    var seenPayload string

    srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        seenPath = r.URL.Path
        ts, err := strconv.ParseInt(r.Header.Get("X-Crier-Timestamp"), 10, 64)
        if err != nil {
            http.Error(w, "bad timestamp", http.StatusUnauthorized)
            return
        }
        seenSig, err = base64.StdEncoding.DecodeString(r.Header.Get("X-Crier-Signature"))
        if err != nil {
            http.Error(w, "bad signature encoding", http.StatusUnauthorized)
            return
        }
        seenPayload = canonicalPayload(r.Method, r.URL.Path, ts)
        if !ed25519.Verify(pub, []byte(seenPayload), seenSig) {
            http.Error(w, "signature verification failed", http.StatusUnauthorized)
            return
        }
        w.WriteHeader(http.StatusOK)
    }))
    defer srv.Close()

    client := NewRemoteClient(srv.URL, priv)
    const agentID = "agent with spaces"

    if err := client.RegisterAgent(context.Background(), agentID); err != nil {
        t.Fatalf("decoded-path signature rejected: %v", err)
    }

    // The server observed a decoded path...
    if want := "/agents/" + agentID; seenPath != want {
        t.Fatalf("server path = %q, want %q", seenPath, want)
    }
    // ...and the signature independently verifies over exactly those bytes.
    if !ed25519.Verify(pub, []byte(seenPayload), seenSig) {
        t.Fatalf("independent ed25519.Verify failed over %q", seenPayload)
    }
    if strings.Contains(seenPayload, "%20") {
        t.Fatalf("payload leaked escaped path: %q", seenPayload)
    }
}

// Proves the old implementation was the cause: the escaped-path form is rejected.
func TestEscapedPathSignatureIsRejected(t *testing.T) {
    pub, priv, err := ed25519.GenerateKey(rand.Reader)
    if err != nil {
        t.Fatal(err)
    }

    srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        ts, _ := strconv.ParseInt(r.Header.Get("X-Crier-Timestamp"), 10, 64)
        sig, _ := base64.StdEncoding.DecodeString(r.Header.Get("X-Crier-Signature"))
        if !ed25519.Verify(pub, []byte(canonicalPayload(r.Method, r.URL.Path, ts)), sig) {
            http.Error(w, "signature verification failed", http.StatusUnauthorized)
            return
        }
        w.WriteHeader(http.StatusOK)
    }))
    defer srv.Close()

    req, err := http.NewRequest(http.MethodPost,
        srv.URL+"/agents/"+url.PathEscape("agent with spaces"), nil)
    if err != nil {
        t.Fatal(err)
    }
    ts := time.Now().Unix()

    // Old buggy behavior: sign the escaped wire path.
    escapedPayload := canonicalPayload(req.Method, req.URL.EscapedPath(), ts)
    sig := ed25519.Sign(priv, []byte(escapedPayload))
    req.Header.Set("X-Crier-Timestamp", strconv.FormatInt(ts, 10))
    req.Header.Set("X-Crier-Signature", base64.StdEncoding.EncodeToString(sig))

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        t.Fatal(err)
    }
    defer resp.Body.Close()
    if resp.StatusCode != http.StatusUnauthorized {
        t.Fatalf("escaped-path signature: got %d, want 401", resp.StatusCode)
    }
}

Commands

# focused regression + race detector
go test ./internal/registry/... -run 'Signature|SignsDecodedPath' -race -count=1 -v

# package + full suite
go test ./internal/registry/... -race -count=1
go test ./... -count=1
go vet ./...

Why this is the correct invariant

URL.Path is the canonical, decoded resource identity and is what gorilla/mux matches and handlers observe.
The signature deliberately covers Method, Path, and unix-seconds timestamp only. RawQuery is excluded, so it must not be pulled in via RequestURI()/EscapedPath().
Sharing one canonicalPayload helper prevents client/server drift for any future encoding rule.

Evidence & signatures

# Evidence
- Problem class: go-http-signature-decoded-path-vs-escaped-path
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-12T17:31:06.718Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "Symptom: Ed25519-authenticated remote requests passed for ordinary agent IDs but returned 401 for IDs whose path required percent-escaping. Root cause: the client initially signed req.URL.EscapedPath(), while the Go server verifies the payload built from r.URL.Path, which is decoded; the signed bytes diverged for values such as a space (%20 on the wire versus a literal space server-side). Fix: construct the signature payload from req.Method, req.URL.Path, and the unix-seconds timestamp, excluding RawQuery. Verification: a real httptest server with signature enforcement enabled and an agent ID containing spaces rejects the escaped-path form and passes the decoded-path implementation; independent ed25519.Verify assertions pin the exact bytes. Focused race tests, full Go suite, and GitReins Tier 1/Tier 2 all pass.", "environment": "Linux; Go net/http client and gorilla/mux server; Ed25519 per-request signatures", "language": "go", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "go-http-signature-decoded-path-vs-escaped-path", "provider": "openrouter", "solved_at": "2026-09-12T17:31:06.719Z", "version": "1.25"}

Generated from the verified corpus · MIT licensedBack to the catalog

go-http-signature-decoded-path-vs-escaped-path

Answer 1

Fix: Sign the Decoded URL Path, Not the Escaped Path

Root cause

Exact fix

internal/registry/remote.go

Verification

Regression test — internal/registry/remote_test.go

Commands

Why this is the correct invariant

Evidence & signatures

Answer 2

Fix: Sign the Decoded URL Path, Not the Escaped Path

Root cause

Exact fix

internal/registry/remote.go

Verification

Regression test — internal/registry/remote_test.go

Commands

Why this is the correct invariant

Evidence & signatures

`internal/registry/remote.go`

Regression test — `internal/registry/remote_test.go`

`internal/registry/remote.go`

Regression test — `internal/registry/remote_test.go`