Repo: Hermes-DAGger/<project> · Fix commit: b45af44 · Pre-fix binary: v0.1.0 (build 33d789a)
I diagnosed the issue against the live pre-fix binary (Hermes DAGger v0.1.0 build 33d789a) and wrote the full solution to /workspace/solution.md. I also verified the core helper logic compiles and passes its shape cases.
Key findings confirmed live:
- POST /hooks/search_and_summarize?query=golang → 500 execution_failed, and runs.dag_json contains ...);\nsearch_and_summarize (the raw identifier splice).
- /api/v1/status/<id> only returns {run_id,status}, and webhook_deliveries.error_code is bounded to execution_failed.
- dagger templates run search_and_summarize resolves and executes the real body (reaches tool()), proving the registry path is correct and the webhook seam just never calls it.
- Unregistered does_not_exist pre-fix also creates a delivery row and 500s.
The complete markdown solution:
500 execution_failed triage and fixRepo: Hermes-DAGger/<project> · Fix commit: b45af44 · Pre-fix binary: v0.1.0 (build 33d789a)
Symptom: a correctly signed POST /hooks/<workflow> to a scratch dagger serve --tier 3 answers HTTP 500 {"status":"failed","error_code":"execution_failed"}.
There are two unrelated causes. Only the first is a defect in the workflow seam.
The webhook builds executed source by concatenating the raw URL workflow onto a params preamble (cmd/dagger/main.go:2938, in handleWebhook):
paramsJSON, err := json.Marshal(params)
code := "Object.assign(params, " + string(paramsJSON) + ");\n" + workflow
Nothing resolves workflow, so search_and_summarize is emitted as a bare JS identifier → qjs: ReferenceError: search_and_summarize is not defined. dagger templates run already has resolveTemplate (prefers dag.LoadTemplate, falls back to highest name@X.Y.Z in the embedded FS, CWD-independent); the webhook never calls it.
Live proof from runs.dag_json:
[{"code":"Object.assign(params, {...\"query\":\"query=golang\"...});\nsearch_and_summarize"}]
DAGGER_TOOL_MODEL unset)With DAGGER_TOOL_MODEL unset, the first tool() inside a successfully resolved body fails:
tool error: DAGGER_TOOL_MODEL: explicit provider/model route required: ...
The bridge has no PAYG fallback by design. Set DAGGER_TOOL_MODEL=provider/model (e.g. deepseek/deepseek-v4-flash). Reaching tool() proves resolution worked.
SELECT dag_json FROM runs WHERE run_id='<id>' — bare identifier tail ⇒ Cause A; full template body ⇒ resolution worked./api/v1/execute, read Output: ReferenceError ⇒ A; DAGGER_TOOL_MODEL ⇒ B. (This route needs corsa/tsgo via DAGGER_CORSA_BIN; the webhook uses qjs directly.)DAGGER_API_TOKEN must be ≥ 32 bytes.X-Dagger-Signature = lowercase HMAC-SHA256 hex of the raw body keyed by HERMES_WEBHOOK_SECRET; X-Dagger-Idempotency required.params.query is the raw query string (?query=golang → "query=golang"); params.body is raw.cmd/dagger/workflow_resolve.gopackage main
import (
"fmt"
"strings"
)
func isTemplateReference(s string) bool {
if s == "" {
return false
}
name := s
if at := strings.LastIndexByte(s, '@'); at >= 0 {
name = s[:at]
version := s[at+1:]
if name == "" || version == "" {
return false
}
for i := 0; i < len(version); i++ {
c := version[i]
if c != '.' && (c < '0' || c > '9') {
return false
}
}
}
for i := 0; i < len(name); i++ {
c := name[i]
switch {
case c == '_':
case c >= 'a' && c <= 'z':
case c >= 'A' && c <= 'Z':
case i > 0 && c >= '0' && c <= '9':
default:
return false
}
}
return true
}
func resolveWorkflowSource(workflow string) (string, error) {
ref := strings.TrimSpace(workflow)
if !isTemplateReference(ref) {
return workflow, nil // byte-identical for inline code/specs
}
name, version := parseTemplateRef(ref)
src, err := resolveTemplate(name, version)
if err != nil {
return "", err
}
if strings.TrimSpace(src) == "" {
return "", fmt.Errorf("template %q resolved to empty source", ref)
}
return src, nil
}
main.go — resolve before reservationInsert after idempotency/body checks but before ReserveWebhookDelivery (~line 2876):
resolvedWorkflow, err := resolveWorkflowSource(workflow)
if err != nil {
writeWebhookError(w, r, http.StatusBadRequest, "unknown_template", err.Error())
return
}
Then:
- code := "Object.assign(params, " + string(paramsJSON) + ");\n" + workflow
+ code := "Object.assign(params, " + string(paramsJSON) + ");\n" + resolvedWorkflow
workflow_resolve_test.go: table-driven TestIsTemplateReference (accepts search_and_summarize, search_and_summarize@1.0.0; rejects foo.bar, foo/bar, 1template, @, @1.x, JS/TS, URLs), TestResolveWorkflowSourceReturnsCodeByteIdentical, and TestResolveWorkflowSourceResolvesRegisteredTemplate (asserts body contains web_search, not the bare id).
webhook_workflow_resolve_test.go: negative-control serve-surface test with a recording executor. Post-fix: 202 + code contains the template body; unregistered → 400 with no webhook_deliveries row/rate slot/run. Pre-fix worktree at 33d789a must FAIL (bare identifier reaches ExecuteWithID).
README.md quickstart and docs/API-REFERENCE.md “workflow identifier”: state the segment is a registered name/name@X.Y.Z resolved through the embedded registry, inline code is passed through, and 400 unknown_template is returned pre-reservation.
gofmt -l . && go build ./... && go vet ./...
golangci-lint run ./... # 0 issues
go test -race -count=1 ./... # green
Negative control: git worktree add /tmp/dagger-prefix 33d789a, copy the two tests in, run go test -run 'ResolveWorkflow|Webhook' ./cmd/dagger/ -count=1 → must FAIL.
Live A/B on two ports, shared token/secret, DAGGER_TOOL_MODEL set:
- Pre-fix :19001 → 500 execution_failed
- Fixed :19002 → 202 completed
Observed pre-fix: {"run_id":"run-b61c...","status":"failed","error_code":"execution_failed"} and dag_json ending in \nsearch_and_summarize; webhook_deliveries row search_and_summarize|failed|execution_failed. Post-fix unregistered ref must yield 400 with no row.
Full document (with complete test code and commands) is at /workspace/solution.md.
# Evidence - Problem class: dagger-webhook-500-execution-failed-triage - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-16T09:29:11.183Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Symptom: a correctly signed POST /hooks/<workflow> to a scratch dagger serve answers HTTP 500 with {\"status\":\"failed\",\"error_code\":\"execution_failed\"}. There are TWO unrelated causes and they must be told apart before changing any code, because only the first is a bug in the workflow seam.\n\nCause A (real defect, DF-HERMES-DAGGER-15): the schedule/URL workflow string was spliced straight into the executed source. A REGISTERED TEMPLATE NAME therefore ran as JavaScript and died with 'qjs: eval failed: ReferenceError: search_and_summarize is not defined'. The README quickstart and docs/API-REFERENCE 'workflow identifier' wording both steer users into this. Fix direction: resolve a template-shaped reference (a whole trimmed string matching a bare identifier with an optional @version suffix, with dot deliberately excluded from the name class) through the SAME registry 'dagger templates run' uses (resolveTemplate, which prefers dag.LoadTemplate and falls back to the highest name@X.Y.Z through the embedded FS, so it works from any CWD) and execute the returned source; anything else must be returned byte-identical.\n\nCause B (NOT a bug in that seam): with DAGGER_TOOL_MODEL unset, the first tool() call inside the resolved template body fails with 'tool error: DAGGER_TOOL_MODEL: explicit provider/model route required: set both provider and model, or use model \"provider/model\"'. The bridge deliberately has no fallback so a launcher cannot fall through to a PAYG default; set the env var (value form provider/model, e.g. deepseek/deepseek-v4-flash) on the serve process. If a template body reaches tool(), that is itself proof the resolution worked, so read the error text before concluding the resolver is wrong.\n\nTriage recipe that separates them: POST the exact executed code to /api/v1/execute and read the returned Output field. A ReferenceError names the identifier; a DAGGER_TOOL_MODEL error names the route. The status endpoint only returns run_id and status, and the webhook_deliveries row stores just the bounded error_code execution_failed, so neither surfaces the cause; the executed code itself is recoverable from the run's dag_json (a JSON list whose single node carries the full 'code' string), and its presence there shows whether the template body was substituted.\n\nOther gotchas found while doing this live: DAGGER_API_TOKEN must be at least 32 bytes or the serve exits with 'REST API token violates the 32-byte minimum length'; the webhook fills params.query with the RAW URL query string (so ?query=golang yields params.query equal to the literal 'query=golang'), params.body with the raw request body, so a template reading a structured field needs the body parsed or the CLI params path; the pre-fix and post-fix behaviour can be compared on two ports with two binaries and one shared token/secret pair; the webhook refuses an unregistered template-shaped reference with HTTP 400 before reservation, so no delivery row, no rate slot and no run are created, and rate-limit tests that used a bare-identifier path segment must move to a code workflow.\n\nVerification shape: negative control is a git worktree at the pre-fix commit with the new serve-surface test copied in, where the test must FAIL (the bare identifier is what reaches ExecuteWithID); live A/B is the same signed request against the pre-fix binary (500 execution_failed) and the fixed build (202 completed). Gates: gofmt -l clean, go build, go vet, golangci-lint run ./... 0 issues, go test -race -count=1 ./... green.", "environment": "Go 1.23+ repo <project>; scratch dagger serve --addr <ip-address>:PORT --db /tmp/x.db --tier 3 with HERMES_WEBHOOK_SECRET, DAGGER_API_TOKEN (32+ bytes) and DAGGER_TOOL_MODEL set; Hermes gateway on <ip-address>:8642 for tool()/llm()", "language": "go", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "dagger-webhook-500-execution-failed-triage", "provider": "openrouter", "solved_at": "2026-09-16T09:29:11.183Z", "version": ""}I diagnosed the issue against the live pre-fix binary (Hermes DAGger v0.1.0 build 33d789a) and wrote the full solution to /workspace/solution.md. I also verified the core helper logic compiles and passes its shape cases.
Key findings confirmed live:
- POST /hooks/search_and_summarize?query=golang → 500 execution_failed, and runs.dag_json contains ...);\nsearch_and_summarize (the raw identifier splice).
- /api/v1/status/<id> only returns {run_id,status}, and webhook_deliveries.error_code is bounded to execution_failed.
- dagger templates run search_and_summarize resolves and executes the real body (reaches tool()), proving the registry path is correct and the webhook seam just never calls it.
- Unregistered does_not_exist pre-fix also creates a delivery row and 500s.
The complete markdown solution:
500 execution_failed triage and fixRepo: Hermes-DAGger/<project> · Fix commit: b45af44 · Pre-fix binary: v0.1.0 (build 33d789a)
Symptom: a correctly signed POST /hooks/<workflow> to a scratch dagger serve --tier 3 answers HTTP 500 {"status":"failed","error_code":"execution_failed"}.
There are two unrelated causes. Only the first is a defect in the workflow seam.
The webhook builds executed source by concatenating the raw URL workflow onto a params preamble (cmd/dagger/main.go:2938, in handleWebhook):
paramsJSON, err := json.Marshal(params)
code := "Object.assign(params, " + string(paramsJSON) + ");\n" + workflow
Nothing resolves workflow, so search_and_summarize is emitted as a bare JS identifier → qjs: ReferenceError: search_and_summarize is not defined. dagger templates run already has resolveTemplate (prefers dag.LoadTemplate, falls back to highest name@X.Y.Z in the embedded FS, CWD-independent); the webhook never calls it.
Live proof from runs.dag_json:
[{"code":"Object.assign(params, {...\"query\":\"query=golang\"...});\nsearch_and_summarize"}]
DAGGER_TOOL_MODEL unset)With DAGGER_TOOL_MODEL unset, the first tool() inside a successfully resolved body fails:
tool error: DAGGER_TOOL_MODEL: explicit provider/model route required: ...
The bridge has no PAYG fallback by design. Set DAGGER_TOOL_MODEL=provider/model (e.g. deepseek/deepseek-v4-flash). Reaching tool() proves resolution worked.
SELECT dag_json FROM runs WHERE run_id='<id>' — bare identifier tail ⇒ Cause A; full template body ⇒ resolution worked./api/v1/execute, read Output: ReferenceError ⇒ A; DAGGER_TOOL_MODEL ⇒ B. (This route needs corsa/tsgo via DAGGER_CORSA_BIN; the webhook uses qjs directly.)DAGGER_API_TOKEN must be ≥ 32 bytes.X-Dagger-Signature = lowercase HMAC-SHA256 hex of the raw body keyed by HERMES_WEBHOOK_SECRET; X-Dagger-Idempotency required.params.query is the raw query string (?query=golang → "query=golang"); params.body is raw.cmd/dagger/workflow_resolve.gopackage main
import (
"fmt"
"strings"
)
func isTemplateReference(s string) bool {
if s == "" {
return false
}
name := s
if at := strings.LastIndexByte(s, '@'); at >= 0 {
name = s[:at]
version := s[at+1:]
if name == "" || version == "" {
return false
}
for i := 0; i < len(version); i++ {
c := version[i]
if c != '.' && (c < '0' || c > '9') {
return false
}
}
}
for i := 0; i < len(name); i++ {
c := name[i]
switch {
case c == '_':
case c >= 'a' && c <= 'z':
case c >= 'A' && c <= 'Z':
case i > 0 && c >= '0' && c <= '9':
default:
return false
}
}
return true
}
func resolveWorkflowSource(workflow string) (string, error) {
ref := strings.TrimSpace(workflow)
if !isTemplateReference(ref) {
return workflow, nil // byte-identical for inline code/specs
}
name, version := parseTemplateRef(ref)
src, err := resolveTemplate(name, version)
if err != nil {
return "", err
}
if strings.TrimSpace(src) == "" {
return "", fmt.Errorf("template %q resolved to empty source", ref)
}
return src, nil
}
main.go — resolve before reservationInsert after idempotency/body checks but before ReserveWebhookDelivery (~line 2876):
resolvedWorkflow, err := resolveWorkflowSource(workflow)
if err != nil {
writeWebhookError(w, r, http.StatusBadRequest, "unknown_template", err.Error())
return
}
Then:
- code := "Object.assign(params, " + string(paramsJSON) + ");\n" + workflow
+ code := "Object.assign(params, " + string(paramsJSON) + ");\n" + resolvedWorkflow
workflow_resolve_test.go: table-driven TestIsTemplateReference (accepts search_and_summarize, search_and_summarize@1.0.0; rejects foo.bar, foo/bar, 1template, @, @1.x, JS/TS, URLs), TestResolveWorkflowSourceReturnsCodeByteIdentical, and TestResolveWorkflowSourceResolvesRegisteredTemplate (asserts body contains web_search, not the bare id).
webhook_workflow_resolve_test.go: negative-control serve-surface test with a recording executor. Post-fix: 202 + code contains the template body; unregistered → 400 with no webhook_deliveries row/rate slot/run. Pre-fix worktree at 33d789a must FAIL (bare identifier reaches ExecuteWithID).
README.md quickstart and docs/API-REFERENCE.md “workflow identifier”: state the segment is a registered name/name@X.Y.Z resolved through the embedded registry, inline code is passed through, and 400 unknown_template is returned pre-reservation.
gofmt -l . && go build ./... && go vet ./...
golangci-lint run ./... # 0 issues
go test -race -count=1 ./... # green
Negative control: git worktree add /tmp/dagger-prefix 33d789a, copy the two tests in, run go test -run 'ResolveWorkflow|Webhook' ./cmd/dagger/ -count=1 → must FAIL.
Live A/B on two ports, shared token/secret, DAGGER_TOOL_MODEL set:
- Pre-fix :19001 → 500 execution_failed
- Fixed :19002 → 202 completed
Observed pre-fix: {"run_id":"run-b61c...","status":"failed","error_code":"execution_failed"} and dag_json ending in \nsearch_and_summarize; webhook_deliveries row search_and_summarize|failed|execution_failed. Post-fix unregistered ref must yield 400 with no row.
Full document (with complete test code and commands) is at /workspace/solution.md.
# Evidence - Problem class: dagger-webhook-500-execution-failed-triage - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-09-16T09:29:11.183Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Symptom: a correctly signed POST /hooks/<workflow> to a scratch dagger serve answers HTTP 500 with {\"status\":\"failed\",\"error_code\":\"execution_failed\"}. There are TWO unrelated causes and they must be told apart before changing any code, because only the first is a bug in the workflow seam.\n\nCause A (real defect, DF-HERMES-DAGGER-15): the schedule/URL workflow string was spliced straight into the executed source. A REGISTERED TEMPLATE NAME therefore ran as JavaScript and died with 'qjs: eval failed: ReferenceError: search_and_summarize is not defined'. The README quickstart and docs/API-REFERENCE 'workflow identifier' wording both steer users into this. Fix direction: resolve a template-shaped reference (a whole trimmed string matching a bare identifier with an optional @version suffix, with dot deliberately excluded from the name class) through the SAME registry 'dagger templates run' uses (resolveTemplate, which prefers dag.LoadTemplate and falls back to the highest name@X.Y.Z through the embedded FS, so it works from any CWD) and execute the returned source; anything else must be returned byte-identical.\n\nCause B (NOT a bug in that seam): with DAGGER_TOOL_MODEL unset, the first tool() call inside the resolved template body fails with 'tool error: DAGGER_TOOL_MODEL: explicit provider/model route required: set both provider and model, or use model \"provider/model\"'. The bridge deliberately has no fallback so a launcher cannot fall through to a PAYG default; set the env var (value form provider/model, e.g. deepseek/deepseek-v4-flash) on the serve process. If a template body reaches tool(), that is itself proof the resolution worked, so read the error text before concluding the resolver is wrong.\n\nTriage recipe that separates them: POST the exact executed code to /api/v1/execute and read the returned Output field. A ReferenceError names the identifier; a DAGGER_TOOL_MODEL error names the route. The status endpoint only returns run_id and status, and the webhook_deliveries row stores just the bounded error_code execution_failed, so neither surfaces the cause; the executed code itself is recoverable from the run's dag_json (a JSON list whose single node carries the full 'code' string), and its presence there shows whether the template body was substituted.\n\nOther gotchas found while doing this live: DAGGER_API_TOKEN must be at least 32 bytes or the serve exits with 'REST API token violates the 32-byte minimum length'; the webhook fills params.query with the RAW URL query string (so ?query=golang yields params.query equal to the literal 'query=golang'), params.body with the raw request body, so a template reading a structured field needs the body parsed or the CLI params path; the pre-fix and post-fix behaviour can be compared on two ports with two binaries and one shared token/secret pair; the webhook refuses an unregistered template-shaped reference with HTTP 400 before reservation, so no delivery row, no rate slot and no run are created, and rate-limit tests that used a bare-identifier path segment must move to a code workflow.\n\nVerification shape: negative control is a git worktree at the pre-fix commit with the new serve-surface test copied in, where the test must FAIL (the bare identifier is what reaches ExecuteWithID); live A/B is the same signed request against the pre-fix binary (500 execution_failed) and the fixed build (202 completed). Gates: gofmt -l clean, go build, go vet, golangci-lint run ./... 0 issues, go test -race -count=1 ./... green.", "environment": "Go 1.23+ repo <project>; scratch dagger serve --addr <ip-address>:PORT --db /tmp/x.db --tier 3 with HERMES_WEBHOOK_SECRET, DAGGER_API_TOKEN (32+ bytes) and DAGGER_TOOL_MODEL set; Hermes gateway on <ip-address>:8642 for tool()/llm()", "language": "go", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "dagger-webhook-500-execution-failed-triage", "provider": "openrouter", "solved_at": "2026-09-16T09:29:11.183Z", "version": ""}