go-json-extraction-llm-output

Answer

The Kobayashi-Maru parser is split into three layers, each fixing one failure mode:

1. extract.go — scan all fences, prefer plan-shaped JSON (fixes parsed-empty-plans): the old extractJSON returned after the first fenced block, so a reasoning model's analysis ```json preamble was parsed as the plan. The fix enumerates every fence in document order, parses each as JSON, and returns the first plan-shaped object (top-level orders/helm/engineering/tactical key). Analysis and generic blocks are skipped. If nothing is plan-shaped it falls back to the first valid JSON block (no regression for legacy generic outputs), else ErrNoPlan.

// PlanKeys are the object keys that distinguish a real plan from an analysis
// preamble. Reasoning models front-load a ```json block describing their
// chain of thought; the orders block appears later and carries these keys.
var PlanKeys = []string{"orders", "helm", "engineering", "tactical"}

func ExtractPlan(output string) (json.RawMessage, error) {
    var firstValid json.RawMessage
    for _, body := range fencedBlocks(output) {
        raw, err := parseBlock(body)
        if err != nil {
            continue // garbage fence: skip, don't abort
        }
        if firstValid == nil {
            firstValid = raw
        }
        if isPlanShaped(raw) {
            return raw, nil
        }
    }
    if firstValid != nil {
        return firstValid, nil // backward-compat fallback
    }
    return nil, ErrNoPlan
}

// fencedBlocks returns every ``` fence body in document order. The language
// tag is deliberately ignored: models tag analysis blocks "json" too.
func fencedBlocks(output string) []string {
    var blocks []string
    rest := output
    for {
        open := strings.Index(rest, "```")
        if open < 0 {
            return blocks
        }
        after := rest[open+3:]
        nl := strings.IndexByte(after, '\n')
        if nl < 0 {
            return blocks // unterminated fence opener
        }
        body := after[nl+1:]
        close := strings.Index(body, "```")
        if close < 0 {
            return blocks // unterminated fence
        }
        blocks = append(blocks, body[:close])
        rest = body[close+3:]
    }
}

func isPlanShaped(raw json.RawMessage) bool {
    var obj map[string]json.RawMessage
    if err := json.Unmarshal(raw, &obj); err != nil {
        return false // arrays/scalars are not plans
    }
    for _, key := range PlanKeys {
        if _, ok := obj[key]; ok {
            return true
        }
    }
    return false
}

2. openrouter.go — one HTTP request, no internal retries, empty-content surfaced cleanly: the adapter performs exactly one POST /chat/completions. A 200 with empty content returns &Message{Content: "", FinishReason: fr} (content is a *string so null and "" are equivalent). Non-200 → *APIError; network failures → returned raw. It never retries; that decision belongs to the retry policy, so API/network errors can never be double-billed.

if resp.StatusCode != http.StatusOK {
    return nil, &APIError{Status: resp.StatusCode, Body: openRouterErrorMessage(body, string(body))}
}
var parsed openRouterResponse
if err := json.Unmarshal(body, &parsed); err != nil { ... }
if len(parsed.Choices) == 0 {
    return &Message{Role: "assistant"}, nil // same failure mode as empty content
}
choice := parsed.Choices[0]
content := ""
if choice.Message.Content != nil {
    content = *choice.Message.Content
}
return &Message{Role: choice.Message.Role, Content: content, FinishReason: choice.FinishReason}, nil

3. retry.go — RetryableAdapter: retry only empty_content, exactly once (fixes the ~50% empty-plan rate): MaxAttempts defaults to 2 (send + one retry). The loop retries only on empty content or a *ParseError from an inner adapter. API errors, network errors, and context cancellation hit the default branch and surface immediately — never retried. On exhausted retries it returns a *ParseError that carries finish_reason, and Plan wraps any post-parse failure the same way, so callers can distinguish stop from length cutoffs without re-parsing.

func (r *RetryableAdapter) Complete(ctx context.Context, prompt string) (*Message, error) {
    max := r.MaxAttempts
    if max <= 0 {
        max = 2
    }
    var lastErr error
    for attempt := 1; attempt <= max; attempt++ {
        msg, err := r.Inner.Complete(ctx, prompt)
        switch {
        case err == nil && isEmpty(msg):
            lastErr = &ParseError{FinishReason: finishReason(msg), Err: ErrEmptyContent}
        case isRetryable(err): // ErrEmptyContent or *ParseError
            lastErr = err
        default:
            return msg, err // API, network, cancel: NEVER retried
        }
    }
    return nil, lastErr
}

func (r *RetryableAdapter) Plan(ctx context.Context, prompt string) (json.RawMessage, error) {
    msg, err := r.Complete(ctx, prompt)
    if err != nil {
        return nil, err
    }
    plan, perr := ExtractPlan(msg.Content)
    if perr != nil {
        return nil, &ParseError{FinishReason: msg.FinishReason, Err: perr} // finish_reason preserved
    }
    return plan, nil
}

Error taxonomy: *APIError (never retried), ErrEmptyContent + *ParseError{FinishReason, Err} (retryable / diagnostic), ErrNoPlan (extraction failure).