go-idle-tick-cooldown-reversion

Answer

Root cause. The Go project exists only in daemon runtime state (scheduler-created, no fleet.toml entry). On daemon restart the project re-materializes from defaults, so CooldownS falls back 43200→900. The tick loop only checked its own in-memory copy ("no drift") — it never did a GET against the daemon, so the revert went unnoticed until too late.

Fix (two parts).

1. Operational fix — re-apply via API with exact PascalCase key, then GET-verify. Go encoding/json matches struct tags case-insensitively on decode, but many daemons decode into map[string]json.RawMessage or a strict decoder where {"cooldownS":…} is silently ignored/400'd. Use the exact tag CooldownS and never trust the PUT echo or tick-local state — always follow with a separate GET and assert the value.

package fleetctl

import (
    "bytes"
    "encoding/json"
    "fmt"
    "net/http"
)

const DaemonBase = "http://<ip-address>:8080" // daemon REST base
const TargetCooldownS = 43200

type project struct {
    Name      string `json:"Name"`
    CooldownS int    `json:"CooldownS"`
}

// ApplyCooldown PUTs the PascalCase field and then GET-verifies persistence.
// Returns error if the daemon's persisted value drifts from want.
func ApplyCooldown(name string, want int) error {
    // 1) PUT with the EXACT PascalCase json tag the daemon struct uses.
    body, _ := json.Marshal(struct {
        CooldownS int `json:"CooldownS"`
    }{CooldownS: want})

    req, _ := http.NewRequest(http.MethodPut,
        DaemonBase+"/api/v1/projects/"+name, bytes.NewReader(body))
    req.Header.Set("Content-Type", "application/json")
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return fmt.Errorf("PUT %s: %w", name, err)
    }
    resp.Body.Close()
    if resp.StatusCode != http.StatusOK {
        return fmt.Errorf("PUT %s: %s", name, resp.Status)
    }

    // 2) SEPARATE GET verify — never trust the PUT echo or tick-local state.
    greq, _ := http.NewRequest(http.MethodGet,
        DaemonBase+"/api/v1/projects/"+name, nil)
    gresp, err := http.DefaultClient.Do(greq)
    if err != nil {
        return fmt.Errorf("GET %s: %w", name, err)
    }
    defer gresp.Body.Close()

    var got project
    if err := json.NewDecoder(gresp.Body).Decode(&got); err != nil {
        return fmt.Errorf("GET %s: decode: %w", name, err)
    }
    if got.CooldownS != want {
        return fmt.Errorf("drift after verify: want CooldownS=%d got %d", want, got.CooldownS)
    }
    return nil
}

Usage from the scheduler tick:

if err := fleetctl.ApplyCooldown("idle-worker", fleetctl.TargetCooldownS); err != nil {
    // drift detected — alert, do not claim "no drift"
    log.Fatalf("cooldown not persisted: %v", err)
}

1. Permanent fix — add a fleet.toml entry. API-only projects are volatile; only fleet.toml sources are re-applied at boot, so the cooldown survives restarts without a re-PUT.

# fleet.toml
[[projects]]
name = "idle-worker"
cooldown_s = 43200   # or CooldownS = 43200 per daemon schema

After adding it, restart the daemon and confirm via GET /api/v1/projects/idle-worker that it returns {"CooldownS":43200} without any PUT.