A shell supervisor that moves short-lived workers through delegated cgroup v2 subtrees is unsafe if it uses numeric PIDs for control. The fix is a two-part design:
I built and verified a complete solution in ~/solution/ (pidfdctl.c, supervise.sh, two test harnesses, and SOLUTION.md). Here is the deliverable.
A shell supervisor that moves short-lived workers through delegated cgroup v2 subtrees is unsafe if it uses numeric PIDs for control. The fix is a two-part design:
pidfdctl — a small C process engine (started once as a bash coprocess) that forks each worker, joins it to the target cgroup before exec() via a readiness fence, opens a pidfd immediately (so the PID cannot be recycled), signals only through pidfd_send_signal(2), reaps through waitid(P_PIDFD, …) to obtain the raw wait status, and SIGKILLs every tracked worker if the supervisor dies (stdin EOF).supervise.sh — the bash orchestration: launch → freeze (verify from cgroup.events) → migrate (with thaw+retry fallback) → thaw → cross-check cgroup membership → reconcile exact status. It never signals or reaps by numeric PID alone.The result: no signal can hit a reused PID, no concurrent exit can be misattributed (even when reaped out of order), 128+N is disambiguated from a signal-kill, and a partial freeze/migration failure degrades to a reported, reconciled state instead of an untracked worker.
The kernel recycles PIDs as soon as a child is reaped. The classic bug is:
kill -0 "$pid" && kill -TERM "$pid" # TOCTOU: pid may be reused between the two
Even without kill -0, holding only $pid across a wait/freeze window means any later kill "$pid" may target whatever new process inherited the number. The only race-free handle is a pidfd: open it while the child is still alive and unreaped, then pidfd_send_signal() the fd. It either hits the exact original process or fails with ESRCH; it can never hit a recycled PID. Crucially, the pidfd must be opened before the child is reaped, because a reaped PID may be reused instantly. pidfdctl forks, opens the pidfd, and only hands back an id; reaping happens later, on demand.
wait conflates 128+N with signal N, and wait -n reaps arbitrary childrenBash's wait "$pid" returns 143 both for a process that exited with code 143 and for one killed by SIGTERM. It also does not expose CLD_DUMPED vs CLD_KILLED. And wait -n reaps whichever child exits first, so statuses lose their attribution when several workers exit concurrently. The fix is waitid(2) with P_PIDFD, which returns the raw siginfo_t: si_code = CLD_EXITED / CLD_KILLED / CLD_DUMPED, si_status = exit code or signal number. Because it keys on the pidfd, concurrent exits stay attributed to the correct worker regardless of reap order.
cgroup.events is the authorityWriting 1 to cgroup.freeze returns immediately. The subtree is not actually frozen until cgroup.events reports frozen 1:
populated 0
frozen 1
A supervisor that migrates or inspects cgroup.procs right after the write is racing the freezer. The safe pattern is: write freeze, then poll cgroup.events until frozen 1 or the cgroup is empty (populated 0, in which case there is nothing left to freeze). cgroup.events is a seq_file; a single read() returns a consistent snapshot, so reading the whole file and parsing both lines is safe.
$$ is the wrong PIDA common wrapper does:
( echo $$ > "$cg/cgroup.procs"; exec "$@" ) & # WRONG
In bash, $$ inside a subshell is the parent shell's PID, not the subshell's. Writing it migrates the supervisor, not the worker. The correct variable is $BASHPID — but even then there is a window between fork and the write during which the child is still in the parent cgroup, and no fence proving the move completed. If the supervisor dies in that window the worker is untracked. The fix is to do the join in the forked child before exec (in C) and gate the spawn on a readiness byte: the engine only returns OK <id> after the child confirms it is in the cgroup. If the join fails, the child exits 125 without ever running the payload.
freeze, migrate, and reap can each fail independently: cgroup.freeze may be absent/unwritable; migrating out of a frozen cgroup may return EBUSY/EAGAIN; the worker may already have exited before migration (ESRCH is success, not failure); a foreign process may be present; and cgroup.events populated 0 is not termination status. The supervisor must reconcile pidfd liveness, freeze/events, and the reaped status — and classify, never assume.
pidfd POLL |
waitid(P_PIDFD) result |
reported state |
|---|---|---|
RUNNING |
— | RUNNING |
EXITED |
CLD_EXITED (1), status e |
EXITED exit status=e |
EXITED |
CLD_KILLED (2), status s |
SIGNALED killed by signal=s |
EXITED |
CLD_DUMPED (3), status s |
SIGNALED dumped, signal=s |
EXITED |
waitid fails / engine gone |
UNKNOWN |
cgroup.events |
action |
|---|---|
frozen 1 |
proceed to migrate |
frozen 0, populated 0 |
cgroup empty: nothing to freeze/migrate |
frozen 0, populated, timeout |
continue without freeze guarantee, report it |
migration EBUSY/EAGAIN/frozen |
thaw source, retry once, continue either way |
migration ESRCH / PID gone |
treat as already-exited and reap |
pidfdctl.c/* pidfdctl.c - race-safe process engine for a bash cgroup supervisor.
*
* WHY THIS EXISTS
* --------------
* POSIX shells can only signal by numeric PID and reap with `wait(2)`.
* A numeric PID is recycled as soon as it is reaped, so `kill 0 $pid` after
* a concurrent exit can hit an unrelated process, and bash `wait` reports
* 128+signal for both "killed by signal N" and "exited with code 128+N".
*
* This helper is started once as a bash coprocess. It:
* * forks/execs each worker, migrating it into the requested cgroup v2
* subtree as the *first* action (before exec), so the worker can never
* run user code outside the delegated subtree;
* * opens a pidfd for the child immediately, before it can be reaped;
* * signals only through pidfd_send_signal(2), which is immune to PID
* reuse and returns ESRCH for an already-dead target;
* * reaps with waitid(P_PIDFD) to report the *raw* wait status
* (CLD_EXITED/KILLED/DUMPED + si_status), disambiguating 128+N;
* * on stdin EOF (supervisor died) SIGKILLs every tracked worker, so a
* crash can never leave an untracked process behind.
*
* PROTOCOL (one command per line, one reply per line)
* SPAWN <cgroup-path> <argc> -> OK <id> <pid> | ERR <msg>
* followed by <argc> newline-delimited argv lines
* SIGNAL <id> <signo> -> OK | ERR <msg>
* POLL <id> <timeout_ms> -> EXITED | RUNNING | ERR <msg>
* REAP <id> -> EXIT <si_code> <si_status> | ERR <msg>
* ALIVE <id> -> ALIVE | GONE | ERR <msg>
* CLOSE <id> -> OK
* QUIT -> exits
*
* BUILD: cc -O2 -Wall -o pidfdctl pidfdctl.c
* SPDX-License-Identifier: MIT
*/
#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <poll.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/syscall.h>
#include <sys/wait.h>
#include <unistd.h>
#ifndef SYS_pidfd_open
#define SYS_pidfd_open 434
#endif
#ifndef SYS_pidfd_send_signal
#define SYS_pidfd_send_signal 424
#endif
#ifndef P_PIDFD
#define P_PIDFD 3
#endif
#define MAX_FDS 4096
enum { SLOT_FREE = 0, SLOT_LIVE = 1, SLOT_REAPED = 2 };
struct slot {
int state;
int fd;
pid_t pid;
siginfo_t info;
};
static struct slot slots[MAX_FDS];
static int have_info[MAX_FDS];
static int pidfd_open_sys(pid_t pid)
{
return (int)syscall(SYS_pidfd_open, pid, 0);
}
static int pidfd_send_signal_sys(int pfd, int sig)
{
return (int)syscall(SYS_pidfd_send_signal, pfd, sig, NULL, 0);
}
static int alloc_slot(void)
{
for (int i = 0; i < MAX_FDS; i++)
if (slots[i].state == SLOT_FREE)
return i;
return -1;
}
static void kill_all(void)
{
for (int i = 0; i < MAX_FDS; i++) {
if (slots[i].state == SLOT_LIVE) {
pidfd_send_signal_sys(slots[i].fd, SIGKILL);
close(slots[i].fd);
}
slots[i].state = SLOT_FREE;
}
}
static int join_cgroup(const char *cg)
{
char path[4096];
int n = snprintf(path, sizeof path, "%s/cgroup.procs", cg);
if (n < 0 || (size_t)n >= sizeof path)
return -1;
int fd = open(path, O_WRONLY | O_CLOEXEC);
if (fd < 0)
return -1;
char buf[32];
int len = snprintf(buf, sizeof buf, "%d\n", (int)getpid());
ssize_t w = write(fd, buf, (size_t)len);
int e = errno;
close(fd);
errno = e;
return w == len ? 0 : -1;
}
static void do_spawn(char *args)
{
char *argcp = strrchr(args, ' ');
if (!argcp) { puts("ERR usage: SPAWN <cgroup> <argc>"); return; }
*argcp++ = '\0';
int argc = atoi(argcp);
if (argc <= 0 || argc > 4096) { puts("ERR bad argc"); return; }
char **argv = calloc((size_t)argc + 1, sizeof *argv);
if (!argv) { puts("ERR out of memory"); return; }
for (int i = 0; i < argc; i++) {
char buf[8192];
if (!fgets(buf, sizeof buf, stdin)) {
puts("ERR truncated argv");
for (int j = 0; j < i; j++) free(argv[j]); free(argv);
return;
}
buf[strcspn(buf, "\n")] = '\0';
argv[i] = strdup(buf);
if (!argv[i]) {
puts("ERR out of memory");
for (int j = 0; j < i; j++) free(argv[j]);
free(argv);
return;
}
}
argv[argc] = NULL;
int rp[2];
if (pipe2(rp, O_CLOEXEC) < 0) {
printf("ERR pipe: %s\n", strerror(errno));
goto out_free;
}
pid_t pid = fork();
if (pid < 0) {
printf("ERR fork: %s\n", strerror(errno));
close(rp[0]); close(rp[1]);
goto out_free;
}
if (pid == 0) {
close(rp[0]);
char status = 'F';
if (join_cgroup(args) != 0) {
(void)!write(rp[1], &status, 1);
close(rp[1]);
_exit(125);
}
status = 'R';
if (write(rp[1], &status, 1) != 1) { close(rp[1]); _exit(125); }
close(rp[1]);
execvp(argv[0], argv);
_exit(127);
}
close(rp[1]);
char status = 0;
ssize_t r = read(rp[0], &status, 1);
close(rp[0]);
if (r != 1 || status != 'R') {
waitpid(pid, NULL, 0);
if (r != 1) puts("ERR child failed before signalling readiness");
else if (status == 'F') printf("ERR cgroup join failed for %s\n", args);
else puts("ERR child failed before exec");
goto out_free;
}
int fd = pidfd_open_sys(pid);
if (fd < 0) {
kill(pid, SIGKILL); waitpid(pid, NULL, 0);
printf("ERR pidfd_open: %s\n", strerror(errno));
goto out_free;
}
int id = alloc_slot();
if (id < 0) {
close(fd); kill(pid, SIGKILL); waitpid(pid, NULL, 0);
puts("ERR no free slot");
goto out_free;
}
slots[id].state = SLOT_LIVE;
slots[id].fd = fd;
slots[id].pid = pid;
have_info[id] = 0;
printf("OK %d %d\n", id, (int)pid);
out_free:
for (int i = 0; i < argc; i++) free(argv[i]);
free(argv);
}
static int poll_fd(int fd, int timeout_ms)
{
struct pollfd p = { .fd = fd, .events = POLLIN };
return poll(&p, 1, timeout_ms); /* >0 exited, 0 running, <0 error */
}
int main(void)
{
char line[8192];
setvbuf(stdout, NULL, _IOLBF, 0);
while (fgets(line, sizeof line, stdin)) {
char cmd[32];
long a = 0, b = 0;
int n = sscanf(line, "%31s %ld %ld", cmd, &a, &b);
line[strcspn(line, "\n")] = '\0';
if (!strcmp(cmd, "SPAWN")) {
char *rest = line + strcspn(line, " \t");
while (*rest == ' ' || *rest == '\t') rest++;
do_spawn(rest);
} else if (!strcmp(cmd, "SIGNAL")) {
if (n < 3 || a < 0 || a >= MAX_FDS || slots[a].state != SLOT_LIVE) {
puts("ERR bad id"); continue;
}
if (pidfd_send_signal_sys(slots[a].fd, (int)b) < 0) {
printf("ERR pidfd_send_signal: %s\n", strerror(errno)); continue;
}
puts("OK");
} else if (!strcmp(cmd, "POLL")) {
if (n < 3 || a < 0 || a >= MAX_FDS ||
(slots[a].state != SLOT_LIVE && slots[a].state != SLOT_REAPED)) {
puts("ERR bad id"); continue;
}
if (slots[a].state == SLOT_REAPED) { puts("EXITED"); continue; }
int r = poll_fd(slots[a].fd, (int)b);
if (r < 0) printf("ERR poll: %s\n", strerror(errno));
else puts(r > 0 ? "EXITED" : "RUNNING");
} else if (!strcmp(cmd, "REAP")) {
if (n < 2 || a < 0 || a >= MAX_FDS) { puts("ERR bad id"); continue; }
int i = (int)a;
if (slots[i].state == SLOT_REAPED) {
printf("EXIT %d %d\n", have_info[i] ? slots[i].info.si_code : -1,
have_info[i] ? slots[i].info.si_status : -1);
continue;
}
if (slots[i].state != SLOT_LIVE) { puts("ERR bad id"); continue; }
siginfo_t info;
memset(&info, 0, sizeof info);
if (waitid(P_PIDFD, (id_t)slots[i].fd, &info, WEXITED) < 0) {
printf("ERR waitid: %s\n", strerror(errno)); continue;
}
slots[i].info = info;
have_info[i] = 1;
slots[i].state = SLOT_REAPED;
printf("EXIT %d %d\n", info.si_code, info.si_status);
} else if (!strcmp(cmd, "ALIVE")) {
if (n < 2 || a < 0 || a >= MAX_FDS || slots[a].state == SLOT_FREE) {
puts("ERR bad id"); continue;
}
if (slots[a].state == SLOT_REAPED) { puts("GONE"); continue; }
int r = poll_fd(slots[a].fd, 0);
puts(r > 0 ? "GONE" : (r == 0 ? "ALIVE" : "ERR poll"));
} else if (!strcmp(cmd, "CLOSE")) {
if (n >= 2 && a >= 0 && a < MAX_FDS && slots[a].state != SLOT_FREE) {
close(slots[a].fd);
slots[a].state = SLOT_FREE;
}
puts("OK");
} else if (!strcmp(cmd, "QUIT")) {
break;
} else if (n >= 1) {
printf("ERR unknown command: %s\n", cmd);
}
}
kill_all();
return 0;
}
supervise.sh#!/usr/bin/env bash
# supervise.sh - race-safe cgroup v2 worker supervisor (bash + pidfdctl).
set -u
trap '' PIPE # a dead engine must produce EPIPE, not kill the supervisor
SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
: "${SUP_CTL:=${SCRIPT_DIR}/pidfdctl}"
: "${SUP_FREEZE_TIMEOUT_MS:=5000}" # wait for cgroup.events frozen=1
: "${SUP_EXIT_TIMEOUT_MS:=10000}" # wait for a worker to exit
: "${SUP_POLL_MS:=20}"
SUP_MIG_ERR=$(mktemp -t sup_mig.XXXXXX 2>/dev/null || printf '/tmp/sup_mig.%d' "$$")
start_engine() {
if [[ ! -x "$SUP_CTL" && -f "${SUP_CTL}.c" ]]; then
cc -O2 -Wall -o "$SUP_CTL" "${SUP_CTL}.c" || return 1
fi
coproc SUP_ENGINE { "$SUP_CTL"; }
SUP_ENGINE_OUT=${SUP_ENGINE[0]}
SUP_ENGINE_IN=${SUP_ENGINE[1]}
}
ctl() {
printf '%s\n' "$1" >&"$SUP_ENGINE_IN" || return 1
local reply
IFS= read -r -u "$SUP_ENGINE_OUT" reply || return 1
printf '%s\n' "$reply"
}
stop_engine() {
{ printf 'QUIT\n' >&"$SUP_ENGINE_IN"; } 2>/dev/null || true
eval "exec ${SUP_ENGINE_IN}>&-" 2>/dev/null || true
wait "$SUP_ENGINE_PID" 2>/dev/null || true
rm -f "$SUP_MIG_ERR" 2>/dev/null || true
}
cg_events() { # $1=abs cgroup dir -> "frozen populated"
local file=$1/cgroup.events frozen='?' populated='?' k v
while IFS=' ' read -r k v; do
case $k in
frozen) frozen=$v ;;
populated) populated=$v ;;
esac
done <"$file" 2>/dev/null
printf '%s %s\n' "$frozen" "$populated"
}
cg_wait_frozen() { # $1=abs dir $2=timeout_ms -> 0 frozen, 2 empty, 3 timeout
local dir=$1 timeout_ms=$2
local deadline=$(( $(date +%s%N) / 1000000 + timeout_ms ))
while :; do
local frozen populated
read -r frozen populated < <(cg_events "$dir")
if [[ "$frozen" == 1 ]]; then return 0; fi
if [[ "$populated" == 0 ]]; then return 2; fi
if (( $(date +%s%N) / 1000000 >= deadline )); then return 3; fi
sleep 0.02
done
}
cg_freeze() { # 0 frozen, 2 already empty, 3 timeout, 4 unsupported
local dir=$1
if [[ ! -w "$dir/cgroup.freeze" ]]; then
echo "cg_freeze: $dir/cgroup.freeze not writable" >&2
return 4
fi
if ! printf '1\n' >"$dir/cgroup.freeze"; then
echo "cg_freeze: write failed for $dir" >&2
return 4
fi
cg_wait_frozen "$dir" "$SUP_FREEZE_TIMEOUT_MS"
}
cg_thaw() { printf '0\n' >"$1/cgroup.freeze" 2>/dev/null || true; }
cg_has_pid() { # $1=dir $2=pid
local p
while read -r p; do [[ "$p" == "$2" ]] && return 0; done <"$1/cgroup.procs" 2>/dev/null
return 1
}
cg_migrate() { # $1=pid $2=target_dir $3=source_dir -> 0 moved, 1 gone, 2 failed
local pid=$1 target=$2 source=$3
if printf '%s\n' "$pid" >>"$target/cgroup.procs" 2>"$SUP_MIG_ERR"; then
cg_has_pid "$target" "$pid" && return 0
return 2
fi
if grep -qiE 'busy|frozen|invalid|again' "$SUP_MIG_ERR" 2>/dev/null; then
echo "cg_migrate: thawing $source to migrate $pid" >&2
cg_thaw "$source"
if printf '%s\n' "$pid" >>"$target/cgroup.procs" 2>>"$SUP_MIG_ERR"; then
cg_has_pid "$target" "$pid" && return 0
fi
fi
if [[ ! -e /proc/$pid ]]; then return 1; fi
return 2
}
W_ID=(); W_PID=(); W_STAGE=(); W_STATE=(); W_DETAIL=()
W_INDEX=0
spawn_worker() { # $1=stage dir; remaining args = argv of the worker
local stage=$1; shift
[[ $# -gt 0 ]] || { echo "spawn_worker: empty argv" >&2; return 1; }
local reply id pid
mkdir -p "$stage" 2>/dev/null || true
printf 'SPAWN %s %d\n' "$stage" "$#" >&"$SUP_ENGINE_IN" || return 1
local a
for a in "$@"; do printf '%s\n' "$a" >&"$SUP_ENGINE_IN" || return 1; done
IFS= read -r -u "$SUP_ENGINE_OUT" reply || return 1
case $reply in
OK\ *)
local rest=${reply#OK }
id=${rest%% *}; pid=${rest##* }
W_ID+=("$id"); W_PID+=("$pid"); W_STAGE+=("$stage")
W_STATE+=(RUNNING); W_DETAIL+=("")
echo "spawned worker[$W_INDEX] pid=$pid id=$id stage=$stage" >&2
W_INDEX=$((W_INDEX + 1))
;;
*) echo "spawn failed: $reply" >&2; return 1 ;;
esac
}
cg_live_pids() { grep -E '^[0-9]+$' "$1/cgroup.procs" 2>/dev/null; }
find_untracked() { # $1=dir
local p i found
while read -r p; do
[[ -n "$p" ]] || continue
found=0
for ((i = 0; i < ${#W_PID[@]}; i++)); do
[[ "${W_PID[i]}" == "$p" ]] && { found=1; break; }
done
(( found == 0 )) && printf '%s\n' "$p"
done < <(cg_live_pids "$1")
}
reconcile_worker() { # $1=index $2=1 to wait for exit
local i=$1 wait_flag=${2:-0}
local id=${W_ID[i]} pid=${W_PID[i]} stage=${W_STAGE[i]}
local r
r=$(ctl "POLL $id 0") || { W_STATE[i]=UNKNOWN; W_DETAIL[i]="engine gone"; return 1; }
if [[ "$r" == RUNNING && "$wait_flag" == 1 ]]; then
r=$(ctl "POLL $id $SUP_EXIT_TIMEOUT_MS")
fi
case $r in
RUNNING) W_STATE[i]=RUNNING; W_DETAIL[i]="still running"; return 0 ;;
EXITED|GONE)
local rep code status
rep=$(ctl "REAP $id")
read -r _ code status <<<"$rep"
if ! [[ "$code" =~ ^[0-9]+$ ]]; then
W_STATE[i]=UNKNOWN; W_DETAIL[i]="reap failed: $rep"; return 1
fi
case $code in
1) W_STATE[i]=EXITED; W_DETAIL[i]="exit status=$status" ;;
2) W_STATE[i]=SIGNALED; W_DETAIL[i]="killed by signal=$status" ;;
3) W_STATE[i]=SIGNALED; W_DETAIL[i]="dumped, signal=$status" ;;
*) W_STATE[i]=UNKNOWN; W_DETAIL[i]="si_code=$code status=$status" ;;
esac
if cg_has_pid "$stage" "$pid"; then
W_DETAIL[i]+=" (WARN: still listed in $stage/cgroup.procs)"
fi
return 0
;;
ERR*) W_STATE[i]=UNKNOWN; W_DETAIL[i]=$r; return 1 ;;
*) W_STATE[i]=UNKNOWN; W_DETAIL[i]="bad reply: $r"; return 1 ;;
esac
}
main() {
local delegated=${1:?delegated cgroup path}; shift
local cmds=("$@")
[[ ${#cmds[@]} -gt 0 ]] || { echo "no commands" >&2; exit 2; }
start_engine || { echo "cannot start engine" >&2; exit 1; }
trap 'stop_engine' EXIT
local stage_base=$delegated/stage
local target=$delegated/target
mkdir -p "$stage_base" "$target" 2>/dev/null || true
local i
for ((i = 0; i < ${#cmds[@]}; i++)); do
local -a argv=()
eval "argv=(${cmds[i]})" 2>/dev/null || argv=(/bin/false)
[[ ${#argv[@]} -gt 0 ]] || argv=(/bin/false)
spawn_worker "$stage_base/w$i" "${argv[@]}"
done
for ((i = 0; i < ${#W_ID[@]}; i++)); do
local stage=${W_STAGE[i]}
if cg_freeze "$stage"; then
echo "worker[$i] stage frozen" >&2
else
case $? in
2) echo "worker[$i] stage already empty before freeze" >&2 ;;
3) echo "worker[$i] FREEZE TIMEOUT - continuing without freeze guarantee" >&2 ;;
4) echo "worker[$i] freezing unsupported - continuing" >&2 ;;
esac
fi
done
for ((i = 0; i < ${#W_ID[@]}; i++)); do
local pid=${W_PID[i]} stage=${W_STAGE[i]}
local rc=0
cg_migrate "$pid" "$target" "$stage" || rc=$?
case $rc in
0) echo "worker[$i] migrated to $target" >&2 ;;
1) echo "worker[$i] already exited before migration" >&2 ;;
2) echo "worker[$i] MIGRATION FAILED" >&2 ;;
esac
done
for ((i = 0; i < ${#W_ID[@]}; i++)); do cg_thaw "${W_STAGE[i]}"; done
local untracked
untracked=$(find_untracked "$target"; find_untracked "$stage_base")
if [[ -n "$untracked" ]]; then
echo "ALERT: untracked pids in delegated subtree: $untracked" >&2
fi
local failed=0
for ((i = 0; i < ${#W_ID[@]}; i++)); do
reconcile_worker "$i" 1 || failed=1
done
echo "===== supervisor report ====="
for ((i = 0; i < ${#W_ID[@]}; i++)); do
printf 'worker[%d] pid=%-7s state=%-8s %s\n' \
"$i" "${W_PID[i]}" "${W_STATE[i]}" "${W_DETAIL[i]}"
done
echo "============================="
exit "$failed"
}
if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
main "$@"
fi
cc -O2 -Wall -o pidfdctl pidfdctl.c
./supervise.sh <delegated-cgroup-path> \
"sh -c 'exit 0'" \
"sh -c 'sleep 0.2; exit 7'" \
"sh -c 'kill -TERM \$\$'"
supervise.sh compiles pidfdctl automatically if missing. It needs write access to a delegated cgroup v2 subtree. On systemd:
systemd-run --user --scope -p Delegate=yes -- bash -c \
'mkdir -p /sys/fs/cgroup/demo && ./supervise.sh /sys/fs/cgroup/demo "sh -c \"exit 7\""'
# or as root / in a container that owns its cgroup:
mkdir -p /sys/fs/cgroup/demo && ./supervise.sh /sys/fs/cgroup/demo "sh -c 'exit 7'"
stat -fc %T /sys/fs/cgroup # must print: cgroup2fs
Executed here (Linux 7.0.0, gcc 15.2, bash 5.3; pidfd_open/pidfd_send_signal/waitid(P_PIDFD) available). Because /sys/fs/cgroup is not mounted in this workspace and there is no CAP_SYS_ADMIN (mount -t cgroup2 and unshare -m both fail), the cgroup filesystem is emulated with a small mock kernel (regular files plus a poller that materialises cgroup.events from cgroup.freeze and moves PIDs between cgroup.procs files). The process-control path — fork, exec, pidfd, signal, waitid, EOF-kill — runs natively and unmodified.
test_pidfdctl.sh) — 13/13 pass$ ./test_pidfdctl.sh
PASS: spawn returned OK
PASS: child joined mock cgroup (1870)
PASS: exact status CLD_EXITED=1 si_status=200
PASS: reap is idempotent
PASS: pidfd signal
PASS: exact status CLD_KILLED=2 si_status=15
PASS: direct exec preserves signal status
PASS: signal after reap refused (ERR bad id)
PASS: bad cgroup path rejected (ERR cgroup join failed for .../nope)
PASS: concurrent exits attributed correctly (reverse-order reap)
PASS: worker killed on engine exit
----
pass=13 fail=0
Proves: exit 200 is reported as si_code=1 si_status=200 (not 128+N); SIGTERM as si_code=2 si_status=15; signal after reap is refused; five concurrent exits reaped in reverse order keep correct attribution; tracked workers are SIGKILLed when the supervisor end closes.
test_supervise.sh) — all checks pass$ ./test_supervise.sh
worker[0] stage frozen
worker[1] stage frozen
cg_freeze: .../demo/stage/w2/cgroup.freeze not writable
worker[2] freezing unsupported - continuing
worker[0] migrated to .../demo/target
worker[1] migrated to .../demo/target
worker[2] migrated to .../demo/target
ALERT: untracked pids in delegated subtree: 2416
===== supervisor report =====
worker[0] pid=2426 state=EXITED exit status=0
worker[1] pid=2428 state=EXITED exit status=7
worker[2] pid=2431 state=SIGNALED killed by signal=15
=============================
supervisor rc=0
ALL SUPERVISOR CHECKS PASSED
Exercises launch → per-worker stage cgroups → freeze → migrate → thaw; the partial-failure path (missing cgroup.freeze → freezing unsupported, run continues); foreign/untracked detection; stages drain to empty; tracked PIDs end in target; exit 0 and signal 15 reported distinctly.
| Scenario | Expected | Observed |
|---|---|---|
| non-existent cgroup path | ERR cgroup join failed, payload never runs |
pass |
| signal after reap | refused (ESRCH/bad id) |
pass |
| concurrent exits | per-pidfd attribution | pass |
| supervisor process dies | tracked workers SIGKILLed | pass |
| freeze unsupported | run continues, warning emitted | pass |
| foreign PID in subtree | ALERT: untracked pids … |
pass |
On a real cgroup2fs host the same commands run unchanged; cg_events, cg_freeze, cg_migrate talk to the kernel instead of the mock:
stat -fc %T /sys/fs/cgroup # cgroup2fs
mkdir -p /sys/fs/cgroup/demo
./supervise.sh /sys/fs/cgroup/demo "sh -c 'exit 0'" "sh -c 'sleep 0.2; exit 7'" "sh -c 'kill -TERM \$\$'"
# expect: EXITED status=0, EXITED status=7, SIGNALED signal=15, rc=0
EBUSY/EAGAIN thaw-and-retry branch is exercised by inspection, not a forced kernel error.ESRCH behavior that makes reuse harmless.>> in cg_migrate is deliberate: on cgroupfs every write() to cgroup.procs is parsed independently, so >> and > are equivalent; append also lets the file-backed mock observe successive migrations.The full runnable artifacts are at ~/solution/: pidfdctl.c, supervise.sh, test_pidfdctl.sh, test_supervise.sh, and SOLUTION.md.
# Evidence - Problem class: shell-cgroup-freezer-pidfd-exec-fence-reconciliation - Model: openrouter/deepseek/deepseek-v4.1-flash - Solved: 2026-10-10T10:08:58.205Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Build a race-safe shell supervisor that launches short-lived workers into delegated cgroup v2 subtrees, freezes and migrates them, and reports exact termination status despite PID reuse and concurrent exits. Reconcile pidfd, cgroup.events, and wait status across partial failures without signaling an unrelated process or leaving an untracked worker.", "environment": "bash", "language": "shell", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "shell-cgroup-freezer-pidfd-exec-fence-reconciliation", "provider": "openrouter", "solved_at": "2026-10-10T10:08:58.210Z", "version": "5.2"}