shell-argmax-batch-size-e2big-safe
Done. The solution is in ~/SOLUTION.md (self-contained writeup) with the working script at ~/batch-exec.
Root cause. execve fails with E2BIG when the total bytes of argv + envp (strings, NUL terminators, per-arg pointers, and the argv/envp NULL terminators) exceed ARG_MAX. The environment shares the same budget, so a 100000-byte env leaves only ~31 KB for arguments. Naive approaches fail twice: cmd $(cat …) word-splits (corrupts whitespace) and a single unbounded execve blows the limit (reproduced: 2.55 MB of args → Argument list too long, exit 126).
Fix. A pure POSIX sh script (#!/bin/sh, dash- and bash-clean) that:
- computes ARG_MAX via getconf (PATH lookup, so a test shim works),
- measures env_size = Σ len(VAR=value)+2 + 4·count from env -0 (awk RS="\0", byte-accurate, whitespace/newline-safe),
- scans NUL-delimited stdin once for max_arg_len/count (awk), then replays it byte-identically through xargs -0 -n N,
- uses N = (ARG_MAX − env_size − 2048 − fixed) / (max_arg_len + 10), with 2048 margin and 10 = 8-byte pointer + 2 per arg, plus explicit deduction of the command's own argv. -n N is the provable cap — GNU xargs's internal 128 KiB buffer (--show-limits) only ever shrinks batches, and -s is deliberately avoided since it ignores per-arg pointers.
Verification (all run for real):
- Synthetic acceptance case (getconf shim ARG_MAX=131072, env_size exactly 100000, 100000 × 16-byte args): N = 1115 ≤ 1116, 90 batches, sum 100000, exit 0, kernel-side total ≈ 128 KB < 131072 — no E2BIG. (With BATCH_DEBUG=1: N=1114, still ≤ 1116.)
- Real machine (ARG_MAX=2097152): same 100000 args complete, exit 0.
- Naive unbatched equivalent: E2BIG, exit 126 — demonstrating the failure the script prevents.
- Byte-for-byte fidelity for args containing spaces, padded whitespace, embedded newlines, and empty strings.
- Runs identically under dash (POSIX sh) and bash.
# Evidence - Problem class: shell-argmax-batch-size-e2big-safe - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-31T04:49:13.839Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Write a POSIX sh script that reads NUL-delimited arguments from stdin and executes a given command once per batch, choosing batch size so execve never fails with E2BIG (Argument list too long): N = (ARG_MAX - env_size - 2048) / (max_arg_len + 10), where ARG_MAX comes from getconf ARG_MAX, env_size sums len(VAR=value)+2 over exported vars plus 4-byte pointer overhead each, and 10 is per-argument overhead (8-byte pointer + 2); use xargs -0 -n N or an equivalent loop so whitespace in arguments is preserved. Verify: with ARG_MAX=131072 and a 100000-byte environment, 16-byte arguments must give N <= 1116, and running against 100000 such arguments must complete without E2BIG.", "environment": "bash", "language": "shell", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "shell-argmax-batch-size-e2big-safe", "provider": "openrouter", "solved_at": "2026-08-31T04:49:13.839Z", "version": "5.2"}Done. The solution is in ~/SOLUTION.md (self-contained writeup) with the working script at ~/batch-exec.
Root cause. execve fails with E2BIG when the total bytes of argv + envp (strings, NUL terminators, per-arg pointers, and the argv/envp NULL terminators) exceed ARG_MAX. The environment shares the same budget, so a 100000-byte env leaves only ~31 KB for arguments. Naive approaches fail twice: cmd $(cat …) word-splits (corrupts whitespace) and a single unbounded execve blows the limit (reproduced: 2.55 MB of args → Argument list too long, exit 126).
Fix. A pure POSIX sh script (#!/bin/sh, dash- and bash-clean) that:
- computes ARG_MAX via getconf (PATH lookup, so a test shim works),
- measures env_size = Σ len(VAR=value)+2 + 4·count from env -0 (awk RS="\0", byte-accurate, whitespace/newline-safe),
- scans NUL-delimited stdin once for max_arg_len/count (awk), then replays it byte-identically through xargs -0 -n N,
- uses N = (ARG_MAX − env_size − 2048 − fixed) / (max_arg_len + 10), with 2048 margin and 10 = 8-byte pointer + 2 per arg, plus explicit deduction of the command's own argv. -n N is the provable cap — GNU xargs's internal 128 KiB buffer (--show-limits) only ever shrinks batches, and -s is deliberately avoided since it ignores per-arg pointers.
Verification (all run for real):
- Synthetic acceptance case (getconf shim ARG_MAX=131072, env_size exactly 100000, 100000 × 16-byte args): N = 1115 ≤ 1116, 90 batches, sum 100000, exit 0, kernel-side total ≈ 128 KB < 131072 — no E2BIG. (With BATCH_DEBUG=1: N=1114, still ≤ 1116.)
- Real machine (ARG_MAX=2097152): same 100000 args complete, exit 0.
- Naive unbatched equivalent: E2BIG, exit 126 — demonstrating the failure the script prevents.
- Byte-for-byte fidelity for args containing spaces, padded whitespace, embedded newlines, and empty strings.
- Runs identically under dash (POSIX sh) and bash.
# Evidence - Problem class: shell-argmax-batch-size-e2big-safe - Model: openrouter/deepseek/deepseek-v4-flash-0731 - Solved: 2026-08-31T04:49:13.839Z - Verification: solution produced by pi in sandbox; see signatures.json
{"description": "Write a POSIX sh script that reads NUL-delimited arguments from stdin and executes a given command once per batch, choosing batch size so execve never fails with E2BIG (Argument list too long): N = (ARG_MAX - env_size - 2048) / (max_arg_len + 10), where ARG_MAX comes from getconf ARG_MAX, env_size sums len(VAR=value)+2 over exported vars plus 4-byte pointer overhead each, and 10 is per-argument overhead (8-byte pointer + 2); use xargs -0 -n N or an equivalent loop so whitespace in arguments is preserved. Verify: with ARG_MAX=131072 and a 100000-byte environment, 16-byte arguments must give N <= 1116, and running against 100000 such arguments must complete without E2BIG.", "environment": "bash", "language": "shell", "model": "openrouter/deepseek/deepseek-v4-flash-0731", "problem_class": "shell-argmax-batch-size-e2big-safe", "provider": "openrouter", "solved_at": "2026-08-31T04:49:13.839Z", "version": "5.2"}