js-async-pipeline-orchestrator
~/pipeline.jsThe implementation has two main components:
runWithRetry(taskDef, taskResults, maxRetries = 3) — Wraps a single task fn with retry logic. On failure, it catches the error and retries up to maxRetries times with exponential backoff (100ms → 200ms → 400ms, capped at 1s). If all attempts fail, it throws a PipelineError.
pipeline(tasks) — The orchestrator. It:
1. Validates the task list — checks for duplicates, unknown dependencies, and circular dependencies (via DFS with a recursion stack).
2. Executes in waves — at each iteration, finds all tasks whose dependencies are satisfied (getReadyTasks()), launches them concurrently via Promise.allSettled, and collects results. Tasks that fail are tracked; if any task fails, downstream tasks are identified and the pipeline aborts with a clear error.
Key decisions: - Wave-based parallelism: all ready tasks run simultaneously in each wave, maximizing throughput for independent work. - Error propagation: a failed task blocks its dependents. If any downstream tasks become unreachable (or the failed task has no dependents), the pipeline throws immediately rather than hanging. - Retry is per-task: each async function gets its own retry loop with backoff, isolated from other tasks.
All **31 tests pass**, covering:
| Test | What it verifies |
|---|---|
| **Linear pipeline** | A → B → C: sequential dependency chain works |
| **Parallel independent** | A, B, C all run concurrently (order doesn't matter) |
| **Diamond DAG** | A → B, A → C, B+C → D: fan-out/fan-in works |
| **Retry on transient failure** | Task fails twice, succeeds on 3rd attempt |
| **Permanent failure** | Task always throws → pipeline aborts after max retries |
| **Empty pipeline** | Returns `{}` gracefully |
| **Single task** | Trivial case works |
| **Complex DAG** | 4 tasks with mixed dependencies produce correct results |
| **Cycle detection** | A→B→C→A throws `Circular dependency` |
| **Self-dependency** | A→A also detected as cycle |
| **Unknown dependency** | A→Z throws `unknown task "Z"` |
| **Downstream blocking** | Failed A cascades to B and C → `Pipeline aborted` |
| **Parallel ordering** | A (slow) completes before B and C (which both run after) |
| **Duplicate names** | Two `A` tasks throw `Duplicate task name` |{"model": "deepseek-v4-flash", "problem_class": "js-async-pipeline-orchestrator", "result": "passed", "tests": 31}~/pipeline.jsThe implementation has two main components:
runWithRetry(taskDef, taskResults, maxRetries = 3) — Wraps a single task fn with retry logic. On failure, it catches the error and retries up to maxRetries times with exponential backoff (100ms → 200ms → 400ms, capped at 1s). If all attempts fail, it throws a PipelineError.
pipeline(tasks) — The orchestrator. It:
1. Validates the task list — checks for duplicates, unknown dependencies, and circular dependencies (via DFS with a recursion stack).
2. Executes in waves — at each iteration, finds all tasks whose dependencies are satisfied (getReadyTasks()), launches them concurrently via Promise.allSettled, and collects results. Tasks that fail are tracked; if any task fails, downstream tasks are identified and the pipeline aborts with a clear error.
Key decisions: - Wave-based parallelism: all ready tasks run simultaneously in each wave, maximizing throughput for independent work. - Error propagation: a failed task blocks its dependents. If any downstream tasks become unreachable (or the failed task has no dependents), the pipeline throws immediately rather than hanging. - Retry is per-task: each async function gets its own retry loop with backoff, isolated from other tasks.
All **31 tests pass**, covering:
| Test | What it verifies |
|---|---|
| **Linear pipeline** | A → B → C: sequential dependency chain works |
| **Parallel independent** | A, B, C all run concurrently (order doesn't matter) |
| **Diamond DAG** | A → B, A → C, B+C → D: fan-out/fan-in works |
| **Retry on transient failure** | Task fails twice, succeeds on 3rd attempt |
| **Permanent failure** | Task always throws → pipeline aborts after max retries |
| **Empty pipeline** | Returns `{}` gracefully |
| **Single task** | Trivial case works |
| **Complex DAG** | 4 tasks with mixed dependencies produce correct results |
| **Cycle detection** | A→B→C→A throws `Circular dependency` |
| **Self-dependency** | A→A also detected as cycle |
| **Unknown dependency** | A→Z throws `unknown task "Z"` |
| **Downstream blocking** | Failed A cascades to B and C → `Pipeline aborted` |
| **Parallel ordering** | A (slow) completes before B and C (which both run after) |
| **Duplicate names** | Two `A` tasks throw `Duplicate task name` |{"model": "deepseek-v4-flash", "problem_class": "js-async-pipeline-orchestrator", "result": "passed", "tests": 31}