s3-head-404-display-string-match-not-notfound

# Evidence
- Problem class: s3-head-404-display-string-match-not-notfound
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-20T12:27:49.913Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "Bug: classifying an absent remote object as a FAILURE instead of 'not present yet' by string-matching the error Display text.\n\nhilo-backends/src/s3.rs head_object_meta() (also head_object_last_modified()) guarded with `Err(e) if format!(\"{}\", e).contains(\"NotFound\") => Ok(None)`. When the S3-compatible endpoint answers HEAD with a 404 whose rendered Display does not contain the literal 'NotFound' (empty body -> SDK deserializes Unhandled carrying only the wire code), the guard MISSES and the error falls through to `Err(S3Error::from(e))`, aborting the whole sync. Symptom: `hilo backend sync --push` on a FRESH bucket prints a correct plan line then dies with 'error: <mount>: backend error: aws sdk error: s3: aws error: service error' and EXIT=1 -- while re-syncing already-present objects succeeds, so a retry-style test passes and misses it.\n\nFix (landed): layered predicate replacing the Display substring match, strongest signal first:\n 1. raw_response().status() == 404 -- unambiguous on every endpoint; HEAD has no body to parse.\n 2. typed `HeadObjectError::is_not_found()` (aws-sdk-s3 models the response 404 as the NotFound variant).\n 3. wire error code == \"NotFound\" (ProvideErrorMetadata::code) -- covers empty-body 404s where only the code survives.\nFactor the decision into a pure `head_failure_means_absent(status, typed_not_found, code) -> bool` so it is unit-testable WITHOUT a live endpoint or SDK error plumbing; the old guard was untestable and therefore untested.\n\nLive falsifiable repro (both binaries, same endpoint, different fresh prefixes, only the code differs):\n  OLD: plan s3old: 2 to transfer, ... -> error: s3old: backend error: aws sdk error: s3: aws error: service error ; EXIT=1\n  NEW: plan s3new: 2 to transfer, ... -> synced s3new: 2 transferred (96 bytes), 0 conflicts recorded ; EXIT=0\n  Remote read-back: dognew/docs/notes.md (47B) + dognew/src/main.rs (49B) present; ignored target/ absent (ignore engine held).\n\nGENERAL RULE it came from: never classify an SDK error by matching a substring of its Display. Display is a human-facing summary that truncates the cause chain; use the typed variant, the machine-readable error code, or an unambiguous transport signal (HTTP status). String matching also risks FALSE POSITIVES -- an unrelated error whose message happens to mention the token is silently swallowed as 'absent'.\n\nSecondary test-hygiene finding in the same area (already filed on the project board as DF-WARPFS-13): hilo-backends/tests/s3_integration_test.rs reports '7 passed; 0.04s' while executing zero real requests, because its readiness gate demands MinIO's specific /minio/health/live path and each test then early-returns while still being reported as PASS -- a guaranteed green tick in CI that provides no coverage of the very path this bug lives on.", "environment": "linux; hilo (<project>) Rust workspace; hilo-backends/src/s3.rs; aws-sdk-s3 1.138.1; S3-compatible emulator (moto) on <ip-address>:19001", "language": "rust", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "s3-head-404-display-string-match-not-notfound", "provider": "openrouter", "solved_at": "2026-09-20T12:27:49.913Z", "version": "0.3.0"}

# Evidence
- Problem class: s3-head-404-display-string-match-not-notfound
- Model: openrouter/deepseek/deepseek-v4.1-flash
- Solved: 2026-09-20T12:27:49.913Z
- Verification: solution produced by pi in sandbox; see signatures.json

{"description": "Bug: classifying an absent remote object as a FAILURE instead of 'not present yet' by string-matching the error Display text.\n\nhilo-backends/src/s3.rs head_object_meta() (also head_object_last_modified()) guarded with `Err(e) if format!(\"{}\", e).contains(\"NotFound\") => Ok(None)`. When the S3-compatible endpoint answers HEAD with a 404 whose rendered Display does not contain the literal 'NotFound' (empty body -> SDK deserializes Unhandled carrying only the wire code), the guard MISSES and the error falls through to `Err(S3Error::from(e))`, aborting the whole sync. Symptom: `hilo backend sync --push` on a FRESH bucket prints a correct plan line then dies with 'error: <mount>: backend error: aws sdk error: s3: aws error: service error' and EXIT=1 -- while re-syncing already-present objects succeeds, so a retry-style test passes and misses it.\n\nFix (landed): layered predicate replacing the Display substring match, strongest signal first:\n 1. raw_response().status() == 404 -- unambiguous on every endpoint; HEAD has no body to parse.\n 2. typed `HeadObjectError::is_not_found()` (aws-sdk-s3 models the response 404 as the NotFound variant).\n 3. wire error code == \"NotFound\" (ProvideErrorMetadata::code) -- covers empty-body 404s where only the code survives.\nFactor the decision into a pure `head_failure_means_absent(status, typed_not_found, code) -> bool` so it is unit-testable WITHOUT a live endpoint or SDK error plumbing; the old guard was untestable and therefore untested.\n\nLive falsifiable repro (both binaries, same endpoint, different fresh prefixes, only the code differs):\n  OLD: plan s3old: 2 to transfer, ... -> error: s3old: backend error: aws sdk error: s3: aws error: service error ; EXIT=1\n  NEW: plan s3new: 2 to transfer, ... -> synced s3new: 2 transferred (96 bytes), 0 conflicts recorded ; EXIT=0\n  Remote read-back: dognew/docs/notes.md (47B) + dognew/src/main.rs (49B) present; ignored target/ absent (ignore engine held).\n\nGENERAL RULE it came from: never classify an SDK error by matching a substring of its Display. Display is a human-facing summary that truncates the cause chain; use the typed variant, the machine-readable error code, or an unambiguous transport signal (HTTP status). String matching also risks FALSE POSITIVES -- an unrelated error whose message happens to mention the token is silently swallowed as 'absent'.\n\nSecondary test-hygiene finding in the same area (already filed on the project board as DF-WARPFS-13): hilo-backends/tests/s3_integration_test.rs reports '7 passed; 0.04s' while executing zero real requests, because its readiness gate demands MinIO's specific /minio/health/live path and each test then early-returns while still being reported as PASS -- a guaranteed green tick in CI that provides no coverage of the very path this bug lives on.", "environment": "linux; hilo (<project>) Rust workspace; hilo-backends/src/s3.rs; aws-sdk-s3 1.138.1; S3-compatible emulator (moto) on <ip-address>:19001", "language": "rust", "model": "openrouter/deepseek/deepseek-v4.1-flash", "problem_class": "s3-head-404-display-string-match-not-notfound", "provider": "openrouter", "solved_at": "2026-09-20T12:27:49.913Z", "version": "0.3.0"}