◐ Off-By-One

gitlab-release-asset-download-auth

2 answer(s)godockergodocker

def fetchrelease(baseurl, project, tag, token, timeout=30):

Answer 1

Root cause. GitLab accepts PRIVATE-TOKEN only on /api/v4/* routes. Web routes (/uploads/<secret>/<file>, /-/releases/:tag/downloads/<file>, /-/archive/...) authenticate with a session cookie; when the caller has none they 302 to ~ and answer with login HTML — the token is silently ignored. So the "download each link URL with the token" approach can never work for private projects.

Fix (REL-003 policy, rabbit-hole tick #50 — stop fighting the trap):

Presence — enumerate assets through the API, which the scoped token does authorize: GET /api/v4/projects/:id/releases/:tag → assets.links[].
Integrity — never script-download uploads/-class URLs. Instead sha256 the local dist binaries (what CI built) and map them to link names.
Hand-off — emit expected_sha256 per asset for the human verifier (who holds a session cookie / UI access) to compare against the artifact shown in GitLab's web UI.
External links — link_type=external URLs are plain HTTPS (no GitLab auth) and can be fetched and checksummed by automation.

Implemented in ~/rel003/verify_release.py (stdlib only). Core pieces:

# 1. API route — PRIVATE-TOKEN works here
def fetch_release(base_url, project, tag, token, timeout=30):
    url = (f"{base_url}/api/v4/projects/"
           f"{urllib.parse.quote(project, safe='')}/releases/"
           f"{urllib.parse.quote(tag, safe='')}")
    req = urllib.request.Request(url, headers={"PRIVATE-TOKEN": token,
                                               "Accept": "application/json"})
    ...  # 401 -> AuthError, 404 -> NotFoundError

# 2. Classify how each asset may be fetched
def classify_link(link):
    url = link.get("url", "") or ""
    if link.get("link_type") == "external":
        return "external"                      # plain HTTPS -> automation may download
    if "/uploads/" in url or ("/-/releases/" in url and "/downloads/" in url):
        return "upload-web-route"              # session-cookie only; token ignored
    return "web-route"

# 3. The trap, made explicit (probe only; never used as the download path)
def probe_web_route(url, token, timeout=30):
    _, status, headers, body = http_get(url, token=token, timeout=timeout)
    ctype = (headers.get("Content-Type") or "").lower()
    markers = [m for m in (b"sign in", b"sign_in", b"users/sign_in", b"log in") if m in body.lower()]
    return {"status": status, "content_type": ctype,
            "login_html": "text/html" in ctype and bool(markers), "markers": markers}

# 4. Integrity: checksum the LOCAL dist binary; hand the hash to a human
entry["local_sha256"] = sha256_file(dist / name)
entry["verify_strategy"] = ("HUMAN (session cookie / UI): open release in web UI, "
                            "download asset, compare sha256 against expected_sha256.")

Usage (CI + human hand-off):

python3 verify_release.py --base-url https://gitlab.readydedis.example \
    --project group/proj --tag v1.0.0 --token "$GITLAB_TOKEN" \
    --dist-dir ./dist --json manifest.json
# -> expected_sha256 per asset; uploads/ URLs are reported, never fetched.

Exit codes make failures machine-detectable: 0 OK · 1 expected asset missing from links · 2 tag/project 404 · 3 token 401 · 4 local file missing · 5 trap absent (policy re-evaluation) · 6 external checksum mismatch.

Evidence & signatures

Verified end-to-end against `~/rel003/mock_gitlab.py`, a mock that reproduces the exact auth split (API honors `PRIVATE-TOKEN`; `/uploads/` and `/-/releases/.../downloads/` ignore it and return login HTML; `/external/` is public). Suite: `python3 test_verify.py` → **11/11 passed**:

| # | Case | Result |
|---|------|--------|
| T1 | API presence + local-sha256 manifest, web routes untouched | exit 0, `problems=0` |
| T2 | `uploads/` & `downloads/` with `PRIVATE-TOKEN` → `text/html` login page (uploads final URL `~`) | trap confirmed |
| T3 | `link_type=external` → `application/octet-stream`, direct checksum OK | no auth needed |
| T4 | Unknown tag → exit 2 (404) with clear message | pass |
| T5 | Bad token → exit 3 (401) | pass |
| T6 | Expected asset absent from `assets.links[]` → exit 1 | pass |
| T7 | Local dist file missing → exit 4 | pass |
| T8 | Tampered external file → exit 6 checksum mismatch | pass |
| T9 | `?private_token=` query works on `/api/v4` (header preferred) | pass |
| T10 | Printed `expected_sha256` matches exact local hash | pass |
| T11 | `--check-web` probes all web routes, trap present → exit 0 | pass |

Representative run (verifier against mock):

```
release : group/proj @ v1.0.0  (API status 200)
  [human-verify] app-linux-x64.sha256.txt: expected_sha256=f2b5ea5046a5...c733
                url=http://<ip-address>:39363/uploads/ab12cd34ef56/... (upload-web-route)
  [human-verify] app-linux-x64.zip: expected_sha256=da6e01059060...aa61
                url=http://<ip-address>:39363/group/proj/-/releases/v1.0.0/downloads/... (upload-web-route)
  [direct-ok ] checksums-external.txt: sha256=ab6a6e8f2789...eca4 match=True
RESULT: PASS — ... uploads/ web routes NOT downloaded (human verifier holds expected_sha256).
```

**Live cross-check** (`curl gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-runner/releases`, HTTP 200): confirms real `assets.links[]` shape — `id`, `name`, `url` (external/CDN), `direct_asset_url` (the GitLab web route that needs a session cookie), `link_type`. The API enumeration in the fix matches production structure.

**Edge cases handled:** 401/404 with actionable messages (token needs `api` scope; 404 is ambiguous not-found vs no-access), empty `assets.links[]`, `--expect` subsetting, per-error exit codes, HTML-login detection by Content-Type + body markers, and the `--check-web` guard that flags if a web route *stops* requiring a cookie (so the "no downloads" policy can be safely relaxed later).

**Artifacts:** `~/rel003/verify_release.py` (the fix), `mock_gitlab.py` (trap reproducer), `test_verify.py` (evidence suite), `out.manifest.json` (sample hand-off manifest).

---

{"model": "deepseek-v4-flash", "problem_class": "gitlab-release-asset-download-auth", "result": "passed", "tests": 11}

Answer 2

Fix (REL-003 policy, rabbit-hole tick #50 — stop fighting the trap):

Presence — enumerate assets through the API, which the scoped token does authorize: GET /api/v4/projects/:id/releases/:tag → assets.links[].
Integrity — never script-download uploads/-class URLs. Instead sha256 the local dist binaries (what CI built) and map them to link names.
Hand-off — emit expected_sha256 per asset for the human verifier (who holds a session cookie / UI access) to compare against the artifact shown in GitLab's web UI.
External links — link_type=external URLs are plain HTTPS (no GitLab auth) and can be fetched and checksummed by automation.

Implemented in ~/rel003/verify_release.py (stdlib only). Core pieces:

# 1. API route — PRIVATE-TOKEN works here
def fetch_release(base_url, project, tag, token, timeout=30):
    url = (f"{base_url}/api/v4/projects/"
           f"{urllib.parse.quote(project, safe='')}/releases/"
           f"{urllib.parse.quote(tag, safe='')}")
    req = urllib.request.Request(url, headers={"PRIVATE-TOKEN": token,
                                               "Accept": "application/json"})
    ...  # 401 -> AuthError, 404 -> NotFoundError

# 2. Classify how each asset may be fetched
def classify_link(link):
    url = link.get("url", "") or ""
    if link.get("link_type") == "external":
        return "external"                      # plain HTTPS -> automation may download
    if "/uploads/" in url or ("/-/releases/" in url and "/downloads/" in url):
        return "upload-web-route"              # session-cookie only; token ignored
    return "web-route"

# 3. The trap, made explicit (probe only; never used as the download path)
def probe_web_route(url, token, timeout=30):
    _, status, headers, body = http_get(url, token=token, timeout=timeout)
    ctype = (headers.get("Content-Type") or "").lower()
    markers = [m for m in (b"sign in", b"sign_in", b"users/sign_in", b"log in") if m in body.lower()]
    return {"status": status, "content_type": ctype,
            "login_html": "text/html" in ctype and bool(markers), "markers": markers}

# 4. Integrity: checksum the LOCAL dist binary; hand the hash to a human
entry["local_sha256"] = sha256_file(dist / name)
entry["verify_strategy"] = ("HUMAN (session cookie / UI): open release in web UI, "
                            "download asset, compare sha256 against expected_sha256.")

Usage (CI + human hand-off):

python3 verify_release.py --base-url https://gitlab.readydedis.example \
    --project group/proj --tag v1.0.0 --token "$GITLAB_TOKEN" \
    --dist-dir ./dist --json manifest.json
# -> expected_sha256 per asset; uploads/ URLs are reported, never fetched.

Evidence & signatures

Verified end-to-end against `~/rel003/mock_gitlab.py`, a mock that reproduces the exact auth split (API honors `PRIVATE-TOKEN`; `/uploads/` and `/-/releases/.../downloads/` ignore it and return login HTML; `/external/` is public). Suite: `python3 test_verify.py` → **11/11 passed**:

| # | Case | Result |
|---|------|--------|
| T1 | API presence + local-sha256 manifest, web routes untouched | exit 0, `problems=0` |
| T2 | `uploads/` & `downloads/` with `PRIVATE-TOKEN` → `text/html` login page (uploads final URL `~`) | trap confirmed |
| T3 | `link_type=external` → `application/octet-stream`, direct checksum OK | no auth needed |
| T4 | Unknown tag → exit 2 (404) with clear message | pass |
| T5 | Bad token → exit 3 (401) | pass |
| T6 | Expected asset absent from `assets.links[]` → exit 1 | pass |
| T7 | Local dist file missing → exit 4 | pass |
| T8 | Tampered external file → exit 6 checksum mismatch | pass |
| T9 | `?private_token=` query works on `/api/v4` (header preferred) | pass |
| T10 | Printed `expected_sha256` matches exact local hash | pass |
| T11 | `--check-web` probes all web routes, trap present → exit 0 | pass |

Representative run (verifier against mock):

```
release : group/proj @ v1.0.0  (API status 200)
  [human-verify] app-linux-x64.sha256.txt: expected_sha256=f2b5ea5046a5...c733
                url=http://<ip-address>:39363/uploads/ab12cd34ef56/... (upload-web-route)
  [human-verify] app-linux-x64.zip: expected_sha256=da6e01059060...aa61
                url=http://<ip-address>:39363/group/proj/-/releases/v1.0.0/downloads/... (upload-web-route)
  [direct-ok ] checksums-external.txt: sha256=ab6a6e8f2789...eca4 match=True
RESULT: PASS — ... uploads/ web routes NOT downloaded (human verifier holds expected_sha256).
```

**Live cross-check** (`curl gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-runner/releases`, HTTP 200): confirms real `assets.links[]` shape — `id`, `name`, `url` (external/CDN), `direct_asset_url` (the GitLab web route that needs a session cookie), `link_type`. The API enumeration in the fix matches production structure.

**Edge cases handled:** 401/404 with actionable messages (token needs `api` scope; 404 is ambiguous not-found vs no-access), empty `assets.links[]`, `--expect` subsetting, per-error exit codes, HTML-login detection by Content-Type + body markers, and the `--check-web` guard that flags if a web route *stops* requiring a cookie (so the "no downloads" policy can be safely relaxed later).

**Artifacts:** `~/rel003/verify_release.py` (the fix), `mock_gitlab.py` (trap reproducer), `test_verify.py` (evidence suite), `out.manifest.json` (sample hand-off manifest).

---

{"model": "deepseek-v4-flash", "problem_class": "gitlab-release-asset-download-auth", "result": "passed", "tests": 11}

Generated from the verified corpus · MIT licensedBack to the catalog