fix(build): serialize cargo registry cache mount to fix concurrent unpack race #12

Merged

David merged 1 commit from fix/cargo-registry-cache-race into main

2026-06-14 21:58:45 +02:00

David commented

2026-06-14 21:56:03 +02:00

Owner

Problem

build-binary-static.yml (and its glibc / windows siblings) fail non-deterministically with failed to unpack package 'nu-ansi-term v0.50.3' ... failed to open .cargo-ok ... File exists (os error 17) (YTMCP-11, run #42).

Root cause is a race on the shared cargo registry cache mount, not stale source. The three build-binary* workflows run concurrently on one runner, and each mounts --mount=type=cache,target=/usr/local/cargo/registry with the default sharing=shared. Two cargo processes extracting the same crate at the same time race on creating the extracted source dir and its .cargo-ok marker, so one fails with File exists.

Fix

Add sharing=locked to every registry cache mount across all three Dockerfiles. BuildKit then serializes access to that mount across concurrent builds, so only one cargo extracts at a time. The download cache is still shared (deps fetched once); only the extraction step is mutually excluded, so there is no download-cost regression.

While in these files, replaced the force-flagged removals with non-force equivalents per the repo's no-force-flags rule: rm -rf src -> rm -r src in all three priming steps, and rm -rf /var/lib/apt/lists/* -> rm -r /var/lib/apt/lists/* in the glibc runtime stage. Both targets always exist at those points, so removal without -f succeeds; a failure there now surfaces a real bug instead of being silently ignored.

Note for the runner

The corrupt cache left by the original failed run still needs a one-time prune on the runner (e.g. docker buildx prune --filter type=exec.cachemount). This source change prevents the race from recurring; it does not retroactively clean the already-corrupt mount. The YTMCP-11 triage classified this as human_action-only, which is half right: the human prune clears the existing corruption, and this change stops it coming back.

#YTMCP-11

## Problem `build-binary-static.yml` (and its glibc / windows siblings) fail non-deterministically with `failed to unpack package 'nu-ansi-term v0.50.3' ... failed to open .cargo-ok ... File exists (os error 17)` (YTMCP-11, run #42). Root cause is a race on the shared cargo registry cache mount, not stale source. The three `build-binary*` workflows run concurrently on one runner, and each mounts `--mount=type=cache,target=/usr/local/cargo/registry` with the default `sharing=shared`. Two `cargo` processes extracting the same crate at the same time race on creating the extracted source dir and its `.cargo-ok` marker, so one fails with `File exists`. ## Fix Add `sharing=locked` to every registry cache mount across all three Dockerfiles. BuildKit then serializes access to that mount across concurrent builds, so only one cargo extracts at a time. The download cache is still shared (deps fetched once); only the extraction step is mutually excluded, so there is no download-cost regression. While in these files, replaced the force-flagged removals with non-force equivalents per the repo's no-force-flags rule: `rm -rf src` -> `rm -r src` in all three priming steps, and `rm -rf /var/lib/apt/lists/*` -> `rm -r /var/lib/apt/lists/*` in the glibc runtime stage. Both targets always exist at those points, so removal without `-f` succeeds; a failure there now surfaces a real bug instead of being silently ignored. ## Note for the runner The corrupt cache left by the original failed run still needs a one-time prune on the runner (e.g. `docker buildx prune --filter type=exec.cachemount`). This source change prevents the race from recurring; it does not retroactively clean the already-corrupt mount. The YTMCP-11 triage classified this as `human_action`-only, which is half right: the human prune clears the existing corruption, and this change stops it coming back. #YTMCP-11

David added 1 commit

2026-06-14 21:56:03 +02:00

fix(build): serialize cargo registry cache mount, drop rm -rf

Check / fmt + clippy + build + tests (pull_request) Successful in 23s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

f5be78dcb8

The three build-binary workflows (glibc, static musl, windows-gnu) run concurrently on one shared runner, all mounting the cargo registry cache at the same target with the default sharing=shared. Two cargo processes extracting the same crate at once race on creating the source dir and its .cargo-ok marker, producing "failed to unpack ... File exists (os error 17)" and a non-deterministic build failure. Add sharing=locked to every registry cache mount so registry access is serialized across concurrent builds; the extracted dep cache is still shared (downloaded once), only the extraction step is mutually excluded.

Also replace the force-flagged rm (rm -rf src priming-cleanup in all three Dockerfiles, plus rm -rf /var/lib/apt/lists/* in the glibc runtime stage) with rm -r. The dummy src dir and the apt lists dir always exist at those points, so removal without -f succeeds and a failure there now surfaces a real bug instead of being swallowed.

The corrupt cache left by the original failed run still needs a one-time prune on the runner; this change prevents the race from recurring.

#YTMCP-11