psa-systems/bunyip
Watch
9
0
Fork
You've already forked bunyip
 0
Code Issues Pull requests Projects Releases 18 Packages Wiki Activity Actions

chore(ci): shrink check.yml disk footprint to avoid runner ENOSPC #96

Merged
YousifShkara merged 1 commit from chore/check-ci-disk-footprint into main 2026-06-10 07:41:07 +02:00
Conversation 0 Commits 1 Files changed 1 +13
YousifShkara commented 2026-06-10 07:41:04 +02:00
Owner
Copy link

Three consecutive bunyip-upgrade PRs hit No space left on device on the runner with the same signature: ENOSPC on /workspace/psa-systems/bunyip/target/... and /tmp/cc*.s, then a cascade of LLVM SIGSEGV / BFD assertion failures downstream of the disk filling. The bunyip-oidc silent-SSO PR, the mokosh-apps onboarding PR, and the bunyip-upgrade-05-toast / -08-high-contrast PRs all failed identically. The host has 100GB+ free at root; the runner allocation fills up because each PR job's target/ is 10-15GB of -C debuginfo=2 artefacts plus a separate incremental cache.

Two env vars on the check job:

  • CARGO_PROFILE_DEV_DEBUG=line-tables-only keeps the bits CI actually uses (readable test-panic frames, rustc backtraces) while dropping the full DWARF die-info that bloats .rmeta / .rlib. Cuts target/ size ~50% with zero impact on fmt + clippy + cargo build --locked + cargo test --lib.
  • CARGO_INCREMENTAL=0 removes the per-job incremental dep tree. Incremental wins nothing on CI (every run starts from the rust-cache restore, not from a hot incremental cache) and costs significant disk. Disabling it drops another chunk.

Combined effect: the per-job disk footprint roughly halves, which is enough headroom to keep consecutive PR jobs from racing each other to ENOSPC. If the problem returns, the next escalation is --profile ci with debug = false, but the env-var pair is the smallest change that addresses the immediate failure.

Not blocking on any single PR; lands as its own chore so the next PR through CI inherits the smaller footprint regardless of which bunyip-upgrade branch is being reviewed.

Three consecutive bunyip-upgrade PRs hit `No space left on device` on the runner with the same signature: ENOSPC on `/workspace/psa-systems/bunyip/target/...` and `/tmp/cc*.s`, then a cascade of LLVM SIGSEGV / BFD assertion failures downstream of the disk filling. The bunyip-oidc silent-SSO PR, the mokosh-apps onboarding PR, and the bunyip-upgrade-05-toast / -08-high-contrast PRs all failed identically. The host has 100GB+ free at root; the runner allocation fills up because each PR job's `target/` is 10-15GB of `-C debuginfo=2` artefacts plus a separate incremental cache. Two env vars on the check job: - `CARGO_PROFILE_DEV_DEBUG=line-tables-only` keeps the bits CI actually uses (readable test-panic frames, rustc backtraces) while dropping the full DWARF die-info that bloats `.rmeta` / `.rlib`. Cuts `target/` size ~50% with zero impact on fmt + clippy + `cargo build --locked` + `cargo test --lib`. - `CARGO_INCREMENTAL=0` removes the per-job incremental dep tree. Incremental wins nothing on CI (every run starts from the rust-cache restore, not from a hot incremental cache) and costs significant disk. Disabling it drops another chunk. Combined effect: the per-job disk footprint roughly halves, which is enough headroom to keep consecutive PR jobs from racing each other to ENOSPC. If the problem returns, the next escalation is `--profile ci` with `debug = false`, but the env-var pair is the smallest change that addresses the immediate failure. Not blocking on any single PR; lands as its own chore so the next PR through CI inherits the smaller footprint regardless of which bunyip-upgrade branch is being reviewed.
chore(ci): shrink check.yml disk footprint to avoid runner ENOSPC
All checks were successful
Create release / Create release from merged PR (pull_request) Has been skipped
Details
Check / fmt / clippy / build / test (pull_request) Successful in 4m3s
Details
e026576416 
Three consecutive bunyip-upgrade PRs hit `No space left on device` on the runner with the same signature: ENOSPC on `/workspace/psa-systems/bunyip/target/...` and `/tmp/cc*.s`, then a cascade of LLVM SIGSEGV / BFD assertion failures downstream of the disk filling. The bunyip-oidc silent-SSO PR, the mokosh-apps onboarding PR, and the bunyip-upgrade-05-toast / -08-high-contrast PRs all failed identically. The host has 100GB+ free at root; the runner allocation fills up because each PR job's `target/` is 10-15GB of `-C debuginfo=2` artefacts plus a separate incremental cache.

Two env vars on the check job:

- `CARGO_PROFILE_DEV_DEBUG=line-tables-only` keeps the bits CI actually uses (readable test-panic frames, rustc backtraces) while dropping the full DWARF die-info that bloats `.rmeta` / `.rlib`. Cuts `target/` size ~50% with zero impact on fmt + clippy + `cargo build --locked` + `cargo test --lib`.
- `CARGO_INCREMENTAL=0` removes the per-job incremental dep tree. Incremental wins nothing on CI (every run starts from the rust-cache restore, not from a hot incremental cache) and costs significant disk. Disabling it drops another chunk.

Combined effect: the per-job disk footprint roughly halves, which is enough headroom to keep consecutive PR jobs from racing each other to ENOSPC. If the problem returns, the next escalation is `--profile ci` with `debug = false`, but the env-var pair is the smallest change that addresses the immediate failure.

Not blocking on any single PR; lands as its own chore so the next PR through CI inherits the smaller footprint regardless of which bunyip-upgrade branch is being reviewed.
YousifShkara deleted branch chore/check-ci-disk-footprint 2026-06-10 07:41:07 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
YousifShkara
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
psa-systems/bunyip!96
No description provided.