feat(download): add dunite-download generic binary distribution engine #2
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/dunite-download"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What
New
dunite-downloadworkspace crate: the generic, storage-agnostic engine for proxying binary artifacts from an upstream Forgejo instance. Tracks BUNYIP-29 (subtask of the BUNYIP-28 distribution story).Why
Bunyip's binary download vertical is currently hand-rolled inside
bunyip-domain(ForgejoClient, ReleaseCache, DownloadCache, DownloadLimiter with a hard PgPool dependency), which violates the dunite/bunyip split: generic mechanism belongs here, persistence belongs to the consumer. This crate is that mechanism, extracted and generalized; the follow-up (BUNYIP-30) makes bunyip consume it and deletes the hand-rolled code.Design
Mirrors
dunite-ociexactly:store::AssetStore+store::DownloadCounterare the persistence traits the consumer implements against its own schema. No PgPool, no named tables, no domain types in the crate.ForgejoAssetClientspeaks both Forgejo artifact APIs, selected per lookup via theArtifactSourceenum: release attachments (/api/v1/repos/{owner}/{repo}/releases/tags/{tag}) and the generic package registry (/api/v1/packages/{owner}/generic/{package}/{version}/files). Download URLs are validated against the configured base host before the API token is forwarded (same token-safety rule asdunite-oci).ReleaseCacheis a moka TTL cache for artifact metadata, shared by both sources.DownloadCache<S: AssetStore>is the on-disk content-addressed asset cache: single-flight fetches viaArc<OnceCell>, SHA-256 hashing while streaming to a temp file, atomic rename, replaced-SHA orphan cleanup, async LRU eviction over the byte cap.DownloadLimiterenforces per-user concurrency in process and delegates the durable daily count toDownloadCounter, mirroringOciLimiter/PullCounter.Testing
23 unit tests, no DB and no network: wiremock for both upstream APIs (releases, generic packages, 404s, token-forwarding refusal on foreign hosts) and an in-memory
AssetStoreexercising cache hit/miss, stale-row refetch, replaced-SHA cleanup, invalidation, and LRU eviction.cargo test --workspace,cargo clippy -D warnings, andcargo fmt --checkare green (run inside the rust-builder-glibc 1.94 image).New workspace crate providing the storage-agnostic mechanism for proxying binary artifacts from an upstream Forgejo instance, extracted and generalized from the hand-rolled download code in bunyip-domain (BUNYIP-29). - ForgejoAssetClient speaks both artifact APIs, selected per lookup via the ArtifactSource enum: release attachments (/api/v1/repos/{owner}/{repo}/releases/tags/{tag}) and the generic package registry (/api/v1/packages/{owner}/generic/{package}/{version}/files), with download URLs always validated against the configured base host before the API token is forwarded. - ReleaseCache is a moka TTL cache for artifact metadata keyed by (scope id, version), shared by both sources. - DownloadCache<S: AssetStore> is the on-disk content-addressed asset cache: single-flight fetches, SHA-256 hashing while streaming to a temp file, atomic rename, replaced-SHA orphan cleanup, and async LRU eviction over the byte cap. Persistence goes through the AssetStore trait the consumer implements. - DownloadLimiter enforces per-user concurrency in process and delegates the durable daily count to the DownloadCounter trait, mirroring dunite-oci's OciLimiter/PullCounter split. - 23 unit tests: wiremock for both upstream APIs and an in-memory AssetStore exercising hit/miss, refetch, orphan cleanup, invalidation, and eviction. cargo test/clippy/fmt green on the workspace. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>