feat(admin): backup + restore archive (LC-95) #155
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/lc-95-backup-restore"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
LC-95: on-demand admin backup + stage-on-restart restore in one PR.
Backup
POST /admin/backupcalls SQLiteVACUUM INTOfor consistent snapshots ofauth.db,chat.db,settings.dbinto a uuid-named work dir while the server keeps serving.${DATA_DIR}/uploads/and${DATA_DIR}/avatars/, skipping.tmp/in-flight siblings.manifest.json(version + git_hash + per-file size + sha256).tokio::fs::File+tokio_util::io::ReaderStream; the tempfile isunlinked while the read handle is still open so the bytes free the moment the stream completes or aborts.Restore
POST /admin/restoremultipart upload (singlearchivefield, no body cap viaDefaultBodyLimit::disable()).backup::verify_archivere-hashes every entry against the manifest BEFORE touching the live data dir; refuses on size mismatch, sha256 mismatch, or version mismatch.backup::stage_extractwrites to${DATA_DIR}.staged-restore/and rejects any archive path containing..or starting with/.${DATA_DIR}/.restore-pendingsignals the next startup.backup::apply_pending_restoreruns inmain.rsBEFORE any pool opens: renames live data dir aside to${DATA_DIR}.replaced-{ts}and renames staged into place. Orphan markers (staged dir missing) self-clear with a warning.Locked design choices
Test plan
just check(both feature builds, clippy -D warnings, fmt --check).just testandjust test-saasboth green; newroutes_backup.rs(8 cases) covers build/verify round-trip, tampered-payload + version-mismatch refusal, swap-on-marker + orphan-marker cleanup, HTTP backup stream, HTTP restore stage + audit, non-admin 403./admin/backup-restore, restart container, upload same archive, confirm marker + staged sibling appear, restart, confirm old data is preserved under.replaced-{ts}.Limitations / follow-ups
VACUUM INTOsilently no-ops against ansqlite::memory:source via sqlx (every pool connection is its own DB); test pools are file-backed under a per-test tempdir to work around it. Documented inline next to the helper. Production is unaffected (file-backed pools).db::data_dir()helper went from private to public so the admin routes can locate the staging path / marker without re-implementing the env-var + default chain.On-demand admin backup and stage-on-restart restore in one PR. Backup: `POST /admin/backup` calls SQLite's `VACUUM INTO` to snapshot the three databases into a uuid-named work dir, walks `uploads/` + `avatars/` (skipping the in-flight `.tmp` siblings), writes the lot into a zip with a `manifest.json` listing every entry's size + sha256, and streams the archive to the admin. The tempfile is unlinked while the response handle is still open so the bytes vanish from disk the moment the client finishes (or aborts). Restore: `POST /admin/restore` accepts a multipart upload (single `archive` field, no body cap), validates the manifest version + every per-file sha256 BEFORE touching the live data dir, extracts into a sibling `${DATA_DIR}.staged-restore/`, and drops a `.restore-pending` marker file. Returns the admin to the page with a banner instructing them to restart. On the next startup `backup::apply_pending_restore` (called from main.rs before any pool opens) atomically renames the live data dir aside to `${DATA_DIR}.replaced-{ts}` and renames the staged dir into place. A marker without a staged sibling (corrupted state from a prior failed restore) logs a warning and self-clears. Refusals: archive built on a different lets-chat version (exact-match for first cut; semver-loosening is a follow-up); any size or sha256 drift between manifest + payload; any path containing `..` or starting with `/` so a malicious archive can't write outside the staging dir. Encryption is intentionally out of scope per the locked design choice; the archive is a plain zip. Operators who need at-rest encryption pipe the download through their own tool. Scheduled backups + remote targets land in a follow-up. Tests: 8-case `routes_backup.rs` covering the build-and-verify round trip, mismatched-version refusal, tampered-payload refusal, swap-on-marker, orphan-marker cleanup, HTTP backup stream, HTTP restore stage + audit, non-admin 403. The HTTP tests are `#[cfg(feature = "standalone")]` (admin router is standalone-only). Test pools are file-backed under a per-test subdir because `VACUUM INTO` silently no-ops against the default sqlx in-memory pool (multiple connections, each its own DB) - documented inline next to the helper. Adds the `zip` crate (single new dep) and a `Backup` tab in the admin layout. The `db::data_dir()` helper goes from private to pub so the route can locate the staging path without re-implementing the env-var chain.- `build_archive` splits its work between the async path (`VACUUM INTO` via sqlx) and a `spawn_blocking` task for the zip build. A multi-GB uploads tree no longer parks a tokio worker for the duration of the snapshot. `verify_archive` and `stage_extract` stay synchronous; the `post_restore` route wraps both in `spawn_blocking`. - `stream_file_into_zip` replaces `write_file_into_zip`: chunked 64 KiB reads + on-the-fly sha256 + direct write into the zip writer. Peak memory is bounded by the buffer size instead of growing with the file. SQLite snapshots that are GBs in size no longer pull into a `Vec<u8>` before encoding. - `VACUUM INTO` goes back to `sqlx::raw_sql` (was temporarily switched to `sqlx::query` during debugging). - `apply_pending_restore` now wraps `std::fs::rename` with `cross_device_aware_rename`. EXDEV (data dir and parent on different mounts) gets an actionable error message instead of an opaque "Invalid cross-device link". - Version gate loosens from exact match to same-major-minor: a `v0.1.0 -> v0.1.1` patch upgrade restores fine, only `v0.1.x -> v0.2.x` blocks. Comment + helper `semver_major_minor`. - `post_backup` audit log: drops the tempfile path (system internal, no admin value) in favor of `"{N} files, {size} bytes"`. The log fires after the file is sized and ready to stream, matching what actually got built. - `post_restore` body cap: `DefaultBodyLimit::max(10 GiB)` instead of `disable()`. Admin-only route so the threat model is narrow, but unlimited would let a compromised admin (or typo'd file picker) fill the disk before validation runs. - `admin/backup_restore.html` gains a doc note about `.replaced-{ts}` accumulation. No auto-purge; admin removes them manually once a restore is confirmed healthy. - `Cargo.toml` pinning comment on `zip = "2"` explaining why we're not on v3+.