feat(admin): surface email-ingress / IMAP-poll / retention-sweep health on /admin/settings (LC-207-OBSERVABILITY) #281

Merged

longjacksonle merged 1 commit from feat/lc-207-observability-ingress-sweep-imap into main

2026-05-31 04:31:00 +02:00

longjacksonle commented

2026-05-31 04:30:38 +02:00

Owner

Closes #278 (LC-207-OBSERVABILITY). Three background subsystems had a tracing log line as their sole operator-facing diagnostic. This persists runtime status for each and surfaces it read-only on the existing /admin/settings page, mirroring the LC-207 bridge-avatar diagnostic precedent, so an operator can self-serve "is ingress alive / why didn't my email post? / did the destructive sweep run?" without container-log access.

What lands

IMAP poll health (O3). spawn_email_poll writes a settings.db singleton (imap_poll_status, migration settings/0008) every tick: last poll, last success, consecutive-failure counter, last error, and last-tick fetched/posted/dropped. A failure increments the counter and preserves the last-success time; a success resets both and clears the error.

Email-ingress drops (O1). The three drop sites in poll_once (FETCH-fail, oversize, per-message drop) record to a chat.db log (email_ingress_drops, migration chat/0058): structured DropReason, the IMAP UID, and a bounded (200-char) non-body diagnostic detail only. It never stores the message body, subject, or any correspondent address. Rows are swept at 30 days by the hourly orphan sweeper alongside the dedup table. The page shows a by-reason summary (last 24h) plus the most recent 20 drops.

Retention sweep (O2). spawn_message_retention_sweeper records every completed tick (including zero-delete) to a chat.db singleton (retention_sweep_status, migration chat/0059): last run, last/total messages deleted, run count, last error. Failures record the error without advancing last_run_at (a failed tick is not a completed run). The page shows enabled/disabled (the LETS_CHAT_RETENTION_SWEEP_ENABLED gate) plus the run snapshot, so an operator who enabled the destructive sweep can confirm it ran and how much it deleted.

Notes

All status writes are best-effort: a write failure is logged and never blocks the poll loop, a message drop, or the sweep.
New admin surface is reached through the existing #[cfg(feature = "standalone")] /admin/settings route, so saas is unaffected. i18n keys added to en + es.
Push-notification delivery was the one log-only subsystem deliberately excluded (it self-heals by auto-pruning 410-Gone subscriptions); the #278 scope notes this.
No [operator-action]: this is additive observability with no new env var, config, or required operator step.

Validation

just test and just test-saas green. just check clean: server standalone + saas clippy (-D warnings, --all-targets), desktop compile, cargo fmt --check. New tests: db_observability_status (round-trips for all three modules - failure-counter climb + recovery, drop recent/counts/sweep/truncation, run accumulation + error-without-run) and routes_admin_observability (standalone-gated render: seeded health + drops appear, placeholders when empty, retention disabled-hint names the env var). migration_enclaves hand-rolled chat array updated for the two new chat migrations (CLAUDE.md category-2 drift).

🤖 Generated with Claude Code

Closes #278 (LC-207-OBSERVABILITY). Three background subsystems had a `tracing` log line as their sole operator-facing diagnostic. This persists runtime status for each and surfaces it read-only on the existing `/admin/settings` page, mirroring the LC-207 bridge-avatar diagnostic precedent, so an operator can self-serve "is ingress alive / why didn't my email post? / did the destructive sweep run?" without container-log access. ## What lands **IMAP poll health** (`O3`). `spawn_email_poll` writes a `settings.db` singleton (`imap_poll_status`, migration `settings/0008`) every tick: last poll, last success, consecutive-failure counter, last error, and last-tick fetched/posted/dropped. A failure increments the counter and preserves the last-success time; a success resets both and clears the error. **Email-ingress drops** (`O1`). The three drop sites in `poll_once` (FETCH-fail, oversize, per-message drop) record to a `chat.db` log (`email_ingress_drops`, migration `chat/0058`): structured `DropReason`, the IMAP UID, and a bounded (200-char) non-body diagnostic `detail` only. It never stores the message body, subject, or any correspondent address. Rows are swept at 30 days by the hourly orphan sweeper alongside the dedup table. The page shows a by-reason summary (last 24h) plus the most recent 20 drops. **Retention sweep** (`O2`). `spawn_message_retention_sweeper` records every completed tick (including zero-delete) to a `chat.db` singleton (`retention_sweep_status`, migration `chat/0059`): last run, last/total messages deleted, run count, last error. Failures record the error without advancing `last_run_at` (a failed tick is not a completed run). The page shows enabled/disabled (the `LETS_CHAT_RETENTION_SWEEP_ENABLED` gate) plus the run snapshot, so an operator who enabled the destructive sweep can confirm it ran and how much it deleted. ## Notes - All status writes are best-effort: a write failure is logged and never blocks the poll loop, a message drop, or the sweep. - New admin surface is reached through the existing `#[cfg(feature = "standalone")]` `/admin/settings` route, so saas is unaffected. i18n keys added to `en` + `es`. - Push-notification delivery was the one log-only subsystem deliberately excluded (it self-heals by auto-pruning 410-Gone subscriptions); the #278 scope notes this. - No `[operator-action]`: this is additive observability with no new env var, config, or required operator step. ## Validation `just test` and `just test-saas` green. `just check` clean: server standalone + saas clippy (`-D warnings`, `--all-targets`), desktop compile, `cargo fmt --check`. New tests: `db_observability_status` (round-trips for all three modules - failure-counter climb + recovery, drop recent/counts/sweep/truncation, run accumulation + error-without-run) and `routes_admin_observability` (standalone-gated render: seeded health + drops appear, placeholders when empty, retention disabled-hint names the env var). `migration_enclaves` hand-rolled chat array updated for the two new chat migrations (CLAUDE.md category-2 drift). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

longjacksonle added 1 commit

2026-05-31 04:30:38 +02:00

feat(admin): surface email-ingress, IMAP-poll, and retention-sweep health on /admin/settings (LC-207-OBSERVABILITY, #278 )

check-secrets / Nosey parker (push) Successful in 4s

Details

check-secrets / Kingfisher (push) Successful in 6s

Details

check-secrets / TruffleHog (push) Successful in 6s

Details

check-secrets / Kingfisher (pull_request) Successful in 4s

Details

check-secrets / TruffleHog (pull_request) Successful in 5s

Details

check-secrets / Nosey parker (pull_request) Successful in 6s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

Check / clippy + fmt + tests (pull_request) Successful in 3m32s

Details

e5cbca212d

Three background subsystems previously had a tracing log line as their sole operator-facing diagnostic. This adds persisted runtime status for each and surfaces it read-only on the existing admin settings page, mirroring the LC-207 bridge-avatar diagnostic precedent, so an operator can answer "is ingress alive / why didn't my email post? / did the destructive sweep run?" without container-log access.

IMAP poll health: spawn_email_poll writes a settings.db singleton (imap_poll_status, migration 0008) every tick - last poll, last success, consecutive-failure counter, last error, last-tick fetched/posted/dropped. A failure increments the counter and preserves last-success; a success resets both.

Email-ingress drops: the three drop sites in poll_once record to a chat.db log (email_ingress_drops, migration 0058) - structured DropReason, IMAP UID, and a bounded (200-char) non-body diagnostic detail only. Never the message body, subject, or correspondent address. Swept at 30 days by the hourly orphan sweeper alongside the dedup table. The settings page shows a by-reason summary (last 24h) plus the most recent 20 drops.

Retention sweep: spawn_message_retention_sweeper records every completed tick (including zero-delete) to a chat.db singleton (retention_sweep_status, migration 0059) - last run, last/total messages deleted, run count, last error - and records errors separately without advancing last_run_at. The page shows enabled/disabled (the LETS_CHAT_RETENTION_SWEEP_ENABLED gate) plus the run snapshot.

All status writes are best-effort: a write failure is logged and never blocks the poll loop, a message drop, or the sweep. i18n keys added to en + es. New admin surface is reached through the existing standalone-gated /admin/settings route, so saas is unaffected.

Tests: db_observability_status (round-trips for all three modules: failure-counter climb + recovery, drop recent/counts/sweep/truncation, run accumulation + error-without-run); routes_admin_observability (standalone-gated render test: seeded ingress health + drops appear, placeholders show when empty, retention disabled-hint names the env var). migration_enclaves array updated for the two new chat migrations. just test + just test-saas + just check all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>