docs(email-ingress): operator guide + threat-model integration tests (LC-77) #199

Merged

longjacksonle merged 1 commit from feat/lc-77-docs into main 2026-05-25 21:08:26 +02:00

Conversation 0 Commits 1 Files changed 4 +541

longjacksonle commented 2026-05-25 20:34:05 +02:00

Owner

Copy link

Summary

Closer PR for the LC-77 v1 scope. With this merged, the email-ingress feature ships with an operator-facing deployment guide and a dedicated threat-model integration test suite that anchors the named invariants from the parent brainstorm.

What changed

• docs/email-ingress.md (new): operator deployment guide. Covers what the feature is (and isn't), the threat model in operator terms, the load-bearing header-precedence requirement (Delivered-To / X-Original-To / To / Cc), step-by-step setup, the full failure-log taxonomy as a diagnostic table, limits, the "Not supported (deferred)" section naming the four scoped-out followup tickets, plus privacy + security notes. ~250 lines. Designed so an operator hitting reason=address_no_match in the server logs can diagnose without reading code.

• tests/email_ingress_threat_model.rs (new): 8 dedicated tests that pin the named threat-model lines from the parent brainstorm in one place so the security review can find them. Each test docstring quotes the invariant it anchors. The same surfaces are covered piecemeal in email_ingress_process.rs and email_ingress_attachments.rs; this file consolidates the load-bearing assertions.

• README.md: one-line mention of email ingress under Integrations with a link to the operator docs.

• CLAUDE.md: new "Email ingress (LC-77)" section pointing at the operator docs + naming the spawn gate + the named link-filter-skip decision.

Threat-model tests anchored in this PR

Test	Threat-model line it pins
forged_from_still_posts_as_inbox_actor	Identity is the secret, not the From header.
raw_html_never_appears_in_stored_body	No <script / onerror= / <img / </script ever reaches the stored body.
unknown_secret_silent_drop_no_message_row	Unknown token drops silently; no bounce, no row.
loop_headers_drop_consistently	Auto-Submitted, Precedence: bulk/list/junk, X-Autoreply, X-Autorespond, List-Id all drop with a detail naming the matching header.
auto_submitted_no_does_not_drop_but_anything_else_does	RFC 3834's "no" opt-out is honored.
revoked_inbox_post_revoke_silent_drop_no_message_row	Drop is silent + no row after admin revoke.
token_at_wrong_domain_fails_to_resolve	The right token at the wrong domain MUST NOT match - cross-tenant safety.
address_no_match_drop_detail_carries_tried_addresses	Drop log's detail lists every address checked, so the operator can see what the resolver saw.

Plan position after this merge

PR #194 (commit 1, refactor)              MERGED
PR #195 (commit 2, schema)                MERGED
PR #196 (commit 3, IMAP poll + send)      MERGED
PR #197 (commit 4, admin lifecycle UI)    MERGED
PR #198 (commit 5, MIME + attachments)    MERGED
This PR (commit 6, docs + threat-model)   OPEN, awaiting review + merge

LC-77 v1 SCOPE COMPLETE after this PR merges.

Followups (not in scope)

Four tickets to file after this PR merges, all already named in docs/email-ingress.md's "Not supported (deferred)" section:

• LC-77-SMTP-SEAL: migrate SMTP password from plaintext settings.db to the same VAPID-sealed pattern this PR's IMAP password uses.
• LC-77-REPLY: reply-by-email, depends on a per-message notification email surface that doesn't exist yet.
• LC-77-MID-DEDUP: exactly-once dedup via Message-ID, defer until duplicates observed in practice.
• LC-77-DEAD-LETTER: optional dead-letter IMAP folder for poison messages, defer until operators ask for recoverability.

Test plan

• 8 new tests in email_ingress_threat_model.rs pass.
• Full server test suite green under both --features standalone and --no-default-features --features saas.
• cargo clippy --tests -- -D warnings clean.
• cargo fmt --check clean.
• Walked the operator guide on a fresh setup mentally; the header-precedence section matches what email_ingress::resolve::HEADER_ORDER actually checks, the failure-log table matches the DropReason enum field-for-field, the limits table matches the constants in parse.rs + poll.rs + attachments.rs.

## Summary Closer PR for the LC-77 v1 scope. With this merged, the email-ingress feature ships with an operator-facing deployment guide and a dedicated threat-model integration test suite that anchors the named invariants from the parent brainstorm. ## What changed - **`docs/email-ingress.md` (new)**: operator deployment guide. Covers what the feature is (and isn't), the threat model in operator terms, the load-bearing header-precedence requirement (`Delivered-To` / `X-Original-To` / `To` / `Cc`), step-by-step setup, the full failure-log taxonomy as a diagnostic table, limits, the "Not supported (deferred)" section naming the four scoped-out followup tickets, plus privacy + security notes. ~250 lines. Designed so an operator hitting `reason=address_no_match` in the server logs can diagnose without reading code. - **`tests/email_ingress_threat_model.rs` (new)**: 8 dedicated tests that pin the named threat-model lines from the parent brainstorm in one place so the security review can find them. Each test docstring quotes the invariant it anchors. The same surfaces are covered piecemeal in `email_ingress_process.rs` and `email_ingress_attachments.rs`; this file consolidates the load-bearing assertions. - **`README.md`**: one-line mention of email ingress under Integrations with a link to the operator docs. - **`CLAUDE.md`**: new "Email ingress (LC-77)" section pointing at the operator docs + naming the spawn gate + the named link-filter-skip decision. ## Threat-model tests anchored in this PR | Test | Threat-model line it pins | |---|---| | `forged_from_still_posts_as_inbox_actor` | Identity is the secret, not the From header. | | `raw_html_never_appears_in_stored_body` | No `<script` / `onerror=` / `<img` / `</script` ever reaches the stored body. | | `unknown_secret_silent_drop_no_message_row` | Unknown token drops silently; no bounce, no row. | | `loop_headers_drop_consistently` | Auto-Submitted, Precedence: bulk/list/junk, X-Autoreply, X-Autorespond, List-Id all drop with a detail naming the matching header. | | `auto_submitted_no_does_not_drop_but_anything_else_does` | RFC 3834's "no" opt-out is honored. | | `revoked_inbox_post_revoke_silent_drop_no_message_row` | Drop is silent + no row after admin revoke. | | `token_at_wrong_domain_fails_to_resolve` | The right token at the wrong domain MUST NOT match - cross-tenant safety. | | `address_no_match_drop_detail_carries_tried_addresses` | Drop log's detail lists every address checked, so the operator can see what the resolver saw. | ## Plan position after this merge ``` PR #194 (commit 1, refactor) MERGED PR #195 (commit 2, schema) MERGED PR #196 (commit 3, IMAP poll + send) MERGED PR #197 (commit 4, admin lifecycle UI) MERGED PR #198 (commit 5, MIME + attachments) MERGED This PR (commit 6, docs + threat-model) OPEN, awaiting review + merge LC-77 v1 SCOPE COMPLETE after this PR merges. ``` ## Followups (not in scope) Four tickets to file after this PR merges, all already named in `docs/email-ingress.md`'s "Not supported (deferred)" section: - **LC-77-SMTP-SEAL**: migrate SMTP password from plaintext `settings.db` to the same VAPID-sealed pattern this PR's IMAP password uses. - **LC-77-REPLY**: reply-by-email, depends on a per-message notification email surface that doesn't exist yet. - **LC-77-MID-DEDUP**: exactly-once dedup via Message-ID, defer until duplicates observed in practice. - **LC-77-DEAD-LETTER**: optional dead-letter IMAP folder for poison messages, defer until operators ask for recoverability. ## Test plan - [x] 8 new tests in `email_ingress_threat_model.rs` pass. - [x] Full server test suite green under both `--features standalone` and `--no-default-features --features saas`. - [x] `cargo clippy --tests -- -D warnings` clean. - [x] `cargo fmt --check` clean. - [x] Walked the operator guide on a fresh setup mentally; the header-precedence section matches what `email_ingress::resolve::HEADER_ORDER` actually checks, the failure-log table matches the `DropReason` enum field-for-field, the limits table matches the constants in `parse.rs` + `poll.rs` + `attachments.rs`.

longjacksonle added 1 commit 2026-05-25 20:34:05 +02:00

docs(email-ingress): operator guide + threat-model integration tests (LC-77) ... d242a9e862

Closes the LC-77 v1 scope.

docs/email-ingress.md: operator-facing deployment guide for the email-ingress feature shipped across PRs #194-#198. Covers:
- The threat model in operator terms (identity is the secret not From, magic-byte trust boundary, EXIF strip, no raw HTML, silent drops on unknown/revoked secrets, per-inbox rate limit, mailbox provider as first-line spam defense).
- The load-bearing header-precedence requirement: mail MUST arrive at the polled mailbox with the token in one of Delivered-To / X-Original-To / To / Cc, in that order. The IMAP-poll model's main operational gotcha; named explicitly so an operator hitting reason=address_no_match in the logs can diagnose the forwarder.
- Setup steps end-to-end (mailbox provision, LETS_CHAT_SECRET_KEY, admin IMAP settings, restart, per-room inbox create).
- Verification path: which logs to filter on, what reason=X means, how to diagnose each drop class.
- The full failure-log taxonomy as a table so a support conversation can lean on it.
- Limits table (poll interval, rate cap, sizes, MIME allowlist) with explicit "Adjustable?" column.
- "Not supported (deferred)" section with the four scoped-out followup tickets named: LC-77-REPLY, LC-77-DEAD-LETTER, LC-77-MID-DEDUP, LC-77-SMTP-SEAL.
- Privacy + security sections: EXIF strip, no From in stored chat, sealed IMAP password at rest, ingress address is a bearer secret.

tests/email_ingress_threat_model.rs (new): 8 dedicated tests that anchor the named threat-model lines from the parent brainstorm in one place so a security review can find them. Each test docstring quotes the invariant it pins:
- forged_from_still_posts_as_inbox_actor: identity is the secret, not From.
- raw_html_never_appears_in_stored_body: no <script / onerror= / <img / </script ever reaches the stored body even when the HTML payload contains them.
- unknown_secret_silent_drop_no_message_row: drop is silent (no bounce, no row) on unknown token.
- loop_headers_drop_consistently: walks Auto-Submitted (replied/generated), Precedence (bulk/list/junk), X-Autoreply, X-Autorespond, and List-Id; each must drop with a detail that names the matching header.
- auto_submitted_no_does_not_drop_but_anything_else_does: RFC 3834's "no" opt-out is honored.
- revoked_inbox_post_revoke_silent_drop_no_message_row: drop is silent + no row after admin revoke.
- token_at_wrong_domain_fails_to_resolve: the right token at the wrong domain MUST NOT match - the cross-tenant safety property.
- address_no_match_drop_detail_carries_tried_addresses: the drop log's detail must list every address checked so the operator can see what the resolver saw.

The same surfaces are already covered piecemeal in email_ingress_process.rs (commit 3 / PR #196) and email_ingress_attachments.rs (commit 5 / PR #198); this file consolidates the load-bearing assertions in one place so the security review and a future change touching the assertion both grapple with the documented invariant, not just see "this test broke".

README.md: one-line mention of email ingress under Integrations with a link to docs/email-ingress.md.

CLAUDE.md: new "Email ingress (LC-77)" section pointing at the operator docs + naming the spawn gate + the named link-filter-skip decision (anchored in routes::room::finalize_email_inbox_message_send).

Verification: 8 new tests pass. Full server test suite green under both default and saas. cargo clippy --tests -- -D warnings clean. cargo fmt --check clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

longjacksonle merged commit 3c83c9c75c into main 2026-05-25 21:08:26 +02:00

longjacksonle deleted branch feat/lc-77-docs 2026-05-25 21:08:26 +02:00

longjacksonle referenced this pull request from a commit 2026-05-25 21:08:28 +02:00

Merge pull request 'docs(email-ingress): operator guide + threat-model integration tests (LC-77)' (#199) from feat/lc-77-docs into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

psa-systems/lets-chat!199

No description provided.