test: retire hand-rolled migration lists, grep-ban drift (LC-204) #266

Merged
longjacksonle merged 3 commits from fix/lc-204-retire-handrolled-migration-lists into main 2026-05-29 20:11:10 +02:00

Summary

Retires the hand-rolled include_str!("../migrations/...) migration lists in the integration test suite. Each pinned a stale migration subset that silently went out of date every time a migration was added, surfacing later as "no such column" 500s (hit twice in recent PRs: 0054/0055 in LC-78, 0056 in avatar-proxy; plus 0052/0053 that were never added to any list pre-LC-78). The cure: build every test pool via the drift-immune common::auth_pool() / common::chat_pool() / common::settings_pool() helpers, which run the full set via sqlx::migrate! so new migrations land in every test automatically.

Audit (gated the whole change)

The ticket estimated 18 files; the actual population is 35. Every file was classified before any conversion:

Class Count Disposition
ACCIDENTALLY-STALE 34 converted to common::*_pool()
LOAD-BEARING-PARTIAL 1 migration_enclaves.rs — stays hand-rolled

migration_enclaves.rs is the one file the audit existed to catch: it is a migration-BEHAVIOR test (asserts the 0009 enclaves data-migration's effect — General created, rooms moved, membership-backfill-is-separate, partial-unique-owner-index enforced), not a schema consumer. Converting it to the full sqlx::migrate! set would silently re-point its assertions from "0009 produced exactly this state" to "the full-migration end-state happens to have these properties" — a different test that could stay green while no longer catching a 0009 regression. It keeps its explicit list, carries a justifying comment, and is the sole grep-ban allow-list entry.

Conversion mechanics

Mechanical and uniform: each pool-builder's hand-rolled list → the matching common::*_pool().await, function name/signature preserved so call sites and all caller-side harness setup (create_user, backfill_general_membership, totp_enabled=1, room seeding) are untouched. Verified drop-in: the helpers are pure pool-builders (migrate + return SqlitePool), and harness setup always lives AFTER the pool build, never inside it.

  • Single-domain (24): one helper call.
  • Multi-domain (8): N helper calls (auth+chat, auth+settings, auth+chat+settings); AppState wiring unchanged.
  • db_mentions / db_enclave: finished their already-half-converted auth halves.

Diff signature: 116 insertions, 2269 deletions across 35 files — deleting lists, adding helper calls.

This is also the cleanup for the pre-existing rot: 0052/0053 (never listed), the accidental 0034-0040 gaps, and 0057 are now present in every test by construction.

Drift prevention (structural, not conventional)

tests/lc204_no_handrolled_migrations.rs walks tests/ and fails the build on any include_str!("../migrations/) outside the allow-list:

  • Anchored literal (opening quote + trailing slash) so it can't false-positive on a sibling dir like migrations_test_data/.
  • Full-path allow-list (tests/migration_enclaves.rs), not basename — a future tests/foo/migration_enclaves.rs can't inherit the exemption.
  • Sanity meta-test asserts every allow-list entry exists on disk AND still contains the pattern, so the exemption can't silently defang (same shape as LC-152's grep-ban sanity test).
  • The allow-list IS the load-bearing-partial registry: an entry is a DECLARATION ("this file is a migration-behavior test"), not a bypass. A future partial-list file gets added here WITH a justifying comment — the ban forces the justification to be explicit.

Item-5 triage (newly-surfaced failures)

Converting partial→full runs every previously-skipped migration, which can surface breakage the stale lists were hiding. Triage question per failure: does fixing it change what the test ASSERTS (load-bearing → split to follow-up) or only what it FEEDS IN (test-data fix → in-PR commit)?

Result: zero genuine conversion failures. Both suite failures are pre-existing, confirmed identical on clean main:

Failure Verdict
lc77_webhook_render_fixture (4 tests) Pre-existing, unrelated. Renders HTML to golden fixtures; touches no DB/migrations; fails identically on clean main (fixtures drifted from a template edit). Out of LC-204 scope — separate finding, worth its own ticket to regenerate the fixtures.
routes_uploads::other_user_cannot_fetch_orphan_upload Pre-existing documented flake. Isolated binary 12/12 × 4 runs, single test 3/3, controlled full-suite 0/4 flake rate; only fails under heavy cross-binary contention. CLAUDE.md documents routes_uploads as the known concurrent-load flake, explicitly out of "make tests pass" scope. The early failures coincided with concurrent baseline-check builds inflating machine load.

No test-data fixes were needed and no hidden load-bearing-partials surfaced beyond migration_enclaves.rs.

Test plan

  • ./dev/cargo check -p lets-chat-server --tests clean; exactly one hand-rolled list remains (migration_enclaves.rs, the deliberate exemption).
  • just test (standalone): 123 binaries, only the 2 pre-existing failures above (both confirmed on clean main).
  • just test-saas: 59 binaries (standalone-gated files correctly skip), only the pre-existing lc77 fixture failure.
  • Grep-ban (lc204_no_handrolled_migrations): 2/2 pass — the ban + the sanity meta-test.
  • cargo fmt --all applied; cargo clippy --tests clean.

Discovered finding to file separately

lc77_webhook_render_fixture's 4 golden-fixture tests fail on clean main (not caused by this PR). The rendered webhook/email-inbox message HTML diverged from the committed fixtures — likely an un-regenerated fixture after a template edit. Needs FIXTURE_WRITE=1 regeneration (after confirming the new render is correct) under its own ticket.

## Summary Retires the hand-rolled `include_str!("../migrations/...)` migration lists in the integration test suite. Each pinned a stale migration subset that silently went out of date every time a migration was added, surfacing later as "no such column" 500s (hit twice in recent PRs: 0054/0055 in LC-78, 0056 in avatar-proxy; plus 0052/0053 that were never added to any list pre-LC-78). The cure: build every test pool via the drift-immune `common::auth_pool()` / `common::chat_pool()` / `common::settings_pool()` helpers, which run the full set via `sqlx::migrate!` so new migrations land in every test automatically. ## Audit (gated the whole change) The ticket estimated 18 files; the actual population is **35**. Every file was classified before any conversion: | Class | Count | Disposition | |---|---|---| | ACCIDENTALLY-STALE | 34 | converted to `common::*_pool()` | | LOAD-BEARING-PARTIAL | 1 | `migration_enclaves.rs` — stays hand-rolled | `migration_enclaves.rs` is the one file the audit existed to catch: it is a migration-BEHAVIOR test (asserts the 0009 enclaves data-migration's effect — General created, rooms moved, membership-backfill-is-separate, partial-unique-owner-index enforced), not a schema consumer. Converting it to the full `sqlx::migrate!` set would silently re-point its assertions from "0009 produced exactly this state" to "the full-migration end-state happens to have these properties" — a different test that could stay green while no longer catching a 0009 regression. It keeps its explicit list, carries a justifying comment, and is the sole grep-ban allow-list entry. ## Conversion mechanics Mechanical and uniform: each pool-builder's hand-rolled list → the matching `common::*_pool().await`, function name/signature preserved so call sites and all caller-side harness setup (`create_user`, `backfill_general_membership`, `totp_enabled=1`, room seeding) are untouched. Verified drop-in: the helpers are pure pool-builders (migrate + return `SqlitePool`), and harness setup always lives AFTER the pool build, never inside it. - **Single-domain (24):** one helper call. - **Multi-domain (8):** N helper calls (auth+chat, auth+settings, auth+chat+settings); `AppState` wiring unchanged. - **`db_mentions` / `db_enclave`:** finished their already-half-converted auth halves. Diff signature: **116 insertions, 2269 deletions** across 35 files — deleting lists, adding helper calls. This is also the cleanup for the pre-existing rot: 0052/0053 (never listed), the accidental 0034-0040 gaps, and 0057 are now present in every test by construction. ## Drift prevention (structural, not conventional) `tests/lc204_no_handrolled_migrations.rs` walks `tests/` and fails the build on any `include_str!("../migrations/)` outside the allow-list: - **Anchored literal** (opening quote + trailing slash) so it can't false-positive on a sibling dir like `migrations_test_data/`. - **Full-path allow-list** (`tests/migration_enclaves.rs`), not basename — a future `tests/foo/migration_enclaves.rs` can't inherit the exemption. - **Sanity meta-test** asserts every allow-list entry exists on disk AND still contains the pattern, so the exemption can't silently defang (same shape as LC-152's grep-ban sanity test). - The allow-list IS the load-bearing-partial registry: an entry is a DECLARATION ("this file is a migration-behavior test"), not a bypass. A future partial-list file gets added here WITH a justifying comment — the ban forces the justification to be explicit. ## Item-5 triage (newly-surfaced failures) Converting partial→full runs every previously-skipped migration, which can surface breakage the stale lists were hiding. Triage question per failure: **does fixing it change what the test ASSERTS (load-bearing → split to follow-up) or only what it FEEDS IN (test-data fix → in-PR commit)?** Result: **zero genuine conversion failures.** Both suite failures are pre-existing, confirmed identical on clean `main`: | Failure | Verdict | |---|---| | `lc77_webhook_render_fixture` (4 tests) | Pre-existing, unrelated. Renders HTML to golden fixtures; touches no DB/migrations; fails identically on clean main (fixtures drifted from a template edit). **Out of LC-204 scope — separate finding, worth its own ticket to regenerate the fixtures.** | | `routes_uploads::other_user_cannot_fetch_orphan_upload` | Pre-existing documented flake. Isolated binary 12/12 × 4 runs, single test 3/3, controlled full-suite 0/4 flake rate; only fails under heavy cross-binary contention. CLAUDE.md documents routes_uploads as the known concurrent-load flake, explicitly out of "make tests pass" scope. The early failures coincided with concurrent baseline-check builds inflating machine load. | No test-data fixes were needed and no hidden load-bearing-partials surfaced beyond `migration_enclaves.rs`. ## Test plan - [x] `./dev/cargo check -p lets-chat-server --tests` clean; exactly one hand-rolled list remains (`migration_enclaves.rs`, the deliberate exemption). - [x] `just test` (standalone): 123 binaries, only the 2 pre-existing failures above (both confirmed on clean main). - [x] `just test-saas`: 59 binaries (standalone-gated files correctly skip), only the pre-existing `lc77` fixture failure. - [x] Grep-ban (`lc204_no_handrolled_migrations`): 2/2 pass — the ban + the sanity meta-test. - [x] `cargo fmt --all` applied; `cargo clippy --tests` clean. ## Discovered finding to file separately `lc77_webhook_render_fixture`'s 4 golden-fixture tests fail on clean `main` (not caused by this PR). The rendered webhook/email-inbox message HTML diverged from the committed fixtures — likely an un-regenerated fixture after a template edit. Needs `FIXTURE_WRITE=1` regeneration (after confirming the new render is correct) under its own ticket.
Each file's pool builder hand-rolled an include_str! migration list that silently went stale every time a migration was added, surfacing later as "no such column" 500s (hit twice in recent PRs: 0054/0055 in LC-78, 0056 in avatar-proxy; plus 0052/0053 never added pre-LC-78). Swap each for the drift-immune common::auth_pool()/chat_pool()/settings_pool() helpers, which run the full set via sqlx::migrate! so new migrations land automatically. Function names/signatures kept so call sites and caller-side harness setup are untouched. Multi-domain files call N helpers; db_mentions/db_enclave finish their already-half-converted auth halves. This is also the cleanup for the pre-existing rot: every silently-missing migration is now present by construction.

migration_enclaves.rs deliberately KEEPS its hand-rolled list (separate commit): it is a migration-behavior test, not a schema consumer.
test: grep-ban hand-rolled migration lists + pin migration_enclaves (LC-204)
Some checks failed
check-secrets / TruffleHog (push) Successful in 5s
check-secrets / Nosey parker (push) Successful in 3s
check-secrets / Kingfisher (push) Successful in 4s
check-secrets / Kingfisher (pull_request) Successful in 5s
check-secrets / Nosey parker (pull_request) Successful in 5s
check-secrets / TruffleHog (pull_request) Successful in 6s
Check / clippy + fmt + tests (pull_request) Has been cancelled
37b50669c7
Structural drift-prevention so the converted-away pattern can't return:
lc204_no_handrolled_migrations walks tests/ and fails on any
include_str!("../migrations/) outside the allow-list. Anchored literal
(opening quote + trailing slash) so it can't false-positive on a sibling
dir; full-path allow-list so a future tests/foo/migration_enclaves.rs
can't inherit the exemption by basename. A sanity meta-test asserts every
allow-list entry exists AND still contains the pattern, so the exemption
can't silently defang.

The allow-list is the load-bearing-partial registry: an entry is a
DECLARATION (this file is a migration-behavior test), not a bypass.
migration_enclaves.rs is the sole entry; it asserts the 0009 enclaves
data-migration's effect, so converting it to the full sqlx::migrate! set
would silently re-point its assertions. Carries a justifying comment.
Merge branch 'main' into fix/lc-204-retire-handrolled-migration-lists
All checks were successful
check-secrets / Nosey parker (pull_request) Successful in 6s
check-secrets / Kingfisher (pull_request) Successful in 9s
check-secrets / TruffleHog (pull_request) Successful in 9s
Create release / Create release from merged PR (pull_request) Has been skipped
check-secrets / Nosey parker (push) Successful in 9s
check-secrets / Kingfisher (push) Successful in 10s
check-secrets / TruffleHog (push) Successful in 10s
Check / clippy + fmt + tests (pull_request) Successful in 5m46s
d42ad50849
longjacksonle deleted branch fix/lc-204-retire-handrolled-migration-lists 2026-05-29 20:11:10 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
psa-systems/lets-chat!266
No description provided.