feat(data): admin-only tenant data export endpoint (PMS-647) #438

Merged

nrupard merged 1 commit from feat/PMS-647-data-export into main

2026-07-09 20:55:48 +02:00

nrupard commented

2026-07-09 20:24:52 +02:00

Owner

First slice of PMS-646 (tenant data import/export via the admin UI): the backend export endpoint.

What

GET /api/v1/data/export (admin-only) returns a single versioned JSON snapshot of the caller's tenant business data as a browser download, and audits it as an export action.

How

New src/modules/data_transfer module, schema-driven (no per-table code):

Reads every table that carries a tenant_id from information_schema (auto-includes new business tables as migrations add them - no hand-maintained list to drift), ordered for a deterministic file.
Excludes a fixed set of tables entirely (EXCLUDE_TABLES): integration secrets (payment_gateway_configs, credential_vault, api_keys, rmm_connections), auth identity/session/one-shot-token tables, and the audit/log tables.
Redacts secret COLUMNS from every remaining row (SECRET_COLUMN_SUBSTRINGS: encrypted, password_hash, _secret, mfa_secret, api_key, api_secret, private_key) - defense-in-depth for secrets that live inside otherwise-business tables (e.g. contacts.portal_password_hash, email_mailboxes.smtp_password_encrypted).
Dumps each table with row_to_json(t) ... WHERE tenant_id = $1 under begin_with_tenant (RLS) - tenant-scoped on both belts. Table names come from the catalog and are additionally guarded by an identifier check before interpolation.
Emits AuditAction::Export (the variant was already defined, previously unused), then returns the envelope (schema_version, tenant_id, included_tables, excluded_tables, redacted_column_patterns, notes, entities) as an application/json attachment download.

Admin-gated (RequireAuth + RequireAdmin), mounted on the /api/v1 PSA router next to settings/audit.

Scope / follow-ups (siblings under PMS-646)

Import (wipe-and-replace + FK id remap) and the mokosh-apps Import/Export UI are separate children.
Blob payloads (attachments/files) are metadata-only here; binary bytes are a follow-up (recorded in the envelope notes).

Tests

Unit tests on the security-critical redaction logic: secret columns are dropped and benign ones kept, every known secret column (from the migrations) is caught, and the identifier guard rejects non-identifiers. The endpoint-level round-trip (tenant scoping, full no-secret-substring scan of the output) is integration-level and lands with the import sibling.

#PMS-647

First slice of PMS-646 (tenant data import/export via the admin UI): the backend export endpoint. ## What `GET /api/v1/data/export` (admin-only) returns a single versioned JSON snapshot of the caller's tenant business data as a browser download, and audits it as an `export` action. ## How New `src/modules/data_transfer` module, schema-driven (no per-table code): - Reads every table that carries a `tenant_id` from `information_schema` (auto-includes new business tables as migrations add them - no hand-maintained list to drift), ordered for a deterministic file. - Excludes a fixed set of tables entirely (`EXCLUDE_TABLES`): integration secrets (`payment_gateway_configs`, `credential_vault`, `api_keys`, `rmm_connections`), auth identity/session/one-shot-token tables, and the audit/log tables. - Redacts secret COLUMNS from every remaining row (`SECRET_COLUMN_SUBSTRINGS`: `encrypted`, `password_hash`, `_secret`, `mfa_secret`, `api_key`, `api_secret`, `private_key`) - defense-in-depth for secrets that live inside otherwise-business tables (e.g. `contacts.portal_password_hash`, `email_mailboxes.smtp_password_encrypted`). - Dumps each table with `row_to_json(t) ... WHERE tenant_id = $1` under `begin_with_tenant` (RLS) - tenant-scoped on both belts. Table names come from the catalog and are additionally guarded by an identifier check before interpolation. - Emits `AuditAction::Export` (the variant was already defined, previously unused), then returns the envelope (`schema_version`, `tenant_id`, `included_tables`, `excluded_tables`, `redacted_column_patterns`, `notes`, `entities`) as an `application/json` attachment download. Admin-gated (`RequireAuth` + `RequireAdmin`), mounted on the `/api/v1` PSA router next to settings/audit. ## Scope / follow-ups (siblings under PMS-646) - Import (wipe-and-replace + FK id remap) and the mokosh-apps Import/Export UI are separate children. - Blob payloads (attachments/files) are metadata-only here; binary bytes are a follow-up (recorded in the envelope `notes`). ## Tests Unit tests on the security-critical redaction logic: secret columns are dropped and benign ones kept, every known secret column (from the migrations) is caught, and the identifier guard rejects non-identifiers. The endpoint-level round-trip (tenant scoping, full no-secret-substring scan of the output) is integration-level and lands with the import sibling. #PMS-647

nrupard added 1 commit

2026-07-09 20:24:52 +02:00

feat(data): admin-only tenant data export endpoint

E2E / Playwright against staging (pull_request) Successful in 1m22s

Details

Check / fmt + clippy + build + tests (pull_request) Successful in 3m2s

Details

Integration / integration tests (pull_request) Successful in 10m18s

Details

Create release / Gate (release-branch merges only) (pull_request) Successful in 1s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

25a583e4d2

First slice of PMS-646. GET /api/v1/data/export returns a single versioned JSON snapshot of the caller's tenant business data as a browser download, audited as an export action.

Schema-driven generic dump (no per-table code): reads every table carrying a tenant_id from information_schema (so new business tables are picked up automatically), minus an explicit exclude set (integration secrets - payment_gateway_configs/credential_vault/api_keys/rmm_connections; auth identity/session/one-shot-token tables; audit/log tables). Every remaining row has secret-looking columns dropped (encrypted/password_hash/_secret/mfa_secret/api_key/api_secret/private_key) as defense-in-depth for secrets that live inside otherwise-business tables (e.g. contacts.portal_password_hash). Each table is dumped with row_to_json ... WHERE tenant_id = $1 under begin_with_tenant, tenant-scoped on both RLS and the explicit filter, and catalog table names are identifier-guarded before interpolation. Admin-gated (RequireAuth + RequireAdmin); emits the previously-unused AuditAction::Export.

Import (wipe-and-replace + FK remap) and the mokosh-apps UI are sibling children under PMS-646. Blob payloads are metadata-only for now (noted in the envelope).

#PMS-647