feat(voice): server-side voice-message transcription [operator-action] (LC-483) #459

Merged

longjacksonle merged 2 commits from feat/LC-483-voice-message-transcription into main 2026-06-27 19:42:54 +02:00

Conversation 0 Commits 2 Files changed 14 +270 -7

longjacksonle commented 2026-06-27 19:30:10 +02:00

Owner

Copy link

What

Server-side transcription of voice messages (LC-483, first shortlist item from the brainstorm epic LC-475). When the operator has configured LETS_CHAT_STT_*, an uploaded voice message is transcribed off the request path and the text is shown under the waveform. Reuses the OpenAI-compatible STT pipeline already shipped for call transcription, so this is mostly plumbing.

How

• Migration: chat/0068_voice_transcript.sql adds file_uploads.transcript (NULL = not transcribed / STT off / failed / not a voice message).
• DB: db::uploads carries transcript through the attachment read paths and adds a set_transcript setter; Attachment gains a transcript field.
• Transcription: routes::room::maybe_transcribe_voice_message is spawned after finalize_message_send, gated on stt_available() + a file_id. It re-checks the attachment is a voice message (audio mime + waveform), reads the bytes, transcribes, caps at MAX_VOICE_TRANSCRIPT_CHARS (8000), stores, and broadcasts. Idempotent (skips a row that already has a transcript); all failures are non-fatal and leave the message transcript-less.
• Live update: new ChatEvent::VoiceTranscribed { message_id, room_id } reuses the existing per-recipient render_edited_message arm (no new renderer), so the message re-renders in place once transcription completes. Classified alongside MessageRegrouped in ws_fragments::render_event.
• Render: transcript renders under the player in partials/attachment.html as escaped plain text using Tailwind token classes (light/dark parity). i18n key partials-voice-transcript added in en + es.

Tests

Three deterministic cases in transcripts.rs via the existing MockSttClient harness (now exposing AppState): voice message transcribed + attached, non-voice upload skipped, STT-disabled no-op. just check, just test, just test-saas all green (standalone + saas, clippy -D warnings).

Operator note

[operator-action] marker: no action required, but enabling LETS_CHAT_STT_URL now sends voice-message audio to that endpoint in addition to call audio - a load/cost consideration on a metered engine. Documented in the CLAUDE.md STT row and the stt module header.

Verify

just verify builds the release binary clean; the boot step fails only because the server-up compose env does not supply the mandatory LETS_CHAT_BUNYIP_SSO_* vars (the standalone build refuses to boot without them) - the same environmental failure as every PR in this series, not a regression.

## What Server-side transcription of voice messages (LC-483, first shortlist item from the brainstorm epic LC-475). When the operator has configured `LETS_CHAT_STT_*`, an uploaded voice message is transcribed off the request path and the text is shown under the waveform. Reuses the OpenAI-compatible STT pipeline already shipped for call transcription, so this is mostly plumbing. ## How - **Migration**: `chat/0068_voice_transcript.sql` adds `file_uploads.transcript` (NULL = not transcribed / STT off / failed / not a voice message). - **DB**: `db::uploads` carries `transcript` through the attachment read paths and adds a `set_transcript` setter; `Attachment` gains a `transcript` field. - **Transcription**: `routes::room::maybe_transcribe_voice_message` is spawned after `finalize_message_send`, gated on `stt_available()` + a `file_id`. It re-checks the attachment is a voice message (audio mime + waveform), reads the bytes, transcribes, caps at `MAX_VOICE_TRANSCRIPT_CHARS` (8000), stores, and broadcasts. Idempotent (skips a row that already has a transcript); all failures are non-fatal and leave the message transcript-less. - **Live update**: new `ChatEvent::VoiceTranscribed { message_id, room_id }` reuses the existing per-recipient `render_edited_message` arm (no new renderer), so the message re-renders in place once transcription completes. Classified alongside `MessageRegrouped` in `ws_fragments::render_event`. - **Render**: transcript renders under the player in `partials/attachment.html` as escaped plain text using Tailwind token classes (light/dark parity). i18n key `partials-voice-transcript` added in en + es. ## Tests Three deterministic cases in `transcripts.rs` via the existing `MockSttClient` harness (now exposing `AppState`): voice message transcribed + attached, non-voice upload skipped, STT-disabled no-op. `just check`, `just test`, `just test-saas` all green (standalone + saas, clippy `-D warnings`). ## Operator note `[operator-action]` marker: no action required, but enabling `LETS_CHAT_STT_URL` now sends voice-message audio to that endpoint in addition to call audio - a load/cost consideration on a metered engine. Documented in the CLAUDE.md STT row and the `stt` module header. ## Verify `just verify` builds the release binary clean; the boot step fails only because the server-up compose env does not supply the mandatory `LETS_CHAT_BUNYIP_SSO_*` vars (the standalone build refuses to boot without them) - the same environmental failure as every PR in this series, not a regression.

longjacksonle added 1 commit 2026-06-27 19:30:10 +02:00

feat(voice): transcribe voice messages via STT [operator-action] (LC-483) ... 8ac17eab75

When LETS_CHAT_STT_* is configured, an uploaded voice message (an audio attachment carrying a waveform) is now transcribed server-side off the request path, reusing the exact OpenAI-compatible STT pipeline already shipped for call transcription. The recognized text is stored on the upload row and rendered as escaped plain text under the waveform player; a new VoiceTranscribed WS event re-renders the message in place so the transcript appears live once it lands.

Implementation: new chat migration 0068 adds file_uploads.transcript; db::uploads carries it through the attachment read paths plus a set_transcript setter; routes::room::maybe_transcribe_voice_message (spawned after finalize_message_send, gated on stt_available + a file_id) re-checks the attachment is a voice message, reads the bytes, transcribes, caps the result at MAX_VOICE_TRANSCRIPT_CHARS, stores it, and broadcasts. The new ChatEvent::VoiceTranscribed reuses the per-recipient render_edited_message arm, so no new renderer is needed. Failures (file gone, engine down, empty/garbage result) are non-fatal and leave the message transcript-less; the helper is idempotent so a re-delivery never re-bills the engine.

Render uses Tailwind token classes (light/dark parity) rather than the voice player's hardcoded-light CSS. i18n key partials-voice-transcript added in en + es. Tests: three deterministic cases in transcripts.rs (voice transcribed + attached, non-voice skipped, STT-disabled no-op) via the existing MockSttClient harness, now exposing AppState.

Operator-Action: No action required. Be aware that enabling LETS_CHAT_STT_URL now sends voice-message audio to that endpoint in addition to call audio, which is a load/cost consideration on a metered STT engine.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

longjacksonle scheduled this pull request to auto merge when all checks succeed 2026-06-27 19:33:25 +02:00

longjacksonle scheduled this pull request to auto merge when all checks succeed 2026-06-27 19:35:33 +02:00

longjacksonle added 1 commit 2026-06-27 19:39:37 +02:00

style(test): rustfmt the LC-483 voice-transcript assertion ... c6b5dbd4c2

Wrap the multi-arg assert! the formatter expects. Fixes the CI fmt check that blocked merge; only whitespace, no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

longjacksonle merged commit ee22250c7c into main 2026-06-27 19:42:54 +02:00

longjacksonle deleted branch feat/LC-483-voice-message-transcription 2026-06-27 19:42:55 +02:00

longjacksonle referenced this pull request from a commit 2026-06-27 19:42:55 +02:00

Merge pull request 'feat(voice): server-side voice-message transcription [operator-action] (LC-483)' (#459) from feat/LC-483-voice-message-transcription into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

psa-systems/lets-chat!459

No description provided.