feat(calls): meeting transcription - live captions + saved transcript (LC-393) #405
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/lc-393-call-transcription"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Implements Phase 1 of LC-393: meeting transcription for 1:1 DM calls - live captions during the call and a saved, reviewable transcript posted back into the DM.
The constraint that shaped it
Call media is peer-to-peer WebRTC; the server never sees the audio (
ws/events.rs,call.js). So transcription is captured client-side: each participant's browser transcribes its OWN mic with the Web Speech API and POSTs final segments. The happy consequence - speaker attribution is automatic (a segment's speaker is whoever's browser produced it), no diarization.What's here
Server (fully tested):
call_transcripts+transcript_segments(migration0066),db::transcriptsmodule (start/append/list/end + a disconnect-backstop closer). Segment text length-capped atMAX_SEGMENT_CHARS(stored + rendered as plain text).POST /call/{room}/transcript/start,POST /call/transcript/{id}/segment,POST /call/transcript/{id}/end(idempotent; posts the linked "transcript saved" notice exactly once),GET /transcripts/{id}(saved page).ChatEvent::{TranscriptStarted,Segment,Ended}rendered per-recipient like the call signals; control events to#lc-transcript-bus, caption lines OOB-appended straight into the visible#lc-caption-log.Consent (non-negotiable): transcription is opt-in and the moment it starts a "this call is being transcribed" banner is broadcast to BOTH parties for the duration. No silent recording.
Client:
transcribe.jsrunsSpeechRecognitionon the existing local mic, POSTs finals, and drives the banner/toggle; three tiny hooks incall.jspublish the active room and a call-ended signal. Feature-detectsSpeechRecognitionand disables the toggle with a localized note where unsupported (Firefox; Chrome works but routes audio to Google - noted in the ticket as a reason to add a self-hosted Whisper provider in Phase 3).Pluggable STT: the data model + UI are engine-agnostic, so the Phase 2 (enclave voice channels) and Phase 3 (self-hosted Whisper) work in the ticket drop in without schema/UI changes.
Testing
tests/transcripts.rs(4 tests, all green): member-start + outsider-403 + non-DM-404, segment persistence + ordering + outsider-403 + gated saved page, length cap, idempotent /end posting exactly one notice + late-segment drop.just check,just test,just test-saasall green (exit 0).just build-cssregenerated.Operator-visible
New tables via migration only (auto-applied, no operator action). No new env var. Phase 1 transcription uses the browser engine, so no server STT dependency is added. Not tagging
[operator-action].Out of scope (tracked in LC-393)
Phase 2 enclave voice-channel transcription; Phase 3 self-hosted Whisper provider; Phase 4 transcript search/export/summary.
Phase 1 of call transcription for 1:1 DM calls. Call media is peer-to-peer (the server never sees the audio), so each participant's browser transcribes its OWN mic with the Web Speech API and POSTs final segments; the server stores them, fans out attributed live captions to both parties, and on hangup saves the transcript and drops a linked notice in the DM. Speaker attribution is automatic because each browser only transcribes its own mic - no diarization. Server (fully tested): - New chat.db tables call_transcripts + transcript_segments (migration 0066), db module db::transcripts (start/append/list/end + a disconnect-backstop closer), segment text length-capped at MAX_SEGMENT_CHARS. - Routes (DM-member gated): POST /call/{room}/transcript/start, POST /call/transcript/{id}/segment, POST /call/transcript/{id}/end (idempotent; posts the "transcript saved" notice exactly once), GET /transcripts/{id} (saved-transcript page). - ChatEvent TranscriptStarted/Segment/Ended, rendered per-recipient like the call signals; control events into #lc-transcript-bus, caption lines OOB-appended straight into #lc-caption-log. - WS-disconnect backstop finalizes a session a hard drop left open (mirrors the LC-186 remote-control backstop). Consent: transcription is opt-in and the moment it starts a "this call is being transcribed" banner is broadcast to BOTH parties; no silent recording. Client: transcribe.js runs SpeechRecognition on the existing local mic, POSTs finals, and drives the banner/toggle; three small hooks in call.js publish the active room and a call-ended signal. Feature-detects SpeechRecognition and disables the toggle with a localized note where unsupported. STT source is pluggable: the data model + UI are engine-agnostic so a self-hosted Whisper provider (Phase 3) and enclave voice-channel transcription (Phase 2) drop in without schema/UI changes. See LC-393 for the phase plan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>