feat(calls): Phase 3 - self-hosted server-side STT provider (LC-393) #407
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/lc-393-phase3-server-stt"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Phase 3 of LC-393: an operator-selectable self-hosted server-side STT engine for call transcription. Phases 1-2 (browser Web Speech) are merged.
Architecture
Rather than bundle whisper.cpp (a large native dep + model file) into the single binary, lets-chat forwards audio to an operator-configured OpenAI-compatible
/v1/audio/transcriptionsendpoint (whisper.cpp server, faster-whisper, LocalAI, ...). The binary stays lean, the operator runs/chooses the engine - on localhost if they like. Benefits over the browser engine: browser-agnostic (works in Firefox/Safari, which lack SpeechRecognition) and keeps audio off third-party clouds (Chrome's Web Speech ships audio to Google).What's here
sttmodule (mirrors thepush_clientpattern):SttConfig::from_env(LETS_CHAT_STT_URLenables it;_API_KEY,_MODELoptional), anSttClienttrait withReqwestSttClient(multipart POST) +MockSttClient(tests).AppStategainsstt_client: Option<...>.POST /call/transcript/{id}/audio: participant-gated, forwards the clip to the STT endpoint, records the returned text as a segment + live caption via the sharedrecord_and_broadcastpath (same persistence/broadcast as the browser-textsegment). 400 when STT is unconfigured.http_client::outbound_trusted_posthelper that lives inside the LC-152 chokepoint module, so the grep-ban stays a single-file allowance (the existinglc152_no_raw_reqwest_in_srctest passes).GET /call/configadvertisessttServer;transcribe.jsreads it and, when server STT is on, captures fixed-lengthMediaRecorderclips and POSTs them instead of using Web Speech. Toggle-disable + the engine pick key on whichever browser API the chosen engine needs.reqwestgains themultipartfeature; CLAUDE.md documents the three env vars.When
LETS_CHAT_STT_URLis unset, behavior is unchanged (in-browser Web Speech).Testing
tests/transcripts.rsextended to 7 (all green):server_stt_audio_records_segment(mock engine -> segment stored + attributed),audio_rejected_when_server_stt_disabled(400), plus the existing Phase 1/2 gating + the voice-participant test.just check,just test,just test-saasall green (exit 0). The 88 testAppStateliterals updated withstt_client: None.Operator-action
Optional: set
LETS_CHAT_STT_URL(+ optional_API_KEY/_MODEL) to an OpenAI-compatible transcription endpoint to switch call transcription to self-hosted server-side STT; leave unset to keep the in-browser engine. The endpoint is trusted/unfiltered - point it only at an operator-run service.Note for review
Worth
/code-review ultragiven the size + the new trusted-HTTP path: the one security-relevant decision isoutbound_trusted_postdeliberately skipping the SSRF filter for an operator-config URL (same posture as SMTP/IMAP), kept inside the chokepoint module.Add an operator-selectable server-side speech-to-text engine for call transcription, so it works browser-agnostically (Firefox/Safari) and keeps audio off third-party clouds. Architecture: rather than bundle whisper.cpp (huge native dep + model) into the single binary, lets-chat forwards audio to an operator-configured OpenAI-compatible /v1/audio/transcriptions endpoint (whisper.cpp server, faster-whisper, LocalAI, ...). The binary stays lean and the operator runs/chooses the engine, on localhost if they like. - New `stt` module: SttConfig::from_env (LETS_CHAT_STT_URL enables it; _API_KEY, _MODEL optional), an SttClient trait with ReqwestSttClient (multipart POST) + MockSttClient (tests), mirroring the push_client pattern. AppState gains `stt_client: Option<...>`. - New endpoint POST /call/transcript/{id}/audio: participant-gated, forwards the clip to the STT endpoint, records the returned text as a segment + live caption via the shared record_and_broadcast path. 400 when STT is unconfigured. - The STT endpoint is operator-trusted (SMTP/IMAP class), so it is NOT public-IP SSRF-filtered (must reach localhost/internal). Routed through a single new blessed http_client::outbound_trusted_post helper that lives inside the LC-152 chokepoint module, so the grep-ban stays a single-file allowance. - GET /call/config advertises the engine (`sttServer`); transcribe.js reads it and, when server STT is on, captures fixed-length MediaRecorder clips and POSTs them instead of using the Web Speech API. Toggle-disable + the engine pick key on whichever API the chosen engine needs. - reqwest gains the `multipart` feature. CLAUDE.md documents the three env vars. When LETS_CHAT_STT_URL is unset, behavior is unchanged (in-browser Web Speech, Phases 1-2). All 88 test AppState literals updated; tests cover the audio path via the mock engine + the disabled-400 + participant gating. Operator-Action: optional - set LETS_CHAT_STT_URL (+ optional _API_KEY / _MODEL) to an OpenAI-compatible transcription endpoint to switch call transcription to self-hosted server-side STT; leave unset to keep the in-browser engine. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>