feat(realtime): remote-control input capture + data-channel transport (LC-184, controller)

nrupard commented

2026-05-27 15:34:48 +02:00

Owner

Controller-side half of the TeamViewer-style remote-control story (LC-181). A controller captures their pointer + keyboard over a verified, consented 1:1 call and ships a compact input stream peer-to-peer over a WebRTC data channel. Native OS injection is LC-185; kill-switch / audit / abuse limits are LC-186. The request/grant/deny/revoke consent gate (both peers email-verified, neither blocked) is already enforced server-side in LC-183 (#227); this PR wires the client half.

What lands

Negotiated data channel. call.js opens an lc-control channel (negotiated, id 0) on both peers in createPc, so it rides the initial SDP and never triggers a mid-call onnegotiationneeded renegotiation. Input rides this channel peer-to-peer, off the signaling WS.
Input capture (controller). While a session is ACTIVE: pointer move/down/up + wheel on the remote <video>, keys at the window with preventDefault (so the controller's own browser shortcuts do not fire on their machine). Pointer-move is rAF-coalesced; clicks/keys send immediately.
Coordinate mapping. normCoords maps a viewport point on the letterboxed object-contain video to normalized [0,1] surface coordinates and drops points in the letterbox bars, keeping the protocol resolution/DPI-independent (the controlled side maps [0,1] back to absolute screen pixels).
Consent state machine. none -> requesting -> controlling, with a 12s request timeout so a server-dropped request (unverified/blocked peer) self-reverts; Request control <-> Stop controlling toggle; auto-stop on remote-video drop, inbound revoke, and call teardown.
Controlled (sharer) side. Grant/deny prompt; arms/disarms a native injector via DOM lc:control-start / lc:control-end, and re-emits each inbound data-channel frame as lc:control-input (the LC-185 bridge seam). Channel onclose/onerror ends the session on both roles as a fast path on abrupt drop.
layout.html. #lc-control-bus slot (static target for the OOB consent fragment from LC-183), the grant/deny prompt overlay, and the Request-control affordance.
docs. Wire-frame shapes + the lc:control-* DOM-event seam recorded for LC-185.

Wire frames (controller -> controlled)

{t:'m',x,y} move, {t:'d',x,y,b} down, {t:'u',x,y,b} up, {t:'w',x,y,dx,dy} wheel, {t:'k',c,m} key down, {t:'K',c,m} key up. x,y normalized [0,1]; b mouse button; c is KeyboardEvent.code (physical key, layout-independent); m modifier bitmask (ctrl=1, shift=2, alt=4, meta=8).

Out of scope (later subtasks)

LC-185: native OS input injection in the desktop app (Windows SendInput first) + the JS->native bridge re-asserting grant before injecting.
LC-186: sharer kill-switch (button + global hotkey), server-side request rate-limit, audit log, max session duration. Note LC-185/186 keep their own independent heartbeat timeout for held-key release; the channel-drop handler here is only the fast path.

Verification

cargo check -p lets-chat-server clean (askama template compile).
call.js bundles clean via bun build (no syntax errors).
just test and just test-saas both green (no Rust logic changed; JS + template + doc only).
Code review (cavecrew-reviewer) addressed: added the channel onclose/onerror fast-path end. The flagged wheel removeEventListener "leak" is a non-issue (removal matches on the capture flag only, which matches).

Controller-side half of the TeamViewer-style remote-control story (LC-181). A controller captures their pointer + keyboard over a verified, consented 1:1 call and ships a compact input stream peer-to-peer over a WebRTC data channel. Native OS injection is LC-185; kill-switch / audit / abuse limits are LC-186. The request/grant/deny/revoke consent gate (both peers email-verified, neither blocked) is already enforced server-side in LC-183 (#227); this PR wires the client half. ## What lands - **Negotiated data channel.** `call.js` opens an `lc-control` channel (negotiated, id 0) on both peers in `createPc`, so it rides the initial SDP and never triggers a mid-call `onnegotiationneeded` renegotiation. Input rides this channel peer-to-peer, off the signaling WS. - **Input capture (controller).** While a session is ACTIVE: pointer move/down/up + wheel on the remote `<video>`, keys at the window with `preventDefault` (so the controller's own browser shortcuts do not fire on their machine). Pointer-move is rAF-coalesced; clicks/keys send immediately. - **Coordinate mapping.** `normCoords` maps a viewport point on the letterboxed `object-contain` video to normalized `[0,1]` surface coordinates and drops points in the letterbox bars, keeping the protocol resolution/DPI-independent (the controlled side maps `[0,1]` back to absolute screen pixels). - **Consent state machine.** `none -> requesting -> controlling`, with a 12s request timeout so a server-dropped request (unverified/blocked peer) self-reverts; `Request control` <-> `Stop controlling` toggle; auto-stop on remote-video drop, inbound `revoke`, and call teardown. - **Controlled (sharer) side.** Grant/deny prompt; arms/disarms a native injector via DOM `lc:control-start` / `lc:control-end`, and re-emits each inbound data-channel frame as `lc:control-input` (the LC-185 bridge seam). Channel `onclose`/`onerror` ends the session on both roles as a fast path on abrupt drop. - **layout.html.** `#lc-control-bus` slot (static target for the OOB consent fragment from LC-183), the grant/deny prompt overlay, and the Request-control affordance. - **docs.** Wire-frame shapes + the `lc:control-*` DOM-event seam recorded for LC-185. ## Wire frames (controller -> controlled) `{t:'m',x,y}` move, `{t:'d',x,y,b}` down, `{t:'u',x,y,b}` up, `{t:'w',x,y,dx,dy}` wheel, `{t:'k',c,m}` key down, `{t:'K',c,m}` key up. `x,y` normalized `[0,1]`; `b` mouse button; `c` is `KeyboardEvent.code` (physical key, layout-independent); `m` modifier bitmask (ctrl=1, shift=2, alt=4, meta=8). ## Out of scope (later subtasks) - LC-185: native OS input injection in the desktop app (Windows `SendInput` first) + the JS->native bridge re-asserting grant before injecting. - LC-186: sharer kill-switch (button + global hotkey), server-side request rate-limit, audit log, max session duration. Note LC-185/186 keep their own independent heartbeat timeout for held-key release; the channel-drop handler here is only the fast path. ## Verification - `cargo check -p lets-chat-server` clean (askama template compile). - `call.js` bundles clean via `bun build` (no syntax errors). - `just test` and `just test-saas` both green (no Rust logic changed; JS + template + doc only). - Code review (cavecrew-reviewer) addressed: added the channel `onclose`/`onerror` fast-path end. The flagged wheel `removeEventListener` "leak" is a non-issue (removal matches on the `capture` flag only, which matches).

nrupard added 1 commit

2026-05-27 15:34:48 +02:00

feat(realtime): remote-control input capture + data-channel transport (LC-184, controller)

check-secrets / Nosey parker (push) Successful in 3s

Details

check-secrets / TruffleHog (push) Successful in 4s

Details

check-secrets / Kingfisher (push) Successful in 5s

Details

check-secrets / TruffleHog (pull_request) Successful in 3s

Details

check-secrets / Nosey parker (pull_request) Successful in 4s

Details

check-secrets / Kingfisher (pull_request) Successful in 4s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

Check / clippy + fmt + tests (pull_request) Successful in 1m36s

Details

b255670eb1

Controller-side half of the TeamViewer-style remote-control story (LC-181). Captures the controller's pointer + keyboard over a verified, consented 1:1 call and ships a compact input stream peer-to-peer over a WebRTC data channel; native OS injection (LC-185) and the kill-switch/audit/limits (LC-186) land separately. The request/grant/deny/revoke consent gate is already enforced server-side (LC-183); this wires the client half.

call.js: open a negotiated `lc-control` data channel (id 0) on both peers in createPc, so it rides the initial SDP and never triggers a mid-call renegotiation. While a control session is ACTIVE, capture pointer move/down/up + wheel on the remote `<video>` and keys at the window (preventDefault so the controller's own browser shortcuts do not fire); pointer-move is rAF-coalesced, clicks/keys send immediately. normCoords maps the letterboxed object-contain video point to normalized [0,1] and drops points in the letterbox bars, keeping the protocol resolution/DPI-independent.

Consent state machine (none -> requesting -> controlling) with a 12s request timeout so a server-dropped (unverified/blocked) request self-reverts; Request control <-> Stop controlling toggle; auto-stop on remote-video drop, inbound revoke, and call teardown. Controlled (sharer) side shows a grant/deny prompt and arms/disarms the native injector via DOM events lc:control-start / lc:control-end, re-emitting each inbound frame as lc:control-input (the LC-185 bridge seam). A channel onclose/onerror ends the session on both roles as a fast path on abrupt drop (LC-185/186 keep their own independent heartbeat timeout for stuck-key prevention).

layout.html: #lc-control-bus slot (static target for the OOB consent fragment), the grant/deny prompt overlay, and the Request-control affordance.

docs: record the wire-frame shapes and the lc:control-* DOM-event seam LC-185 depends on.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>