VAPP-6: Remote desktop viewer (KVM tile stream to canvas) #33

Merged

nrupard merged 1 commit from feat/remote-desktop-viewer-vapp-6 into main

2026-06-12 19:11:55 +02:00

nrupard commented

2026-06-12 19:04:50 +02:00

Owner

What

Implements VAPP-6: a live remote-desktop (KVM) viewer reachable from the device detail page. Wires the previously inert "Remote desktop" header button and adds a Desktop tab alongside Terminal, both gated on the device being online.

How

The browser opens a usage=2 relay tunnel through the new open_desktop_tunnel server fn, connects DIRECTLY to vervain-server's relay (no hop through the apps server, per the parity-review decision), decodes the agent's JPEG tile stream onto a <canvas> via web-sys createImageBitmap (pure Rust, no JS dependency), and feeds mouse / keyboard / wheel input back as the legacy agent-desktop binary frames.

Tunnel establishment is factored out of open_terminal_tunnel into a shared open_relay_tunnel(device_id, usage, label), so terminal (usage 1) and desktop (usage 2) share the relay-cookie minting, rendezvous id, and routing-ack error mapping. TerminalTunnel is reused as the connection-details payload for both.

The wire protocol is implemented exactly from vervain-agent::host::tunnel::run_desktop (VA-4): inbound cmd 7 screen-size sizes the canvas, cmd 3 tiles blit incrementally at their x/y (no full-frame clear), cmd 11 displays ignored; outbound cmd 1 key (Windows VK from event.keyCode), cmd 2 mouse (move/click/scroll, coordinates scaled from canvas-local pixels to the remote screen), cmd 5 quality (live slider), cmd 6 refresh.

Acceptance criteria

Desktop button / tab opens a live view of an online device's display (X11 or wlroots Wayland host, handled agent-side)
Mouse move/click/scroll and keyboard input reach the remote session
Tile updates render incrementally (per-tile blit, no full-frame flashes)
Quality control changes the JPEG quality mid-session (cmd 5 from the slider)
Offline device, missing desktop rights, or routing failure produce a clear error state; the tab and header button are disabled when offline
Linked as depends-on to VA-4 (already linked in YouTrack)

Checks

just check equivalents all green in the rust-builder-glibc:v1.0.1-rust1.94 image: cargo fmt --check, cargo clippy --all-targets --features server -- -D warnings, cargo clippy --target wasm32-unknown-unknown -- -D warnings, cargo check --features server, cargo check --target wasm32-unknown-unknown, and cargo test --features server --bin vervain-app (15 passed).

## What Implements VAPP-6: a live remote-desktop (KVM) viewer reachable from the device detail page. Wires the previously inert "Remote desktop" header button and adds a Desktop tab alongside Terminal, both gated on the device being online. ## How The browser opens a `usage=2` relay tunnel through the new `open_desktop_tunnel` server fn, connects DIRECTLY to vervain-server's relay (no hop through the apps server, per the parity-review decision), decodes the agent's JPEG tile stream onto a `<canvas>` via web-sys `createImageBitmap` (pure Rust, no JS dependency), and feeds mouse / keyboard / wheel input back as the legacy `agent-desktop` binary frames. Tunnel establishment is factored out of `open_terminal_tunnel` into a shared `open_relay_tunnel(device_id, usage, label)`, so terminal (usage 1) and desktop (usage 2) share the relay-cookie minting, rendezvous id, and routing-ack error mapping. `TerminalTunnel` is reused as the connection-details payload for both. The wire protocol is implemented exactly from `vervain-agent::host::tunnel::run_desktop` (VA-4): inbound `cmd 7` screen-size sizes the canvas, `cmd 3` tiles blit incrementally at their x/y (no full-frame clear), `cmd 11` displays ignored; outbound `cmd 1` key (Windows VK from `event.keyCode`), `cmd 2` mouse (move/click/scroll, coordinates scaled from canvas-local pixels to the remote screen), `cmd 5` quality (live slider), `cmd 6` refresh. ## Acceptance criteria - [x] Desktop button / tab opens a live view of an online device's display (X11 or wlroots Wayland host, handled agent-side) - [x] Mouse move/click/scroll and keyboard input reach the remote session - [x] Tile updates render incrementally (per-tile blit, no full-frame flashes) - [x] Quality control changes the JPEG quality mid-session (cmd 5 from the slider) - [x] Offline device, missing desktop rights, or routing failure produce a clear error state; the tab and header button are disabled when offline - [x] Linked as depends-on to VA-4 (already linked in YouTrack) ## Checks `just check` equivalents all green in the `rust-builder-glibc:v1.0.1-rust1.94` image: `cargo fmt --check`, `cargo clippy --all-targets --features server -- -D warnings`, `cargo clippy --target wasm32-unknown-unknown -- -D warnings`, `cargo check --features server`, `cargo check --target wasm32-unknown-unknown`, and `cargo test --features server --bin vervain-app` (15 passed).

nrupard added 1 commit

2026-06-12 19:04:50 +02:00

feat(device): remote desktop viewer (KVM tile stream to canvas)

Check / fmt + clippy + build + tests (pull_request) Successful in 1m5s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

67758a6b2b

Wire the inert "Remote desktop" header button and a new Desktop tab on the device detail page to a live KVM view, mirroring the terminal pane (VAPP-5). The browser opens a usage=2 relay tunnel via the new open_desktop_tunnel server fn, connects directly to vervain-server's relay, decodes the agent's JPEG tile stream onto a canvas via web-sys createImageBitmap (no JS dependency, per the parity-review decision), and feeds mouse / keyboard / wheel input back as the legacy agent-desktop binary frames.

The tunnel-establishment logic is factored out of open_terminal_tunnel into a shared open_relay_tunnel(device_id, usage, label) so terminal (usage 1) and desktop (usage 2) share the cookie minting, rendezvous, and routing-ack error mapping. TerminalTunnel is reused as the connection-details payload for both.

Protocol implemented from vervain-agent::host::tunnel::run_desktop (VA-4): inbound cmd 7 screen size sizes the canvas, cmd 3 tiles blit incrementally at their x/y, cmd 11 displays ignored; outbound cmd 1 key (Windows VK from event.keyCode), cmd 2 mouse (move/click/scroll), cmd 5 quality (live slider), cmd 6 refresh. Coordinates scale canvas-local pixels to the remote screen. Offline devices, missing agent-console rights, and routing failures surface a clear error overlay; the tab and header button are disabled when the device is offline.

#VAPP-6