feat(kvm): macOS KVM backend (VA-93) #142

Merged

longjacksonle merged 1 commit from feat/VA-93-macos-kvm into main

2026-07-15 17:29:43 +02:00

longjacksonle commented

2026-07-15 17:14:40 +02:00

Owner

Implements VA-93: the macOS vervain-agent-kvm backend, so remote desktop (capture, input, display enumeration) works on macOS agents. VA-89 shipped the macOS agent with KVM stubbed to NotAvailable; this brings it to parity with the Linux (X11 / Wayland) and Windows (DXGI) backends, feeding the existing tile-diff + JPEG pipeline unchanged.

What landed

Capture: CGDisplayCreateImage via the core-graphics safe wrappers (same rationale the Windows backend uses the windows crate). Returns BGRA, mapping directly onto PixelLayout::Bgrx (the DXGI layout), so stream.rs is reused rather than forked. capture_primary, capture_display (by display-<id> name), and capture_raw_frame (streaming) are all wired.
Input: Quartz Event Services. Mouse move/button (with drag events emitted while a button is held), scroll wheel (CGEventCreateScrollWheelEvent2, hence the highsierra feature), and keyboard. Named keysyms map to macOS virtual keycodes; other single-character keysyms inject as a layout-independent Unicode string (mirrors the Windows KEYEVENTF_UNICODE path).
Display enumeration: CGGetActiveDisplayList + CGDisplayBounds, same DisplayInfo shape; pixel sizes, offsets scaled points->pixels per display.
TCC permissions: capture preflights Screen Recording, input preflights Accessibility. A missing grant returns a clear NotAvailable naming the permission and pointing at docs/macos-permissions.md, never a silent black frame or dropped input. The doc also covers the LaunchAgent-vs-LaunchDaemon session implication for capture.

Scope / follow-ups

CGDisplayCreateImage is functional but deprecated on macOS 14; a ScreenCaptureKit path for lower-overhead streaming is a noted follow-up (mirrors how Windows shipped a working MVP first).
Multi-display offsets are exact under a uniform scale factor; mixed-DPI layouts are approximated (noted follow-up).

Testing

All macOS code is cfg(target_os = "macos") gated: the Linux and Windows builds are unaffected.
Locally validated with cross-target cargo check and cargo clippy --all-targets --deny warnings for both aarch64-apple-darwin and x86_64-apple-darwin (the macOS Rust typechecks and lints clean; cargo check skips linking, so no macOS SDK was needed).
Full Linux workspace clippy --deny warnings and the test suite pass (93 kvm tests, 0 regressions).
The real macOS link/build runs here in CI on the VA-90 runner. Runtime acceptance (capture renders, input lands, multi-display parity) needs a human on a real Mac with the two TCC grants in place.

Acceptance criteria

macOS capture feeds the tile-stream pipeline (wired via capture_raw_frame; renders pending on-device check)
Mouse move/click/scroll and keyboard injection via CGEvent
Multi-display enumeration returns DisplayInfo
Missing Screen Recording / Accessibility permission produces a clear error, grant documented
Linux/Windows KVM builds unaffected (macOS code cfg-gated)
Check suite passes; macOS build (VA-89) still succeeds (validated by this PR's CI)

🤖 Generated with Claude Code

Implements VA-93: the macOS `vervain-agent-kvm` backend, so remote desktop (capture, input, display enumeration) works on macOS agents. VA-89 shipped the macOS agent with KVM stubbed to `NotAvailable`; this brings it to parity with the Linux (X11 / Wayland) and Windows (DXGI) backends, feeding the existing tile-diff + JPEG pipeline unchanged. ## What landed - Capture: `CGDisplayCreateImage` via the `core-graphics` safe wrappers (same rationale the Windows backend uses the `windows` crate). Returns BGRA, mapping directly onto `PixelLayout::Bgrx` (the DXGI layout), so `stream.rs` is reused rather than forked. `capture_primary`, `capture_display` (by `display-<id>` name), and `capture_raw_frame` (streaming) are all wired. - Input: Quartz Event Services. Mouse move/button (with drag events emitted while a button is held), scroll wheel (`CGEventCreateScrollWheelEvent2`, hence the `highsierra` feature), and keyboard. Named keysyms map to macOS virtual keycodes; other single-character keysyms inject as a layout-independent Unicode string (mirrors the Windows `KEYEVENTF_UNICODE` path). - Display enumeration: `CGGetActiveDisplayList` + `CGDisplayBounds`, same `DisplayInfo` shape; pixel sizes, offsets scaled points->pixels per display. - TCC permissions: capture preflights Screen Recording, input preflights Accessibility. A missing grant returns a clear `NotAvailable` naming the permission and pointing at `docs/macos-permissions.md`, never a silent black frame or dropped input. The doc also covers the LaunchAgent-vs-LaunchDaemon session implication for capture. ## Scope / follow-ups - `CGDisplayCreateImage` is functional but deprecated on macOS 14; a ScreenCaptureKit path for lower-overhead streaming is a noted follow-up (mirrors how Windows shipped a working MVP first). - Multi-display offsets are exact under a uniform scale factor; mixed-DPI layouts are approximated (noted follow-up). ## Testing - All macOS code is `cfg(target_os = "macos")` gated: the Linux and Windows builds are unaffected. - Locally validated with cross-target `cargo check` and `cargo clippy --all-targets --deny warnings` for both `aarch64-apple-darwin` and `x86_64-apple-darwin` (the macOS Rust typechecks and lints clean; `cargo check` skips linking, so no macOS SDK was needed). - Full Linux workspace `clippy --deny warnings` and the test suite pass (93 kvm tests, 0 regressions). - The real macOS link/build runs here in CI on the VA-90 runner. Runtime acceptance (capture renders, input lands, multi-display parity) needs a human on a real Mac with the two TCC grants in place. ## Acceptance criteria - [x] macOS capture feeds the tile-stream pipeline (wired via `capture_raw_frame`; renders pending on-device check) - [x] Mouse move/click/scroll and keyboard injection via CGEvent - [x] Multi-display enumeration returns `DisplayInfo` - [x] Missing Screen Recording / Accessibility permission produces a clear error, grant documented - [x] Linux/Windows KVM builds unaffected (macOS code cfg-gated) - [x] Check suite passes; macOS build (VA-89) still succeeds (validated by this PR's CI) 🤖 Generated with [Claude Code](https://claude.com/claude-code)

longjacksonle added 1 commit

2026-07-15 17:14:40 +02:00

feat(kvm): macOS KVM backend (VA-93)

Check / fmt + clippy + tests (pull_request) Successful in 14m1s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

c851756f74

Implement the macOS `vervain-agent-kvm` backend so remote desktop works on macOS agents (VA-89 shipped with KVM stubbed to `NotAvailable`), reaching parity with the Linux (X11 / Wayland) and Windows (DXGI) backends and feeding the existing tile-diff + JPEG pipeline unchanged.

Capture uses `CGDisplayCreateImage` (via the `core-graphics` safe wrappers, chosen for the same ergonomic reason the Windows backend uses the `windows` crate): synchronous, feeds the existing `spawn_blocking` path, and returns BGRA which maps directly onto `PixelLayout::Bgrx`, the same layout the DXGI path produces. `capture_primary` / `capture_display` / `capture_raw_frame` are all wired. A ScreenCaptureKit path for lower-overhead streaming is a documented follow-up; `CGDisplayCreateImage` is functional (deprecated on macOS 14 but still works).

Input uses Quartz Event Services: `CGEventCreateMouseEvent` (move/button, with drag events when a button is held), `CGEventCreateScrollWheelEvent2` (wheel), and `CGEventCreateKeyboardEvent` (named keys map to macOS virtual keycodes; other single-character keysyms inject as a layout-independent Unicode string, mirroring the Windows `KEYEVENTF_UNICODE` path).

Display enumeration uses `CGGetActiveDisplayList` + `CGDisplayBounds`, producing the same `DisplayInfo` shape (pixel sizes; offsets scaled points->pixels per display, exact under a uniform scale factor).

TCC: capture preflights Screen Recording (`CGPreflightScreenCaptureAccess`) and input preflights Accessibility (`AXIsProcessTrusted`); a missing grant returns a clear `NotAvailable` pointing at `docs/macos-permissions.md` rather than a silent black frame or dropped input. The doc covers the two grants and the LaunchAgent-vs-LaunchDaemon session implication for capture.

All macOS code is `cfg(target_os = "macos")` gated, so the Linux and Windows builds are unaffected. Validated by cross-target `cargo check` + `clippy --deny warnings` for both `aarch64-apple-darwin` and `x86_64-apple-darwin`, plus the full Linux workspace clippy and test suite; the real macOS link/build runs on the VA-90 runner via CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LYb1fkK32XQdwkGkStxXR9
Signed-off-by: longjacksonle <longjacksonle@gmail.com>