feat(agent): Wayland screen capture via dlopen PipeWire (VA-82) #87

Merged

David merged 3 commits from fix/VA-82-pipewire-dlopen into main

2026-06-29 13:20:24 +02:00

David commented

2026-06-29 12:59:55 +02:00

Owner

What

Completes VA-82: the agent captures the screen on a Wayland desktop, while the single shipped binary still execs on headless hosts with no hard libpipewire/libgstreamer dependency. This is the end-state the maintainer asked for (one Linux binary like every other platform), achieved by loading PipeWire at runtime.

Background

VA-80 stopped the silent black X11 fallback on Wayland; a --features wayland-portal test build proved the existing ScreenCast + PipeWire consumer captures correctly on the GNOME/Wayland host. The blocker to shipping it: the pipewire crate's libspa-sys links libpipewire-0.3 via pkg-config, adding a NEEDED entry that would break the agent on headless hosts (binary won't exec). RustDesk avoids a libpipewire link only by going through GStreamer's plugin loader, trading it for a libgstreamer hard link. This PR removes the hard dependency entirely via dlopen.

Changes

pw_ffi: dlopens libpipewire-0.3.so.0 with libloading and binds the ~dozen pw_* symbols the consumer needs. The spa_buffer / spa_data / spa_chunk / pw_buffer / pw_stream_events / spa_hook layouts are declared #[repr(C)] verbatim from the installed spa-0.2 / pipewire-0.3 headers. Loads lazily; absent library -> clean NotAvailable (the VA-80 surfaced error), never a crash.
spa_pod: pure-Rust SPA POD codec (no libspa link). Builds the EnumFormat object sent to pw_stream_connect and parses the negotiated Format read in param_changed. All SPA constants taken verbatim from headers. 7 unit tests (byte layout, fixated + Choice-wrapped parsing, 4K dims, non-Format/non-object/truncated rejection).
portal::run_pipewire_loop: rewritten off the pipewire crate onto pw_ffi + spa_pod; convert_to_rgb now takes a SPA_VIDEO_FORMAT_* id. ashpd RemoteDesktop+ScreenCast session and input paths unchanged (still pure-Rust DBus; restore-token persistence via PersistMode::Application retained).
wayland-portal feature drops pipewire, adds libloading; now pulls only pure-Rust deps (ashpd/zbus, image, libloading) so the feature build needs no system libs or libclang. Cargo.lock refreshed (pipewire/libspa removed).
release.Dockerfile: builds with --features wayland-portal and asserts via readelf -d that the binary has no NEEDED libpipewire, guarding the headless-safe property in CI.

Verification (Docker only)

just pre-commit green (default build: fmt, clippy -D warnings, build, host+windows, lib+tpm tests; 7 spa_pod tests pass).
Feature path clippy -D warnings green.
Release build (--features wayland-portal --locked) succeeds; readelf -d on the resulting binary shows only libc, libgcc_s, libm - no pipewire/spa - and it runs --version.

Functional on-device capture (actual frames on GNOME/Wayland) is validated separately on desktop-02 with a binary from this branch; the consumer logic is the same one already proven with the pipewire crate, so the risk here is confined to the FFI port, which the no-link release build compiles and links clean.

Fixes VA-82. Relates to VA-80, VA-81.

🤖 Generated with Claude Code

## What Completes VA-82: the agent captures the screen on a Wayland desktop, while the single shipped binary still execs on headless hosts with **no hard libpipewire/libgstreamer dependency**. This is the end-state the maintainer asked for (one Linux binary like every other platform), achieved by loading PipeWire at runtime. ## Background VA-80 stopped the silent black X11 fallback on Wayland; a `--features wayland-portal` test build proved the existing ScreenCast + PipeWire consumer captures correctly on the GNOME/Wayland host. The blocker to shipping it: the `pipewire` crate's `libspa-sys` links `libpipewire-0.3` via pkg-config, adding a `NEEDED` entry that would break the agent on headless hosts (binary won't exec). RustDesk avoids a libpipewire link only by going through GStreamer's plugin loader, trading it for a libgstreamer hard link. This PR removes the hard dependency entirely via dlopen. ## Changes - `pw_ffi`: `dlopen`s `libpipewire-0.3.so.0` with `libloading` and binds the ~dozen `pw_*` symbols the consumer needs. The `spa_buffer` / `spa_data` / `spa_chunk` / `pw_buffer` / `pw_stream_events` / `spa_hook` layouts are declared `#[repr(C)]` verbatim from the installed spa-0.2 / pipewire-0.3 headers. Loads lazily; absent library -> clean `NotAvailable` (the VA-80 surfaced error), never a crash. - `spa_pod`: pure-Rust SPA POD codec (no libspa link). Builds the EnumFormat object sent to `pw_stream_connect` and parses the negotiated Format read in `param_changed`. All SPA constants taken verbatim from headers. 7 unit tests (byte layout, fixated + Choice-wrapped parsing, 4K dims, non-Format/non-object/truncated rejection). - `portal::run_pipewire_loop`: rewritten off the `pipewire` crate onto `pw_ffi` + `spa_pod`; `convert_to_rgb` now takes a `SPA_VIDEO_FORMAT_*` id. ashpd RemoteDesktop+ScreenCast session and input paths unchanged (still pure-Rust DBus; restore-token persistence via `PersistMode::Application` retained). - `wayland-portal` feature drops `pipewire`, adds `libloading`; now pulls only pure-Rust deps (ashpd/zbus, image, libloading) so the feature build needs no system libs or libclang. `Cargo.lock` refreshed (pipewire/libspa removed). - `release.Dockerfile`: builds with `--features wayland-portal` and asserts via `readelf -d` that the binary has **no NEEDED libpipewire**, guarding the headless-safe property in CI. ## Verification (Docker only) - `just pre-commit` green (default build: fmt, clippy `-D warnings`, build, host+windows, lib+tpm tests; 7 `spa_pod` tests pass). - Feature path clippy `-D warnings` green. - Release build (`--features wayland-portal --locked`) succeeds; `readelf -d` on the resulting binary shows only `libc`, `libgcc_s`, `libm` - no pipewire/spa - and it runs `--version`. Functional on-device capture (actual frames on GNOME/Wayland) is validated separately on desktop-02 with a binary from this branch; the consumer logic is the same one already proven with the `pipewire` crate, so the risk here is confined to the FFI port, which the no-link release build compiles and links clean. Fixes VA-82. Relates to VA-80, VA-81. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

David added 2 commits

2026-06-29 12:59:55 +02:00

feat(agent): hand-rolled SPA POD codec for dlopen PipeWire consumer 0431c4b9b4

First step of the VA-82 dlopen PipeWire capture path. The `pipewire`/`libspa` crates can't be used because `libspa-sys` links libpipewire via pkg-config, which would re-introduce the hard runtime dependency VA-82 exists to remove (the agent binary must still exec on headless hosts). So the consumer will talk to libpipewire over dlopen, and the SPA PODs it exchanges are encoded/decoded here in pure Rust.

`spa_pod` builds the EnumFormat object passed to `pw_stream_connect` (media Video / Raw, packed RGB(A)/BGR(A) formats with BGRx preferred, wide size/framerate ranges) and parses the negotiated Format object delivered in `param_changed` (video format id + size, tolerating fixated plain values or Choice wrappers). All SPA type/format constants are taken verbatim from the installed spa-0.2 headers. Pure Rust, no libpipewire/libspa link; six unit tests cover the EnumFormat byte layout, fixated and Choice-wrapped parsing, 4K dimensions, and malformed input.

The dlopen FFI bindings and the consumer run-loop rewrite (replacing the pipewire-crate `run_pipewire_loop`) follow; this lands the foundational, fully unit-tested codec.

#VA-82

feat(agent): Wayland screen capture via dlopen PipeWire (no hard link)

Check / fmt + clippy + tests (pull_request) Has been cancelled

Details

e8922157f9

Completes VA-82: the agent now captures the screen on a Wayland desktop while the single shipped binary still execs on headless hosts, with no hard libpipewire/libgstreamer dependency.

The ScreenCast frame consumer (`portal::run_pipewire_loop`) is rewritten off the `pipewire` crate (whose `libspa-sys` links libpipewire via pkg-config and would add a NEEDED entry that breaks headless hosts) onto a `dlopen` path: `pw_ffi` loads `libpipewire-0.3.so.0` lazily with libloading and binds the dozen `pw_*` symbols the consumer needs, with the `spa_buffer`/`spa_data`/`spa_chunk`/`pw_buffer`/`pw_stream_events`/`spa_hook` layouts declared `#[repr(C)]` verbatim from the spa-0.2/pipewire-0.3 headers. The SPA PODs exchanged with the library (the EnumFormat sent to `pw_stream_connect`, the negotiated Format read in `param_changed`) are encoded/parsed by the pure-Rust `spa_pod` codec. If libpipewire is absent the consumer returns a clean NotAvailable (the VA-80 surfaced error), never a load-time crash.

The `wayland-portal` feature drops `pipewire` and adds `libloading`; it now pulls only pure-Rust deps (ashpd/zbus, image, libloading) so the feature build needs no system libraries or libclang. The release build enables `--features wayland-portal` and asserts via `readelf -d` that the binary has no NEEDED libpipewire, guarding the headless-safe property in CI. Verified: the release binary links only libc/libgcc_s/libm.

#VA-82

David added 1 commit

2026-06-29 13:09:13 +02:00

Merge branch 'main' into fix/VA-82-pipewire-dlopen

Check / fmt + clippy + tests (pull_request) Successful in 11m7s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

4afdb204de