fix(kvm): composite DRM cursor plane into scanout capture (VA-110)

Claude-Run commented

2026-07-04 18:23:19 +02:00

Member

The DRM/KMS scanout path reads only the CRTC's primary framebuffer, but the hardware cursor lives on a separate DRM cursor plane, so a primary-plane readback captured the desktop without the pointer. On desktop-02 the DRM path is now the active backend (after the VA-104/VA-106 scanout work), which is why the operator could move the remote pointer on the physical screen while it stayed invisible in the Desktop tab.

Enumerate the active CRTC's planes, find the DRM_PLANE_TYPE_CURSOR plane, read its (alpha-carrying) framebuffer through the same explicit-linear-mmap / gbm de-tile readback the primary plane uses, and alpha-blend it onto the captured RGB frame at the plane's CRTC_X/CRTC_Y position. The compositor already bakes the hot spot into that position, so the overlay tracks the physical cursor pixel-for-pixel and is re-read every frame. Cursor compositing is strictly best-effort: a hidden cursor (no framebuffer bound), a driver with no cursor plane, or an unreadable cursor format leaves the primary frame untouched rather than failing the capture.

The portal PipeWire path already requests CursorMode::Embedded, so the compositor composites the cursor into that stream; no change needed there. Added a once-per-backend log in wayland::capture_rgb so the active capture path (drm-kms / grim-wlr-screencopy / xdg-portal-pipewire) is confirmable from the journal.

Shared the gbm import/map/unmap into map_bo so the new RGBA cursor read reuses it, and unit-tested the RGBA conversion and the alpha-blend (opaque overwrite, transparent no-op, half-alpha mix, off-screen clipping) so the compositing logic is covered without a DRM device.

#VA-110

The DRM/KMS scanout path reads only the CRTC's primary framebuffer, but the hardware cursor lives on a separate DRM cursor plane, so a primary-plane readback captured the desktop without the pointer. On desktop-02 the DRM path is now the active backend (after the VA-104/VA-106 scanout work), which is why the operator could move the remote pointer on the physical screen while it stayed invisible in the Desktop tab. Enumerate the active CRTC's planes, find the `DRM_PLANE_TYPE_CURSOR` plane, read its (alpha-carrying) framebuffer through the same explicit-linear-mmap / gbm de-tile readback the primary plane uses, and alpha-blend it onto the captured RGB frame at the plane's `CRTC_X`/`CRTC_Y` position. The compositor already bakes the hot spot into that position, so the overlay tracks the physical cursor pixel-for-pixel and is re-read every frame. Cursor compositing is strictly best-effort: a hidden cursor (no framebuffer bound), a driver with no cursor plane, or an unreadable cursor format leaves the primary frame untouched rather than failing the capture. The portal PipeWire path already requests `CursorMode::Embedded`, so the compositor composites the cursor into that stream; no change needed there. Added a once-per-backend log in `wayland::capture_rgb` so the active capture path (drm-kms / grim-wlr-screencopy / xdg-portal-pipewire) is confirmable from the journal. Shared the gbm import/map/unmap into `map_bo` so the new RGBA cursor read reuses it, and unit-tested the RGBA conversion and the alpha-blend (opaque overwrite, transparent no-op, half-alpha mix, off-screen clipping) so the compositing logic is covered without a DRM device. #VA-110

Claude-Run added 1 commit

2026-07-04 18:23:19 +02:00

fix(kvm): composite DRM cursor plane into scanout capture (VA-110)

Check / fmt + clippy + tests (pull_request) Successful in 5m12s

Details

Create release / Create release from merged PR (pull_request) Has been skipped

Details

ee9a47099f

The DRM/KMS scanout path reads only the CRTC's primary framebuffer, but the hardware cursor lives on a separate DRM cursor plane, so a primary-plane readback captured the desktop without the pointer. On desktop-02 the DRM path is now the active backend (after the VA-104/VA-106 scanout work), which is why the operator could move the remote pointer on the physical screen while it stayed invisible in the Desktop tab.

Enumerate the active CRTC's planes, find the `DRM_PLANE_TYPE_CURSOR` plane, read its (alpha-carrying) framebuffer through the same explicit-linear-mmap / gbm de-tile readback the primary plane uses, and alpha-blend it onto the captured RGB frame at the plane's `CRTC_X`/`CRTC_Y` position. The compositor already bakes the hot spot into that position, so the overlay tracks the physical cursor pixel-for-pixel and is re-read every frame. Cursor compositing is strictly best-effort: a hidden cursor (no framebuffer bound), a driver with no cursor plane, or an unreadable cursor format leaves the primary frame untouched rather than failing the capture.

The portal PipeWire path already requests `CursorMode::Embedded`, so the compositor composites the cursor into that stream; no change needed there. Added a once-per-backend log in `wayland::capture_rgb` so the active capture path (drm-kms / grim-wlr-screencopy / xdg-portal-pipewire) is confirmable from the journal.

Shared the gbm import/map/unmap into `map_bo` so the new RGBA cursor read reuses it, and unit-tested the RGBA conversion and the alpha-blend (opaque overwrite, transparent no-op, half-alpha mix, off-screen clipping) so the compositing logic is covered without a DRM device.

#VA-110

Claude-Run merged commit c946a46f7b into main

2026-07-04 18:28:54 +02:00

Claude-Run deleted branch fix/VA-110-drm-cursor-overlay

2026-07-04 18:28:54 +02:00

Claude-Run referenced this pull request from a commit

2026-07-04 18:28:55 +02:00

Merge pull request 'fix(kvm): composite DRM cursor plane into scanout capture (VA-110)' (#113) from fix/VA-110-drm-cursor-overlay into main

Rows
Columns

fix(kvm): composite DRM cursor plane into scanout capture (VA-110) #113