perf(desktop-gpui): convert camera preview frames off the main thread by richiemcilroy · Pull Request #2761 · CapSoftware/Cap

richiemcilroy · 2026-10-20T12:33:12Z

What

Every camera frame shown in the bubble went through FrameConverter::convert on the main thread. That is a synchronous VTPixelTransferSession transfer (420v → BGRA IOSurface), plus a VTPixelRotationSession pass when mirrored.

The work runs on the GPU scaler, but the call blocks until the hardware finishes. In a symbolized profile of a Studio recording with the camera on (signed build, M4 Max), the main thread spent 438 of 2930 samples waiting in mach_msg inside VTPixelTransferChainDoTransfer. That is about 15% of wall time, roughly 4–5 ms of every 30 fps frame, during which the UI couldn't draw or handle input.

Change

frame_arrived now hands the frame to a dedicated camera-preview-convert thread (bounded(1), latest wins) and returns immediately.
The thread owns the FrameConverter, converts the newest frame, and sends the result back over a bounded(1) channel. A send blocks while the main thread still holds the previous result, so the thread is never more than one frame ahead.
The new converted_arrived does what the tail of frame_arrived did before, unchanged: blur hand-off or raw paint, window resize on a dimension change, first-frame refresh, effect feedback.
The ring grows from 4 to 6 IOSurface buffers. Besides the painted frame and the previous scene's, one converted frame can now wait for the main thread, one can sit with the blur worker, and one is being written, so a slot is never overwritten while a scene or the blur worker still reads it.
Toggling mirror used to drop the converter, which restarted its ring generation at 0 (a possible alias in the blur worker's imported-texture cache). It now sends a rebuild counter in each job, and the converter rebuilds with the next generation (force_rebuild).
If the thread can't be spawned, frames are converted inline exactly as before.
Mirroring, blur (including its 720p downscale), cutout/shape, and the painted cadence are unchanged. A blur mode switched off mid-conversion paints that one frame raw.

Not changed here: converting to the bubble's on-screen size when blur is off. The transfer runs on the hardware scaler, so the cost was the main-thread wait, not the scaling. Smaller targets would also need cover-fit sizing and resize-aware ring rebuilds. That would be a separate, measured change.

Only apps/desktop-gpui/src/camera_window.rs changes; no Hona-authored lines are touched.

Confidence Score: 3/5

Fix the Windows and Linux build failure and the stale blur-setting path before merging.

Findings

Windows and Linux builds fail ▶
Blur misses queued frames ▶
Comments repeat the code ▶

Fix with agent prompt

### Issue 1
apps/desktop-gpui/src/camera_window.rs:1503
The insertion moves the existing macOS guard onto `ConvertJob`, leaving `BlurBridge` unguarded. `BlurBridge` still names `camera_blur::BlurJob`, but `camera_blur` only exists on macOS. Windows and Linux builds now fail to resolve that type. Restore `#[cfg(target_os = "macos")]` immediately before `BlurBridge`.

### Issue 2
apps/desktop-gpui/src/camera_window.rs:1960
If the user enables blur while a frame submitted with `blur_mode: None` is still converting, this filter preserves `None`. `paint_raw` then displays that unblurred frame after blur has been enabled. The old synchronous path could not cross a settings change this way. Drop results whose captured effect settings no longer match the current settings, or apply the currently selected effect before painting.

### Issue 3
apps/desktop-gpui/src/camera_window.rs:1941-1942
This new comment restates what `converted_arrived` does. The repository guide reserves comments for non-obvious context and explicitly bans comments that narrate the code. Remove this description and the similar one above `ConvertJob`; keep the comments explaining timing and buffer ownership. This repository requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Moves macOS camera preview conversion off the main thread.

Camera preview frames now convert off the main thread.

richiemcilroy explicitly described raw painting after blur is switched off mid-conversion as intentional in the PR description.

_{Reviews (1) · Last reviewed commit: "perf(desktop-gpui): convert camera previ..." · Reviewed by Greptile}

vercel · 2026-10-20T12:33:18Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
cap-status	Ready	Preview	Oct 20, 2026 7:27pm UTC

capy-ai

Capy found 2 potential issues (1 high, 1 medium).

View 1 other finding in Capy.

capy-ai · 2026-10-20T12:35:24Z

 /// `release_blur_resources` behaviour (`camera.rs:1477-1484`).
 #[cfg(target_os = "macos")]
+/// A camera frame and the preview settings it is converted with.
+#[cfg(target_os = "macos")]


BlurBridge is no longer gated to macOS

High Severity

The unchanged BlurBridge declaration below this insertion loses its platform guard: the original #[cfg(target_os = "macos")] now applies to ConvertJob instead. BlurBridge still references camera_blur::BlurJob, but both the import here and the module declaration in main.rs are macOS-only. Windows and Linux builds therefore fail to resolve camera_blur. Restore a macOS guard on BlurBridge.

Addressed in the latest commits:

BlurBridge has its doc comment and #[cfg(target_os = "macos")] back; Windows and Linux build again.

Each ConvertJob carries ConvertSettings (camera preview epoch, blur mode, mirroring, rebuild counter) captured when the frame was handed off. converted_arrived drops any result whose settings no longer match the window's current ones, so a frame converted for the previous camera, or for the blur/mirror state before a toggle, is never painted or sent to the blur worker. The next frame arrives within a frame interval.

The narrating comments on ConvertJob and converted_arrived are removed; the comments on buffer ownership and timing stay.

capy-ai · 2026-10-20T12:35:24Z

+        window: &mut Window,
+        cx: &mut Context<Self>,
+    ) {
+        let Some(epoch) = Feeds::global(cx).read(cx).camera_preview_epoch() else {


Converted results lose their originating camera epoch

Medium Severity

If conversion or result delivery is delayed across a camera switch, this check accepts an old frame whenever the replacement camera has become ready. deliver_camera_frame checked freshness before enqueueing, but neither ConvertJob nor its result preserves that epoch or timestamp. The raw path then calls set_frame, which stamps the old image with the current selection revision; the blur path likewise submits it under the new epoch, bypassing its stale-output guard. This can display the previous camera as the newly selected camera. Carry the originating epoch through conversion and reject mismatched results before painting or submitting blur work.

Addressed in the latest commits:

BlurBridge has its doc comment and #[cfg(target_os = "macos")] back; Windows and Linux build again.

Each ConvertJob carries ConvertSettings (camera preview epoch, blur mode, mirroring, rebuild counter) captured when the frame was handed off. converted_arrived drops any result whose settings no longer match the window's current ones, so a frame converted for the previous camera, or for the blur/mirror state before a toggle, is never painted or sent to the blur worker. The next frame arrives within a frame interval.

The narrating comments on ConvertJob and converted_arrived are removed; the comments on buffer ownership and timing stay.

greptile-apps · 2026-10-20T12:37:25Z

+            return;
+        }
+    }
+}


Windows and Linux builds fail

The insertion moves the existing macOS guard onto ConvertJob, leaving BlurBridge unguarded. BlurBridge still names camera_blur::BlurJob, but camera_blur only exists on macOS. Windows and Linux builds now fail to resolve that type. Restore #[cfg(target_os = "macos")] immediately before BlurBridge.

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop-gpui/src/camera_window.rs Line: 1503 Comment: **Windows and Linux builds fail** The insertion moves the existing macOS guard onto `ConvertJob`, leaving `BlurBridge` unguarded. `BlurBridge` still names `camera_blur::BlurJob`, but `camera_blur` only exists on macOS. Windows and Linux builds now fail to resolve that type. Restore `#[cfg(target_os = "macos")]` immediately before `BlurBridge`. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Addressed in the latest commits:

BlurBridge has its doc comment and #[cfg(target_os = "macos")] back; Windows and Linux build again.

Each ConvertJob carries ConvertSettings (camera preview epoch, blur mode, mirroring, rebuild counter) captured when the frame was handed off. converted_arrived drops any result whose settings no longer match the window's current ones, so a frame converted for the previous camera, or for the blur/mirror state before a toggle, is never painted or sent to the blur worker. The next frame arrives within a frame interval.

The narrating comments on ConvertJob and converted_arrived are removed; the comments on buffer ownership and timing stay.

greptile-apps · 2026-10-20T12:37:25Z

+        let failures_before = self.effect_failures;
+        // The blur mode may have been switched off (or failed) while this
+        // frame was converting; follow the current state.
+        let blur_mode = blur_mode.filter(|_| self.active_blur_mode().is_some());


Blur misses queued frames

If the user enables blur while a frame submitted with blur_mode: None is still converting, this filter preserves None. paint_raw then displays that unblurred frame after blur has been enabled. The old synchronous path could not cross a settings change this way. Drop results whose captured effect settings no longer match the current settings, or apply the currently selected effect before painting.

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop-gpui/src/camera_window.rs Line: 1960 Comment: **Blur misses queued frames** If the user enables blur while a frame submitted with `blur_mode: None` is still converting, this filter preserves `None`. `paint_raw` then displays that unblurred frame after blur has been enabled. The old synchronous path could not cross a settings change this way. Drop results whose captured effect settings no longer match the current settings, or apply the currently selected effect before painting. --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Addressed in the latest commits:

BlurBridge has its doc comment and #[cfg(target_os = "macos")] back; Windows and Linux build again.

Each ConvertJob carries ConvertSettings (camera preview epoch, blur mode, mirroring, rebuild counter) captured when the frame was handed off. converted_arrived drops any result whose settings no longer match the window's current ones, so a frame converted for the previous camera, or for the blur/mirror state before a toggle, is never painted or sent to the blur worker. The next frame arrives within a frame interval.

The narrating comments on ConvertJob and converted_arrived are removed; the comments on buffer ownership and timing stay.

greptile-apps · 2026-10-20T12:37:25Z

+    /// A frame came back from the conversion thread (or the inline fallback):
+    /// blur or paint it, and resize the window if its dimensions changed.


Comments repeat the code

This new comment restates what converted_arrived does. The repository guide reserves comments for non-obvious context and explicitly bans comments that narrate the code. Remove this description and the similar one above ConvertJob; keep the comments explaining timing and buffer ownership. This repository requirement must be satisfied before merging.

Context Used: AGENTS.md (source)

Prompt To Fix With AI

This is a comment left during a code review. Path: apps/desktop-gpui/src/camera_window.rs Line: 1941-1942 Comment: **Comments repeat the code** This new comment restates what `converted_arrived` does. The repository guide reserves comments for non-obvious context and explicitly bans comments that narrate the code. Remove this description and the similar one above `ConvertJob`; keep the comments explaining timing and buffer ownership. This repository requirement must be satisfied before merging. **Context Used:** AGENTS.md ([source](https://github.com/capsoftware/cap/blob/main/AGENTS.md)) --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Addressed in the latest commits:

BlurBridge has its doc comment and #[cfg(target_os = "macos")] back; Windows and Linux build again.

Each ConvertJob carries ConvertSettings (camera preview epoch, blur mode, mirroring, rebuild counter) captured when the frame was handed off. converted_arrived drops any result whose settings no longer match the window's current ones, so a frame converted for the previous camera, or for the blur/mirror state before a toggle, is never painted or sent to the blur worker. The next frame arrives within a frame interval.

The narrating comments on ConvertJob and converted_arrived are removed; the comments on buffer ownership and timing stay.

capy-ai

Capy found no issues.

…r effect setting

richiemcilroy · 2026-10-20T13:30:43Z

@greptileai review

capy-ai

Capy found no issues.

richiemcilroy · 2026-10-20T19:26:36Z

@greptileai review (the macOS guard on BlurBridge and stale-setting drops landed in 598b462 and 66cf6c3 after your first pass)

capy-ai

Capy found no issues.

blacksmith-sh · 2026-10-20T19:45:20Z

Found 2 test failures on Blacksmith runners:

Failures

Test	View Logs
`crash/recorder-kill/inproc/static-screen/static-screen`	View Logs
`crash/recorder-kill/oop/static-screen/static-screen`	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

perf(desktop-gpui): convert camera preview frames off the main thread

493f501

vercel Bot deployed to Preview – cap-status October 20, 2026 12:33

View deployment

vercel Bot deployed to Preview – cap-web October 20, 2026 12:34

View deployment

capy-ai Bot reviewed Oct 20, 2026

View reviewed changes

greptile-apps Bot reviewed Oct 20, 2026

View reviewed changes

fix(desktop-gpui): keep the blur bridge macOS-only

598b462

vercel Bot deployed to Preview – cap-status October 20, 2026 12:54

View deployment

vercel Bot deployed to Preview – cap-web October 20, 2026 12:56

View deployment

capy-ai Bot reviewed Oct 20, 2026

View reviewed changes

fix(desktop-gpui): drop preview frames converted for another camera o…

66cf6c3

…r effect setting

vercel Bot deployed to Preview – cap-status October 20, 2026 13:30

View deployment

vercel Bot deployed to Preview – cap-web October 20, 2026 13:31

View deployment

capy-ai Bot reviewed Oct 20, 2026

View reviewed changes

richiemcilroy added 2 commits October 21, 2026 03:26

chore(desktop-gpui): drop a comment that narrated the inline fallback

e0426fc

Merge main into perf/camera-preview-off-main

aa3c30f

vercel Bot deployed to Preview – cap-status October 20, 2026 19:27

View deployment

vercel Bot deployed to Preview – cap-web October 20, 2026 19:28

View deployment

capy-ai Bot reviewed Oct 20, 2026

View reviewed changes

		/// A frame came back from the conversion thread (or the inline fallback):
		/// blur or paint it, and resize the window if its dimensions changed.

Conversation

richiemcilroy commented Oct 20, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Change

Confidence Score: 3/5

Findings

Summary

Uh oh!

vercel Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

capy-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

capy-ai Bot Oct 20, 2026

Choose a reason for hiding this comment

BlurBridge is no longer gated to macOS

Uh oh!

richiemcilroy Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

capy-ai Bot Oct 20, 2026

Choose a reason for hiding this comment

Converted results lose their originating camera epoch

Uh oh!

richiemcilroy Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

richiemcilroy Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

richiemcilroy Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

richiemcilroy Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

capy-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

richiemcilroy commented Oct 20, 2026

Uh oh!

capy-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

richiemcilroy commented Oct 20, 2026

Uh oh!

capy-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

blacksmith-sh Bot commented Oct 20, 2026

Failures

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

richiemcilroy commented Oct 20, 2026 •

edited by greptile-apps Bot

Loading

vercel Bot commented Oct 20, 2026 •

edited

Loading