AIMPACT News, April 24 (UTC+8), according to Beating monitoring, the open-source computer-use infrastructure project Cua released cua-driver, a native macOS driver that allows any agent to control Mac applications in the background. When the agent clicks, types, or takes screenshots, the user's cursor does not move, the focus does not change, and macOS does not switch desktops across Spaces. The core technology comes from reverse engineering Apple's private framework SkyLight. Conventional synthetic events posted via CGEventPost through the HID event stream move the cursor; \CGEvent.postToPid\ can send events directly but Chromium's render process filters them out. cua-driver uses SkyLight's SLEventPostToPid to send events through WindowServer's trusted channel, bypassing HID, so Chromium can also receive them. Window activation borrows from the window manager yabai: using SLPSPostEventRecordTo to only toggle the AppKit activation state of the target application without raising the window level, avoiding triggering Spaces following. For Electron apps (Slack, VS Code, Discord, etc.), it uses the unpublished _AXObserverAddNotificationAndCheckRemote to keep the accessibility tree updated even when the window is obscured. cua-driver provides three capture modes: ax mode returns only the accessibility tree, no screen recording permission needed; vision mode returns only screenshots; som mode (default) returns both, allowing the agent to click via element index or pixel coordinates. The driver supports the MCP protocol and can be connected to clients like Claude Code and Cursor, or called via command line. Two known limitations: right-clicking on Chromium web content does not work, and Canvas-based applications (Blender, Unity, game engines) still require brief foreground activation. After OpenAI acquired the former Apple Shortcuts team Sky, Codex first launched a background computer-use feature but did not open-source it. Francesco Bonacci of Cua stated that background computer-use drivers should be common infrastructure rather than a proprietary feature of a single product. (Source: BlockBeats)

View Original

This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.

Reward
like
Comment
Repost
Share

Comment

Add a comment

No comments

Trending Topics
View More
#
gStocksTokenizedStocksLive
4.75M Popularity
#
WeakNFPShakesRateHikeOdds
1.04M Popularity
#
PredictWorldCup🇧🇷vs🇳🇴
200.44K Popularity
#
ETHBreaks1700
152.56M Popularity
#
MetaSellsComputeTriggersChipSlump
1.39M Popularity

Pinned

Sitemap

Cua open-source macOS background computer-use driver: reverse-engineering Apple’s private framework, an agent controls applications without stealing the cursor

Trending Topics

gStocksTokenizedStocksLive

WeakNFPShakesRateHikeOdds

PredictWorldCup🇧🇷vs🇳🇴

ETHBreaks1700

MetaSellsComputeTriggersChipSlump

Pinned