I found an open-source project that truly allows AI to "use its hands" to work


cua, 16.7k stars, not a chat robot, but the infrastructure that enables AI to directly control a computer
The core idea is simple: provide AI with a sandbox environment where it can click the mouse, type on the keyboard, and see the screen, allowing it to operate software like a human
Several subprojects have clear divisions of labor:
✅ Cua Driver: macOS background operation driver, AI clicks buttons and types without stealing your cursor, doesn't switch your window, silently works in the background
✅ Cua Sandbox: unified API control for Linux/macOS/Windows/Android, runs in the cloud or locally
✅ CuaBot: provides sandbox environments for coding agents like Claude Code and Cursor, ready to use out of the box
✅ Cua-Bench: benchmark tests to evaluate agent operation capabilities, runs datasets like OSWorld and ScreenSpot
✅ Lume: runs macOS virtual machines on Apple Silicon, performance close to native
Looking at the code, it's a mix of Rust, Swift, and Python, with a pretty hardcore architecture. If you're interested in agent automation, desktop task orchestration, or just curious about "how AI uses computers," this is worth checking out the source code
The project link is in the comment section 👇
View Original
post-image
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Repost
  • Share
Comment
Add a comment
Add a comment
No comments
  • Pinned