Can AI actually control your computer like a human would?
Anthropic taught Claude to see your screen, move the mouse, click buttons, and type - just like a person sitting at your desk.
The answer
On October 22, 2024, Anthropic announced 'computer use' - a groundbreaking capability allowing Claude 3.5 Sonnet to interact with a computer's desktop environment by looking at screenshots, moving the cursor, clicking buttons, and typing text. Unlike previous models that couldn't provide precise screen coordinates, Claude can now identify and click specific UI elements across any application. Early adopters including Asana, Canva, DoorDash, and Replit began building workflows involving dozens or even hundreds of sequential steps. The capability launched in public beta on the Anthropic API, Amazon Bedrock, and Google Cloud's Vertex AI simultaneously. While still imperfect at tasks like scrolling and dragging, computer use represents a fundamental shift - AI can now operate any software with a graphical interface, not just tools with dedicated APIs.
Source: Anthropic
- computer use
- anthropic
- claude
- automation
More from AI & Machine Learning
- What radical question did Alan Turing ask in 1950 that launched the entire field of artificial intelligence?
- Can a machine fool you into thinking it's human - and does that mean it's truly intelligent?
- What happened at a small Dartmouth workshop in 1956 that gave artificial intelligence its name?
- How did a 1958 invention called the Perceptron convince people that machines could learn like brains?
- Could a 1966 chatbot really make people believe they were talking to a therapist?
- What was the first robot that could 'reason' about its own actions?