Anthropic
Anthropic launches Claude Computer Use and can control your computer just like human
Anthropic has launched a new Claude Computer Use capability, which controls the computer with a user prompt just like the web and mobile versions of the chatbot.
The computer use is designed to control the computer and conduct tasks on your behalf. These tasks could be related to your project, code development or even finding something for you online. While completing this user request, Claude will move the cursor, click buttons, and type text automatically.
Claude has shared three demos for this new feature.
The first demo shows how computer users look for the data on the screen and fill out the required details on a form. The user writes the following request:
“Please fill out the vendor request form for ‘Ant Equipment Co.’ using data from either the vendor spreadsheet to search portal tabs in window one. List & Verify each field as you complete the from in window two”
Right after, Claude starts completing the tasks in this request sequentially. Throughout these actions, the bot keeps the user updated with the next actions.
Coding
The second demo showed the computer use in action for website development with the following prompt:
“Please navigate to a new Chrome window, navigate to claude.ai, and once loaded, ask Claude to create a personal website with a 90s-style theme.”
Then the user shows how Claude starts a server for this HTML website file and fixes any run-time error.
Tasks
This third demo shows Claude performing some search for a user with the following prompt:
“My friend is coming to San Francisco and I want to watch the sunrise with him at the Golden Gate Bridge tomorrow morning. We’ll be coming from Pacific Heights. Could you find us a great viewing spot, check the drive time and sunrise time, and then set up a calendar event that gives us enough time to get there?”
We’re trying something fundamentally new.
Instead of making specific tools to help Claude complete individual tasks, we’re teaching it general computer skills—allowing it to use a wide range of standard tools and software programs designed for people. pic.twitter.com/42u8VeTvXd
— Anthropic (@AnthropicAI) October 22, 2024
Availability
Claude Computer Use is available on the API for developers with the new Claude 3.5 Sonnet. However, it’s still in early access and could fire some errors while completing any request.