Microsoft Copilot Studio Introduces UI Automation with Computer Use Feature

Microsoft has announced a groundbreaking addition to Copilot Studio: the computer use feature, enabling AI agents to interact directly with websites and desktop applications. This innovation marks a significant leap in automation, allowing agents to perform tasks like clicking buttons, selecting menus, and typing into fields—just as a human would.

Key Features of Computer Use

• UI Interaction: Agents can now navigate and manipulate graphical interfaces, even without APIs.
• Adaptive Reasoning: The tool adjusts to changes in real time, ensuring uninterrupted workflows.
• Enterprise-Grade Security: Built on Microsoft’s compliance frameworks, it keeps data within Microsoft Cloud boundaries.
• Cross-Browser Support: Works with Edge, Chrome, and Firefox.

High-Value Use Cases

• Automated Data Entry: Streamlines large-scale data input with minimal errors.
• Market Research: Automates data collection from online sources for analysis.
• Invoice Processing: Extracts and inputs invoice data into accounting systems.

Transforming RPA

Traditional Robotic Process Automation (RPA) often struggles with dynamic interfaces. Copilot Studio’s computer use feature addresses this by:

• Real-Time Adaptation: Adjusts to UI changes without breaking workflows.
• Natural Language Commands: No coding required—describe tasks in plain language.
• Full Visibility: Makers can review activity logs, including screenshots and reasoning steps.

How to Join the Preview

Interested users can apply for the limited preview. Microsoft will also showcase the feature at Microsoft Build 2025—register here to attend.

This release follows recent Copilot Studio updates, including deep reasoning and agent flows.