AI OS Agents Pose Security Risks as Tech Giants Accelerate Development

New research reveals the accelerating development of OS Agents—AI systems capable of autonomously controlling computers and mobile devices by interacting with their interfaces. The comprehensive survey by researchers from Zhejiang University and OPPO AI Center highlights both the potential and risks of this emerging technology.

Tech Giants Race to Deploy AI Agents

Major companies are rapidly commercializing this technology:

• OpenAI launched Operator
• Anthropic released Computer Use
• Apple enhanced Apple Intelligence
• Google unveiled Project Mariner

These systems work by analyzing screenshots, understanding interfaces through computer vision, and executing precise actions like clicks and form entries. The most advanced can handle multi-step workflows across applications.

Critical Security Vulnerabilities Emerge

The research identifies serious risks:

• Web Indirect Prompt Injection: Hidden web page instructions can hijack agent behavior
• Environmental injection attacks: Malicious content can trigger unauthorized actions
• Limited existing defenses for OS Agent-specific threats

Traditional security models fail against these novel attack vectors, creating urgent challenges for enterprise adoption.

Current Limitations and Future Potential

While promising, current systems show mixed performance:

• 50%+ success rates on simple tasks
• Struggles with complex, context-dependent workflows
• Personalization remains a key challenge

The technology excels at routine tasks but isn't yet ready to replace human judgment in sophisticated scenarios.

As development accelerates, the window to establish proper security frameworks is narrowing. The survey maintains an open-source repository tracking progress in this transformative field.