AI Agents Challenge Corporate Governance in Financial Transactions

The technology sector’s famous phrase, “Move fast and break things,” has taken on new urgency with the advent of autonomous artificial intelligence (AI) agents.

Key Points:

• Governance Revolution: Kathryn McCall, Chief Legal and Compliance Officer at Trustly, emphasizes that AI agents represent a governance revolution, not just a technical upgrade.
• Autonomous Actions: AI agents are now executing financial transactions, booking flights, and signing documents, raising concerns about privacy, security, and legality.
• Risks and Controls: McCall advocates for “bounded autonomy,” including layered governance, sandbox environments, and human oversight at critical decision points.

Compliance Is No Longer Reactive

• Non-Deterministic Behavior: AI agents introduce unpredictable outputs and evolving capabilities, requiring new security measures like prompt injection defenses and audit logs.
• Human Accountability: McCall stresses the need for human-readable reasoning and forensic replay to maintain accountability.

Regulatory Vacuum Demands Corporate Responsibility

• Patchwork Regulations: Current regulations like PCI-DSS, Gramm-Leach-Bliley Act, and GDPR must be reinterpreted for AI agents.
• Ethical Boundaries: McCall warns against opaque decision-making, urging companies to design explainability into AI systems from the start.

“The chief legal officer’s role becomes even more important to translate complex AI behavior and inform the organization about legal exposure,” McCall said.

For more AI coverage, subscribe to the daily AI Newsletter.