In a nutshell: Organizations must fundamentally adapt their security policies to AI system autonomy and establish differentiated governance controls for each maturity stage (Assistant, Agent, Operator).
As companies expand AI tools from mere assistants to autonomous agents, security and governance practices are falling behind. The previous control model based on the assistant principle no longer suffices when AI systems operate independently with access rights and sensitive data.
Since ChatGPT’s release nearly four years ago, the gap has widened between the proliferation of AI tools and mature security structures. Users routinely upload corporate data, financial documents and health information into large language models, often without formal approval or security review. To date, the risk has persisted in many organizations without catastrophic consequences — yet this is no evidence of sound practices.
The transition from assistive to autonomous AI use fundamentally changes the risk profile. Stephen Wilson, Field Chief Technology Officer at HashiCorp (IBM), distinguishes three maturity stages: (1) AI as Assistant — humans remain in control and validate every output; (2) AI as Agent — the system performs tasks partly autonomously, for example by delegating work between multiple AI systems; (3) AI as Operator — largely autonomous systems make decisions with minimal human oversight. The problem: organizations are not updating their governance models accordingly.
Even in Assistant mode, the dangers are already evident. Privileged users easily enter API keys, database credentials or access tokens into prompts without realizing that the LLM context windows store this sensitive data and could relay it to third parties. This requires strict identity separation: humans retain their privileges, the machine identity receives only minimal rights necessary for the task.
Agents require a tiered model for access control. Different agents need different permissions; each agent needs a traceable identity and an audit trail. These agent identities must be managed, their reliability for core tasks continuously assessed. Only then can an organization safely depart from human supervision and have agents work sequentially — for example, a writing agent handing off to an editorial agent.
The greatest challenge lies in the next step: AI Operators that autonomously make business-critical decisions or access systems. There is as yet no established governance norm here. It will be critical to set clearly defined boundaries for when an Operator may intervene, what escalation paths exist, and who ultimately bears responsibility when an autonomous system acts on behalf of the organization.
Source: www.csoonline.com · Published 6 July 2026
Lumi AI News — AI-assisted curation pursuant to Article 50 EU AI Act. Paraphrase and classification by Lumi News Pipeline v1.7.3.