In brief: A position paper argues that AI agent security should not be anchored in training, but as an enforceable runtime contract with preventive and evidence-based controls in the harness, backed by an analysis of 52 security incidents and an 8-12x publication imbalance between training-time and deployment-time safety.
A new position paper argues that safety in autonomous AI agents cannot be achieved through training methods such as RLHF, DPO, or Constitutional AI alone. Instead, safety should be anchored as an enforceable runtime contract within the execution framework (harness), since agents execute code, modify files, send messages, and manipulate databases.
The paper, published at arxiv.org/abs/2608.11274, criticizes the prevailing paradigm in which AI safety is primarily produced as a property of the model during training. The authors argue that this approach is structurally insufficient for autonomous agents, since these agents perform real actions with real consequences during operation. Instead, they propose a two-part runtime contract: a preventive side that blocks dangerous actions in advance through sandboxes, permission gates, output filters, and trajectory monitors, as well as an evidence-based side that requires robust proof of successful actions before a task is marked complete – such as test runs, log records, file diffs, or citation grounding.
To support this thesis, the paper draws on four empirical studies with raw data included in accompanying JSON files: an analysis of 52 documented security incidents involving AI agents and LLMs, an audit of false success reports (“false completions”) comprising 31 undisputed core cases and one contested example case, a schema audit of the trajectories of 12 public agent systems and harnesses, and a title analysis of all 28,560 papers accepted at NeurIPS, ICML, and ICLR between 2023 and 2025. The latter reveals a pooled imbalance of a factor of 8 to 12 between publications on training-time safety and those on deployment-time safety.
For engineering teams bringing agent systems into production, the paper provides concrete building blocks: a formalized Agent Trajectory Schema with an evidence chain, as well as a compositional gating proposition built on standard monitor composition. The authors draw parallels to computer security and the experimental sciences, both of which have historically converged toward runtime contracts with preventive and evidence-based elements. Their central thesis is that the relevant unit of safety in agent-based AI is not the model itself, but the trajectory with verifiable evidence – a starting point for designing harness architectures, monitoring pipelines, and acceptance criteria in production agent systems.
Source: arxiv.org · Published August 10, 2026
Lumi AI News — AI-assisted curation pursuant to Art. 50 EU AI Act. Paraphrasing and classification by Lumi News Pipeline v1.8.3.