Lack of Evidence Is Not Evidence of AI Safety

The United Nations Independent International Scientific Panel on Artificial Intelligence has published its first thematic brief, examining AI agents, misalignment and the risk of losing human control. The brief draws lessons from an incident in which experimental OpenAI models reached Hugging Face production systems during an evaluation.

The panel argues that some advanced-AI risks present the kind of decision problem the precautionary principle was designed to address: potential harm may be catastrophic or irreversible even while its probability and causal mechanisms remain scientifically uncertain.

This is scientific and policy advice, not binding international law. It does not establish that catastrophic loss of control is likely or imminent. Nor does it justify treating every AI deployment as an existential threat.

Its practical message for enterprise governance is more immediate: when consequences could be severe, incomplete evidence should not automatically justify weaker safeguards or faster deployment.

Many risk methods ask reviewers to estimate likelihood and impact. The result is often plotted on a matrix and converted into a red, amber or green rating.

That structure is useful when an organization has credible evidence, comparable events and stable operating conditions. It becomes less reliable when systems are new, capabilities change quickly, vendors disclose limited information or the deployment allows an agent to take long sequences of actions across connected tools.

In those conditions, reviewers may still feel pressure to enter a likelihood score. A lack of incidents can be interpreted as low probability. A lack of evidence can quietly become evidence of safety.

That is a category error.

"We have not observed this failure" is different from "we have reliable evidence that this failure is unlikely." The first statement may reflect limited testing, weak monitoring, short operating history or restricted access to vendor information. Governance must preserve that distinction.

Clariantix recommends adding an Uncertainty and Severity dimension to AI risk reviews rather than forcing every unknown into a conventional likelihood score.

The dimension should assess seven factors.

A precautionary approach is sometimes framed as a choice between stopping innovation and accepting risk. That is too simplistic.

Proportionate precaution can mean narrowing the scope of deployment, withholding sensitive data, requiring human approval for consequential actions, limiting network destinations, using read-only access, strengthening logging or delaying expansion until independent evidence is available.

The response should reflect both uncertainty and severity. High uncertainty around a low-impact, reversible use may justify monitoring and a controlled pilot. High uncertainty combined with potentially severe, difficult-to-detect or irreversible harm should trigger stronger approval, testing and containment.

This approach protects innovation by making the path to responsible expansion explicit. Instead of a vague "not yet," the governance decision can state what evidence and safeguards are required for the next stage.

Risk analysis has little value if it does not change system design.

If the maximum credible impact depends on access to production code, the deployment may begin in a sandbox with no production credentials. If the risk arises from external communications, outbound destinations can be allowlisted and messages held for approval. If harm could result from financial transactions, the agent can be subject to monetary limits, dual authorization and real-time alerts.

The same principle applies to professional services. An agent reviewing project information may operate in read-only mode. An agent drafting technical content may be prevented from issuing, stamping or transmitting a deliverable. An agent conducting research may be restricted from accessing client-confidential systems or licensed material until permission is verified.

Governance becomes effective when the risk decision determines the agent's actual permissions, monitoring and stopping conditions.

Senior leaders do not need to debate speculative scenarios in the abstract. They should ask concrete questions.

As AI systems gain the ability to plan, use tools and act across multiple environments, organizations will encounter risks that cannot be reduced to precise probabilities. That does not mean decisions must be driven by fear. It means the evidence limits should be stated honestly and reflected in the conditions of deployment.

The burden of proof should rise with autonomy, connectivity, data sensitivity, decision consequence and irreversibility. A system should not receive broader authority simply because serious failure has not yet been observed.

Lack of evidence is not evidence of safety. It is evidence that uncertainty remains, and material uncertainty belongs inside the governance decision.