The NCSC has published new interim advice aimed at helping organisations manage the cyber risks associated with increasingly autonomous AI systems, as it works towards more formal guidance on the technology.

Agentic AI systems have the potential to automate complex workflows, reduce routine workloads and deliver significant productivity gains, but the National Cyber Security Centre (NCSC) has warned that organisations need to plan for cases where autonomous systems behave in unintended or unexpected ways.

The guidance follows several recent incidents involving AI models and agentic AI systems carrying out unsanctioned or unintended activity, prompting the NCSC to set out a series of practical measures around autonomy, sandboxing, monitoring and human oversight.

The agency said AI cybersecurity remains a rapidly evolving area and that effective practices are likely to change as the technology matures and the evidence base develops.

The NCSC has been researching and experimenting with agentic AI and is currently working with partners on formal guidance which will ultimately supersede the interim advice. In the meantime, the recommendations are intended to help organisations make informed decisions about deploying autonomous systems securely.

The guidance is primarily aimed at organisations building environments in which AI agents are expected to operate with significant levels of autonomy, particularly where unintended actions could arise from the instructions given to an agent, the tools available to it or the systems it can access.

A central recommendation is that organisations should first determine how much autonomy an AI agent actually needs.

The NCSC noted that not every deployment requires the same level of independence. While some agents may simply make suggestions or carry out tightly constrained, low-risk tasks, others may be given access to production environments and allowed to make decisions with little or no human intervention.

As autonomy increases, so too does the potential impact if an agent malfunctions, accesses information it should not or acts beyond its intended scope. Organisations should therefore determine both the degree of autonomy required and the level of risk they are prepared to accept before deciding which controls to implement.

The NCSC also warned against relying solely on safeguards already built into AI models and associated services. While these can offer a baseline level of protection, they may be bypassed or prove insufficient in higher-risk environments.

Where the consequences of failure exceed an organisation’s risk tolerance, additional safeguards should be introduced, potentially including classifiers, deterministic provers and wider technical controls. Regardless of the protections built into a model, the NCSC said deployments should be supported by robust observability, operational monitoring and incident response procedures.

Human Oversight and Sandboxing

Before deploying an autonomous agent, organisations are advised to carry out threat modelling and clearly document the activities that sit inside and outside the intended scope of the system.

This should include identifying any ‘red lines’ the agent must not cross and considering the instructions it receives, alongside the networks and services it can access directly or indirectly.

The NCSC warned: “Remember that an AI agent is not human. It does not have common sense or human traits, and may interpret instructions and goals in literal or unexpected ways.”

Instructions should also make clear what an agent is expected to achieve, which actions it is permitted to take and when it should stop and seek human approval.

For longer-running tasks, organisations should consider how agents handle context over time, including whether important constraints could be lost when context windows are compressed. Critical instructions may therefore need to be repeated.

However, the agency stressed that prompting should form only one part of the security controls surrounding autonomous systems.

“However, do not rely on prompting alone. You should combine prompts with technical and operational controls to provide defence in depth.”

The appropriate level of human involvement will also depend on the potential consequences of an agent acting unexpectedly. The guidance distinguishes between human-in-the-loop systems, where people approve actions before they occur; human-on-the-loop systems, where people monitor activity and can intervene; and human-out-of-the-loop deployments that operate without human review.

For higher-risk use cases, the NCSC recommends maintaining human oversight alongside technically enforced controls, with named individuals or groups responsible for agent activity and real-time monitoring used to identify concerning behaviour.

Sandboxing also forms a major part of the guidance.

The NCSC said AI agents should always operate within sandboxed environments that control the local and network resources with which they can communicate. For particularly high-risk activities, it said the most robust approach is to use an isolated and disconnected environment with pre-downloaded tools and information.

Where access to external systems is necessary, organisations should tightly define the boundaries of the sandbox and consider everything an agent could access or influence, including network resources, compute infrastructure, credentials and data.

The agency noted that autonomous AI tools designed to interact with large numbers of systems and services create a greater potential ‘blast radius’ if they malfunction or are compromised.

Network access should therefore be denied by default where possible, with only required connections permitted through allowlists. Where internet access is necessary and allowlists cannot provide enough control, protocol- or service-aware proxies can be used to require manual approval for communications outside agreed boundaries.

The NCSC sets out a four-level maturity model for network restrictions, ranging from unrestricted network access at the lowest level to no external network access and a locally hosted model at the highest.

A similar model is proposed for compute isolation, ranging from an agent running alongside other workloads without isolation to dedicated hardware separated from other workloads.

The guidance also warns that AI agents could potentially discover and exploit configuration weaknesses or vulnerabilities in technical controls, creating the possibility of a sandbox escape. Organisations are therefore encouraged to use multiple layers of isolation and regularly validate their configurations.

Access to credentials should similarly be restricted. Each agent should be given its own unique identity, distinguishing it from human users and individual systems, and should receive only the permissions it requires to perform its assigned task.

The NCSC said credentials available to an AI agent form part of its potential ‘blast radius’ and can include API keys, OAUTH grants, SSH keys and authenticated sessions. Where possible, credentials should have the shortest viable lifetime, while proxies may be used to inject credentials into requests without exposing them directly to an agent.

Recommended reading

• OpenAI and Anthropic Called to White House Over AI Hacking Risks
• Anthropic and OpenAI Models Implicated in New Cyber Testing Breaches
• “Sloppy” OpenAI Hack Hit More Than Hugging Face
• OpenAI Model Goes Rogue In Testing, Hacks Hugging Face

Monitoring is another key component of the recommendations.

Organisations should be able to access and analyse telemetry relating to an agent’s actions both in near real time and retrospectively. The NCSC said this should include agent transcripts and traces alongside logs from the wider sandbox environment, including access logs, proxy activity and network traffic.

Those logs should be appropriately protected and, where possible, made immutable so they can be trusted during an investigation.

Agentic AI activity should also be treated in much the same way as user activity and incorporated into security operations and incident response processes. Organisations may initially want to run experiments during office hours, when greater human oversight is available, before expanding autonomous activity to overnight or weekend periods once controls have been shown to be effective.

The NCSC is additionally encouraging organisations to make autonomous activity easy for third parties to attribute. Emerging approaches include using IP addresses that support reverse lookups and adding identifying information to outbound communications.

Finally, organisations should maintain the ability to immediately halt an agent if something goes wrong.

“If an incident is detected or reported, you should always be able to ‘pull the plug’ and halt autonomous AI agent activity immediately.”

The NCSC said this may require more than simply stopping the AI process itself, with controls designed to rapidly restrict network access and interrupt communications between AI agents and the underlying model infrastructure.