The National Cyber Security Council and the AI Security Institute  (AISI) have issued support for open sourcing solutions to AI safeguard bypass threats.

The UK cyber and AI authorities warned of the growing threats frontier AI systems pose when they are not properly defended.

Cyber-criminals have grown adept at bypassing security safeguards in common large language models such as ChatGPT, Gemini, Claude, and Llama. These guardrails typically stop users from employing AI chatbots from creating illegal material, such as malicious code or instructions on how to create a bomb.

These threats seem to be increasing, as researchers discovered what may be the first known example of AI-generated malware which used OpenAI’s technology.

The NCSC and AISI have supported the use of industry collaboration and vulnerability disclosure to conjoin efforts to fight these threats.

While crowdsourcing can be a positive way in mitigating threats and sharing defence techniques, the NCSC and AISI provided guidance on best practices to maintain security in these collaborations.

Disclosure programmes can follow a range of best practices as suggested by the NCSC.

A good disclosure programme should have a clearly defined scope, providing specific definitions for prompts to not execute in a clear and communicative way.

The NCSC says that developers should only launch their disclosure programme after they have conducted internal reviews and fixed any discovered weaknesses. Similarly, the duration of the programme should reflect and support its goals.

Recommended reading

• Researchers Discover “First Known AI-powered Ransomware”
• CISOs Seeing More Impact from AI Cyber Threats
• AI in Cybersecurity | Navigating the Promises and Risks

Further, developers must make effective reports by tracking what they found, and providing effective tools to share information.

The NCSC does say that just because an organisation does have a disclosure programme does not necessarily mean its model is safe or secure.

The cyber authority also says that there is still a lot of open questions regarding the role of public programmes in maintaining AI security.

Things to consider include if other areas of cybersecurity can offer insights into detection bypasses, and how do approaches to incentives need to change or expand regarding people who work in AI.

Therefore, the NCSC and AISI are urging researchers to explore questions such as how to mitigate safeguarding weaknesses once exposed, how to handle cross-model attacks, and how to judge the severity of safeguard bypass weaknesses.