The National Cyber Security Centre (NCSC) has shared critical insights cautioning cybersecurity professionals against comparing prompt injection and more classical application vulnerabilities classed as SQL injection.

A new blog advises that, contrary to first impressions, prompt injection attacks against generative artificial intelligence applications may never be totally mitigated in the way SQL injection attacks can be.

Unlike SQL mitigation techniques, which hinge on enforcing a clear separation between data and instructions, prompt injection exploits the inability of Large Language Models (LLMs) to distinguish between the two.

Without action addressing this misconception, the NCSC warns, websites risk falling victim to data breaches exceeding those seen from SQL injection attacks in the 2010s, impacting UK businesses and citizens into the next decade.

Backing proactive adoption of cyber risk management standards, the NCSC challenges claims that prompt injections can be ‘stopped’.

Instead, it suggests efforts should turn to reducing the risk and impact of prompt injection and driving up resilience across AI supply chains.

As AI technologies become embedded in more UK business operations, the NCSC calls on AI system designers, builders and operators to take control of manageable variables, acknowledging that LLM systems are “inherently confusable” and their risks managed in different ways.

“Confused deputy vulnerabilities are where a system can be coerced to perform a function that benefits the attacker, typically where a privileged component is coerced into making a privileged request on behalf of a less-privileged attacker,” the blog says.

Recommended reading

• Perplexity AI Makes $34.5bn Bid for Google Chrome
• Is OpenAI’s Atlas Browser ‘Inherently Dangerous’?
• Report: Only 2.8% of Websites Protected Against AI Bots

These cannot be mitigated, but can be reduced by secure design principles, developer and organisational awareness, appropriate data marking, and better monitoring.

The blog discussed many techniques in making it harder for an LLM to act on instructions that are included within data, such as marking data sections to be separate from instructions.

Developers can also block commonly used injection phrases like “ignore previous instructions,” though this is limited in scope as LLM prompts can be so varied.

As developers and organisations seek solutions, the NCSC warns to veer away from vendors and services that say they can ‘stop’ prompt injection, and opt for those that can explain how they can ‘reduce’ the risk.