The Cyber and Fraud Centre – Scotland has expanded its cyber security testing services to help organisations identify vulnerabilities introduced by AI-powered chatbots and other AI functionality integrated into websites and web applications.

The new AI and Web Application Security Assessment service will see the Centre’s specialist team assess traditional web application vulnerabilities alongside security risks specific to the way AI has been implemented.

As organisations increasingly use AI-powered chatbots and virtual assistants for customer service, online support and access to information, these tools can also create new opportunities for attackers if they are not configured securely.

Rather than relying solely on traditional hacking techniques, attackers may attempt to manipulate an AI assistant through carefully constructed prompts, potentially causing it to ignore its instructions, disclose information it should not reveal or interact with connected systems in unintended ways.

Testing will assess areas including prompt injection, sensitive information disclosure, access controls, guardrails and the systems and data sources connected to an AI application.

Recommended reading

• OpenAI and Anthropic Called to White House Over AI Hacking Risks
• “Sloppy” OpenAI Hack Hit More Than Hugging Face
• OpenAI Model Goes Rogue In Testing, Hacks Hugging Face

Thaïs Ramdani, Cyber and Fraud Centre – Scotland’s lead pentester comments: “Organisations are understandably keen to explore what AI can do for their customers and help with efficiencies in teams. But security needs to be part of that conversation.

“Adding an AI assistant to a website isn’t simply adding another customer service tool. Depending on how it’s been configured, the AI assistant could have access to confidential information or documents, which risk being exposed.

“Our team essentially approach the technology from an attacker’s point of view – what can we make it do that it wasn’t intended to do? Finding these weaknesses through controlled testing gives our clients the opportunity to address them before someone else finds them.”

The service combines testing with practical reporting – organisations will receive a breakdown of vulnerabilities identified during testing, their potential business impact and practical recommendations to address them.