The Information Commissioner’s Office (ICO) has urged organisations to strengthen their cyber resilience as criminals increasingly use artificial intelligence to carry out faster, more advanced and harder-to-detect attacks.

In a blog, Ian Hulme, the ICO’s Interim Executive Director for Regulatory Supervision, warned that cyber-criminals are using AI across a growing range of attack methods, including AI-generated phishing emails, deepfake social engineering, automated vulnerability scanning, AI-powered malware, credential stuffing, data poisoning, and indirect prompt injection attacks.

The regulator said the evolving threat landscape means cybersecurity must become “a shared responsibility across every part of the economy”, with organisations expected to take proactive steps to prepare for emerging threats.

“As the data protection regulator, we can provide clear expectations and practical support, but all organisations must take proactive steps to prepare themselves for emerging threats,” Hulme said.

“By investing in cyber resilience and ensuring appropriate security measures are in place, you can build public trust and confidence in how your organisation protects the personal data you hold.”

According to the blog, AI-enhanced phishing can allow attackers to generate highly convincing, personalised messages that impersonate colleagues, clients, or trusted suppliers. Deepfake audio and video can also be used to impersonate colleagues or IT staff to trick employees into resetting credentials or granting system access.

The regulator also highlighted the risk of AI tools being used to rapidly scan systems, identify weaknesses, and launch targeted attacks, while AI-powered malware can adapt its behaviour in real time to evade detection by conventional antivirus and security tools.

The ICO said credential stuffing and password attacks can also be accelerated by AI, making weak or reused passwords more vulnerable. Where organisations use AI models in their services, attackers may also attempt data poisoning, corrupting training data or manipulating model outputs to cause harm or extract sensitive information.

The blog also drew attention to indirect prompt injection attacks, where malicious instructions are embedded in external content that an AI system processes and misinterprets as legitimate commands. This includes tool poisoning, where instructions are hidden within the metadata of tools that an AI agent interacts with.

Five Steps to Strengthen AI Cyber Resilience

Hulme noted that the National Cyber Security Centre has updated its Cyber Assessment Framework to reflect growing AI threats, with a stronger emphasis on organisations understanding how criminals may use AI technologies so they can respond effectively.

The ICO said organisations should begin by understanding the risks they face, but warned that basic security measures remain essential. The regulator said most successful cyber attacks exploit fundamental security failures, and that organisations using or storing personal data are expected to have the five technical controls outlined in the Cyber Essentials scheme in place, as well as actions set out in the Cyber Governance Code of Practice.

However, the ICO warned that foundational security alone is not enough when dealing with AI-powered threats. Organisations should apply layered defences, ensuring that if one control fails, others are in place to contain the damage.

The regulator also urged organisations to maintain strong patching and updating processes, noting that AI tools can identify and exploit known vulnerabilities at speed. This means available security fixes should be applied in a timely manner.

Access control was also identified as a major area of risk. The ICO said weak access points are a primary target for cyber attacks, including through third-party suppliers. Organisations should implement multi-factor authentication on all remote access, admin accounts, and email, while enforcing strong password policies.

The blog also advised organisations to apply the “principle of least privilege”, ensuring users, systems, and applications can only access what they genuinely need. Privileged accounts should be audited regularly and access should be removed as soon as it is no longer required.

Where AI is integrated into access control systems, the ICO said organisations must understand the privacy and security implications of any behavioural and identity data being used.

Supply chains were also highlighted as a potential weakness. The regulator said organisations should map what third parties can access, hold suppliers to appropriate security standards, include security requirements in contracts, and conduct proportionate due diligence.

The ICO also called for improved detection, monitoring, and incident response. Organisations should implement comprehensive security monitoring for suspicious activity, including unusual login patterns, unexpected data transfers, and abnormal API usage. They should also regularly identify weaknesses through vulnerability scanning and penetration testing.

The regulator acknowledged that AI can also be used defensively, helping organisations flag and contain threats at speed. However, it said these systems should operate within a clear framework of human oversight and accountability to prevent misuse and exploitation by malicious actors.

Incident response plans should also be maintained and tested regularly, with staff clear on their roles and reporting contacts. The ICO said key contact details and offline copies of critical documentation should remain accessible if systems are compromised.

The regulator placed particular emphasis on protecting personal data, warning that AI-powered attacks increasingly target data that can be used to facilitate further attacks.

Recommended reading

• UK Unveils New Cyber Bill to Protect Critical Services
• UK Public Wants More Regulation to Feel Okay About AI
• UK Gov Launches £5M Challenge Fund to Strengthen AI Security

Under UK GDPR, organisations are required to implement appropriate technical and organisational measures to protect personal data. The ICO said this can include data minimisation and storage limitation, regular audits of what personal data is held, where it is stored and who has access to it, and staff training to recognise AI-powered social engineering attacks such as AI-generated phishing, voice cloning, and deepfake techniques.

The ICO also said organisations using AI tools that process high-risk personal data should have a data protection impact assessment and appropriate safeguards in place, including protections against attacks targeting AI systems. The blog also advised organisations to follow the government’s AI Cyber Security Code of Practice.

Encryption and pseudonymisation were also highlighted as measures that could reduce the impact of a breach.

“None of this is new, but AI brings a renewed urgency and greater speed,” Hulme said. “Organisations can prepare themselves for future cyber threats by establishing robust security fundamentals early, applying layered defences, and ensuring human oversight in their detection and response processes.”