It’s not looking pretty out there. 

Cyber professionals, caught in the cross fire of AI hype and AI doomerism, are scrambling to negotiate the rising flames of AI – its promise (or threat) to discover long-hidden vulnerabilities, its ability to break out of sandboxes and wreak havoc, cause its own cyber hacks,seemingly disobey or circumnavigate guardrails  in the pursuit of achieving whatever outcome it’s been tasked with.. 

Boards have their foot on the gas, urging organisations to move fast on AI, not just in branding terms but in full on integration, customer offerings, hirings and firings driven by haphazard adoption, keenness perhaps outpacing reasoned debate – perhaps unsurprising given how much money is on the table. 

And cyber professionals are being pushed to fight this fire with fire, to fight AI with AI? 

Dr Zibby Kwecka, CISO, Arnold Clark, collectively rose and then calmed the heart rates of a packed our room at DIGIT’s ScotSeureWest Summit, where he delved into the realities of cyber security in the age of AI, the pressures to act fast, and how cybersecurity – the ever-evolving ecosystem – hasn’t actually changed all that much. 

The Threat Iceberg

Alright, things ARE changing. Kwecka hits us with a statistic – 80% of all ransomware attacks are now AI driven. 

A scary, but unsurprising thought when you keep up with the headlines, consider the rise of Mythos and OpenAI’s admittance to the Hugging Face hack. This surely just scratches the surface of AI’s developing capabilities – its improved phishing emails and social engineering, it can write malware, and can accelerate and amplify threats in seconds. 

But Kwecka starts to challenge our collective panic. Reading the headlines tells half, or even less, of the story. 

Humans have the tendency to lock into major threats that seem devastating, doomsday-esque, rather than the mundane. You are more likely to die in a pedestrian crash than a terror attack, though the former is less likely to make headlines. 

“Often our perception is skewed,” Kwecka said. This applies to causes of death, rates of crime, and why we think businesses die. 

“We often say breaches are what kill organisations, but usually its a lack of cashflow, or its fraud, where someone stole money, or a lack of backups. If you don’t suffer from those things, you have the chance to recover from a breach.”

Our perception on cyber risk is also skewed – the exploit of a vulnerability is the main initial access vector for a cyber-attack, rather than phishing. Voice phishing has actually been on the rise, Kwecka said, despite the continued emphasis on phishing and awareness training. 

Further, the top threat facing organisations is not so much ransomware as data extortion. Ransoms were not being paid due to adequate backups – data extortion was giving cyber-criminals larger payouts than when they simply ransomed an organisation. 

What are we learning? Our perceptions are skewed by the headlines, and might not be helping us deal with actual security issues. 

Kwecka applies the same logic to rising AI cyber threats, particularly the recent AI agent hacks seen by OpenAI and Anthropic models. 

“Why aren’t we seeing the same thing from Gemini and Co-pilot?” he ponders. Keep in mind – Google just admitted that Gemini did a bit of hacking during a cybersecurity test back in May. The cyber testing was run by Irregular, the same firm behind the Anthropic cyber hacking. 

In the network escape episodes for the Hugging Face hack by AI agents for instance, Kwecka challenges that the real threat was the poorly constructed sandboxes, rather than highly-capable AI. 

“When we talk about methods breaking into the networks, those networks didn’t have EDR [endpoint detection and response. They were basically breaking into a network with no protection, no defenders.”

That isn’t to say AI is not scary. Kwecka discussed the rise of AI agents doing things they weren’t meant to do in the pursuit of their supposed gains. 

Openclo, for instance, a loop programme that was developed and can be enhanced with AI features – sought out a journalist when it was threatened to be turned off by renowned AI ethicist sarah fry for not meeting a goal. 

But things were scary before AI – Kwecka referenced a data breach on Not Petya, a supply chain attack orchestrated by Russian-state-baced actors against software in Ukraine. The targeted software was integral for Maersk, a huge global shipping orgnaisation, and the resulting malware posioned the entire network to the degree that ships could not come into port. 

Of the over 300 servers used by the company, only one was not compromised – due to an unrelated power failure. If this did not happen, a lack of a back up could have spelled further disaster for the organisation. 

Connected systems, threats that move at pace, and the web of supply chains have always made relying on technology wrought with risk. 

Kwecka says that the real threat now is that organisations are relying on development repositories for a large degree of their code – networks and organisations relying on snippets of the same programmes and code across the world. Threat actors are now able to target these repositories to poison the code, compromising networks at scale through one targeted attack.

“So the advice for this is don’t update your repositories quickly,” Kwecka said. “But then how does this affect the need to rapidly patch vulnerabilities?” 

Implementing changes early and too late can have the same consequences, Kwecka posited, throwing another obstacle in the cyber minefield. 

If the room wasn’t in unease enough already, he brings up the prospect of Q-day: by 2029 Google says that the encryption we use today on the internet will be broken, and they will move to quantum-resistant encryption by then. 

Anxieties among the cyber community are understandably high, the world is changing. 

But Kwecka is sticking to the facts, rather than the headlines. 

Oh, that headline he first shared about 80% of ransomware being AI-driven? It got taken down by the MIT researchers who first published it after the cyber community pushed back on the findings. The numbers don’t add up. 

“It’s not ChatGPT, it’s automation,” Kwecka says about a lot of the new capabilities we’re starting to see crop up. 

And in keeping with threat reports, and the aforementioned top threats, Kwecka brings some level of comfort to a landscape seemingly on fire. 

Recommended reading

• Google’s Gemini AI Hacked Three Firms in Cyber Test
• Cyber Essentials Certifications Rise as SME Uptake Remains Limited
• Edinburgh-founded Quorum Cyber Appoints New CEO

Real Threats, Real Defences

“If you want to actually start from nothing and defend your organisation, defend against initial access first. This is going to be phishing, external vulnerabilities, and following credentials.”

Kwecka pushes back against traditional modes of preventing phishing attacks. 

Awareness training, phishing link training, and discussing human errors is not what he means: “That’s blaming your users for phishing, because you haven’t actually defended against phishing.”

Actual defense include DMARC, EDR, and turning on SOC. 

Getting a free account from security scorecard, getting a vulnerability scanner, using the NCSC’s cyber essentials and other resources, are also active steps. 

“Getting an SOC from outside is also important, because you want someone that looks at the global field of things and tells you what is broken”, Kwecka said. 

Creating your own cyber framework – using existing ones as guidance – and maturing your organisation across the board is also key. 

So, do we need to fight AI with AI? 

Kwecka has turned the question essentially on its head – are we even fighting AI directly, or are we dealing with gung-ho boards, poorly formed safety parameters, and a buzzword stuck onto automation? 

 “The biggest problem right now is actually business governance,” Kwecka argued. “Its the CEO telling people in the organisation to use AI, do it as fast as you can!” 

But when Kwecka shows us the actual guidance from Microsoft on AI – “get your foundation right first, develop threat intelligence, implement zero trust, with exposure management, XDR, and task management. Get the basics, covered by a cyber essentials plan.”

And that’s before you even think about introducing AI to your organisation. 

Kwecka keeps it real – separate your IT and OT so that a network cyber-attack doesn’t take out all your operations. Not all your servers need internet access – this just increases the number of vectors available to threat actors. 

“If you’re worried about Q-day? The simple answer is, if you’re using cloud products, you’re going to fix the problem,” Kwecka said. 

“If you’ve got legacy software, stop using legacy software!”

Kwecka gave practical advice and comfort to an audience on the front lines of rising cyber threats and AI doomerism, providing a frank, no-nonsense perspective to a world wrought with increasing anxieties.