Deepfakes are videos, pictures or audio clips made with artificial intelligence to look real.

New Ofcom research has found that 43% of people aged 16+ say they have seen at least one deepfake online in the last six months – rising to 50% among children aged 8-15.

Historically, creating deepfakes required advanced tools such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), which demanded significant technical expertise and high costs. However, the advent of genAI models has revolutionised this process, drastically changing the dynamics of deepfake production.

Anyone can make a deepfake now

One of the biggest changes introduced by genAI is the simplification and reduction in cost of deepfake creation.

Traditional tools like GANs and VAEs required extensive technical knowledge and substantial financial resources. For example, cloning a voice with older technologies could cost upwards of $10,000. In contrast, the cost of voice cloning has plummeted to just a few dollars within a year.

GenAI tools are designed to be user-friendly, requiring only basic technical skills. Users can create deepfakes by entering simple prompts into model interfaces, a significant shift from the complex processes previously required.

This ease of use has broadened the pool of potential deepfake creators, including individuals with minimal technical knowledge. According to the Ofcom, there have been reports of children in the UK, US, and Spain using genAI to produce deepfake nude images of peers – an alarming development that would have been nearly impossible with older technology.

Deepfakes are now more realistic and versatile

The capabilities of genAI models extend beyond just making deepfake creation easier and cheaper. These models produce highly realistic content with ease.

A study from the University of Waterloo found that only 61% of people could differentiate AI-generated images of people from real ones. GenAI models can now create synthetic characters that exhibit lifelike voices, nuanced facial expressions, and subtle body movements.

Furthermore, genAI’s flexibility allows for the creation and manipulation of a broader range of content. Unlike GAN and VAE tools, which are better suited to altering existing content, genAI can generate entirely new content and edit existing materials. This includes the creation of long audio passages from just a few seconds of real speech.

Some of the latest genAI audio models can produce undoubtedly lifelike voice clones with minimal input.

Unsurprisingly, deepfakes are rarely used for good

The proliferation of genAI has not only increased the quality and accessibility of deepfakes but has also escalated their prevalence and potential for harm. Deepfakes can be categorised into three main types based on their intended harm:

1. Deepfakes that Demean: These deepfakes are created to humiliate or abuse victims by falsely depicting them in compromising or demeaning scenarios. Research shows that intimate image abuse, including deepfake content, can cause significant social and psychological harm. A recent survey from Ofcom found that 14% of respondents aged 16 and older believed they had encountered a sexual deepfake in the past six months. Among those, 64% identified the content as being of celebrities or public figures, while 15% were of someone they knew personally.
2. Deepfakes that Defraud: These are used to deceive individuals for financial gain or other malicious purposes. Fraudulent deepfakes can include fake advertisements or romance scams. A notable example is a deepfake featuring Martin Lewis, used in a fraudulent Facebook ad promoting a non-existent investment. The number of attempts to use fraudulent deepfakes to bypass identity verification systems surged by 3,000% between 2022 and 2023, according to digital identity company Onfido.
3. Deepfakes that Disinform: These deepfakes aim to spread misinformation or shape public opinion on political and social issues. For instance, a deepfake audio recording of Slovakian politician Michal Simecka purportedly discussing election rigging was shared on social media just hours before votes were cast. Microsoft reported observing Chinese-affiliated actors using deepfakes to create politically divisive content, while extremist networks have used deepfakes to spread propaganda and recruit followers.

How prevalent are deepfakes?

According to the report, accurately measuring the prevalence of deepfakes is challenging due to their deceptive nature and varying definitions. Nevertheless, available data sheds light on the scope of the issue, with the report stating:

• Ofcom’s Poll: Found that 43% of respondents aged 16 and older, and 50% of those aged 8-15, believed they had encountered a deepfake in the past six months. Additionally, 10% of adults and 13% of children in this group believed they had seen deepfakes over ten times in this period.
• Alan Turing Institute and Oxford Internet Institute Survey (2024): Indicated that younger participants aged 18-25 reported the highest exposure to non-consensual and political deepfakes.
• Channel 4 News Analysis (2024): Identified nearly 4,000 famous individuals featured on the five most visited non-consensual deepfake websites.
• Internet Matters Survey (2023): Revealed that 10% of children aged 13-16 had either experienced or known someone who had been featured in fake nude images or videos.
• Sensity (2020): Found that deepfake bots on Telegram had generated over 100,000 fake nude images, predominantly targeting women.
• My Image My Choice (2023): Discovered 276,149 intimate image abuse deepfake videos on top sites, with these videos amassing over four billion views.
• Home Security Heroes (2023): Reported a 550% increase in deepfake videos from 2019 to 2023, with 98% of these being sexual deepfakes predominantly targeting women.
• Advertising Standards Authority (2023): Noted an increase in scam ads featuring deepfake footage of celebrities endorsing cryptocurrency and trading apps.
• Fenimore Harper: Found that 143 deepfake advertisements featuring Rishi Sunak reached over 400,000 people on Facebook between December 2023 and January 2024.

Mitigation Strategies

According to Ofcom, addressing the rise of harmful deepfakes requires a multifaceted approach:

1. Prevention: Implement safeguards within model development, such as prompt and output filters, and content removal from training datasets.
2. Embedding: Attach provenance metadata and invisible watermarks to content to trace its origin and verify authenticity.
3. Detection: Use automated and human-led tools to identify and flag deepfake content.
4. Enforcement: Establish and enforce clear rules regarding synthetic content creation and distribution, and take action against violators by removing content and suspending accounts.

Addressing deepfakes is likely to require action from all actors in the technology supply chain – from the developers that create genAI models and related tools, to the platforms that host this technology, through to the user-facing services that act as spaces for deepfake content to be shared and amplified.

Recommended reading

• YouTube Introduces Tool to Flag AI Content Ahead of Global Elections
• OpenAI Is Trying to Stop AI Meddling in Elections. Will It Be Enough?
• Meta To Start Labelling AI-generated Images

In its report overview, Ofcom states that ‘As the new regulator for online safety, Ofcom is committed to doing its part to curtail the circulation of this malicious content. There will be circumstances where services regulated under the Online Safety Act 2023 (‘the Act’) will need to address the dissemination of some types of deepfake (though not all types).

To effectively regulate this type of content, we need to have a clear understanding of the types of deepfakes that can be created, the types of harm they are most likely to be implicated in, their prevalence, and the merits and limitations of different mitigation techniques.