February saw the spread of a new viral social media trend that has taken the world by storm.

It isn’t clear where the trend originated, but it hasn’t stopped people from uploading photos into ChatGPT and generating cartoon versions of themselves. 

The original prompt people have been using seems to follow the lines of “create a cartoon based on this image of me, and what I talk about most to you”. If ChatGPT doesn’t get it quite right the first time, users are then uploading extra details about their hobbies, interests, and personality, and within seconds, they receive a perfectly personal digital caricature.

It is a trend that, to most, feels creative, harmless, playful even, but data and security experts these past few weeks have been broadcasting warnings. They believe that beneath the surface, something much more consequential is happening.

Recent Warnings

The warnings from cybersecurity experts on the dark side of this trend centre on the long-term training implications for ChatGPT, mainly where data will actually end up, as millions of users are feeding extensive personal data into the LLM.

Obviously, users are always providing data every time they create a prompt in the platform, but a photo and highly specific personal information, like their physical features, lifestyle and identity, are details that most would not normally upload.

Individually, these details may seem insignificant, however they collectively form something far more valuable: an increasingly rich, structured identity profile. 

Imagine if you got a phone call from someone you don’t know or have never met in person, and they asked you for all of this information – you would feel quite disconcerted.

The data people have shared is data most people would never consciously exchange in a single moment. Yet when it’s framed as a fun trend, millions are quick to hand it over in exchange for little more than a symbolic marker of identity or social status.

Unlike a traditional search prompt, where interactions are fleeting and functional – like users asking a question or for something as simple as a recipe request – these interactions are more obviously deeply personal. 

They describe not just what you want to have for dinner that night, but who you interact with, what you do on the weekend, and your personal opinions.

It is a data gathering effort that used to take the likes of old-school Facebook years to cultivate, done in minutes, but the issue for users is the same: once this data has been shared, the exchange is permanent, you can’t get it back.

The Actual Risks

Building a digital clone is never a good idea. Once that level of detail exists in a digital system, it becomes far harder to control how it could be used, replicated, or interpreted in the future.

The rise of prompt injection attacks highlights how sensitive this kind of data can be. In the wrong hands, detailed personal data and images could be used to quickly build convincing impersonations or manipulate AI-driven tools. 

This also then increases the risk of deepfake scams and highly personalised phishing attacks. The more realistic and personalised an AI-generated image or profile is, the easier it becomes for attackers to impersonate someone convincingly. 

Threat actors could use AI-generated likenesses to trick colleagues, friends, or family members, or to create highly tailored scam messages that appear legitimate. These attacks are effective because they exploit familiarity and trust, and detailed digital personas make that deception significantly easier.

So what happens next? OpenAI now holds far more detailed information about many of its users than it did before, both built up over time and largely because of trends like the caricature phenomenon. 

Recommended reading

• Are Tech Leaders Prioritising AI Speed Over Safety?
• OpenAI Launches ChatGPT Health in US Sparking Privacy Concerns
• UK Businesses Turn to ChatGPT for Financial Advice Over Accountants
  

What happens next really comes down to how that data is handled, governed, protected, and potentially commercialised. As AI platforms continue to evolve, this kind of personal context could be used to make experiences feel more tailored, improve recommendations, or support new business models like advertising or agent-driven services, regardless of user preferences

At the same time, it naturally raises some important questions around boundaries, consent, and security. People will want clear answers about how long that information is kept, whether it might be used to train future models, and what safeguards exist to stop it from being misused or exposed.

Ultimately, how OpenAI balances innovation, monetisation, and user trust will have a big influence on how comfortable people feel sharing personal information with AI systems going forward.