Meta has revealed it discovered “likely AI-generated” content used across its social media platforms including fake profiles posting comments supporting the state of Israel’s assault on Gaza.

In a quarterly security report, Meta found profiles in Facebook and Instagram with AI-generated profiles, including profile pictures, posting comments pretending to be Jewish students, African Americans, and others posting comments under news organisations’ and US lawmakers’ posts.

The comments largely targeted audiences across North America, and praised the ongoing genocide of Palestinians perpetrated by Israel.

In its report, Meta credited the social media AI campaign to STOIC, an Israeli political marketing company based in Tel Aviv.

This is the first time Meta has disclosed the use of generative AI for text in influence operations on any of its platforms since it emerged in 2022, though Meta has previously found profile photos generated by AI since 2019.

Meta has said that they removed the Israeli campaign, and were not overly concerned in how the rise of AI could mitigate their efforts to quell the spread of rampant misinformation and disinformation across their platforms.

Since the advancement in large language models, experts have been concerned on how these tools can be used by governments and threat actors alike to conduct misinformation campaigns and conduct operations across social media.

Recommended reading

• YouTube Introduces Tool to Flag AI Content Ahead of Global Elections
• OpenAI Is Trying to Stop AI Meddling in Elections. Will It Be Enough?
• Meta To Start Labelling AI-generated Images

Amid many global conflicts, and during an important election year across many countries in the globe, the rise of AI misinformation is stoking fears across international governments.