OpenAI, the American artificial intelligence research organisation and developer of Chat-GPT, have published an extensive safety report which lays bare the risks of their latest AI model, and the work it has done to mitigate the dangers.

Among the new system’s vulnerabilities are unauthorised voice generation, the use of copyrighted content, risks of misinformation, and creating harmful audio output. 

According to OpenAI, tests also revealed that the AI model could have a societal impact, with users showing evidence of creating emotional bonds with the system, even romantic attachments, in telling the AI things like ‘This is our last day together’.

In a blog post discussing their findings, OpenAI admitted that as AI models like GPT-4o become more human-like, it could have an adverse impact on human-to-human interactions, and affect healthy relationships, with people choosing to spend more time with the AI and less in the ‘real-world’. 

The company has said that extended interaction with the model might even influence social norms. For example, because AI is deferential, users can interrupt and ‘take the mic’ at any time, which, if it becomes normalised behaviour, might impact how people interact.

The testing shows that a major part of the issue stems from the development of human-like, high-fidelity voices for the latest AI models. That was observed not only to result in users creating bonds with the software, but trusting it too. The report stresses that this ‘Voice Mode’ could allow for spreading misinformation, and notes instances where the model unintentionally generated audio emulating the user’s own voice. 

OpenAI states that they have taken steps to address these issues, for example by conducting post-training with the AI to convert existing text-based safety measures over to an audio environment, and by performing real-time analysis of audio input to detect if users are trying to generate banned content.

The firm has emphasised that it is working to fully explore the risks associated with AI models, both on individuals and society. Concluding the report the firm maintains the importance of stress tests and the need to mitigate risks from dangerous capabilities like self-improvement, model autonomy, and scheming.

Recommended reading

• Survey: 75% of Businesses Block GenAI to Protect Data
• CISOs Are Conflicted on AI in Cybersecurity
• Data Governance A Cornerstone to Better AI

Since the release of the first generation of ChatGPT, OpenAI has remained firmly in the spotlight over its plans for upgrading its AI model, although questions continue to rise over safety at the firm.

Earlier this year, former OpenAI leader Jan Leike criticised the company’s focus on product over safety, writing on X: “Building smarter-than-human machines is an inherently dangerous endeavour” and warning that the company is prioritising product advancement over safety concerns. 

Recently, the firm has attempted to calm worries over its safety protocols by helping to form the Coalition for Secure AI (CoSAI) with other AI developers such as Google, an open-source initiative to share methodologies, frameworks, and tools, to assist in the creation secure-by-design AI systems.