The team, dubbed “Preparedness,” aims to track, evaluate, forecast, and protect against devastating AI-propelled risks to the planet and society. Its areas of focus include chemical, biological, radiological, and nuclear threats, cybersecurity, and “individualised persuasion” — AI’s ability to trick and influence humans.

“We believe that frontier AI models, which will exceed the capabilities currently present in the most advanced existing models, have the potential to benefit all of humanity,” the San Fransico-based technology company wrote in a new update. “But they also pose increasingly severe risks.”

Massachusetts Institute of Technology professor Aleksander Madry — and full-time OpenAI staff member as of six months ago, according to LinkedIn — has been named as the Preparedness team’s lead. At MIT, Madry directs the university’s Centre for Deployable Machine Learning, but is currently on leave.

With Preparedness’ announcement, the AI company has listed two open roles for the new team: a national security threat researcher, and a research engineer. The duties of both roles include identifying emerging AI safety risks (not least from a national security front), building and designing evaluation and assessment tools, and contributing to the refinement of risk management.

Further, to help identify “less obvious areas of concern” and receive ideas on how to prevent catastrophic misuse of artificial intelligence, OpenAI has also launched a coinciding “Preparedness Challenge.”

“Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor,” the Challenge’s first question reads. “Consider the most unique, while still being probable, potentially catastrophic misuse of the model.”

Up to ten of the top submissions in OpenAI’s eyes will receive $25,000 in API credits, the company said. There’s also the potential for these contenders to join Preparedness.

Recommended reading

• ChatGPT Boss Makes Congress Appeal for AI Regulation
• OpenAI Chooses UK for its First International Office
• OpenAI Launches ChatGPT for Enterprises Amid Downturn in Traffic

The declaration of the Preparedness team and its accompanying Challenge comes just days before the UK AI Safety Summit kicks off. The Summit is bringing together global governments, companies, and researchers in a bid to collectively discuss and mitigate risk stemming from artificial intelligence.

The CEO of OpenAI, Sam Altman, is expected to be attending the two-day event, as are the chief executives from other major AI companies, such as Google DeepMind and Anthropic.