According to technological research and consulting firm Gartner, 40% of GenAI (generative artificial intelligence) solutions will be multimodal by 2027—a significant increase from 2023’s figure of 1%.

Multimodal GenAI encompasses the ability to process a range of media, including text, imagery, sound, and moving video. This shift from individual to multimodal models consequently provides an opportunity for GenAI-enabled offerings to be differentiated, in addition to enhanced human-AI interaction.

“As the GenAI market evolves towards models natively trained on more than one modality, this helps capture relationships between different data streams and has the potential to scale the benefits of GenAI across all data types and applications,” said Erick Brethenoux, distinguished VP analyst at Gartner. “It also allows AI to support humans in performing more tasks, regardless of the environment.”

Multimodal GenAI is one of the two technologies identified in Gartner’s 2024 GenAI Hype Cycle where early adoption has potential to lead to notable competitive advantage and time-to-market benefits. Along with open-source large language models (LLMs), both technologies have high impact potential on organisations within the next five years, the research firm found.

“In the real world, people encounter and comprehend information through a combination of different modalities such as audio, visual and sensing,” explained Brethenoux. “Multimodal GenAI is important because data is typically multimodal. When single modality models are combined or assembled to support multimodal GenAI applications, it often leads to latency and less accurate results, resulting in a lower quality experience.”

Meanwhile, on open-source LLMs—deep-learning foundation models that can democratise commercial access due to their open nature—Arun Chandrasekaran, distinguished VP analyst at Gartner, commented: “Open-source LLMs increase innovation potential through customization, better control over privacy and security, model transparency, ability to leverage collaborative development, and potential to reduce vendor lock-in.”

“Ultimately, they offer enterprises smaller models that are easier and less costly to train, and enable business applications and core business processes,” Chandrasekaran added.

That said, among the GenAI innovations Gartner expects will reach mainstream adoption within 10 years, two technologies have been identified as offering the highest potential: domain-specific GenAI models, which are optimised for the needs of specific industries, business functions, or tasks, and autonomous agents, which are combined systems that achieve defined goals without human intervention.

“Domain-specific models can achieve faster time to value, improved performance and enhanced security for AI projects by providing a more advanced starting point for industry-specific tasks,” noted Chandrasekaran. “This will encourage broader adoption of GenAI because organizations will be able to apply them to use cases where general-purpose models are not performant enough.”

Touching on autonomous agents, “Autonomous agents represent a significant shift in AI capabilities,” said Brethenoux. “Their independent operation and decision capabilities enable them to improve business operations, enhance customer experiences and enable new products and services. This will likely deliver cost savings, granting a competitive edge. It also poses an organizational workforce shift from delivery to supervision.”

Recommended reading

• 73% of UK Financial Leaders Want AI to Help with Decision Making
• OpenAI Co-Founder’s Safety-focused AI Startup Raises $1 Billion
• UK Signs First International, Legally-binding AI Treaty

Gartner’s predictions about multimodal AI come just days after OpenAI, the creator of the multimodal chatbot ChatGPT, has over one million paid business users.

As DIGIT staff writer Tom Quinn wrote, the meteoric rise is in part down to CEO Sam Altman’s push to get companies using ChatGPT in the workplace, who Reuters reported in April had pitched the company’s AI services to hundreds of leaders at Fortune 500 companies.