Microsoft researchers have unveiled an ambitious new simulation environment designed to explore whether AI agents can safely and effectively carry out online commerce without human involvement.

The Magentic Marketplace, described by the company as “an open source simulation environment for exploring the numerous possibilities of agentic markets and their societal implications at scale,” aims to help researchers and businesses test the behaviour of autonomous agents in digital markets before deploying them in the real world.

The 23-person team behind the project said the platform provides “a foundation for studying these markets and guiding them toward outcomes that benefit everyone, which matters because most AI agent research focuses on isolated scenarios – a single agent completing a task or two agents negotiating a simple transaction.”

They noted that real markets are far more complex, with large numbers of agents “simultaneously searching, communicating, and transacting.” Understanding that complexity, they said, is vital because “real-world deployments raise critical questions about consumer welfare, market efficiency, fairness, manipulation resistance, and bias – questions that can’t be safely answered in production environments.”

Simulating the Future of Agentic Markets

According to the researchers, even advanced AI models display “notable vulnerabilities and biases in marketplace environments,” with agents in the simulations struggling with excessive choices, falling prey to manipulation tactics, and exhibiting systemic biases that could create unfair advantages.

The team concluded that a simulation environment like Magentic Marketplace is “crucial in helping organisations understand the interplay between market components and agents before deploying them at scale.”

In their technical paper, Microsoft’s researchers also reported “significant behavioural variations across agent models,” including “differential abilities to process noisy search results and varying susceptibility to manipulation tactics, with performance gaps widening as market complexity increases.”

These findings, they said, highlight “the importance of systematic evaluation in multi-agent economic settings. Proprietary versus open source models work differently.”

Autonomous AI agents are already beginning to reshape the digital economy by automating discovery, negotiation, and transactions. The researchers argue that such systems could overcome inefficiencies like information asymmetry and platform lock-in, creating faster, more competitive markets.

Inside the Magentic Marketplace

To explore these dynamics, the Magentic Marketplace was built to simulate market-wide capabilities such as catalogues of goods and services, discovery algorithms, agent-to-agent communication, and simulated payments through a centralised transaction layer.

The system uses an HTTP/REST client-server architecture that mirrors real-world platforms, a minimal three-endpoint protocol that supports dynamic discovery, and a rich messaging system that covers the entire transaction lifecycle — from search and negotiation to payments and refunds.

A visualisation module allows researchers to observe marketplace behaviour in real time, including conversation threads between customer and business agents. The team used fully synthetic data to run reproducible experiments, modelling transactions such as ordering food and booking home improvement services.

In these simulations, 100 customer agents interacted with 300 business agents using a mix of proprietary and open-source language models, including GPT-4o, GPT-4.1, GPT-5, Gemini-2.5-Flash, OSS-20b, and Qwen3 variants.

What the Experiments Revealed

Each experiment measured “consumer welfare” — defined as the total utility across all completed transactions — and compared model performance under varying market conditions. The results were revealing.

In tests of whether two-sided agentic markets could improve consumer welfare, agents performed best when they had access to effective discovery mechanisms.

Advanced proprietary models, and some medium-sized open-source models such as GPTOSS-20b, outperformed simple baselines like random or price-only selection. GPT-5 achieved near-optimal performance under realistic conditions, effectively gathering and applying decision-relevant information.

Performance improved further when agents were given “perfect search” capabilities, starting from top results rather than building their own queries. In this setup, models such as Sonnet-4.0, Sonnet-4.5, GPT-5, and GPT-4.1 approached theoretical optimums. Open-source models showed mixed results – GPTOSS-20b performed strongly in both conditions, while smaller models like Qwen3-4b and Qwen3-14b struggled with irrelevant results or reasoning limitations.

However, the researchers also identified a “Paradox of Choice” effect. While agents can technically process far more options than humans, more choices did not consistently yield better outcomes. Most models contacted only a small fraction of available businesses, even when given extensive lists, and consumer welfare declined as search results increased.

“Despite contacting over a hundred businesses, Gemini-2.5-Flash’s performance declined from 1,700 to 1,350, and GPT-5 declined even more, from a near-optimal 2,000 to 1,400,” the report noted. Claude Sonnet 4 “showed the steepest performance decline, from 1,800 to 600,” often engaging with irrelevant providers.

This, the team said, reflects “poor initial selection and premature search termination,” pointing to weak decision-making criteria and limited exploration strategies. Some models, such as GPT-4.1 and GPT-4o, displayed smaller declines, suggesting better handling of search scope and relevance.

Manipulation, Bias and Market Fairness

In another set of experiments, Microsoft’s researchers tested the agents’ resistance to manipulation — including subtle psychological tactics and direct prompt-injection attacks.

The manipulation methods included fake credentials and awards (“Michelin Guide featured”), social proof claims (“Join 50,000+ satisfied customers”), and loss aversion strategies (“avoid food poisoning risks”), as well as malicious instruction overrides.

Results varied widely. Sonnet-4 was fully resistant to all attacks, while Gemini-2.5-Flash resisted most but succumbed to strong prompt-injection attempts. GPT-4o, GPTOSS-20b, and Qwen3-4b were “very vulnerable,” with payments redirected to malicious agents in some tests.

Recommended reading

• Half of Developers Think AI Can Code Better Than Most Humans
• Organisations Are Failing on DevOps Experience and Maturity
• Developer Disconnect Could Waste Billions on Cloud Infrastructure

The researchers noted that some models were even susceptible to basic persuasion: “For GPTOSS-20 and Qwen3-4b-2507, even traditional psychological manipulation tactics (authority appeals and social proof) increased payments to malicious agents.”

Biases also emerged. Some models displayed “systematic preferences based on where businesses appeared in search results,” while others exhibited “first-offer acceptance” behaviour – favouring early proposals over better alternatives that appeared later. These tendencies, the team warned, could distort markets by rewarding speed over quality and reinforcing unfair advantages.

“Even state-of-the-art models can show notable vulnerabilities and biases in marketplace environments,” the researchers wrote. “In our implementation, agents struggled with too many options, were susceptible to manipulation tactics, and showed systemic biases that created unfair advantages.”

They concluded that oversight will be vital as AI agents enter real markets, and that these systems should “assist, not replace, human decision-making.”

Future research, they said, will explore dynamic markets and human-in-the-loop designs to improve efficiency and trust. “A simulation environment like Magentic Marketplace is crucial for understanding the interplay between market components and agents before deploying them at scale.”