Gartner, the business and technology insights company, has predicted that by 2029 most privacy incidents won’t result from the direct exposure of personally identifiable information (PII), but from AI-generated inferences about individuals instead.

As organisations reduce the amount of personal data they store due to regulatory and cost pressures, threat actors’ access to artificial intelligence now lets them perform inference-based attacks.

Advances in generative AI and machine learning are enabling the extraction of sensitive attributes, such as health conditions or behavioural patterns, from seemingly innocuous, anonymised, or aggregated data.

Speaking on this shift, Bart Willemsen, VP Analyst at Gartner, said: “Organisations have historically focused on protecting raw personal data, but AI can now reconstruct deeply personal insights without ever breaching traditional data controls.

“Privacy risks are increasingly emerging from what AI algorithms infer about individuals rather than what data is directly exposed.”

“Inference attacks are particularly dangerous because they often evade conventional detection mechanisms,” Willemsen added.

“Individuals can be exposed through AI-generated conclusions rather than leaked records, creating privacy risks that undermine data integrity and are difficult to detect, explain and mitigate.”

Recommended reading

• Report: Only 4% of Firms Turn AI Investment into Scalable Value
• AI Bias Locking Women Out of Tech Growth
• Tech CEOs Defend Open-Weight AI Models to Lawmakers

In light of this, Gartner expects spending on data integrity protections to reach parity with data confidentiality investments by 2028, as organisations respond to the risk of inaccurate, biased, or unauthorised AI-generated profiles.

The insights firm has also advised chief information security officers and other privacy leaders to undergo a series of actions to address these emerging inference-based privacy risks.

For instance, enhancing cybersecurity for AI-driven threats by investing in advancing monitoring, anomaly detection, and scenario-planning capabilities designed to identify indirect exploitation patterns and inference-based threats.

Adopting privacy-enhancing technologies (PETs), by implementing tech such as differential privacy, synthetic data, and privacy-aware machine learning to process data in a protected state and reduce reidentification risks was also suggested.

Further, embedding AI governance into privacy programmes, strengthening data minimisation and lifecycle controls, and fostering transparency and human oversight were also advocated for amid the shift from data exposure to insight exposure.