Fresh reports from two major civil liberties groups claim that the UK Government’s use of unchecked and ‘secretive’ automation and AI is putting welfare at risk, and leading to the exclusion of people with disabilities and other marginalised groups.

Over the last week both Amnesty International and the UK privacy group Big Brother Watch published separate research investigating the roll-out of new technology by the Department of Work and Pensions (DWP), each finding significant problems impacting citizens rights.

Amnesty’s report, Too Much Technology, Not Enough Empathy, claims that the ‘constant’ testing, rolling out, and rolling back of AI and digital technologies for Universal Credit, Personal Independence Payments, and other social security schemes has created an inaccessible social security system for those already marginalised.

Specifically, Amnesty’s study found that people with disabilities, those living in poverty, and people with serious health conditions are most at risk by the DWP’s ‘unchecked’ use of tech, arguing that the introduction of automation and AI into social security assessments is risking significant errors in decision making, due to biased or discriminatory algorithms.

Big Brother Watch’s report, Suspicion by Design, claims that these algorithms were used to profile around one million people last year, with more than two million Universal Credit recipients expected to be reviewed by the end of this decade.

Calling the system ‘riddled with algorithmic bias’, Big Brother Watch found disparities around age, nationality, relationship status, and reported illnesses after analysing the DWP’s pilot tools, leading to questions about how and why people are flagged on welfare systems.

Digital ‘Dehumanisation’

According to Amnesty, even social security advisors have misgivings about the extent of emerging tech’s role in welfare.

Many raised concerns about the over-reliance of the DWP on digital decision making, with one advisor telling Amnesty that the new technology “lacks the subtlety to give a fair assessment” of claimants’ needs. 

Earlier this year, a Labour minister claimed that the DWP had used fifty-eight automated technologies to process nearly 50 million benefit claims, saving more than three million staff hours, and said the government would ‘continue to explore additional automation and digitisation prospects’.

Amnesty’s report argues, however, that the DWP’s mission to reduce costs and save time is based on a system of pre-existing flaws coupled with new problems, creating a ‘perfect storm’ with severe human rights implications.  

Recommended reading

• Is the UK’s ‘Prevent’ Data Retention Programme Illegal?
• Facial Recognition Trials by South Wales Police Ruled Unlawful
• UK Police Use of Facial Recognition Found to be ‘Unethical’

“People are struggling to make ends meet and put food on the table due to cuts in social security, and yet the DWP is more concerned about experimental technologies to surveil claimants,” said Imogen Richmond-Bishop, an Amnesty technology researcher.

“The tech-enabled system to claim and manage welfare benefits is resulting in relentless dehumanisation and strain for people who are already wrestling to access their basic needs in a broken system.”

These studies add to growing concerns about the state’s use of emerging technologies. Last year, Amnesty published another report which delved into the use of automation by UK police forces, particularly around live facial recognition (LRF) technology. 

Likewise, another study into what Amnesty called ‘dangerous discrimination’ in police prediction technology found that current profiling systems are overly focused on racialised people, who go on to be repeatedly targeted by police, adding to an ongoing cycle of prejudice.