AI scribes may be saving social workers time on cumbersome paperwork and painstaking admin, but AI bias and prejudice could be permeating across use cases unchecked.

This is according to research from the Ada Lovelace Institute, which found that the rapid roll-out of AI transcription tools in social work is being met with enthusiasm by social workers, with many already reporting benefits from the time saved.

However, the research also finds that these tools are introducing new risks to people and society, such as bias and ‘hallucinations’, that aren’t being fully assessed or mitigated, leaving frontline workers to navigate these challenges on their own.

Researchers at the Institute interviewed 39 social workers with experience of using AI transcription tools, from across 17 local authorities, as well as senior staff members involved in procuring and evaluating them.

The report found that resource constraints in social care are driving widespread piloting, adoption, and evaluation of AI transcription tools.

The institute also found a sad reality: local authorities focus their evaluations on efficiency rather than impact on people who draw on their care.

Almost all of the social workers who reported that using AI transcription tools brings meaningful benefits to their work, but they do not experience these benefits uniformly.

While social workers assume full responsibility for AI transcription tools, perceptions of reliability, accuracy, and the need for oversight (‘human in the loop’) vary significantly.

Further, the Institute found that there is currently no consensus on when it is appropriate to use AI transcription tools in social care.

To mitigate risk amid growing use of AI transcription tools, the report set out a series of recommendations to improve the governance and regulations of these AI innovations.

Recommended reading

• UK Policing AI and Automation Called Out Over Racial Bias
• New Report Reveals The Quiet Biases Built Into AI Models
• Civil Rights Alarm Bells Raised over PM’s Surveillance Plans
• UK Police Use of Facial Recognition Found to be ‘Unethical’

These include recording the use of AI transcription tools, extending pilots of these tools to include various locations and contexts, and creating context-specific evidence for the benefits of the tools.

Further, the Institute calls on the UK government to create a What Works Centre for AI in Public Services, and calls on a coalition of researchers and policymakers to generate research into the impacts of AI transcription tools.

Social care regulators and local authorities should also work together to produce guidance for the use of these tools, as well as specifying the outcomes and expected impact of procuring these tools so staff can better understand their proper use.

“AI may be able to help make aspects of public services more efficient, and so policymakers should absolutely be looking at how technology can help public sector workers,” Lara Groves, Senior Researcher at the Ada Lovelace Institute and co-author of the research, said.

“However, delivering time savings is not necessarily the same thing as delivering public benefit, especially if these come at the cost of inaccuracy or unaccountability. In the rush to adopt AI in the public sector, it is essential that policymakers don’t lose sight of the wider risks to people and society and the need for responsible governance.”