Researchers from across Scottish universities are taking part in a UK-wide project developing ‘hearing glasses’ that could dramatically improve how people with hearing loss experience sound by combining lip-reading technology, AI and cloud computing.

The COG-MHEAR programme, led by Edinburgh Napier University alongside researchers from the universities of Edinburgh, Stirling, Glasgow and Heriot-Watt, aims to help those with hearing loss by creating a device that filters out background noise in real time, even in loud environments.

Still under development, the device will use a small camera built into glasses to track the speaker’s lip movements, while a smartphone app using 5G will be able to send both audio and visual data to a powerful cloud server.

AI can then isolate the speaker’s voice from surrounding noise and send the cleaned-up sound back to the listener’s hearing aid or headphones almost instantly.

While the researchers are working towards a functional version of the glasses, they’re already speaking to hearing aid manufacturers about future partnerships and hoping to reduce costs to make the devices widely available.

“We’re not trying to reinvent hearing aids. We’re trying to give them superpowers,” said Professor Mathini Sellathurai from Heriot-Watt, who is co-leading on a strand of the work utilising wireless, 5G, cloud-based noise cleaning.

“One of the most exciting parts is how general the technology could be. It’s aimed to support people who use hearing aids and who have severe visual impairments, but it could help anyone working in noisy places, from oil rigs to hospital wards.”

This approach, known as audio-visual speech enhancement, takes advantage of the close link between lip movements and speech. While some noise-cancelling technologies already exist, they struggle with overlapping voices or complex background sounds, something this system aims to overcome. 

According to researchers, if successful, the COG-MHEAR device will mean that even if two people are speaking at once, the AI can use visual clues to extract the voice of the person being looked at, simply by pointing the camera in their direction.

Over 12 million people in the UK are affected by hearing loss, but currently, most hearing aids are limited by their size and processing power, and often struggle in noisy places like cafés, stations or workplaces.

Recommended reading

• NHS Scotland Hits 10,000 Robotic Surgeries
• Glasgow Researchers to Explore If “Neurobots” Can Help Treat Epilepsy
• Aberdeen’s GEMINI Project Wins Awards for AI Cancer Screening

To tackle this, the researchers are shifting the heavy processing work to cloud servers – some as far away as Stockholm – allowing them to apply powerful deep-learning algorithms without overloading the small, wearable device.

While there is a slight delay as the sound travels to Sweden and back, the researchers said that with 5G, it’s still fast enough to feel instant.

“There are only a few big companies that make hearing aids, and they have limited support in noisy environments,” said Sellathurai. 

“We want to break that barrier and help more people, especially children and older adults, access affordable, AI-driven hearing support.”