Hearing technology experts discuss audio-visual hearing aids

Imagine that you are wearing a hearing aid in a noisy pub or café. You want to chat with a person sitting next to you without having to look so closely to be able to lip read. What if you are conversing with someone wearing a COVID face-covering? These example scenarios were given to COG-MHEAR researchers in a recent workshop, to ensure that new hearing technology is designed to work in everyday situations where hearing aids often do not work well.

One potential solution we are working on is to incorporate cameras or radio frequency sensors to sense lip movement, correlate this with sound from one speaker, and reduce background noise. The participants discussed privacy concerns about use of a camera. There were concerns that cameras, no matter how low the resolution, and how well the technology is explained, could still be considered intrusive. A video demonstration showed how a privacy-preserving camera could select only the border of the lip region to assess speech. It is often legal to film in a wide range of public situations, but not everyone is comfortable being filmed. The bigger issue is what society as a whole is prepared to accept. There is a compromise to be made between legality, good manners, sensibility and consent. These issues will not be solved in the COG-MHEAR research programme, but should be discussed.

The need for privacy could be further allayed by use of radar for lip reading, which could also potentially work in the presence of face masks. There is a need to build trust in the technology, including through audiologists developing rapport with people who wear hearing aids. This includes reassurances about use of the 5G network and radio frequency signals that will be required to enable initial versions of the hearing aids to work quickly to process speech and cut out background noise. On-chip processing technology is currently being developed so that later versions of the devices could work offline, which could further allay privacy concerns.

The most willing initial testers of the new technology are likely to be people of all ages and backgrounds with hearing loss, who are used to wearing hearing aids. A range of venues (in addition to homes) may be considered as testing grounds to account for privacy concerns.  Initial versions of the new technology are likely to be bulkier than current, audio-only hearing aids, but there was a willingness amongst workshop participants to wear slightly larger devices for testing purposes. Join the discussion to help develop new hearing technology by signing up at the "get involved" tab: link

For a hands-on demonstration of the web-based COG-MHEAR prototype, see: https://demo.cogmhear.org