Soapbox Science: Hearing with cameras

Soapbox Science is an annual event in Edinburgh. It is designed to get passers-by in city centres to notice and engage with women and non-binary people who work in science, technology, engineering, maths and medicine.

Dr Dorothy Hardy, the COG-MHEAR Research Programme Manager, stood on a soapbox in the centre of Edinburgh with two halves of a plastic tub held to her ears, to show how a basic hearing aid captures sound and channels it towards the ears of the person wearing it. Holding a model camera helped to explain how videoing a talker’s mouth can help people to hear the one person that they want to in noisy situations. Software compares the video of a talker’s mouth with the sounds collected by microphones in a hearing aid, then cancels out any sounds that are not likely to be associated with the mouth movements.

Several queries about the COG-MHEAR technology came up during the training for the event and from those who stopped find out more on the day. One of these was the idea that using this technology will mean showing that you are compensating for deafness. There’s already a stigma around getting hearing aids, because it can show that you are getting older. It will be interesting to see who the early adopters of this technology will be: the people who don’t mind showing that they are using technology to help them to hear.

Will you be able to hear people far from you that wouldn’t be expecting to be heard as they whisper on the other side of the room? This is not designed to be spyware! So the technology will be designed to compensate for hearing loss rather than creating a superpower.

One of the key queries was about how it will be possible to use a camera to switch from listening to one talker to another when listening to several people at once. Using additional sensors to detect the direction of a person’s gaze could help with that. Plenty of processing power is required to do this quickly enough to hear almost instantaneously. So this hearing technology currently relies on good internet connections. The aim is to develop the device to a stage where it is affordable and where the sound processing happens on a chip, rather than needing to connect to the internet.

The event gave the opportunity to explain the idea of COG-MHEAR hearing technology to a wide range of people who happened to be in Edinburgh centre on a Saturday morning.