Skip to content

Selective Hearing

In real life I want to be able to mute a single person or a single sound while everything else stays audible.

Test track with 330 scenes and 12 sound types is in place

Someone is typing, a siren wails next to it.Computed on a Mac, not live yet.

Siren and keyboard: Freesound, from the FSD50K collection · both CC0

What it is about

Noise-cancelling headphones make everything quieter. I want something else: to mute a single person or a single sound in real life while everything else stays audible.

The sample above is a scene from my fixed exam, and it is the one that has gone best so far: someone is typing, a siren wails next to it, and the siren can be switched off. Not every scene works this well, the numbers are below. It was computed on a Mac, not live yet.

How I go about it

The program receives its job as a sound sample: a few seconds of "something like this", not a fixed list of sounds. The same input is later meant to take a specific voice.

I train the models myself, on a Mac. I started with small networks of my own from scratch. In the current model only the listening part comes from an open research model. The part that takes the sound out I built myself and trained with my own material, which I mix from free recordings of voices, sounds and rooms.

I always measure both: how much quieter the target gets and whether the rest stays intact. A scene only counts as passed when the target is at least 20 decibels quieter and the rest changes by at most 1 decibel. In the end the ear decides: I listen to the fixed exam blind, against a ladder of known steps.

Where others stand

The closest work comes from a group at the University of Washington in Seattle. Their programs run live, even on a phone, but work with a fixed list of 20 sound types. Programs that flexibly follow a sound sample exist too, just not live. I have not found both together anywhere.

I measured the ready-made programs on my test track. At six fixed listening spots the live program from Seattle reduces the target by 21, 3, 1, 6, 26 and 2 decibels. By ear, my current version is at 30 decibels or more at all six. The comparison is not entirely fair: my version does not run live yet.

What is hard

The real room

Programs that separate two cleanly mixed voices on a computer fail at a real dinner table with reverb and noise. They then remove one voice and swallow the other along with it.

Recognising, not removing

Where my version steps in, it is mostly good: of 18 such scenes it passes 14, eleven of them even at 30 decibels. In the other twelve it does next to nothing. So the problem is recognising the target, not the depth of removal.

Measuring honestly

At the start, the best ready-made program passed 5 of the 30 scenes of the fixed exam. My version passes 14 today, 24 are required, without any audible damage to the rest. The outcome is open.

Key facts

Status
computed on a Mac, not live yet
Goal
headphones
Built
since November 2026
Tech
Python, self-trained models, own test track