Splitting drums without a neural network

Most stem splitters today are neural networks: a model of tens or hundreds of megabytes, trained on thousands of songs. seance, the sample workbench on this site, splits drums without one. It runs in a browser tab, starts at once, and every build measures how well it works. Here is how.

Step one: steady against sudden#

Look at a drum loop as a spectrogram, time left to right and pitch bottom to top. A held note is a horizontal line: one frequency, lasting. A drum hit is a vertical line: every frequency at once, for an instant.

That is the whole idea behind harmonic-percussive separation. For every point of the spectrogram, compare it with its neighbours along time and along frequency. If it looks like its neighbours in time, it belongs to something steady. If it looks like its neighbours in frequency, it belongs to a hit. Each point is shared between the two stems in proportion.

The textbook version compares with the median of the neighbours. seance uses the 20th percentile instead, which keeps more of the quiet tails of notes on the harmonic side. On top of that it arms the percussive side at each detected onset, so the start of a hit is taken more firmly than the rest.

Step two: which hit is which#

Now there is a percussive stem, and it still holds the kick, the snare and the cymbals together. To split it, seance uses non-negative matrix factorisation: it describes the percussive spectrogram as a small number of spectral shapes, each switched on and off over time.

• The spectrogram is pooled into 160 bands on a log scale.
• It is factorised into 20 shapes with 60 rounds of updates, started three times from fixed seeds, keeping the best of the three. Fixed seeds mean the same file always gives the same stems.
• Each shape gets a first label from where its energy sits: low is kick, middle is snare, high is cymbals.

Then comes the step that makes it work. A kick drum has a click at the moment the beater hits, and that click is spectrally the same as a cymbal. Only its timing gives it away: it fires exactly when the kick's low thump fires. So every high shape is checked against the timing of the others and moved to the kick if it fires with the kick.

Finally each class gets a soft mask, and the masks add up to one at every point. The three stems therefore sum exactly to the percussive stem: nothing is lost and nothing is invented.

How we measure it#

You cannot measure separation on a real song, because nobody has the true separate parts. So the test suite builds its own: an eight-second groove at 120 BPM, synthesised from a known kick, snare, cymbals and a harmonic part, then mixed. seance separates the mix, and each stem is compared with the part it should be.

The score is SI-SDR, the scale-invariant signal-to-distortion ratio. It asks how much of a stem is the true part and how much is everything else, in decibels, ignoring overall volume. 0 dB means as much wrong as right. Every 6 dB halves the error.

Stem	The plain mix	Band-split baseline	seance
Kick	1.9 dB	7.0 dB	7.0 dB
Snare	−12.7 dB	−0.5 dB	11.6 dB
Cymbals	−19.3 dB	−2.9 dB	8.9 dB

The kick was already easy to get by filtering the low end, so the gain there is small. The snare and the cymbals are where a fixed band split fails and the factorisation earns its keep. The test suite checks these floors on every build and prints the numbers, and the stems rack in seance shows the same ones.

The separator we threw away#

seance used to carry its own harmonic-percussive separator, written before the shared audio library existed. When we measured both on the same groove, the old one did not just lose. It made the harmonic stem worse than not separating at all:

Stem	The plain mix	Old separator	Kept separator
Harmonic	−2.5 dB	−10.5 dB	4.1 dB
Percussive	3.0 dB	−19.3 dB	7.7 dB

It sounded plausible. It produced two stems, one tonal and one clicky. Without a number, nobody would have noticed that the result was worse than the input. The comparison stays in the test suite, so the baseline can still be run instead of quoted.

Two ideas that did not work#

• Smarter starting points. Seeding the factorisation with spectra taken from the hits sounds clever. It locks the updates into shapes that look like the whole mix. Random starts, chosen by how well they fit, did better and stay reproducible with fixed seeds.
• Soft labels. Letting each shape belong partly to several drums let broad snare shapes flood the cymbal stem. One label per shape, corrected by timing, holds up.

Where it stops#

The cymbals are good in a mix but not clean when soloed. Low notes that play with the kick can land in the kick stem. On a full song the percussive stem holds more than drums, and there is no vocal stem at all. For a drum loop or a break, it is quick, it is honest about its numbers, and it never leaves your computer.

Try it on your own loop with drum stem separation in seance.