MelSpectrogram

mel_filterbank @ |STFT|²: the power spectrogram reduced to perceptually-spaced mel bands, the standard front end for speech and music models. Fused into one kernel: each frame’s spectrum is reduced to mel bands while still in shared memory; the spectrogram never materializes. Cepstral coefficients on top of it are MFCC.

specux.melspectrogram(x, *, sr, n_fft=2048, hop_length=None, win_length=None,
                      window="hann", center=True, n_mels=128, fmin=0.0,
                      fmax=None, mel_scale="slaney", norm="slaney", power=2.0,
                      output="power", eps=1e-10, backend="auto")

numpy

import numpy as np

import specux

x = np.random.randn(8, 32768).astype(np.float32)
M = specux.melspectrogram(x, sr=16000, n_fft=1024, n_mels=80)
M.shape        # (8, 80, 129) = (..., n_mels, n_frames)

torch

import torch

import specux

xt = torch.randn(8, 32768, device="cuda", requires_grad=True)
M = specux.melspectrogram(xt, sr=16000, n_fft=1024, n_mels=80)
M.sum().backward()             # differentiable, on the GPU end to end

cupy

import cupy

import specux

xc = cupy.random.standard_normal((8, 32768), dtype=cupy.float32)
M = specux.melspectrogram(xc, sr=16000, n_fft=1024, n_mels=80)
M.shape        # (8, 80, 129), stays on the GPU, no torch

sr is required and keyword-only: the mel scale is meaningless without the true sample rate. Leading dimensions pass through: (..., time) in, (..., n_mels, n_frames) out.

Parameters

• sr: sample rate in Hz.
• n_fft, hop_length, win_length, window, center: framing, as in STFT.
• n_mels: number of mel bands (default 128).
• fmin, fmax: filterbank range; fmax=None means sr / 2.
• mel_scale: "slaney" (default) or "htk" [stevens1937].
• norm: "slaney" area-normalizes each triangle [slaney1998]; None keeps peak-1 triangles.
• power: 2.0 for power (default), 1.0 for magnitude.
• output: "power" (linear mel energies) or "db" (10·log10(mel + eps)).
• backend: "auto", "cuda", "cpu", or "metal".

The filterbank itself

specux.mel_filterbank(sr, n_fft, n_mels=128, fmin=0.0, fmax=None, htk=False, norm="slaney") returns the (n_mels, n_fft // 2 + 1) matrix as a torch tensor, the same one the fused kernel bakes in:

The scale converters behind it are public too:

m = specux.hz_to_mel(440.0)      # 6.6 on the slaney scale (default)
specux.mel_to_hz(m)              # 440.0; htk=True selects the HTK formula

They accept scalars or numpy arrays and are exact inverses of each other, for both the slaney and HTK scale formulas.

The configured form

specux.MelSpectrogram holds the parameters and broadcasts over any leading shape; to_dict() round-trips the configuration:

t = specux.MelSpectrogram(sr=44100, n_fft=1024, n_mels=80)
M = t(x)                                       # (..., n_mels, n_frames)
assert specux.MelSpectrogram(**t.to_dict()) == t

The torch Module for training pipelines is specux.transforms.MelSpectrogram (window and filterbank registered as buffers, nn.Sequential-ready).

References

• [stevens1937]: the mel pitch scale.
• [slaney1998]: the slaney filterbank convention.