musetric/uvr-mdxnet-kara2-onnx

UVR-MDX-NET KARA 2 — lead/backing vocal separation (ONNX, WebGPU)

ONNX graph of the UVR-MDX-NET KARA 2 karaoke model from Ultimate Vocal Remover, packaged for the musetric packages/ai runtime (onnxruntime-web on WebGPU).

The graph is the network only: it takes a complex spectrogram chunk and returns the complex spectrogram of everything except the lead vocal. The weights are those of the UVR release file; the operator layout is rewritten so that the WebGPU execution provider computes it correctly on Adreno GPUs (see the known defect below). STFT, chunking, overlap-add and the lead/backing split run in the host. This is not a drop-in audio -> stems model.

vocal stem, 44100 Hz stereo, peak-normalized
  -> STFT (n_fft 5120, hop 1024, periodic Hann, reflect-padded center frames)
  -> bins 0..2047, bins 0..2 zeroed -> input [1, 4, 2048, 256] per chunk
  -> kara2.onnx                     -> output [1, 4, 2048, 256] per chunk
  -> bins 2048..2560 zeroed, iSTFT, overlap-add across chunks -> separated
  -> backing = separated · peak
  -> lead    = (vocal − 1.065 · separated) · peak

Intended uses & limitations

Intended:

• Splitting an isolated vocal stem into lead and backing vocals, as the second separation stage of an audio pipeline (after vocals/instrumental).
• Client/edge inference via WebGPU through onnxruntime-web, on desktop and on mobile GPUs alike. The wasm EP gives the same result.

Out of scope:

• Use on full mixes: the model expects a vocal stem.
• Standalone use without a host that computes the STFT input and inverts the output (see musetric packages/ai and packages/server).
• Use in other training frameworks — this is an inference-only graph.

Limitations:

• The host contract is exact. 44100 Hz stereo; n_fft 5120, hop 1024, periodic Hann window 0.5 − 0.5·cos(2πn/N), frames centered with reflect padding of n_fft/2; the first 2048 of 2561 bins, with bins 0–2 zeroed; channels ordered [left real, left imaginary, right real, right imaginary]. A different window, bin count or channel order gives wrong output without an error.
• One static chunk: batch 1 and 256 frames (261,120 samples). The source graph declared a dynamic batch; this one does not.
• The lead stem is a residual with UVR's compensation factor 1.065, not a second model output.
• Training-data provenance of the upstream weights is not documented upstream.

Known defect: onnxruntime WebGPU on Adreno 6xx, and the static rewrite (2026-09-14)

On-device verification (one deterministic input, wasm EP of the same device as the reference, the same input bytes on both sides) showed the WebGPU EP returns silently wrong output for the source graph on Adreno 660: 2,094,338 of 2,097,152 elements differ by more than 1e-2 (max |Δ| 9.77 on values near 1), and two runs in a row differ from each other. Desktop NVIDIA matches wasm within 2.3e-5 on the same input. In a separation pipeline this leaves content in the backing vocal stem that is silent on desktop.

Moving operator families to the CPU on the device isolates the cause: BatchNormalization, Conv and ConvTranspose each corrupt the graph, and all three must leave the GPU for the output to match wasm. Transpose (0,1,3,2), MatMul, Relu, Add and Mul compute it correctly.

kara2.onnx ships the static rewrite, built only from the working operators:

Source	Rewrite
BatchNormalization	Mul + Add with folded running statistics
Conv	sum over kernel offsets: Pad → Slice with the stride → Reshape [C_in, H·W] → weight-left MatMul
ConvTranspose, kernel = stride	one weight-left MatMul per kernel offset, interleaved with Concat + Reshape
Conv over more than 65536 positions	the same, band by band of output rows, joined with Concat

No transposition and no index tables are added, so the file stays the size of the source. Rebuild: rewrite_static_adreno.py --max-columns 65536 in musetric-toolkit scripts/onnx/kara2/. Full evidence: the musetric plan gpu/kara2-adreno660-2026-09-14.md.

The row bands keep mobile GPUs responsive. At full resolution a 3×3 convolution is nine MatMuls over all 524,288 positions of the 256 × 2048 plane, and onnxruntime-web submits 16 dispatches at a time, so on Adreno 660 one submission held the GPU for about half a second. The full-resolution convolutions now run in eight bands and the half-resolution ones in two. Every position keeps its arithmetic, and the output is bit-identical to the previous revision on WebGPU.

How to use

The session runs one chunk; the host supplies input and consumes output.

import * as ort from 'onnxruntime-web/webgpu';

const session = await ort.InferenceSession.create('kara2.onnx', {
  executionProviders: [{ name: 'webgpu', storageBufferCacheMode: 'simple' }],
});

// spectrum: Float32Array of shape [1, 4, 2048, 256], produced host-side from one
// 261,120-sample stereo chunk of the vocal stem (see the contract above).
const input = new ort.Tensor('float32', spectrum, [1, 4, 2048, 256]);
const { output } = await session.run({ input });
// output: float32 [1, 4, 2048, 256] -> iSTFT and overlap-add host-side.

See the musetric host code (packages/ai/src/runtime/leadBacking/, packages/server/src/analysis/voices.rs) for the full STFT, chunk plan and lead/backing split.

Files

File	Size	SHA256
kara2.onnx	53,249,592 B	c59e3b9d2288cf8ad1099fe2b99e6de008d825aee2720f401c02bf63c596c127

Signature — float32 weights, opset ai.onnx 13, IR 6:

Tensor	Type	Shape	Meaning
input (in)	float32	[1, 4, 2048, 256]	complex spectrogram of the vocal stem chunk
output (out)	float32	[1, 4, 2048, 256]	complex spectrogram of the backing vocals

Validation

The rewrite, against the source graph on the same deterministic input:

Runtime	Comparison	Max |Δ|
onnxruntime CPU provider	rewrite vs source	1.14e-5
Adreno 660, WebGPU, three runs	rewrite vs wasm of the same device	1.17e-5, deterministic
desktop NVIDIA, WebGPU	rewrite vs wasm	1.17e-5, deterministic

On Adreno 660 a session plus four runs took 39.4–39.5 s for the rewrite against 35.4–35.8 s for the source graph.

End to end, the musetric app on a 10 s song through all processing steps, practice gains of the resulting stems:

Gain, dB	desktop, this graph	Adreno 660, this graph	Adreno 660, source graph
backing	12 (silent stem)	12 (silent stem)	5.461
lead	−1.277	−1.274	0.272
lead spectrogram	−7.201	−7.197	−6.167
instrumental	−6.959	−6.959	−6.959

The row bands, against the previous revision 7d49dab on the same deterministic input, one warm-up run and three runs, each run paced one submission at a time as musetric does. The foreign job wait is how long a one-dispatch job submitted every 100 ms on another GPU device waits for its turn:

Device	Output	Run, previous	Run, this	Foreign job wait p95, previous	This
Adreno 660, Chrome 152	bit-identical	7.8–8.3 s	8.4 s	361 ms	132 ms
Adreno 750, Chrome 153	bit-identical	2.9–3.5 s	3.0–3.1 s	144–155 ms	60 ms
desktop NVIDIA, Chrome 153	bit-identical	541–592 ms	623–637 ms	4 ms	5 ms

The worst wait fell from 456–527 ms to 267 ms on Adreno 660 and from 180–191 ms to 81 ms on Adreno 750.

Re-run the parity gate on the exact published bytes before relying on it.

Source & lineage

Code license and weight license are separate; ONNX conversion does not change the weight license. Documented only as far as it is verifiable.

• Architecture: MDX-Net (TFC-TDF U-Net) over a complex spectrogram: a 1×1 input convolution to 48 channels; at each of six resolutions (48 → 288 channels, 2048 → 64 frequency bins) three 3×3 conv–ReLU layers and a residual dense block that squeezes the frequency axis 8× with batch norm and ReLU; 2×2 strided convolutions down, 2×2 transposed convolutions with batch norm and ReLU up, skip connections merged by multiplication; a 1×1 output convolution. 13.19 M parameters. Original MDX-Net code: kuielab/mdx-net.
• Weights: Ultimate Vocal Remover (Anjok07, aufr33), MIT.
• Source file: UVR_MDXNET_KARA_2.onnx from the UVR model release TRvlvr/model_repo all_public_uvr_models — 52,786,726 B, sha256 bf32e15105a09c0f7dddd2b67346146334d6f3ecb399ed7638eba2ab07cbf5f4. A release asset is not content-addressed, so the sha256 is the pin.
• Export tooling: scripts/onnx/kara2 in musetric-toolkit.
• Host runtime: packages/ai and packages/server in musetric.

This graph preserves the upstream MIT license; we do not claim authorship of the original weights.

License

MIT, inherited from the upstream weights.