SPRINGLab

/

Indic-Mio

like 28

Follow

SPRINGLab 132

The 22-language coverage and emotion tags are really useful, and the model itself works well once the codec is wired up correctly.

I tried the “Approach 2: Directly with Transformers” example from the model card against miocodec commit 7747354437, transformers==4.51.3, and torch==2.5.1.

The snippet currently has a few issues:

1. decode() is being called with the wrong argument

The card uses:

wav = codec.decode(codes_tensor)

but the signature is:

MioCodec.decode(
    self,
    global_embedding=None,
    content_token_indices=None,
    content_embedding=None,
    target_audio_length=None,
    features=None
)

So codes_tensor gets passed as global_embedding, and no content tokens are actually provided:

ValueError: Either content_token_indices or content_embedding must be provided.

2. There is no speaker embedding / reference audio

Even after fixing the argument order, global_embedding is required. The example doesn't provide a speaker embedding, reference audio, preset, or call synthesize_from_tokens().

So this isn't just an argument-order issue — there's a required input missing from the example.

3. The tensor shape is also wrong

The card creates:

torch.tensor([audio_codes]).unsqueeze(0)

which gives [1, 1, T].

decode() internally does an unsqueeze(0) on the content embedding, so passing an already-batched tensor eventually produces a 4-D tensor and fails here:

module/transformer.py:594
ValueError: too many values to unpack (expected 3)

I tested all 9 combinations of:

• content: [T], [1,T], [1,1,T]
• global: [128], [1,128], [1,1,128]

The only working combination for decode() is:

content -> [T]
global  -> [128]

For batched inputs, decode_batch() is the appropriate entry point and accepts [B, ...] inputs.

4. The sample rate doesn't match the codec

The card loads:

Aratako/MioCodec-25Hz-24kHz

whose config says:

codec.config.sample_rate == 24000

but then writes:

sf.write("output.wav", wav, 44100)

That makes the output play ~1.84x too fast.

The sample rate should come directly from the codec config rather than being hardcoded.

Minimal reproduction

This reproduces the issues without downloading the LM:

import torch
from miocodec import MioCodec, MioCodecModel

# The card's loader doesn't work with this checkpoint:
try:
    MioCodec.from_pretrained("Aratako/MioCodec-25Hz-24kHz")
except ValueError as e:
    print(e)
    # No vocoder weights found with prefix 'vocoder.'

m = MioCodecModel.from_pretrained(
    "Aratako/MioCodec-25Hz-24kHz"
).eval()

print(m.config.sample_rate)
# 24000

codes = torch.randint(0, 12800, (50,), dtype=torch.long)
g = torch.randn(128)

# Fails because content is pre-batched
m.decode(
    global_embedding=g,
    content_token_indices=codes[None, None, :]
)

# Also fails: codes are passed as global_embedding
m.decode(torch.tensor([codes.tolist()]).unsqueeze(0))

# Works
m.decode(
    global_embedding=g,
    content_token_indices=codes
)

Suggested replacement

I'd change the card's Approach 2 to something along these lines:

import torch
import soundfile as sf
from transformers import AutoTokenizer, AutoModelForCausalLM
from miocodec import MioCodecModel, load_audio

tokenizer = AutoTokenizer.from_pretrained(
    "SPRINGLab/Indic-Mio",
    trust_remote_code=True
)

model = AutoModelForCausalLM.from_pretrained(
    "SPRINGLab/Indic-Mio",
    dtype=torch.bfloat16,
    device_map="cuda"
)

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "नमस्ते, आप कैसे हैं?"}],
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

output = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.9,
    top_p=0.9
)

generated = output[0][inputs["input_ids"].shape[1]:]

SPEECH_OFFSET = tokenizer.convert_tokens_to_ids("<|s_0|>")

audio_codes = [
    t.item() - SPEECH_OFFSET
    for t in generated
    if SPEECH_OFFSET <= t.item() < SPEECH_OFFSET + 12800
]

codec = MioCodecModel.from_pretrained(
    "Aratako/MioCodec-25Hz-44.1kHz-v2"
).eval().cuda()

reference = load_audio(
    "reference_speaker.wav",
    sample_rate=codec.config.sample_rate
).cuda()

wav = codec.synthesize_from_tokens(audio_codes, reference)

sf.write(
    "output.wav",
    wav.cpu().numpy(),
    codec.config.sample_rate
)

The important distinction here is that decode() itself isn't broken — the issue is that the current model-card example is using the wrong loader, missing the speaker input, passing pre-batched tensors to an unbatched API, and hardcoding the wrong sample rate.

I think updating the card would save people a fair amount of debugging time.