Feature Extraction
NeMo
CasanovaE commited on
Commit
4b5996a
·
verified ·
1 Parent(s): a3b6ceb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -8
README.md CHANGED
@@ -7,7 +7,7 @@ pipeline_tag: feature-extraction
7
  ---
8
 
9
 
10
- # NVIDIA nemo-nano-codec
11
  <style>
12
  img{
13
  display: inline-table;
@@ -21,7 +21,7 @@ padding: 0;
21
  | [![Language](https://img.shields.io/badge/Language-multilingual-lightgrey#model-badge)](#datasets)
22
 
23
 
24
- The [nemo-nano-codec]() is a neural audio codec that leverages finite scalar quantization and adversarial training with large speech language models to achieve state-of-the-art audio compression across different bitrate and frame rate ranges.
25
  Model variant details:
26
 
27
  | Sample Rate | Frame Rate | Bit Rate | # Codebooks | Codebook Size | Embed Dim | FSQ Levels |
@@ -32,7 +32,7 @@ This model is ready for commercial/non-commercial use.
32
 
33
 
34
 
35
- ## nemo-nano-codec variants
36
 
37
  Model | Sample Rate | Frame Rate | Bit Rate | # Codebooks | Codebook Size | Embed Dim | FSQ Levels |
38
  :-----------:|:-----------:|:----------:|:----------:|:-----------:|:-------------:|:-----------:|:------------:|
@@ -55,13 +55,13 @@ Model | Sample Rate | Frame Rate | Bit Rate | # Codebooks | Codebook Size | Em
55
  <br>Huggingface [08/11/2025] via https://huggingface.co/nvidia/nanocodec-22khz-1.78kbps-12.5fps<br>
56
 
57
  ## Model Architecture
58
- nemo-nano-codec is composed of a fully convolutional generator neural network and three discriminators. The generator comprises an encoder, followed by vector quantization, and a [HiFi-GAN-based](https://arxiv.org/abs/2010.05646) decoder.
59
 
60
  The non-causal encoder consists of five residual blocks, each block containing three residual layers similar to the [multi-receptive field fusion (MRF) module](https://arxiv.org/abs/2010.05646). The causal decoder, based on the HiFi-GAN vocoder, uses upsampling rates that are the reverse of the encoder's One-Dimensional (1D) convolutional strides.
61
 
62
  For the vector quantization, we have used [Finite Scalar Quantization (FSQ)](https://arxiv.org/abs/2309.15505) with thirteen codebooks and four dimensions per code and 2016 codes per codebook. For the discriminators, we utilize three neural networks, all employing a squared-GAN and feature-matching loss. We adopt the [multi-period discriminator](https://arxiv.org/abs/2010.05646), [multi-band multi-scale STFT discriminator](https://arxiv.org/abs/2306.06546), and [WavLM-based discriminator](https://arxiv.org/abs/2409.12117).
63
 
64
- For more details please check [our paper]().
65
 
66
  **This model was developed based on [NVIDIA Low Frame-rate Speech Codec](https://huggingface.co/nvidia/low-frame-rate-speech-codec-22khz)**
67
 
@@ -184,11 +184,11 @@ For fine-tuning on another dataset, please follow the steps available at our [Au
184
 
185
  ## Training, Testing, and Evaluation Datasets:
186
 
187
- The nemo-nano-codec was trained on 28.7k hours of speech data spanning 105 languages. The model was evaluated using multilingual audiobook-style data and high-quality English recordings. For further details, refer to [our paper]().
188
 
189
 
190
  ### Training Datasets
191
- The nemo-nano-codec is trained on a total of 28.7k hrs of speech data from 105 languages.
192
 
193
  Link: [MLS English](https://www.openslr.org/94/) [25.5k]
194
 
@@ -243,7 +243,7 @@ Link: [Common Voice](https://huggingface.co/datasets/mozilla-foundation/common_v
243
 
244
  ## Performance
245
 
246
- We evaluated our codec using multiple objective audio quality metrics across two distinct test sets. Additionally, we compared our model's performance with state-of-the-art codecs. For further details, please refer to [our paper]().
247
 
248
 
249
  Variant results:
 
7
  ---
8
 
9
 
10
+ # NVIDIA NeMo NanoCodec
11
  <style>
12
  img{
13
  display: inline-table;
 
21
  | [![Language](https://img.shields.io/badge/Language-multilingual-lightgrey#model-badge)](#datasets)
22
 
23
 
24
+ The [NeMo NanoCodec](https://arxiv.org/abs/2508.05835v1) is a neural audio codec that leverages finite scalar quantization and adversarial training with large speech language models to achieve state-of-the-art audio compression across different bitrate and frame rate ranges.
25
  Model variant details:
26
 
27
  | Sample Rate | Frame Rate | Bit Rate | # Codebooks | Codebook Size | Embed Dim | FSQ Levels |
 
32
 
33
 
34
 
35
+ ## NeMo NanoCodec variants
36
 
37
  Model | Sample Rate | Frame Rate | Bit Rate | # Codebooks | Codebook Size | Embed Dim | FSQ Levels |
38
  :-----------:|:-----------:|:----------:|:----------:|:-----------:|:-------------:|:-----------:|:------------:|
 
55
  <br>Huggingface [08/11/2025] via https://huggingface.co/nvidia/nanocodec-22khz-1.78kbps-12.5fps<br>
56
 
57
  ## Model Architecture
58
+ NeMo NanoCodec is composed of a fully convolutional generator neural network and three discriminators. The generator comprises an encoder, followed by vector quantization, and a [HiFi-GAN-based](https://arxiv.org/abs/2010.05646) decoder.
59
 
60
  The non-causal encoder consists of five residual blocks, each block containing three residual layers similar to the [multi-receptive field fusion (MRF) module](https://arxiv.org/abs/2010.05646). The causal decoder, based on the HiFi-GAN vocoder, uses upsampling rates that are the reverse of the encoder's One-Dimensional (1D) convolutional strides.
61
 
62
  For the vector quantization, we have used [Finite Scalar Quantization (FSQ)](https://arxiv.org/abs/2309.15505) with thirteen codebooks and four dimensions per code and 2016 codes per codebook. For the discriminators, we utilize three neural networks, all employing a squared-GAN and feature-matching loss. We adopt the [multi-period discriminator](https://arxiv.org/abs/2010.05646), [multi-band multi-scale STFT discriminator](https://arxiv.org/abs/2306.06546), and [WavLM-based discriminator](https://arxiv.org/abs/2409.12117).
63
 
64
+ For more details please check [our paper](https://arxiv.org/abs/2508.05835v1).
65
 
66
  **This model was developed based on [NVIDIA Low Frame-rate Speech Codec](https://huggingface.co/nvidia/low-frame-rate-speech-codec-22khz)**
67
 
 
184
 
185
  ## Training, Testing, and Evaluation Datasets:
186
 
187
+ The NeMo NanoCodec was trained on 28.7k hours of speech data spanning 105 languages. The model was evaluated using multilingual audiobook-style data and high-quality English recordings. For further details, refer to [our paper](https://arxiv.org/abs/2508.05835v1).
188
 
189
 
190
  ### Training Datasets
191
+ The NeMo NanoCodec is trained on a total of 28.7k hrs of speech data from 105 languages.
192
 
193
  Link: [MLS English](https://www.openslr.org/94/) [25.5k]
194
 
 
243
 
244
  ## Performance
245
 
246
+ We evaluated our codec using multiple objective audio quality metrics across two distinct test sets. Additionally, we compared our model's performance with state-of-the-art codecs. For further details, please refer to [our paper](https://arxiv.org/abs/2508.05835v1).
247
 
248
 
249
  Variant results: