---
license: other
license_name: pantagruel-research-license
license_link: https://huggingface.co/ina-foss/ssl-vad-base/blob/main/LICENSE.md
extra_gated_prompt: >-
  You agree to use the model according to its
  [license](https://huggingface.co/ina-foss/ssl-vad-base/blob/main/LICENSE.md).
extra_gated_fields:
  Research institute: text
  Institutional email address: text
  Country: country
  I agree to use this model for non-commercial research use only: checkbox
  I have read, and I agree to use the model according to its license: checkbox
pipeline_tag: voice-activity-detection
library_name: transformers
tags:
- speech
- voice
- voice activity detection
- VAD
base_model:
- ina-foss/ssl-audio-1k-base
---

# Voice Activity Detection using SSL features

This model is trained to detect and segment voice activity on an given audio file.
It uses SSL features from the [`ina-foss/ssl-audio-1k-base`](https://huggingface.co/ina-foss/ssl-audio-1k-base) model.

You can find the full list of **other Voice Activity Detection (VAD) models** using SSL features [**here**](https://huggingface.co/collections/ina-foss/ssl-vad).
Their global results on the test set of InaGVAD are the following (see the paper for more details):

| **Link to model** | **Global Acc.** | **Global F1.** |
|-------------------|-----------------|----------------|
| [ssl-vad-music2vec](https://huggingface.co/ina-foss/ssl-vad-music2vec) | 96.4 | 97.0 |
| ssl-vad-base | **96.8** | **97.3** |
| [ssl-vad-no_music](https://huggingface.co/ina-foss/ssl-vad-no_music) | 96.0 | 96.6 |
| [ssl-vad-only_speech](https://huggingface.co/ina-foss/ssl-vad-only_speech) | 96.2 | 96.8 |
| [ssl-vad-only_fr](https://huggingface.co/ina-foss/ssl-vad-only_fr) | 96.3 | 96.9 |
| [ssl-vad-gender](https://huggingface.co/ina-foss/ssl-vad-gender) | 96.5 | 97.0 |

*Music detection models using SSL features can be found [here](https://huggingface.co/collections/ina-foss/ssl-music-detection).*

## Architecture
The model first extract features from the CNN and the first transformer layer of the [`ina-foss/ssl-audio-1k-base`](https://huggingface.co/ina-foss/ssl-audio-1k-base) SSL encoder.
Then, these features are given to a downstream model MLP, which has been trained to binary predict voice activity for each frame.
During inference, the decoding uses a Viterbi decoder (from [Librosa](https://librosa.org/doc/latest/generated/librosa.sequence.viterbi_binary.html)).

## Data and training
It has been trained on the validation subset of the [InaGVAD dataset](https://aclanthology.org/2024.lrec-main.785/) by Doukhan et al. (2024).

For detailed information about training and results associated with this model, please refer to our publication.
The training hyperparameters, original checkpoint and Tensorboard event files are available in the [training directory](https://huggingface.co/ina-foss/ssl-vad-base/tree/main/training).

## Usage

To use this model, you need the packages listed inside the [requirements.txt file](https://huggingface.co/ina-foss/ssl-vad-base/blob/main/requirements.txt).
Then:

```python
import librosa
from transformers import AutoModel

# loading the audio file, need to be sampled at 16kHz
audio, sr = librosa.load('/path/to/your/audio/file.wav', sr=16000)

# loading the music detection model
model = AutoModel.from_pretrained(
    'ina-foss/ssl-vad-base',
    trust_remote_code=True
)

# running the inference
output = model(
    audio=audio,
    sampling_rate=sr
)

print(output)
```

```
[{'start': 0.0, 'stop': 56.58943157192866, 'label': False},
 {'start': 56.58943157192866, 'stop': 60.45007501250208, 'label': True},
 {'start': 60.45007501250208, 'stop': 62.870478413068845, 'label': False},
[...]
 {'start': 117.03950658443074, 'stop': 119.21986997832973, 'label': True},
 {'start': 119.21986997832973, 'stop': 119.97999666611102, 'label': False}]
```

## License and citation

The model is distributed using the [pantagruel-research-license](https://huggingface.co/ina-foss/ssl-vad-base/blob/main/LICENSE.md).

If you use this model or find it useful in your research, publications, or applications, please cite the following work:
```
@inproceedings{pelloin2026lrec,
  author =       "Pelloin, Valentin and Bekkali, Lina and Dehak, Reda and Doukhan, David",
  year =         "2026",
  title =        "Data Selection Effects on Self-Supervised Learning of Audio Representations for French Audiovisual Broadcasts",
  booktitle={Fifteenth International Conference on Language Resources and Evaluation (LREC 2026)},
  address = "Palma, Mallorca, Spain",
  publisher = "European Language Resources Association",
}
```