ESPnet
audio
self-supervised-learning
beats
jaeyeonkim99 commited on
Commit
2f73d42
·
verified ·
1 Parent(s): 1462b93

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +54 -0
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - espnet
4
+ - audio
5
+ - self-supervised-learning
6
+ - beats
7
+ datasets:
8
+ - audioset
9
+ license: cc-by-4.0
10
+ arxiv: 2212.09058
11
+ ---
12
+
13
+ # BEATs base encoder — AudioSet-2M, iteration 1
14
+
15
+ BEATs base audio encoder pre-trained on AudioSet-2M with ESPnet, reproducing
16
+ [BEATs](https://arxiv.org/abs/2212.09058) (Table 4). Iteration 1 is trained against the
17
+ discrete targets of the [iteration-1 VQ tokenizer](https://huggingface.co/jaeyeonkim99/BEATs-tokenizer-AS2M-iter1).
18
+ Produced by the [egs2/audioset/ssl1](https://github.com/espnet/espnet/tree/master/egs2/audioset/ssl1) recipe.
19
+
20
+ ## Usage
21
+
22
+ Load into an ESPnet downstream classification recipe (e.g. `egs2/esc50/asr1`,
23
+ `egs2/as20k/cls1`) via `beats_ckpt_path`:
24
+
25
+ ```yaml
26
+ encoder: beats
27
+ encoder_conf:
28
+ beats_ckpt_path: beats_encoder_iter1.pt
29
+ fbank_mean: 15.66439
30
+ fbank_std: 6.38312
31
+ ```
32
+
33
+ ## Downstream results
34
+
35
+ | | AS-20K test mAP | ESC-50 (5-fold acc) |
36
+ |---|---|---|
37
+ | iter 1 | 31.26 | 93.35 |
38
+
39
+ ## Citation
40
+
41
+ ```bibtex
42
+ @inproceedings{chen2022beats,
43
+ title={BEATs: Audio Pre-Training with Acoustic Tokenizers},
44
+ author={Chen, Sanyuan and Wu, Yu and Wang, Chengyi and Liu, Shujie and
45
+ Tompkins, Daniel and Chen, Zhuo and Wei, Furu},
46
+ booktitle={ICML}, year={2023}
47
+ }
48
+ @inproceedings{bharadwaj2025openbeats,
49
+ title={OpenBEATs: A Fully Open-Source General-Purpose Audio Encoder},
50
+ author={Bharadwaj, Shikhar and Cornell, Samuele and Choi, Kwanghee and
51
+ Fukayama, Satoru and Shim, Hye-jin and Deshmukh, Soham and Watanabe, Shinji},
52
+ booktitle={WASPAA}, year={2025}
53
+ }
54
+ ```