Compactbot commited on
Commit
0396c3b
·
1 Parent(s): 1154cc3

Add BPE tokenizer (vocab 8192) (#3)

Browse files

- Add BPE tokenizer (vocab 8192) (04ed7606e406619515cb5e4d4dc6c6cea7857314)

Files changed (1) hide show
  1. tokenizer.json +0 -0
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff