Convert BPE merges for tokenizers 0.19.1 compatibility
Browse files- tokenizer.json +2 -2
tokenizer.json
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d8d1c97bbabe51a8b9443165d2d20a5df15e42e09ac5c09f4c5c7a5c8f5abdcb
|
| 3 |
+
size 4757461
|