Commit ·
e6024b5
1
Parent(s): 753ccee
Update README.md
Browse files
README.md
CHANGED
|
@@ -2,4 +2,5 @@ Training Details:
|
|
| 2 |
* 200,000 steps on the Pile
|
| 3 |
* Denoising (span corruption) objective
|
| 4 |
* Batch size 256, 512 sequence length
|
| 5 |
-
* 25B tokens total
|
|
|
|
|
|
| 2 |
* 200,000 steps on the Pile
|
| 3 |
* Denoising (span corruption) objective
|
| 4 |
* Batch size 256, 512 sequence length
|
| 5 |
+
* 25B tokens total
|
| 6 |
+
* Modified GPT NeoX Tokenizer
|