andersonbcdefg commited on
Commit
753ccee
·
1 Parent(s): f97db32

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -0
README.md ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ Training Details:
2
+ * 200,000 steps on the Pile
3
+ * Denoising (span corruption) objective
4
+ * Batch size 256, 512 sequence length
5
+ * 25B tokens total