my-silly-t5 / README.md
andersonbcdefg's picture
Update README.md
e6024b5
|
Raw
History Blame Contribute Delete
172 Bytes

Training Details:

  • 200,000 steps on the Pile
  • Denoising (span corruption) objective
  • Batch size 256, 512 sequence length
  • 25B tokens total
  • Modified GPT NeoX Tokenizer