Pragya / tokenizer_config (2).json
ArushBuilds's picture
Upload 2 files
4bb7338 verified
Raw History Blame Contribute Delete
102 Bytes
{
"vocab_size": 16384,
"bos_id": 16383,
"eos_id": 16383,
"pad_id": 16383,
"unk_id": 16383
}