kipasyangin5 commited on
Commit
388ae6d
·
verified ·
1 Parent(s): eedec79

Fix markdown LaTeX formatting in model card

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -20,7 +20,7 @@ widget:
20
 
21
  # 5M Terminal & Multilingual Language Model (Chinchilla Optimal - 100M Tokens)
22
 
23
- This repository contains a **5.0 Million Parameter Causal Language Model** trained from scratch on a compute-optimal token budget of **100 Million Tokens** ($20\times$ parameter count) following Chinchilla scaling laws.
24
 
25
  The model is specialized in **Linux Terminal Commands, Shell Automation, and Multilingual Text Generation** (English, Indonesian, Spanish, French, German).
26
 
@@ -35,7 +35,7 @@ The model was trained on **100,000,000 Tokens** mixed across three distinct sour
35
  | **Linux Terminal & CLI Engine** | Synthesized Shell Engine | **~20.0 Million** | **20%** | **100% Fully Utilized**: Synthesized Linux CLI commands (`cd ..`, `ls -la`, `mkdir`, `grep`, `git`, `chmod`, `curl`, Q&A pairs). |
36
  | **English General Text** | [`wikitext`](https://huggingface.co/datasets/wikitext) (`wikitext-2-raw-v1`) | **~24.0 Million** | **24%** | **Streamed Sample**: Streamed line-by-line via Hugging Face `load_dataset("wikitext", streaming=True)`. |
37
  | **Multilingual Wikipedia** | [`wikimedia/wikipedia`](https://huggingface.co/datasets/wikimedia/wikipedia) | **~56.0 Million** | **56%** | **Streamed Multilingual Sample**: Streamed across 4 language sub-splits (Indonesian `id`, Spanish `es`, French `fr`, German `de`). |
38
- | **TOTAL** | Mixed Corpus | **100.0 Million** | **100%** | **Chinchilla Compute-Optimal** ($20\times$ params) |
39
 
40
  ---
41
 
 
20
 
21
  # 5M Terminal & Multilingual Language Model (Chinchilla Optimal - 100M Tokens)
22
 
23
+ This repository contains a **5.0 Million Parameter Causal Language Model** trained from scratch on a compute-optimal token budget of **100 Million Tokens** (20x parameter count) following Chinchilla scaling laws.
24
 
25
  The model is specialized in **Linux Terminal Commands, Shell Automation, and Multilingual Text Generation** (English, Indonesian, Spanish, French, German).
26
 
 
35
  | **Linux Terminal & CLI Engine** | Synthesized Shell Engine | **~20.0 Million** | **20%** | **100% Fully Utilized**: Synthesized Linux CLI commands (`cd ..`, `ls -la`, `mkdir`, `grep`, `git`, `chmod`, `curl`, Q&A pairs). |
36
  | **English General Text** | [`wikitext`](https://huggingface.co/datasets/wikitext) (`wikitext-2-raw-v1`) | **~24.0 Million** | **24%** | **Streamed Sample**: Streamed line-by-line via Hugging Face `load_dataset("wikitext", streaming=True)`. |
37
  | **Multilingual Wikipedia** | [`wikimedia/wikipedia`](https://huggingface.co/datasets/wikimedia/wikipedia) | **~56.0 Million** | **56%** | **Streamed Multilingual Sample**: Streamed across 4 language sub-splits (Indonesian `id`, Spanish `es`, French `fr`, German `de`). |
38
+ | **TOTAL** | Mixed Corpus | **100.0 Million** | **100%** | **Chinchilla Compute-Optimal** (20x params) |
39
 
40
  ---
41