Instructions to use aisingapore/Llama-SEA-LION-v3-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aisingapore/Llama-SEA-LION-v3-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aisingapore/Llama-SEA-LION-v3-8B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("aisingapore/Llama-SEA-LION-v3-8B")
model = AutoModelForCausalLM.from_pretrained("aisingapore/Llama-SEA-LION-v3-8B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aisingapore/Llama-SEA-LION-v3-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aisingapore/Llama-SEA-LION-v3-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/aisingapore/Llama-SEA-LION-v3-8B

SGLang

How to use aisingapore/Llama-SEA-LION-v3-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aisingapore/Llama-SEA-LION-v3-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aisingapore/Llama-SEA-LION-v3-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use aisingapore/Llama-SEA-LION-v3-8B with Docker Model Runner:
```
docker model run hf.co/aisingapore/Llama-SEA-LION-v3-8B
```

tainc commited on Dec 18, 2024

Commit

09bca8b

verified ·

1 Parent(s): e6014de

Update README.md

Browse files

Files changed (1) hide show

README.md +23 -39

README.md CHANGED Viewed

@@ -72,48 +72,32 @@ Llama3.1 8B CPT SEA-LIONv3 was trained using [MosaicML Composer](https://github.
 ## Data
 Llama3.1 8B CPT SEA-LIONv3 base model was continued pre-trained on 200B tokens of the following data:
-| Data Source                           | Unique Tokens (B) | Multiplier | Total Tokens (B) | Percentage (%)|
-|---------------------------------------|:-----------------:|:----------:|:----------------:|:-------------:|
-| StackV2                               | 40.0              | 1          | 40.0             | 20.00         |
-| Wiki* + News* - English               | 5.0               | 1          | 5.0              | 2.50          |
-| Fineweb-Edu                           | 7.5               | 1          | 7.5              | 3.75          |
-| Dolma Project Gutenberg               | 5.0               | 1          | 5.0              | 2.50          |
-| Dolma arXiv                           | 1.7               | 1          | 1.7              | 0.83          |
-| Dolma StackExchange                   | 1.7               | 1          | 1.7              | 0.83          |
-| Dolma Semantic Scholar                | 1.7               | 1          | 1.7              | 0.83          |
-| Dolma OpenWebMath                     | 2.5               | 1          | 2.5              | 1.25          |
-| Dolma Algebraic Stack                 | 2.5               | 1          | 2.5              | 1.25          |
-| Dolma Flan                            | 5.0               | 1          | 5.0              | 2.50          |
-| Dolma Reddit                          | 5.0               | 1          | 5.0              | 2.50          |
-| Dolma Megawika                        | 5.0               | 1          | 5.0              | 2.50          |
-| Dolma CC News                         | 7.5               | 1          | 7.5              | 3.75          |
-| Wiki* + News* - Chinese               | 3.5               | 4          | 14.0             | 7.00          |
-| SEA-LION Pile - Chinese               | 12.0              | 1          | 12.0             | 6.00          |
-| Wiki* + News* - Vietnamese            | 2.4               | 4          | 9.4              | 4.70          |
-| VinBigData - Vietnamese               | 2.1               | 4          | 8.2              | 4.10          |
-| SEA-LION Pile - Vietnamese            | 8.4               | 1          | 8.4              | 4.20          |
-| Wiki* + News* - Indonesian            | 1.3               | 4          | 5.2              | 2.60          |
-| SEA-LION Pile - Indonesian            | 20.8              | 1          | 20.8             | 10.40         |
-| Wiki* + News* + WangChanBERTa - Thai  | 1.3               | 4          | 5.2              | 2.60          |
-| SEA-LION Pile - Thai                  | 14.8              | 1          | 14.8             | 7.40          |
-| Wiki* + News - Filipino               | 0.2               | 4          | 0.9              | 0.43          |
-| SEA-LION Pile - Filipino              | 2.1               | 1          | 2.1              | 1.07          |
-| Wiki* + News - Tamil                  | 0.1               | 4          | 0.3              | 0.14          |
-| SEA-LION Pile - Tamil                 | 0.7               | 1          | 0.7              | 0.36          |
-| Wiki* + News - Malay                  | 0.1               | 4          | 0.6              | 0.29          |
-| SEA-LION Pile - Malay                 | 1.4               | 1          | 1.4              | 0.71          |
-| Wiki* + News - Khmer                  | 0.1               | 4          | 0.3              | 0.17          |
-| SEA-LION Pile - Khmer                 | 2.3               | 1          | 2.3              | 1.13          |
-| Wiki* + News - Lao                    | 0.0               | 4          | 0.1              | 0.03          |
-| SEA-LION Pile - Lao                   | 0.3               | 1          | 0.3              | 0.17          |
-| Wiki* + News - Burmese                | 0.1               | 4          | 0.4              | 0.20          |
-| SEA-LION Pile - Burmese               | 2.6               | 1          | 2.6              | 1.30          |
 Note:
 - All token counts are counted using Llama 3.1 8B Instruct tokenizer
-- Wiki* sources includes Wikipedia, Wiki Books, Wiki Source, Wiki Voyage and Fandom Wiki
-- News* sources includes VOA, Global Voices, MediaCorp, VinBigData-News
 - Tamil news is sourced with permission from [Seithi](https://seithi.mediacorp.sg/)
 ## Call for Contributions

 ## Data
 Llama3.1 8B CPT SEA-LIONv3 base model was continued pre-trained on 200B tokens of the following data:
+| Language                 | Source                                 | Total Tokens (B) | Percentage (%) | Total percentage (%) |
+| ------------------------ | -------------------------------------- | ---------------- | -------------- | -------------------- |
+| Code                     | StackV2                                | 40               | 20             | 20                   |
+| English                  | Dolma                                  | 37.5             | 18.75          | 25                   |
+|                          | Fineweb-Edu                            | 7.5              | 3.75           |
+|                          | Others                                 | 5                | 2.5            |
+| Chinese                  | SEA-LION Pile v1                       | 12               | 6              | 13                   |
+|                          | Others                                 | 14               | 7              |
+| Vietnamese               | SEA-LION Pile v1                       | 8.4              | 4.2            | 13                   |
+|                          | VinBigData                             | 16               | 8              |
+|                          | Others                                 | 1.6              | 0.8            |
+| Indonesian               | SEA-LION Pile v1                       | 7                | 3.5            | 13                   |
+|                          | SEA-LION Pile v2                       | 7                | 3.5            |
+|                          | Others                                 | 12               | 6              |
+| Thai                     | SEA-LION Pile v1                       | 10.7             | 5.35           | 10                   |
+|                          | WangChanBERTa                          | 8.5              | 4.25           |
+|                          | Others                                 | 0.8              | 0.4            |
+| Filipino - Malay - Tamil | SEA-LION Pile v1, AI4Bharat, Sangraha  | 4.28             | 2.14           | 3                    |
+|                          | Others                                 | 1.72             | 0.86           |
+| Khmer - Lao - Burmese    | SEA-LION Pile v1                       | 5.2              | 2.6            | 3                    |
+|                          | Others                                 | 0.8              | 0.4            |
 Note:
 - All token counts are counted using Llama 3.1 8B Instruct tokenizer
+- SEA-LION Pile v1 is processed from Common Crawl WET, which is published [here](https://huggingface.co/datasets/aisingapore/sea-lion-pile). The cutoff date of this version is September 2020.
+- SEA-LION Pile v2 is processed from Common Crawl WARC from October 2020 to April 2024.
 - Tamil news is sourced with permission from [Seithi](https://seithi.mediacorp.sg/)
 ## Call for Contributions