Instructions to use aisingapore/Llama-SEA-LION-v3-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aisingapore/Llama-SEA-LION-v3-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aisingapore/Llama-SEA-LION-v3-8B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("aisingapore/Llama-SEA-LION-v3-8B")
model = AutoModelForCausalLM.from_pretrained("aisingapore/Llama-SEA-LION-v3-8B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aisingapore/Llama-SEA-LION-v3-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aisingapore/Llama-SEA-LION-v3-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/aisingapore/Llama-SEA-LION-v3-8B

SGLang

How to use aisingapore/Llama-SEA-LION-v3-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aisingapore/Llama-SEA-LION-v3-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aisingapore/Llama-SEA-LION-v3-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use aisingapore/Llama-SEA-LION-v3-8B with Docker Model Runner:
```
docker model run hf.co/aisingapore/Llama-SEA-LION-v3-8B
```

Jian-Gang commited on Dec 18, 2024

Commit

2799f09

verified ·

1 Parent(s): b282b19

Update README.md

Browse files

Files changed (1) hide show

README.md +8 -9

README.md CHANGED Viewed

@@ -11,8 +11,7 @@ language:
 - km
 - lo
 - my
-- jv
-- su
 license: llama3.1
 library_name: transformers
 pipeline_tag: text-generation
@@ -21,19 +20,19 @@ base_model: meta-llama/Llama-3.1-8B-Instruct
 # Llama3.1 8B CPT SEA-LIONv3
 SEA-LION is a collection of Large Language Models (LLMs) which has been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
-Llama3.1 8B CPT SEA-LIONv3 Base is a multilingual model which has undergone continued pre-training on approximately **200B** tokens across the 11 official Southeast Asian languages: English, Chinese, Vietnamese, Indonesian, Thai, Tamil, Filipino, Malay, Khmer, Lao, Burmese.
 SEA-LION stands for <i>Southeast Asian Languages In One Network</i>.
 - **Developed by:** Products Pillar, AI Singapore
 - **Funded by:** Singapore NRF
 - **Model type:** Decoder
-- **Languages:** English, Chinese, Vietnamese, Indonesian, Thai, Filipino, Tamil, Malay, Khmer, Lao, Burmese, Javanese, Sundanese
 - **License:** [Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)
 ## Model Details
 ### Model Description
-We performed continued pre-training in English and ASEAN languages on [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct), a decoder model using the Llama 3.1 architecture, to create Llama3.1 8B CPT SEA-LIONv3 Base.
 For tokenisation, the model employs the default tokenizer used in Llama 3.1 8B Instruct.
@@ -41,14 +40,14 @@ For tokenisation, the model employs the default tokenizer used in Llama 3.1 8B I
 We evaluated Llama3.1 8B CPT SEA-LIONv3 base model on general language capabilities.
 #### General Language Capabilities
-For the evaluation of general language capabilities, we employed the [SEA HELM (also known as BHASA) evaluation benchmark](https://arxiv.org/abs/2309.06085v2) across a variety of tasks.
-These tasks include Question Answering (QA), Sentiment Analysis (Sentiment), Toxicity Detection (Toxicity), Translation in both directions (Eng>Lang & Lang>Eng), Abstractive Summarization (Summ), Causal Reasoning (Causal) and Natural Language Inference (NLI).
-Note: SEA HELM is implemented using prompts to elicit answers in a strict format. For all tasks, the model is expected to provide an answer tag from which the answer is automatically extracted. For tasks where options are provided, the answer should comprise one of the pre-defined options. The scores for each task is normalised to account for baseline performance due to random chance.
 The evaluation was done **five-shot** with native prompts on a sample of 100-1000 instances for each dataset.
-For more details on Llama3.1 8B CPT SEA-LIONv3 base benchmark performance, please refer to the SEA HELM leaderboard, https://leaderboard.sea-lion.ai/
 ## Technical Specifications
 ### Infrastructure

 - km
 - lo
 - my
 license: llama3.1
 library_name: transformers
 pipeline_tag: text-generation
 # Llama3.1 8B CPT SEA-LIONv3
 SEA-LION is a collection of Large Language Models (LLMs) which has been pretrained and instruct-tuned for the Southeast Asia (SEA) region.
+Llama3.1 8B CPT SEA-LIONv3 Base is a multilingual model which has undergone continued pre-training on approximately **200B** tokens across 11 SEA languages: English, Chinese, Vietnamese, Indonesian, Thai, Tamil, Filipino, Malay, Khmer, Lao, Burmese.
 SEA-LION stands for <i>Southeast Asian Languages In One Network</i>.
 - **Developed by:** Products Pillar, AI Singapore
 - **Funded by:** Singapore NRF
 - **Model type:** Decoder
+- **Languages supported:** English, Chinese, Vietnamese, Indonesian, Thai, Filipino, Tamil, Malay, Khmer, Lao, Burmese
 - **License:** [Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)
 ## Model Details
 ### Model Description
+We performed continued pre-training in English and SEA languages on [Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct), a decoder model using the Llama 3.1 architecture, to create Llama3.1 8B CPT SEA-LIONv3 Base.
 For tokenisation, the model employs the default tokenizer used in Llama 3.1 8B Instruct.
 We evaluated Llama3.1 8B CPT SEA-LIONv3 base model on general language capabilities.
 #### General Language Capabilities
+For the evaluation of general language capabilities, we employed the [SEA-HELM (also known as BHASA) evaluation benchmark](https://arxiv.org/abs/2309.06085v2) across a variety of tasks.
+These tasks include Question Answering (QA), Sentiment Analysis (Sentiment), Toxicity Detection (Toxicity), Translation in both directions (Eng>Lang & Lang>Eng), Abstractive Summarisation (Abssum), Causal Reasoning (Causal) and Natural Language Inference (NLI).
+Note: SEA-HELM is implemented using prompts to elicit answers in a strict format. For all tasks, the model is expected to provide an answer tag from which the answer is automatically extracted. For tasks where options are provided, the answer should comprise one of the pre-defined options. The scores for each task is normalised to account for baseline performance due to random chance.
 The evaluation was done **five-shot** with native prompts on a sample of 100-1000 instances for each dataset.
+For more details on Llama3.1 8B CPT SEA-LIONv3 base benchmark performance, please refer to the SEA-HELM leaderboard, https://leaderboard.sea-lion.ai/
 ## Technical Specifications
 ### Infrastructure