Instructions to use aisingapore/Llama-SEA-LION-v3-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aisingapore/Llama-SEA-LION-v3-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aisingapore/Llama-SEA-LION-v3-8B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("aisingapore/Llama-SEA-LION-v3-8B")
model = AutoModelForCausalLM.from_pretrained("aisingapore/Llama-SEA-LION-v3-8B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aisingapore/Llama-SEA-LION-v3-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aisingapore/Llama-SEA-LION-v3-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/aisingapore/Llama-SEA-LION-v3-8B

SGLang

How to use aisingapore/Llama-SEA-LION-v3-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aisingapore/Llama-SEA-LION-v3-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aisingapore/Llama-SEA-LION-v3-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aisingapore/Llama-SEA-LION-v3-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use aisingapore/Llama-SEA-LION-v3-8B with Docker Model Runner:
```
docker model run hf.co/aisingapore/Llama-SEA-LION-v3-8B
```

Jian-Gang commited on Dec 19, 2024

Commit

d12aece

verified ·

1 Parent(s): 45d3b8e

Update README.md

Browse files

Files changed (1) hide show

README.md +7 -1

README.md CHANGED Viewed

@@ -27,7 +27,7 @@ SEA-LION stands for <i>Southeast Asian Languages In One Network</i>.
 - **Developed by:** Products Pillar, AI Singapore
 - **Funded by:** Singapore NRF
 - **Model type:** Decoder
-- **Languages supported:** English, Chinese, Vietnamese, Indonesian, Thai, Filipino, Tamil, Malay, Khmer, Lao, Burmese
 - **License:** [Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)
 ## Model Details
@@ -47,8 +47,14 @@ Note: SEA-HELM is implemented using prompts to elicit answers in a strict format
 The evaluation was done **five-shot** with native prompts on a sample of 100-1000 instances for each dataset.
 For more details on Llama3.1 8B CPT SEA-LIONv3 base benchmark performance, please refer to the SEA-HELM leaderboard, https://leaderboard.sea-lion.ai/
 ## Technical Specifications
 ### Infrastructure
 Llama3.1 8B CPT SEA-LIONv3 was trained using [MosaicML Composer](https://github.com/mosaicml/composer) on the following hardware:

 - **Developed by:** Products Pillar, AI Singapore
 - **Funded by:** Singapore NRF
 - **Model type:** Decoder
+- **Languages supported:** Burmese, Chinese, English, Filipino, Indonesia, Khmer, Lao, Malay, Tamil, Thai, Vietnamese
 - **License:** [Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)
 ## Model Details
 The evaluation was done **five-shot** with native prompts on a sample of 100-1000 instances for each dataset.
+Following the implementation of IFEval in OpenLLM leaderboard, we also implement SEA-IFEval to provide a comparison of the ability of the model to follow specific constraints in English and in SEA languages.
 For more details on Llama3.1 8B CPT SEA-LIONv3 base benchmark performance, please refer to the SEA-HELM leaderboard, https://leaderboard.sea-lion.ai/
+**SEA-IFEval**
+SEA-IFEval evaluates a model's ability to adhere to constraints provided in the prompt, for example beginning a response with a specific word/phrase or answering with a certain number of sections. Additionally, accuracy is normalised by the proportion of responses in the correct language (if the model performs the task correctly but responds in the wrong language, it is judged to have failed the task).
 ## Technical Specifications
 ### Infrastructure
 Llama3.1 8B CPT SEA-LIONv3 was trained using [MosaicML Composer](https://github.com/mosaicml/composer) on the following hardware: