Instructions to use assix-research/lebanese-llama-3.1-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use assix-research/lebanese-llama-3.1-8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="assix-research/lebanese-llama-3.1-8b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("assix-research/lebanese-llama-3.1-8b")
model = AutoModelForCausalLM.from_pretrained("assix-research/lebanese-llama-3.1-8b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use assix-research/lebanese-llama-3.1-8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "assix-research/lebanese-llama-3.1-8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "assix-research/lebanese-llama-3.1-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/assix-research/lebanese-llama-3.1-8b

SGLang

How to use assix-research/lebanese-llama-3.1-8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "assix-research/lebanese-llama-3.1-8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "assix-research/lebanese-llama-3.1-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "assix-research/lebanese-llama-3.1-8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "assix-research/lebanese-llama-3.1-8b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use assix-research/lebanese-llama-3.1-8b with Docker Model Runner:
```
docker model run hf.co/assix-research/lebanese-llama-3.1-8b
```

lebanese-llama-3.1-8b / README.md

Anthony Assi

Update README.md

4e6df8b verified 8 months ago

3.65 kB

license: mit
base_model: unsloth/meta-llama-3.1-8b-instruct-bnb-4bit
language:
  - ar
  - en
library_name: transformers
tags:
  - lebanese
  - ammiya
  - arabizi
  - nlp
  - dgx-spark
  - blackwell
  - sm_121a
datasets:
  - assix/lebanese-dialect-fusion-v1
metrics:
  - loss

🇱🇧 Lebanese-Llama-3.1-8B

Lebanese-Llama-3.1-8B is a high-performance LLM fine-tuned specifically for the Lebanese dialect (Ammiya). It bridges the gap between Modern Standard Arabic (MSA) and the multi-modal nature of Lebanese communication, seamlessly blending Arabic script, French/English influences, and Arabizi (Romanized Arabic with numbers).

This model was trained and validated on the NVIDIA DGX Spark, the world’s first personal AI supercomputer powered by the Grace Blackwell (GB10) architecture.

🎮 Live Demo

Try the model instantly in your browser without any setup: 👉 Lebanese Llama Chat Demo

🚀 Model Features

Dialectal Authenticity: Fine-tuned to recognize and generate Lebanese syntax, specifically the "B" prefix for verbs and regional idioms (e.g., mni7, shou fi ma fi).
Arabizi Mastery: Expertly handles Romanized Lebanese using numbers (e.g., 3 for 'ayn, 7 for ha, 2 for hamza).
Blackwell Optimized: Merged into 16-bit (Bfloat16) to leverage the 5th Gen Tensor Cores and 128GB Unified Memory of the DGX Spark.
Cultural Nuance: Enhanced understanding of Lebanese culinary, geographic, and social context compared to base Llama-3.1.

🛠 Training Specifications

Infrastructure: NVIDIA DGX Spark (Grace Blackwell Superchip)
Architecture: Llama-3.1-8B-Instruct (4-bit QLoRA base)
Training Steps: 300 steps
Loss Convergence: Dropped from 5.3 to 2.2285
Optimizer: Unsloth (Stable Bfloat16 Path)

💻 Usage & Implementation

Because this model was developed on the sm_121a architecture, it is best loaded using the "Stable Path" to avoid Triton compiler conflicts.

Standard Inference (Hugging Face Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "esix117/lebanese-llama-3.1-8b"
tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Recommended System Prompt for a Native Persona
system_prompt = "You are a helpful Lebanese assistant speaking strictly in Lebanese Ammiya (dialect)."

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Marhaba! Kifak el yom? Khabbirni shway shou fi ma fi."}
]

inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=150, temperature=0.7)
# Slice the output to remove the prompt and only show the assistant's reply
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

⚠️ Known Limitations & Blackwell Optimization

Triton Compatibility: If running on Blackwell hardware (sm_121a), you may encounter a ptxas fatal : Value 'sm_121a' is not defined error when using custom kernels. To fix this, use the standard PyTorch RMS Norm fallback:

import unsloth.kernels.rms_layernorm
unsloth.kernels.rms_layernorm.fast_rms_layernorm = torch.nn.functional.rms_norm

🤝 Contribution & Acknowledgements

Developed by esix117 on the DGX Spark infrastructure. Special thanks to the researchers behind the open-source dialect datasets used in this fusion.

License: MIT