Instructions to use osiria/diablo-italian-base-354m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use osiria/diablo-italian-base-354m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="osiria/diablo-italian-base-354m")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("osiria/diablo-italian-base-354m", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use osiria/diablo-italian-base-354m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "osiria/diablo-italian-base-354m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "osiria/diablo-italian-base-354m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/osiria/diablo-italian-base-354m

SGLang

How to use osiria/diablo-italian-base-354m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "osiria/diablo-italian-base-354m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "osiria/diablo-italian-base-354m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "osiria/diablo-italian-base-354m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "osiria/diablo-italian-base-354m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use osiria/diablo-italian-base-354m with Docker Model Runner:
```
docker model run hf.co/osiria/diablo-italian-base-354m
```

Model: DIABLO 354M 🔥
Lang: IT

Model description

This model is a causal language model for the Italian language, based on a GPT-like [1] architecture (more specifically, the model has been obtained by modifying Meta's XGLM architecture [2] and exploiting its 564M checkpoint).

The model has ~354M parameters and a vocabulary of 50.335 tokens. It is a foundation model, pre-trained for causal language modeling, so it is mainly suitable for basic natural language generation, and you will have to fine-tune it in order to use it on more specific downstream tasks.

Quick usage

In order to use the model for inference on GPU, the following pipeline is needed:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("osiria/diablo-italian-base-354m")
model = AutoModelForCausalLM.from_pretrained("osiria/diablo-italian-base-354m", torch_dtype=torch.float16)

device = torch.device("cuda")
model = model.to(device)

pipeline_nlg = pipeline("text-generation", model = model, tokenizer = tokenizer, device = 0)
pipeline_nlg("Ciao, mi chiamo Marco Rossi e")

# [{'generated_text': 'Ciao, mi chiamo Marco Rossi e sono un ragazzo di 23 anni.'}]

Limitations

The model might behave erratically when presented with prompts which are too far away from its pre-training and, because of the probabilistic nature of its generation, it might occasionally produce biased or offensive content with respect to gender, race, ideologies, and political or religious beliefs. These limitations imply that the model and its outputs should be used with caution, and should not be involved in situations that require the generated text to be fair or true.