Instructions to use openeurollm/kda-sigmoid-hybrid-1.3B-100B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openeurollm/kda-sigmoid-hybrid-1.3B-100B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openeurollm/kda-sigmoid-hybrid-1.3B-100B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("openeurollm/kda-sigmoid-hybrid-1.3B-100B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use openeurollm/kda-sigmoid-hybrid-1.3B-100B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openeurollm/kda-sigmoid-hybrid-1.3B-100B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openeurollm/kda-sigmoid-hybrid-1.3B-100B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openeurollm/kda-sigmoid-hybrid-1.3B-100B

SGLang

How to use openeurollm/kda-sigmoid-hybrid-1.3B-100B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openeurollm/kda-sigmoid-hybrid-1.3B-100B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openeurollm/kda-sigmoid-hybrid-1.3B-100B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openeurollm/kda-sigmoid-hybrid-1.3B-100B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openeurollm/kda-sigmoid-hybrid-1.3B-100B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openeurollm/kda-sigmoid-hybrid-1.3B-100B with Docker Model Runner:
```
docker model run hf.co/openeurollm/kda-sigmoid-hybrid-1.3B-100B
```

korbip commited on 11 days ago

Commit

51088a3

verified ·

1 Parent(s): 92db682

Condense the tokenizer section of the model card

Browse files

Files changed (1) hide show

README.md +6 -29

README.md CHANGED Viewed

@@ -38,35 +38,12 @@ The bundled tokenizer is configured the way the training corpus was encoded:
 repository's own defaults differ on both points, so encode through the
 tokenizer shipped here rather than re-fetching it by name.
-**To condition on the start of a document, prefix the EOS token, not a BOS.**
-Training packed documents as `[text..., EOS]`, so the token preceding any
-document's first token is always the EOS; a BOS was never seen at any position.
-Measured on held-out documents at 1.3B, against a bare prompt:
-| prefix | mean NLL | first 8 tokens |
-|---|---|---|
-| none | 2.0104 | 3.2947 |
-| **EOS** | **1.9960** | **2.9606** |
-| BOS | 2.0150 | 3.3768 |
-So the EOS is worth ~0.33 nats on the opening tokens and the BOS costs ~0.08.
-**Only at a document start.** The prefix is a "a new document begins here"
-signal, and mid-document it is a false one — on continuations the same EOS
-prefix *costs* 0.34 nats over the first 8 tokens (2.19 → 2.22 overall). Prefix
-it when you mean a fresh document; leave a continuation bare. Nothing in the
-weights or the tokenizer can tell these apart, which is why this is the
-caller's decision.
-`bos_token` is remapped to `</s>` here, so a pipeline that asks for "the BOS"
-(lm-eval's `--add_bos_token`, a generic wrapper) gets the separator rather than
-the unused `<s>`. `add_bos_token` remains `False`, so the default is still a
-bare prompt — which is the convention every number quoted for these models was
-measured under.
-If you are coming from `fla-hub` checkpoints, note that they use the opposite
-convention -- trained as `[BOS, text...]` with the BOS as the document
-separator -- so the habit does not carry over.
 ## Usage

 repository's own defaults differ on both points, so encode through the
 tokenizer shipped here rather than re-fetching it by name.
+**To condition on the start of a document, prefix the EOS token** -- that is
+what precedes every document's first token in training, and a BOS was never
+seen at any position. Leave a continuation bare: mid-document the prefix is a
+false signal and costs accuracy. `bos_token` is remapped to `</s>` here, so a
+caller that asks for "the BOS" gets the separator, while `add_bos_token` stays
+`False` and the default remains a bare prompt.
 ## Usage