Instructions to use poolside-laguna-hackathon/laguna-xs2-dense-stage2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use poolside-laguna-hackathon/laguna-xs2-dense-stage2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="poolside-laguna-hackathon/laguna-xs2-dense-stage2", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("poolside-laguna-hackathon/laguna-xs2-dense-stage2", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("poolside-laguna-hackathon/laguna-xs2-dense-stage2", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use poolside-laguna-hackathon/laguna-xs2-dense-stage2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "poolside-laguna-hackathon/laguna-xs2-dense-stage2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside-laguna-hackathon/laguna-xs2-dense-stage2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/poolside-laguna-hackathon/laguna-xs2-dense-stage2

SGLang

How to use poolside-laguna-hackathon/laguna-xs2-dense-stage2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "poolside-laguna-hackathon/laguna-xs2-dense-stage2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside-laguna-hackathon/laguna-xs2-dense-stage2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "poolside-laguna-hackathon/laguna-xs2-dense-stage2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside-laguna-hackathon/laguna-xs2-dense-stage2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use poolside-laguna-hackathon/laguna-xs2-dense-stage2 with Docker Model Runner:
```
docker model run hf.co/poolside-laguna-hackathon/laguna-xs2-dense-stage2
```

duico commited on May 30

Commit

3e5ecef

verified ·

1 Parent(s): 78009fd

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -20,7 +20,7 @@ A **~3B dense** model distilled from **[poolside/Laguna-XS.2](https://huggingfac
 Two stages, both distilling from the frozen FP8 XS.2 teacher:
-1. **Stage 1 — per-layer MoE→dense init** (RADLADS-style). Each of the 39 sparse MoE blocks is replaced by a dense SwiGLU FFN (intermediate 4608) and trained *independently, in parallel* to match the teacher MoE block's output (NMSE on the residual contribution), fed the teacher's own hidden states (no error compounding). ~90M tokens. Result: a dense init with held-out perplexity **~25** (vs teacher **~4.4**) — functional but rough, because cross-layer error compounding is left uncorrected by design.
 2. **Stage 2 — synchronous logit-KD** (this model). The stitched ~3B dense student is trained **end-to-end** against the fp8 teacher's full-vocab logits (forward-KL), on a 50/50 code+general corpus. ~14M tokens, one H100, **KL 2.5 → 1.40**.
 Data: 50% [DCLM](https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0) + 50% [StarCoder2/the-stack-v2-train](https://huggingface.co/datasets/bigcode/the-stack-v2-train-smol-ids).
@@ -58,7 +58,7 @@ print(tok.decode(m.generate(**ids, max_new_tokens=64)[0], skip_special_tokens=Tr
 | Model | Params (resident) | Held-out PPL | HumanEval pass@1 |
 |---|---|---|---|
 | Teacher (Laguna XS.2, fp8) | 33B (3B active) | ~4.4 | _running_ |
-| Stage-1 dense | ~3B | ~25 | 0.0% (0/164) |
 | **Stage-2 dense (this)** | ~3B | _tbd_ | **0.6%** (1/164) |
 _(HumanEval pass@1 via [evalplus](https://github.com/evalplus/evalplus), greedy. Teacher number being measured; will be filled in.)_
@@ -90,4 +90,4 @@ The MoE keeps all 33B params resident even though only ~3B are active per token;
 - **Severely under-trained.** ~14M KD tokens is 20–400× below typical recovery budgets (RADLADS used 250–700M; MoE→dense work ~4B). Expect near-zero on real coding tasks.
 - **Next:** extend Stage 2 substantially — ideally with **cached teacher top-K logits** to remove the teacher forward from the loop (3–5× throughput), reaching 50–100M+ tokens. Then run the paper-faithful agentic evals (SWE-bench / Terminal-Bench via Harbor).
-Code: https://github.com/postscarcity-inc/laguna-xs.2-dense · Stage-1 model: `poolside-laguna-hackathon/laguna-xs2-dense-stage1`

 Two stages, both distilling from the frozen FP8 XS.2 teacher:
+1. **[Stage 1](https://huggingface.co/poolside-laguna-hackathon/laguna-xs2-dense-stage1) — per-layer MoE→dense init** (RADLADS-style). Each of the 39 sparse MoE blocks is replaced by a dense SwiGLU FFN (intermediate 4608) and trained *independently, in parallel* to match the teacher MoE block's output (NMSE on the residual contribution), fed the teacher's own hidden states (no error compounding). ~90M tokens. Result: a dense init with held-out perplexity **~25** (vs teacher **~4.4**) — functional but rough, because cross-layer error compounding is left uncorrected by design.
 2. **Stage 2 — synchronous logit-KD** (this model). The stitched ~3B dense student is trained **end-to-end** against the fp8 teacher's full-vocab logits (forward-KL), on a 50/50 code+general corpus. ~14M tokens, one H100, **KL 2.5 → 1.40**.
 Data: 50% [DCLM](https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0) + 50% [StarCoder2/the-stack-v2-train](https://huggingface.co/datasets/bigcode/the-stack-v2-train-smol-ids).
 | Model | Params (resident) | Held-out PPL | HumanEval pass@1 |
 |---|---|---|---|
 | Teacher (Laguna XS.2, fp8) | 33B (3B active) | ~4.4 | _running_ |
+| [Stage-1 dense](https://huggingface.co/poolside-laguna-hackathon/laguna-xs2-dense-stage1) | ~3B | ~25 | 0.0% (0/164) |
 | **Stage-2 dense (this)** | ~3B | _tbd_ | **0.6%** (1/164) |
 _(HumanEval pass@1 via [evalplus](https://github.com/evalplus/evalplus), greedy. Teacher number being measured; will be filled in.)_
 - **Severely under-trained.** ~14M KD tokens is 20–400× below typical recovery budgets (RADLADS used 250–700M; MoE→dense work ~4B). Expect near-zero on real coding tasks.
 - **Next:** extend Stage 2 substantially — ideally with **cached teacher top-K logits** to remove the teacher forward from the loop (3–5× throughput), reaching 50–100M+ tokens. Then run the paper-faithful agentic evals (SWE-bench / Terminal-Bench via Harbor).
+Code: https://github.com/postscarcity-inc/laguna-xs.2-dense · [Stage-1 model](https://huggingface.co/poolside-laguna-hackathon/laguna-xs2-dense-stage1)