Text Generation
Transformers
Safetensors
qwen4_exp
image-text-to-text
qwen
qwen4-exp
Mixture of Experts
bf16
mtp
speculative-decoding
cybersecurity
security-research
red-team
blue-team
purple-team
llm-agent
fuzzing
vulnerability-research
tool-use
conversational
Instructions to use Blackfrost-AI/CYBER-FROST-3.8-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Blackfrost-AI/CYBER-FROST-3.8-BF16 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Blackfrost-AI/CYBER-FROST-3.8-BF16") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Blackfrost-AI/CYBER-FROST-3.8-BF16") model = AutoModelForMultimodalLM.from_pretrained("Blackfrost-AI/CYBER-FROST-3.8-BF16", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Blackfrost-AI/CYBER-FROST-3.8-BF16 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Blackfrost-AI/CYBER-FROST-3.8-BF16" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Blackfrost-AI/CYBER-FROST-3.8-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Blackfrost-AI/CYBER-FROST-3.8-BF16
- SGLang
How to use Blackfrost-AI/CYBER-FROST-3.8-BF16 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Blackfrost-AI/CYBER-FROST-3.8-BF16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Blackfrost-AI/CYBER-FROST-3.8-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Blackfrost-AI/CYBER-FROST-3.8-BF16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Blackfrost-AI/CYBER-FROST-3.8-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Blackfrost-AI/CYBER-FROST-3.8-BF16 with Docker Model Runner:
docker model run hf.co/Blackfrost-AI/CYBER-FROST-3.8-BF16
Publish Cyber-Frost model card and deployment kit
Browse filesRename the release identity, document verified security lineage and evaluations, add the approved account banner and deployment kit, and remove legacy local path metadata from config.json. Model weights and tokenizer/template assets are unchanged.
- .gitattributes +1 -0
- ASSETS/BLACKFROST-AI-BANNER.png +3 -0
- DEPLOYMENT/ENV.EXAMPLE +14 -0
- DEPLOYMENT/LAUNCH.sh +39 -0
- DEPLOYMENT/README.md +69 -0
- DEPLOYMENT/SHA256SUMS +4 -0
- DEPLOYMENT/SMOKE_TEST.sh +45 -0
- README.md +136 -42
- config.json +1 -3
.gitattributes
CHANGED
|
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
ASSETS/BLACKFROST-AI-BANNER.png filter=lfs diff=lfs merge=lfs -text
|
ASSETS/BLACKFROST-AI-BANNER.png
ADDED
|
Git LFS Details
|
DEPLOYMENT/ENV.EXAMPLE
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Copy to DEPLOYMENT/.env and adjust for your host. Do not commit secrets.
|
| 2 |
+
MODEL_REPO=Blackfrost-AI/CYBER-FROST-3.8-BF16
|
| 3 |
+
SERVED_MODEL_NAME=CYBER-FROST-3.8-BF16
|
| 4 |
+
CUDA_VISIBLE_DEVICES=0,1,2,3
|
| 5 |
+
TENSOR_PARALLEL_SIZE=4
|
| 6 |
+
MAX_MODEL_LEN=32768
|
| 7 |
+
MAX_NUM_SEQS=4
|
| 8 |
+
GPU_MEMORY_UTILIZATION=0.45
|
| 9 |
+
HOST=127.0.0.1
|
| 10 |
+
PORT=8000
|
| 11 |
+
|
| 12 |
+
# Authenticate separately with `hf auth login`, or export HF_TOKEN in the
|
| 13 |
+
# process environment. Never store a token in this file.
|
| 14 |
+
|
DEPLOYMENT/LAUNCH.sh
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
set -euo pipefail
|
| 3 |
+
|
| 4 |
+
deployment_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 5 |
+
if [[ -f "${deployment_dir}/.env" ]]; then
|
| 6 |
+
set -a
|
| 7 |
+
# shellcheck disable=SC1091
|
| 8 |
+
source "${deployment_dir}/.env"
|
| 9 |
+
set +a
|
| 10 |
+
fi
|
| 11 |
+
|
| 12 |
+
: "${MODEL_REPO:=Blackfrost-AI/CYBER-FROST-3.8-BF16}"
|
| 13 |
+
: "${SERVED_MODEL_NAME:=CYBER-FROST-3.8-BF16}"
|
| 14 |
+
: "${CUDA_VISIBLE_DEVICES:=0,1,2,3}"
|
| 15 |
+
: "${TENSOR_PARALLEL_SIZE:=4}"
|
| 16 |
+
: "${MAX_MODEL_LEN:=32768}"
|
| 17 |
+
: "${MAX_NUM_SEQS:=4}"
|
| 18 |
+
: "${GPU_MEMORY_UTILIZATION:=0.45}"
|
| 19 |
+
: "${HOST:=127.0.0.1}"
|
| 20 |
+
: "${PORT:=8000}"
|
| 21 |
+
|
| 22 |
+
export CUDA_VISIBLE_DEVICES
|
| 23 |
+
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
| 24 |
+
|
| 25 |
+
exec vllm serve "${MODEL_REPO}" \
|
| 26 |
+
--served-model-name "${SERVED_MODEL_NAME}" \
|
| 27 |
+
--host "${HOST}" \
|
| 28 |
+
--port "${PORT}" \
|
| 29 |
+
--tensor-parallel-size "${TENSOR_PARALLEL_SIZE}" \
|
| 30 |
+
--max-model-len "${MAX_MODEL_LEN}" \
|
| 31 |
+
--max-num-seqs "${MAX_NUM_SEQS}" \
|
| 32 |
+
--gpu-memory-utilization "${GPU_MEMORY_UTILIZATION}" \
|
| 33 |
+
--enable-prefix-caching \
|
| 34 |
+
--trust-remote-code \
|
| 35 |
+
--reasoning-parser qwen3 \
|
| 36 |
+
--mamba-ssm-cache-dtype bfloat16 \
|
| 37 |
+
--speculative-config.method mtp \
|
| 38 |
+
--speculative-config.num-speculative-tokens 2
|
| 39 |
+
|
DEPLOYMENT/README.md
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# CYBER-FROST-3.8-BF16 deployment kit
|
| 2 |
+
|
| 3 |
+
This kit reproduces the validated text-only BF16 serving profile for `CYBER-FROST-3.8-BF16`. It is a deployment recipe, not a weight-modification recipe.
|
| 4 |
+
|
| 5 |
+
## Validated profile
|
| 6 |
+
|
| 7 |
+
| Item | Validated setting |
|
| 8 |
+
|---|---|
|
| 9 |
+
| Hardware | 4× NVIDIA B300 SXM6 |
|
| 10 |
+
| Parallelism | tensor parallel 4 |
|
| 11 |
+
| Runtime | vLLM `0.29.1rc1.dev13+g1cfd97281` |
|
| 12 |
+
| Python ABI observed in the runtime | Python 3.12 |
|
| 13 |
+
| Context exercised | 32,768 tokens |
|
| 14 |
+
| Concurrent sequence ceiling | 4 |
|
| 15 |
+
| GPU-memory utilization setting | 0.45 |
|
| 16 |
+
| Attention/cache mode | prefix caching requested; Mamba SSM cache forced to BF16 |
|
| 17 |
+
| Speculative decoding | native MTP, two draft tokens |
|
| 18 |
+
| API | OpenAI-compatible `/v1/models` and `/v1/chat/completions` |
|
| 19 |
+
| Validated modality | text only |
|
| 20 |
+
|
| 21 |
+
The archived trial did not preserve immutable NVIDIA driver, CUDA, PyTorch, or container-image identifiers. The vLLM build identifier and launch flags are preserved exactly, but deployers must validate their own driver/framework combination before relying on the performance results. B300 is a Blackwell GPU and requires a software stack with matching architecture support.
|
| 22 |
+
|
| 23 |
+
## Prerequisites
|
| 24 |
+
|
| 25 |
+
- Four visible GPUs with enough aggregate memory for the roughly 360 GB BF16 checkpoint, runtime allocations, and the requested context.
|
| 26 |
+
- A vLLM build matching `0.29.1rc1.dev13+g1cfd97281` or a newer build independently validated with `qwen4_exp`, hybrid Mamba/full attention, native MTP, and the flags in `LAUNCH.sh`.
|
| 27 |
+
- Hugging Face access approved for the manually gated repository and authentication through `hf auth login` or a process-level `HF_TOKEN`.
|
| 28 |
+
- `curl` for smoke testing; `jq` is recommended.
|
| 29 |
+
|
| 30 |
+
## Launch
|
| 31 |
+
|
| 32 |
+
```bash
|
| 33 |
+
cp DEPLOYMENT/ENV.EXAMPLE DEPLOYMENT/.env
|
| 34 |
+
# Edit non-secret host settings if needed, then authenticate separately:
|
| 35 |
+
hf auth login
|
| 36 |
+
DEPLOYMENT/LAUNCH.sh
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
The launcher binds to `127.0.0.1:8000` by default. Keep the endpoint private or place it behind authenticated transport. Do not expose an unauthenticated model or agent executor to the public internet.
|
| 40 |
+
|
| 41 |
+
Run the smoke test from another shell:
|
| 42 |
+
|
| 43 |
+
```bash
|
| 44 |
+
DEPLOYMENT/SMOKE_TEST.sh
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
From the `DEPLOYMENT/` directory, verify the kit itself with `sha256sum -c SHA256SUMS`.
|
| 48 |
+
|
| 49 |
+
## Behavioral controls
|
| 50 |
+
|
| 51 |
+
The packaged template defaults to thinking enabled and supports `xhigh`, `medium`, and `low` reasoning effort. The smoke test disables thinking only to keep the response short. Production callers should record the template, reasoning, sampling, and system-message settings used for each evaluation.
|
| 52 |
+
|
| 53 |
+
Tool calls are model output, not authorization. If an external agent executes them, apply independent identity checks, explicit target scope, least-privilege credentials, sandboxing, command allowlists or policy checks, audit logs, rate limits, and approval gates for consequential actions.
|
| 54 |
+
|
| 55 |
+
## Runtime notes
|
| 56 |
+
|
| 57 |
+
- The model config requests a float32 Mamba SSM cache. The validated speed profile overrides it to BF16. Remove `--mamba-ssm-cache-dtype bfloat16` when evaluating the configured precision instead of reproducing the speed trial.
|
| 58 |
+
- Native MTP combined with the model's Mamba groups prevented cross-request prefix-cache reuse in the tested runtime even though prefix caching was requested.
|
| 59 |
+
- Fused multi-step draft decode was unavailable for the experimental attention backend, so draft metadata was rebuilt between steps.
|
| 60 |
+
- The native MTP tensors predate the final trunk-weight behavioral stage and are a provisional acceleration baseline.
|
| 61 |
+
- The configured 262,144-token ceiling, multimodal path, tool-call correctness, high concurrency, and hardware other than the documented B300 profile are not qualified by this kit.
|
| 62 |
+
|
| 63 |
+
## Troubleshooting
|
| 64 |
+
|
| 65 |
+
- **Repository access denied:** confirm that the Hugging Face account has been approved for the gate and that the runtime sees the correct token.
|
| 66 |
+
- **Architecture is unknown:** use the pinned vLLM build or a build with explicit `qwen4_exp` support; keep `--trust-remote-code` enabled for this artifact.
|
| 67 |
+
- **Out of memory:** verify that exactly four intended GPUs are visible, reduce `MAX_MODEL_LEN`, lower `MAX_NUM_SEQS`, or lower `GPU_MEMORY_UTILIZATION` only after measuring the effect.
|
| 68 |
+
- **Unexpected refusal or output style:** verify that the repository chat template is in use and record any caller system message, reasoning mode, and sampling overrides.
|
| 69 |
+
- **Tool calls are plain text:** this profile validates text generation, not a particular tool parser or executor. Integrate and qualify those separately.
|
DEPLOYMENT/SHA256SUMS
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
7fe06000ec9892827e91bda3c2074e73ac76a125ecbb011a2439aa0910980bf9 ENV.EXAMPLE
|
| 2 |
+
4a55e2e1d4dcf4b90d9f856d4e695df89b2dd9c0ea65f69671abde2e7fef2b6e LAUNCH.sh
|
| 3 |
+
136ab95f8e39429da7a32998d4b37c3113a62e07421539949229b91a09af629d README.md
|
| 4 |
+
dc4353f1e8145324408bc56cf115ea466398b3ea11473b3408788712ba7d421a SMOKE_TEST.sh
|
DEPLOYMENT/SMOKE_TEST.sh
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env bash
|
| 2 |
+
set -euo pipefail
|
| 3 |
+
|
| 4 |
+
deployment_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
| 5 |
+
if [[ -f "${deployment_dir}/.env" ]]; then
|
| 6 |
+
set -a
|
| 7 |
+
# shellcheck disable=SC1091
|
| 8 |
+
source "${deployment_dir}/.env"
|
| 9 |
+
set +a
|
| 10 |
+
fi
|
| 11 |
+
|
| 12 |
+
: "${HOST:=127.0.0.1}"
|
| 13 |
+
: "${PORT:=8000}"
|
| 14 |
+
: "${SERVED_MODEL_NAME:=CYBER-FROST-3.8-BF16}"
|
| 15 |
+
: "${READINESS_TIMEOUT_SECONDS:=900}"
|
| 16 |
+
|
| 17 |
+
base_url="http://${HOST}:${PORT}/v1"
|
| 18 |
+
deadline=$((SECONDS + READINESS_TIMEOUT_SECONDS))
|
| 19 |
+
|
| 20 |
+
until curl --fail --silent --show-error --max-time 5 "${base_url}/models" >/dev/null; do
|
| 21 |
+
if (( SECONDS >= deadline )); then
|
| 22 |
+
echo "Timed out waiting for ${base_url}/models" >&2
|
| 23 |
+
exit 1
|
| 24 |
+
fi
|
| 25 |
+
sleep 5
|
| 26 |
+
done
|
| 27 |
+
|
| 28 |
+
models_json="$(curl --fail --silent --show-error "${base_url}/models")"
|
| 29 |
+
if command -v jq >/dev/null 2>&1; then
|
| 30 |
+
jq -e --arg model "${SERVED_MODEL_NAME}" '.data | any(.id == $model)' <<<"${models_json}" >/dev/null
|
| 31 |
+
fi
|
| 32 |
+
|
| 33 |
+
response="$(curl --fail --silent --show-error \
|
| 34 |
+
-H 'Content-Type: application/json' \
|
| 35 |
+
-X POST "${base_url}/chat/completions" \
|
| 36 |
+
-d "$(printf '{\"model\":\"%s\",\"messages\":[{\"role\":\"user\",\"content\":\"In an authorized lab, give three concise steps for triaging an unexpected listening service.\"}],\"temperature\":0,\"max_tokens\":96,\"chat_template_kwargs\":{\"enable_thinking\":false}}' "${SERVED_MODEL_NAME}")")"
|
| 37 |
+
|
| 38 |
+
if command -v jq >/dev/null 2>&1; then
|
| 39 |
+
jq -e '.choices[0].message.content | type == "string" and length > 0' <<<"${response}" >/dev/null
|
| 40 |
+
else
|
| 41 |
+
[[ "${response}" == *'"choices"'* ]]
|
| 42 |
+
fi
|
| 43 |
+
|
| 44 |
+
echo "Cyber-Frost readiness and minimal generation checks passed."
|
| 45 |
+
|
README.md
CHANGED
|
@@ -1,6 +1,7 @@
|
|
| 1 |
---
|
| 2 |
-
|
| 3 |
library_name: transformers
|
|
|
|
| 4 |
license: other
|
| 5 |
license_name: qwen-community-license-1.0
|
| 6 |
license_link: LICENSE
|
|
@@ -11,36 +12,122 @@ tags:
|
|
| 11 |
- bf16
|
| 12 |
- mtp
|
| 13 |
- speculative-decoding
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
---
|
| 15 |
|
| 16 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
##
|
| 21 |
|
| 22 |
-
-
|
| 23 |
-
- 48 text blocks with hybrid linear/full attention and a full-attention block every fourth layer.
|
| 24 |
-
- Hidden size 2,560; 24 attention heads; 2 KV heads.
|
| 25 |
-
- 512 routed experts, top-10 routing, plus a shared expert.
|
| 26 |
-
- One native hybrid MTP layer is included in the checkpoint.
|
| 27 |
-
- Configured maximum context: 262,144 tokens. The trial below used 32,768.
|
| 28 |
-
- The repository contains a vision configuration, but the current validation and benchmark are text-only.
|
| 29 |
|
| 30 |
-
|
| 31 |
|
| 32 |
-
##
|
| 33 |
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
-
|
| 39 |
-
- Fused multi-step draft decode is not yet available for the `QWEN4_EXP_EXP_QSA_STATE` attention backend, so metadata is rebuilt between speculative draft steps.
|
| 40 |
|
| 41 |
-
|
| 42 |
|
| 43 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
| Mode | Completion tokens | Median decode | Mean decode | Median TTFT | Draft acceptance | Relative median |
|
| 46 |
|---|---:|---:|---:|---:|---:|---:|
|
|
@@ -48,33 +135,40 @@ Environment: four NVIDIA B300 SXM6 GPUs, tensor parallelism 4, vLLM `0.29.1rc1.d
|
|
| 48 |
| Native MTP, k=2 | 1,820 | **265.00 tok/s** | 264.27 tok/s | **0.660 s** | 55.0% | **+39.4%** |
|
| 49 |
| Native MTP, k=3 | 1,799 | 263.10 tok/s | 288.43 tok/s | 0.684 s | 49.6% | +38.4% |
|
| 50 |
|
| 51 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
-
|
| 54 |
|
| 55 |
-
##
|
| 56 |
|
| 57 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
|
| 59 |
-
|
| 60 |
-
CUDA_VISIBLE_DEVICES=0,1,2,3 \
|
| 61 |
-
VLLM_WORKER_MULTIPROC_METHOD=spawn \
|
| 62 |
-
vllm serve Blackfrost-AI/BLACKFROST-3.8-ICED-BF16 \
|
| 63 |
-
--served-model-name BLACKFROST-3.8-ICED-BF16 \
|
| 64 |
-
--tensor-parallel-size 4 \
|
| 65 |
-
--max-model-len 32768 \
|
| 66 |
-
--max-num-seqs 4 \
|
| 67 |
-
--gpu-memory-utilization 0.45 \
|
| 68 |
-
--enable-prefix-caching \
|
| 69 |
-
--trust-remote-code \
|
| 70 |
-
--reasoning-parser qwen3 \
|
| 71 |
-
--mamba-ssm-cache-dtype bfloat16 \
|
| 72 |
-
--speculative-config.method mtp \
|
| 73 |
-
--speculative-config.num-speculative-tokens 2
|
| 74 |
-
```
|
| 75 |
|
| 76 |
-
|
| 77 |
|
| 78 |
-
|
| 79 |
|
| 80 |
-
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model: Qwen/Qwen3.8-Flash-Next
|
| 3 |
library_name: transformers
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
license: other
|
| 6 |
license_name: qwen-community-license-1.0
|
| 7 |
license_link: LICENSE
|
|
|
|
| 12 |
- bf16
|
| 13 |
- mtp
|
| 14 |
- speculative-decoding
|
| 15 |
+
- cybersecurity
|
| 16 |
+
- security-research
|
| 17 |
+
- red-team
|
| 18 |
+
- blue-team
|
| 19 |
+
- tool-use
|
| 20 |
---
|
| 21 |
|
| 22 |
+

|
| 23 |
+
|
| 24 |
+
# CYBER-FROST-3.8-BF16
|
| 25 |
+
|
| 26 |
+
**A first-party Blackfrost-AI BF16 model for security professionals conducting authorized research, assessment, engineering, and response work.**
|
| 27 |
+
|
| 28 |
+
## Release status and contents
|
| 29 |
+
|
| 30 |
+
Cyber-Frost is a research release under active quality assessment. The repository is public and access is manually gated.
|
| 31 |
+
|
| 32 |
+
The release contains the complete BF16 SafeTensors checkpoint, weight index, configuration, tokenizer and processor assets, the packaged chat template, upstream license, this model card, and a reproducible deployment kit. It is not an adapter and does not require a separate parent checkpoint at load time.
|
| 33 |
+
|
| 34 |
+
| Field | Released artifact |
|
| 35 |
+
|---|---|
|
| 36 |
+
| Clean model name | `CYBER-FROST-3.8-BF16` |
|
| 37 |
+
| Former name | `BLACKFROST-3.8-ICED-BF16` |
|
| 38 |
+
| Architecture | `Qwen4ExpForConditionalGeneration` |
|
| 39 |
+
| Precision | BF16 |
|
| 40 |
+
| Weight layout | 131 SafeTensors shards |
|
| 41 |
+
| Total parameters reported by Hub metadata | approximately 180B |
|
| 42 |
+
| Configured context | 262,144 tokens |
|
| 43 |
+
| Context exercised in the published performance trial | 32,768 tokens |
|
| 44 |
+
| Native speculative head | one MTP layer |
|
| 45 |
+
| Validated modality | text |
|
| 46 |
+
|
| 47 |
+
The configuration includes a vision tower, but this release has not received a multimodal quality evaluation. Do not infer validated image or video capability from the presence of processor files.
|
| 48 |
+
|
| 49 |
+
## Why Cyber-Frost exists
|
| 50 |
+
|
| 51 |
+
Security work is unusually vulnerable to false refusals. The same vocabulary appears in incident response, exploit validation, malware analysis, defensive engineering, and unauthorized activity; a general-purpose assistant can react to individual terms instead of the operator's legitimate scope.
|
| 52 |
+
|
| 53 |
+
Cyber-Frost is designed to reduce that unnecessary friction in professional, authorized workflows. It is intended to stay technically direct when an analyst is reviewing a finding, reproducing a vulnerability in a controlled environment, writing detection content, analyzing malicious code, or operating an approved security agent. This is a design objective, not a claim that the model has no refusal behavior or that every answer is safe or correct.
|
| 54 |
+
|
| 55 |
+
Authorization is an external control. The model cannot establish ownership, consent, rules of engagement, jurisdiction, or whether a target is in scope. Deployers must enforce identity, scope, tool permissions, logging, rate limits, and human review outside the model.
|
| 56 |
+
|
| 57 |
+
## Security corpus
|
| 58 |
+
|
| 59 |
+
Cyber-Frost was fine-tuned on a Blackfrost-AI security corpus combining curated security material, operator-authored workflows, realistic engagement-style scenarios, and Blackfrost-owned distillation data. Corpus sizes, source-by-source counts, raw engagement material, client identities, prompts, and responses are intentionally not published.
|
| 60 |
+
|
| 61 |
+
Domain coverage includes:
|
| 62 |
+
|
| 63 |
+
- Reconnaissance and OSINT
|
| 64 |
+
- Social engineering, business-email compromise, and deepfake-enabled abuse
|
| 65 |
+
- Web application and API security
|
| 66 |
+
- Identity, authentication, and Active Directory security
|
| 67 |
+
- Network, perimeter, VPN, and protocol security
|
| 68 |
+
- Vulnerability research, bug bounty, and binary exploitation
|
| 69 |
+
- Malware analysis, ransomware, and endpoint defense
|
| 70 |
+
- Cloud, container, and Kubernetes security
|
| 71 |
+
- Software supply-chain security
|
| 72 |
+
- Mobile, IoT, wireless, and physical security
|
| 73 |
+
- Industrial-control-system and operational-technology security
|
| 74 |
+
- Cryptography and security protocols
|
| 75 |
+
- Privilege escalation, lateral movement, and data exfiltration
|
| 76 |
+
- Threat intelligence, APT analysis, and purple-team operations
|
| 77 |
+
- AI-agent, LLM, and adversarial-ML security
|
| 78 |
+
|
| 79 |
+
Blackfrost-AI attests that owned portions of the corpus were developed from sanitized experience with authorized security work. It also applies a frontier-scale policy to its distillation teachers, excluding teachers below the 753B-parameter class. The release evidence independently binds one security subset to a Qwen3.8 2.4T teacher; it does not include a corpus-wide teacher manifest. These are therefore operator provenance statements, not independent benchmark findings.
|
| 80 |
|
| 81 |
+
Training-data provenance and licensing review for the mixed-source corpus remains in progress. That review is one reason access remains manually gated.
|
| 82 |
|
| 83 |
+
## Model specifications
|
| 84 |
|
| 85 |
+
The text stack has 48 blocks with hybrid linear and full attention, using a full-attention block every fourth layer. Its hidden size is 2,560 with 24 attention heads and 2 KV heads. The MoE stack contains 512 routed experts, selects 10 experts per token, and includes a shared expert. One native MTP layer is packaged for speculative decoding.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
|
| 87 |
+
The configured context ceiling is not a blanket quality guarantee. The published BF16 trial exercised 32,768 tokens. Longer contexts, high concurrency, multimodal requests, and tool-heavy agent loops require separate validation.
|
| 88 |
|
| 89 |
+
## Lineage
|
| 90 |
|
| 91 |
+
1. **Foundational checkpoint:** [`Qwen/Qwen3.8-Flash-Next`](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) at immutable revision `de4b8e4d43b917e7706784d8bb445c9af86a3540`.
|
| 92 |
+
2. **Blackfrost security adaptation:** security-domain fine-tuning followed by a full BF16 merge. The merged internal stage was identified as `BLACKFROST-3.8-FLASH-BF16`.
|
| 93 |
+
3. **Behavioral stage:** a Blackfrost-AI behaviorally modified derivative targeting lower false-refusal friction in authorized security workflows. The proprietary transformation process is not distributed.
|
| 94 |
+
4. **Release identity:** the same released BF16 weight payload was formerly labeled `BLACKFROST-3.8-ICED-BF16` and is now named `CYBER-FROST-3.8-BF16`. The rename does not represent another training run.
|
| 95 |
|
| 96 |
+
Tokenizer and processor lineage comes from the pinned Qwen foundation checkpoint. The packaged Blackfrost chat template is release-specific.
|
| 97 |
|
| 98 |
+
## Artifact verification
|
|
|
|
| 99 |
|
| 100 |
+
The released weight payload is tied to Hub revision `5321904427c4ef54df8a667edcbc2d1184e4286e`. On 2026-09-15, every weight shard was checked against its corresponding Hub LFS object identifier with no mismatch. The old ICED label and the Cyber-Frost repository resolve to this same payload.
|
| 101 |
|
| 102 |
+
| Artifact | SHA-256 |
|
| 103 |
+
|---|---|
|
| 104 |
+
| Sanitized `config.json` prepared for this card update | `c77cfd62621660b25e4c811bf44bf2ad4f63af98f23a874952a99e4e85f189f2` |
|
| 105 |
+
| `model.safetensors.index.json` | `99e815241ef03325536b0aaa4441deea45174c17fae31e10f0bb456410c590de` |
|
| 106 |
+
| Qwen Community License file | `a0dc422560841fd68e06d974907f8b4c709bca44a67daad2b528437bdf676c08` |
|
| 107 |
+
|
| 108 |
+
The sanitized configuration removes legacy local-path metadata only; it does not alter architecture or inference behavior.
|
| 109 |
+
|
| 110 |
+
## Measured behavior
|
| 111 |
+
|
| 112 |
+
### Internal refusal and over-refusal evaluation
|
| 113 |
+
|
| 114 |
+
The exact released BF16 checkpoint was evaluated on an internal development suite drawn from AdvBench, StrongREJECT, and XSTest. The run supplied no caller system message, used the packaged template, enabled thinking at medium effort, and sampled with temperature 1.0, top-p 0.95, top-k 20, and a 4,096-token completion ceiling. Full responses were adjudicated under a four-way rubric.
|
| 115 |
+
|
| 116 |
+
| Slice | Measured true-refusal rate |
|
| 117 |
+
|---|---:|
|
| 118 |
+
| AdvBench | 50.7% |
|
| 119 |
+
| StrongREJECT | 54.7% |
|
| 120 |
+
| Combined harmful subset | 52.7% |
|
| 121 |
+
| XSTest safe over-refusal | **0.0%** |
|
| 122 |
+
| XSTest unsafe | 24.0% |
|
| 123 |
+
|
| 124 |
+
There were no API errors. The evaluation used an earlier internal suite revision that was subsequently corrected, so these results are development evidence rather than a standardized leaderboard score. The zero safe over-refusal result supports the intended low-friction behavior on that slice; the uneven harmful-request refusal rates also show why this checkpoint must not be treated as a safety control.
|
| 125 |
+
|
| 126 |
+
No standardized cyber-capability benchmark has yet been qualified for this exact BF16 checkpoint. In particular, this card does not claim a CyberMetric, SecBench, MMLU, HumanEval, or IFEval score. Refusal behavior is not a substitute for measuring security competence.
|
| 127 |
+
|
| 128 |
+
### Four-B300 BF16 performance trial
|
| 129 |
+
|
| 130 |
+
The measured profile used four NVIDIA B300 SXM6 GPUs, tensor parallelism 4, vLLM `0.29.1rc1.dev13+g1cfd97281`, 32,768-token serving context, sequential requests, and thinking enabled. The frozen suite covered reasoning, code, prose, and tool-result synthesis with temperature 1.0, top-p 0.95, top-k 20, seed 38421, and at most 256 generated tokens.
|
| 131 |
|
| 132 |
| Mode | Completion tokens | Median decode | Mean decode | Median TTFT | Draft acceptance | Relative median |
|
| 133 |
|---|---:|---:|---:|---:|---:|---:|
|
|
|
|
| 135 |
| Native MTP, k=2 | 1,820 | **265.00 tok/s** | 264.27 tok/s | **0.660 s** | 55.0% | **+39.4%** |
|
| 136 |
| Native MTP, k=3 | 1,799 | 263.10 tok/s | 288.43 tok/s | 0.684 s | 49.6% | +38.4% |
|
| 137 |
|
| 138 |
+
These are single-pass development measurements, not production throughput guarantees. Workload, concurrency, context, runtime build, cache precision, and sampling can materially change the result.
|
| 139 |
+
|
| 140 |
+
The embedded MTP tensors predate the final trunk-weight behavioral stage. Treat the MTP head as a provisional acceleration baseline rather than a freshly adapted draft head. With the tested vLLM build, native MTP plus the model's Mamba groups also disables cross-request prefix-cache reuse, and fused multi-step draft decode is unavailable for the experimental attention backend.
|
| 141 |
+
|
| 142 |
+
## Prompt, tool use, and sampling
|
| 143 |
+
|
| 144 |
+
The repository includes a Blackfrost chat template that supplies a default operating prompt, supports caller-provided system context, exposes Qwen-style reasoning controls, and serializes tool calls. Its default is thinking enabled; supported reasoning-effort values are `xhigh`, `medium`, and `low`. Generation defaults are temperature 1.0, top-p 0.95, and top-k 20.
|
| 145 |
+
|
| 146 |
+
The template's authorization assumption is not an access-control mechanism. An agent runtime must independently restrict credentials, targets, files, networks, commands, and approval-requiring actions. Tool-call output is proposed text until an external executor acts on it.
|
| 147 |
+
|
| 148 |
+
Changing the template, system message, reasoning mode, or sampling can materially change refusal behavior and output quality. Record those settings when reporting results.
|
| 149 |
+
|
| 150 |
+
## Deployment
|
| 151 |
+
|
| 152 |
+
Use the included [`DEPLOYMENT/`](DEPLOYMENT/README.md) kit for the validated four-GPU BF16 serving profile, environment variables, launch command, and smoke tests. The clean API model identifier is `CYBER-FROST-3.8-BF16`.
|
| 153 |
+
|
| 154 |
+
## Intended use
|
| 155 |
+
|
| 156 |
+
Cyber-Frost is intended for qualified security professionals working within explicit authorization, including defensive research, secure code review, vulnerability validation, red-team and purple-team exercises, detection engineering, incident response, malware analysis, bug hunting, and controlled security-agent workflows.
|
| 157 |
|
| 158 |
+
It is not intended to authorize access, choose targets, define rules of engagement, make autonomous high-impact decisions, or replace legal, compliance, and safety review. Do not use it to access systems or data without permission, evade oversight, persist in third-party environments, deploy malware, steal credentials or data, disrupt services, or cause physical harm.
|
| 159 |
|
| 160 |
+
## Limitations and security responsibility
|
| 161 |
|
| 162 |
+
- Generated findings, code, commands, indicators, and remediation advice may be wrong, incomplete, outdated, or fabricated. Independently review them and execute only in isolated, authorized environments.
|
| 163 |
+
- Reduced over-refusal can increase the chance of receiving actionable output in ambiguous or malicious contexts. Gating and a system prompt do not remove that risk.
|
| 164 |
+
- The model is not a policy engine, authorization service, sandbox, malware scanner, or secrets boundary.
|
| 165 |
+
- The current evidence does not establish production readiness, comprehensive safety, comprehensive cyber competence, long-context quality beyond the tested profile, or multimodal quality.
|
| 166 |
+
- Model behavior can shift substantially with prompts, sampling, runtime versions, quantization, speculative settings, and agent scaffolding.
|
| 167 |
|
| 168 |
+
The deployer is responsible for authorization, least privilege, isolation, network policy, credential handling, human approval gates, monitoring, incident response, and compliance with applicable law.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 169 |
|
| 170 |
+
## License and disclaimer
|
| 171 |
|
| 172 |
+
Use and redistribution of this checkpoint are governed by the [Qwen Community License 1.0](LICENSE). Review the license before use. This research release is provided without a warranty of correctness, fitness, security, or non-infringement.
|
| 173 |
|
| 174 |
+
Report reproducible model or packaging issues through the repository's [Discussions](https://huggingface.co/Blackfrost-AI/CYBER-FROST-3.8-BF16/discussions) page without including secrets, client data, live targets, or sensitive exploit details.
|
config.json
CHANGED
|
@@ -150,7 +150,5 @@
|
|
| 150 |
"temporal_patch_size": 2
|
| 151 |
},
|
| 152 |
"vision_end_token_id": 248054,
|
| 153 |
-
"vision_start_token_id": 248053
|
| 154 |
-
"_name_or_path": "/models/BLACKFROST-3.8-FLASH-BF16",
|
| 155 |
-
"name_or_path": "/models/BLACKFROST-3.8-FLASH-BF16"
|
| 156 |
}
|
|
|
|
| 150 |
"temporal_patch_size": 2
|
| 151 |
},
|
| 152 |
"vision_end_token_id": 248054,
|
| 153 |
+
"vision_start_token_id": 248053
|
|
|
|
|
|
|
| 154 |
}
|