Instructions to use Anomly/SmolLM2-135M-Instruct-bposit8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Anomly/SmolLM2-135M-Instruct-bposit8 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Anomly/SmolLM2-135M-Instruct-bposit8 # Run inference directly in the terminal: llama cli -hf Anomly/SmolLM2-135M-Instruct-bposit8
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Anomly/SmolLM2-135M-Instruct-bposit8 # Run inference directly in the terminal: llama cli -hf Anomly/SmolLM2-135M-Instruct-bposit8
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Anomly/SmolLM2-135M-Instruct-bposit8 # Run inference directly in the terminal: ./llama-cli -hf Anomly/SmolLM2-135M-Instruct-bposit8
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Anomly/SmolLM2-135M-Instruct-bposit8 # Run inference directly in the terminal: ./build/bin/llama-cli -hf Anomly/SmolLM2-135M-Instruct-bposit8
Use Docker
docker model run hf.co/Anomly/SmolLM2-135M-Instruct-bposit8
- LM Studio
- Jan
- Ollama
How to use Anomly/SmolLM2-135M-Instruct-bposit8 with Ollama:
ollama run hf.co/Anomly/SmolLM2-135M-Instruct-bposit8
- Unsloth Desktop
- Docker Model Runner
How to use Anomly/SmolLM2-135M-Instruct-bposit8 with Docker Model Runner:
docker model run hf.co/Anomly/SmolLM2-135M-Instruct-bposit8
- Lemonade
How to use Anomly/SmolLM2-135M-Instruct-bposit8 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Anomly/SmolLM2-135M-Instruct-bposit8
Run and chat with the model
lemonade run user.SmolLM2-135M-Instruct-bposit8-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
SmolLM2-135M-Instruct-bposit8
HuggingFaceTB/SmolLM2-135M-Instruct converted to b-posit8 (32-code blocks with a power-of-two scale, 8-bit posit
codes, es = 2) for the exact profile of INVAR:
every matmul accumulates in a 256-bit quire with one rounding, so a deterministic runtime
produces bit-identical activations and logits on x86, CUDA and aarch64, and independent
reference implementations reproduce a served answer from these weights and the token ids.
- File:
SmolLM2-135M-Instruct-bposit8.gguf(0.14 GB), SHA-256f09fe23eb1ec46dfedfd4c6603c2928b3e349190c13b39b7993be8251b52d6d0 - Format: GGUF
general.file_type42 (b-posit8), llama-cpp-et fork (deterministic backend) - Spec:
docs/EXACT-PROFILE-SPEC.mdin the INVAR repo; conformance fixture and tokenizer vectors ingo/crverify/testdata - Licence: this is a re-quantised copy of
HuggingFaceTB/SmolLM2-135M-Instruct; the upstream licence (apache-2.0) applies unchanged, including any use restrictions and attribution requirements.
Run it: invar serve --model SmolLM2-135M-Instruct-bposit8.gguf --binary llama-cli --spot-check --spot-check-units, then
invar verify worldline.jsonl --model SmolLM2-135M-Instruct-bposit8.gguf --binary llama-cli --spot-check --units --reexec.
- Downloads last month
- -
We're not able to determine the quantization variants.
Model tree for Anomly/SmolLM2-135M-Instruct-bposit8
Base model
HuggingFaceTB/SmolLM2-135M