Instructions to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Use Docker

docker model run hf.co/IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

LM Studio
Jan

vLLM

How to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Ollama
How to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with Ollama:
```
ollama run hf.co/IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0
```
Unsloth Desktop

How to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with Docker Model Runner:
```
docker model run hf.co/IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0
```

Lemonade

How to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Run and chat with the model

lemonade run user.Occamy-1.0-APEX-I-NanoPlus-GGUF-Q8_0

List all available models

lemonade list

Hermes Agent

How to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Quick Navigation Index

Optimization History & Transparency Notice
Model Specifications
Inference Quickstart
1. llama-cli (Console Generation)
2. llama-server (OpenAI-Compatible API)
Optional Support

Occamy-1.0 APEX-I-NanoPlus GGUF

The Next-Generation Frontier MoE · Extreme 12–13GB Footprint · Fast System RAM Streaming & Massive Context on 16GB VRAM

🚀 EXPLORE THE ESTABLISHED 35B MoE MINIPLUS & NANOPLUS LINEUP

These are complementary APEX-I releases, not alternate downloads of the same model. Each receives the same surgical tensor-by-tensor approach and a design suitable for full or partial system-RAM inference:

Occamy-1.0 APEX-I-MiniPlus-V2.1 — versatile frontier MoE for broad reasoning, multimodal tasks, and deep research (14.75 GB / Q5_K_M tier).

Occamy-1.0 APEX-I-MiniPlus-V2.1 Abliterated — the V2.1 refusal-ablated edition with authentic Heretic TPE directional abliteration.

Qwen3.6-35B-A3B-MTP APEX-I-NanoPlus — the ~13.0 GB NanoPlus pioneer achieving Q4 quality at 2.93 BPW.

Ornith 1.5 APEX-I-NanoPlus — software engineering MoE streamlined for ultra-lean memory footprints.

THE DEFINITIVE SPECIFICATION IN THE ~12–13 GB CEILING

This APEX-I-NanoPlus release marks the official debut of our specialized tensor-by-tensor architectural configuration for sparse Mixture-of-Experts quantization within an extreme ~12–13 GB envelope. Every tensor across its 40 layers and 256 micro-experts has been mathematically allocated to maximize reasoning precision, preserve routing behavior, and prevent avoidable CPU dequantization stalls during hybrid and system-RAM inference.

🏆 BUILD & VERIFIED REFERENCE COMPARISON

Quantization Specification File Size (Disk) Memory Footprint (RAM/VRAM) Average BPW WikiText-2 Perplexity ΔPPL vs. approx. BF16 Quality Tier Equivalent

Unquantized BF16 Base ~70.0 GB ~65.2 GiB 16.00 BPW ~6.18 (Reference) 0.000 Full precision baseline

APEX-I-MiniPlus V2.1 14.75 GB 13.74 GiB 3.40 BPW 6.2432 ± 0.1622 +0.0632 (+1.02%) Q5_K_M tier

APEX-I-NanoPlus (CURRENT) 12.55 GB 11.69 GiB ~2.93 BPW 6.1695 ± 0.15632 ~0.00 (within error bounds) Solid Q4_K_M / Q4_K_L Tier

Looking for higher precision? APEX-I-MiniPlus V2.1 offers the full 14.75 GB (3.40 BPW) release of this Occamy family, delivering full Q5_K_M tier fidelity with 120 shared experts in physical Q5_K.

Routing: all recipe-designated gate_inp and gate_shexp tensors remain in uncompressed F32, preserving zero routing drift.

Evaluation status: WikiText-2 perplexity successfully measured on final GGUF (6.1695 ± 0.15632).

ARC-Challenge (0-shot, 1,172 questions): 95.73%.

Q4_K_L Tier in Reasoning & Routing: 100% uncompressed F32 routers (gate_inp) and a Q6_K output head eliminate router drift, matching or exceeding standard Q4_K_L baselines on logic benchmarks.

Solid Q4_K_M Tier in Language Modeling: WikiText-2 perplexity preserves 4-bit distributional fidelity across standard generation in an ultra-lean footprint.

Quantization Specification	File Size (Disk)	Memory Footprint (RAM/VRAM)	Average BPW	WikiText-2 Perplexity	ΔPPL vs. approx. BF16	Quality Tier Equivalent
Unquantized BF16 Base	~70.0 GB	~65.2 GiB	16.00 BPW	~6.18 (Reference)	0.000	Full precision baseline
APEX-I-MiniPlus V2.1	14.75 GB	13.74 GiB	3.40 BPW	6.2432 ± 0.1622	+0.0632 (+1.02%)	Q5_K_M tier
APEX-I-NanoPlus (CURRENT)	12.55 GB	11.69 GiB	~2.93 BPW	6.1695 ± 0.15632	~0.00 (within error bounds)	Solid Q4_K_M / Q4_K_L Tier

DO NOT CONFUSE APEX-I-NANOPLUS WITH GENERIC COMMUNITY SUB-3-BIT QUANTS!

Regardless of release version, NEVER confuse handcrafted APEX-I-NanoPlus builds with generic community sub-3-bit releases:

Generic Community IQ2_S / IQ2_XXS: Uniformly crushes all core MoE experts down to aggressive 2-bit codebooks without importance calibration, leaves the sensitive token output head unarmored at 3-bit, and compresses attention projections. In deep reasoning models, this triggers severe perplexity spikes, syntax errors, and broken code brackets.

Handcrafted APEX-I-NanoPlus: Applies a surgical tensor-by-tensor architecture that preserves 100% of expert routing matrices in uncompressed F32 (zero router drift), armors the token output head in high-precision Q6_K, safeguards attention gates in Q8_0, fortifies the critical MoE down-projection residual stream (ffn_down_exps) in IQ3_XXS (3.06 bpw), and restricts 2-bit compression strictly to redundant gating/up projections guided by the official imatrix.

SYSTEM RAM INFERENCE: FULL OR PARTIAL

This APEX-I-NanoPlus release is designed for full or partial system-RAM inference. Depending on the processor, memory bandwidth, and DDR4/DDR5 configuration, generation can range from 20 to 45 tok/s. With partial GPU offload, systems that cannot fit 128K or more context entirely in VRAM can place the remaining model and context load in system RAM, maintaining stable, responsive generation at longer context lengths.

Optimization History & Transparency Notice

We maintain our previous releases publicly as a transparent engineering record of continuous optimization. Below is the exact evolutionary roadmap of our architectures:

Specification	Core Experts (2–37)	Edge Experts (0–1, 38–39)	Shared Expert (`shexp`)	Full Attention (L3, 7, 11, ...)	Attention Gates	Output Head (`output.weight`)	Routers (`gate_inp`)	Size / Overhead	Real-World Impact
Generic APEX Mini	`IQ2_S` (2.50 bpw)	`Q3_K` (only 5 layers)	`Q4_K` / `Q3_K`	`Q3_K`	Compressed	`Q3_K_M`	Compressed	Baseline (~12.5 GB)	Severe syntax errors, broken code indentation, high perplexity in `<think>`.
MiniPlus V2.1 (Current)	`IQ3_XXS` + `Q3_K`	`Q3_K` (10 layers)	`Q5_K`	`Q4_K` (`q/k/v`) + `Q6_K` (`output`)	`Q8_0`	`Q6_K`	`F32`	14.75 GB (13.74 GiB)	Measured Q5_K_M tier. Fits 24GB GPUs effortlessly.
NanoPlus (NEW)	`IQ3_XXS` (down) + `IQ2_S` (gate) + `IQ2_XXS` (up)	`Q3_K` (down) + `IQ3_XXS` (gate/up)	`Q4_K`	`Q4_K` (`q/k/v`) + `Q6_K` (`attn_output`)	`Q8_0`	`Q6_K`	`F32`	12.55 GB (11.69 GiB)	Calibrated 12–13 GB tier. Leaves >3 GB free VRAM on 16GB cards for 32k context with zero AVX2 CPU stalls.

Deployment & System Architecture Guide

Full GPU VRAM Offload (16GB+ VRAM, -ngl 99): Effortless full offload with native 32K–64K context support on 16GB cards (RTX 4080 / RTX 4070 Ti Super), and native 256K context on 24GB workstations (RTX 3090 / 4090 / 5090).

System RAM Streaming Specialist (DDR4/DDR5 & Massive Context): Specially engineered to run either partially or entirely out of system RAM across large or full context windows. By utilizing linear SIMD-optimized Q4_K attention projections and preserving critical down-projections in IQ3_XXS, AVX2 CPU dequantization stalls are eliminated.

Explore our official collection: APEX-I-NanoPlus Collection.

Model Specifications

Attribute	Specification
Architecture	Hybrid MoE + Linear Attention / DeltaNet SSM
Base Model	Accio-Lab/occamy-1.0
Total Parameters	~35 Billion
Active Parameters	~3.0 Billion per token
Context Window	Up to 256,000 tokens
Primary File	`Occamy-1.0.APEX-I-NanoPlus.gguf` (12.55 GB)
Vision Projector	`mmproj-Q8_0.gguf` (Q8_0 8-bit quantization)

Inference Quickstart

1. `llama-cli` (Console Generation)

llama-cli \
  -m Occamy-1.0.APEX-I-NanoPlus.gguf \
  --mmproj mmproj-Q8_0.gguf \
  -p "<|im_start|>user\nHello! Explain your architecture.<|im_end|>\n<|im_start|>assistant\n" \
  -ngl 99 -c 4096 --temp 0.6 --top-p 0.95

2. `llama-server` (OpenAI-Compatible API)

llama-server \
  -m Occamy-1.0.APEX-I-NanoPlus.gguf \
  --mmproj mmproj-Q8_0.gguf \
  --port 8080 \
  -ngl 99 -c 16384

Optional Support

If these MiniPlus or NanoPlus releases have been useful to you and you would like to support the work, you can do so voluntarily through https://ko-fi.com/isvalorum. Your contribution helps with evaluation, hosting, and future handcrafted quantizations. Every release will always remain free to download and use; there are no paywalled files, updates, or features.

Downloads last month: 791

GGUF

Model size

35B params

Architecture

qwen35moe

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

Accio-Lab/occamy-1.0

Quantized

(24)

this model

Collection including IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF

APEX-I-NanoPlus

Collection

Architectural quantization line targeting ~13 GB with solid Q4 quality and fast RAM streaming on 16GB VRAM. • 4 items • Updated 3 days ago

Quick Navigation Index

Occamy-1.0 APEX-I-NanoPlus GGUF

The Next-Generation Frontier MoE · Extreme 12–13GB Footprint · Fast System RAM Streaming & Massive Context on 16GB VRAM

🚀 EXPLORE THE ESTABLISHED 35B MoE MINIPLUS & NANOPLUS LINEUP

THE DEFINITIVE SPECIFICATION IN THE ~12–13 GB CEILING

🏆 BUILD & VERIFIED REFERENCE COMPARISON

DO NOT CONFUSE APEX-I-NANOPLUS WITH GENERIC COMMUNITY SUB-3-BIT QUANTS!

SYSTEM RAM INFERENCE: FULL OR PARTIAL

Optimization History & Transparency Notice

Deployment & System Architecture Guide

Model Specifications

Inference Quickstart

1. llama-cli (Console Generation)

2. llama-server (OpenAI-Compatible API)

Optional Support

Model tree for IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF

Collection including IsValorum/Occamy-1.0-APEX-I-NanoPlus-GGUF

1. `llama-cli` (Console Generation)

2. `llama-server` (OpenAI-Compatible API)