Instructions to use FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16
# Run inference directly in the terminal:
./llama-cli -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Use Docker

docker model run hf.co/FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

LM Studio
Jan
Ollama
How to use FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF with Ollama:
```
ollama run hf.co/FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16
```
Unsloth Desktop

How to use FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF with Docker Model Runner:
```
docker model run hf.co/FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16
```

Lemonade

How to use FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Run and chat with the model

lemonade run user.Qwen-AgentWorld-35B-A3B-NVFP4-GGUF-F16

List all available models

lemonade list

Hermes Agent

How to use FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "FreedomAISVR/Qwen-AgentWorld-35B-A3B-NVFP4-GGUF:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

FreedomAISVR commited on Jun 26

Commit

ccd2374

verified ·

1 Parent(s): d324434

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +15 -20

README.md CHANGED Viewed

@@ -19,9 +19,7 @@ base_model: Qwen/Qwen-AgentWorld-35B-A3B
 # Qwen AgentWorld 35B-A3B — NVFP4 GGUF
-NVFP4 quantization of [Qwen/Qwen-AgentWorld-35B-A3B](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B), a 35B parameter Mixture-of-Experts model with 3B active parameters, designed for agent tasks and world modeling .
-> **Note: This model has NO vision support.** The official Qwen/Qwen-AgentWorld-35B-A3B checkpoint ships with \language_model_only: True\ — the vision encoder weights are not included. Despite the \image-text-to-text\ tag on the source model and the vision config in config.json, this GGUF is text-only. Do not attempt to send images to this model.
 ## About the Model
@@ -29,14 +27,16 @@ Qwen AgentWorld is a specialized variant of the Qwen 3.5 MoE architecture optimi
 - **Agent tasks** — tool calling, function execution, environment simulation
 - **World modeling** — understanding and predicting environment states
-**35B total parameters** with only **3B active per token** (256 experts, 8 active)
 - **Efficient inference** — MoE architecture activates only a fraction of parameters
 ## Architecture
 - **Text model**: Qwen3.5 MoE — 40 layers, 2048 hidden, 256 experts (8 active/token)
-- **Vision encoder**: 27-layer SigLIP-style, 1152 hidden, patch_size 16
 - **Vocabulary**: 248,320 tokens
 ## Quantization
 This GGUF was quantized from the BF16 safetensors using [llama.cpp](https://github.com/ggerganov/llama.cpp) (build 537). The source weights were converted to F16 GGUF, then quantized to NVFP4 format.
@@ -47,39 +47,34 @@ NVFP4 (NVIDIA FP4) uses 4-bit floating point quantization optimized for NVIDIA B
 | File | Size | Description |
 |------|------|-------------|
-| `qwen-agentworld-35b-a3b-nvfp4.gguf` | ~18.4 GB | NVFP4 quantized model weights
-| `mmproj-qwen-agentworld-35b-a3b-f16.gguf` | ~0.84 GB | Vision projector (BF16) |
 ## Usage
 ### llama.cpp
-```bash
-# Server mode with OpenAI-compatible API
 llama-server \
   -m qwen-agentworld-35b-a3b-nvfp4.gguf \
   -ngl 99 \
   --host 0.0.0.0 \
   --port 8080
-# Direct inference
-llama-cli \
-  -m qwen-agentworld-35b-a3b-nvfp4.gguf \
-  -ngl 99 \
-  -p "Analyze this image and describe what you see"
-```
 ### LM Studio
-1. Download the GGUF file from this repository
-2. Load the GGUF file in LM Studio (vision is embedded, no mmproj needed)
-3. Set GPU offload layers to maximum
 ## Hardware Requirements
 - **Minimum**: 20 GB VRAM for partial offload
 - **Recommended**: 24+ GB VRAM for full GPU offload
-- **Disk**: ~18.4 GB
 ## License

 # Qwen AgentWorld 35B-A3B — NVFP4 GGUF
+NVFP4 quantization of [Qwen/Qwen-AgentWorld-35B-A3B](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B), a 35B parameter Mixture-of-Experts model with 3B active parameters, designed for agent tasks and world modeling with vision support.
 ## About the Model
 - **Agent tasks** — tool calling, function execution, environment simulation
 - **World modeling** — understanding and predicting environment states
+- **Vision understanding** — multimodal image input via separate mmproj vision projector
+- **35B total parameters** with only **3B active per token** (256 experts, 8 active)
 - **Efficient inference** — MoE architecture activates only a fraction of parameters
 ## Architecture
 - **Text model**: Qwen3.5 MoE — 40 layers, 2048 hidden, 256 experts (8 active/token)
+- **Vision encoder**: 27-layer SigLIP-style, 1152 hidden, patch_size 16 (via mmproj)
 - **Vocabulary**: 248,320 tokens
 ## Quantization
 This GGUF was quantized from the BF16 safetensors using [llama.cpp](https://github.com/ggerganov/llama.cpp) (build 537). The source weights were converted to F16 GGUF, then quantized to NVFP4 format.
 | File | Size | Description |
 |------|------|-------------|
+| qwen-agentworld-35b-a3b-nvfp4.gguf | ~18.4 GB | NVFP4 quantized model weights |
+| mmproj-qwen-agentworld-35b-a3b-f16.gguf | ~843 MB | Vision projector (BF16) |
 ## Usage
 ### llama.cpp
+`ash
 llama-server \
   -m qwen-agentworld-35b-a3b-nvfp4.gguf \
+  --mmproj mmproj-qwen-agentworld-35b-a3b-f16.gguf \
   -ngl 99 \
   --host 0.0.0.0 \
   --port 8080
+`
 ### LM Studio
+1. Download both files from this repository
+2. Load the main GGUF file in LM Studio
+3. Load the mmproj file for vision support
+4. Set GPU offload layers to maximum
 ## Hardware Requirements
 - **Minimum**: 20 GB VRAM for partial offload
 - **Recommended**: 24+ GB VRAM for full GPU offload
+- **Disk**: ~19.2 GB
 ## License