Instructions to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S
# Run inference directly in the terminal:
llama cli -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S
# Run inference directly in the terminal:
llama cli -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S
# Run inference directly in the terminal:
./llama-cli -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Use Docker

docker model run hf.co/Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

LM Studio
Jan

vLLM

How to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Ollama
How to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with Ollama:
```
ollama run hf.co/Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S
```
Unsloth Desktop

How to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with Docker Model Runner:
```
docker model run hf.co/Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S
```

Lemonade

How to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Run and chat with the model

lemonade run user.Qwen3.8-27B-Intel-Arc-Tuned-GGUF-IQ3_S

List all available models

lemonade list

Hermes Agent

How to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Greatjedi/Qwen3.8-27B-Intel-Arc-Tuned-GGUF:IQ3_S" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Greatjedi commited on 8 days ago

Commit

be0ccb8

verified ·

1 Parent(s): 6a9a3ba

docs: Add Swift-1.5 and bytkim Qwen3.8-27B-pi Intel Arc Tuned MTP models

Browse files

Files changed (1) hide show

README.md +24 -0

README.md CHANGED Viewed

@@ -20,6 +20,7 @@ tags:
 base_model:
 - empero-ai/Qwen3.8-27B-Ridge-GGUF
 - ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
 ---
 # Qwen3.8-27B Intel Arc Tuned Weights (Native Systolic DPAS GGUF)
@@ -166,6 +167,19 @@ The new **IQ3_S-MTP tuned base is ~1 GiB lighter** (14.31 GiB), buying ~1 GiB of
 - **Embedded MTP Included**: 90.5% draft acceptance.
 - ⚠️ *See Architecture Advisory below regarding Intel Arc Xe2 decode speed on sub-4 BPW weights.*
 ---
 ## ⚠️ CRITICAL ARCHITECTURAL ADVISORY: Sub-4 BPW Models (`3.7BPW` & `3.2BPW`) on Intel Arc
@@ -257,6 +271,16 @@ This repository provides hardware-adapted conversions specifically optimized for
 - **Georgi Gerganov & the llama.cpp community**: For `llama.cpp` and the open-source GGML SYCL backend.
 - **Intel oneAPI Team**: For the DPC++ / SYCL compiler and Level Zero runtime drivers.
 ---
 ## Methodology Notes (how these numbers were measured)

 base_model:
 - empero-ai/Qwen3.8-27B-Ridge-GGUF
 - ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
+- bytkim/Qwen3.8-27B-pi
 ---
 # Qwen3.8-27B Intel Arc Tuned Weights (Native Systolic DPAS GGUF)
 - **Embedded MTP Included**: 90.5% draft acceptance.
 - ⚠️ *See Architecture Advisory below regarding Intel Arc Xe2 decode speed on sub-4 BPW weights.*
+### 6. `Swift-1.5-Qwen3.8-27B-GSQ-RCO-Intel-Arc-Tuned-MTP-Q4_K.gguf` (14.30 GiB) — **Swift 1.5 Fast Agentic + Embedded MTP (Q4_K)**
+- Based on **Swift-1.5** fine-tuned for high-speed agentic coding, tooling, and refactoring workflows.
+- Transquantized from `IQ3_S-mtp` to pure linear systolic `Q4_K` super-blocks (**0 IQ tensors**, eliminates all SYCL host dequantization stalls).
+- **Embedded MTP**: Intact Layer 64 NextN prediction head (`blk.64.nextn.*`) delivering high speculative decode speed with **zero drafter VRAM**.
+- Verified up to **200,000 context** with `q4_0` KV cache on Intel Arc Pro B60.
+### 7. `Qwen3.8-27B-pi-Intel-Arc-Tuned-MTP-Q4_K.gguf` (15.11 GiB) — **bytkim Qwen3.8-27B-pi GRPO Agentic + Embedded MTP (Q4_K)** ⭐
+- Based on **`bytkim/Qwen3.8-27B-pi`** (GRPO-aligned deep reasoning and agentic problem solving).
+- Transquantized from `IQ4_XS` to pure linear systolic `Q4_K` super-blocks (**0 IQ tensors** across all 866 tensors), eliminating the ~8.1 tok/s SYCL Level Zero non-linear vector lattice bottleneck.
+- **Embedded MTP**: Intact Layer 64 NextN prediction head (`blk.64.nextn.*` in `Q4_0`) sustaining **~78–82% draft acceptance**.
+- Verified up to **170,000 context** with `q4_0` KV cache on Intel Arc Pro B60.
+- Superior performance on complex mathematical modeling, calculus, quantitative strategy analysis, and empirical protocol verification under `reasoningEffort: xhigh`.
 ---
 ## ⚠️ CRITICAL ARCHITECTURAL ADVISORY: Sub-4 BPW Models (`3.7BPW` & `3.2BPW`) on Intel Arc
 - **Georgi Gerganov & the llama.cpp community**: For `llama.cpp` and the open-source GGML SYCL backend.
 - **Intel oneAPI Team**: For the DPC++ / SYCL compiler and Level Zero runtime drivers.
+### 6. Swift-1.5 Fine-Tune
+- **Swift Team**: For the Swift-1.5 agentic coding fine-tune.
+- **Notice of Modification (Apache 2.0 §4b)**: Transquantized non-linear lattice tensors (`IQ3_S-mtp`) into linear systolic `Q4_K` super-blocks for Intel Arc DPAS execution.
+### 7. bytkim Qwen3.8-27B-pi
+- **bytkim**: Creator of `bytkim/Qwen3.8-27B-pi` (GRPO reasoning and agentic alignment).
+- **Original Weights**: [bytkim/Qwen3.8-27B-pi](https://huggingface.co/bytkim/Qwen3.8-27B-pi)
+- **GGUF Quantization Source**: [bartowski/bytkim_Qwen3.8-27B-pi-GGUF](https://huggingface.co/bartowski/bytkim_Qwen3.8-27B-pi-GGUF)
+- **Notice of Modification (Apache 2.0 §4b)**: Transquantized 866 tensors from non-linear `IQ4_XS` into linear systolic `Q4_K` super-blocks for Intel Arc Xe2 systolic DPAS execution, retaining the Layer 64 NextN draft head in `Q4_0`.
 ---
 ## Methodology Notes (how these numbers were measured)