Mark validated artifact upload in progress

README.md

@@ -1,261 +1,9 @@
 ---
 license: apache-2.0
-base_model: Agnes-AI/Agnes-3.0-Flash
-pipeline_tag: image-text-to-text
-language:
-- en
-- zh
-tags:
-- gguf
-- agnes
-- multimodal
 ---
 
-# Agnes-3.0-Flash GGUF
+# Validated release upload in progress
 
-**Follow [@procrastiness on Twitter/X](https://twitter.com/procrastiness) for more model releases and updates.**
-
-GGUF quantizations of the original [Agnes-AI/Agnes-3.0-Flash](https://huggingface.co/Agnes-AI/Agnes-3.0-Flash), released by 0xKitkat. **This is the normal model, with no abliteration or fine-tuning.** Approximately 33.1B parameters; all 72 decoder blocks and the original vision tower are retained.
-
-The original Apache-2.0 license is preserved. Downloads and local use require no hosted API subscription; you supply the hardware and electricity. The source revision is `8f0c484c363cdda8384195be4a5f7730f3915bde`.
-
-Available validated quants: Q4_K_M, Q5_K_M. Additional quants are still being built and validated.
-
-<!-- GGUF-USAGE-BEGIN -->
-## Recommended starting setup
-
-Start with **Q4_K_M**, a **4,096-token context**, **one concurrent request**, and **thinking disabled**. This is the configuration closest to the functional checks reported below. Use the embedded chat template; do not select a generic ChatML or Llama template manually.
-
-For normal chat, use the upstream sampling defaults: **temperature 1.0, top-p 0.95, top-k 20**. For repeatable troubleshooting or checking exact answers, use **temperature 0**. Start with **512 output tokens** for short answers or **2,048** for longer responses, keeping prompt + image tokens + output within the configured context.
-
-### Choose a quant and hardware
-
-Download **one text GGUF**. Add the **F16 vision projector** only if you want image input. The projector is shared by all quants in this release.
-
-| Quant | When to choose it |
-|---|---|
-| **Q4_K_M** | Recommended first download; smallest planned file and the easiest fit on consumer hardware. |
-| **Q5_K_M** | More weight precision if you have additional memory; compare the measured results below. |
-| **Q6_K** | Higher precision with a larger memory footprint. |
-| **Q8_0** | Highest precision in this quant set; intended for larger-memory machines. |
-
-The validation table lists exact sizes for published files. **A quant being listed here does not mean its upload has finished**; check the available-files list and the Files tab.
-
-| Hardware | Suggested starting point |
-|---|---|
-| Single 8–16 GB GPU + 32 GB or more system RAM | Q4_K_M with CPU offloading. Use automatic fitting initially; lower GPU layers if needed. Generation will be slower than a full GPU fit. |
-| Single 24 GB GPU + 32 GB or more system RAM | Try Q4_K_M at 4K context. Leave room for buffers, display usage, and the projector; reduce GPU layers if allocation fails. |
-| Two 12 GB GPUs + about 32 GB system RAM | Q4_K_M, layer split `1,1`, 4K context. This release was exercised on two RTX 2060 12 GB GPUs. |
-| 48 GB or more GPU memory | Consider Q6_K or Q8_0 at 4K first, then increase context after checking memory use. |
-| CPU only, or Apple Silicon unified memory | Start with Q4_K_M and preferably 32 GB or more available system/unified memory. CPU-only and Metal performance were not benchmarked for this release. |
-
-These are starting recommendations, not exact minimums. Weight-file size excludes runtime buffers, recurrent state, attention cache, images, and other applications. A 16 GB machine is a poor fit for this approximately 33B model. CPU-only operation benefits from 48–64 GB RAM when other applications are running.
-
-## 1. Install llama.cpp and the download tool
-
-Use a recent **llama.cpp** build with Qwen3.5 text and vision support. The release was tested with commit `56381e407c0ccfb3a6f71e668a27a901001d22ce`. Older bundled runtimes can reject the model architecture or load it incorrectly.
-
-### Windows
-
-Install Python 3.10+ if needed, then open PowerShell:
-
-```powershell
-winget install llama.cpp
-python -m pip install --upgrade huggingface_hub requests
-llama-server --version
-```
-
-Open a new terminal after installing if commands are not found. For NVIDIA acceleration, use a compatible CUDA build from the [official llama.cpp releases](https://github.com/ggml-org/llama.cpp/releases), extract the complete archive, and keep its DLLs alongside `llama-server.exe`. From that folder, replace `llama-server` in the commands below with `.\llama-server.exe`. Confirm the startup log detects your GPU; installing a package alone does not establish which backend it uses.
-
-### macOS
-
-With [Homebrew](https://brew.sh/) and Python installed:
-
-```bash
-brew install llama.cpp
-python3 -m venv .venv
-source .venv/bin/activate
-python -m pip install --upgrade huggingface_hub requests
-llama-server --version
-```
-
-### Ubuntu / Debian: reproducible NVIDIA build
-
-Install a compatible NVIDIA driver and CUDA Toolkit first; `nvidia-smi` and `nvcc --version` should work. Then:
-
-```bash
-sudo apt-get update
-sudo apt-get install -y git cmake build-essential libcurl4-openssl-dev python3-venv
-git clone https://github.com/ggml-org/llama.cpp
-cd llama.cpp
-git checkout 56381e407c0ccfb3a6f71e668a27a901001d22ce
-cmake -S . -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
-cmake --build build --config Release -j 4 --target llama-server
-export PATH="$PWD/build/bin:$PATH"
-cd ..
-python3 -m venv .venv
-source .venv/bin/activate
-python -m pip install --upgrade huggingface_hub requests
-```
-
-For a CPU-only build, configure with `-DGGML_CUDA=OFF` and start the server with `-ngl 0`. Other backends and installation options are covered in the [llama.cpp installation guide](https://github.com/ggml-org/llama.cpp/blob/56381e407c0ccfb3a6f71e668a27a901001d22ce/docs/install.md) and [build guide](https://github.com/ggml-org/llama.cpp/blob/56381e407c0ccfb3a6f71e668a27a901001d22ce/docs/build.md).
-
-## 2. Download the model
-
-Run these in the directory where you want the `agnes` folder. The public download does not require a paid account or hosted inference subscription.
-
-```bash
-hf download 0xKitkat/Agnes-3.0-Flash-GGUF Agnes-3.0-Flash-Q4_K_M.gguf --local-dir agnes
-```
-
-For image input, also download:
-
-```bash
-hf download 0xKitkat/Agnes-3.0-Flash-GGUF mmproj-Agnes-3.0-Flash-F16.gguf --local-dir agnes
-```
-
-For another published quant, replace `Q4_K_M` in the filename with `Q5_K_M`, `Q6_K`, or `Q8_0`. Rerun the same download command after an interruption. Allow disk space for the chosen file plus the approximately 0.93 GB projector if used.
-
-SHA-256 checksums are recorded in [release-manifest.json](./release-manifest.json). To calculate a local checksum:
-
-```powershell
-Get-FileHash agnes/Agnes-3.0-Flash-Q4_K_M.gguf -Algorithm SHA256
-```
-
-On Linux use `sha256sum agnes/Agnes-3.0-Flash-Q4_K_M.gguf`; on macOS use `shasum -a 256` with the same path.
-
-## 3. Start the local server
+Local validation has passed. Files are uploading; the release is not complete yet. The final model card and evaluation tables will appear after remote hash verification.
 
-### Text-only: single GPU or automatic fitting
-
-```bash
-llama-server -m agnes/Agnes-3.0-Flash-Q4_K_M.gguf --alias agnes -c 4096 --parallel 1 --fit on --jinja --flash-attn on --batch-size 256 --ubatch-size 128 --threads 6 --temp 1.0 --top-p 0.95 --top-k 20 --host 127.0.0.1 --port 8080
-```
-
-Open **http://127.0.0.1:8080** for the built-in chat UI. Keep the terminal running. The API is at **http://127.0.0.1:8080/v1**; its model name is **`agnes`**. This local server does not need an API key. If a client requires a nonempty key field, use `local`.
-
-The command lets llama.cpp choose GPU offloading with `--fit on`. If it runs out of GPU memory, specify a smaller layer count, for example `-ngl 20`, and adjust from there. Use `-ngl 0` for CPU-only inference. Six CPU threads is a starting value used in our checks; tune it for your CPU.
-
-### Text and images: tested dual-GPU layout
-
-```bash
-llama-server -m agnes/Agnes-3.0-Flash-Q4_K_M.gguf --mmproj agnes/mmproj-Agnes-3.0-Flash-F16.gguf --alias agnes -c 4096 --parallel 1 -ngl 99 --split-mode layer --tensor-split 1,1 --jinja --flash-attn on --batch-size 256 --ubatch-size 128 --threads 6 --temp 1.0 --top-p 0.95 --top-k 20 --host 127.0.0.1 --port 8080
-```
-
-`-ngl 99` requests all eligible layers on the GPUs; it is not a claim that the model has 99 layers. `--tensor-split 1,1` distributes layers between two GPUs. On one GPU, use the first command and add `--mmproj agnes/mmproj-Agnes-3.0-Flash-F16.gguf`. For larger quants, lower the GPU layer count or let automatic fitting choose it.
-
-### Thinking and context settings
-
-The API examples below explicitly disable thinking, matching release testing. To make this the server default for the web UI too, set `LLAMA_ARG_CHAT_TEMPLATE_KWARGS` **before** starting the server:
-
-```powershell
-# PowerShell
-$env:LLAMA_ARG_CHAT_TEMPLATE_KWARGS = '{"enable_thinking":false}'
-```
-
-```bash
-# Bash / zsh
-export LLAMA_ARG_CHAT_TEMPLATE_KWARGS='{"enable_thinking":false}'
-```
-
-For experimental thinking-on use, send `"chat_template_kwargs": {"enable_thinking": true, "reasoning_effort": "xhigh"}` in the request. The source template accepts `low`, `medium`, and `xhigh` effort (the actual pinned template uses `xhigh`, even though the upstream card describes `high`). Allocate more output tokens and context for reasoning. This release's measured results do **not** validate reasoning-on quality, tool calling, video, or the advertised 262,144-token context. Start at 4K; try 8K only after confirming adequate memory and prompt-plus-output space.
-
-## 4. Use the API from Python
-
-Save as `chat_agnes.py`, then run `python chat_agnes.py` while the server is running:
-
-```python
-import unicodedata
-import requests
-
-prompt = unicodedata.normalize("NFC", "Explain how a rainbow forms in three sentences.")
-response = requests.post(
-    "http://127.0.0.1:8080/v1/chat/completions",
-    json={
-        "model": "agnes",
-        "messages": [{"role": "user", "content": prompt}],
-        "temperature": 1.0,
-        "top_p": 0.95,
-        "top_k": 20,
-        "max_tokens": 512,
-        "chat_template_kwargs": {"enable_thinking": False},
-        "stream": False,
-    },
-    timeout=1800,
-)
-response.raise_for_status()
-print(response.json()["choices"][0]["message"]["content"])
-```
-
-For deterministic checks, change `temperature` to `0` and add `"seed": 20260912`. Sampling defaults above come from the [original model](https://huggingface.co/Agnes-AI/Agnes-3.0-Flash); they are a starting point, not a new tuning benchmark.
-
-### Ask about a local image
-
-Start the server with the projector. Save this as `image_agnes.py`, place an `image.jpg` alongside it, and run `python image_agnes.py`:
-
-```python
-import base64
-from pathlib import Path
-import requests
-
-encoded = base64.b64encode(Path("image.jpg").read_bytes()).decode("ascii")
-response = requests.post(
-    "http://127.0.0.1:8080/v1/chat/completions",
-    json={
-        "model": "agnes",
-        "messages": [{
-            "role": "user",
-            "content": [
-                {"type": "text", "text": "Describe the main objects in this image."},
-                {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encoded}"}},
-            ],
-        }],
-        "temperature": 0,
-        "max_tokens": 256,
-        "chat_template_kwargs": {"enable_thinking": False},
-        "stream": False,
-    },
-    timeout=1800,
-)
-response.raise_for_status()
-print(response.json()["choices"][0]["message"]["content"])
-```
-
-For PNG input, change the file path and use `data:image/png;base64,`. Begin with one modest-size image; image tokens also consume context. The API format follows [llama.cpp's multimodal server documentation](https://github.com/ggml-org/llama.cpp/blob/56381e407c0ccfb3a6f71e668a27a901001d22ce/docs/multimodal.md).
-
-## Troubleshooting
-
-| Symptom | What to change |
-|---|---|
-| Unknown architecture, missing tensor, or unsupported operation | Update llama.cpp to the tested revision or a newer compatible build. This is a Qwen3.5 GGUF graph conversion, so older runtimes may not support it. |
-| GPU out of memory | Use Q4_K_M; reduce `-ngl`, context, or `--ubatch-size` to `64`; keep `--parallel 1`. Add the projector only when needed. |
-| Very slow generation | Check the startup log for actual GPU offloading. Q5/Q6/Q8 may spill heavily to CPU on 24 GB total VRAM. Try Q4 and close memory-heavy applications. |
-| Thought tags or unexpectedly long reasoning | Set `chat_template_kwargs.enable_thinking` to `false`, or set the server environment variable shown above. |
-| Output stops early | Check the response's `finish_reason`. Raise `max_tokens` if it is `length`, while ensuring sufficient context remains. |
-| Image request fails | Check that the matching projector is loaded and that the data URL's MIME type matches the image file. |
-| Unicode tokenization differs from Transformers | NFC-normalize text before sending it. Stock llama.cpp does not reproduce the upstream tokenizer's NFC normalization automatically. |
-
-Other applications must bundle a sufficiently recent compatible backend. LM Studio and Ollama were not part of this release's validation; the commands above use llama.cpp directly.
-<!-- GGUF-USAGE-END -->
-
-## Validation
-
-| Quant | File GiB | 24 deterministic tasks | Native top-10 token overlap | 3 synthetic image checks | WikiText PPL |
-|---|---:|---:|---:|---:|---:|
-| Q4_K_M | 18.40 | 95.8% | 93.1% | 100.0% | 7.160 |
-| Q5_K_M | 21.42 | 95.8% | 95.6% | 100.0% | 7.113 |
-
-These are small functional tests, not comprehensive capability benchmarks. Native-reference checks compare next-token distributions on 16 prompts with a layer-streamed upstream calculation using FP32 computation and FP16 residual storage. All generation checks use thinking disabled. Per-quant reports and SHA-256 hashes are included. Long-context limits, video, reasoning-on quality, and tool calling are not exhaustively tested here.
-
-WikiText perplexity uses eight 512-token chunks from a pinned WikiText-2 test corpus. Each normal quant must stay within 5% of the unmodified Q4 baseline. This is a subset loss check, not a full-corpus benchmark.
-
-Q4_K_M, Q5_K_M and Q6_K use a 32-chunk importance matrix computed from the unmodified Q8_0 model. Q8_0 is quantized directly from the original BF16 weights. The projector is F16 and is shared by all quants.
-
-## Architecture mapping
-
-Agnes uses the same attention computation as the supported Qwen3.5 GGUF graph. Its additional parallel SwiGLU branch is preserved exactly by concatenating both branches' gate/up matrices and concatenating their down matrices on the input dimension. This is an algebraic graph conversion, not a trained modification. MTP speculative decoding weights are omitted from GGUF. The vision tower maps to the Qwen3.5 vision projector graph. Structural, rotary, tokenizer, and FP32 next-token equivalence tests are included.
-
-The upstream tokenizer applies NFC Unicode normalization; stock llama.cpp's Qwen3.5 tokenizer does not. For exact agreement on decomposed Unicode, normalize text with `unicodedata.normalize("NFC", text)` before rendering the chat prompt. Already-normalized English and Chinese text is unaffected.
-
-
-Original model and implementation: Agnes AI. Build code was generated with AI assistance and tested as reported. See the original license and upstream model card for attribution.
+**Follow [@procrastiness on Twitter/X](https://twitter.com/procrastiness) for more model releases and updates.**