Update README.md

README.md

@@ -7,14 +7,114 @@ library_name: transformers
 pipeline_tag: image-text-to-text
 tags:
 - agnes-ai
+- uncensored
 - reasoning
 - multimodal
+- fp8
+- w8a8
+- vllm
+- sglang
 - long-context
 - hybrid-attention
 ---
-<p align="center">
-  <img width="132" src="assets/agnes_logo.svg" alt="Agnes AI logo">
-</p>
+
+
+# Agnes 3.0 Flash FP8 (W8A8, calibrated)
+
+Full FP8 quantization of Agnes-3.0-Flash Preview (33B parameters, multimodal).
+Weights and activations are FP8 e4m3 with static per-tensor scales, calibrated
+with LLM Compressor. This gives you both the model and context at FP8. Text, 
+image, and video inputs are preserved, and the chat template is upstream's.
+
+The Sglang patch from the original model checkpoint is included here as well.
+
+*Note that this model seems to be naturally uncensored. I ran Heretic's assessment
+and it gave 0/300 refusals. So no need to abliterate this model. And with that,
+my usual warning here:*
+
+> **Uncensored model:** the language checkpoint has undergone abliteration 
+to reduce refusal behavior. Treat outputs as untrusted, apply application-level 
+safeguards, and do not assume the model will decline harmful requests.
+
+> **User responsibility:** this model is provided without warranty. The 
+creators, uploaders, and maintainers are not responsible or liable for what 
+others generate, publish, deploy, or otherwise do with this abliterated model. 
+Users must operate it responsibly, apply appropriate safeguards, comply with 
+applicable law, and respect third-party rights. This model is for research 
+purposes only and is not intended for production use.
+
+
+## Quantization recipe
+
+| Field | Value |
+|---|---|
+| Format | `compressed-tensors`, `float-quantized` |
+| Weights | FP8 (e4m3), per-tensor symmetric, static |
+| Activations | FP8 (e4m3), per-tensor symmetric, static, calibrated |
+| Targets | all `Linear` modules except the protected list below |
+| Toolkit | llm-compressor 0.13.0, compressed-tensors 0.18.0 |
+| Calibration | `HuggingFaceH4/ultrachat_200k`, split `train_sft`, revision `8049631c405ae6576f93f445c6b8166f76f5505a` |
+| Calibration size | 512 samples, max sequence length 2048, batch 1, seed 42 |
+
+Kept at original precision: `lm_head`, `embed_tokens`, the full `model.visual`
+tower, the recurrent-state layers of delta attention (`conv1d`, `in_proj_a`,
+`in_proj_b`), and every MTP weight (shipped unquantized in
+`model-mtp.safetensors`).
+
+`agnes_quantization_manifest.json` records the full build, including the
+SHA-256 of the calibration prompt ids used, so the run is reproducible piece
+for piece. `recipe.yaml` is the machine-readable form of the table above.
+
+## Why the quantization runs on a fused checkpoint
+
+The upstream FFN carries two branches per layer: a main branch (width 17,408)
+and a parallel branch (width 2,048). This checkpoint folds them into a single
+set of projections per layer (`intermediate_size: 19456`,
+`parallel_ffn_intermediate_size: 0`).
+
+Fusing in bf16 is concatenation and it is exact: gate/up join along the output
+dimension, down along the input dimension. Static per-tensor FP8 scales cannot
+be merged after the fact, because each branch carries its own scale and a
+single fused matrix has room for exactly one. Quantizing the fused layout
+means every serving tensor is quantized once, from the values the model will
+actually run, and calibration observes the same matrix geometry as inference.
+
+## Numerics
+
+The fused layout measures 5.9e-4 full-vocabulary KL against the two-branch
+layout at identical weights. The residual comes from reduction order in the
+fused projections, and it sits below the 6.5e-4 that independent inference
+stacks produce from the same unquantized checkpoint. Perplexity moves 17.05 to
+17.07 across the fusion.
+
+The FP8 conversion itself has no formal before/after eval on this card. Run
+your own benchmarks before trusting it for a workload.
+
+## Files
+
+| Group | Contents |
+|---|---|
+| Weights | `model-00001-of-00002.safetensors`, `model-00002-of-00002.safetensors`, `model-mtp.safetensors`, `model.safetensors.index.json` |
+| Config | `config.json`, `generation_config.json`, custom `*_agnes.py` modeling and processor code |
+| Tokenizer | `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja` |
+| Provenance | `recipe.yaml`, `agnes_quantization_manifest.json` |
+| Extras | `sglang_patch/`, `serve.sh` |
+
+The checkpoint is standard `compressed-tensors float-quantized`, so any engine
+that reads that format can load it.
+
+## Upstream and license
+
+Derived from the `Agnes-AI/Agnes-3.0-Flash` Preview checkpoint at revision
+`891ce4f9ffb89b22888aa7fcc2bb2f3618867684`, Apache 2.0, unchanged. This is a
+community quantization and is not affiliated with Agnes AI. The open-weight
+Preview differs from the production/API model, and the upstream benchmark
+figures below describe the Preview checkpoint before quantization.
+
+---
+
+**The original upstream model card follows, unchanged.**
+
 
 <p align="center">
   <a href="https://agnes-ai.com/"><img src="https://img.shields.io/badge/Agnes_AI-Website-3248AF" alt="Agnes AI website"></a>