Upload README.md with huggingface_hub

README.md

@@ -1,199 +1,266 @@
 ---
+license: apache-2.0
 library_name: transformers
-tags: []
+pipeline_tag: text-generation
+tags:
+- hunyuan
+- hy3
+- moe
+- text-generation
+- heretic
+- uncensored
+- decensored
+- abliterated
 ---
+# This is a decensored version of a model, made using [Heretic](https://github.com/p-e-w/heretic) v1.2.0+custom
 
-# Model Card for Model ID
+## Abliteration parameters
 
-<!-- Provide a quick summary of what the model is/does. -->
+| Parameter | Value |
+| :-------- | :---: |
+| **direction_index** | 36.95 |
+| **attn.o_proj.max_weight** | 2.72 |
+| **attn.o_proj.max_weight_position** | 62.94 |
+| **attn.o_proj.min_weight** | 2.44 |
+| **attn.o_proj.min_weight_distance** | 60.89 |
+| **mlp.down_proj.max_weight** | 1.47 |
+| **mlp.down_proj.max_weight_position** | 67.79 |
+| **mlp.down_proj.min_weight** | 0.54 |
+| **mlp.down_proj.min_weight_distance** | 31.09 |
 
+## Performance
 
+| Metric | This model | Original model (a model) |
+| :----- | :--------: | :---------------------------: |
+| **KL divergence** | 0.2398 | 0 *(by definition)* |
+| **Refusals** | 1/100 | 99/100 |
 
-## Model Details
+-----
 
-### Model Description
+<p align="left">
+    <a href="https://huggingface.co/tencent/Hy3/blob/main/README_CN.md">中文</a>&nbsp;｜&nbsp;English
+</p>
+<br>
 
-<!-- Provide a longer summary of what this model is. -->
+<p align="center">
+ <img src="assets/logo-en.png" width="400"/> <br>
+</p>
 
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+<div align="center" style="line-height: 1;">
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
 
-### Model Sources [optional]
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue)](#license)
+&nbsp;&nbsp;
+[![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Tencent%20Hy-ffc107?color=ffc107&logoColor=white)](https://huggingface.co/tencent/Hy3)
+&nbsp;&nbsp;
+[![ModelScope](https://img.shields.io/badge/ModelScope-Tencent%20Hy-624aff)](https://modelscope.cn/models/Tencent-Hunyuan/Hy3)
+&nbsp;&nbsp;
+[![cnb.cool](https://img.shields.io/badge/cnb.cool-Tencent%20Hy-blue?logoColor=white)](https://cnb.cool/ai-models/tencent/Hy3)
+&nbsp;&nbsp;
+[![GitCode](https://img.shields.io/badge/GitCode-Tencent%20Hy-red?logoColor=white)](https://ai.gitcode.com/tencent_hunyuan/Hy3)
 
-<!-- Provide the basic links for the model. -->
+</div>
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+<p align="center">
+    🖥️&nbsp;<a href="https://aistudio.tencent.com/"><b>Official Website</b></a>&nbsp;&nbsp;|&nbsp;&nbsp;
+    💬&nbsp;<a href="https://github.com/Tencent-Hunyuan/Hy3"><b>GitHub</b></a></p>
 
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+---
 
-#### Preprocessing [optional]
+## Table of Contents
+
+- [Model Introduction](#model-introduction)
+- [Stronger Agent Capabilities](#stronger-agent-capabilities)
+- [More Reliable Product Experiences](#more-reliable-product-experiences)
+- [Benchmark Appendix](#benchmark-appendix)
+- [News](#news)
+- [Model Links](#model-links)
+- [Quickstart](#quickstart)
+- [Deployment](#deployment)
+  - [vLLM](#vllm)
+  - [SGLang](#sglang)
+- [Finetuning](#finetuning)
+- [Quantization](#quantization)
+- [License](#license)
+- [Contact Us](#contact-us)
 
-[More Information Needed]
+---
 
+## Model Introduction
 
-#### Training Hyperparameters
+**Hy3** is a 295B-parameter Mixture-of-Experts (MoE) model with 21B active parameters and 3.8B MTP layer parameters, developed by the Tencent Hy Team. Following the Hy3 Preview launch in late April, we gathered feedback from 50+ products and scaled up post-training with higher quality data. Today, we introduce Hy3, which outperforms similar-size models and rivals flagship open-source models with 2-5x parameters. It also shows significant gains in utility across various products and productivity tasks.
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
-#### Speeds, Sizes, Times [optional]
+| Property | Value |
+|:---|:---|
+| Architecture | Mixture-of-Experts (MoE) |
+| Total Parameters | 295B |
+| Activated Parameters | 21B |
+| MTP Layer Parameters | 3.8B |
+| Number of Layers (excluding MTP layer) | 80 |
+| Number of MTP Layers | 1 |
+| Attention Heads | 64 (GQA, 8 KV heads, head dim 128) |
+| Hidden Size | 4096 |
+| Intermediate Size | 13312 |
+| Context Length | 256K |
+| Vocabulary Size | 120832 |
+| Number of Experts | 192 experts, top-8 activated |
+| Supported Precisions | BF16 |
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+## Stronger Agent Capabilities
 
-[More Information Needed]
+Building on Hy3 Preview, we further improved the quality and diversity of post-training data while scaling up RL training. Hy3 shows solid gains across reasoning, agentic, and long-context tasks, competitive with much larger flagship models.
 
-## Evaluation
+<p align="center">
+  <img src="assets/benchmark.png" width="100%"/>
+</p>
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+In productivity scenarios such as coding, office work, financial modeling, frontend design, and game development, Hy3 has made remarkable progress and can now serve as a reliable, cost-effective model option.
 
-### Testing Data, Factors & Metrics
+We don't think public benchmark scores tell the full story. So we ran a blind evaluation with 270 experts using tasks from their work, and Hy3 scored 2.67/4, outperforming GLM-5.1 at 2.51/4. The advantage was most substantial in frontend development, data & storage, and CI/CD tasks.
 
-#### Testing Data
+## More Reliable Product Experiences
 
-<!-- This should link to a Dataset Card if possible. -->
+Model usefulness is not fully captured by benchmarks. Based on extensive product feedback, we identified and fixed the following issues, receiving consistently positive feedback from product teams.
 
-[More Information Needed]
+**Stability of tool calls and output formats**: We fixed multiple baseline reliability issues, bringing the model to production-grade standards across tool configurations and output constraints. Tool-call error recovery and overall efficiency improved. Hy3 also generalizes across different agent scaffoldings. On SWE-Bench Verified, accuracy variance across scaffoldings like CodeBuddy, Cline, and KiloCode remains within 4%.
 
-#### Factors
+**Knowledge and anti-hallucination**: Guided by the ideal of "answer when grounded, state when evidence is missing, do not conflate sources or fabricate data," we implemented fine-grained data cleaning and training constraints. In internal evaluations based on real-world scenarios, Hy3's hallucination rate dropped from 12.5% to 5.4%, and commonsense error rates fell from 25.4% to 12.7%. These improvements materially reduce fact conflation, fabrication, and logical contradiction.
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+**Complex context retention and multi-turn intent tracking**: Through joint optimization of SFT and RL, Hy3 improved on operational pain points like coreference resolution, ellipsis recovery, and multi-turn constraint inheritance. On internal comprehensive multi-turn tests, the issue rate dropped from 17.4% to 7.9%. Hy3 also improved markedly on long-dialogue evals like MRCR. Its outputs are more concise while ensuring complex intents do not decay or drift over long-horizon interactions.
 
-[More Information Needed]
+## Benchmark Appendix
 
-#### Metrics
+<p align="center">
+  <img src="assets/benchmark-appendix.png" width="100%"/>
+</p>
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+## News
 
-[More Information Needed]
 
-### Results
+* 🔥 We open-source **Hy3** and **Hy3-FP8** model weights on [Hugging Face](https://huggingface.co/tencent/Hy3), [ModelScope](https://modelscope.cn/models/Tencent-Hunyuan/Hy3), [GitCode](https://ai.gitcode.com/tencent_hunyuan/Hy3), and [CNB](https://cnb.cool/ai-models/tencent/Hy3).
 
-[More Information Needed]
+## Model Links
 
-#### Summary
 
+| Model Name | Description | Hugging Face | ModelScope | GitCode | CNB |
+|:---|:---|:---:|:---:|:---:|:---:|
+| Hy3 | Instruct model | 🤗 [Model](https://huggingface.co/tencent/Hy3) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3) | [Model](https://cnb.cool/ai-models/tencent/Hy3) |
+| Hy3-FP8 | FP8 quantized instruct model | 🤗 [Model](https://huggingface.co/tencent/Hy3-FP8) | [Model](https://modelscope.cn/models/Tencent-Hunyuan/Hy3-FP8) | [Model](https://ai.gitcode.com/tencent_hunyuan/Hy3-FP8) | [Model](https://cnb.cool/ai-models/tencent/Hy3-FP8) |
 
+## Quickstart
 
-## Model Examination [optional]
+Deploy Hy3 with [vLLM](#vllm) or [SGLang](#sglang) first, then call the OpenAI-compatible API:
 
-<!-- Relevant interpretability work for the model goes here -->
+```python
+from openai import OpenAI
 
-[More Information Needed]
+client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")
 
-## Environmental Impact
+response = client.chat.completions.create(
+    model="hy3",
+    messages=[
+        {"role": "user", "content": "Hello! Can you briefly introduce yourself?"},
+    ],
+    temperature=0.9,
+    top_p=1.0,
+    # reasoning_effort: "no_think" (default, direct response), "low", "high" (deep chain-of-thought)
+    extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
+)
+print(response.choices[0].message.content)
+```
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+> **Recommended parameters**: `temperature=0.9`, `top_p=1.0`.
+>
+> **Reasoning mode**: Set `reasoning_effort` to `"high"` for complex tasks (math, coding, reasoning) or `"no_think"` for direct responses.
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+See the [Deployment](#deployment) section below for how to start the API server.
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+## Deployment
 
-## Technical Specifications [optional]
+Hy3 has 295B parameters in total. To serve it on 8 GPUs, we recommend using H20-3e or other GPUs with larger memory capacity.
 
-### Model Architecture and Objective
+For production serving, we recommend using vLLM or SGLang, both of which provide dedicated recipes for Hy3:
 
-[More Information Needed]
+- [vLLM](https://github.com/vllm-project/vllm) - see [vLLM recipes](https://recipes.vllm.ai/tencent/Hy3)
 
-### Compute Infrastructure
+- [SGLang](https://docs.sglang.io/) - see [SGLang cookbook](https://lmsysorg.mintlify.app/cookbook/autoregressive/Tencent/Hy3)
 
-[More Information Needed]
+### vLLM
 
-#### Hardware
+Build vLLM from source:
+```bash
+uv venv --python 3.12 --seed --managed-python
+source .venv/bin/activate
+git clone https://github.com/vllm-project/vllm.git
+cd vllm
+uv pip install --editable . --torch-backend=auto
+```
 
-[More Information Needed]
+Start the vLLM server with MTP enabled:
 
-#### Software
+```bash
+# Switch to trtllm backend to work-around mnnvl workspace size issue.
+export VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm
+vllm serve tencent/Hy3 \
+  --tensor-parallel-size 8 \
+  --speculative-config.method mtp \
+  --speculative-config.num_speculative_tokens 2 \
+  --tool-call-parser hy_v3 \
+  --reasoning-parser hy_v3 \
+  --enable-auto-tool-choice \
+  --port 8000 \
+  --served-model-name hy3
+```
 
-[More Information Needed]
+### SGLang
 
-## Citation [optional]
+Build SGLang from source:
+```bash
+git clone https://github.com/sgl-project/sglang
+cd sglang
+pip3 install pip --upgrade
+pip3 install "transformers>=5.6.0"
+pip3 install -e "python"
+```
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+Launch SGLang server with MTP enabled:
 
-**BibTeX:**
+```bash
+python3 -m sglang.launch_server \
+  --model tencent/Hy3 \
+  --tp-size 8 \
+  --tool-call-parser hunyuan \
+  --reasoning-parser hunyuan \
+  --speculative-num-steps 2 \
+  --speculative-eagle-topk 1 \
+  --speculative-num-draft-tokens 3 \
+  --speculative-algorithm EAGLE \
+  --port 8000 \
+  --served-model-name hy3
+```
 
-[More Information Needed]
+## Finetuning
 
-**APA:**
+Hy3 provides a complete model finetuning pipeline. For detailed documentation, please refer to: [Finetuning Guide](https://huggingface.co/tencent/Hy3/blob/main/finetune/README.md)
 
-[More Information Needed]
+## Quantization
 
-## Glossary [optional]
+We provide [AngelSlim](https://github.com/tencent/AngelSlim), a more accessible, comprehensive, and efficient toolkit for large model compression. AngelSlim supports a comprehensive suite of compression tools for large-scale multimodal models, including common quantization algorithms, low-bit quantization, and speculative sampling.
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+## License
 
-[More Information Needed]
 
-## More Information [optional]
+Hy3 is released under the **Apache License 2.0**. See [LICENSE](https://huggingface.co/tencent/Hy3/blob/main/LICENSE) for details.
 
-[More Information Needed]
+## Contact Us
 
-## Model Card Authors [optional]
+If you would like to leave a message for our R&D and product teams, welcome to contact us. You can also reach us via email:
 
-[More Information Needed]
+📧 **hunyuan_opensource@tencent.com**
 
-## Model Card Contact
+---
 
-[More Information Needed]
+<p align="center">
+  <i>Hy3 is developed by the Tencent Hy Team.</i>
+</p>