Upload folder using huggingface_hub

LICENSE

@@ -0,0 +1,446 @@
+================================================================================
+NOESIS-Hy-MT2-1.8B-BF16 — NOESIS Bundle License
+================================================================================
+
+This bundle is a NOESIS-fine-tuned derivative of the upstream model
+    tencent/Hy-MT2-1.8B
+Upstream weights and training code © upstream authors.
+NOESIS SFT-LoRA fine-tuning, provenance metadata, bundle inventory,
+sealed-rule annotations, and DHCF-FNO integration notes
+© AMAImedia 2026 (NOESIS DHCF-FNO project).
+
+================================================================================
+LICENSE REGIME — Tencent HY Community License Agreement
+================================================================================
+Three license gates apply: (1) Territory — EU excluded; (2) 100M MAU cap (separate Tencent license required above); (3) No-KD: outputs cannot be used to train other AI models. Full text in `LICENSE.md`.
+
+================================================================================
+STOP — R-NEVER-DELETE-WITHOUT-EXPLICIT-CONSENT (sealed 2026-05-21)
+    This bundle MUST NOT be deleted without an explicit operator
+    instruction "удали <path>" / "delete <path>".
+================================================================================
+
+NOESIS provenance:
+  - Founder:        Ilia Bolotnikov
+  - Organization:   AMAImedia.com (https://www.amaimedia.com)
+  - X (Twitter):    @AMAImediacom (https://x.com/AMAImediacom)
+  - Telegram:       @AMAImediacom (https://t.me/AMAImediacom)
+  - NOESIS version: v15.10
+  - Bundle date:    2026-06-06
+
+NOESIS A/B test (2026-06-06, nt318):
+  garbage% / iso_fit / overlap — measurable improvement vs upstream baseline.
+  See README.md for full table + sample outputs.
+
+================================================================================
+UPSTREAM LICENSE TEXT FOLLOWS
+================================================================================
+
+# Tencent HY Community License Agreement
+
+**Tencent Hy-MT2 Release Date:** May 21, 2026
+
+> ⚠️ **Territorial limitation:** THIS LICENSE AGREEMENT DOES NOT APPLY
+> IN THE **EUROPEAN UNION** AND IS EXPRESSLY LIMITED TO THE TERRITORY,
+> AS DEFINED BELOW.
+>
+> ✅ **EXPANDED TERRITORY vs HY-MT1.5:** Hy-MT2 (2026-05-21) excludes
+> **only the EU** — UK and South Korea are now **INCLUDED** in the
+> Territory (HY-MT1.5 had excluded EU + UK + South Korea).
+
+By using this bundle, You are deemed to have recognized and accepted
+this Agreement, which is effective immediately.
+
+---
+
+## 1. Definitions
+
+**(a) "Acceptable Use Policy"** — policy in **Exhibit A**.
+
+**(b) "Agreement"** — these terms and conditions.
+
+**(c) "Documentation"** — Tencent HY specifications, manuals, docs.
+
+**(d) "Hosted Service"** — service via API, web, or remote means.
+
+**(e) "Licensee," "You" or "Your"** — person or entity using Tencent HY Works.
+
+**(f) "Materials"** — Tencent HY + Documentation.
+
+**(g) "Model Derivatives"** — modifications, works based on, or any
+other ML model created by transfer of patterns of weights / parameters
+/ operations / Output of Tencent HY (including distillation,
+intermediate-data, or synthetic-data Outputs for training). Outputs by
+themselves are NOT Model Derivatives.
+
+**(h) "Output"** — content output of Tencent HY or a Model Derivative.
+
+**(i) "Tencent," "We" or "Us"** — Tencent corporate family.
+
+**(j) "Tencent HY"** — LLMs, generation models, multimodal LLMs and
+software/algorithms, including **Hy-MT2-1.8B / 7B / 30B-A3B** (+ FP8 /
+GGUF / 2bit-GGUF / 1.25bit-GGUF variants).
+
+**(k) "Tencent HY Works"** — Materials + Model Derivatives + derivatives.
+
+**(l) "Territory"** — **worldwide territory, excluding the European Union**.
+
+**(m) "Third Party"** — entities not under common control with Us or You.
+
+**(n) "including"** — including but not limited to.
+
+---
+
+## 2. Grant of Rights
+
+For the **Territory only**, non-exclusive, non-transferable, royalty-free
+limited license to use, reproduce, distribute, create derivative works
+(including Model Derivatives), and make modifications.
+
+---
+
+## 3. Distribution
+
+Distribute Tencent HY Works to Third Parties **exclusively in the
+Territory**, provided:
+
+**(a)** Provide all Third Party recipients a copy of this Agreement;
+
+**(b)** Modified files must carry prominent change notices;
+
+**(c)** Encouraged: technology blogpost + **"Powered by Tencent HY"** label;
+
+**(d)** Non-Hosted-Service distributions must include "Notice" text:
+*"Tencent HY is licensed under the Tencent HY Community License
+Agreement, Copyright © 2026 Tencent. All Rights Reserved. The trademark
+rights of 'Tencent HY' are owned by Tencent or its affiliate."*
+
+**(e)** **Provider disclosure** — clearly state actual provider's full
+legal name + expressly state Tencent is NOT affiliated / sponsoring.
+
+---
+
+## 4. Additional Commercial Terms
+
+If MAU > **100 million** on release-date snapshot (2026-05-21), request
+separate Tencent license. Tencent's sole discretion.
+
+---
+
+## 5. Rules of Use
+
+**(a)** Comply with laws and the Acceptable Use Policy.
+
+**(b)** **You must not use Tencent HY Works or any Output to improve
+any other AI model** (only Tencent HY / its Model Derivatives).
+
+**(c)** **You must not use Tencent HY Works outside the Territory**
+(i.e., not in the EU).
+
+---
+
+## 6-9. Standard provisions
+
+IP / trademark / patent termination / Output ownership; **"AS IS"**, no
+warranties; termination on breach; **Hong Kong SAR** law + exclusive
+jurisdiction.
+
+---
+
+## Exhibit A — Acceptable Use Policy
+
+**Last modified:** December 30, 2025
+
+You agree NOT to use Tencent HY or Model Derivatives:
+1. Outside the Territory (i.e., in the EU);
+2. In violation of applicable laws;
+3. To harm Yourself or others;
+4. To repurpose / distribute outputs to harm others;
+5. To override / circumvent safety guardrails;
+6. To exploit / harm minors;
+7. To generate verifiably false info to harm others / influence elections;
+8. To facilitate false online engagement (fake reviews, bot traffic);
+9. To defame / harass others;
+10. To generate / disseminate malware (incl. ransomware);
+11. To generate / disseminate PII to harm others;
+12. To publish machine-generated content without conspicuous disclosure;
+13. To impersonate another individual without consent;
+14. To make high-stakes automated decisions in sensitive domains;
+15. In a manner violating social ethics / moral standards;
+16. To perform / facilitate violent extremism or terrorism;
+17. To discriminate based on protected characteristics;
+18. To exploit vulnerabilities of specific groups;
+19. **For military purposes;**
+20. To engage in unauthorized professional practice.
+
+**END OF TERMS AND CONDITIONS**
+
+---
+
+## Copyright Notice
+
+```
+Copyright © 2026 Tencent. All Rights Reserved.
+        — Tencent Hy-MT2-7.5B BF16 base model. Hunyuan Translation
+          Model 2.0 dense 7B variant (HunYuanDenseV1ForCausalLM,
+          hidden 4096, 32 layers, GQA 32/8, 262K context, vocab
+          128 167), released 2026-05-21 as part of the Hy-MT2
+          family. "Fast-thinking" multilingual translation —
+          surpasses DeepSeek-V4-Pro and Kimi K2.6 in fast-thinking
+          mode. WMT26 video subtitle translation partner.
+          Distributed under the Tencent HY Community License
+          Agreement (release date: 2026-05-21).
+
+Copyright © 2026 AMAImedia
+        — NF4 (bitsandbytes 4-bit, double_quant, bf16 compute)
+          quantization step on top of the local BF16 mirror
+          `NOESIS-3.5B-A0.5B-DUBBING-FILM/Hy-MT2-7.5B-BF16`
+          (which is itself a verbatim mirror of tencent/Hy-MT2-7.5B).
+          Used as **PRIMARY DUB-LM translation backend** inside the
+          NOESIS dubbing pipeline (Tier-1 / Tier-2 language coverage,
+          live voice translate scenarios) per the new tier rule
+          R-MT-PRIMARY-7B-NF4-FALLBACK-1.8B.
+          NF4 quantization is a Model Derivative per Section 1(g)
+          and is explicitly permitted under Section 2 of the
+          Tencent HY Community License Agreement.
+          NF4 quant date: 2026-05-22 03:20:52.
+```
+
+Licensed under the Tencent HY Community License Agreement above. You
+may not use this bundle except in compliance with that License, in
+particular:
+- The Territory exclusion (**EU only** for Hy-MT2 — UK + South Korea
+  now permitted, unlike HY-MT1.5);
+- The 100M MAU commercial gate (Section 4);
+- The Section 5(b) prohibition on using Outputs to train other AI models;
+- The Section 3(d) "Powered by Tencent HY" notice obligation;
+- The Section 3(e) provider-disclosure obligation.
+
+---
+
+## NOTICE — Hy-MT2-7.5B-NOESIS-NF4
+
+> ⚠️ **License regime:** **Tencent HY Community License Agreement**
+> (NOT Apache 2.0, NOT MIT). This bundle CANNOT be combined into
+> Apache-2.0-only NF4 distributions without preserving the Tencent HY
+> License terms across the entire downstream chain. Three critical gates:
+> 1. **Territory** (EU excluded; UK + South Korea NOW INCLUDED)
+> 2. **100M MAU** (Section 4)
+> 3. **No-KD** (Section 5(b))
+
+✅ This bundle is licensed under the **Tencent HY Community License
+Agreement** (release date: 2026-05-21, Hong Kong SAR governing law).
+
+✅ AMAImedia/NOESIS is currently below the 100M MAU threshold (no
+investor at the time of NF4 quantization — see CLAUDE.md SCOPE LOCK
+2026-05-15).
+
+✅ **PRIMARY DUB-LM translation backend** in the NOESIS production
+stack per **`R-MT-PRIMARY-7B-NF4-FALLBACK-1.8B`** (sealed 2026-05-22):
+this 7B NF4 bundle is the canonical translation engine; the
+`Hy-MT2-1.8B-NOESIS-NF4` sibling (1.29 GB, VRAM peak 2.59 GB) is the
+**FALLBACK** for low-VRAM clients (≤ 4 GB GPU, mobile/edge, WebGPU
+browser deployment).
+
+✅ **OWN NF4, NOT GGUF** per **`R-MT-OWN-NF4-NOT-GGUF`**: NOESIS
+quantizes the BF16 mirror to NF4 in-house rather than relying on
+upstream `tencent/Hy-MT2-7.5B-GGUF`, which depends on the **STQ kernel
+released only in llama.cpp PR #22836** (not yet merged to stable
+llama.cpp). NF4 is production-stable on transformers 5.8.1 +
+bitsandbytes 0.49.2 per CLAUDE.md ENV_LOCK.
+
+⚠️ **Section 5(b) NO-KD prohibition:** Outputs of this model **MAY NOT
+be used** as training data, reward signal, distillation target, or
+synthetic data source for **any other AI model** (only for Tencent HY
+itself or Model Derivatives of Tencent HY). This is the critical
+incompatibility with the NOESIS knowledge-distillation pipeline:
+**Hy-MT2 outputs cannot enter the ensemble label dataset, cannot be
+mixed into KD soft-label pools, and cannot be used as DPO/RLAIF reward
+signal for any non-Hunyuan student model**. Inference outputs delivered
+directly to the end user (translated text in the dubbing pipeline
+output) are fine — that is not "training" or "improving" another AI
+model under Section 5(b).
+
+⚠️ **Section 5(c) territorial restriction:** This model **MAY NOT** be
+served to end users physically located in the **European Union**.
+NOESIS production routing must geo-fence Hunyuan-MT requests originating
+from EU and fall back to a different translation backend
+(NLLB-200 / Qwen3-MT / Madlad-400) for that region. **UK and South
+Korea are now PERMITTED** in Hy-MT2 (relaxed vs HY-MT1.5).
+
+⚠️ **Section 3(d) attribution obligation:** Display the verbatim Section
+3(d) notice (Copyright © **2026** Tencent — NOT 2025 like HY-MT1.5) in
+the application's license / about / acknowledgments surface.
+
+### Source chain
+
+```
+tencent/Hy-MT2-7.5B
+    HunYuanDenseV1ForCausalLM, 7B params, 32 layers,
+    hidden 4096, GQA 32/8, 262K context, vocab 128 167
+    "Fast-thinking" multilingual translation — surpasses
+    DeepSeek-V4-Pro / Kimi K2.6 in fast-thinking mode
+    © Tencent 2026
+    Tencent HY Community License Agreement (2026-05-21)
+    https://huggingface.co/tencent/Hy-MT2-7.5B
+        │
+        │  Local mirror, no value transformation
+        ▼
+NOESIS-3.5B-A0.5B-DUBBING-FILM/Hy-MT2-7.5B-BF16 (sibling, 4 shards, ~16.05 GB)
+    NOESIS source-of-truth mirror (R-BF16-SOURCE-FOR-NF4-STUDENT)
+    © AMAImedia 2026, inherits Tencent HY Community License
+        │
+        │  bitsandbytes NF4 + double-quant + bf16 compute
+        │  (single-pass load → quantize → save, no calibration needed)
+        │  R-MT-OWN-NF4-NOT-GGUF (we quant, not upstream GGUF)
+        │  trust_remote_code=True per NF4 manifest
+        ▼
+Hy-MT2-7.5B-NOESIS-NF4 (this folder, 1 shard, 4.65 GB)
+    PRIMARY DUB-LM translation backend (R-MT-PRIMARY-7B-NF4-FALLBACK-1.8B)
+    VRAM peak load 7.27 GB → on RTX 3060 Laptop 6 GB requires sequential
+    swapping window (release ASR/TTS specialists during translation pass)
+    OR fall back to 1.8B NF4 sibling for fixed-VRAM clients
+    © AMAImedia 2026 (NOESIS DHCF-FNO project)
+    Inherits Tencent HY Community License Agreement
+    Quant date: 2026-05-22 03:20:52
+```
+
+### File inventory + quantization specifications
+
+| File | Size | Format | Role |
+|------|-----:|--------|------|
+| `model.safetensors` | 4.65 GB | **NF4** (bitsandbytes) | Quantized weights (single shard) |
+| `config.json` | 1.5 KB | JSON | HunYuanDenseV1 config + `quantization_config` block |
+| `generation_config.json` | small | JSON | Sampling defaults (temp=0.7, top_p=0.6, top_k=20, rep_pen=1.05) |
+| `chat_template.jinja` | small | Jinja2 | Hy-MT2 chat template (translation-optimized) |
+| `tokenizer.json` | — | JSON | Fast tokenizer (vocab 128 167) |
+| `tokenizer_config.json` | — | JSON | Tokenizer metadata |
+| `main_result.png` | — | PNG | Upstream benchmark visualization |
+| `LICENSE.txt` | 16 KB | text | Original Tencent HY License (verbatim, preserved per Section 3(a)) |
+| `LICENSE.md` | (this file) | Markdown | Same License in Markdown + NOESIS NOTICE |
+| `README.md` | — | Markdown | NOESIS-standard model card + Tencent upstream content |
+| `NOESIS_NF4_MANIFEST.json` | small | JSON | NOESIS quant pipeline manifest |
+
+### Quantization specifications (preserved verbatim from NF4 manifest)
+
+```
+Library                       : bitsandbytes (0.49.2 per CLAUDE.md
+                                ENV_LOCK 2026-05-17)
+Method                        : NF4 (Normalized Float 4-bit)
+bnb_4bit_use_double_quant     : True (additional ~5% saving)
+bnb_4bit_compute_dtype        : bfloat16
+bnb_4bit_quant_storage        : uint8
+Device map                    : {"": 0} (R-NF4-DEVICE-MAP-EXPLICIT)
+trust_remote_code_used        : True (per NF4 manifest)
+Output disk size              : 4.65 GB (single safetensors, down from
+                                ~16.05 GB BF16, ~3.45× compression)
+VRAM peak (load)              : 7.27 GB per manifest
+                                ⚠️ EXCEEDS RTX 3060 6 GB on cold start —
+                                requires sequential window swapping
+                                (release ASR/TTS specialists during
+                                translation pass), or fall back to 1.8B
+                                NF4 sibling for fixed-VRAM clients
+Architecture                  : HunYuanDenseV1ForCausalLM (Tencent native)
+Hidden size                   : 4 096
+Intermediate size             : 14 336
+num_hidden_layers             : 32
+num_attention_heads           : 32
+num_key_value_heads           : 8 (GQA 4 : 1)
+head_dim                      : 128
+use_qk_norm                   : True
+Norm                          : RMSNorm (eps 1e-5)
+Tie word embeddings           : True
+Vocab size                    : 128 167 (Hy-MT2-7.5B-specific; differs
+                                from 1.8B 120 818; see
+                                R-HUNYUAN-VOCAB-128167-7B-NEW)
+Max position embeddings       : 262 144 (262K context, dynamic NTK)
+RoPE                          : dynamic NTK (alpha 1000, factor 1.0)
+transformers_version          : 5.8.1 (matches ENV_LOCK 2026-05-17,
+                                R-TRANSFORMERS-5-8-1)
+Quant date                    : 2026-05-22 03:20:52
+NOESIS role                   : PRIMARY DUB-LM translation backend
+                                (R-MT-PRIMARY-7B-NF4-FALLBACK-1.8B)
+```
+
+---
+
+## Required attribution
+
+**Tencent Hy-MT2 base model:**
+> tencent/Hy-MT2-7.5B
+> © Tencent 2026, licensed under Tencent HY Community License Agreement
+> https://huggingface.co/tencent/Hy-MT2-7.5B
+> https://github.com/Tencent-Hunyuan/Hy-MT2
+
+**Mandatory Section 3(d) NOTICE:**
+> *"Tencent HY is licensed under the Tencent HY Community License
+> Agreement, Copyright © 2026 Tencent. All Rights Reserved. The
+> trademark rights of 'Tencent HY' are owned by Tencent or its
+> affiliate."*
+
+**Encouraged Section 3(c) "Powered by Tencent HY" label** — render in
+the product's translation-backend credits surface.
+
+**Mandatory Section 3(e) provider disclosure:**
+> The actual provider of this dubbing automation service is
+> **AMAImedia.com** (NOESIS DHCF-FNO project, https://amaimedia.com).
+> Tencent is **not** affiliated with, associated with, sponsoring,
+> or endorsing the NOESIS Professional Multilingual Dubbing
+> Automation Platform.
+
+**NF4 quantization + NOESIS bundling (this folder):**
+> Hy-MT2-7.5B-NOESIS-NF4
+> bitsandbytes NF4 + double_quant + bfloat16 compute
+> © AMAImedia 2026 (NOESIS DHCF-FNO project)
+> Inherits Tencent HY Community License Agreement
+> Released as part of NOESIS Professional Multilingual Dubbing Platform
+
+---
+
+## NOESIS sealed-rule constraints
+
+| Rule | Summary |
+|------|---------|
+| `R-TENCENT-HY-COMMUNITY-CC` | Tencent HY Community License regime, NOT Apache 2.0 |
+| `R-TERRITORY-EXCLUDE-EU` | **CRITICAL** — Section 5(c): EU **excluded** (UK + KR NOW INCLUDED in Hy-MT2 — relaxed vs HY-MT1.5) |
+| `R-100M-MAU-GATE` | Section 4 — separate Tencent license above 100M MAU |
+| `R-TENCENT-HY-NO-KD-TRAIN` | **CRITICAL** — Section 5(b): Outputs cannot train other AI models |
+| `R-POWERED-BY-TENCENT-HY-LABEL` | Section 3(c) — encouraged "Powered by Tencent HY" badge |
+| `R-DISCLOSE-AMAIMEDIA-PROVIDER` | Section 3(e) — non-affiliation disclosure |
+| `R-TENCENT-HY-NOTICE-3D` | Section 3(d) verbatim notice (Copyright © **2026** Tencent) |
+| **`R-MT-PRIMARY-7B-NF4-FALLBACK-1.8B`** | **This bundle = PRIMARY DUB-LM backend**; 1.8B NF4 sibling = FALLBACK |
+| **`R-MT-OWN-NF4-NOT-GGUF`** | NOESIS quantizes BF16 → NF4 in-house, NOT upstream Hy-MT2-7.5B-GGUF (depends on llama.cpp PR #22836 STQ kernel) |
+| **`R-HF-NAMING-QUANT-ONLY-PRESERVE-UPSTREAM`** | Folder name preserves upstream model name + quant suffix only |
+| `R-NF4-DEVICE-MAP-EXPLICIT` | Load with `device_map={"": 0}`; never `device_map="auto"` with NF4 (CLAUDE.md GOLDEN RULE 2) |
+| `R-HUNYUAN-VOCAB-128167-7B-NEW` | Vocab 128 167 is **incompatible** with Qwen3 family (151 936) AND with sibling Hy-MT2-1.8B (120 818) for logit-level KD mixing |
+| `R-HY-MT-DUB-LM-TRANSLATION-ROLE` | DUB-LM translation backend only, not chat/agent/reasoning |
+| `R-HUNYUAN-DENSE-V1-NATIVE-LOADER` | `transformers >= 5.6.0` (Hy-MT2 baseline); NOESIS env uses 5.8.1; `trust_remote_code=True` per manifest |
+| `R-HUNYUAN-262K-CONTEXT-DYNAMIC-NTK` | 262K context via dynamic NTK (alpha 1000) |
+| `R-HUNYUAN-LIVE-VOICE-TRANSLATE` | Qualified for live voice translate scenarios |
+| `R-VENDORED-INTERNAL` | NOESIS bundle. `LICENSE.txt` preserved verbatim per Section 3(a) |
+
+---
+
+## NOESIS Professional Multilingual Dubbing Automation Platform
+
+**Framework:** DHCF-FNO (Deterministic Hybrid Control Framework for Frozen Neural Operators)
+
+| Field | Value |
+|-------|-------|
+| Founder | Ilia Bolotnikov |
+| Organization | AMAImedia.com — https://www.amaimedia.com |
+| X (Twitter) | https://x.com/AMAImediacom |
+| LinkedIn | https://www.linkedin.com/in/ilia-bolotnikov |
+| Telegram | https://t.me/AMAImediacom |
+
+| Field | Value |
+|-------|-------|
+| NOESIS version at quant | v15.8 |
+| Quantization date | 2026-05-22 03:20:52 |
+| Parent BF16 source | `NOESIS-3.5B-A0.5B-DUBBING-FILM/Hy-MT2-7.5B-BF16` (4 shards, ~16.05 GB) |
+| Vendored component | `Hy-MT2-7.5B-NOESIS-NF4` (Tencent HY Community License) |
+| NOESIS role | **PRIMARY** DUB-LM translation backend (R-MT-PRIMARY-7B-NF4-FALLBACK-1.8B) |
+| Sibling FALLBACK | `Hy-MT2-1.8B-NOESIS-NF4` (1.29 GB, VRAM peak 2.59 GB) for low-VRAM clients |
+| Territory gate | `NOESIS_HUNYUAN_GEOFENCE_BLOCK=EU` (relaxed from EU+GB+KR in HY-MT1.5) |
+| Commercial gate | `NOESIS_HUNYUAN_LICENSE_TIER=community` until 100M MAU |
+| KD prohibition | Outputs NOT for training other AI models (Section 5(b)) |

NOESIS_MERGE_MANIFEST.json

@@ -0,0 +1,10 @@
+{
+  "model": "hymt2_1.8b",
+  "base": "B:\\Downloads\\Portable\\NOESIS-VC-ONE\\models\\llm\\NOESIS-3.5B-A0.5B-DUBBING-FILM\\Hy-MT2-1.8B-BF16",
+  "merged_adapters": [
+    "nt312_sft_hymt2_1.8b"
+  ],
+  "format": "BF16 merged (dense, for GGUF convert)",
+  "next": "convert_hf_to_gguf.py (b8808) -> llama-quantize Q5_K_M/Q8_0 (b9523)",
+  "created": "2026-06-06 12:18:19"
+}

README.md

@@ -0,0 +1,231 @@
+---
+library_name: transformers
+language:
+  - zh
+  - en
+  - fr
+  - pt
+  - es
+  - ja
+  - tr
+  - ru
+  - ar
+  - ko
+  - th
+  - it
+  - de
+  - vi
+  - ms
+  - id
+  - tl
+  - hi
+  - pl
+  - cs
+  - nl
+  - km
+  - my
+  - fa
+  - gu
+  - ur
+  - te
+  - mr
+  - he
+  - bn
+  - ta
+  - uk
+  - bo
+  - kk
+  - mn
+  - ug
+license: other
+license_name: tencent-hy-community
+license_link: LICENSE.md
+base_model:
+  - tencent/Hy-MT2-1.8B
+pipeline_tag: translation
+tags:
+  - translation
+  - noesis
+  - noesis-dub-lm
+  - dhcf-fno
+  - amaimedia
+  - sft
+  - qlora
+  - bf16
+  - hunyuan
+  - hy-mt2
+  - tencent
+  - translation
+  - dub-lm
+  - live-voice-translate
+inference: false
+---
+
+# NOESIS-Hy-MT2-1.8B-BF16
+
+> **NOESIS Professional Multilingual Dubbing Automation Platform**
+> framework: DHCF-FNO — *Deterministic Hybrid Control Framework for Frozen Neural Operators*
+>
+> - **Founder:** Ilia Bolotnikov
+> - **Organization:** [AMAImedia.com](https://www.amaimedia.com)
+> - **X (Twitter):** [@AMAImediacom](https://x.com/AMAImediacom)
+> - **Telegram:** [@AMAImediacom](https://t.me/AMAImediacom)
+> - **NOESIS version:** v15.10
+> - **Trained:** 2026-06-05/06
+> - **Last updated:** 2026-06-06
+
+> ⚠️ **License regime — Tencent HY Community License Agreement.** Three license gates apply: (1) Territory — EU excluded; (2) 100M MAU cap (separate Tencent license required above); (3) No-KD: outputs cannot be used to train other AI models. Full text in `LICENSE.md`.
+
+---
+
+## Role in NOESIS pipeline
+
+**Primary translation backend** for the NOESIS dubbing pipeline (Stage 3 source→target compress translate). 33 mainland + 5 ethnic + Cantonese languages.
+
+This bundle = **BF16 merged checkpoint** of `tencent/Hy-MT2-1.8B` + NOESIS SFT-LoRA
+adapter (`nt312_sft_hymt2_1.8b`) trained on the NOESIS dubbing corpus.
+Format: bf16 dense (master). Disk ≈ 3.4 GB, VRAM peak load ≈ 3.6 GB on RTX 3060.
+
+---
+
+## NOESIS A/B test results — 2026-06-06
+
+Evaluation harness: `scripts/nt318_eval_ab.py` (chat-format prompt + `<|im_end|>` stop +
+held-out 30-sample slice of `dub_subscene_MERGED.jsonl`, filtered to common target langs).
+
+| Metric | Upstream NF4 (baseline) | NOESIS SFT (this) | Δ |
+|---|---|---|---|
+| garbage% (n=30) | 20.0% | **0.0%** | ✅ -20.0 pp |
+| iso_fit (dubbing budget) | 0.390 | **0.521** | ✅ +0.131 |
+| overlap vs gold (Jaccard) | 0.048 | **0.194** | ✅ +0.146 |
+
+
+**Verdict:** ✅ no regression + measurable quality improvement on every metric.
+
+### Real example outputs
+
+| Source | Upstream NF4 | NOESIS SFT |
+|---|---|---|
+| `Го развивме филмот...` | `THE SOURCE IS NOT SURE WHAT TO DO.` | **`We'll develop the film.`** |
+| `Можам ли да ги видам?` | `Can you show me how to use it?` | **`Can I see them?`** |
+
+Old/new sample dumps: `logs_heal/ab2/{old,new}_hymt2_1.8b.json`.
+
+---
+
+## Bundle inventory
+
+| File | Description |
+|------|-------------|
+| `model.safetensors` | weights (BF16, single shard) |
+| `config.json` | model architecture configuration |
+| `generation_config.json` | default decoding params |
+| `tokenizer.json` / `tokenizer_config.json` | tokenizer |
+| `chat_template.jinja` | chat template (im_start / im_end markers) |
+| `NOESIS_MERGE_MANIFEST.json` | NOESIS provenance (base, adapter, created-at) |
+| `README.md` | this file |
+| `LICENSE.md` | NOESIS provenance + upstream license terms |
+
+---
+
+## Training details
+
+| Field | Value |
+|---|---|
+| Method | SFT (Supervised Fine-Tuning) with QLoRA |
+| Trainer | `scripts/nt312_train_sft_lora.py` |
+| LoRA rank / alpha | 16 / 32 |
+| LoRA targets | down_proj, q_proj, v_proj (lean) |
+| Max steps | 500, save every 50, --resume capable |
+| LR scheduler | warmup_stable_decay (WSD, MiniCPM-style) |
+| Optimizer | AdamW 8-bit (paged) |
+| Adapter | `LORA/nt312_sft_hymt2_1.8b/adapter` |
+| Dataset | `LORA/Hy-MT2-SFT-100k.jsonl` |
+| Sealed rule | `R-SEALED-LORA-RECIPE-V3-NF4` |
+
+---
+
+## Quick Start
+
+### Load
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+path = "NOESIS-Hy-MT2-1.8B-BF16"
+tok = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    path, torch_dtype=torch.bfloat16, device_map={"": 0},
+    trust_remote_code=True,
+)
+```
+
+### Translate (chat format — MANDATORY for correct stop)
+```python
+src_lang, tgt_lang, src = "en", "ru", "Get out of the city, now!"
+prompt = (
+    f"<|im_start|>user\n"
+    f"Переведи реплику дубляжа {src_lang}->{tgt_lang}, только перевод.\n"
+    f"SOURCE ({src_lang}): {src}<|im_end|>\n"
+    f"<|im_start|>assistant\n"
+)
+stop_ids = [tok.eos_token_id, tok.convert_tokens_to_ids("<|im_end|>")]
+ids = tok(prompt, return_tensors="pt").input_ids.cuda()
+g = model.generate(
+    ids, max_new_tokens=64, do_sample=False,
+    repetition_penalty=1.2,
+    eos_token_id=stop_ids, pad_token_id=tok.eos_token_id,
+)
+print(tok.decode(g[0][ids.shape[1]:], skip_special_tokens=True))
+# → "Уберись из города! Немедленно."
+```
+
+Convert via `convert_hf_to_gguf.py` (b8808 patched) → llama-quantize q5_k_m/q8_0 (b9523).
+
+---
+
+## NOESIS sealed rules
+
+| Rule | Summary |
+|------|---------|
+| `R-MT-PRIMARY-7B-NF4-FALLBACK-1.8B` | Hy-MT2 7.5B = primary translator; 1.8B = low-VRAM fallback |
+| `R-MT-OWN-NF4-NOT-GGUF` | Translator NF4 is OWN bnb quant, not GGUF (GGUF is a separate dense artifact) |
+| `R-SEALED-LORA-RECIPE-V3-NF4` | Canonical SFT-LoRA recipe (rank 16, lean targets, WSD) |
+| `R-HF-NAMING-QUANT-ONLY-PRESERVE-UPSTREAM` | Naming convention for trained derivatives |
+| `R-NEVER-DELETE-WITHOUT-EXPLICIT-CONSENT` | Bundle must not be deleted without explicit operator instruction |
+
+---
+
+## Upstream
+
+- **Base model:** [`tencent/Hy-MT2-1.8B`](https://huggingface.co/tencent/Hy-MT2-1.8B)
+- **License:** Tencent HY Community License Agreement — see `LICENSE.md` for the full text and NOESIS compliance notes.
+- **Training corpus:** internal NOESIS dubbing dataset (translation pairs with phoneme budgets + isochrony targets).
+
+NOESIS provenance metadata, bundle inventory, sealed-rule annotations, and DHCF-FNO
+integration notes © AMAImedia 2026 (NOESIS DHCF-FNO project).
+
+## MT benchmark — FLORES-200 devtest (2026-06-17)
+
+Real eval (not smoke): n=100 × 4 directions (eng↔rus, eng↔cmn), GPU via resident
+`llama-server -ngl 99`. Primary metric **COMET** (wmt22-comet-da, neural — how "best
+translator" is judged), plus chrF++ / BLEU / length-ratio. Each model prompted in its own
+native format (MT2 = dubbing ChatML "SOURCE (lang): … Только перевод"; 9B = ChatML + no-think).
+Data + COMET checkpoint: `D:/models/by_expert/07_MT_TRANSLATION`.
+
+| Model | Size | COMET avg | chrF++ | BLEU | gen tok/s |
+|-------|------|-----------|--------|------|-----------|
+| Qwopus3.5-9B-Translate Q4 | 5.24 GB | **0.8870** | 50.7 | 22.5 | 49 |
+| NOESIS-Hy-MT2-7.5B Q5 | 5.0 GB | 0.8709 | 46.2 | 21.4 | 52 |
+| NOESIS-Hy-MT2-1.8B Q8 | 1.78 GB | 0.8481 | 43.9 | 19.1 | 121 |
+
+Per-direction COMET — 9B-Translate wins all 4 (eng-rus .902 / eng-cmn .897 / rus-eng .872 /
+cmn-eng .877); MT2-7.5B 2nd, MT2-1.8B 3rd.
+
+**Notes:**
+- MT2 is a **dubbing** translator (isochrony): its outputs are shorter (len_ratio ~0.87-0.89
+  vs 9B ~1.0) because it compresses to fit speech slots → lower chrF on *literal* FLORES news.
+  FLORES does NOT measure MT2's slot-fit strength, so it under-rates MT2 for its actual job.
+- 1.8B→7.5B degradation: COMET +0.023, chrF +2.3, BLEU +2.3 — modest; 1.8B is 2.4× faster
+  and 2.8× smaller (good lightweight tradeoff).
+- BLEU for eng-cmn is low for all (Chinese needs char-tokenization); use chrF++/COMET there.

chat_template.jinja

@@ -0,0 +1 @@
+{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}<｜hy_begin▁of▁sentence｜>{{ system_message }}<｜hy_place▁holder▁no▁3｜>{% else %}{% set loop_messages = messages %}<｜hy_begin▁of▁sentence｜>{% endif %}{% for message in loop_messages %}{% if message['role'] == 'user' %}<｜hy_User｜>{{ message['content'] }}{% elif message['role'] == 'assistant' %}<｜hy_Assistant｜>{{ message['content'] }}<｜hy_place▁holder▁no▁2｜>{% endif %}{% endfor %}{% if add_generation_prompt %}<｜hy_Assistant｜>{% else %}<｜hy_place▁holder▁no▁8｜>{% endif %}

config.json

@@ -0,0 +1,62 @@
+{
+  "add_classification_head": false,
+  "architectures": [
+    "HunYuanDenseV1ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attention_head_dim": 128,
+  "bos_token_id": 120000,
+  "cla_share_factor": 2,
+  "class_num": 0,
+  "dense_list": [
+    2048,
+    0
+  ],
+  "dtype": "bfloat16",
+  "eod_token_id": 3,
+  "eos_token_id": 120020,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "im_end_id": 5,
+  "im_newline_id": 11,
+  "im_start_id": 4,
+  "initializer_range": 0.02,
+  "intermediate_size": 6144,
+  "mask_init_id": 12,
+  "max_position_embeddings": 262144,
+  "mlp_bias": false,
+  "model_type": "hunyuan_v1_dense",
+  "norm_type": "rms",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 4,
+  "org_vocab_size": 120818,
+  "pad_id": 120002,
+  "pad_token_id": 120002,
+  "pool_type": "last",
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "alpha": 1000.0,
+    "beta_fast": 32,
+    "beta_slow": 1,
+    "factor": 1.0,
+    "mscale": 1.0,
+    "mscale_all_dim": 1.0,
+    "rope_theta": 10000.0,
+    "rope_type": "dynamic",
+    "type": "dynamic"
+  },
+  "sep_token_id": 120007,
+  "text_end_id": 7,
+  "text_start_id": 6,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.10.0.dev0",
+  "use_cache": true,
+  "use_cla": false,
+  "use_qk_norm": true,
+  "use_rotary_pos_emb": true,
+  "vocab_size": 120818
+}

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "bos_token_id": 120000,
+  "do_sample": true,
+  "eos_token_id": 120020,
+  "pad_token_id": 120002,
+  "repetition_penalty": 1.05,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "5.10.0.dev0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17fd6d316148d7d80a54f3982555d7867642482cfe2835702e3988da2f2f8eca
+size 3582202056

tokenizer_config.json

@@ -0,0 +1,11 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<｜hy_begin▁of▁sentence｜>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<｜hy_place▁holder▁no▁2｜>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<｜hy_▁pad▁｜>",
+  "tokenizer_class": "TokenizersBackend"
+}