---
license: mit
language:
- es
- an
library_name: transformers
pipeline_tag: translation
tags:
- translation
- onnx
- ctranslate2
- opennmt
base_model: proxectonos/es-arg
base_model_relation: quantized
---

# nos-mt-es-arg-onnx

ONNX build of [`proxectonos/es-arg`](https://huggingface.co/proxectonos/es-arg), translating
**Spanish (es) to Aragonese (an)**.

The original is published only as a CTranslate2 binary, which runs only inside
CTranslate2. This repository holds a HuggingFace `PegasusForConditionalGeneration`
checkpoint and an ONNX export that were **reconstructed from that binary**.
Nothing was retrained. The weights are the original weights.

Credit for the model belongs to [Proxecto Nós](https://nos.gal/) (Universidade
de Santiago de Compostela). Licence `mit`, the same as the source.

## Contents

| Path | What it is |
|---|---|
| `model.safetensors`, `config.json` | PyTorch `PegasusForConditionalGeneration` |
| `encoder_model.onnx`, `decoder_model.onnx`, `decoder_with_past_model.onnx` | ONNX, float32 |
| `int8/` | the same three graphs, dynamic int8 |
| `source.bpe` | the subword-nmt BPE codes for the source language |
| `nos_vocab.json` | source vocabulary, target vocabulary, and the id offset |
| `nos_tokenizer.py` | the tokenizer, reproducing the published pipeline |
| `ct2_reader.py`, `ct2_to_pegasus_dual.py` | the converter |

## Use

This model has **no sentencepiece tokenizer**. Its published pipeline is Moses
tokenization, then subword-nmt BPE with the `@@` continuation marker. Install
the two helpers and use the tokenizer shipped here:

```bash
pip install optimum[onnxruntime] sacremoses subword-nmt
```

```python
from huggingface_hub import hf_hub_download
from optimum.onnxruntime import ORTModelForSeq2SeqLM
import importlib.util, sys

path = hf_hub_download("TigreGotico/nos-mt-es-arg-onnx", "nos_tokenizer.py")
spec = importlib.util.spec_from_file_location("nos_tokenizer", path)
mod = importlib.util.module_from_spec(spec); spec.loader.exec_module(mod)

tok = mod.NosTokenizer.from_pretrained("TigreGotico/nos-mt-es-arg-onnx", src_lang="es", tgt_lang="an")
model = ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/nos-mt-es-arg-onnx", use_cache=True, use_merged=False)

ids = tok('El gato duerme en el sofá.', return_tensors="pt")
out = model.generate(**ids, num_beams=4, max_new_tokens=256)
print(tok.decode(out[0]))
# Lo <unk> <unk> me en o *sofá.
```

For the int8 build add `subfolder="int8"`.

## Preprocessing

Read this before you replace the tokenizer.

- The source text gets **no end-of-sentence token**. The CT2 binary ships
  `add_source_eos: false`, so the encoder never saw `</s>`.
- Source and target have **separate vocabularies**. Pegasus has one, so the
  single table is `[target vocabulary | source vocabulary]` and every encoder
  input id is offset by `14904` (the target vocabulary size). The source half
  is suppressed at decode time with `final_logits_bias = -1e9`, so the decoder
  can never emit a source-side id.
- Words must be Moses-tokenized and BPE-applied first. The output is joined,
  `@@ ` is removed, and the result is Moses-detokenized.
- Decoding starts from `<s>`.
- Both vocabularies are frequency-filtered, so rare words come back as `<unk>`.
  CTranslate2 prints `<unk>` as well. The upstream `translate.py` hides it with
  `replace_unknowns=True`, which copies the aligned source word; that needs
  attention alignments, which `generate` does not expose. `nos_tokenizer.py`
  therefore keeps `<unk>` in the output, so what you read is what the model
  produced.
- Feed one sentence at a time. The model has no document context and no
  language tag.

## Parity with the original

15 source sentences, greedy and beam 4, **exact string match** against
`ctranslate2.Translator` running the source `model.bin`:

| Comparison | greedy | beam 4 |
|---|---|---|
| reconstructed PyTorch | 100% | 100% |
| ONNX float32 | 100% | 100% |
| ONNX int8 | 100% | 100% |

Any remaining string difference is a beam-search tie, not a weight error.

## Sample output

| Spanish | Aragonese |
|---|---|
| El gato duerme en el sofá. | Lo <unk> <unk> me en o *sofá. |
| Mañana iremos a la playa si hace buen tiempo. | Manyana iremos a la placha si fa buen tiempo. |
| La reunión se ha aplazado hasta el próximo lunes por la tarde. | La reunión s'ha *aplazado dica lo proximo luns per la tarde. |
| No entiendo por qué siempre llegas tarde a clase. | No entiendo per qué siempre plegas tarde a clase. |
| El gobierno aprobó una nueva ley sobre el cambio climático. | Lo gubierno aprebó una nueva lei sobre lo cambio climatico. |
| ¿Podrías decirme dónde está la estación de tren más cercana? | <unk> Podrías decir-me án ye la estación de tren mas cercana? |
| Los niños jugaban en el parque mientras sus padres charlaban. | Los ninos chugaban en o parque mientres los suyos pais charraban. |
| Este restaurante sirve la mejor paella de toda la ciudad. | Este restaurant sirve la millor paella de tota la ciudat. |

## How the reconstruction works

A CTranslate2 `model.bin` is a flat self-describing binary: `binary_version`,
the spec name and revision, then one record per variable (name, rank,
dimensions, dtype code, byte count, raw bytes), then a table of aliases for tied
weights. `ct2_reader.py` reads it. `ct2_to_pegasus_dual.py` recovers the
architecture from the spec scalars and maps every variable onto a HuggingFace
parameter.

This model reports:

```json
{
  "encoder_layers": 12,
  "decoder_layers": 12,
  "source_vocab_size": 14728,
  "target_vocab_size": 14904,
  "d_model": 512,
  "heads": 16,
  "ffn_dim": 2048,
  "pre_norm": true,
  "activation": "relu",
  "layernorm_embedding": false,
  "relative_position": false,
  "scale_embeddings": true,
  "output_bias": true,
  "stored_positions": true,
  "attention_bias": false,
  "ct2_spec": "TransformerSpec rev 7, binary_version 6",
  "source_eos": false,
  "source_bos": false,
  "decoder_start_token": "<s>"
}
```

### Why Pegasus

1. **Pre-norm blocks with a final encoder and decoder layer norm**, and no
   `layernorm_embedding`. That rules out BART, mBART and PLBart, whose
   `layernorm_embedding` cannot be neutralised — a LayerNorm with weight 1 and
   bias 0 still normalises. It also rules out Marian, which is post-norm.
2. **An output bias** (`decoder/projection/bias`). Neither Marian nor M2M100 has
   one. Pegasus does, as `final_logits_bias`.

### Traps

- **Pegasus refuses to save its position table.** `embed_positions.weight` is in
  `_keys_to_ignore_on_save` and is rebuilt on load with `10000^(2i/dim)`.
  OpenNMT-py interleaves sin and cos instead. This binary *does* store the real
  table, so the converter writes it in, clears `_keys_to_ignore_on_save`, and
  reloads the checkpoint to assert the table survived. Skipping this produces a
  model that runs and translates plausibly but wrongly.
- CTranslate2 fuses self-attention Q, K and V into one `linear_0` of shape
  `(3d, d)` in that order. Cross-attention splits differently: `linear_0` is Q
  alone, `linear_1` is `[K; V]` fused, `linear_2` is the output projection.
- These models were trained with `add_qkvbias=False`, so the attention and
  feed-forward projections carry no bias. Zeros are written where HuggingFace
  insists on one.
- Weights are stored `(out, in)`, the layout `torch.nn.Linear` uses, so nothing
  is transposed. `gamma` and `beta` are the layer-norm weight and bias.
- int8 quantization is restricted to `MatMul` with `/lm_head/MatMul` excluded.
  Quantizing every operator destroys a 512-dimension NMT decoder.

## Attribution

Model and training data: [Proxecto Nós](https://nos.gal/), licence `mit`.
Source repository: [`proxectonos/es-arg`](https://huggingface.co/proxectonos/es-arg).
The model was built for the paper *Training and fine-tuning NMT models for
low-resource languages using Apertium-based synthetic corpora* (Sant et al.,
2023), within the Nós Project funded by the Ministerio para la Transformación
Digital y de la Función Pública and the EU NextGenerationEU programme
(ILENIA, 2022/TL22/00215336).

This repository only changes the file format.