nos-mt-es-arg-onnx

ONNX build of proxectonos/es-arg, translating Spanish (es) to Aragonese (an).

The original is published only as a CTranslate2 binary, which runs only inside CTranslate2. This repository holds a HuggingFace PegasusForConditionalGeneration checkpoint and an ONNX export that were reconstructed from that binary. Nothing was retrained. The weights are the original weights.

Credit for the model belongs to Proxecto Nós (Universidade de Santiago de Compostela). Licence mit, the same as the source.

Contents

Path	What it is
model.safetensors, config.json	PyTorch PegasusForConditionalGeneration
encoder_model.onnx, decoder_model.onnx, decoder_with_past_model.onnx	ONNX, float32
int8/	the same three graphs, dynamic int8
source.bpe	the subword-nmt BPE codes for the source language
nos_vocab.json	source vocabulary, target vocabulary, and the id offset
nos_tokenizer.py	the tokenizer, reproducing the published pipeline
ct2_reader.py, ct2_to_pegasus_dual.py	the converter

Use

This model has no sentencepiece tokenizer. Its published pipeline is Moses tokenization, then subword-nmt BPE with the @@ continuation marker. Install the two helpers and use the tokenizer shipped here:

pip install optimum[onnxruntime] sacremoses subword-nmt

from huggingface_hub import hf_hub_download
from optimum.onnxruntime import ORTModelForSeq2SeqLM
import importlib.util, sys

path = hf_hub_download("TigreGotico/nos-mt-es-arg-onnx", "nos_tokenizer.py")
spec = importlib.util.spec_from_file_location("nos_tokenizer", path)
mod = importlib.util.module_from_spec(spec); spec.loader.exec_module(mod)

tok = mod.NosTokenizer.from_pretrained("TigreGotico/nos-mt-es-arg-onnx", src_lang="es", tgt_lang="an")
model = ORTModelForSeq2SeqLM.from_pretrained("TigreGotico/nos-mt-es-arg-onnx", use_cache=True, use_merged=False)

ids = tok('El gato duerme en el sofá.', return_tensors="pt")
out = model.generate(**ids, num_beams=4, max_new_tokens=256)
print(tok.decode(out[0]))
# Lo <unk> <unk> me en o *sofá.

For the int8 build add subfolder="int8".

Preprocessing

Read this before you replace the tokenizer.

• The source text gets no end-of-sentence token. The CT2 binary ships add_source_eos: false, so the encoder never saw </s>.
• Source and target have separate vocabularies. Pegasus has one, so the single table is [target vocabulary | source vocabulary] and every encoder input id is offset by 14904 (the target vocabulary size). The source half is suppressed at decode time with final_logits_bias = -1e9, so the decoder can never emit a source-side id.
• Words must be Moses-tokenized and BPE-applied first. The output is joined, @@ is removed, and the result is Moses-detokenized.
• Decoding starts from <s>.
• Both vocabularies are frequency-filtered, so rare words come back as <unk>. CTranslate2 prints <unk> as well. The upstream translate.py hides it with replace_unknowns=True, which copies the aligned source word; that needs attention alignments, which generate does not expose. nos_tokenizer.py therefore keeps <unk> in the output, so what you read is what the model produced.
• Feed one sentence at a time. The model has no document context and no language tag.

Parity with the original

15 source sentences, greedy and beam 4, exact string match against ctranslate2.Translator running the source model.bin:

Comparison	greedy	beam 4
reconstructed PyTorch	100%	100%
ONNX float32	100%	100%
ONNX int8	100%	100%

Any remaining string difference is a beam-search tie, not a weight error.

Sample output

Spanish	Aragonese
El gato duerme en el sofá.	Lo me en o *sofá.
Mañana iremos a la playa si hace buen tiempo.	Manyana iremos a la placha si fa buen tiempo.
La reunión se ha aplazado hasta el próximo lunes por la tarde.	La reunión s'ha *aplazado dica lo proximo luns per la tarde.
No entiendo por qué siempre llegas tarde a clase.	No entiendo per qué siempre plegas tarde a clase.
El gobierno aprobó una nueva ley sobre el cambio climático.	Lo gubierno aprebó una nueva lei sobre lo cambio climatico.
¿Podrías decirme dónde está la estación de tren más cercana?	Podrías decir-me án ye la estación de tren mas cercana?
Los niños jugaban en el parque mientras sus padres charlaban.	Los ninos chugaban en o parque mientres los suyos pais charraban.
Este restaurante sirve la mejor paella de toda la ciudad.	Este restaurant sirve la millor paella de tota la ciudat.

How the reconstruction works

A CTranslate2 model.bin is a flat self-describing binary: binary_version, the spec name and revision, then one record per variable (name, rank, dimensions, dtype code, byte count, raw bytes), then a table of aliases for tied weights. ct2_reader.py reads it. ct2_to_pegasus_dual.py recovers the architecture from the spec scalars and maps every variable onto a HuggingFace parameter.

This model reports:

{
  "encoder_layers": 12,
  "decoder_layers": 12,
  "source_vocab_size": 14728,
  "target_vocab_size": 14904,
  "d_model": 512,
  "heads": 16,
  "ffn_dim": 2048,
  "pre_norm": true,
  "activation": "relu",
  "layernorm_embedding": false,
  "relative_position": false,
  "scale_embeddings": true,
  "output_bias": true,
  "stored_positions": true,
  "attention_bias": false,
  "ct2_spec": "TransformerSpec rev 7, binary_version 6",
  "source_eos": false,
  "source_bos": false,
  "decoder_start_token": "<s>"
}

Why Pegasus

1. Pre-norm blocks with a final encoder and decoder layer norm, and no layernorm_embedding. That rules out BART, mBART and PLBart, whose layernorm_embedding cannot be neutralised — a LayerNorm with weight 1 and bias 0 still normalises. It also rules out Marian, which is post-norm.
2. An output bias (decoder/projection/bias). Neither Marian nor M2M100 has one. Pegasus does, as final_logits_bias.

Traps

• Pegasus refuses to save its position table. embed_positions.weight is in _keys_to_ignore_on_save and is rebuilt on load with 10000^(2i/dim). OpenNMT-py interleaves sin and cos instead. This binary does store the real table, so the converter writes it in, clears _keys_to_ignore_on_save, and reloads the checkpoint to assert the table survived. Skipping this produces a model that runs and translates plausibly but wrongly.
• CTranslate2 fuses self-attention Q, K and V into one linear_0 of shape (3d, d) in that order. Cross-attention splits differently: linear_0 is Q alone, linear_1 is [K; V] fused, linear_2 is the output projection.
• These models were trained with add_qkvbias=False, so the attention and feed-forward projections carry no bias. Zeros are written where HuggingFace insists on one.
• Weights are stored (out, in), the layout torch.nn.Linear uses, so nothing is transposed. gamma and beta are the layer-norm weight and bias.
• int8 quantization is restricted to MatMul with /lm_head/MatMul excluded. Quantizing every operator destroys a 512-dimension NMT decoder.

Attribution

Model and training data: Proxecto Nós, licence mit. Source repository: proxectonos/es-arg. The model was built for the paper Training and fine-tuning NMT models for low-resource languages using Apertium-based synthetic corpora (Sant et al., 2023), within the Nós Project funded by the Ministerio para la Transformación Digital y de la Función Pública and the EU NextGenerationEU programme (ILENIA, 2022/TL22/00215336).

This repository only changes the file format.