N7-T5-small-EN-MNI

It achieves the following results on the evaluation set:

• Loss: 3.3832
• Bleu: 2.1755
• Gen Len: 127.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 2000
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
5.7624	1.0	10381	5.4722	0.6341	126.9983
5.0926	2.0	20762	4.7662	1.0765	126.9999
4.6415	3.0	31143	4.2904	1.4149	127.0
4.2391	4.0	41524	3.9571	1.663	127.0
4.0196	5.0	51905	3.7264	1.8588	127.0
3.8029	6.0	62286	3.5647	2.0018	127.0
3.7119	7.0	72667	3.4628	2.0933	127.0
3.664	8.0	83048	3.4082	2.1472	127.0
3.7063	9.0	93429	3.3857	2.1719	127.0
3.6087	10.0	103810	3.3832	2.1755	127.0

EXAMPLE USAGE

  from transformers import AutoTokenizer
  from transformers import AutoModelForSeq2SeqLM
  model = 'DayanandaThokchom/N7-T5-small-EN-MNI'
  
  
  model = AutoModelForSeq2SeqLM.from_pretrained(model)
  tokenizer = AutoTokenizer.from_pretrained(model)
  
  
  
  text = "translate English to meitei mayek: what is the most important things for you"
  
  inputs = tokenizer(text, return_tensors="pt").input_ids
  # Generate text based on the input
  outputs = model.generate(
      inputs,  # The tokenized input text
      max_new_tokens=50,  # The maximum number of new tokens to generate
      do_sample=True,  # Whether to use sampling for generation
      top_k=10,  # In sampling, consider only the top_k most likely tokens
      top_p=0.50 # In sampling, consider the smallest set of tokens whose cumulative probability exceeds top_p
  )
  decoded_output = tokenizer.decode(outputs[0], skip_special_tokens=True)
  
  cleaned_output = decoded_output.split('.')[0] if '.' in decoded_output else decoded_output
  
  print(f"result:- {cleaned_output}")

Framework versions

• Transformers 4.55.0
• Pytorch 2.7.1+cu128
• Datasets 4.0.0
• Tokenizers 0.21.4