SinisterLlama/anlp-a1-c5

Config 5: Byte Latent Transformer (BLT) - Final 60 Epochs

This model is part of the ANLP Assignment 1: Custom Transformers & Byte Latent Transformers at IIIT Hyderabad.

Architecture

• Description: Byte Latent Transformer with entropy-based dynamic patching, local byte encoder/decoder, global latent cross-attention transformer.
• Parameters: 8.48M

Test Evaluation Results (Epoch 60)

• Bit-Level Accuracy: 98.73%
• Sequence Accuracy: 87.12%
• Average Levenshtein Distance: 0.45
• BLEU-4 Score: 96.17%
• ROUGE-1 / ROUGE-2 / ROUGE-L: 98.07% / 95.77% / 98.07%

Repository & Links

• GitHub: https://github.com/SinisterLlamma/ANLP-Assignment1
• Author: Eshaan Sharma (SinisterLlama)