---
library_name: pytorch
tags: [transformer, ablation, anlp]
---

# ANLP Assignment 1 — Architectural Ablation

Encoder-decoder transformers built from scratch for a binary-cipher →
plaintext sequence task. Five configurations differing by exactly one
component each.

| Config | Positional | Attention | Norm | Tokenization |
|---|---|---|---|---|
| C1 | Sinusoidal | MHA | LayerNorm | Subword BPE |
| C2 | **RoPE** | MHA | LayerNorm | Subword BPE |
| C3 | Sinusoidal | **GQA** | LayerNorm | Subword BPE |
| C4 | Sinusoidal | MHA | **RMSNorm** | Subword BPE |
| C5 | Sinusoidal | MHA | LayerNorm | **BLT (token-free)** |

Uploaded: C1, C2, C3, C4, C5

Trained on IIIT-H Ada (RTX 2080 Ti, 11 GB).