metadata
library_name: pytorch
tags:
- transformer
- ablation
- anlp
ANLP Assignment 1 — Architectural Ablation
Encoder-decoder transformers built from scratch for a binary-cipher → plaintext sequence task. Five configurations differing by exactly one component each.
| Config | Positional | Attention | Norm | Tokenization |
|---|---|---|---|---|
| C1 | Sinusoidal | MHA | LayerNorm | Subword BPE |
| C2 | RoPE | MHA | LayerNorm | Subword BPE |
| C3 | Sinusoidal | GQA | LayerNorm | Subword BPE |
| C4 | Sinusoidal | MHA | RMSNorm | Subword BPE |
| C5 | Sinusoidal | MHA | LayerNorm | BLT (token-free) |
Uploaded: C1, C2, C3, C4, C5
Trained on IIIT-H Ada (RTX 2080 Ti, 11 GB).