README.md

---
license: mit
library_name: transformers
datasets:
- uoft-cs/cifar10
pipeline_tag: image-classification
tags:
- computer-vision
- cnn
- cifar10
- adversarial-robustness
- downsampling
- anti-aliasing
metrics:
- accuracy
---

## NULA 

NULA, an anti-aliased residual convolutional neural network for CIFAR-10 image classification, trained to be robust against perturbations that exploit downsampling operations.

Classical image models rely on fragile high-frequency cues, which downsampling operators destroy or alias exactly to those components.

NULA is trained to reduce this dependence and instead form representations that remain stable under information-destroying transformations such as resizing, decimations, and aliasing-style perturbation.


# Problem

Downsampling operations are linear maps from a high-dimensional space to a lower-dimensional one.

By the Rank-Nullity theorem, this matrix has a massive NULL space.

An attacker can exploit this: they utilize the discarded samples of these downsampling operations as extra degrees of freedom. 

By sculpting perturbations with components in the null space of the downsampling operator, they spread energy across frequencies that are discarded during striding.

The result is an image perceptually identical to the original, with a manipulated activation pattern.

## Approach

[BlurPool: what it does mechanically and why it addresses the problem]
[SE blocks: what they add]
- For augmentation functions, see [`augmentations.py`](augmentations.py)
- For the adversarial training loop, see [`train_robust.py`](train_robust.py)


### FIRST EVALUATION (Base)

The first evaluation was trained for clean classification performance without the robust training procedure described above.

| Perturbation | Accuracy | Drop from clean |
|---|---:|---:|
| Clean | 91.95% | — |
| Resize ×0.5 (bilinear) | 59.83% | −32.12% |
| Resize ×0.25 (bilinear) | 24.82% | −67.13% |
| Decimate ×2 | 30.03% | −61.92% |
| Checkerboard \( \varepsilon = 0.03 \) | 75.47% | −16.48% |
| Checkerboard \( \varepsilon = 0.05 \) | 44.99% | −46.96% |

This model achieves high clean accuracy, but its performance collapses under resolution loss and aliasing-sensitive perturbations. In other words, it is accurate but fragile.

### SECOND EVALUATION (Robust)

The second evaluation of NULA was trained from scratch under a modified training distribution. 

During training, image were stochastically exposed to resolution-degrading transformations such as:

- resize-down/up
- hard decimation
- anti-aliased blur-decimation

The objective becomes invariance learning.
The model is forced to reduce sensitivity to unstable fine-scale structure and form representations that survive frequency loss and sampling artifacts.

| Perturbation | Accuracy | Change vs. baseline |
|---|---:|---:|
| Clean | 89.42% | −2.53% |
| Resize ×0.5 | 85.37% | +25.54% |
| Resize ×0.25 | 71.80% | +46.98% |
| Decimate ×2 | 85.02% | +54.99% |
| Checkerboard \( \varepsilon = 0.03 \) | 89.43% | +13.96% |
| Checkerboard \( \varepsilon = 0.05 \) | 89.39% | +44.40% |

The robust variant trades 2.53 percentage points of clean accuracy for a substantial increase in robustness across all tested perturbations.

Most notably, under the checkerboard perturbation with \( \varepsilon = 0.05 \), performance improves from 44.99% to 89.39%, which is nearly clean-level behavior.

## Interpretation

The baseline model seemingly relies on brittle high-frequency cues. When these cues are removed, aliased, or perturbed, classification performance degrades sharply.

The robust variant instead learns representations that are much less sensitive to these fine-scale disturbances. It does not become perfectly invariant to severe information destruction, but it shifts the model toward more stable, lower-frequency, structurally meaningful features.

This is the central result of NULA: not merely high clean accuracy, but a substantially better tradeoff between accuracy and robustness under downsampling-related perturbations.

## Usage

Nula is hosted on the HuggingFace Hub and can be loaded directly via the transformers library. 
```python
import torch
from transformers import AutoModelForImageClassification

model = AutoModelForImageClassification.from_pretrained(
    "MamaPearl/nula-cifar10-robust-v0", 
    trust_remote_code=True
)
model.eval()

# prepare an input (CIFAR-10 expected size: 32x32)
# NOTE: real images should be normalized to mean=0.5, std=0.5 for best results
image = torch.randn(1, 3, 32, 32) 

with torch.no_grad():
    output = model(pixel_values=image)
logits = output.logits
predicted_class = logits.argmax(dim=-1).item()

print(f"Predicted Class ID: {predicted_class}")
print(f"Label: {model.config.id2label[predicted_class]}")
```
Input tensors should be shape (B, C, H, W).

## Architecture
```
stem → s1 → s2 → s3 → global pool → head
```

| Component | Details |
|---|---|
| Stem | 3 → 128, Conv3×3, BatchNorm, SiLU |
| Stage 1 | 128 → 128, residual, no downsample |
| Stage 2 | 128 → 256, residual, BlurPool downsample |
| Stage 3 | 256 → 512, residual, BlurPool downsample |
| Head | GlobalAvgPool → Linear(512, 512) → SiLU → Dropout(0.3) → Linear(512, 10) |

SE blocks applied at each stage with reduction factor 16.

## Citation

If you use this model or repository in your research, please cite:

```bibtex
@misc{mamapearl_nula_2026,
  author = {MamaPearl},
  title = {NULA: Anti-Aliased SE-CNN for CIFAR-10},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/MamaPearl/nula-base-cifar10-v0}
}
```

# Authors

* **MamaPearl** ([@MamaPearl](https://github.com))  
  *Main Contributor* 

## Contact & Socials
| Platform | Link |
| :--- | :--- |
| **GitHub** | [github.com/MamaPearl](https://github.com) |
| **Instagram** | [@mamapearli](https://www.instagram.com/mamapearli/) | 
## License

This project is licensed under the MIT License. See [LICENSE](LICENSE) for more information.