README.md

---
license: mit
library_name: transformers
datasets:
- uoft-cs/cifar10
pipeline_tag: image-classification
tags:
- computer-vision
- cnn
- cifar10
- adversarial-robustness
- downsampling
- anti-aliasing
metrics:
- accuracy
---

<div align="center">

# NULA

**V.0.1.0 · Anti-Aliased Residual CNN · CIFAR-10 · Adversarial Robustness**

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![HuggingFace](https://img.shields.io/badge/🤗-MamaPearl%2FNULA-blue)](https://huggingface.co/MamaPearl/nula-cifar10-robust-v0)

</div>

NULA, an anti-aliased residual convolutional neural network for CIFAR-10 image classification, trained to be robust against perturbations that exploit downsampling operations.

Classical image models rely on fragile high-frequency cues, which downsampling operators destroy or alias exactly to those components.

NULA is trained to reduce this dependence and instead form representations that remain stable under information-destroying transformations such as resizing, decimations, and aliasing-style perturbation.

NULA explicitly targets robustness to operators that change the sampling structure of the input, rather than generic adversarial perturbations.

## Problem

Downsampling operations are linear maps from a high-dimensional space to a lower-dimensional one.

By the Rank-Nullity theorem, this matrix has a massive NULL space.

An attacker can exploit this: they utilize the discarded samples of these downsampling operations as extra degrees of freedom. 

By sculpting perturbations with components in the null space of the downsampling operator, they spread energy across frequencies that are discarded during striding.

The result is an image perceptually identical to the original, with a manipulated activation pattern.

![perturbation examples](https://cdn-uploads.huggingface.co/production/uploads/6921e98c32171f09bfa0329a/ALG2BPCYeXIxeTjYfcIsl.png)

## Approach

### Anti-aliased Downsampling (BlurPool)

Standard strided convolutions perform subsampling without enforcing a band-limit, causing aliasing.

BlurPool2d introduces a low-pass filter before subsampling:

  - x -> (low-pass filter) -> subsample

The filter is a normalized binomial kernel [1, 2, 1] ⊗ [1, 2, 1], applied depthwise: one filter per channel, no cross-channel mixing.

This enforces approximate band-limitedness prior to resolution reduction, reducing aliasing artifacts and making feature extraction more stable under downsampling.

### Squeeze-and-Excitation (SE) blocks

SE blocks perform channel-wise reweighting. 

  - s = σ(W₂ δ(W₁ GAP(x)))

  - x → s ⊙ x

The bottleneck dimension is max(C // r, 1) where r = 16, keeping the recalibration 
lightweight relative to the feature dimension.

The network learns to suppress channels that carry unstable high-frequency information 
and amplify channels that carry structurally stable features.

## FIRST EVALUATION (Base)

The first evaluation was trained for clean classification only, without adversarial augmentation.

| Perturbation | Accuracy | Drop from clean |
|---|---:|---:|
| Clean | 91.95% | — |
| Resize ×0.5 (bilinear) | 59.83% | −32.12% |
| Resize ×0.25 (bilinear) | 24.82% | −67.13% |
| Decimate ×2 | 30.03% | −61.92% |
| Checkerboard \( \varepsilon = 0.03 \) | 75.47% | −16.48% |
| Checkerboard \( \varepsilon = 0.05 \) | 44.99% | −46.96% |

This is a catastrophic collapse of model performance under representation instability.
The representation inside the network is highly sensitive to aliasing and not stable under non-invertible transformations.
  
  - for checkerboard perturbations: ∃δ s.t. ||δ||∞ ≤ ε, but argmax f(x + δ) ≠ argmax f(x)

  - for resize/decimate: the transformation preserves class identity while destroying high-frequency structure
    the model fails because its representations are not 
    invariant to the loss of this structure


## SECOND EVALUATION (Robust)

The second evaluation of NULA was retrained from scratch under a modified training distribution. 

During training, images were stochastically exposed to resolution-degrading transformations such as:

- resize-down/up
- hard decimation
- anti-aliased blur-decimation

The training objective becomes:

  - min_θ E_{ (x, y) ~ D, T ~ 𝓣 } [ L(f_θ(T(x)), y) ]

where T is a distribution over resolution-degrading operators
  
The model is forced to learn representations that remain predictive under transformations that destroy or corrupt high-frequency information.

| Perturbation | Accuracy | Change vs. baseline |
|---|---:|---:|
| Clean | 89.42% | −2.53% |
| Resize ×0.5 | 85.37% | +25.54% |
| Resize ×0.25 | 71.80% | +46.98% |
| Decimate ×2 | 85.02% | +54.99% |
| Checkerboard \( \varepsilon = 0.03 \) | 89.43% | +13.96% |
| Checkerboard \( \varepsilon = 0.05 \) | 89.39% | +44.40% |

- For augmentation functions, see [`augmentations.py`](augmentations.py)
- For the adversarial training loop, see [`train_robust.py`](train_robust.py)

## Interpretation

The baseline model relies on high-frequency components that are not stable under downsampling or aliasing. 
These components lie in regions of the input space that are not preserved by common sampling operators.

As a result, small perturbations aligned with these unstable directions cause large changes in the model’s activations.

The robust variant shifts reliance toward features that occupy the range of the downsampling operator — the subspace that survives projection. 

These features encode structural information at scales that are preserved under frequency loss, rather than fine-grained detail that is 
annihilated by the null space. 

The result is a model whose decision boundary is anchored to 
geometry that persists through information destruction.

## Usage

Nula is hosted on the HuggingFace Hub and can be loaded directly via the transformers library. 
```python
import torch
from transformers import AutoModelForImageClassification

model = AutoModelForImageClassification.from_pretrained(
    "MamaPearl/nula-cifar10-robust-v0", 
    trust_remote_code=True
)
model.eval()

# prepare an input (CIFAR-10 expected size: 32x32)
# NOTE: real images should be normalized to mean=0.5, std=0.5 for best results
image = torch.randn(1, 3, 32, 32) 

with torch.no_grad():
    output = model(pixel_values=image)
logits = output.logits
predicted_class = logits.argmax(dim=-1).item()

print(f"Predicted Class ID: {predicted_class}")
print(f"Label: {model.config.id2label[predicted_class]}")
```
Input tensors should be shape (B, C, H, W).

## Architecture

| Component | Details |
|---|---|
| Stem | 3 → 128, Conv3×3, BatchNorm, SiLU |
| Stage 1 | 128 → 128, residual, no downsample |
| Stage 2 | 128 → 256, residual, BlurPool downsample |
| Stage 3 | 256 → 512, residual, BlurPool downsample |
| Head | GlobalAvgPool → Linear(512, 512) → SiLU → Dropout(0.3) → Linear(512, 10) |

SE blocks applied at each stage with reduction factor 16.

## Citation

If you use this model or repository in your research, please cite:

```bibtex
@misc{mamapearl_nula_2026,
  author       = {MamaPearl},
  title        = {NULA: Robust CIFAR-10 Classification via Anti-Aliased Downsampling and Adversarial Augmentation},
  month        = apr,
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/MamaPearl/nula-cifar10-robust-v0}},
}
```

## Authors

**MamaPearl** · [@mamapearli](https://www.instagram.com/mamapearli/)

## License

This project is licensed under the MIT License. See [LICENSE](LICENSE) for more information.