README.md

metadata

license: mit
library_name: transformers
datasets:
  - uoft-cs/cifar10
pipeline_tag: image-classification
tags:
  - computer-vision
  - cnn
  - cifar10
  - adversarial-robustness
  - downsampling
  - anti-aliasing
metrics:
  - accuracy

NULA

V.0.1.0 · Anti-Aliased Residual CNN · CIFAR-10 · Adversarial Robustness

NULA, an anti-aliased residual convolutional neural network for CIFAR-10 image classification, trained to be robust against perturbations that exploit downsampling operations.

Classical image models rely on fragile high-frequency cues, which downsampling operators destroy or alias exactly to those components.

NULA is trained to reduce this dependence and instead form representations that remain stable under information-destroying transformations such as resizing, decimations, and aliasing-style perturbation.

NULA explicitly targets robustness to operators that change the sampling structure of the input, rather than generic adversarial perturbations.

Problem

Downsampling operations are linear maps from a high-dimensional space to a lower-dimensional one.

By the Rank-Nullity theorem, this matrix has a massive NULL space.

An attacker can exploit this: they utilize the discarded samples of these downsampling operations as extra degrees of freedom.

By sculpting perturbations with components in the null space of the downsampling operator, they spread energy across frequencies that are discarded during striding.

The result is an image perceptually identical to the original, with a manipulated activation pattern.

Approach

Anti-aliased Downsampling (BlurPool)

Standard strided convolutions perform subsampling without enforcing a band-limit, causing aliasing.

BlurPool2d introduces a low-pass filter before subsampling:

• x -> (low-pass filter) -> subsample

The filter is a normalized binomial kernel [1, 2, 1] ⊗ [1, 2, 1], applied depthwise: one filter per channel, no cross-channel mixing.

This enforces approximate band-limitedness prior to resolution reduction, reducing aliasing artifacts and making feature extraction more stable under downsampling.

Squeeze-and-Excitation (SE) blocks

SE blocks perform channel-wise reweighting.

• s = σ(W₂ δ(W₁ GAP(x)))

• x → s ⊙ x

The bottleneck dimension is max(C // r, 1) where r = 16, keeping the recalibration lightweight relative to the feature dimension.

The network learns to suppress channels that carry unstable high-frequency information and amplify channels that carry structurally stable features.

FIRST EVALUATION (Base)

The first evaluation was trained for clean classification only, without adversarial augmentation.

Perturbation	Accuracy	Drop from clean
Clean	91.95%	—
Resize ×0.5 (bilinear)	59.83%	−32.12%
Resize ×0.25 (bilinear)	24.82%	−67.13%
Decimate ×2	30.03%	−61.92%
Checkerboard ( \varepsilon = 0.03 )	75.47%	−16.48%
Checkerboard ( \varepsilon = 0.05 )	44.99%	−46.96%

This is a catastrophic collapse of model performance under representation instability. The representation inside the network is highly sensitive to aliasing and not stable under non-invertible transformations.

• for checkerboard perturbations: ∃δ s.t. ||δ||∞ ≤ ε, but argmax f(x + δ) ≠ argmax f(x)

• for resize/decimate: the transformation preserves class identity while destroying high-frequency structure the model fails because its representations are not invariant to the loss of this structure

SECOND EVALUATION (Robust)

The second evaluation of NULA was retrained from scratch under a modified training distribution.

During training, images were stochastically exposed to resolution-degrading transformations such as:

• resize-down/up
• hard decimation
• anti-aliased blur-decimation

The training objective becomes:

• min_θ E_{ (x, y) ~ D, T ~ 𝓣 } [ L(f_θ(T(x)), y) ]

where T is a distribution over resolution-degrading operators

The model is forced to learn representations that remain predictive under transformations that destroy or corrupt high-frequency information.

Perturbation	Accuracy	Change vs. baseline
Clean	89.42%	−2.53%
Resize ×0.5	85.37%	+25.54%
Resize ×0.25	71.80%	+46.98%
Decimate ×2	85.02%	+54.99%
Checkerboard ( \varepsilon = 0.03 )	89.43%	+13.96%
Checkerboard ( \varepsilon = 0.05 )	89.39%	+44.40%

• For augmentation functions, see augmentations.py
• For the adversarial training loop, see train_robust.py

Interpretation

The baseline model relies on high-frequency components that are not stable under downsampling or aliasing. These components lie in regions of the input space that are not preserved by common sampling operators.

As a result, small perturbations aligned with these unstable directions cause large changes in the model’s activations.

The robust variant shifts reliance toward features that occupy the range of the downsampling operator — the subspace that survives projection.

These features encode structural information at scales that are preserved under frequency loss, rather than fine-grained detail that is annihilated by the null space.

The result is a model whose decision boundary is anchored to geometry that persists through information destruction.

Usage

Nula is hosted on the HuggingFace Hub and can be loaded directly via the transformers library.

import torch
from transformers import AutoModelForImageClassification

model = AutoModelForImageClassification.from_pretrained(
    "MamaPearl/nula-cifar10-robust-v0", 
    trust_remote_code=True
)
model.eval()

# prepare an input (CIFAR-10 expected size: 32x32)
# NOTE: real images should be normalized to mean=0.5, std=0.5 for best results
image = torch.randn(1, 3, 32, 32) 

with torch.no_grad():
    output = model(pixel_values=image)
logits = output.logits
predicted_class = logits.argmax(dim=-1).item()

print(f"Predicted Class ID: {predicted_class}")
print(f"Label: {model.config.id2label[predicted_class]}")

Input tensors should be shape (B, C, H, W).

Architecture

Component	Details
Stem	3 → 128, Conv3×3, BatchNorm, SiLU
Stage 1	128 → 128, residual, no downsample
Stage 2	128 → 256, residual, BlurPool downsample
Stage 3	256 → 512, residual, BlurPool downsample
Head	GlobalAvgPool → Linear(512, 512) → SiLU → Dropout(0.3) → Linear(512, 10)

SE blocks applied at each stage with reduction factor 16.

Citation

If you use this model or repository in your research, please cite:

@misc{mamapearl_nula_2026,
  author       = {MamaPearl},
  title        = {NULA: Robust CIFAR-10 Classification via Anti-Aliased Downsampling and Adversarial Augmentation},
  month        = apr,
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/MamaPearl/nula-cifar10-robust-v0}},
}

Authors

MamaPearl · @mamapearli

License

This project is licensed under the MIT License. See LICENSE for more information.