---
library_name: diffusers
pipeline_tag: image-to-image
base_model: neonforestmist/Clover-Image-Tiny
license: creativeml-openrail-m
tags:
  - clover-image
  - inpainting
  - stable-diffusion
  - coreml
---

# Clover Image Tiny Inpaint 🍀

Clover Image Tiny Inpaint is a compact, context-aware inpainting model for the
SD 1.4-class 512×512 architecture. Paint the region to replace in white; black
pixels are preserved. This v2 checkpoint was selected from bounded
teacher-distillation sweeps for semantic prompt following, reconstruction, and
clean mask-edge blending.

The model uses the standard nine-channel inpainting contract:

```text
[noisy latent (4), mask (1), masked-image latent (4)]
```

The text encoder, VAE, scheduler, safety checker, and tokenizer remain
compatible with Clover Image Tiny. A VAE encoder is additionally required to
prepare the masked-image latent.

## Diffusers example

```python
import torch
from diffusers import AutoPipelineForInpainting, DPMSolverMultistepScheduler
from diffusers.utils import load_image

pipe = AutoPipelineForInpainting.from_pretrained(
    "neonforestmist/Clover-Image-Tiny-Inpaint",
    torch_dtype=torch.float16,
).to("cuda")
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)

result = pipe(
    prompt="a glossy red enamel kettle resting naturally on the countertop",
    negative_prompt="blurry, distorted, low detail",
    image=load_image("source.png"),
    mask_image=load_image("mask.png"),
    num_inference_steps=20,
    guidance_scale=6.0,
    padding_mask_crop=96,
).images[0]
result.save("clover-inpaint.png")
```

Recommended interactive defaults are DPM-Solver++, 20 steps, CFG 6.0, and
`padding_mask_crop=96`. Keep runs at or below 50 steps. For production UI,
composite the generated image through the exact binary mask so that every
unmasked source pixel remains unchanged.

## Inpainting examples

All examples below use the selected v2 checkpoint, CFG 6.0, 30 evaluation
steps, and a 96-pixel context crop. White is the region regenerated by Clover.

### Context-aware object insertion

Prompt: `a tabby cat sitting naturally on the wooden park bench`

| Source | White mask | Result |
|:---:|:---:|:---:|
| ![Park bench source](examples/cat-source.png) | ![Cat edit mask](examples/cat-mask.png) | ![Cat inpaint result](examples/cat-result.png) |

Prompt: `a glossy red enamel kettle resting naturally on the countertop`

| Source | White mask | Result |
|:---:|:---:|:---:|
| ![Kitchen source](examples/kettle-source.png) | ![Kettle edit mask](examples/kettle-mask.png) | ![Kettle inpaint result](examples/kettle-result.png) |

### Irregular mask replacement

Prompt: `a bright red bicycle standing naturally on the city street`

| Source | White mask | Result |
|:---:|:---:|:---:|
| ![Street source](examples/bicycle-source.png) | ![Bicycle edit mask](examples/bicycle-mask.png) | ![Bicycle inpaint result](examples/bicycle-result.png) |

## Evaluation

The release gate used a deterministic 12-image held-out reconstruction set and
six context-rich semantic edits with ellipse, rounded-rectangle, polygon, and
brush masks.

| Metric | Previous release | v2 |
|---|---:|---:|
| Held-out masked-region MAE (lower is better) | 0.3120 | **0.2517** |
| Mean object-text CLIP similarity | 0.2726 | **0.2890** |
| Mean CLIP image similarity to the SD inpainting teacher | 0.7651 | **0.8183** |
| Black-collapse outputs | 0/6 | **0/6** |
| Changed pixels outside the mask | 0 | **0** |

Masked reconstruction MAE improved by 19.3%. Metrics are useful regression
signals, not guarantees of photorealism; the visual contact sheet and per-case
outputs were also reviewed before selection.

## Training provenance

- Warm start: `neonforestmist/Clover-Image-Tiny-Inpaint` at revision
  `1b6f8ae3db51900520369d5522c7dc7c2a97e21e`
- Teacher: `stable-diffusion-v1-5/stable-diffusion-inpainting` at revision
  `8a4288a76071f7280aedbdb3253bdb9e9d5d84bb`
- Dataset: `prithivMLmods/Caption3o-Opt` at revision
  `17e893f785fcd3f5d6fc4a5d65a914b9f7b1ff5b`
- Optimizer steps: 500, with cosine decay after warmup
- Objective: teacher noise prediction plus ground-truth denoising, Min-SNR
  weighting, and extra masked-region/boundary weighting
- Masks: brush, multi-brush, rectangle, ellipse, polygon, multi-region, and
  outpainting distributions
- Caption dropout: 0.1
- Training platform: Modal A10 in the `guccichungus69` workspace

The longer run was rejected after its held-out and semantic gates regressed;
the published checkpoint is the best-performing bounded sweep, not simply the
last checkpoint.

## Core ML

The companion SD 1.4-class Core ML resources are published at
[`neonforestmist/Clover-Image-Tiny-Inpaint-CoreML`](https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint-CoreML).
The batch-one U-Net accepts `[1, 9, 64, 64]`; classifier-free guidance is run as
two serial U-Net passes to reduce peak memory.

## LoRA compatibility

Diffusers can load a LoRA trained against this nine-channel inpainting U-Net.
Regular Clover Image Tiny LoRAs target a four-channel U-Net and are not
interchangeable. The Core ML package does not dynamically load inpainting
LoRAs; fuse an inpainting-specific adapter before conversion if needed.

## Limitations

The model can still distort small, highly structured objects, text, hands, and
faces. Very small masks may not provide enough latent resolution without the
recommended context crop. Results depend on the source, mask, prompt, seed,
and scheduler. This release inherits the limitations and license obligations
of its base and teacher models.

## Citation

```bibtex
@software{lozadaperez2026cloverimagetinyinpaint,
  author = {Lukas Lozada Perez},
  title = {Clover Image Tiny Inpaint: Compact SD 1.4-Class Image Inpainting},
  year = {2026},
  url = {https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint}
}
```