library_name: diffusers
pipeline_tag: image-to-image
base_model: neonforestmist/Clover-Image-Tiny
license: creativeml-openrail-m
tags:
- clover-image
- inpainting
- stable-diffusion
- coreml
Clover Image Tiny Inpaint 🍀
An inpainting adaptation of Clover Image Tiny for 512×512 local generation and on-device Core ML deployment. White mask pixels are regenerated; black pixels are preserved. The intended target is an SD 1.4-class architecture, rather than a device-specific model.
The model uses a 9-channel U-Net input:
[noisy latent (4), mask (1), masked-image latent (4)]
The base text encoder, VAE, scheduler, safety checker, and tokenizer remain compatible with Clover Image Tiny. The inpainting export additionally includes the VAE encoder needed to prepare the masked-image latent on device.
Diffusers example
from diffusers import AutoPipelineForInpainting, DPMSolverMultistepScheduler
from diffusers.utils import load_image
pipe = AutoPipelineForInpainting.from_pretrained(
"neonforestmist/Clover-Image-Tiny-Inpaint",
torch_dtype="auto",
)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
image = pipe(
prompt="a tiny glass greenhouse glowing in a moonlit garden",
image=load_image("input.png"),
mask_image=load_image("mask.png"),
num_inference_steps=20,
guidance_scale=6.0,
padding_mask_crop=96,
).images[0]
image.save("clover-inpaint.png")
The mask is a grayscale image: white means “regenerate” and black means “preserve.” A few useful inpainting prompts are:
replace the masked area with a tiny glass greenhouse glowing at nightremove the person from the masked area and continue the background naturallyadd a red enamel kettle on the masked countertop
Use DPM-Solver++, 20 steps, CFG 6.0, and a 96-pixel context margin as the interactive defaults. Exact-mask compositing preserves every source pixel outside the edit. The selected v2 checkpoint reduced deterministic held-out masked MAE from 0.3120 to 0.2517 and improved mean semantic CLIP alignment from 0.2726 to 0.2890 over the previous release.
For repeatable experiments, keep the input image, mask, seed, scheduler, and step count together. Inpainting is local editing: the unmasked region is provided as the masked-image conditioning and is also preserved by the native runtime compositor.
Core ML and SD 1.4-class deployment
The companion Core ML resource bundle is converted for iOS 18 with a
batch-one U-Net and chunked U-Net resources for the SD 1.4-class 512×512
architecture. The Swift runtime performs classifier-free guidance as two
serial passes to reduce peak memory. The
bundled VAEEncoder.mlmodelc creates the masked-image latent locally, so the
input image and mask do not leave the device.
Conversion and the native iOS integration live in the source Clover repo:
Training provenance
The context-aware v2 recipe distills the pinned official SD 1.5 inpainting
U-Net into Clover's compact nine-channel U-Net while retaining a ground-truth
diffusion loss. Training uses diverse synthetic free-form, multi-region,
object-like, and outpainting masks over the pinned Apache-2.0
prithivMLmods/Caption3o-Opt image-caption dataset. A deterministic 128-image
holdout is excluded from optimization and reused for same-mask, same-seed
comparisons against the previous release.
The job runs on Modal under the guccichungus69 workspace and stores rolling
resumable checkpoints plus the final Diffusers pipeline in the
clover-image-tiny-inpaint-output Volume before Core ML conversion. Exact
revisions and objective weights are recorded in inpainting/config.json and
the generated training-summary.json.
Citation
If Clover Image Tiny Inpaint is useful in your work, please cite the release:
@software{lozadaperez2026cloverimagetinyinpaint,
author = {Lukas Lozada Perez},
title = {Clover Image Tiny Inpaint: Compact SD 1.4-Class Image Inpainting},
year = {2026},
url = {https://huggingface.co/neonforestmist/Clover-Image-Tiny-Inpaint}
}