Publish Alpha Gen IC-LoRA

6184df1 5 days ago

7.33 kB

base_model:
  - Lightricks/LTX-2.5
base_model_relation: adapter
license: other
license_name: ltx-2.x-community-license
license_link: https://github.com/Lightricks/LTX-2/blob/main/LICENSE-2_x
language:
  - en
tags:
  - alpha-matte
  - matting
  - background-removal
  - ic-lora
  - ltx-2.5
  - video-to-video
  - vfx
  - ltx
pipeline_tag: video-to-video
extra_gated_description: >-
  By clicking "Agree and Access" you acknowledge the [Privacy
  Policy](https://static.lightricks.com/legal/Privacy%20Policy%20-%20LTX%20Platform.pdf)
  and consent to receive offers and updates including targeted and personalized
  advertisements. You can unsubscribe at any time.
extra_gated_button_content: Agree and Access
widget:
  - text: No prompt required
    output:
      url: examples/dragon_fire_sbs.mp4
  - text: No prompt required
    output:
      url: examples/dog_fur_sbs.mp4
  - text: No prompt required
    output:
      url: examples/sheer_curtain_sbs.mp4
  - text: No prompt required
    output:
      url: examples/water_pour_sbs.mp4
  - text: No prompt required
    output:
      url: examples/backlit_hair_sbs.mp4
  - text: No prompt required
    output:
      url: examples/dancer_smoke_sbs.mp4

LTX-2.5 22B IC-LoRA Alpha Gen

This is an Alpha Gen IC-LoRA trained on top of LTX-2.5-22B. It takes an ordinary RGB ("beauty") video and generates a matching alpha matte for it, with no green screen, masks, or prompt required. It handles hard subjects as well as soft and semi-transparent elements such as hair, fur, smoke, fire, sheer fabric, glass, and water.

It is based on the LTX-2.5 foundation model.

Prompt: No prompt required

Prompt: No prompt required

Prompt: No prompt required

Prompt: No prompt required

Prompt: No prompt required

Prompt: No prompt required

Model Files

ltx-2.5-22b-ic-lora-alpha-gen-0.9.safetensors: the released checkpoint (training step 25000). Recommended inference on the -distilled base.

Model Details

Base Model: LTX-2.5-22B Video
Training Type: IC-LoRA (video-to-video, reference-conditioned)
Control Type: Reference video: the RGB clip to be matted. The model outputs a grayscale alpha matte (white = foreground, black = background) aligned frame-for-frame with the input.
Reference Downscale Factor: 1 (the reference is processed at the same resolution as the output).
Prompt: Always empty. The RGB video is the only guide.

Intended Use & Out-of-Scope

Intended use: Pulling an alpha matte from live-action or generated footage for compositing and VFX: isolating people, animals, objects, logos, and characters so they can be placed over new backgrounds, including shots with soft or semi-transparent edges that are hard to key or rotoscope.

Out of scope: Clips longer than 145 frames: above that, RGB content starts leaking into the matte, so trim or split longer clips. Resolutions above 1920×1088. Choosing which object to matte: there is no prompt or mask control, and the model decides the foreground on its own.

Control Signal Requirements

Control signal type: The RGB clip to be matted (the "beauty" pass).
Expected input: A single video at up to 1920×1088 and up to 145 frames.
Preprocessing: Pad (don't stretch) the clip to dimensions divisible by 32 and a frame count of 8n+1 (e.g. 1920×1080 → 1920×1088; 145 frames already fits). No extractor or normalization is needed; the RGB clip is VAE-encoded directly.
Alignment: The matte is generated at the same size and length as the reference and is frame-aligned with it.
Mask support: Not supported. The model predicts the foreground for the whole frame.

How It Works

The IC-LoRA conditions on the latents of the RGB reference video and generates the corresponding alpha matte as a video, with an empty text prompt. Because the reference stays attached for the entire denoise (stage-1-only inference at native resolution), the matte stays aligned with the source down to fine edges.

To composite, use the matte as the alpha channel of the original RGB clip:

out = rgb * alpha + background * (1 - alpha)

Usage

🔌 ComfyUI

Copy ltx-2.5-22b-ic-lora-alpha-gen-0.9.safetensors into models/loras.
Load the LTX-2.5-22B distilled base model and add the LoRA.
Use the IC-LoRA (video-to-video) workflow from the LTX-2 ComfyUI repository. Select this LoRA in the IC-LoRA loader node, connect your RGB clip as the reference video, and leave the prompt empty.
Use LoRA strength 1.0.
The output video is the alpha matte. Combine it with your original clip as its alpha channel (e.g. in your compositor, or export RGBA / ProRes 4444).

Python pipeline

Use the IC-LoRA video-to-video pipeline (python -m ltx_pipelines.ic_lora) from LTX-2 with an empty prompt, --skip-stage-2, and --width/--height set to 2× the source size, so the stage-1 render comes back at native resolution:

# --width/--height are 2x the 1920x1088 source: stage 1 renders at half the requested size
# and --skip-stage-2 skips the 2x upscale, so the matte comes out at 1920x1088.
python -m ltx_pipelines.ic_lora \
  --transformer-path ltx-2.5-22b-distilled-transformer-bf16.safetensors \
  --text-encoder-path gemma4-12b-with-proj-ltx-2.5-bf16.safetensors \
  --video-vae-path ltx-2.5-video-vae-bf16.safetensors \
  --audio-vae-path ltx-2.5-audio-vae-bf16.safetensors \
  --spatial-upsampler-path ltx-2.5-latent-spatial-upscaler-x2-bf16-1.0.safetensors \
  --prompt "" \
  --width 3840 --height 2176 --num-frames 121 --frame-rate 24 \
  --video-conditioning input_1920x1088.mp4 1.0 \
  --lora ltx-2.5-22b-ic-lora-alpha-gen-0.9.safetensors 1.0 \
  --skip-stage-2 \
  --seed 1234 \
  --output-path matte.mp4

Recommended Settings

Setting	Recommended value
LoRA strength	1.0
Base checkpoint	`ltx-2.5-22b-distilled`
Prompt	empty (`""`); do not add a prompt
Sampler	Distilled, stage-1 only at native resolution (`--skip-stage-2`, CLI width/height = 2× source)
Reference conditioning strength	1.0
Resolution	up to 1920×1088, dimensions divisible by 32
Frames	up to 145, `8n+1`
Seed	any (1234 used for the published samples)

Tips & Troubleshooting

RGB bleeding into the matte: the clip is too long. Keep it at 145 frames or fewer, and split longer shots into chunks.
Out-of-memory: full-HD 145-frame clips need an H100/B200-class GPU. Shorten the clip or lower the resolution.
Don't prompt: the model was built for an empty prompt; adding text doesn't steer which object is matted.

References

Code: GitHub Repository
ComfyUI: ComfyUI-LTXVideo
IC-LoRA docs: IC-LoRA usage guide

License

See the LTX-2-community-license for full terms.

Acknowledgments

Base model by Lightricks
Training infrastructure: LTX-2 Community Trainer