add README.md

9292e18 verified 3 months ago

4.71 kB

license: apache-2.0
base_model: Tongyi-MAI/Z-Image
tags:
  - text-to-image
  - lora
  - z-image
  - diffusers
  - peft
  - witcher3
  - game-map
library_name: peft
pipeline_tag: text-to-image
inference: false

witcher3-velen-z-image-lora

LoRA adapter for Tongyi-MAI/Z-Image (6B Single-Stream DiT, base / non-Turbo) that produces top-down fantasy RPG game-map tiles in the visual style of The Witcher 3: Wild Hunt's Velen region — muted greens and browns, rivers and marshland, forest, villages, ruins.

Trained on PNW-GM/witcher3_velen (private), a captioned tile dataset cut from the in-game Velen world map.

Files

File	Purpose
`adapter_model.safetensors`	LoRA weights (rank 128).
`adapter_config.json`	PEFT config — `target_modules`, rank, alpha.
`README.md`	This card.

Training summary


Base model	`Tongyi-MAI/Z-Image` (6B base, not the Turbo distilled variant — Turbo is explicitly marked non-fine-tunable on its model card)
LoRA rank / alpha	128 / 128
Target modules	`to_q, to_k, to_v, to_out.0, feed_forward.w1, feed_forward.w2, feed_forward.w3` (attention + SwiGLU FFN)
Resolution	1024²
Optimizer	AdamW, lr=1e-4, bf16 mixed precision, gradient checkpointing
Effective batch	16 (bs=8/rank × 2 ranks DDP, no grad accumulation)
Steps	720 (≈1.13 epoch over ~10 176 rows after 4× repetitions)
Hardware	2× NVIDIA A40 (48 GB), ~5 h wall-time

The FFN module names follow Z-Image's SwiGLU convention (feed_forward.w1 = gate, w2 = down, w3 = up) — not the ff.net.0.proj / ff.net.2 pattern from generic diffusers examples. Verified by inspecting transformer.layers[0].feed_forward.named_modules().

Inference

Requires diffusers @ main (the version that exposes ZImagePipeline), peft >= 0.13, and safetensors.

import torch
from diffusers import ZImagePipeline
from peft import PeftModel

BASE = "Tongyi-MAI/Z-Image"
LORA = "PNW-GM/witcher3-velen-z-image-lora"

pipe = ZImagePipeline.from_pretrained(BASE, torch_dtype=torch.bfloat16)
pipe.to("cuda")
pipe.transformer = PeftModel.from_pretrained(pipe.transformer, LORA)
pipe.transformer.eval()

prompt = (
    "This fragment of a fantasy RPG game map depicts a small, elongated river "
    "running centrally from the bottom left to the top right of the frame. The "
    "river's waters are depicted in greenish-blue with wavy textures. Surrounding "
    "the river, there are patches of forest in darker brown shades, interspersed "
    "with lighter clearings. To the left side of the river, several irregularly "
    "shaped small ponds are surrounded by dense vegetation. A narrow dirt path "
    "winds along the right bank, connecting two clusters of wooden buildings."
)

image = pipe(
    prompt=prompt,
    height=1024,
    width=1024,
    num_inference_steps=50,
    guidance_scale=5.0,
    generator=torch.Generator("cuda").manual_seed(441),
).images[0]
image.save("velen_z.png")

Notes on inference settings

The dataset captions are long, structured descriptions (a few hundred to ~2000 chars) of map contents — landforms, water bodies, roads, buildings. Short prompts work but the style locks in more tightly with caption-like prompts.
num_inference_steps=50 + guidance_scale=5.0 matches Z-Image base defaults and the training distribution. The Turbo variant (8 steps, CFG=0) can be tried as a base, but its model card explicitly says it is not fine-tunable — the LoRA may be partially or wholly ignored by the distilled schedule.
Single A40 (48 GB) is enough for bf16 inference at 1024².

Intended use & limitations

Intended: generating Velen-style top-down RPG map tiles for game-design exploration, asset prototyping, and research into style-conditioned generation.
Limitations: the LoRA was trained on tiles from a single in-game region; coverage of other biomes (deserts, snow, urban) is whatever the base model already does. Captioning was generated automatically and is sometimes imprecise. Image content reflects the dataset's source material.
License: the LoRA adapter weights are released under Apache 2.0 (same as the Z-Image base). The underlying Witcher 3 imagery is © CD PROJEKT RED — these adapter weights are released for research and are not a redistribution of the source artwork.

Citation

@misc{pnwgm_witcher3_velen_z_image_lora,
  title  = {Witcher 3 Velen Z-Image LoRA},
  author = {PNW-GM},
  year   = {2026},
  url    = {https://huggingface.co/PNW-GM/witcher3-velen-z-image-lora},
}