Add comprehensive README documentation

README.md

@@ -1,3 +1,423 @@
 ---
 license: apache-2.0
+base_model: stabilityai/stable-diffusion-xl-base-1.0
+tags:
+- stable-diffusion-xl
+- sdxl
+- text-to-image
+- diffusers
+- lora
+- control
+- controlnet
+- control-lora
+- brightness
+- grayscale
+- template:sd-lora
+widget:
+- text: "a beautiful garden scene with colorful flowers and butterflies, highly detailed, professional photography, vibrant colors"
+  output:
+    url: "https://huggingface.co/Oysiyl/controlnet-lora-brightness-sdxl-10k/resolve/main/examples/example.png"
+inference: true
 ---
+
+# ControlNet LoRA SDXL - Brightness Control (10k @ 1024×1024)
+
+A Control LoRA model trained on Stable Diffusion XL to control image generation through brightness/grayscale information. This model uses **LoRA (Low-Rank Adaptation)** combined with ControlNet architecture for efficient control, providing an **ultra-lightweight alternative** to full ControlNet with excellent pattern preservation.
+
+## Model Description
+
+This Control LoRA enables brightness-based conditioning for SDXL image generation. By providing a grayscale image as input, you can control the brightness distribution and lighting structure while maintaining creative freedom through text prompts.
+
+### Key Features:
+
+- 🎨 **Excellent brightness and pattern control** across multiple scales (0.5-2.0)
+- 🚀 **196x smaller than full ControlNet**: ~24MB vs ~4.7GB
+- ⚡ **Ultra-fast loading**: LoRA weights load in <1 second
+- 💡 **Flexible scale control**: Adjustable conditioning scale from 0.5 to 2.0+
+- 🔄 **Compatible with ControlLoRA v3**: Uses the efficient ControlLoRA v3 architecture
+- 📦 **Minimal storage**: All checkpoints + final model = ~120MB total
+- 🖼️ **Native SDXL resolution**: Trained at 1024×1024
+
+### Intended Uses:
+
+- **Artistic QR code generation** (scale 1.0-1.5 recommended)
+- Image recoloring and colorization
+- Lighting control in text-to-image generation
+- Brightness-based pattern integration
+- Watermark and subtle pattern embedding
+- Photo enhancement and stylization
+
+## Training Details
+
+### Training Data
+
+Trained on 10,000 samples from `latentcat/grayscale_image_aesthetic_3M`:
+- High-quality aesthetic images
+- Paired with grayscale/brightness versions
+- Native resolution: 1024×1024 (SDXL native)
+
+### Training Configuration
+
+| Parameter | Value |
+|-----------|-------|
+| **Base Model** | `stabilityai/stable-diffusion-xl-base-1.0` |
+| **Architecture** | ControlLoRA v3 (~7M trainable parameters) |
+| **LoRA Rank** | 16 |
+| **Extra Conv Rank** | 64 (conv_in layer) |
+| **Training Resolution** | 1024×1024 |
+| **Training Steps** | 313 (1 epoch) |
+| **Batch Size** | 8 per device |
+| **Gradient Accumulation** | 4 (effective batch: 32) |
+| **Learning Rate** | 1e-4 |
+| **Empty Prompts** | 10% (ControlLoRA v3 recommendation) |
+| **Mixed Precision** | BF16 |
+| **Hardware** | NVIDIA H100 80GB |
+| **Training Time** | ~21 minutes |
+| **Final Loss** | ~0.10-0.12 |
+
+### Model Size Comparison
+
+| Model | Parameters | Size | Training | Resolution |
+|-------|-----------|------|----------|-----------|
+| **This Control LoRA** | ~7M | **~24MB** | 10k @ 1024 | 1024×1024 |
+| ControlNet (SDXL) | ~700M | 4.7GB | 100k @ 512 | 512×512 |
+| T2I Adapter (SDXL) | ~77M | 302MB | 100k @ 1024 | 1024×1024 |
+| Flux Control LoRA | ~7M | 25MB | 10k @ 512 | 512×512 |
+
+## Usage
+
+### Installation
+
+```bash
+pip install diffusers transformers accelerate torch peft
+# Install ControlLoRA v3
+git clone https://github.com/HighCWu/control-lora-v3
+```
+
+### Basic Usage
+
+```python
+import torch
+import sys
+sys.path.insert(0, '/path/to/control-lora-v3')
+
+from pipeline_sdxl import StableDiffusionXLControlLoraV3Pipeline
+from model import UNet2DConditionModelEx
+from PIL import Image
+
+# Load UNet with LoRA support
+unet = UNet2DConditionModelEx.from_pretrained(
+    "stabilityai/stable-diffusion-xl-base-1.0",
+    subfolder="unet",
+    torch_dtype=torch.bfloat16,
+)
+unet = unet.add_extra_conditions(["brightness"])
+
+# Load SDXL Control LoRA pipeline
+pipe = StableDiffusionXLControlLoraV3Pipeline.from_pretrained(
+    "stabilityai/stable-diffusion-xl-base-1.0",
+    unet=unet,
+    torch_dtype=torch.bfloat16,
+)
+
+# Load Control LoRA weights
+pipe.load_lora_weights("Oysiyl/controlnet-lora-brightness-sdxl-10k", adapter_name="brightness")
+pipe.to("cuda")
+
+# Load grayscale/brightness control image
+control_image = Image.open("path/to/grayscale_image.png")
+control_image = control_image.resize((1024, 1024))
+
+# Generate image
+prompt = "a beautiful garden scene with colorful flowers and butterflies, highly detailed, professional photography, vibrant colors"
+
+image = pipe(
+    prompt=prompt,
+    image=control_image,
+    num_inference_steps=30,
+    guidance_scale=7.5,
+    extra_condition_scale=1.0,  # Controls conditioning strength
+    height=1024,
+    width=1024,
+).images[0]
+
+image.save("output.png")
+```
+
+### Adjusting Control Strength
+
+The `extra_condition_scale` parameter controls how strongly the brightness map influences generation:
+
+```python
+# Subtle control (scale 0.5-0.7)
+image = pipe(
+    prompt=prompt,
+    image=control_image,
+    extra_condition_scale=0.5,
+    ...
+).images[0]
+
+# Balanced control (scale 1.0-1.5) - Recommended for artistic QR codes
+image = pipe(
+    prompt=prompt,
+    image=control_image,
+    extra_condition_scale=1.0,
+    ...
+).images[0]
+
+# Strong control (scale 1.5-2.0)
+image = pipe(
+    prompt=prompt,
+    image=control_image,
+    extra_condition_scale=1.5,
+    ...
+).images[0]
+```
+
+### Artistic QR Code Generation
+
+```python
+import qrcode
+from PIL import Image
+
+# Generate QR code
+qr = qrcode.QRCode(
+    version=1,
+    error_correction=qrcode.constants.ERROR_CORRECT_H,
+    box_size=10,
+    border=4
+)
+qr.add_data("https://your-url.com")
+qr.make(fit=True)
+
+qr_image = qr.make_image(fill_color="black", back_color="white")
+qr_image = qr_image.resize((1024, 1024), Image.LANCZOS).convert("RGB")
+
+# Generate artistic QR code (scale 1.0-1.5 works best)
+image = pipe(
+    prompt="a beautiful garden with colorful flowers and butterflies, highly detailed, professional photography",
+    image=qr_image,
+    num_inference_steps=30,
+    guidance_scale=7.5,
+    extra_condition_scale=1.0,
+    height=1024,
+    width=1024,
+).images[0]
+
+image.save("artistic_qr.png")
+```
+
+### Using Different Checkpoints
+
+The model includes intermediate checkpoints from throughout training:
+
+```python
+# Early checkpoint (25% - 2,500 samples)
+pipe.load_lora_weights("Oysiyl/controlnet-lora-brightness-sdxl-10k",
+                       adapter_name="brightness",
+                       revision="checkpoint-78")
+
+# Mid checkpoint (50% - 5,000 samples)
+pipe.load_lora_weights("Oysiyl/controlnet-lora-brightness-sdxl-10k",
+                       adapter_name="brightness",
+                       revision="checkpoint-156")
+
+# Late checkpoint (75% - 7,500 samples)
+pipe.load_lora_weights("Oysiyl/controlnet-lora-brightness-sdxl-10k",
+                       adapter_name="brightness",
+                       revision="checkpoint-234")
+
+# Near-final checkpoint (99% - 9,984 samples)
+pipe.load_lora_weights("Oysiyl/controlnet-lora-brightness-sdxl-10k",
+                       adapter_name="brightness",
+                       revision="checkpoint-312")
+
+# Final model (10,000 samples, main branch - recommended)
+pipe.load_lora_weights("Oysiyl/controlnet-lora-brightness-sdxl-10k",
+                       adapter_name="brightness")
+```
+
+## Conditioning Scale Guide
+
+The `extra_condition_scale` parameter controls how strongly the brightness map influences generation:
+
+### Recommended Scale Ranges
+
+| Scale | Behavior | Best For |
+|-------|----------|----------|
+| **0.5-0.7** | Subtle artistic integration with hints of pattern | Natural images, soft lighting hints |
+| **0.7-1.0** | Light control - visible structure with artistic freedom | Artistic images, creative reinterpretation |
+| **1.0-1.5** | **🔥 Balanced control** | **Artistic QR codes, watermarks** (recommended) |
+| **1.5-2.0** | Strong control - clear patterns with artistic overlay | Geometric patterns, structured designs |
+| **2.0+** | Maximum control - dominant patterns | Strong brightness maps, technical applications |
+
+## Performance Comparison
+
+### vs Full ControlNet (SDXL)
+
+| Metric | ControlNet (SDXL) | This Control LoRA | Advantage |
+|--------|-------------------|-------------------|-----------|
+| **Parameters** | ~700M | ~7M | **100x smaller** |
+| **Model Size** | 4.7GB | 24MB | **196x smaller** |
+| **Load Time** | ~5-10 seconds | **<1 second** | **10x faster loading** |
+| **Storage (w/ checkpoints)** | ~18.8GB | **~120MB** | **157x less storage** |
+| **Training Time** | ~3 hours | **21 minutes** | **8.5x faster** |
+| **Pattern Preservation @ Scale 1.0** | Excellent | **Excellent** | Comparable quality |
+| **Flexibility** | Fixed architecture | **Adjustable weights** | More versatile |
+
+### vs T2I Adapter (SDXL)
+
+| Metric | T2I Adapter (SDXL) | This Control LoRA | Advantage |
+|--------|-------------------|-------------------|-----------|
+| **Parameters** | ~77M | ~7M | **11x smaller** |
+| **Model Size** | 302MB | 24MB | **12.6x smaller** |
+| **Training Samples** | 100k | 10k | **More efficient** |
+| **Architecture** | Separate adapter | **Integrated LoRA** | Simpler loading |
+
+## Checkpoint Progression Analysis
+
+The model includes checkpoints from throughout training:
+
+1. **checkpoint-78**: 25% complete (2,500 samples)
+2. **checkpoint-156**: 50% complete (5,000 samples)
+3. **checkpoint-234**: 75% complete (7,500 samples)
+4. **checkpoint-312**: 99% complete (9,984 samples)
+5. **Final model**: 100% complete (10,000 samples - main branch)
+
+### Visual Comparison
+
+![Checkpoint Progression](https://huggingface.co/Oysiyl/controlnet-lora-brightness-sdxl-10k/resolve/main/examples/controlnet_lora_checkpoint_comparison_scale_1.0.png)
+
+*Checkpoint progression at scale 1.0, showing improvement from 25% to 100% training*
+
+### Key Observations
+
+All checkpoints show consistent, high-quality performance across scales. The progression analysis reveals:
+
+1. **Early Checkpoint (78 steps, 2.5k samples)**:
+   - Good pattern awareness, developing control
+   - More artistic interpretation of prompts
+   - Recommended scales: 0.7-1.2
+
+2. **Mid Checkpoints (156-234 steps, 5k-7.5k samples)**:
+   - Strong balance between control and creativity
+   - Stable pattern preservation
+   - Recommended scales: 0.8-1.5
+
+3. **Final Model (313 steps, 10k samples)**:
+   - Maximum control capability
+   - Excellent pattern preservation at all scales
+   - Recommended scales: 0.7-2.0
+
+### No Overfitting Observed
+
+Unlike larger models trained on 100k samples, this 10k Control LoRA shows **no signs of overfitting**:
+- Consistent improvement throughout training
+- Final checkpoint is recommended for production use
+- 10k samples appears optimal for LoRA-based control training
+
+## When to Use This Model
+
+### ✅ Use This Control LoRA When:
+
+- Creating **artistic QR codes** with SDXL quality (scale 1.0-1.5)
+- Need **minimal storage overhead** (<30MB per checkpoint)
+- Want **fast model loading** (<1 second)
+- Building **production applications** requiring small model sizes
+- Working with **SDXL** as base model
+- Require **flexible control strength** via extra_condition_scale
+- Need **multiple checkpoints** without massive storage (120MB total vs 18.8GB)
+- Training on **limited data** (10k samples sufficient)
+
+### ⚠️ Consider Alternatives When:
+
+- Need **full ControlNet** features with extremely precise control
+- Working with existing **T2I Adapter** pipelines
+- Require **different control types** (pose, depth, etc.) - train separate LoRAs
+
+## Limitations
+
+### Current Limitations
+
+- **ControlLoRA v3 dependency**: Requires custom pipeline code (not in main diffusers yet)
+- **Grayscale conditioning only**: Trained specifically for brightness/grayscale control
+- **Single control type**: Only brightness, not other conditioning types
+- **Custom code required**: Need to include ControlLoRA v3 files
+
+### Recommendations
+
+- For **SDXL generation**, use this Control LoRA
+- For **multiple control types**, train separate LoRAs and combine
+- Experiment with **scales 1.0-1.5** for most use cases
+- Use **final model** for best results
+
+## Training Script
+
+```bash
+accelerate launch --mixed_precision="bf16" train_sdxl.py \
+  --pretrained_model_name_or_path="stabilityai/stable-diffusion-xl-base-1.0" \
+  --dataset_name="<path_to_10k_dataset>" \
+  --conditioning_image_column="conditioning_image" \
+  --image_column="image" \
+  --caption_column="text" \
+  --output_dir="./controlnet-lora-brightness-sdxl-10k" \
+  --mixed_precision="bf16" \
+  --resolution=1024 \
+  --learning_rate=1e-4 \
+  --proportion_empty_prompts=0.1 \
+  --rank=16 \
+  --lora_adapter_name="brightness" \
+  --extra_lora_rank_modules conv_in \
+  --extra_lora_ranks 64 \
+  --half_or_full_lora=half_skip_attn \
+  --train_batch_size=8 \
+  --num_train_epochs=1 \
+  --gradient_accumulation_steps=4 \
+  --gradient_checkpointing \
+  --checkpointing_steps=78 \
+  --validation_steps=78 \
+  --validation_image="validation_qr.png" \
+  --validation_prompt="a beautiful garden scene with colorful flowers and butterflies, highly detailed, professional photography, vibrant colors" \
+  --num_validation_images=4 \
+  --seed=42 \
+  --dataloader_num_workers=4 \
+  --tracker_project_name="controlnet-lora-brightness-sdxl-10k" \
+  --report_to="wandb" \
+  --enable_xformers_memory_efficient_attention \
+  --use_8bit_adam \
+  --init_lora_weights="gaussian"
+```
+
+## Available Branches
+
+- `main`: Final model (10,000 samples, recommended)
+- `checkpoint-78`: Early checkpoint (2,500 samples)
+- `checkpoint-156`: Mid checkpoint (5,000 samples)
+- `checkpoint-234`: Late checkpoint (7,500 samples)
+- `checkpoint-312`: Near-final checkpoint (9,984 samples)
+
+## Citation
+
+```bibtex
+@misc{controlnet-lora-brightness-sdxl-10k,
+  author = {Oysiyl},
+  title = {ControlNet LoRA SDXL - Brightness Control (10k @ 1024×1024)},
+  year = {2026},
+  publisher = {HuggingFace},
+  journal = {HuggingFace Model Hub},
+  howpublished = {\url{https://huggingface.co/Oysiyl/controlnet-lora-brightness-sdxl-10k}}
+}
+```
+
+## Acknowledgments
+
+- Built with [🤗 Diffusers](https://github.com/huggingface/diffusers)
+- Base model: [Stable Diffusion XL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0) by Stability AI
+- ControlLoRA v3: [control-lora-v3](https://github.com/HighCWu/control-lora-v3) by HighCWu
+- Dataset: [grayscale_image_aesthetic_3M](https://huggingface.co/datasets/latentcat/grayscale_image_aesthetic_3M) by latentcat
+- Training infrastructure: NVIDIA H100 80GB
+- LoRA implementation: [PEFT](https://github.com/huggingface/peft) by Hugging Face
+
+## License
+
+Apache 2.0 License. The base SDXL model has separate license terms at [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0).