Card: eager is the better default; torch.compile recompiles on mixed workloads

README.md

@@ -39,7 +39,6 @@ from diffusers import DiffusionPipeline
 pipe = DiffusionPipeline.from_pretrained(
     "joseplcam/Qwen-Image-2.1-NVFP4", dtype=torch.bfloat16, trust_remote_code=True
 ).to("cuda")
-pipe.transformer.compile_repeated_blocks()  # optional: ~1.2x faster; the first call compiles (~10 s)
 
 image = pipe("A capybara reading a book by candlelight", height=512, width=512, num_inference_steps=40).images[0]
 edited = pipe("Make it night time with moonlight", image=image, output_resolution=512, num_inference_steps=40).images[0]
@@ -117,16 +116,20 @@ Peak VRAM is PyTorch's peak allocation.
 
 Batching several prompts gives no extra throughput in Diffusers on this GPU; it is saturated at batch 1.
 
-Runtime options (text-to-image, 512x512, 40 steps, mean of 3 runs after warmup):
+Runtime options (text-to-image, 512x512, 40 steps, one fixed prompt, mean of 3 runs after warmup):
 
 | Setup | BF16 (official) | This repo |
 |---|---|---|
 | Eager, PyTorch SDPA | 3.20 s | 1.79 s |
 | Eager, cuDNN SDPA | 3.26 s | 1.88 s |
-| **`compile_repeated_blocks()`, PyTorch SDPA** | 2.99 s | **1.50 s** (2.1x vs BF16 eager) |
-
-- `torch.compile` without CUDA graphs is the fastest setup here and does not change the images beyond the
-  run-to-run variation described below.
+| `compile_repeated_blocks()`, PyTorch SDPA | 2.99 s | 1.50 s |
+
+- **Use eager (the default) for mixed workloads.** `torch.compile` only wins while every call has the same
+  shapes. Prompt length and the reference image's aspect ratio change the prefix KV cache, so varied
+  text-to-image prompts and edits keep recompiling until TorchDynamo's recompile limit, after which new shapes
+  run uncompiled anyway. In a 12-call mix (6 prompts of different lengths, 6 edits with different reference
+  shapes) eager took 37.2 s in total, `compile_repeated_blocks()` 44.3 s and `dynamic=True` 56.5 s; steady-state
+  edits were 2.0 s, 2.0 s and 1.7 s. Compile is only worth it for fixed-shape batch jobs.
 - CUDA graphs (`mode="reduce-overhead"`) do not work: the prefix KV cache keeps tensors that graph replays
   overwrite, and the Nunchaku kernels cannot be captured (`cudaErrorStreamCaptureInvalidated`). With the KV
   cache off, BF16 gained only 6% from them.