# Prompting this model This is a finetune of `krea/Krea-2-Raw`, merged with the Krea-2-Turbo distillation delta. It samples in 8 steps without CFG. It understands ordinary prose prompts. But about 90% of its training used a **compact grounding DSL**, a plain-text layout language that puts named things in named boxes. That is where this finetune differs from stock Krea-2. It is what this document covers: multi-panel comics, speech bubbles that land inside their panel, characters that stay themselves across panels, and text that appears where you asked for it. --- ## 1. Two ways to prompt **Prose.** Works as you'd expect. Nothing here is required. ``` A 1960s screen-printed travel poster for a fictional mountain railway. A stylised red funicular climbs a steep green slope on the right. Large cream letters across the top read 'ALPENBAHN'. ``` **Grounded.** A line-per-element layout spec. Same model, same call. It is just a differently shaped string. ``` A four-panel comic in which an anime handywoman and an anthropomorphic tabby cat assemble a flat-pack shelf and fail completely. @clean line art, flat colors, light cel shading; Digital illustration ~A bare apartment living room with cardboard packaging and loose screws on a pale wood floor. p[0,0,500,500] Top-left panel: the handywoman holding up the instruction sheet, confident. p[500,0,1000,500] Top-right panel: the cat batting a single screw off the floor. p[0,500,500,1000] Bottom-left panel: a lopsided half-built shelf wobbling. p[500,500,1000,1000] Bottom-right panel: the shelf collapsed, the cat sitting smugly on the wreckage. ac:1[60,80,340,470] A young woman with short dark hair in blue overalls, holding an instruction sheet, confident grin. fc:2[560,90,880,470] An anthropomorphic tabby cat in a tool belt, one paw raised mid-swat, innocent expression. t[80,40,420,120]"SOME ASSEMBLY REQUIRED." black text in a rounded speech bubble o[80,700,440,960] The collapsed shelf, planks fanned out across the floor. t[540,530,960,610]"IT HAS FIVE LEGS NOW." black text in a rounded speech bubble ``` No JSON, no braces, no keys. Whitespace is not significant, except that **every element sits on its own line**. --- ## 2. Coordinates. Read this part twice. ``` [x0, y0, x1, y1] ``` * **X FIRST.** Not `[y0,x0,y1,x1]`. This is the most common way to get a grounded prompt wrong, and it fails quietly. A transposed box still renders, still generates an image, and just puts everything in the wrong place. If your layouts come out mirrored about the diagonal, this is why. * **0 to 1000 on both axes**, whatever the image's pixel size or aspect ratio. `[0,0,1000,1000]` is the whole canvas. * **Origin is top-left.** `y` increases downward. * Integers, with `x0 < x1` and `y0 < y1`. So `p[500,0,1000,500]` is the **top-right** quadrant. X runs from the middle to the right edge, y from the top to the middle. --- ## 3. The format ### Header lines (all optional, in this order) | Line | Meaning | |---|---| | *(first line, no sigil)* | One sentence describing the whole image | | `@...` | Style: `; ` | | `~...` | Background / setting | ``` An anime key visual of a crimson-cloaked archer drawing a longbow on a windswept hillside at dusk. @clean line art, soft cel shading, high saturation; Digital illustration ~A long grassy ridge under a banded orange and deep-blue sky with distant birds. ``` ### Element lines ``` [x0,y0,x1,y1] :[x0,y0,x1,y1] t[x0,y0,x1,y1]"" ``` No space between the tag and the `[`. The description runs free-form to the end of the line. | Tag | Element | Notes | |---|---|---| | `p` | panel | Comic panel. Put these first. | | `o` | object | Anything inanimate. | | `t` | text | Carries the literal string to render, in quotes. | | `pe` | person | Takes a `character_id`. | | `ac` | anime character | Takes a `character_id`. | | `fc` | furry / anthro character | Takes a `character_id`. | ### Character identity `ac`, `fc` and `pe` can carry an id: `ac:[...]`. **Reuse the same id to mean the same character.** That is how you keep one person consistent across the panels of a strip. ``` p[0,0,1000,333] Top panel: the barista takes an order, alert and cheerful. p[0,333,1000,666] Middle panel: the barista pulling shots, sleeves pushed up, focused. p[0,666,1000,1000] Bottom panel: the barista slumped against the counter at closing time. ac:1[80,40,400,320] A barista with a messy green hoodie under an apron and short auburn hair, smiling brightly. ac:1[560,370,900,650] The same barista mid-shift, sleeves rolled, brow furrowed at the machine. ac:1[100,700,420,980] The same barista slumped over the counter, hollow-eyed, apron askew. ``` **Use small integers: `1`, `2`, `3`.** Any lowercase `[a-z0-9_]+` parses, but every character id in the training data is a number, assigned in first-appearance order within each image. The labelling pipeline renumbers descriptive slugs on purpose: short ids cost fewer of the 512 tokens, and they avoid the spaces, parens and capitals that would break the line format. Ids are labels local to one prompt, not references to anything the model memorised, so `1` does not mean a particular person. The character's identity comes from the `desc`, which is why you describe them again at each occurrence. The id only says "this is the same one as that". ### Text elements The quoted string is what should appear in the image, verbatim. Anything after the closing quote describes how it should look (bubble, lettering style, colour). That part is optional. ``` t[80,40,420,120]"SOME ASSEMBLY REQUIRED." black text in a rounded speech bubble t[120,540,320,960]"KRAK!!" huge jagged yellow SFX lettering, no bubble t[600,590,860,860]"TONKOTSU\nMISO\nSHOYU" white chalk lettering, three stacked lines ``` Escapes inside the quotes: `\"` for a quote, `\\` for a backslash, `\n` for a line break. --- ## 4. Conventions that match the training data None of this is enforced. The model saw it consistently, so following it helps. * **Panels first.** Then characters, objects and text. * **Keep contents inside their panel.** A speech bubble's box should sit within the box of the panel it belongs to. * **A box is a region, not a pixel-tight bound.** Rough is fine. The model reads it as "put this here", not as a hard mask. * **Keep descriptions short.** The whole prompt is encoded into a **fixed 512-token** window, 507 of them usable. Over-long prompts are truncated from the end, which usually drops your text elements, the things you most wanted placed. The gallery prompts run 190 to 400 tokens. If you are near the limit, shorten the prose, not the number of elements. * Dense scenes in the training data went through a budgeter that dropped whole elements rather than truncating mid-element, keeping panels first and objects last. The model has effectively never seen a half-written element line. --- ## 5. Building the string from code Nothing fancy is needed. Writing it by hand invites coordinate mistakes, though. This helper takes **x-first** boxes and emits exactly the format above. ```python def build_grounding(summary="", style="", medium="", background="", elements=()): """elements: dicts with tag, bbox=(x0,y0,x1,y1), and optionally id / text / desc.""" lines = [] if summary: lines.append(summary) style_line = "; ".join(b for b in (style, medium) if b) if style_line: lines.append("@" + style_line) if background: lines.append("~" + background) for el in elements: x0, y0, x1, y1 = (int(v) for v in el["bbox"]) head = el["tag"] if el.get("id"): head += ":" + el["id"] line = f"{head}[{x0},{y0},{x1},{y1}]" if el.get("text") is not None: escaped = el["text"].replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n") line += f'"{escaped}"' if el.get("desc"): line += " " + el["desc"].replace("\n", " ").strip() lines.append(line) return "\n".join(lines) prompt = build_grounding( summary="A two-panel comic where a knight destroys a birthday piñata.", style="heavy outlines, saturated flat colors, speed lines", medium="Digital illustration", background="A suburban back garden strung with paper bunting.", elements=[ {"tag": "p", "bbox": (0, 0, 500, 1000), "desc": "Left panel: the knight winding up an enormous swing."}, {"tag": "p", "bbox": (500, 0, 1000, 1000), "desc": "Right panel: the piñata detonating in a shower of sweets."}, {"tag": "pe", "bbox": (80, 200, 420, 900), "id": "armored_knight", "desc": "A knight in full plate armour swinging a wooden stick like a greatsword."}, {"tag": "t", "bbox": (540, 120, 960, 320), "text": "KRAK!!", "desc": "huge jagged yellow SFX lettering, no bubble"}, ], ) ``` --- ## 6. Running it ```python import torch from diffusers import Krea2Pipeline, Krea2Transformer2DModel tf = Krea2Transformer2DModel.from_pretrained( "jimmycarter/krea2-turbo-bbox", subfolder="/transformer", torch_dtype=torch.bfloat16) pipe = Krea2Pipeline.from_pretrained("krea/Krea-2-Raw", transformer=tf, torch_dtype=torch.bfloat16) pipe.to("cuda") image = pipe(prompt, num_inference_steps=8, guidance_scale=0.0, width=1296, height=1824).images[0] ``` * **8 steps, `guidance_scale=0.0`, `mu=1.15`.** These are distilled weights, so CFG is off. `mu` is the distilled timestep shift. Pass it if your pipeline does not infer it from `is_distilled`. * **Size:** roughly 1536x1536 worth of area, with both dimensions a multiple of 16. Non-square is fine and often better. Comics in the training data run about 1.4 tall for every 1 wide, so `1296x1824` suits a comic page far better than a square. Keep the area roughly constant when you change the aspect. * Grounded prompts and prose prompts both go in the same `prompt` argument. There is no flag. --- ## 7. What to expect Good at: panel layout, putting a speech bubble in the panel you asked for, short verbatim text, keeping a `character_id` recognisable across panels, anime and anthro character work. Weaker at: long paragraphs of rendered text, more than six to eight characters in one image, exact typography, and boxes whose shape fights their content. A very wide `t[]` box holding one short word will stretch or repad it. The boxes are guidance, not a constraint. Placement accuracy falls off as you add elements and as boxes get small.