malcolmrey

/

krea2

Like 101

I've trained dozens of LoRAs for Krea 2 on AiToolKit and have never had a character overfit. They come out like this because you use too many steps, perhaps because you don't test the intermediate checkpoints. If you use a dataset with 10-15 photos with 2000 and beyond steps, very high linear and linear alpha, and then test only the final checkpoint, it's perfectly normal for them to come out poorly. Use linear at 32, linear alpha at 16, and 1200 steps with 15 images. Prepare the yaml so that it generates intermediate checkpoints every 250 steps, so that you can test them at the end of the training, generally with the data I gave you the 1000 and 1200 steps are excellent. Use 1024x1024 regardless of whether you train at this resolution or a lower one. If you train at a lower resolution, still use 1024p for the dataset and then have AiToolKit resize the dataset. This preserves much more detail than using a much lower resolution directly.

I've trained dozens of LoRAs for Krea 2 on AiToolKit and have never had a character overfit. They come out like this because you use too many steps, perhaps because you don't test the intermediate checkpoints. If you use a dataset with 10-15 photos with 2000 and beyond steps, very high linear and linear alpha, and then test only the final checkpoint, it's perfectly normal for them to come out poorly. Use linear at 32, linear alpha at 16, and 1200 steps with 15 images. Prepare the yaml so that it generates intermediate checkpoints every 250 steps, so that you can test them at the end of the training, generally with the data I gave you the 1000 and 1200 steps are excellent. Use 1024x1024 regardless of whether you train at this resolution or a lower one. If you train at a lower resolution, still use 1024p for the dataset and then have AiToolKit resize the dataset. This preserves much more detail than using a much lower resolution directly.

I use about 20-25 Photos, Train within 2500-3000 Checkpoints, I test them all from 1000 onwards. Linear and Linear alpha stay at 16, All Images in my Dataset are 4K or 2K, 4K on Full Body, 2K on face/upper body and I completely Leave the Resizing to AiToolkit.
By Overfitting I don't mean that the Background or Character's Accessories get baked in, It's that the Character's Face isn't in there, If I ask for a Selfie photo, It only Comes down from the Chin although the Chin Does match my Character's. Similarly in a Top-Down view lying on bed, If I describe the Clothing a bit too in detail then Only the Character's body shows up without their head of Hair-Color even.

Is this Underfitting? I have seen Malcolm's OneTrainer LoRAs and they are Insane and Almost no Problems like mine, heck it's almost as if he trained the Character as a Person in the Model's own training dataset so it appears almost always naturally in Any scenario.

Am I doing something wrong? and for Reference, my LR is 0.0001

I've trained dozens of LoRAs for Krea 2 on AiToolKit and have never had a character overfit. They come out like this because you use too many steps, perhaps because you don't test the intermediate checkpoints. If you use a dataset with 10-15 photos with 2000 and beyond steps, very high linear and linear alpha, and then test only the final checkpoint, it's perfectly normal for them to come out poorly. Use linear at 32, linear alpha at 16, and 1200 steps with 15 images. Prepare the yaml so that it generates intermediate checkpoints every 250 steps, so that you can test them at the end of the training, generally with the data I gave you the 1000 and 1200 steps are excellent. Use 1024x1024 regardless of whether you train at this resolution or a lower one. If you train at a lower resolution, still use 1024p for the dataset and then have AiToolKit resize the dataset. This preserves much more detail than using a much lower resolution directly.

I use about 20-25 Photos, Train within 2500-3000 Checkpoints, I test them all from 1000 onwards. Linear and Linear alpha stay at 16, All Images in my Dataset are 4K or 2K, 4K on Full Body, 2K on face/upper body and I completely Leave the Resizing to AiToolkit.
By Overfitting I don't mean that the Background or Character's Accessories get baked in, It's that the Character's Face isn't in there, If I ask for a Selfie photo, It only Comes down from the Chin although the Chin Does match my Character's. Similarly in a Top-Down view lying on bed, If I describe the Clothing a bit too in detail then Only the Character's body shows up without their head of Hair-Color even.

Is this Underfitting? I have seen Malcolm's OneTrainer LoRAs and they are Insane and Almost no Problems like mine, heck it's almost as if he trained the Character as a Person in the Model's own training dataset so it appears almost always naturally in Any scenario.

Am I doing something wrong? and for Reference, my LR is 0.0001

2500-3000 steps for 25 photos is way too much for Krea 2 (RAW?), not to mention it also becomes very long. Do this: if you use 15 photos, use 1200 steps; if you use 20 photos, use 1500 steps, 25 photos 1800 step. Set the "linear" value to 32, and the "linear alpha" value to 16 and use adamw8bit. Generate intermediate checkpoints every 250 steps; you'll see that the 1000 and 1200 steps will be perfect for 15 photos and 1200 step, and you'll also gain calculation speed, which doesn't hurt. I now only ever take 15 photos at 1200 steps; the characters always come out well, provided, however, that you have good prompting in the dataset and carefully chosen photos. Your LR is good. Try it and let me know how it goes, I'm curious.
PS: In the dataset, use close-up photos, medium shot photos, medium close-up photos (from the chest up), and at least one three-quarter shot and one profile photo. If you use low resolutions like 512p, don't use full-body photos because the face would have too few pixels and the model wouldn't learn it. In the dataset .txt, always include the position, expression, where the face is looking, the type of framing, etc.

this is an example of my prompts in the dataset:
TR1GG3R: medium close-up shot, three-quarter view, front portrait (if the subject is framed frontally but the head is at three-quarters or the opposite you must write it), looking away, wide grin, showing teeth, standing, wearing a red tank top, room with several wooden-framed doors, paintings on the walls, a round white chandelier, a dark floor, a staircase at the end and a red object on a light brick wall in the background, indoor lighting, sharp focus

I think I far outdid myself by a Longshot, The Realism is insane the Quality is insane, And yeah I think my Problem before was Having too many Full body Images in the Dataset, This time I had only 3, Total 26 Images.
3000Steps with Differential Guidance, Linear 32, Linear Alpha 16, Dataset was all 4K, captions were like your Example. I trained on 512p and 1024p since I was reading Krea 2's Documentation and the Model was Trained on 256, 512 and 1024 resolution steps, and my Dataset was perfectly Fitting for that, so I did 512 and 1024, skipped 768 since The last attempt was including that and that didn't turn out so well.

The Character is insanely Flexible, and is Coherent with their Natural look. I have tried insane Yoga poses and Complex Dimensional Drifting. Even when out of Camera's Focus their Features can be discerned just clearly.
I think my Config might be Kinda better than Malcolm's lol 😉

But there was one downside, It Considers an Upper-Body shot as just a Photo of the Upper Body no face only from Chin and down. Answer to this was making the Prompt Brief. and if In a Short prompt just say Natural look of the character, The Upper-Body Problem might have come from my Dataset, I have an Idea where and why.

Thanks a lot bro.

you guys should try https://github.com/shootthesound/Fizgig , i've tested it for krea2 only and it's awesome . Made one of myself to test it and i got better results than civi , Aitoolkit i did't test on krea 2 . ~8s/it on a 3060 12GB , ultra fast settings ( i think rank 8 lora ) .

With my setup—an RTX 5070ti and 64GB of DDR4 RAM (bought when they had human prices)—with the settings, yaml file, and the type of step/dataset I use, namely 15 images and 1200 steps, I can train a LoRA character in about 35 minutes at 1.65 s/it with 15,1 gb usage, but I had to do crazy optimizations to the operating system, crazy stuff to save possible vram stolen by the operating system. I noticed that by keeping the terminal closed in a icon, I get a 27% increase performance. It's crazy, I realized it after years. Just having the terminal open, both during training and in ComfyUI, eats up an insane amount of VRAM. In fact, if I keep the terminal open, I can get up from 1.65 to 2.1s/it without any problems.
Generally, these are my values, i try to get the best possible result combined with the best possible speed, so I can train multiple LoRAs per day. I do about 3 per day
For LoRA characters
15 images - 1200 steps (my favorite, good results, 35 minutes of calculation)
20 images - 1500 steps
25 images - 1800 steps

For poses
15 images - 1500 steps with linear and linear alpha raised to 64
20 images - 1800 steps
If you're interested in learning about my entire setup, I wrote a beginner's guide on Reddit a few days ago. I'm not a super expert, and these are just the result of testing, but they can help.
https://www.reddit.com/r/StableDiffusion/comments/1vt6gck/guide_training_krea_2_character_pose_loras_with/

Sooner or later I would like to try to train a character for LTX-2.3, but I don't know if I can do it with 16 GB and above all I think I'll spend all night calculating