ff7r_4e-5cosine_everything-cont-20-ep50-3e-7_bs1-ep04-half / ff7r_4e-5cosine_everything-cont-20-ep50-3e-7_bs1-20241228-170700.log

damian0815

Upload folder using huggingface_hub

f8b4557 verified over 1 year ago

Raw

History Blame Contribute Delete

3.23 kB

	12/28/2024 05:07:00 PM [93m Disabling AMP, not recommended.[0m
	12/28/2024 05:07:00 PM Seed: 756498848
	12/28/2024 05:07:00 PM unet attention_head_dim: 8
	12/28/2024 05:07:00 PM Inferred yaml: v1-inference.yaml, attn: sd1, prediction_type: epsilon
	12/28/2024 05:07:01 PM * Using default (DDPM) noise scheduler for training: ddpm
	12/28/2024 05:07:01 PM * Using SDP attention *
	12/28/2024 05:07:01 PM * DLMA resolution 512, buckets: [[512, 512], [576, 448], [448, 576], [640, 384], [384, 640], [768, 320], [320, 768], [896, 256], [256, 896], [1024, 256], [256, 1024]]
	12/28/2024 05:07:01 PM Preloading images...
	12/28/2024 05:07:06 PM * Loaded 40 validation images for validation set 'val' from ../data/val
	12/28/2024 05:07:06 PM * DLMA initialized with 40 images.
	12/28/2024 05:07:06 PM ** Dataset 'val': 40 batches, num_images: 40, batch_size: 1
	12/28/2024 05:07:06 PM * [91mAspect ratio bucket ('default_batch', 768, 320) has only 1 images[0m. At batch size 1 this makes for an effective multiplier of 2.0, which may cause problems. Consider adding 1 or more images with aspect ratio 12:5, or reducing your batch_size.
	12/28/2024 05:07:06 PM - Plugin plugins.interruptible.InterruptiblePlugin loaded to <class 'plugins.interruptible.InterruptiblePlugin'>
	12/28/2024 05:07:06 PM * DLMA initialized with 1913 images.
	12/28/2024 05:07:06 PM ** Dataset 'train': 1913 batches, num_images: 1913, batch_size: 1
	12/28/2024 05:07:06 PM [36m * unet optimizer: AdamW (686 parameters) *[0m
	12/28/2024 05:07:06 PM [36m lr: 3e-07, betas: [0.9, 0.99], epsilon: 1e-08, weight_decay: 0.01 *[0m
	12/28/2024 05:07:06 PM Grad scaler enabled: False (amp mode)
	12/28/2024 05:07:06 PM Pretraining GPU Memory: 5102 / 24564 MB
	12/28/2024 05:07:06 PM saving ckpts every 1000000000.0 minutes
	12/28/2024 05:07:06 PM saving ckpts every 4 epochs
	12/28/2024 05:07:06 PM unet device: cuda:0, precision: torch.float32, training: True
	12/28/2024 05:07:06 PM text_encoder device: cuda:0, precision: torch.float32, training: False
	12/28/2024 05:07:06 PM vae device: cuda:0, precision: torch.float32, training: False
	12/28/2024 05:07:06 PM scheduler: <class 'diffusers.schedulers.scheduling_ddpm.DDPMScheduler'>
	12/28/2024 05:07:06 PM [32mProject name: [0m[92mff7r_4e-5cosine_everything-cont-20-ep50-3e-7_bs1[0m
	12/28/2024 05:07:06 PM [32mgrad_accum: [0m[92m1[0m
	12/28/2024 05:07:06 PM [32mbatch_size: [0m[92m1[0m
	12/28/2024 05:07:06 PM [32mepoch_len: [92m1913[0m
	12/28/2024 05:45:02 PM * Saving diffusers model to ../logs/ff7r_4e-5cosine_everything-cont-20-ep50-3e-7_bs1-20241228-170700/ckpts/ff7r_4e-5cosine_everything-cont-20-ep50-3e-7_bs1-ep04-gs07652
	12/28/2024 05:45:04 PM Saving optimizer state to ../logs/ff7r_4e-5cosine_everything-cont-20-ep50-3e-7_bs1-20241228-170700/ckpts/ff7r_4e-5cosine_everything-cont-20-ep50-3e-7_bs1-ep04-gs07652
	12/28/2024 05:45:08 PM [36mTraining complete[0m
	12/28/2024 05:45:08 PM Total training time took 38.03 minutes, total steps: 7652
	12/28/2024 05:45:08 PM Average epoch time: 9.46 minutes
	12/28/2024 05:45:08 PM [97m ***************************[0m
	12/28/2024 05:45:08 PM [97m ** Finished training **[0m
	12/28/2024 05:45:08 PM [97m ***************************[0m