Train

token name : silhumil

!accelerate launch train_dreambooth_lora_sdxl_mod.py \\
  --pretrained_model_name_or_path=$MODEL_NAME  \\
  --pretrained_vae_model_name_or_path="madebyollin/sdxl-vae-fp16-fix" \\
  --instance_data_dir=$INSTANCE_DIR \\
  --class_data_dir=$CLASS_DIR \\
  --output_dir=$OUTPUT_DIR \\
  --with_prior_preservation --prior_loss_weight=1.0 \\
  --mixed_precision="fp16" \\
  --instance_prompt="a photo of silhumil man" \\
  --class_prompt="A photo of man" \\
  --resolution=512 \\
  --train_batch_size=2 \\
  --gradient_accumulation_steps=4 --gradient_checkpointing \\
  --learning_rate=1e-4 \\
  --lr_scheduler="constant" \\
  --lr_warmup_steps=0 \\
  --max_train_steps=500 \\
  --num_class_images=200 \\
  --sample_batch_size=2 \\
  --seed=1337 \\
  --train_text_encoder

seconds per iteration : 6초 초반

trained_time : 48m

Instance image

num : 4

target : Eminem

Inference

Backbone model : "stabilityai/stable-diffusion-xl-base-0.9”

pipeline : StableDiffusionXLPipeline

n_steps = 50
prompt = "A photo of silhumil man singing ballad in concert"

refine : X

Inference Image

n_steps = 50