Train

token name : silhumil

!accelerate launch train_dreambooth_lora_sdxl_mod.py \\
  --pretrained_model_name_or_path=$MODEL_NAME  \\
  --pretrained_vae_model_name_or_path="madebyollin/sdxl-vae-fp16-fix" \\
  --instance_data_dir=$INSTANCE_DIR \\
  --class_data_dir=$CLASS_DIR \\
  --output_dir=$OUTPUT_DIR \\
  --with_prior_preservation --prior_loss_weight=1.0 \\
  --mixed_precision="fp16" \\
  --instance_prompt="a photo of silhumil man" \\
  --class_prompt="A photo of man" \\
  --resolution=512 \\
  --train_batch_size=4 \\
  --gradient_accumulation_steps=1 --gradient_checkpointing \\
  --learning_rate=1e-4 \\
  --lr_scheduler="constant" \\
  --lr_warmup_steps=0 \\
  --max_train_steps=1000 \\
  --num_class_images=200 \\
  --sample_batch_size=1 \\
  --seed=1337 \\
  --train_text_encoder

seconds per iteration : 1.53s

trained_time : 26m

Instance image

num : 4

target : eminem (exp1과 같음)

Inference

Backbone model : "stabilityai/stable-diffusion-xl-base-0.9”

pipeline : StableDiffusionXLPipeline

n_steps = 50
prompt = "A photo of silhumil man standing in the moonlight."

달 자체가 없음..?

있어도 작고 얼굴을 잘 그리지 못함. 동양인 이미지 위주 학습이 되어 있어 서양인을 잘 못 그리는게 맞는 것 같음

Inference Image