Train

token name : silhume

!accelerate launch train_dreambooth_lora_sdxl_mod.py \\
  --pretrained_model_name_or_path=$MODEL_NAME  \\
  --pretrained_vae_model_name_or_path="madebyollin/sdxl-vae-fp16-fix" \\
  --instance_data_dir=$INSTANCE_DIR \\
  --class_data_dir=$CLASS_DIR \\
  --output_dir=$OUTPUT_DIR \\
  --with_prior_preservation --prior_loss_weight=1.0 \\
  --mixed_precision="fp16" \\
  --instance_prompt="a photo of silhume woman" \\
  --class_prompt="A photo of woman" \\
  --resolution=512 \\
  --train_batch_size=4 \\
  --gradient_accumulation_steps=2 --gradient_checkpointing \\
  --learning_rate=1e-4 \\
  --lr_scheduler="constant" \\
  --lr_warmup_steps=0 \\
  --max_train_steps=1000 \\
  --num_class_images=200 \\
  --sample_batch_size=2 \\
  --seed=1337 \\
  --train_text_encoder

seconds per iteration : 3초

trained_time : 51m

Instance image

num : 6

target : 고마츠 나나

Inference

Backbone model : "stabilityai/stable-diffusion-xl-base-0.9”

pipeline : StableDiffusionXLPipeline

refine : X

Inference Image

n_steps = 50
prompt = "A photo of silhume woman in blue ocean, beautiful sky and palm trees"

n_steps = 100
prompt = "Create a stunning album cover with a silhume woman standing in the moonlight."
#prompt by ChatGPT, "write me a good prompt to generate woman stand in moon, and It's like a album cover. but prompt should not be over 30 words."