스타일 자체를 학습하는 것은 가능할까?

token name : silsam

!accelerate launch train_dreambooth_lora_sdxl_mod.py \\
  --pretrained_model_name_or_path=$MODEL_NAME  \\
  --pretrained_vae_model_name_or_path="madebyollin/sdxl-vae-fp16-fix" \\
  --instance_data_dir=$INSTANCE_DIR \\
  --class_data_dir=$CLASS_DIR \\
  --output_dir=$OUTPUT_DIR \\
  --with_prior_preservation --prior_loss_weight=1.0 \\
  --mixed_precision="fp16" \\
  --instance_prompt="A photo of silsam album cover" \\
  --class_prompt="A photo of album cover" \\
  --resolution=512 \\
  --train_batch_size=2 \\
  --gradient_accumulation_steps=1 --gradient_checkpointing \\
  --learning_rate=1e-4 \\
  --lr_scheduler="constant" \\
  --lr_warmup_steps=0 \\
  --max_train_steps=1000 \\
  --num_class_images=200 \\
  --sample_batch_size=2 \\
  --seed=1337 \\
  --train_text_encoder \\
  --push_to_hub \\
  --hub_model_id="IU_album_cover"

seconds per iteration : 1.21

trained_time : 20m

Instance image

num : 5

target : IU 앨범 커버들

Inference

Backbone model : "stabilityai/stable-diffusion-xl-base-0.9”

pipeline : StableDiffusionXLPipeline

n_steps = 50
prompt = "A photo of Emma Watson in silsam album cover style"

Inference Image

아이유랑 얼굴이 많이 섞여서 엠마 왓슨 같지는 않은 것 같음..