Description
Hello！I am training ip-adapter-plus-face_stxl-vit-h, but I am encountering some issues.
My train code
tutorial_train_plus.py
Config
pretrained_model_name_or_path=“aamXLAnimeMix_v10HalfturboEulera/”
pretrained_ip_adapter_path=“ip-adapter-plus-face_sdxl_vit-h.bin”
image_encoder_path=“models/image_encoder: [OpenCLIP-ViT-H-14]”
Error
Shapes of 'image_proj.latents' in checkpoint ipadapter-train/model/ip-adapter-plus-face_sdxl_vit-h.bin and current model do not match.
Removing 'latents' from checkpoint and loading the rest of the weights.
[rank1]: Traceback (most recent call last):
[rank1]: File "/ipadapter-train/train-process/IP-Adapter-finetune-kj/tutorial_train_plus.py", line 564, in
[rank1]: main()
[rank1]: File "/ipadapter-train/train-process/IP-Adapter-finetune-kj/tutorial_train_plus.py", line 427, in main
[rank1]: ip_adapter = IPAdapter(unet, image_proj_model, adapter_modules, args.pretrained_ip_adapter_path)
[rank1]: File "/ipadapter-train/train-process/IP-Adapter-finetune-kj/tutorial_train_plus.py", line 162, in init
[rank1]: self.load_from_checkpoint(ckpt_path)
[rank1]: File "ipadapter-train/train-process/IP-Adapter-finetune-kj/tutorial_train_plus.py", line 189, in load_from_checkpoint
[rank1]: self.image_proj_model.load_state_dict(state_dict["image_proj"], strict=strict_load_image_proj_model)
[rank1]: File "/opt/conda/envs/comfyui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 2584, in load_state_dict
[rank1]: raise RuntimeError(
[rank1]: RuntimeError: Error(s) in loading state_dict for Resampler:
[rank1]: size mismatch for proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 1280]).
[rank1]: size mismatch for proj_in.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for proj_out.weight: copying a param with shape torch.Size([2048, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 2048]).
[rank1]: size mismatch for layers.0.0.norm1.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.0.0.norm1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.0.0.norm2.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.0.0.norm2.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.0.0.to_q.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([768, 2048]).
[rank1]: size mismatch for layers.0.0.to_kv.weight: copying a param with shape torch.Size([2560, 1280]) from checkpoint, the shape in current model is torch.Size([1536, 2048]).
[rank1]: size mismatch for layers.0.0.to_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 768]).
[rank1]: size mismatch for layers.0.1.0.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.0.1.0.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.0.1.1.weight: copying a param with shape torch.Size([5120, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 2048]).
[rank1]: size mismatch for layers.0.1.3.weight: copying a param with shape torch.Size([1280, 5120]) from checkpoint, the shape in current model is torch.Size([2048, 8192]).
[rank1]: size mismatch for layers.1.0.norm1.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.1.0.norm1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.1.0.norm2.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.1.0.norm2.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is
646B
torch.Size([2048]).
[rank1]: size mismatch for layers.1.0.to_q.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([768, 2048]).
[rank1]: size mismatch for layers.1.0.to_kv.weight: copying a param with shape torch.Size([2560, 1280]) from checkpoint, the shape in current model is torch.Size([1536, 2048]).
[rank1]: size mismatch for layers.1.0.to_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 768]).
[rank1]: size mismatch for layers.1.1.0.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.1.1.0.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.1.1.1.weight: copying a param with shape torch.Size([5120, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 2048]).
[rank1]: size mismatch for layers.1.1.3.weight: copying a param with shape torch.Size([1280, 5120]) from checkpoint, the shape in current model is torch.Size([2048, 8192]).
[rank1]: size mismatch for layers.2.0.norm1.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.2.0.norm1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.2.0.norm2.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.2.0.norm2.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.2.0.to_q.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([768, 2048]).
[rank1]: size mismatch for layers.2.0.to_kv.weight: copying a param with shape torch.Size([2560, 1280]) from checkpoint, the shape in current model is torch.Size([1536, 2048]).
[rank1]: size mismatch for layers.2.0.to_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 768]).
[rank1]: size mismatch for layers.2.1.0.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.2.1.0.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.2.1.1.weight: copying a param with shape torch.Size([5120, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 2048]).
[rank1]: size mismatch for layers.2.1.3.weight: copying a param with shape torch.Size([1280, 5120]) from checkpoint, the shape in current model is torch.Size([2048, 8192]).
[rank1]: size mismatch for layers.3.0.norm1.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.3.0.norm1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.3.0.norm2.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.3.0.norm2.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.3.0.to_q.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([768, 2048]).
[rank1]: size mismatch for layers.3.0.to_kv.weight: copying a param with shape torch.Size([2560, 1280]) from checkpoint, the shape in current model is torch.Size([1536, 2048]).
[rank1]: size mismatch for layers.3.0.to_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 768]).
[rank1]: size mismatch for layers.3.1.0.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.3.1.0.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
[rank1]: size mismatch for layers.3.1.1.weight: copying a param with shape torch.Size([5120, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 2048]).
I have tried many methods, but still encountered an error. I hope you can provide a solution. Very Thanks！！！！ @xiaohu2015