• Lora训练Windows[笔记]

    一. 使用kohya_ss的GUI版本（https://github.com/bmaltais/kohya_ss.git）

    这个版本跟stable-diffusion-webui的界面很像，只不过是训练模型专用而已，打开的端口同样是7860。

    1.双击setup.bat,选择1安装好xformers,pytorch等和cuda相关的库，然后可以control+C退出.将requirements.txt里面的内容除了“-e .”外复制到req.txt,然后在虚拟环境下（{venv}\Scripts=E:\SD_WIN\kohya_ss\venv\Scripts）执行下面代码加速安装：

    1. pip install -r {xxx}/req.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
    2.

    安装结束后，重新双击setup.bat并选择1，查缺补漏。

    2.双击gui.bat运行，这个和stable-diffusion-webui不一样，不会自动打开浏览器的。自行在chrome上输入“http://127.0.0.1:7860/”.

    注意：默认选择的是dreambooth,不要一上来就设置哦，那玩意非常大，通常一次7G+，默认来20次，哈哈
    3.选择Lora标签，SD1.5模块不需要考虑Accelerate launch，设置model，注意“Pretrained model name or path”最好先点 选择好路径，然后点 选好模型。类似这样
    值得一提的是图片和数据应该是放一块的，比如C:/database/1_images/ 里面的数据应该是1个图片对应一个text文本，类似这样：
    这个另外图中的pr是模块输出的文件名，我这是测试，随便按的。
    如果不需要调整啥参数，默认是值就是给SD1.5用的，所以我跳过“Parameters”等，只需要填写好“Folders”
    主要是填好输出路径，logs路径随便填。
    4.start training
    ==========以上是SD-Scripts GUI版本训练SD1.5的测试================
    本来我也想测一下SDXL版本的Lora训练的，奈何机器不行，机器显存8G+，内存16G+的，一直出错，可以考虑改一下“Accelerate launch”的“Mixed precision”=“no”，我这边改了后还是没法通过，主要还是机器问题，out of memory，哈哈 
    1.  
    2. 19:07:38-166454 INFO     Start training LoRA Standard ...
    3. 19:07:38-167453 INFO     Validating lr scheduler arguments...
    4. 19:07:38-168449 INFO     Validating optimizer arguments...
    5. 19:07:38-169446 INFO     Validating E:/SD_WIN/kohya_ss/logs existence and writability... SUCCESS
    6. 19:07:38-171441 INFO     Validating E:/SD_WIN/kohya_ss/outputs existence and writability... SUCCESS
    7. 19:07:38-172439 INFO     Validating E:/SD_WIN/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1.0.safetensors
    8.                          existence... SUCCESS
    9. 19:07:38-173436 INFO     Validating C:/sdxl existence... SUCCESS
    10. 19:07:38-174433 INFO     Folder 1_images: 1 repeats found
    11. 19:07:38-186400 INFO     Folder 1_images: 8 images found
    12. 19:07:38-187399 INFO     Folder 1_images: 8 * 1 = 8 steps
    13. 19:07:38-188396 INFO     Regulatization factor: 1
    14. 19:07:38-189394 INFO     Total steps: 8
    15. 19:07:38-190389 INFO     Train batch size: 1
    16. 19:07:38-191387 INFO     Gradient accumulation steps: 1
    17. 19:07:38-192384 INFO     Epoch: 1
    18. 19:07:38-193383 INFO     Max train steps: 1600
    19. 19:07:38-193383 INFO     stop_text_encoder_training = 0
    20. 19:07:38-194380 INFO     lr_warmup_steps = 160
    21. 19:07:38-221307 INFO     Saving training config to E:/SD_WIN/kohya_ss/outputs\pr1_sdxl_20240515-190738.json...
    22. 19:07:38-256213 INFO     Executing command: E:\SD_WIN\kohya_ss\venv\Scripts\accelerate.EXE launch --dynamo_backend no
    23.                          --dynamo_mode default --mixed_precision no --num_processes 1 --num_machines 1
    24.                          --num_cpu_threads_per_process 2 E:/SD_WIN/kohya_ss/sd-scripts/sdxl_train_network.py
    25.                          --config_file E:/SD_WIN/kohya_ss/outputs/config_lora-20240515-190738.toml
    26. 19:07:38-263218 INFO     Command executed.
    27. 2024-05-15 19:08:39 INFO     Loading settings from                                                    train_util.py:3744
    28.                              E:/SD_WIN/kohya_ss/outputs/config_lora-20240515-190738.toml...
    29.                     INFO     E:/SD_WIN/kohya_ss/outputs/config_lora-20240515-190738                   train_util.py:3763
    30. 2024-05-15 19:08:39 INFO     prepare tokenizers                                                   sdxl_train_util.py:134
    31. 2024-05-15 19:08:41 INFO     update token length: 75                                              sdxl_train_util.py:159
    32.                     INFO     Using DreamBooth method.                                               train_network.py:172
    33.                     INFO     prepare images.                                                          train_util.py:1572
    34.                     INFO     found directory C:\sdxl\1_images contains 8 image files                  train_util.py:1519
    35.                     WARNING  No caption file found for 1 images. Training will continue without       train_util.py:1550
    36.                              captions for these images. If class token exists, it will be used. /
    37.                              1枚の画像にキャプションファイルが見つかりませんでした。これらの画像につ
    38.                              いてはキャプションなしで学習を続行します。class
    39.                              tokenが存在する場合はそれを使います。
    40.                     WARNING  C:\sdxl\1_images\videoplayback[(000859)2023-11-09-22-17-15].jpg          train_util.py:1557
    41.                     INFO     8 train images with repeating.                                           train_util.py:1613
    42.                     INFO     0 reg images.                                                            train_util.py:1616
    43.                     WARNING  no regularization images / 正則化画像が見つかりませんでした              train_util.py:1621
    44.                     INFO     [Dataset 0]                                                              config_util.py:565
    45.                                batch_size: 1
    46.                                resolution: (1024, 1024)
    47.                                enable_bucket: True
    48.                                network_multiplier: 1.0
    49.                                min_bucket_reso: 256
    50.                                max_bucket_reso: 2048
    51.                                bucket_reso_steps: 64
    52.                                bucket_no_upscale: True
    53.  
    54.                                [Subset 0 of Dataset 0]
    55.                                  image_dir: "C:\sdxl\1_images"
    56.                                  image_count: 8
    57.                                  num_repeats: 1
    58.                                  shuffle_caption: False
    59.                                  keep_tokens: 0
    60.                                  keep_tokens_separator:
    61.                                  secondary_separator: None
    62.                                  enable_wildcard: False
    63.                                  caption_dropout_rate: 0.0
    64.                                  caption_dropout_every_n_epoches: 0
    65.                                  caption_tag_dropout_rate: 0.0
    66.                                  caption_prefix: None
    67.                                  caption_suffix: None
    68.                                  color_aug: False
    69.                                  flip_aug: False
    70.                                  face_crop_aug_range: None
    71.                                  random_crop: False
    72.                                  token_warmup_min: 1,
    73.                                  token_warmup_step: 0,
    74.                                  is_reg: False
    75.                                  class_tokens: images
    76.                                  caption_extension: .txt
    77.  
    78.  
    79.                     INFO     [Dataset 0]                                                              config_util.py:571
    80.                     INFO     loading image sizes.                                                      train_util.py:853
    81. 100%|██████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 2025.13it/s]
    82.                     INFO     make buckets                                                              train_util.py:859
    83.                     WARNING  min_bucket_reso and max_bucket_reso are ignored if bucket_no_upscale is   train_util.py:876
    84.                              set, because bucket reso is defined by image size automatically /
    85.                              bucket_no_upscaleが指定された場合は、bucketの解像度は画像サイズから自動計
    86.                              算されるため、min_bucket_resoとmax_bucket_resoは無視されます
    87.                     INFO     number of images (including repeats) /                                    train_util.py:905
    88.                              各bucketの画像枚数（繰り返し回数を含む）
    89.                     INFO     bucket 0: resolution (1024, 1024), count: 8                               train_util.py:910
    90.                     INFO     mean ar error (without repeats): 0.0                                      train_util.py:915
    91.                     WARNING  clip_skip will be unexpected / SDXL学習ではclip_skipは動作しません   sdxl_train_util.py:343
    92.                     INFO     preparing accelerator                                                  train_network.py:225
    93. accelerator device: cuda
    94.                     INFO     loading model for process 0/1                                         sdxl_train_util.py:30
    95.                     INFO     load StableDiffusion checkpoint:                                      sdxl_train_util.py:70
    96.                              E:/SD_WIN/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1
    97.                              .0.safetensors
    98. 2024-05-15 19:08:47 INFO     building U-Net                                                       sdxl_model_util.py:192
    99.                     INFO     loading U-Net from checkpoint                                        sdxl_model_util.py:196
    100. 2024-05-15 19:11:37 INFO     U-Net:                                sdxl_model_util.py:202
    101. 2024-05-15 19:11:38 INFO     building text encoders                                               sdxl_model_util.py:205
    102. 2024-05-15 19:11:41 INFO     loading text encoders from checkpoint                                sdxl_model_util.py:258
    103. 2024-05-15 19:11:47 INFO     text encoder 1:                       sdxl_model_util.py:272
    104. 2024-05-15 19:12:15 INFO     text encoder 2:                       sdxl_model_util.py:276
    105.                     INFO     building VAE                                                         sdxl_model_util.py:279
    106. 2024-05-15 19:12:19 INFO     loading VAE from checkpoint                                          sdxl_model_util.py:284
    107. 2024-05-15 19:12:23 INFO     VAE:                                  sdxl_model_util.py:287
    108. 2024-05-15 19:12:36 INFO     Enable xformers for U-Net                                                train_util.py:2660
    109. import network module: networks.lora
    110. 2024-05-15 19:12:40 INFO     [Dataset 0]                                                              train_util.py:2079
    111.                     INFO     caching latents.                                                          train_util.py:974
    112.                     INFO     checking cache validity...                                                train_util.py:984
    113. 100%|████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00
    114.                     INFO     caching latents...                                                       train_util.py:1021
    115. 100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:37<00:00,  4.75s/it]
    116. 2024-05-15 19:13:19 INFO     create LoRA network. base dim (rank): 8, alpha: 1                               lora.py:810
    117.                     INFO     neuron dropout: p=None, rank dropout: p=None, module dropout: p=None            lora.py:811
    118.                     INFO     create LoRA for Text Encoder 1:                                                 lora.py:902
    119.                     INFO     create LoRA for Text Encoder 2:                                                 lora.py:902
    120. 2024-05-15 19:13:20 INFO     create LoRA for Text Encoder: 264 modules.                                      lora.py:910
    121.                     INFO     create LoRA for U-Net: 722 modules.                                             lora.py:918
    122.                     INFO     enable LoRA for text encoder                                                    lora.py:961
    123.                     INFO     enable LoRA for U-Net                                                           lora.py:966
    124. prepare optimizer, data loader etc.
    125. 2024-05-15 19:13:24 INFO     use 8-bit AdamW optimizer | {}                                           train_util.py:3889
    126. Traceback (most recent call last):
    127.   File "E:\SD_WIN\kohya_ss\sd-scripts\sdxl_train_network.py", line 185, in 
    128.     trainer.train(args)
    129.   File "E:\SD_WIN\kohya_ss\sd-scripts\train_network.py", line 429, in train
    130.     unet = accelerator.prepare(unet)
    131.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\accelerate\accelerator.py", line 1213, in prepare
    132.     result = tuple(
    133.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\accelerate\accelerator.py", line 1214, in 
    134.     self._prepare_one(obj, first_pass=True, device_placement=d) for obj, d in zip(args, device_placement)
    135.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\accelerate\accelerator.py", line 1094, in _prepare_one
    136.     return self.prepare_model(obj, device_placement=device_placement)
    137.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\accelerate\accelerator.py", line 1334, in prepare_model
    138.     model = model.to(self.device)
    139.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\torch\nn\modules\module.py", line 1160, in to
    140.     return self._apply(convert)
    141.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    142.     module._apply(fn)
    143.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    144.     module._apply(fn)
    145.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    146.     module._apply(fn)
    147.   [Previous line repeated 6 more times]
    148.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\torch\nn\modules\module.py", line 833, in _apply
    149.     param_applied = fn(param)
    150.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\torch\nn\modules\module.py", line 1158, in convert
    151.     return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
    152. torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 50.00 MiB. GPU 0 has a total capacty of 4.00 GiB of which 0 bytes is free. Of the allocated memory 10.68 GiB is allocated by PyTorch, and 226.95 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
    153. Traceback (most recent call last):
    154.   File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    155.     return _run_code(code, main_globals, None,
    156.   File "C:\Python310\lib\runpy.py", line 86, in _run_code
    157.     exec(code, run_globals)
    158.   File "E:\SD_WIN\kohya_ss\venv\Scripts\accelerate.EXE\__main__.py", line 7, in 
    159.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\accelerate\commands\accelerate_cli.py", line 47, in main
    160.     args.func(args)
    161.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 1017, in launch_command
    162.     simple_launcher(args)
    163.   File "E:\SD_WIN\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.py", line 637, in simple_launcher
    164.     raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
    165. subprocess.CalledProcessError: Command '['E:\\SD_WIN\\kohya_ss\\venv\\Scripts\\python.exe', 'E:/SD_WIN/kohya_ss/sd-scripts/sdxl_train_network.py', '--config_file', 'E:/SD_WIN/kohya_ss/outputs/config_lora-20240515-190738.toml']' returned non-zero exit status 1.
    166. 19:16:29-337912 INFO     Training has ended.

    二、ComfyUI之插件Lora-Training-in-Comfy（https://github.com/LarryJane491/Lora-Training-in-Comfy.git），顺道也安装一下Image-Captioning-in-ComfyUI（https://github.com/LarryJane491/Image-Captioning-in-ComfyUI.git）和Vector_Sculptor_ComfyUI（https://github.com/Extraltodeus/Vector_Sculptor_ComfyUI.git）

    在“custom_nodes”下clone它下来，重启安装，一般大概率是没法顺顺利利的，自行安装一些库，我这边列一下xformers和pytorch几个需要注意的库，其他的随意吧

    1. accelerate                0.29.3
    2. library                   0.0.0              E:\SD_WIN\ComfyUI_windows_portable\ComfyUI\custom_nodes\Lora-Training-in-Comfy\sd-scripts
    3. torch                     2.3.0+cu121
    4. torchaudio                2.3.0+cu121
    5. torchvision               0.18.0+cu121
    6. xformers                  0.0.26.post1

    xformers优先安装，使用

    1. {venv}/Scripts/pip.exe install xformers --index-url --index-url https://download.pytorch.org/whl/cu121
    2. 然后根据pytorch的版本提示安装torchaudio和torchvision
    3.  
    4. 我的例子：
    5.  
    6. {venv}/Scripts/pip.exe install xformers==0.0.26.post1 torch==2.3.0+cu121 torchaudio==2.3.0+cu121 torchvision==0.18.0+cu121 --index-url https://download.pytorch.org/whl/cu121

    其次要到“custom_nodes/Lora-Training-in-Comfy/sd-scripts/library”目录下运行

    {venv}/Scripts/pip.exe install -e .

    后面这一步没做的话，可能会遇到library模块加不来，要是直接用线上的安装就傻眼了，大概率是对不上号的。

    重新双击run_nvidia_gpu.bat运行ComfyUI,添加节点“LJRE/LORA/LORA training in ComfyUI”，SD1.5的LORA只需要改三个配置就可以运行了。

    output_dir最好是相对run_nvidia_gpu.bat所在的路径，这样得到的lora不需要复制，重启ComfyUI就可以测试。

    这个插件有个大问题，就是很多机器没法正常运行，哈哈，没错，是真的。我建议有两点：

    1.更新sd-scripts,将原来删了，在同路径下运行

    1. ​
    2. git clone https://github.com/kohya-ss/sd-scripts.git
    3.  
    4. ​

    安装参考上面

    2.修改train.py。 搜索"python -m accelerate"，改为“{vevn-path}/python.exe -m accelerate”，vevn-path应该是run_nvidia_gpu.bat同目录下的python_embeded的绝对路径。（注：下载一键安装包，要是clone的版本应该自己知道venv路径的）

    最后补充一张根据图片提取文本的流程图

    用到Comfyui_image2prompt（https://github.com/zhongpei/Comfyui_image2prompt.git）,这玩意要是完整几乎不太可能，低端机器下wd-swinv2-tagger-v3-hf足够了。等有空再聊聊这个插件的安装经历。

    当然也可以安装其他的插件代替的，WD14是不太可能了，还有其他的插件可以考虑。

  • 相关阅读:
    鸿蒙（API 12 Beta3版）【获取支持的编解码能力】 音视频编码
    Hazel引擎学习（九）
    TiDB Lightning 命令行参数
    Nginx配置实例——反向代理
    CTF-sql注入(get)【简单易懂】
    chatgpt赋能python：Python如何快速取出所有元素？
    Hive SQL初级练习（30题）
    Node简介
    第3章 Tomcat&servlet
    STM32 寄存器配置笔记——USART配置 打印
  • 原文地址：https://blog.csdn.net/ftpleopard/article/details/138920326