• lora训练之偷师

    自stable diffusion开源之后AIGC绘画方向定制化百花齐放百家争鸣。
    而c站 https://civitai.com/

    也聚集了全球爱好者的各种微调训练模型分享。

    其中以lora为首，应用最广泛。

    而这些模型是怎么训练出来的，细节到底是什么样的，没有多少人能说得清楚。

    而就博主知道的，

    使用最广泛的训练代码是这份 https://github.com/bmaltais/kohya_ss

    它流行的原因主要是在其做了个GUI界面，便于很多不懂代码的人也能使用。

    这里博主没打算讲解kohya_ss下具体训练方法和其参数。

    想知道具体训练参数和细节，可以移步其wiki

    https://github.com/bmaltais/kohya_ss/wiki/LoRA-training-parameters

    还是不懂，也可以问各种gpt或者直接翻译成中文阅读即可。

    现在主流最广泛使用的模型存储格式为 huggingface的safetensors

    https://github.com/huggingface/safetensors

    kohya_ss里默认会把训练参数填到safetensors的metadata里。

    说到这里，lora训练之偷师 答案呼之欲出。

    假设我们从c站上把safetensors模型下载下来，

    然后读出来它的metadata信息,

    那不就能知道这个模型具体的训练参数吗？

    而在博主实操读出来好几个主流模型的参数之后，很感慨:

    c站上的模型真的很demo，训练的数据集大多都不大，分辨率也不高，有的也就百来张图。

    训练参数也可以大概评估出来，每个模型作者的用心程度，

    当然也可以作为评价模型的一种标准，

    例如：看过了多少张图，什么样的分辨率训练的，训练了多久之类的。

    也就对这个微调的模型能有个大概认知。

    多看几个模型之后，

    新手也可以快速上手训练自己的lora，定制自己的AI女友之类的。

    你说你想直接抄他们的参数训练，当然也是可以的，偷师不可耻很光荣。

    为了方便读取模型的模型的训练信息，博主写了个小工具。

    开源地址为:

    https://github.com/cpuimage/st_metadata_parser

    使用也非常简单:

    python st_metadata_parser.py input_file.safetensors

    输出的参数具体含义可以参考：
    https://github.com/bmaltais/kohya_ss/wiki/LoRA-training-parameters

    附一份C站上某个lora模型文件的训练参数供一览：

    ?
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    68
    69
    70
    71
    72
    73
    74
    75
    76
    77
    78
    79
    80
    81
    82
    83
    84
    85
    86
    87
    88
    89
    90
    91
    92
    93
    94
    95
    96
    97
    98
    99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    133
    [('ss_cache_latents', 'True'),
     ('ss_caption_dropout_every_n_epochs', '0'),
     ('ss_caption_dropout_rate', '0.0'),
     ('ss_caption_tag_dropout_rate', '0.0'),
     ('ss_clip_skip', '2'),
     ('ss_dataset_dirs', '{"rapunzel": {"n_repeats": 5, "img_count": 40}}'),
     ('ss_datasets',
      '[{"is_dreambooth": true, "batch_size_per_device": 2, "num_train_images": '
      '200, "num_reg_images": 0, "resolution": [512, 512], "enable_bucket": true, '
      '"min_bucket_reso": 256, "max_bucket_reso": 1024, "tag_frequency": '
      '{"rapunzel": {"rapunzel 1girl": 40, "solo": 39, "long hair": 37, "smile": '
      '26, "blonde hair": 38, "dress": 21, "very long hair": 18, "green eyes": 30, '
      '"crossed arms": 3, "corset": 5, "purple dress": 12, "breasts": 13, "open '
      'mouth": 7, "medium breasts": 5, "nipples": 7, "swimsuit": 2, "bikini": 1, '
      '"tongue": 2, "striped": 1, "lips": 7, "micro bikini": 1, "freckles": 5, '
      '"looking at viewer": 10, "long sleeves": 7, "cleavage": 3, "multiple boys": '
      '1, "puffy sleeves": 7, "2boys": 1, "full body": 4, "absurdly long hair": 2, '
      '"grin": 2, "hand on hip": 3, "juliet sleeves": 1, "blush": 3, "large '
      'breasts": 4, "hair ornament": 5, "standing": 2, "flower": 7, "cowboy shot": '
      '1, "teeth": 5, "indoors": 1, "hair flower": 5, "one-piece swimsuit": 1, '
      '"covered navel": 1, "makeup": 2, "cameltoe": 1, "highleg": 1, "lamp": 1, '
      '"braid": 4, "blurry": 2, "arm up": 1, "depth of field": 1, "blurry '
      'background": 2, "looking up": 3, "bokeh": 1, "upper body": 4, "short '
      'sleeves": 3, "closed eyes": 1, "small breasts": 1, "lying": 3, "signature": '
      '1, "on back": 1, "arms up": 2, "eyelashes": 1, "from above": 1, "happy": 1, '
      '"grass": 1, "cross-laced clothes": 1, "hair spread out": 1, "sitting": 4, '
      '"outdoors": 2, "barefoot": 7, "water": 1, "tree": 2, "pink dress": 1, '
      '"holding flower": 1, "lily pad": 1, "simple background": 5, "white '
      'background": 3, "frog": 4, "black background": 2, "child": 1, "star print": '
      '1, "lantern": 1, "blue eyes": 2, "navel": 5, "underwear": 2, "panties": 2, '
      '"thigh gap": 1, "bound": 2, "bdsm": 2, "bondage": 2, "gag": 1, "gagged": 1, '
      '"improvised gag": 1, "cloth gag": 1, "nude": 5, "tongue out": 1, "saliva": '
      '1, "arms behind back": 1, "seiza": 1, "bound arms": 1, "on head": 1, '
      '"animal on head": 1, "holding hair": 1, "tiara": 1, "crown": 1, "princess": '
      '1, "hands on own head": 1, "mirror": 2, "frying pan": 2, "nature": 1, '
      '"parted lips": 1, "blue background": 1, "shirt": 1, "choker": 2, "huge '
      'breasts": 1, "colored skin": 1, "bursting breasts": 1, "thighhighs": 2, '
      '"jewelry": 1, "thighs": 1, "earrings": 1, "pussy": 3, "spread legs": 2, '
      '"cum": 1, "high heels": 1, "kneeling": 1, "uncensored": 3, "cum in pussy": '
      '1, "phone": 1, "cellphone": 1, "smartphone": 1, "after sex": 1, "holding '
      'phone": 1, "cumdrip": 1, "brown thighhighs": 1, "gloves": 1, "multicolored '
      'hair": 1, "white gloves": 1, "topless": 1, "pink panties": 1, "fishnet '
      'thighhighs": 1, "ass": 1, "feet": 1, "soles": 1, "on stomach": 1, "the '
      'pose": 1, "nail polish": 1, "tan": 1, "personification": 1, "tanlines": 1, '
      '"detached sleeves": 1, "armpits": 1, "pillow": 1}}, "bucket_info": '
      '{"buckets": {"0": {"resolution": [320, 704], "count": 30}, "1": '
      '{"resolution": [384, 640], "count": 95}, "2": {"resolution": [448, 576], '
      '"count": 50}, "3": {"resolution": [512, 512], "count": 20}, "4": '
      '{"resolution": [576, 448], "count": 5}}, "mean_img_ar_error": '
      '0.042025975009129204}, "subsets": [{"img_count": 40, "num_repeats": 5, '
      '"color_aug": false, "flip_aug": false, "random_crop": false, '
      '"shuffle_caption": true, "keep_tokens": 1, "image_dir": "rapunzel", '
      '"class_tokens": null, "is_reg": false}]}]'),
     ('ss_epoch', '10'),
     ('ss_face_crop_aug_range', 'None'),
     ('ss_full_fp16', 'False'),
     ('ss_gradient_accumulation_steps', '1'),
     ('ss_gradient_checkpointing', 'False'),
     ('ss_learning_rate', '0.0005'),
     ('ss_lowram', 'True'),
     ('ss_lr_scheduler', 'cosine_with_restarts'),
     ('ss_lr_warmup_steps', '50'),
     ('ss_max_grad_norm', '1.0'),
     ('ss_max_token_length', '225'),
     ('ss_max_train_steps', '1010'),
     ('ss_min_snr_gamma', '5.0'),
     ('ss_mixed_precision', 'fp16'),
     ('ss_multires_noise_discount', '0.3'),
     ('ss_multires_noise_iterations', 'None'),
     ('ss_network_alpha', '16'),
     ('ss_network_dim', '32'),
     ('ss_network_module', 'networks.lora'),
     ('ss_new_sd_model_hash',
      'b0c91a8a93ab63b8b513046c2e2181a4ce78b83367ee7e26354ba9e018ea102f'),
     ('ss_noise_offset', 'None'),
     ('ss_num_batches_per_epoch', '101'),
     ('ss_num_epochs', '10'),
     ('ss_num_reg_images', '0'),
     ('ss_num_train_images', '200'),
     ('ss_optimizer', 'bitsandbytes.optim.adamw.AdamW8bit'),
     ('ss_output_name', 'rapunzel'),
     ('ss_prior_loss_weight', '1.0'),
     ('ss_sd_model_hash', '66f58dbb'),
     ('ss_sd_model_name', 'fp16-pruned.ckpt'),
     ('ss_sd_scripts_commit_hash', 'e6ad3cbc66130fdc3bf9ecd1e0272969b1d613f7'),
     ('ss_seed', '42'),
     ('ss_session_id', '1637709685'),
     ('ss_steps', '1010'),
     ('ss_tag_frequency',
      '{"rapunzel": {"rapunzel 1girl": 40, "solo": 39, "long hair": 37, "smile": '
      '26, "blonde hair": 38, "dress": 21, "very long hair": 18, "green eyes": 30, '
      '"crossed arms": 3, "corset": 5, "purple dress": 12, "breasts": 13, "open '
      'mouth": 7, "medium breasts": 5, "nipples": 7, "swimsuit": 2, "bikini": 1, '
      '"tongue": 2, "striped": 1, "lips": 7, "micro bikini": 1, "freckles": 5, '
      '"looking at viewer": 10, "long sleeves": 7, "cleavage": 3, "multiple boys": '
      '1, "puffy sleeves": 7, "2boys": 1, "full body": 4, "absurdly long hair": 2, '
      '"grin": 2, "hand on hip": 3, "juliet sleeves": 1, "blush": 3, "large '
      'breasts": 4, "hair ornament": 5, "standing": 2, "flower": 7, "cowboy shot": '
      '1, "teeth": 5, "indoors": 1, "hair flower": 5, "one-piece swimsuit": 1, '
      '"covered navel": 1, "makeup": 2, "cameltoe": 1, "highleg": 1, "lamp": 1, '
      '"braid": 4, "blurry": 2, "arm up": 1, "depth of field": 1, "blurry '
      'background": 2, "looking up": 3, "bokeh": 1, "upper body": 4, "short '
      'sleeves": 3, "closed eyes": 1, "small breasts": 1, "lying": 3, "signature": '
      '1, "on back": 1, "arms up": 2, "eyelashes": 1, "from above": 1, "happy": 1, '
      '"grass": 1, "cross-laced clothes": 1, "hair spread out": 1, "sitting": 4, '
      '"outdoors": 2, "barefoot": 7, "water": 1, "tree": 2, "pink dress": 1, '
      '"holding flower": 1, "lily pad": 1, "simple background": 5, "white '
      'background": 3, "frog": 4, "black background": 2, "child": 1, "star print": '
      '1, "lantern": 1, "blue eyes": 2, "navel": 5, "underwear": 2, "panties": 2, '
      '"thigh gap": 1, "bound": 2, "bdsm": 2, "bondage": 2, "gag": 1, "gagged": 1, '
      '"improvised gag": 1, "cloth gag": 1, "nude": 5, "tongue out": 1, "saliva": '
      '1, "arms behind back": 1, "seiza": 1, "bound arms": 1, "on head": 1, '
      '"animal on head": 1, "holding hair": 1, "tiara": 1, "crown": 1, "princess": '
      '1, "hands on own head": 1, "mirror": 2, "frying pan": 2, "nature": 1, '
      '"parted lips": 1, "blue background": 1, "shirt": 1, "choker": 2, "huge '
      'breasts": 1, "colored skin": 1, "bursting breasts": 1, "thighhighs": 2, '
      '"jewelry": 1, "thighs": 1, "earrings": 1, "pussy": 3, "spread legs": 2, '
      '"cum": 1, "high heels": 1, "kneeling": 1, "uncensored": 3, "cum in pussy": '
      '1, "phone": 1, "cellphone": 1, "smartphone": 1, "after sex": 1, "holding '
      'phone": 1, "cumdrip": 1, "brown thighhighs": 1, "gloves": 1, "multicolored '
      'hair": 1, "white gloves": 1, "topless": 1, "pink panties": 1, "fishnet '
      'thighhighs": 1, "ass": 1, "feet": 1, "soles": 1, "on stomach": 1, "the '
      'pose": 1, "nail polish": 1, "tan": 1, "personification": 1, "tanlines": 1, '
      '"detached sleeves": 1, "armpits": 1, "pillow": 1}}'),
     ('ss_text_encoder_lr', '0.0001'),
     ('ss_training_comment', 'None'),
     ('ss_training_finished_at', '1694906008.5775223'),
     ('ss_training_started_at', '1694905352.3001792'),
     ('ss_unet_lr', '0.0005'),
     ('ss_v2', 'False'),
     ('sshs_legacy_hash', 'fd5d0c6f'),
     ('sshs_model_hash',
      '0f9a9c96e707b34f1591ed9d24e788769cb72e2040f380aa7c31b6a49ca783b2')]

    有数据就比较直观，不然这篇博文，就显得干巴巴的。

    也算为业界做了一点点贡献。

    提一个额外的话题，

    由于diffusers和stable-diffusion-webui的实现十分臃肿，
    并且屎山越来越大，非常不便于研发以及学习。
    为了远离那堆玩意，博主化身屎壳郎，花了不少精力对stable diffusion进行了整理和重写。
    开源了sd 1.5和 sdxl 1.0 两个仓库，主要以keras/TensorFlow为主，兼容pytroch的模型权重。
    项目地址为:
    sd1.5:
    https://github.com/cpuimage/minSDTF
    sdxl 1.0：
    https://github.com/cpuimage/minSDXLTF
    目前有空会持续更新，目标是把webui的主要功能以比较简洁优雅的方式实现，路漫漫其修远兮。
    如果有小伙伴有能力也想加入，欢迎联系博主，一起摇摆。

    也有业内好友期望博主写一份keras/tensorflow版本的lora训练框架，只能说有时间的话，会写的。

    最后唠叨几句，kohya_ss训练框架的作者，其代码好几个地方逻辑写错了，小伙伴们要注意了。

    最后的最后，

    有stable diffusion相关的训练或者开发定制，不论是需求还是各种问题都可以通过以下方式联系博主，


    微信: Dbgmonks

    QQ: 200759103

    邮箱: gaozhihan@vip.qq.com

    注: 博主提供有偿服务, 不注明来意者一律拒绝。

     
     
     
     
     
  • 相关阅读:
    存在主义和阳明心学
    Linux服务器占用处理手记
    【SpringMVC】拦截器&JSR303的使用
    【LeetCode热题100】--34.在排序数组中查找元素的第一个和最后一个位置
    打印nXn方阵的上三角阵
    【算法面试必刷Java版九】删除链表的倒数第n个节点
    java计算机毕业设计爱心公益网站设计与制作源码+系统+lw文档+mysql数据库+部署
    软件测试/测试开发丨测试用例自动录入 学习笔记
    布隆过滤器及其用法
    JAVA计算机毕业设计网约车管理系统（附源码、数据库）
  • 原文地址：https://www.cnblogs.com/cpuimage/p/17816178.html