sd-webui-controlnet代码分析

controlnet前向代码解析_Kun Li的博客-CSDN博客文章浏览阅读1.5k次。要分析下controlnet的yaml文件，在params中分成了4个部分，分别是control_stage_config、unnet_config、first_stage_config、cond_stage_config。其中control_stage_config对应的是13层的controlnet，unet_config对应的是diffusion model，first_stage_config对应的是vae中的decode部分。cldm：controlnet版本的ldm。https://blog.csdn.net/u012193416/article/details/129956546?ops_request_misc=%257B%2522request%255Fid%2522%253A%2522170028636116800188569967%2522%252C%2522scm%2522%253A%252220140713.130102334.pc%255Fblog.%2522%257D&request_id=170028636116800188569967&biz_id=0&utm_medium=distribute.pc_search_result.none-task-blog-2~blog~first_rank_ecpm_v1~rank_v31_ecpm-3-129956546-null-null.nonecase&utm_term=controlnet&spm=1018.2226.3001.4450上面是之前写的controlnet原版的代码流程。

stable-diffusion-webui源码分析（7）-controlnet - 知乎AUTOMATIC1111的stable-diffusion-webui是近期很流行的stable-diffusion界面应用，它集合stable-diffusion各项常用功能，还通过扩展的形式支持controlnet、lora等技术。分析它的源码，也在一定程度上有助于理解sta…https://zhuanlan.zhihu.com/p/620218436上面介绍的还是挺详细的。

api.py->这个api和alwayson_scripts不同
 
controlnet_ui/controlnet_ui_group.UiControlNetUnit->internal_controlnet/external_code.ControlNetUnit
 
controlnet.py->
ui->
self.ui_group->
ControlNetUiGroup->
controlnet_ui/controlnet_ui_group.ControlNetGroup->render()
 
controlnet.py->process()->
unet = p.sd_model.model.diffusion_model->
unit.module = global_state.get_module_basename(unit.module)
resize_mode = external_code.resize_mode_from_value(unit.resize_mode)
control_mode = external_code.control_mode_from_value(unit.control_mode)->
model_net = Script.load_control_model(p, unet, unit.model, unit.low_vram)->
input_image, image_from_a1111 = Script.choose_input_image(p, unit, idx)->
preprocessor = self.preprocessor[unit.module]->
detected_map, is_image = preprocessor(input_image,res=preprocessor_resolution,thr_a=unit.threshold_a,thr_b=unit.threshold_b) ->
control, detected_map = Script.detectmap_proc(detected_map, unit.module, resize_mode, h, w) ->
control_model_type = ControlModelType.ControlNet->
preprocessor_dict = dict()->
forward_param = ControlParams()->
self.latest_network = UnetHook(lowvram=any(unit.low_vram for unit in self.enabled_units))->
self.latest_network.hook(model=unet, sd_ldm=sd_ldm, control_params=forward_params, process=p)
- hook.py->UnetHook()->
- process.sample = process_sample
- model._original_forward = model.forward
- model.forward = forward_webui.__get__(model,UNetModel) # 重新给model.foward赋值
-- forward(*args,**kwargs)->
-- # handle cond-uncond marker
-- # convert control image to latent
-- # handle prompt token control
-- # handle controlnet
-- # replace x_t to support inpaint models
-- # clear attention and adain cache
-- # unet encoder
-- # unet middle block
-- # unet decoder
-- # unet output
-- # post-processing for color fix
-- # post-processing for restore
scripts.script_callbacks.on_cfg_denoiser(self.guidance_schedule_handler)

Tips：

最大的问题，controlnet的on_ui_setting并没有被启用？

[Bug]: on_ui_settings callback gets called after "ui" method in scripts · Issue #11210 · AUTOMATIC1111/stable-diffusion-webui · GitHub

大概是这么个问题，就是我在插件中写的on_ui_setting注入到webui中，再从插件中使用这些注入的参数时，无效，需要在安装完后，执行reload操作，这样这些参数注入才是有效的？

但是on_ui_setting中的参数多是默认参数，其实也不太影响。