ControlNet

The Doodle ControlNet feature significantly enhances the controllability of AI drawing. It supports over ten pre-trained styles, including doodling, interpolation, depth maps, and more. These styles enhance both the content and quality of AI-generated images. You can find detailed style explanations for the ControlNet feature here: https://github.com/lllyasviel/ControlNet-v1-1-nightly

Steps to Use ControlNet in JoyFusion

1. Installing a Specific Model ControlNet Plugin

ControlNet capabilities for specific models and sizes require installation.

模型控制网插件

2. Installing ControlNet Style Plugin

Specific style plugins for certain types and sizes only need to be installed once. Once installed, they can be used in conjunction with models that have ControlNet capabilities.

控制网风格插件

3. Using in JoyFusion

When you have installed the corresponding plugins, you can use ControlNet in the Creative Center. As shown in the image below, when both plugins display a green indicator, it signifies that the plugins have been recognized (if a downloaded plugin isn't recognized, try closing and reopening the app; this issue occasionally occurs). Currently, JoyFusion supports in-app editing for Scribble and InaPaint (interpolation) styles. For other styles, such as generating line art images via online services like "Photograph to Pattern" (https://online.rapidresizer.com/photograph-to-pattern.php) or converting to skeletal images via "OpenPose AI" (https://openposeai.com/), third-party functionality is needed. Once you submit an entry for a ControlNet-specific style image, it will take effect. JoyFusion also supports blending multiple ControlNet styles.

In-app Editing for InPaint Mode

当前设置的控制网条目展示

ControlNet Functionality Demonstration Video

控制网使用演示视频（Youtube版）

BiliBili： https://www.bilibili.com/video/BV1fh4y1D764