ControlNet

The Doodle ControlNet feature significantly enhances the controllability of AI drawing. It supports over ten pre-trained styles, including doodling, interpolation, depth maps, and more. These styles enhance both the content and quality of AI-generated images. You can find detailed style explanations for the ControlNet feature here: https://github.com/lllyasviel/ControlNet-v1-1-nightly

Steps to Use ControlNet in JoyFusion

1. Installing a Specific Model ControlNet Plugin

ControlNet capabilities for specific models and sizes require installation.

2. Installing ControlNet Style Plugin

Specific style plugins for certain types and sizes only need to be installed once. Once installed, they can be used in conjunction with models that have ControlNet capabilities.

3. Using in JoyFusion

When you have installed the corresponding plugins, you can use ControlNet in the Creative Center. As shown in the image below, when both plugins display a green indicator, it signifies that the plugins have been recognized (if a downloaded plugin isn't recognized, try closing and reopening the app; this issue occasionally occurs). Currently, JoyFusion supports in-app editing for Scribble and InaPaint (interpolation) styles. For other styles, such as generating line art images via online services like "Photograph to Pattern" (https://online.rapidresizer.com/photograph-to-pattern.php) or converting to skeletal images via "OpenPose AI" (https://openposeai.com/), third-party functionality is needed. Once you submit an entry for a ControlNet-specific style image, it will take effect. JoyFusion also supports blending multiple ControlNet styles.