What is Preprocessor?

How do you prepare "control images" like stick figures and depth maps used in ControlNet? While simple edges like Canny might be manageable, drawing a depth map by hand every time is not realistic.

Therefore, the process of automatically creating stick figures, depth maps, line drawings, normal maps, etc. from reference images is collectively called "Preprocessor" for convenience in the ControlNet community.

There is no single technology that does all of these; there are separate technologies for each, such as pose estimation, depth estimation, and line drawing extraction.

The True Nature of Control Images

I said it was hard to draw by hand, but please remember that "it IS possible to draw by hand".

The control image is not a special data type, but just an RGB image. You can paint over interfering parts in a depth map with black, or redraw just the arms of a pose image to change the posture.

Required Custom Nodes

Basically, this is all you need.

Frequently Used Preprocessors

Here we introduce technologies that are actually used frequently. There are things with better performance, but we have chosen them focusing on ease of use, lightness, and usability.

If you are using it for ControlNet, you don't need such extreme accuracy.

WorkflowsFrequently Used Preprocessors
canny_canny_edge.json

Canny

  • 🟩 Canny
  • 🟨 Canny Edgy
hed_soft_edge.json

SoftEdge / HED

  • 🟩 HED Soft-Edge
realistic_anyline_lineart.json

Lineart

  • 🟩 Realistic Lineart
  • 🟨 AnyLine Lineart
depth_anything_v2.json

Depth

  • 🟩 Depth Anything V2
    • Currently developed up to V3, but V2 is sufficient for ControlNet use.
dsine.json

Normal

  • 🟩 DSINE
m_lsd.json

MLSD

  • 🟩 M-LSD Lines
openpose_dwpose.json

Pose

  • 🟩 OpenPose
  • 🟨 DWPose
    • Often treated as a higher compatible version of OpenPose, but it has a clear weakness that it is not good at back views. Use it together with OpenPose depending on the situation.
sdpose.json

SDPose

judian17/ComfyUI-SDPose-OOD

  • 🟩 SDPose
    • OpenPose is very weak with animals and anime illustrations, so try this if it doesn't work well.