What is Preprocessor?

How do you prepare "control images" like stick figures and depth maps used in ControlNet? While simple edges like Canny might be manageable, drawing a depth map by hand every time is not realistic.

Therefore, the process of automatically creating stick figures, depth maps, line drawings, normal maps, etc. from reference images is collectively called "Preprocessor" for convenience in the ControlNet community.

There is no single technology that does all of these; there are separate technologies for each, such as pose estimation, depth estimation, and line drawing extraction.


The True Nature of Control Images

I said it was hard to draw by hand, but please remember that "it IS possible to draw by hand".

The control image is not a special data type, but just an RGB image. You can paint over interfering parts in a depth map with black, or redraw just the arms of a pose image to change the posture.


Required Custom Nodes

Basically, this is all you need.


Frequently Used Preprocessors

Here we introduce technologies that are actually used frequently. There are things with better performance, but we have chosen them focusing on ease of use, lightness, and usability.

If you are using it for ControlNet, you don't need such extreme accuracy.

Canny

Canny-Canny_Edge.json
  • 🟩 Canny
  • 🟨 Canny Edgy

SoftEdge / HED

HED_Soft-Edge.json
  • 🟩 HED Soft-Edge

Lineart

Realistic-AnyLine_Lineart.json
  • 🟩 Realistic Lineart
  • 🟨 AnyLine Lineart

Depth

Depth_Anything_V2.json
  • 🟩 Depth Anything V2
    • Currently developed up to V3, but V2 is sufficient for ControlNet use.

Normal

DSINE.json
  • 🟩 DSINE

MLSD

M-LSD.json
  • 🟩 M-LSD Lines

Pose

OpenPose_DWPose.json
  • 🟩 OpenPose
  • 🟨 DWPose
    • Often treated as a higher compatible version of OpenPose, but it has a clear weakness that it is not good at back views. Use it together with OpenPose depending on the situation.

SDPose

SDPose.json

judian17/ComfyUI-SDPose-OOD

  • 🟩 SDPose
    • OpenPose is very weak with animals and anime illustrations, so try this if it doesn't work well.