AI Mask Generation

Masks are often needed for inpainting and similar workflows, but drawing them by hand or preparing mask images every time is a lot of work. Above all, it cannot be automated.

But there aren't many techniques that let you just say "mask this part" and easily get a clean mask.

You need to think in terms of combining several AI techniques.

  • Object Detection - Finds where the target is in the image.
  • Segmentation - Cuts out the target shape as a mask.
  • Matting - Handles the boundary between foreground and background in more detail.

For example, you might use object detection to find the target, then pass that result to segmentation to turn it into a mask.

Let's see what's out there.

Object Detection

As the name suggests, object detection identifies the position of a specific object in an image and outputs a rectangular area called a BBOX.

YOLO Family

YOLO is an extremely fast detection technique designed for real-time object detection.

Basically, each model is made for one type of object you want to detect (faces only, hands only, and so on). If there's no model for your target, you have to train one yourself, and it's not a good fit when you want to detect several kinds of objects.

In exchange, it's very lightweight, so it's a good choice when you need speed.

Grounding DINO and Others

Grounding DINO detects objects specified by text and outputs BBOXes.

Unlike YOLO, you can name any object you like in text, such as "white dog" or "red car," so it's easy to use, and it can also detect multiple objects at once.

VLM / MLLM

VLM / MLLM are LLMs with the ability to see images.

They can do many things, such as caption generation, and some of them can also perform object detection.

It's quite old now, but a well-known example is Florence-2.

It's slow, but it understands a lot, so you can describe the target with complex text such as "the woman on the right side of the screen wearing a blue hat."

Matting

Much of what's called "background removal" is actually matting.

Matting separates the foreground from the background, and can handle fine boundaries such as hair and semi-transparent areas.

However, it is not for specifying and cutting out one particular object the way segmentation does.

BiRefNet

The detailed usage is covered on the BiRefNet page.

Segmentation

SAM (Segment Anything Model)

SAM is currently the most famous segmentation model.

It understands the shape of objects, so if you specify a car in a photo with text, points, or boxes, it can find the outline and turn it into a mask.

The current latest model is covered on the SAM 3 / 3.1 page.

Practical Examples

Let's combine the techniques above to generate masks for arbitrary text prompts or categories.

The workflows below were commonly used before SAM 3. If your goal is target-specified segmentation, start with SAM 3 / 3.1 now.

They remain here as references for understanding older workflows or reproducing the same setup in an existing environment.

Required Custom Nodes

These custom nodes may be needed to run the practical examples on this page.

YOLO x SAM

yolo_face_sam.json

This combines fast face detection with YOLO and the original SAM.

Grounding DINO x SAM

grounding_dino_hq_sam.json

This combines Grounding DINO with HQ-SAM, an improved version of SAM.

You can pick the target with text and still get a high-precision mask, so it's one of the most commonly used combinations.

Florence2 x SAM2

florence2_sam2_1.json

This combines Florence2 and SAM2.1.

For simple targets like people or animals, anything works, but when you want to specify something more complex, like "a man wearing sunglasses" or "a cat lying under a tree," LLM-based models like this really shine.

SAM 3 x BiRefNet

sam3_birefnet.json

Segmentation is about telling objects apart in the first place, not about making fine cutouts.

By contrast, matting can handle fine details like hair and semi-transparent objects like glass.

Combining the two lets each one make up for what the other lacks.