What is Object Removal?
Just what it sounds like: the task of erasing specific objects from an image.
It cleanly removes things you don't want in the shot, like people, signs, power lines, trash, or passersby, and fills in the background seamlessly.
LaMa
Before diffusion models, CNN-based inpainting models like LaMa were the SOTA and widely used.
It specializes in filling masked areas with the surrounding texture, and it was also popular for removing watermarks.
Removal by Inpainting
The most straightforward approach is to make a mask and fill it in with regular inpainting.
Draw a mask on the object you want to remove, write a prompt that matches the background (e.g., "background lawn only", "empty floor"), and inpaint.
But instead of removing the object, it would sometimes add a new one, so it wasn't always reliable for object removal.
That's why LaMa was sometimes used as a preprocessing step before inpainting, but current models probably don't need it.
Object Removal with Instruction-Based Image Editing
With recent instruction-based image editing models, object removal is becoming a pretty simple task.
You just say "remove this person," "remove this sign," "remove the logo in the bottom right," and so on.
Advantage: No Mask Required
The clear advantage over inpainting is that you don't have to draw a mask.
To automate object removal, you'd need segmentation to mask the object, but really you need to remove not just the object itself but also its shadow and its reflection in glass. That's the hard part.
Instruction-based image editing models remove those too.