What is talking head?
Talking head animates a single image or face photo so it looks like "a person talking." It moves the mouth and expressions of the input image using the motion from a separate reference video or audio as a guide.
It's very similar to lip sync, but lip sync is mostly about "matching just the mouth in an existing video to audio." Talking head is basically about animating a single image, and many methods are driven mainly by the motion in a reference video rather than by audio.
As the name suggests, it started with moving the face, but it's been evolving toward moving the upper body and even the whole body.
Deformation-based talking head
Thin-Plate Spline Motion Model for Image Animation
Give it a single image and a video of someone moving, and the image deforms to mimic that motion.
Rather than using a 3D model, it's more like squishing and twisting the image while it stays 2D. Think Photoshop's Puppet Warp.
LivePortrait
This also takes a single image and a reference video, but it's designed to reproduce the movement of each facial feature, gaze direction, and subtle emotion reliably.
It isn't a diffusion model, so it's relatively light and leans toward real-time use. It also lets you make edits like "tilt the face down a little" or "open the eyes a bit," so it's still widely used today.
Diffusion model-based talking head
The next generation brought talking heads that use diffusion models to "redraw the picture itself," such as X-Portrait and HelloMeme.
They extract signals for things like "head direction" and "changes in expression" from the reference video and pass them to the diffusion model as conditions. It's a lot like generating an image while ControlNet holds the pose and composition in place, as if you were saying "redraw this character's face with this motion."
Video generation model-based talking head
Even newer are talking head / avatar models built on video generation models themselves. OmniAvatar and Wan-Animate belong to this line.
Wan-Animate
Wan-Animate takes a character image and a "reference video with motion" and animates the character to follow that motion.
To Human Motion Transfer
Once talking head techniques could handle the face reliably, it was only natural to want to "move the upper body and whole body too."
Older methods like Thin-Plate Spline could already be applied to the whole body, not just the face, and Wan-Animate handles the whole body perfectly, so honestly I don't think there's much need to separate it from talking head. Still, human motion transfer has evolved along its own path, so let's take a quick look.