What is video2audio?
Apart from a few like Sora 2 and Veo 3, today's video generation models still generate only video. In other words, there's no sound.
That's where video2audio comes in: technology that generates audio from video.
It works out "what is happening" in the footage and generates matching sound, synced to the video.
FoleyCrafter
FoleyCrafter is a Video2Audio framework that adds a "video adapter" on top of an existing Text2Audio model.
Think of it as giving the base Text2Audio model extra information: "what sound fits what's on screen" and "when it should play (timing)."
HunyuanVideo-Foley
HunyuanVideo-Foley is a multimodal diffusion Transformer designed for text + video -> audio from the start.
Rather than bolting features onto a text2audio model the way FoleyCrafter does, it is trained on text, video, and audio together from the outset.