LivePortraitとは？
LivePortrait は、1枚の顔写真を、別の動画やパラメータに合わせて動かすためのキーポイントベースの Talking Head モデルです。
写真の中の顔に打ったキーポイントを手がかりに、Photoshop の「ゆがみツール」を AI が 1 秒間に何十回も自動で実行しているようなイメージで変形させています。
参照動画と同じ表情・首振りをさせたり、顔のパーツごと（目・口・首の向きなど）に微調整することも出来ます。
最近の動画生成モデルを使ったものほどの「何でもあり感」はないものの、拡散モデルではないぶん非常に軽く、ほぼリアルタイムで動かせるのが大きな特徴です。
インスタレーションに使ってもいいですし、「まぶたを少し閉じる」「顔を少し下に向かせる」といった、生成した画像の微調整に使うのもかなり便利です。
カスタムノード
image2image
入力した人物画像の顔の向き・表情を変化させます。
大きく 2 つのコントロール方法があります。
- パラメータで表情を調整する
- 参照画像と同じ表情にする
パラメータで表情を調整する
{
"id": "dba15c18-c2e7-4547-8472-85361bc55454",
"revision": 0,
"last_node_id": 15,
"last_link_id": 15,
"nodes": [
{
"id": 10,
"type": "PreviewImage",
"pos": [
1699.0274397476157,
93.3821815696158
],
"size": [
312.7272727272725,
485.090909090909
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 14
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "PreviewImage"
},
"widgets_values": []
},
{
"id": 8,
"type": "LoadImage",
"pos": [
1052.5387244959265,
93.3821815696158
],
"size": [
282.1988281250001,
508.7
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
13
]
},
{
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"pasted/image (105).png",
"image"
]
},
{
"id": 14,
"type": "ExpressionEditor",
"pos": [
1374.372496184271,
93.3821815696158
],
"size": [
285.0200000000002,
781.96
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "src_image",
"shape": 7,
"type": "IMAGE",
"link": 13
},
{
"name": "motion_link",
"shape": 7,
"type": "EDITOR_LINK",
"link": null
},
{
"name": "sample_image",
"shape": 7,
"type": "IMAGE",
"link": null
},
{
"name": "add_exp",
"shape": 7,
"type": "EXP_DATA",
"link": null
}
],
"outputs": [
{
"name": "image",
"type": "IMAGE",
"links": [
14
]
},
{
"name": "motion_link",
"type": "EDITOR_LINK",
"links": null
},
{
"name": "save_exp",
"type": "EXP_DATA",
"links": null
}
],
"properties": {
"cnr_id": "comfyui-advancedliveportrait",
"ver": "3bba732915e22f18af0d221b9c5c282990181f1b",
"Node name for S&R": "ExpressionEditor"
},
"widgets_values": [
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
1,
"All",
1.7
],
"color": "#232",
"bgcolor": "#353"
}
],
"links": [
[
13,
8,
0,
14,
0,
"IMAGE"
],
[
14,
14,
0,
10,
0,
"IMAGE"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.7513148009015777,
"offset": [
-422.25595692540645,
168.35165163849862
]
},
"frontendVersion": "1.35.0",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true
},
"version": 0.4
}
- 色々なパラメータがありますが、触ってみるのが早いですね。
- この動画のように、
▷ Run (On Change)を使うと便利です。
参照画像から編集する
{
"id": "dba15c18-c2e7-4547-8472-85361bc55454",
"revision": 0,
"last_node_id": 18,
"last_link_id": 19,
"nodes": [
{
"id": 15,
"type": "PreviewImage",
"pos": [
1701.7023235943498,
98.72387954106597
],
"size": [
312.7272727272725,
485.090909090909
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 13
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "PreviewImage"
},
"widgets_values": []
},
{
"id": 9,
"type": "LoadImage",
"pos": [
1081.3225849102464,
207.48210643813542
],
"size": [
271.6843277493424,
470.9630353117956
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
9
]
},
{
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"pasted/image (107).png",
"image"
]
},
{
"id": 14,
"type": "LoadImage",
"pos": [
766.4372083426591,
98.72387954106597
],
"size": [
282.1988281250001,
508.70000000000005
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
14
]
},
{
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"pasted/image (105).png",
"image"
]
},
{
"id": 11,
"type": "ExpressionEditor",
"pos": [
1385.6934611021761,
98.72387954106597
],
"size": [
283.32231404958657,
756
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "src_image",
"shape": 7,
"type": "IMAGE",
"link": 14
},
{
"name": "motion_link",
"shape": 7,
"type": "EDITOR_LINK",
"link": null
},
{
"name": "sample_image",
"shape": 7,
"type": "IMAGE",
"link": 9
},
{
"name": "add_exp",
"shape": 7,
"type": "EXP_DATA",
"link": null
}
],
"outputs": [
{
"name": "image",
"type": "IMAGE",
"links": [
13
]
},
{
"name": "motion_link",
"type": "EDITOR_LINK",
"links": []
},
{
"name": "save_exp",
"type": "EXP_DATA",
"links": null
}
],
"properties": {
"cnr_id": "comfyui-advancedliveportrait",
"ver": "3bba732915e22f18af0d221b9c5c282990181f1b",
"Node name for S&R": "ExpressionEditor"
},
"widgets_values": [
0,
17.5,
5.5,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
1,
"All",
1.7
],
"color": "#232",
"bgcolor": "#353"
}
],
"links": [
[
9,
9,
0,
11,
2,
"IMAGE"
],
[
13,
11,
0,
15,
0,
"IMAGE"
],
[
14,
14,
0,
11,
0,
"IMAGE"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.6830134553650705,
"offset": [
-175.963708342659,
209.17832045893417
]
},
"frontendVersion": "1.35.0",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true
},
"version": 0.4
}
- 先ほどの workflow に
sample_imageを加えるだけです。 sample_partsから、どの部位を参照画像に追従させるか選べます。OnlyExpression… 表情のみOnlyRotation… 顔の向きのみOnlyMouth… 口のみOnlyEyes… 目のみAll… 全て
参照画像の表情（向き）にさせたうえで、パラメータを使って微調整することもできます。
image2video
画像内の人物を、パラメータや参照動画に従って動かします。
motion_link
Expression Editor (PHM) でいくつもの表情を作り、その表情を次へ次へ変化させていくことで動画を作ることが出来ます。
- 🟨
Advanced Live Portrait (PHM)ノードanimate_without_vidをtrueに- その下のコマンド欄に、どの表情にどれだけかけて変化させるかを設定します。
コマンド欄のフォーマットは次のようなイメージです。
表情のインデックス = その表情に「変化していくフレーム数」:「その表情のまま待つフレーム数」
例えば、次のように書いた場合を考えてみます。
1 = 1:0
2 = 15:0
3 = 20:10
-
1 = 1:0- 変化フレーム 1、待ちフレーム 0
- この表情から始まり、すぐに次の表情へ変化します。
-
2 = 15:0- 15 フレームかけて 1 の表情からこの表情へ変化し、すぐに次の表情へ移ります。
-
3 = 20:10- 20 フレームかけて 2 の表情からこの表情へ変化し、そのあと 10 フレーム保持します。
この場合、合計 46 フレームの動画が出来ます。
参照動画から転送する
上では少しトリッキーなことをしましたが、実際にはこちらの使い方のほうがメインになると思います。
{
"id": "dba15c18-c2e7-4547-8472-85361bc55454",
"revision": 0,
"last_node_id": 25,
"last_link_id": 28,
"nodes": [
{
"id": 8,
"type": "LoadImage",
"pos": [
1053.8902782149364,
-0.15546305848340092
],
"size": [
282.1988281250001,
508.7
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
17
]
},
{
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"pasted/image (105).png",
"image"
]
},
{
"id": 21,
"type": "VHS_VideoCombine",
"pos": [
2019.7634896752875,
-1.365463058483401
],
"size": [
344.2576171874998,
780.8260157993859
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 19
},
{
"name": "audio",
"shape": 7,
"type": "AUDIO",
"link": null
},
{
"name": "meta_batch",
"shape": 7,
"type": "VHS_BatchManager",
"link": null
},
{
"name": "vae",
"shape": 7,
"type": "VAE",
"link": null
}
],
"outputs": [
{
"name": "Filenames",
"type": "VHS_FILENAMES",
"links": null
}
],
"properties": {
"cnr_id": "comfyui-videohelpersuite",
"ver": "8923bd836bdab8b7bbdf4ed104b7d045e70c66e2",
"Node name for S&R": "VHS_VideoCombine"
},
"widgets_values": {
"frame_rate": 16,
"loop_count": 0,
"filename_prefix": "LivePortrait",
"format": "video/h264-mp4",
"pix_fmt": "yuv420p",
"crf": 19,
"save_metadata": true,
"trim_to_audio": false,
"pingpong": false,
"save_output": true,
"videopreview": {
"hidden": false,
"paused": false,
"params": {
"filename": "LivePortrait_00002.mp4",
"subfolder": "",
"type": "output",
"format": "video/h264-mp4",
"frame_rate": 8,
"workflow": "LivePortrait_00002.png",
"fullpath": "D:\\AI\\ComfyUI_windows_portable\\ComfyUI\\output\\LivePortrait_00002.mp4"
}
}
}
},
{
"id": 20,
"type": "AdvancedLivePortrait",
"pos": [
1665.2686130082168,
-1.365463058483401
],
"size": [
311.2184264611833,
272
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "src_images",
"shape": 7,
"type": "IMAGE",
"link": 17
},
{
"name": "motion_link",
"shape": 7,
"type": "EDITOR_LINK",
"link": null
},
{
"name": "driving_images",
"shape": 7,
"type": "IMAGE",
"link": 28
}
],
"outputs": [
{
"name": "images",
"type": "IMAGE",
"links": [
19
]
}
],
"properties": {
"cnr_id": "comfyui-advancedliveportrait",
"ver": "3bba732915e22f18af0d221b9c5c282990181f1b",
"Node name for S&R": "AdvancedLivePortrait"
},
"widgets_values": [
0,
0,
1.7,
true,
false,
false,
""
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 25,
"type": "VHS_LoadVideo",
"pos": [
1371.2861499441556,
114.0272015894756
],
"size": [
261.6533203125,
460.08634187370603
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "meta_batch",
"shape": 7,
"type": "VHS_BatchManager",
"link": null
},
{
"name": "vae",
"shape": 7,
"type": "VAE",
"link": null
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
28
]
},
{
"name": "frame_count",
"type": "INT",
"links": null
},
{
"name": "audio",
"type": "AUDIO",
"links": null
},
{
"name": "video_info",
"type": "VHS_VIDEOINFO",
"links": null
}
],
"properties": {
"cnr_id": "comfyui-videohelpersuite",
"ver": "8923bd836bdab8b7bbdf4ed104b7d045e70c66e2",
"Node name for S&R": "VHS_LoadVideo"
},
"widgets_values": {
"video": "7327398-uhd_3840_2160_25fps.mp4",
"force_rate": 16,
"custom_width": 0,
"custom_height": 0,
"frame_load_cap": 48,
"skip_first_frames": 0,
"select_every_nth": 1,
"format": "None",
"choose video to upload": "image",
"videopreview": {
"hidden": false,
"paused": false,
"params": {
"filename": "7327398-uhd_3840_2160_25fps.mp4",
"type": "input",
"format": "video/mp4",
"force_rate": 16,
"custom_width": 0,
"custom_height": 0,
"frame_load_cap": 48,
"skip_first_frames": 0,
"select_every_nth": 1
}
}
}
}
],
"links": [
[
17,
8,
0,
20,
0,
"IMAGE"
],
[
19,
20,
0,
21,
0,
"IMAGE"
],
[
28,
25,
0,
20,
2,
"IMAGE"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.9090909090909091,
"offset": [
-786.1067299591815,
216.72498248265038
]
},
"frontendVersion": "1.35.0",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true
},
"version": 0.4
}
- 🟨
driving_imagesに参照動画を入力するだけです。
参照動画側の表情・首振りを、そのまま入力画像に転送してくれます。
もちろん、上の motion_link と組み合わせて使うことも出来ます。
video2video
動画の中の人物の表情を参照動画に合わせます。
{
"id": "dba15c18-c2e7-4547-8472-85361bc55454",
"revision": 0,
"last_node_id": 26,
"last_link_id": 29,
"nodes": [
{
"id": 20,
"type": "AdvancedLivePortrait",
"pos": [
1673.4228458122946,
-1.365463058483401
],
"size": [
311.2184264611833,
480
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [
{
"name": "src_images",
"shape": 7,
"type": "IMAGE",
"link": 29
},
{
"name": "motion_link",
"shape": 7,
"type": "EDITOR_LINK",
"link": null
},
{
"name": "driving_images",
"shape": 7,
"type": "IMAGE",
"link": 28
}
],
"outputs": [
{
"name": "images",
"type": "IMAGE",
"links": [
19
]
}
],
"properties": {
"cnr_id": "comfyui-advancedliveportrait",
"ver": "3bba732915e22f18af0d221b9c5c282990181f1b",
"Node name for S&R": "AdvancedLivePortrait"
},
"widgets_values": [
0,
0,
1.7,
true,
false,
false,
""
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 25,
"type": "VHS_LoadVideo",
"pos": [
1376.647308097985,
111.5478627465004
],
"size": [
261.6533203125,
460.08634187370603
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [
{
"name": "meta_batch",
"shape": 7,
"type": "VHS_BatchManager",
"link": null
},
{
"name": "vae",
"shape": 7,
"type": "VAE",
"link": null
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
28
]
},
{
"name": "frame_count",
"type": "INT",
"links": null
},
{
"name": "audio",
"type": "AUDIO",
"links": null
},
{
"name": "video_info",
"type": "VHS_VIDEOINFO",
"links": null
}
],
"properties": {
"cnr_id": "comfyui-videohelpersuite",
"ver": "8923bd836bdab8b7bbdf4ed104b7d045e70c66e2",
"Node name for S&R": "VHS_LoadVideo"
},
"widgets_values": {
"video": "7327398-uhd_3840_2160_25fps.mp4",
"force_rate": 16,
"custom_width": 0,
"custom_height": 0,
"frame_load_cap": 48,
"skip_first_frames": 0,
"select_every_nth": 1,
"format": "None",
"choose video to upload": "image",
"videopreview": {
"hidden": false,
"paused": false,
"params": {
"filename": "7327398-uhd_3840_2160_25fps.mp4",
"type": "input",
"format": "video/mp4",
"force_rate": 16,
"custom_width": 0,
"custom_height": 0,
"frame_load_cap": 48,
"skip_first_frames": 0,
"select_every_nth": 1
}
}
}
},
{
"id": 26,
"type": "VHS_LoadVideo",
"pos": [
1079.8717703836753,
-1.365463058483401
],
"size": [
261.6533203125,
460.08634187370603
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{
"name": "meta_batch",
"shape": 7,
"type": "VHS_BatchManager",
"link": null
},
{
"name": "vae",
"shape": 7,
"type": "VAE",
"link": null
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
29
]
},
{
"name": "frame_count",
"type": "INT",
"links": null
},
{
"name": "audio",
"type": "AUDIO",
"links": null
},
{
"name": "video_info",
"type": "VHS_VIDEOINFO",
"links": null
}
],
"properties": {
"cnr_id": "comfyui-videohelpersuite",
"ver": "8923bd836bdab8b7bbdf4ed104b7d045e70c66e2",
"Node name for S&R": "VHS_LoadVideo"
},
"widgets_values": {
"video": "3762907-uhd_3840_2160_25fps.mp4",
"force_rate": 16,
"custom_width": 0,
"custom_height": 0,
"frame_load_cap": 48,
"skip_first_frames": 0,
"select_every_nth": 1,
"format": "None",
"choose video to upload": "image",
"videopreview": {
"hidden": false,
"paused": false,
"params": {
"filename": "3762907-uhd_3840_2160_25fps.mp4",
"type": "input",
"format": "video/mp4",
"force_rate": 16,
"custom_width": 0,
"custom_height": 0,
"frame_load_cap": 48,
"skip_first_frames": 0,
"select_every_nth": 1
}
}
}
},
{
"id": 21,
"type": "VHS_VideoCombine",
"pos": [
2019.7634896752875,
-1.365463058483401
],
"size": [
480.98761718749984,
607.3055346679687
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 19
},
{
"name": "audio",
"shape": 7,
"type": "AUDIO",
"link": null
},
{
"name": "meta_batch",
"shape": 7,
"type": "VHS_BatchManager",
"link": null
},
{
"name": "vae",
"shape": 7,
"type": "VAE",
"link": null
}
],
"outputs": [
{
"name": "Filenames",
"type": "VHS_FILENAMES",
"links": null
}
],
"properties": {
"cnr_id": "comfyui-videohelpersuite",
"ver": "8923bd836bdab8b7bbdf4ed104b7d045e70c66e2",
"Node name for S&R": "VHS_VideoCombine"
},
"widgets_values": {
"frame_rate": 16,
"loop_count": 0,
"filename_prefix": "LivePortrait",
"format": "video/h264-mp4",
"pix_fmt": "yuv420p",
"crf": 19,
"save_metadata": true,
"trim_to_audio": false,
"pingpong": false,
"save_output": true,
"videopreview": {
"hidden": false,
"paused": false,
"params": {
"filename": "LivePortrait_00005.mp4",
"subfolder": "",
"type": "output",
"format": "video/h264-mp4",
"frame_rate": 16,
"workflow": "LivePortrait_00005.png",
"fullpath": "D:\\AI\\ComfyUI_windows_portable\\ComfyUI\\output\\LivePortrait_00005.mp4"
}
}
}
}
],
"links": [
[
19,
20,
0,
21,
0,
"IMAGE"
],
[
28,
25,
0,
20,
2,
"IMAGE"
],
[
29,
26,
0,
20,
0,
"IMAGE"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.8264462809917354,
"offset": [
-698.8176497225181,
308.1820919619897
]
},
"frontendVersion": "1.35.0",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true
},
"version": 0.4
}
- 🟨
src_images、driving_imagesの両方を動画にするだけです。
- ベース動画のカメラワークや背景はそのままに、人物の表情・口パクだけを差し替えることが出来ます。