KSampler (Advanced)とは？
KSampler (Advanced)ノードは、無印KSamplerノードの上位互換です。
パラメータはdenoiseが消えて以下の四つが増えています。
add_noisestart_at_stepend_at_stepreturn_with_leftsover_noise
ComfyUIの中でも、最も面白い機能の一つだと思っているので、ゆっくり見ていきましょう。
拡散モデルとSamplingの復習
拡散モデルというのは、完全なノイズから、徐々にノイズを取り除いていくことで画像を生成する仕組みでした。
上の画像ならば、0ステップ目に満量のノイズを追加し、20ステップ目には全てのノイズが取り除かれ、画像が完成します。
パラメータ
start_at_step と end_at_step
KSampler (Advanced)では、start_at_step と end_at_step を設定することで、どこからサンプリングを開始し、どこまでサンプリングを進めるかを制御できます。
例えば、start_at_stepを4 / end_at_stepを11と設定したとき、画像の白い部分しかサンプリングを行いません。
add_noise
何度も繰り返しになりますが、拡散モデルはノイズ画像を作り、ノイズを徐々に取り除いていく仕組みです。
では、そのノイズはどこから来るのでしょう？
そうです、ノイズを追加するのもKSamplerが行っています。
KSampler (Advanced)では、add_noiseパラメータでノイズを追加するかしないかを選択できます。
{
"id": "8b9f7796-0873-4025-be3c-0f997f67f866",
"revision": 0,
"last_node_id": 15,
"last_link_id": 24,
"nodes": [
{
"id": 8,
"type": "VAEDecode",
"pos": [
1209,
188
],
"size": [
210,
46
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 20
},
{
"name": "vae",
"type": "VAE",
"link": 10
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"slot_index": 0,
"links": [
9
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "VAEDecode"
},
"widgets_values": [],
"color": "#232",
"bgcolor": "#353"
},
{
"id": 9,
"type": "SaveImage",
"pos": [
1451,
189
],
"size": [
354.2876035004722,
433.23967321788405
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 9
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33"
},
"widgets_values": [
"ComfyUI"
],
"color": "#232",
"bgcolor": "#353"
},
{
"id": 15,
"type": "SaveImage",
"pos": [
1451,
706.2502919494433
],
"size": [
354.2876035004722,
433.23967321788405
],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 23
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33"
},
"widgets_values": [
"ComfyUI"
],
"color": "#322",
"bgcolor": "#533"
},
{
"id": 14,
"type": "VAEDecode",
"pos": [
1209,
705.2502919494433
],
"size": [
210,
46
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 24
},
{
"name": "vae",
"type": "VAE",
"link": 22
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"slot_index": 0,
"links": [
23
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "VAEDecode"
},
"widgets_values": [],
"color": "#322",
"bgcolor": "#533"
},
{
"id": 11,
"type": "KSamplerAdvanced",
"pos": [
861.0869279218368,
700.4936138850449
],
"size": [
316.566228693182,
334
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 14
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 11
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 12
},
{
"name": "latent_image",
"type": "LATENT",
"link": 13
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
24
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "KSamplerAdvanced"
},
"widgets_values": [
"disable",
1234,
"fixed",
20,
8,
"euler",
"normal",
0,
20,
"disable"
],
"color": "#322",
"bgcolor": "#533"
},
{
"id": 10,
"type": "VAELoader",
"pos": [
896.5787764497297,
584.438602917244
],
"size": [
281.0743801652891,
58
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [
10,
22
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "VAELoader"
},
"widgets_values": [
"vae-ft-mse-840000-ema-pruned.safetensors"
]
},
{
"id": 7,
"type": "CLIPTextEncode",
"pos": [
347.38431660156334,
544.0837453223864
],
"size": [
410.75801513671877,
158.82607910156253
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 5
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
12,
18
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"text, watermark"
]
},
{
"id": 13,
"type": "KSamplerAdvanced",
"pos": [
861.0869279218368,
192.38359194944317
],
"size": [
316.566228693182,
334
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 16
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 17
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 18
},
{
"name": "latent_image",
"type": "LATENT",
"link": 19
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
20
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "KSamplerAdvanced"
},
"widgets_values": [
"enable",
1234,
"fixed",
20,
8,
"euler",
"normal",
0,
20,
"disable"
],
"color": "#232",
"bgcolor": "#353"
},
{
"id": 6,
"type": "CLIPTextEncode",
"pos": [
346.18730000000085,
334.6177000000005
],
"size": [
411.95503173828126,
151.0030493164063
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 3
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
11,
17
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"beautiful scenery nature glass bottle landscape, , purple galaxy bottle,"
]
},
{
"id": 5,
"type": "EmptyLatentImage",
"pos": [
511.5041499200999,
761.3728204299287
],
"size": [
244.81999999999994,
106
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
13,
19
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "EmptyLatentImage"
},
"widgets_values": [
512,
512,
1
]
},
{
"id": 4,
"type": "CheckpointLoaderSimple",
"pos": [
-32.91270000000002,
499.3077000000004
],
"size": [
315,
98
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"slot_index": 0,
"links": [
14,
16
]
},
{
"name": "CLIP",
"type": "CLIP",
"slot_index": 1,
"links": [
3,
5
]
},
{
"name": "VAE",
"type": "VAE",
"slot_index": 2,
"links": []
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CheckpointLoaderSimple"
},
"widgets_values": [
"v1-5-pruned-emaonly-fp16.safetensors"
]
}
],
"links": [
[
3,
4,
1,
6,
0,
"CLIP"
],
[
5,
4,
1,
7,
0,
"CLIP"
],
[
9,
8,
0,
9,
0,
"IMAGE"
],
[
10,
10,
0,
8,
1,
"VAE"
],
[
11,
6,
0,
11,
1,
"CONDITIONING"
],
[
12,
7,
0,
11,
2,
"CONDITIONING"
],
[
13,
5,
0,
11,
3,
"LATENT"
],
[
14,
4,
0,
11,
0,
"MODEL"
],
[
16,
4,
0,
13,
0,
"MODEL"
],
[
17,
6,
0,
13,
1,
"CONDITIONING"
],
[
18,
7,
0,
13,
2,
"CONDITIONING"
],
[
19,
5,
0,
13,
3,
"LATENT"
],
[
20,
13,
0,
8,
0,
"LATENT"
],
[
22,
10,
0,
14,
1,
"VAE"
],
[
23,
14,
0,
15,
0,
"IMAGE"
],
[
24,
11,
0,
14,
0,
"LATENT"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.620921323059155,
"offset": [
132.91270000000003,
-88
]
},
"frontendVersion": "1.34.6",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true
},
"version": 0.4
}
🟫仮にadd_noiseをdisableに設定してサンプリングを行うと、ノイズのない画像からノイズを除去し続けるため何も生成されません。
サンプリングを分割する
中間のステップしかサンプリングしなかったり、ノイズを追加しなかったり、一体どこで役に立つのでしょうか？
このKSampler (Advanced)だからこそ出来ることとして、一つのサンプリングを2つ以上のKSamplerに分割する、というものがあります。
図のように、前半 0 ~ 10stepは🟪Ksample (Advanced)、11~20stepは🟨KSampler (Advanced)で行ってみましょう。
{
"id": "8b9f7796-0873-4025-be3c-0f997f67f866",
"revision": 0,
"last_node_id": 15,
"last_link_id": 33,
"nodes": [
{
"id": 8,
"type": "VAEDecode",
"pos": [
1588.3386299999995,
185.17101999999997
],
"size": [
210,
46
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 7
},
{
"name": "vae",
"type": "VAE",
"link": 10
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"slot_index": 0,
"links": [
9
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "VAEDecode"
},
"widgets_values": []
},
{
"id": 5,
"type": "EmptyLatentImage",
"pos": [
476.39434173828073,
792.3797499999989
],
"size": [
244.81999999999994,
106
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
2,
14
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "EmptyLatentImage"
},
"widgets_values": [
512,
512,
1
]
},
{
"id": 9,
"type": "SaveImage",
"pos": [
1830.3386299999995,
185.17101999999997
],
"size": [
354.2876035004722,
433.23967321788405
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 9
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33"
},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 14,
"type": "VAEDecode",
"pos": [
1586.692689801927,
682.113174924969
],
"size": [
210,
46
],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 20
},
{
"name": "vae",
"type": "VAE",
"link": 18
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"slot_index": 0,
"links": [
19
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "VAEDecode"
},
"widgets_values": []
},
{
"id": 15,
"type": "SaveImage",
"pos": [
1828.692689801927,
682.113174924969
],
"size": [
354.2876035004722,
433.23967321788405
],
"flags": {},
"order": 11,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 19
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33"
},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 10,
"type": "VAELoader",
"pos": [
1279.285619834711,
539.9456682334871
],
"size": [
281.0743801652891,
58
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [
10,
18
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "VAELoader"
},
"widgets_values": [
"vae-ft-mse-840000-ema-pruned.safetensors"
]
},
{
"id": 11,
"type": "KSamplerAdvanced",
"pos": [
870,
682.113174924969
],
"size": [
313.66404687500017,
334
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 24
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 29
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 26
},
{
"name": "latent_image",
"type": "LATENT",
"link": 14
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
11
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "KSamplerAdvanced"
},
"widgets_values": [
"enable",
1234,
"fixed",
20,
8,
"euler",
"normal",
0,
10,
"enable"
],
"color": "#323",
"bgcolor": "#535"
},
{
"id": 13,
"type": "KSamplerAdvanced",
"pos": [
1246.300000000001,
682.113174924969
],
"size": [
313.66404687500017,
334
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 25
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 32
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 31
},
{
"name": "latent_image",
"type": "LATENT",
"link": 11
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
20
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "KSamplerAdvanced"
},
"widgets_values": [
"disable",
0,
"fixed",
20,
8,
"euler",
"normal",
10,
20,
"disable"
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 7,
"type": "CLIPTextEncode",
"pos": [
310.4563266015633,
578.1782351074208
],
"size": [
410.75801513671877,
158.82607910156253
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 5
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
26,
31,
33
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"text, watermark"
]
},
{
"id": 3,
"type": "KSampler",
"pos": [
861.38949,
185.17101999999997
],
"size": [
315,
262
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 23
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 28
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 33
},
{
"name": "latent_image",
"type": "LATENT",
"link": 2
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
7
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "KSampler"
},
"widgets_values": [
1234,
"fixed",
20,
8,
"euler",
"normal",
1
],
"color": "#232",
"bgcolor": "#353"
},
{
"id": 6,
"type": "CLIPTextEncode",
"pos": [
309.2593100000008,
371.7997499999994
],
"size": [
411.95503173828126,
151.0030493164063
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 3
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
28,
29,
32
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"beautiful scenery nature glass bottle landscape, , purple galaxy bottle,"
]
},
{
"id": 4,
"type": "CheckpointLoaderSimple",
"pos": [
-74.4034738016528,
516.4438097520666
],
"size": [
315,
98
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"slot_index": 0,
"links": [
23,
24,
25
]
},
{
"name": "CLIP",
"type": "CLIP",
"slot_index": 1,
"links": [
3,
5
]
},
{
"name": "VAE",
"type": "VAE",
"slot_index": 2,
"links": []
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CheckpointLoaderSimple"
},
"widgets_values": [
"v1-5-pruned-emaonly-fp16.safetensors"
]
}
],
"links": [
[
2,
5,
0,
3,
3,
"LATENT"
],
[
3,
4,
1,
6,
0,
"CLIP"
],
[
5,
4,
1,
7,
0,
"CLIP"
],
[
7,
3,
0,
8,
0,
"LATENT"
],
[
9,
8,
0,
9,
0,
"IMAGE"
],
[
10,
10,
0,
8,
1,
"VAE"
],
[
11,
11,
0,
13,
3,
"LATENT"
],
[
14,
5,
0,
11,
3,
"LATENT"
],
[
18,
10,
0,
14,
1,
"VAE"
],
[
19,
14,
0,
15,
0,
"IMAGE"
],
[
20,
13,
0,
14,
0,
"LATENT"
],
[
23,
4,
0,
3,
0,
"MODEL"
],
[
24,
4,
0,
11,
0,
"MODEL"
],
[
25,
4,
0,
13,
0,
"MODEL"
],
[
26,
7,
0,
11,
2,
"CONDITIONING"
],
[
28,
6,
0,
3,
1,
"CONDITIONING"
],
[
29,
6,
0,
11,
1,
"CONDITIONING"
],
[
31,
7,
0,
13,
2,
"CONDITIONING"
],
[
32,
6,
0,
13,
1,
"CONDITIONING"
],
[
33,
7,
0,
3,
2,
"CONDITIONING"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.6209213230591553,
"offset": [
174.4034738016528,
-85.17101999999997
]
},
"frontendVersion": "1.34.6",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true,
"reroutes": [
{
"id": 1,
"pos": [
808.5556574908115,
529.8476357277159
],
"linkIds": [
23,
24,
25
]
},
{
"id": 2,
"parentId": 1,
"pos": [
1172.9426440354462,
536.1385953616206
],
"linkIds": [
25
]
},
{
"id": 3,
"pos": [
812.5220874908125,
601.4361057277166
],
"linkIds": [
26,
31,
33
]
},
{
"id": 4,
"parentId": 3,
"pos": [
1176.9420874908117,
604.4861057277171
],
"linkIds": [
31
]
},
{
"id": 5,
"pos": [
811.3220874908124,
565.1461057277166
],
"linkIds": [
28,
29,
32
]
},
{
"id": 6,
"parentId": 5,
"pos": [
1174.1520874908117,
570.3461057277162
],
"linkIds": [
32
]
}
],
"linkExtensions": [
{
"id": 23,
"parentId": 1
},
{
"id": 24,
"parentId": 1
},
{
"id": 25,
"parentId": 2
},
{
"id": 26,
"parentId": 3
},
{
"id": 28,
"parentId": 5
},
{
"id": 29,
"parentId": 5
},
{
"id": 31,
"parentId": 4
},
{
"id": 32,
"parentId": 6
},
{
"id": 33,
"parentId": 3
}
]
},
"version": 0.4
}
2つに分割しても生成される画像が全く同じであるのがわかると思います。 パラメータが少しややこしいので、丁寧に見ていきます。
🟪KSampler (Advanced)
add_noise:enablestart_at_step:0end_at_step:10return_with_leftover_noise:enable- ノイズを残したままlatentを返します。
🟨KSampler (Advanced)
add_noise:disable- ノイズが残ったlatentが渡されるので、ここでノイズは追加しません。
start_at_step:10- 🟪の
end_at_stepと同じにします。
- 🟪の
end_at_step:20- 全体のstep数より大きな値にした場合、内部的には
stepsと同じになっています。
- 全体のstep数より大きな値にした場合、内部的には
return_with_leftover_noise:disable
分割することに意味はなさそうですが、これが面白いのです。
KSampler (Advanced)だからできること
プロンプトの切り替え
最初は"A"のプロンプトを使い、サンプリングの途中から"B"のプロンプトに切り替える、といったことが出来ます。
{
"id": "8b9f7796-0873-4025-be3c-0f997f67f866",
"revision": 0,
"last_node_id": 22,
"last_link_id": 44,
"nodes": [
{
"id": 14,
"type": "VAEDecode",
"pos": [
1800.503389632606,
389.0983567431494
],
"size": [
210,
46
],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 20
},
{
"name": "vae",
"type": "VAE",
"link": 18
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"slot_index": 0,
"links": [
19
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "VAEDecode"
},
"widgets_values": []
},
{
"id": 15,
"type": "SaveImage",
"pos": [
2039.6902352564712,
389.0983567431494
],
"size": [
354.2876035004722,
433.23967321788405
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 19
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33"
},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 5,
"type": "EmptyLatentImage",
"pos": [
442.63634173828075,
742.2317499999979
],
"size": [
244.81999999999994,
106
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
14
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "EmptyLatentImage"
},
"widgets_values": [
512,
512,
1
]
},
{
"id": 10,
"type": "VAELoader",
"pos": [
1488.009637350903,
269.2903779900228
],
"size": [
281.0743801652891,
58
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [
18
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "VAELoader"
},
"widgets_values": [
"vae-ft-mse-840000-ema-pruned.safetensors"
]
},
{
"id": 7,
"type": "CLIPTextEncode",
"pos": [
345.8263266015634,
591.7082351074206
],
"size": [
347.2221151367188,
88
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 5
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
31,
36
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"text, watermark"
]
},
{
"id": 6,
"type": "CLIPTextEncode",
"pos": [
345.71385545454683,
387.2801749249675
],
"size": [
340.95503173828126,
143.0030493164063
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 3
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
37
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"(best quality, high quality, ultra-detailed),a single apple on a table, photorealistic"
],
"color": "#323",
"bgcolor": "#535"
},
{
"id": 13,
"type": "KSamplerAdvanced",
"pos": [
1457.6524971337399,
389.0983567431494
],
"size": [
313.66404687500017,
334
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 25
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 39
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 31
},
{
"name": "latent_image",
"type": "LATENT",
"link": 11
},
{
"name": "start_at_step",
"type": "INT",
"widget": {
"name": "start_at_step"
},
"link": 42
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
20
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "KSamplerAdvanced"
},
"widgets_values": [
"disable",
0,
"fixed",
20,
8,
"euler",
"normal",
10,
20,
"disable"
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 17,
"type": "CLIPTextEncode",
"pos": [
1094.0560743170481,
387.2801749249675
],
"size": [
330.6550317382813,
143.20304931640635
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 40
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
39
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"(best quality, high quality, ultra-detailed), a fluffy red\nrabbit sitting on the same wooden table"
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 4,
"type": "CheckpointLoaderSimple",
"pos": [
-72.4034738016528,
462.4438097520666
],
"size": [
315,
98
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"slot_index": 0,
"links": [
25,
35
]
},
{
"name": "CLIP",
"type": "CLIP",
"slot_index": 1,
"links": [
3,
5,
40
]
},
{
"name": "VAE",
"type": "VAE",
"slot_index": 2,
"links": []
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CheckpointLoaderSimple"
},
"widgets_values": [
"v1-5-pruned-emaonly-fp16.safetensors"
]
},
{
"id": 11,
"type": "KSamplerAdvanced",
"pos": [
751.2051818181817,
387.2801749249675
],
"size": [
313.66404687500017,
334
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 35
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 37
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 36
},
{
"name": "latent_image",
"type": "LATENT",
"link": 14
},
{
"name": "end_at_step",
"type": "INT",
"widget": {
"name": "end_at_step"
},
"link": 41
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
11
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "KSamplerAdvanced"
},
"widgets_values": [
"enable",
432432432,
"fixed",
20,
8,
"euler",
"normal",
0,
10,
"enable"
],
"color": "#323",
"bgcolor": "#535"
},
{
"id": 19,
"type": "PrimitiveInt",
"pos": [
478.4070639854157,
902.7007836657053
],
"size": [
210,
82
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "INT",
"type": "INT",
"links": [
41,
42
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "PrimitiveInt"
},
"widgets_values": [
10,
"fixed"
]
}
],
"links": [
[
3,
4,
1,
6,
0,
"CLIP"
],
[
5,
4,
1,
7,
0,
"CLIP"
],
[
11,
11,
0,
13,
3,
"LATENT"
],
[
14,
5,
0,
11,
3,
"LATENT"
],
[
18,
10,
0,
14,
1,
"VAE"
],
[
19,
14,
0,
15,
0,
"IMAGE"
],
[
20,
13,
0,
14,
0,
"LATENT"
],
[
25,
4,
0,
13,
0,
"MODEL"
],
[
31,
7,
0,
13,
2,
"CONDITIONING"
],
[
35,
4,
0,
11,
0,
"MODEL"
],
[
36,
7,
0,
11,
2,
"CONDITIONING"
],
[
37,
6,
0,
11,
1,
"CONDITIONING"
],
[
39,
17,
0,
13,
1,
"CONDITIONING"
],
[
40,
4,
1,
17,
0,
"CLIP"
],
[
41,
19,
0,
11,
4,
"INT"
],
[
42,
19,
0,
13,
4,
"INT"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.8264462809917356,
"offset": [
172.4034738016528,
-169.29037799002282
]
},
"frontendVersion": "1.34.6",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true,
"reroutes": [
{
"id": 1,
"parentId": 9,
"pos": [
693.3972029453568,
271.44254481862447
],
"linkIds": [
25,
35
]
},
{
"id": 2,
"parentId": 1,
"pos": [
1405.7705531263575,
285.30714081616566
],
"linkIds": [
25
]
},
{
"id": 9,
"pos": [
298.24956024523,
279.391142477911
],
"linkIds": [
25,
35
]
},
{
"id": 10,
"pos": [
301.1467983188592,
310.674804664238
],
"linkIds": [
40
]
},
{
"id": 11,
"parentId": 10,
"pos": [
1052.5567983188585,
314.304804664238
],
"linkIds": [
40
]
}
],
"linkExtensions": [
{
"id": 25,
"parentId": 2
},
{
"id": 35,
"parentId": 1
},
{
"id": 40,
"parentId": 11
}
]
},
"version": 0.4
}
このworkflowでは、最初の10stepを赤いりんご、残りを赤いうさぎというプロンプトで生成しています。
もっとわかりやすくするために、切り替えステップを1ずつ変えたときの生成画像を動画にしてみました。
100%りんごだったものが、徐々にうさぎになっていきます。
Stable Diffusion web UIユーザーであれば、Prompt Editingを思い出すかもしれません。実際、似たようなものです。
その中で1stepsごとにプロンプトを切り替えるというものがありましたが、ComfyUIでは、残念ながらカスタムノードがなければ出来ません。
KSampler (Advanced)を20個並べれば出来ますが、まぁ…やりませんね……
モデル(LoRA)の切り替え
同じように、途中からモデルやLoRAを切り替えることもできます。
{
"id": "8b9f7796-0873-4025-be3c-0f997f67f866",
"revision": 0,
"last_node_id": 26,
"last_link_id": 57,
"nodes": [
{
"id": 15,
"type": "SaveImage",
"pos": [
1369.0734598564702,
1065.2134205431485
],
"size": [
354.2876035004722,
433.23967321788405
],
"flags": {},
"order": 12,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 19
}
],
"outputs": [],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33"
},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 5,
"type": "EmptyLatentImage",
"pos": [
442.63634173828075,
742.2317499999979
],
"size": [
244.81999999999994,
106
],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
14
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "EmptyLatentImage"
},
"widgets_values": [
512,
512,
1
]
},
{
"id": 7,
"type": "CLIPTextEncode",
"pos": [
345.8263266015634,
591.7082351074206
],
"size": [
347.2221151367188,
88
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 5
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
36
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"text, watermark"
]
},
{
"id": 24,
"type": "CLIPTextEncode",
"pos": [
345.8263266015634,
1269.3362381724755
],
"size": [
347.2221151367188,
88
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 49
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
53
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"text, watermark"
]
},
{
"id": 14,
"type": "VAEDecode",
"pos": [
1129.886614232605,
1065.2134205431485
],
"size": [
210,
46
],
"flags": {},
"order": 11,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 20
},
{
"name": "vae",
"type": "VAE",
"link": 18
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"slot_index": 0,
"links": [
19
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "VAEDecode"
},
"widgets_values": []
},
{
"id": 10,
"type": "VAELoader",
"pos": [
805.7005593509015,
941.5080075900203
],
"size": [
281.0743801652891,
58
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [
18
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "VAELoader"
},
"widgets_values": [
"vae-ft-mse-840000-ema-pruned.safetensors"
]
},
{
"id": 13,
"type": "KSamplerAdvanced",
"pos": [
779.2538971337385,
1059.1842567431484
],
"size": [
313.66404687500017,
334
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 57
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 52
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 53
},
{
"name": "latent_image",
"type": "LATENT",
"link": 56
},
{
"name": "start_at_step",
"type": "INT",
"widget": {
"name": "start_at_step"
},
"link": 42
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
20
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "KSamplerAdvanced"
},
"widgets_values": [
"disable",
0,
"fixed",
20,
8,
"euler",
"normal",
10,
20,
"disable"
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 25,
"type": "CLIPTextEncode",
"pos": [
345.8263266015634,
1059.1842567431484
],
"size": [
340.95503173828126,
143.0030493164063
],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 50
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
52
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"(best quality, high quality, ultra-detailed),Pixel art, a single apple on a table, photorealistic"
]
},
{
"id": 6,
"type": "CLIPTextEncode",
"pos": [
345.71385545454683,
384.86017492496757
],
"size": [
340.95503173828126,
143.0030493164063
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 3
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
37
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"(best quality, high quality, ultra-detailed),Pixel art,a single apple on a table, photorealistic"
]
},
{
"id": 26,
"type": "LoraLoader",
"pos": [
75.55685619834745,
1059.1842567431484
],
"size": [
233.59501027624788,
126
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 54
},
{
"name": "clip",
"type": "CLIP",
"link": 55
}
],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
57
]
},
{
"name": "CLIP",
"type": "CLIP",
"links": [
49,
50
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "LoraLoader"
},
"widgets_values": [
"PX64NOCAP_epoch_10.safetensors",
1,
1
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 4,
"type": "CheckpointLoaderSimple",
"pos": [
-317.82656380165326,
384.86017492496757
],
"size": [
315,
98
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"slot_index": 0,
"links": [
35,
54
]
},
{
"name": "CLIP",
"type": "CLIP",
"slot_index": 1,
"links": [
3,
5,
55
]
},
{
"name": "VAE",
"type": "VAE",
"slot_index": 2,
"links": []
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.33",
"Node name for S&R": "CheckpointLoaderSimple"
},
"widgets_values": [
"v1-5-pruned-emaonly-fp16.safetensors"
],
"color": "#323",
"bgcolor": "#535"
},
{
"id": 11,
"type": "KSamplerAdvanced",
"pos": [
779.2538971337385,
384.86017492496757
],
"size": [
313.66404687500017,
334
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 35
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 37
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 36
},
{
"name": "latent_image",
"type": "LATENT",
"link": 14
},
{
"name": "end_at_step",
"type": "INT",
"widget": {
"name": "end_at_step"
},
"link": 41
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"links": [
56
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "KSamplerAdvanced"
},
"widgets_values": [
"enable",
99999,
"fixed",
20,
8,
"euler",
"normal",
0,
10,
"enable"
],
"color": "#323",
"bgcolor": "#535"
},
{
"id": 19,
"type": "PrimitiveInt",
"pos": [
478.4070639854157,
902.7007836657053
],
"size": [
210,
82
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "INT",
"type": "INT",
"links": [
41,
42
]
}
],
"properties": {
"cnr_id": "comfy-core",
"ver": "0.3.76",
"Node name for S&R": "PrimitiveInt"
},
"widgets_values": [
6,
"fixed"
]
}
],
"links": [
[
3,
4,
1,
6,
0,
"CLIP"
],
[
5,
4,
1,
7,
0,
"CLIP"
],
[
14,
5,
0,
11,
3,
"LATENT"
],
[
18,
10,
0,
14,
1,
"VAE"
],
[
19,
14,
0,
15,
0,
"IMAGE"
],
[
20,
13,
0,
14,
0,
"LATENT"
],
[
35,
4,
0,
11,
0,
"MODEL"
],
[
36,
7,
0,
11,
2,
"CONDITIONING"
],
[
37,
6,
0,
11,
1,
"CONDITIONING"
],
[
41,
19,
0,
11,
4,
"INT"
],
[
42,
19,
0,
13,
4,
"INT"
],
[
49,
26,
1,
24,
0,
"CLIP"
],
[
50,
26,
1,
25,
0,
"CLIP"
],
[
52,
25,
0,
13,
1,
"CONDITIONING"
],
[
53,
24,
0,
13,
2,
"CONDITIONING"
],
[
54,
4,
0,
26,
0,
"MODEL"
],
[
55,
4,
1,
26,
1,
"CLIP"
],
[
56,
11,
0,
13,
3,
"LATENT"
],
[
57,
26,
0,
13,
0,
"MODEL"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.6830134553650706,
"offset": [
417.82656380165326,
-284.86017492496757
]
},
"frontendVersion": "1.34.6",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true,
"reroutes": [
{
"id": 12,
"pos": [
1120.2227312731427,
737.0464022260336
],
"linkIds": [
56
]
},
{
"id": 13,
"parentId": 12,
"pos": [
758.4327312731427,
920.9664022260334
],
"linkIds": [
56
]
}
],
"linkExtensions": [
{
"id": 56,
"parentId": 13
}
]
},
"version": 0.4
}
ノードが増えて少しややこしくなってきましたね。
ただ、よく見るとtext2imageのworkflowが2つ並んでおり、途中で切り替えているだけです。
最初の6stepをLoRAなしで、残りをLoRAありで生成しています。
これはドット絵にするLoRAですが、LoRAは"ドット絵にする"という概念以外にも、学習素材にした絵の記憶も持ってしまっています。
そのため、LoRAなしで生成した時の画像がそのままドット絵になるわけではありません。
LoRAを使わなかったときと全step適用したときを見比べてみると、ドット絵スタイルになっただけでなく、りんごの形まで変わってしまっています。
拡散モデルは、序盤のステップで形を作り、後半で細部を描いていきます。
形を作る最初のstepはLoRAをかけず、後半だけにLoRAをかけることで、モデル本来の能力を生かしつつLoRAによる絵柄変換ができるのです。
image2image
途中からノイズを追加する、というのはそのままimage2imageの仕組みつながります。
それでは、このままimage2imageの仕組みを見てみましょう。