ComfyUI Isn't Built for Realtime Processing

To start with the basics: ComfyUI is not well suited to realtime processing. Everything runs as a single workflow, and nothing else can happen until that run finishes.

That said, by automatically feeding in the next run as soon as one finishes, you can make it look like realtime.

Run(Instant) / Run(Change)

ComfyUI has two run modes for this.

Run (Instant)

  • Once execution starts, the same workflow is automatically re-executed every time processing finishes
  • To stop it, switch to another mode (it won't stop on its own)

Run (On Change)

  • Runs only when a value, such as a slider, changes
  • A run is queued automatically every time you move the mouse

About Realtime image2image / video2video

In theory, running image2image / video2video on every frame could get you something close to "realtime video processing." But generative AI inference always takes time, so true realtime processing is very hard.

Recent research is bringing that day closer, but for now it still takes a lot of patient optimization.

  • Lower resolution
  • Switch to lightweight models
  • Use speed-up techniques made for video (covered on another page)

"Realtime image2image" is a special kind of workflow that depends on speed-up techniques.