Introducing Radar
SHARE
Nov. 22, 2021

Introducing Prefect Radar: A new way of visualizing workflows that elegantly clarifies their execution at any level of scale.

Bill Palombi
Bill PalombiHead of Product

Orion, our second-generation workflow engine, can orchestrate dynamic, DAG-free workflows. This evolution was necessary to handle the rich expressiveness of modern, on-demand, dynamic, highly scalable workflows.

DAG-based workflow engines require that all potential task execution paths be explicitly registered before the workflow can run. With Orion, this constraint is dropped. Task execution paths may not be known to Orion prior to a run—the graph “unfolds” as execution proceeds. Radar embraces this dynamism, giving users the clearest possible view of their workflows.

radar running

gif illustrating Prefect Radar updates in real time as the flow run proceeds

Orion’s Radar is based on a structured, radial canvas upon which tasks are rendered as they are orchestrated. We developed a new layout algorithm that updates Radar as tasks run. It’s not just designed to handle simple workflows, but also those with massive dynamic fan-out, fan-in, sidecar tasks, and complex references. The algorithm optimizes readability through consistent node placement and minimal edge crossings. Users can zoom and pan across the canvas to discover and inspect tasks of interest. The mini-map, edge tracing, and node selection tools make workflow inspection a breeze.

radar nav

Gif illustration of Prefect Radar's navigation tools, which allow you to quickly find task runs of interest

Prefect users regularly run workflows with hundreds of thousands, sometimes even millions, of tasks. Orion is built not only to handle this scale, but to allow users to work with it in a meaningful way. Having made the engine itself much more powerful, Prefect’s visualization of the task execution graph needed an upgrade. With Radar, Orion is the only workflow orchestrator capable of performantly rendering large graphs at the scale our users need.

radar overflow

Gif illustration of Prefect Radar scaling to handle large volumes of task runs

With Orion’s first-class support for infinitely-nested subflow runs, we expect users to leverage them often. Radar supports direct click-through to a subflow from its parent, enabling users to move seamlessly between task execution graphs.

radar subflow

gif illustrating Prefect Radar; move from parent flow runs to their subflow runs seamlessly

Radar is Prefect’s latest step towards making you love your workflows again. We’re excited to share it with you. Please install itrun some flows, and share your feedback in our Slack community. We are continuing to design features for Radar, including the ability to monitor data, artifacts, and progress as the system processes. Please note that Orion is still a technical preview, an alpha release of Prefect 2.0. Check out the Orion announcement to learn more.

Happy engineering!

Posted on Nov 22, 2021
Blog Post
Other
Monitoring
Other
RELATED LINKS

Love your workflows again

Orchestrate your stack to gain confidence in your data