Nov. 22, 2021
Introducing Radar
Introducing Prefect Radar: A new way of visualizing workflows that elegantly clarifies their execution at any level of scale.
A new way of visualizing workflows that elegantly clarifies their execution at any level of scale.
Orion, our second-generation workflow engine, can orchestrate dynamic, DAG-free workflows. This evolution was necessary to handle the rich expressiveness of modern, on-demand, dynamic, highly scalable workflows.
DAG-based workflow engines require that all potential task execution paths be explicitly registered before the workflow can run. With Orion, this constraint is dropped. Task execution paths may not be known to Orion prior to a run—the graph “unfolds” as execution proceeds. Radar embraces this dynamism, giving users the clearest possible view of their workflows.
gif illustrating Prefect Radar updates in real time as the flow run proceeds
Orion’s Radar is based on a structured, radial canvas upon which tasks are rendered as they are orchestrated. We developed a new layout algorithm that updates Radar as tasks run. It’s not just designed to handle simple workflows, but also those with massive dynamic fan-out, fan-in, sidecar tasks, and complex references. The algorithm optimizes readability through consistent node placement and minimal edge crossings. Users can zoom and pan across the canvas to discover and inspect tasks of interest. The mini-map, edge tracing, and node selection tools make workflow inspection a breeze.
Gif illustration of Prefect Radar's navigation tools, which allow you to quickly find task runs of interest
Prefect users regularly run workflows with hundreds of thousands, sometimes even millions, of tasks. Orion is built not only to handle this scale, but to allow users to work with it in a meaningful way. Having made the engine itself much more powerful, Prefect’s visualization of the task execution graph needed an upgrade. With Radar, Orion is the only workflow orchestrator capable of performantly rendering large graphs at the scale our users need.
Gif illustration of Prefect Radar scaling to handle large volumes of task runs
With Orion’s first-class support for infinitely-nested subflow runs, we expect users to leverage them often. Radar supports direct click-through to a subflow from its parent, enabling users to move seamlessly between task execution graphs.
gif illustrating Prefect Radar; move from parent flow runs to their subflow runs seamlessly
Radar is Prefect’s latest step towards making you love your workflows again. We’re excited to share it with you. Please install it, run some flows, and share your feedback in our Slack community. We are continuing to design features for Radar, including the ability to monitor data, artifacts, and progress as the system processes. Please note that Orion is still a technical preview, an alpha release of Prefect 2.0. Check out the Orion announcement to learn more.
Happy engineering!