Technical Preview
Love your workflows again
Orion is the second-generation orchestration engine from Prefect.
Not just a new version, it's a whole new way of working.
FLEXIBLE
No DAGs Required
Finally, flows as flexible as you need them to be. If Python can write it, Prefect can run it.
LEARN MORE
DYNAMIC
Go with the Flow
Generate tasks or even nested flows dynamically, in response to runtime conditions.
LEARN MORE
EASY
Batteries Included
All activity is automatically tracked in a beautiful new UI, no setup required.
LEARN MORE
NATIVE
Brand-New Engine
Use if, for, and while loops to control execution and asyncio for concurrency.
LEARN MORE- from prefect import flow, task
- from typing import List
- import httpx
- @task(retries=3)
- def print_stars(repo: str):
- url = f"https://api.github.com/repos/{repo}"
- count = httpx.get(url).json()["stargazers_count"]
- print(f"{repo} has {count} stars!")
- @flow(name="Github Stars")
- def stars(repos: List[str]):
- for repo in repos:
- print_stars(repo)
- stars(["PrefectHQ/Prefect", "PrefectHQ/miter-design"])
Code as Workflows
With Orion, your functions are your flows. Add orchestration features only where you need them. Call flows interactively to see the UI update, or deploy them for automatic scheduling.
LEARN MOREThe flow never stops
Generate tasks in loops to maintain infinitely-long flow runs.
Event based runs
Run any flow, at any time, for any reason, from any place, with or without an API.
Incremental adoption
Take advantage of as much - or as little - orchestration as you want.
Flows all the way down
Organize code as infinitely nested subflows.
Streaming
Launch new flow runs in response to events from streaming systems or message queues.
Loops
Use familiar for and while loops to build dynamic flows.
Conditional logic
Use standard Python if statements to build complex conditional logic.
Tasks as functions
Turn any function into a Prefect task to immediately supercharge its execution.
Flows as functions
Turn any function into a Prefect flow to expose an API for scheduling and running it.
Radial Schematics
See your flows from a whole new angle. Orion's new Miter Design system is optimized for representing dynamic workflow data.
LEARN MORE
Async
Orion fully supports asynchronous tasks, making concurrent parallelism easy.
Scheduling
Trigger runs automatically using cron, intervals, or the iCalendar recurrence standard.
Database Options
Keep it simple with embedded SQLite or use Postgres for heavier workloads.
Get Warmed Up
Prefetch flow runs to give your infrastructure enough time to spin up, then start the run exactly when you expect.
Caching
Highly-configurable task caching both within and across runs.
Dask-native
Harness the power of parallelism with first class Dask execution.
Mapping
Dynamically execute parallel tasks through a flexible map/reduce model.
Performance Optimized
Fast enough to keep up with even the most powerful infrastructure.
COMING SOON
No-code deployments
Create and deploy flows straight from the UI.
COMING SOON
Multi-language support
Don't stop at Python. Orchestrate code in other languages such as R or Julia.
COMING SOON
Third-party Logging
Send logs to your preferred logging system.
COMING SOON
Per-task Environments
Compose flows of tasks to be executed in separate environments.
COMING SOON
Serverless Execution
Trigger your favorite serverless compute tooling to execute flows without persistent infrastructure.
COMING SOON
Storage validation
Configure and validate storage before running your flow.
COMING SOON
Linear Runs
Ensure that only one run of a flow executes at a time.
Always On Call
Understand your workflows' health through metadata visibility and analytics. Orion lets you quickly filter to get exactly the information you need.
LEARN MOREPlay by the rules
Orchestration is governed transparently and consistently through the rules engine.
Deployments
Deploy the same flow to multiple environments, schedules, parameters, or configurations.
Share state
Reuse stateful Python objects, like database connections, across multiple tasks.
Typed parameters
Automatically parse and validate parameters using familiar Python annotations.
Built on Pydantic
Orion’s serialization engine is built on Pydantic for easy extension and customization.
Data documents
Self-describing data objects allow transparent data serialization and lineage.
Custom states
Create named states to denote conditions specific to your system.
Versioning
Automatically track changes to flows and tasks by associating them with fixed versions.
Tag it
Tag every object for flexible organization and discovery.
Start small
Use an embedded database and API by default to leverage Orion without ever starting a server.
Local testing
Zero-compromise local testing, including the full power of the Orion engine.
Write Once, Run Anywhere
Run any flow locally or against multiple Orion backends without modification or registration.
Documentation
Hit the ground running with clear, concise, idiomatic documentation.
Open-Source
Orion is, and always will be, open-source.
Community-Approved
Designed in partnership with Prefect's developer community.
Ready for CI
Run flows, update deployments, and track versions, all from the comfort of your favorite CI tool.
COMING SOON
Configurable storage
Flow storage is independent from flow execution.
COMING SOON
Continuous integration and deployment
Integrate with your preferred preferred CI/CD systems to ensure that you're always running the latest stable version.
COMING SOON
Task expectations
Pre-registration of a flow would allow us to display "expected" steps. For example, in a CI script, users will want to see all expected steps before they run.
COMING SOON
Upgrade from Airflow
Forward Airflow states to Orion to upgrade effortlessly.
COMING SOON
Strongly typed
Track input, output, and intermediate lineage using Python’s builtin annotation system.
COMING SOON
Track Progress
Publish progress from task runs as they execute.
Team Player
The first orchestration engine for the modern data stack, Orion integrates seamlessly with the entire PyData ecosystem and the tools you already use.
LEARN MORE
Miter Design
Prefect's clean, consistent design system takes the UI to the next level.
Dashboard
Define custom UI dashboards with advanced queries.
Accessible
The UI includes built-in support for colorblind themes.
Metadata insights
Understand system health through run analytics.
Run visibility
See flow and task runs as they happened with visualizations.
See into the future
See whats coming by visualizing flow run schedules well into the future.
COMING SOON
Prefect IDE
Load runs into a dedicated IDE for interactive debugging.
COMING SOON
Time travel
Download and replay failed runs to examine data interactively.
