OPEN-SOURCE DATA WORKFLOW ORCHESTRATION
Love your workflows again
Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
PRODUCTIVITY
Code as Workflows
If Python can write it, Prefect can run it.
FLEXIBILITY
Dynamic Workflows
Finally flows as flexible as you need them to be: generate any task at runtime.
ORCHESTRATION
Batteries Included
Everything you need for a complete orchestration platform.
VISIBILITY
Reimagined UI
Customize your view of your stack.
01
Code as Workflows
Turn any function into a task or flow. If Python can write it, Prefect can orchestrate it.
- from prefect import flow, task
- from typing import List
- import httpx
- @task(retries = 3)
- def print_stars(repo:str):
- url = f"https://api.github.com/repos/{repo}"
- count = httpx.get(url).json()["stargazers_count"]
- print(f"{repo} has {count} stars!")
- @flow(name="Github Stars")
- def stars(repos:List[str]):
- for repo in repos:
- print_stars(repo)
- from prefect import flow, task
- from typing import List
- import httpx
- @task(retries = 3)
- def print_stars(repo:str):
- url = f"https://api.github.com/repos/{repo}"
- count = httpx.get(url).json()["stargazers_count"]
- print(f"{repo} has {count} stars!")
- @flow(name="Github Stars")
- def stars(repos:List[str]):
- for repo in repos:
- print_stars(repo)
Continuous Workflows
Use for or while loops to create infinitely-long flow runs, no schedule required.
Event-Based Workflows
Trigger flows from external events such as file or system updates.
Versioning
Know when flow code changes so that you can quickly diagnose and resolve failures.
Asynchronous Task Support
Run concurrent tasks independently so flows can run faster and elegantly handle blocking.
Streaming Support
Launch new flow runs in response to events from streaming systems or message queues.
02
Goodbye, DAGs
Design dynamics workflows that generate tasks at runtime, for more use cases with less code.
03
Batteries Included
Spin up a full orchestration platform from your CLI or deploy for high availability.
05
Ready for your data stack
Prefect has dozens of integrations with the most popular tools across the modern data stack....or you can build your own.
And coming soon...
Any Language
R? Julia? Use the language you love, not just Python. Now more team members can contribute and cover more use cases.
Third Party Logging
Enable workflow logging to third party systems such as Datadog to gain cross-infrastructure insights.
Per Task Environments
Run any flow or task in a specific environment so you can do more from one code base.
Serverless Support
Going serverless? So are we. Execute flows without the need for persistent infrastructure.
GET PLUGGED IN
Come for the code. Stay for the community.
Browse through our community knowledge base and become a contributor.
Try the Tutorial
Work through a simple example with Prefect 2.0 to see the basics in action.
TRY THE TUTORIAL