OPEN-SOURCE DATA WORKFLOW ORCHESTRATION
Dataflows as easy as .py
Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.
SAVE TIME
Build 2x faster
Get started quickly with our Python framework and tons of examples right from your local machine.
INCREASE RELIABILITY
Reduce Failures
Easily add automatic failure detection and handling to your pipelines.
AUTOMATE WORKFLOWS
Supercharge Your Pipelines
Make pipelines smarter with granular scheduling, dependencies, parameters and more.
INTEGRATE TOOLS
Connect It All
Connect your entire data stack with a broad library of pre-built integrations.
SPEED
Fast out of the gate
Prefect is just Python, so you can make code-based data processes into intelligent tasks and flows quickly. Code is easily readable and maintainable, allowing even non-data engineers to build their own flows.
Check Out The Tutorial →
RESILIENCY
Fewer failures, greater peace of mind
Use features such as retries, restarts, and timeouts to automatically and elegantly handle any error or failure. Know when problems occur with real-time notifications, and easily identify the root causes to get back up and running quickly.
Learn More →LIMITLESS
Your workflows, supercharged
If you can think it, you can build it in Prefect. Run tasks in parallel, pass data between them, and create interflow dependencies. And that's just the start.
Learn More →
INTEGRATIONS
Take us to your data stack
Prefect has dozens of integrations with the most popular tools across the modern data stack... or you can build your own. With extensive examples and community support, there’s no task you won’t be able to tackle.
View our integrations →Plus other features
Hover on one of the many features below to view a description.
Plus other features
Secure by Design
With our Hybrid Execution Model, Prefect never sees your code or data, so it can be run in the most sensitive environments.
Universal Deploy
Anywhere you can run Python, you can run Prefect. Instantly deploy your flows and monitor runs from Prefect's UI, no Docker required.
Parameterization
Add parameters to any flow for easy runtime templating and reuse.
Inter-task Communication
Pass data between tasks for complex processing and advanced analytics.
Dynamic Mapping
Powerful map/reduce operators generate dynamic tasks for each element of an input. Mapped tasks can be linked to create parallel pipelines.
Time Travel
Prefect task outputs can be cached or updated at different intervals, even within the same workflow.
Result Handlers
Serialize data in and out of your tasks with customizable result handlers, including local filesystems, S3, and GCS.
Looping
Loop tasks with arbitrary control logic.
Depth-First Execution
Race through mapped pipelines by allowing tasks to start before all tasks of the previous stage have finished.
Event-Driven Flows
Fire off flow runs in response to external events of any frequency.
GET PLUGGED IN
Come for the code. Stay for the community.
Browse through our community knowledge base and become a contributor.