An open-source workflow
management system

Design
/
Build
/
Test
/
Run

An Engine for Innovation

The prefect Python library includes everything you need to design, build, test, and run powerful data applications. Instantly upgrade your existing code with workflow best practices, and use the Prefect UI to orchestrate and monitor everything.

1
2
3
4
5
6
7
8
9
10
11
12
13
from prefect import task, Flow

@task
def say_hello():
    print("Hello, world!")


with Flow("My First Flow") as flow:
    say_hello()


flow.run() # "Hello, world!"

Python is the API

Prefect's beautiful API is familiar the first time you see it. No new languages, no config files, and no boilerplate.

Realtime UI

Prefect's beautiful UI lets you keep an eye on the state of your flows.

Stream realtime state updates and logs, kick off new runs, and receive critical information exactly when you need it.

Task Library

Prefect's growing task library means you don't need to write code for common tasks. Just import prefect and go.

State of the Art

Tasks exchange information via rich state objects.
Users can implement custom logic to react to states.

Practice Makes Prefect

Developed in partnership with hundreds of data scientists and engineers to ensure compliance with best practices, Prefect Core has been successfully deployed everywhere from data-science bootcamps to profressional sports teams to Fortune-100 companies.

and more...

New

Realtime UI

The Prefect UI updates in realtime so you're never behind.

New

Universal Deploy

Anywhere you can run Python, you can run Prefect. Instantly deploy your flows and monitor runs from Prefect's UI, no Docker required.

Flow code

Prefect flows are plain old Python, so you can build and modify them however you like.

Parameters

Add parameters to any flow for easy runtime templating and reuse.

Robust states

Prefect handles every error, whether expected or not. Some tasks might only run if upstream tasks fail.

Dataflow

Pass data between tasks for complex processing and advanced analytics.

Mapping

Powerful map/reduce operators generate dynamic tasks for each element of an input. Mapped tasks can be linked to create parallel pipelines.

Environments

A flexible environment model means flows can be deployed anywhere from a laptop to multi-cloud clusters.

Realtime

When paired with Dask, Prefect's event-driven scheduler can execute tasks with millisecond latency.

Time Travel

Prefect task outputs can be cached or updated at different intervals, even within the same workflow.

Result Handlers

Serialize data in and out of your tasks with customizable result handlers, including local filesystems, S3, and GCS.

Custom Schedules

Specify custom schedule logic including business days, offsets, and blackout windows, or fall back on good old cron.

Looping

Loop tasks with arbitrary control logic.

Coming Soon

Event-Driven Flows

Fire off flow runs in response to external events of any frequency.

Coming Soon

Task Affinity

Run each of a flow's tasks in a completely different environment, including new dependencies or platforms.

Coming Soon

Depth-First Execution

Race through mapped pipelines by allowing tasks to start before all tasks of the previous stage have finished.

OPEN SOURCE

Get Started Now

See why Prefect Core is the easiest, most powerful workflow engine around.

Join our Community

Our public Slack is full of helpful tips and support

A complete solution for dataflow automation