Prefect x Kubernetes x Ephemeral Dask: Power, without responsibility

Prefect is rapidly becoming the standard in dataflow automation. In our last blog we briefly ran through using Prefect Executors to parallelize on a single node.

July 19, 2021

Workflow Orchestration with Prefect and Coiled

July 16, 2021

The Data Exchange Podcast: Chris White on building an engine that enables arbitrary proprietary workflows.

July 15, 2021

Parallel Prefect: A bit about Dask, real fast

The problem: I heard Prefect can run Python in parallel, what is the simplest way to achieve this?

July 8, 2021

Prefect 0.15.0: A New Flow Run Experience

An overhauled flow run CLI, sub-flow result passing, agentless execution, improved run observability, new auth patterns, and more!

July 1, 2021

Escape Velocity: Prefect's Series B

Following our $32M Series B, we're scaling our team, building new products, and forming A More Prefect Union.

June 10, 2021

Prefect Announces $32M Series B Funding Led by Tiger Global to Deliver on Growing Demand for Dataflow Automation

"Any engineer delivering custom code for their organization can easily reap the benefits of Prefect's platform, creating a massive addressable market for the company to capture." Official press release

June 10, 2021

Data flow automation engine Prefect raises $32M

"Prefect’s success aligns with broader industry trends in digital transformation...data flows have raised the global GDP by at least 10% over a decade. The value totaled $7.8 trillion in 2014 alone, contributing to economic growth primarily by raising productivity." Coverage by VentureBeat

June 10, 2021

Prefect raises $32M for dataflow automation technology

"We see the data stack as a whole just exploding in terms of attention and the need for Prefect is growing exponentially." Coverage by TechTarget

June 10, 2021

Prefect Secures $32 Million for Its Dataflow Automation Platform

“Our mission is to help data engineers and data scientists eliminate negative engineering by delivering a world-class orchestration tool for today’s modern data stack" Coverage by StartUp Savant

June 10, 2021

D.C. startup raises tens of millions of dollars in second venture round of 2021

"The new capital will let Prefect hire up, grow its commercial business and further develop its platform" Coverage by Washington Business Journal

June 10, 2021

DC’s Prefect already raised two funding rounds in 2021. Here’s what’s next for the dataflow automation company

"We are chasing our own growth." Coverage by Technical.ly

June 10, 2021

Prefect Raises $32M To Be The ‘Toothpick’ In Companies’ Data Stack

“Suitors will always come, but Prefect has all of the elements to be an enduring business that forges its own path,” said Ethan Kurzweil of BVP. Coverage by Crunchbase News

June 10, 2021

Introducing: The Prefect KV Store

A flexible building block for workflow metadata

June 3, 2021

MLOps: Task and Workflow Orchestration Tools on Kubernetes

If you need a simple and clear answer―use Prefect.

May 28, 2021

Orchestrating Image Preprocessing Workflows for Label Studio with Prefect

Kevin Kho, Open Source Community Engineer at Prefect, showcases Prefect workflows with Label Studio

May 26, 2021

Prefect + Microsoft: Advancing Dataflow Automation Together

Unveiling the first details of our deepening relationship with Microsoft.

May 21, 2021

Enhance positive engineering with Prefect and Microsoft for Startups

We are delighted to welcome Prefect into the Microsoft for Startups portfolio. Working with Prefect will help our joint customers easily deploy on trusted infrastructure with the convenience of Prefect Cloud.

May 21, 2021

If you give a data engineer a job...

May 11, 2021

Don’t Panic! The Prefect Guide to Building a High-Performance Team

Building the startup is just as important as building the product. Here's how we establish a high-performance and collaborative culture.

April 28, 2021

Introducing a Dataflow Management System Backed Up by Prefect, AWS, and Github Actions

Take your local Prefect workflows to the next level by provisioning AWS & Prefect Cloud infrastructure, with Github Actions playing an important role.

April 27, 2021

Best Workflow and Pipeline Orchestration Tools – Machine Learning Guide

"I believe that [Prefect] is one of the best automated workflow management tools out there."

April 20, 2021

Data Engineering & Business Intelligence – An Exciting Quest

Prefect. dbt. Redshift. BI tools. Learn about Cloud Academy Staff Engineer Giuliano Sofi's (literally) epic journey to redefine and transform their data and reporting!

April 16, 2021

Building a Healthy Open Source Community with Jeremiah Lowin, Founder and CEO of Prefect

Join Jeremiah Lowin and Anu Sharma for a deep dive into open-source community building from 0 to 5,000.

April 9, 2021

Simple, Success-Based Pricing

Eliminate negative engineering with Prefect Cloud’s new success-based pricing, a generous free tier, and automatic discounts.

March 24, 2021

Liftoff: Prefect's Series A

Welcoming a new cohort of strategic investors.

February 25, 2021

GitHub CEO Joins Prefect’s $11.5 Million Funding Round for Dataflow Automation

Watch Jeremiah Lowin's interview about our Series A, future plans, and upcoming announcements!

February 17, 2021

Prefect Raises $11.5M in Series A Funding to Eliminate Negative Engineering in Dataflow Automation

February 9, 2021

Liftoff

A preview of our upcoming announcements, including our Series A, Microsoft partnership, updated product vision, and new community programs.

February 9, 2021

Serverless Data Pipelines Made Easy with Prefect and AWS ECS Fargate

Even though there are so many workflow orchestration solutions and cloud services for building data workloads, it’s hard to find one which is actually pleasant to use and allows you to get started quickly. One of my favorite tools for building data pipelines in Python is Prefect — a workflow management platform with a hybrid agent-based execution model.

February 8, 2021

Prefect Workflow Automation with Azure DevOps and AKS

The team at Infinite Lambda shares how to automate Prefect flow deployments on Azure.

February 5, 2021

Understanding Great Expectations and How to Use It

Use Prefect and Great Expectations to load valid data into Snowflake.

February 4, 2021

Getting Started with Parallel Computation

Prefect provides many simple and expressive building blocks that users can snap together to add advanced features to their workflows. This post shows how easily Prefect tasks can be executed in parallel without requiring deep knowledge of distributed computing. In fact, it only requires two extra lines of code!

January 20, 2021

What We Love About Prefect

It’s open source, plays nicely with Kubernetes, and there’s a great community! We love how easy it was to set this up, and the clean UI and dashboards make it a pleasure to use.

January 15, 2021

Livestream - 0.14.0: your deploy just got easier

December 18, 2020

Prefect 0.14.0: Improved Flow Run Configuration

We’re excited to announce the release of Prefect 0.14.0

December 17, 2020

Introducing: The Artifacts API

Create native Prefect UI integrations with one line of code

December 8, 2020

Getting started with operationalizing your Python code

In the first part of this blog series we saw how quickly Prefect can help you run parallelized Python code from a fresh OS install

December 2, 2020

Scheduling & Optimizing RAPIDS Workflows with Dask and Prefect

Scheduling and optimizing accelerated data science workflows is really easy with Prefect, RAPIDS, and Dask. Get started with this tutorial.

December 9, 2020

Integrating Prefect & Databricks to Manage your Spark Jobs

This tutorial explains how to run Databricks notebooks and Spark jobs in Prefect flows.

December 3, 2020

How to Automate Python Workflows in Prefect (Step-by-Step Guide)

There was quite a steep learning curve for me to pick up Airflow. Thankfully, in the midst of my struggles, I discovered a library that would make the entire process ridiculously simple. Enter Prefect.

November 21, 2020

The Prefect Task Library

The Prefect community is a welcoming place for developers of all experience levels, and our task library is an excellent way to get involved with contributing to an open-source project.

November 8, 2020

How Prefect Kept my Family Cool in the Pool this Summer

The data workflow platform Prefect is used commercially for serious data engineering and machine learning pipelines. It's being rapidly adopted in industries like financial services, e-commerce, healthcare, and many others.

September 22, 2020

Managing dependencies between data pipelines in Apache Airflow & Prefect

A simple approach to managing dependencies between your workflows.

September 4, 2020

Distributed data pipelines made easy with AWS EKS and Prefect

Anna shows you how you can obtain a highly available, scalable, distributed system that will make the orchestration of your data pipelines for ETL & ML much more enjoyable

August 25, 2020

The All-New Prefect Server and UI

Two key Prefect products are becoming open-source projects in our biggest release ever.

August 6, 2020

Workflow Automation: Empowering teams through our in-house, self-service framework

See how the Journal team used Prefect to build a self-serve automation framework for their data teams

July 20, 2020

Empiric Health Leverages Prefect’s Hybrid Model for HIPAA Compliance & Platform Stability

Empiric Health makes patient care decisions with predictive analytics, reducing patient costs and improving surgical outcomes.

January 7, 2020

Introducing: Prefect Universal Deploy

We’re excited to announce our best release yet: Prefect 0.8.0

December 11, 2019

Snowflake: the details of our first Data Warehousing project in the Cloud

Learn how Trade Me adopted Prefect to build a workflow for Snowflake.

July 28, 2019

Toward Dataflow Automation

Prefect Cloud is coming

June 10, 2019

Why Not Airflow?

An overview of the Prefect engine for Airflow users

May 1, 2019

Pipeline Pitfalls

Data engineers are often responsible for the planning, building and maintaining of data pipelines. During that process, they may face the challenging decision to either create a custom system or use an existing framework.

May 1, 2019

Prefect is Open-Source! 🎉

Today, on its two-year anniversary, we’re excited to announce that Prefect has launched as an open-source project with an Apache 2.0 license.

March 24, 2019

Prefect Runs on Prefect

Every morning, Marvin runs standup for our team. That’s not unusual for a tech startup, except that Marvin is actually a Prefect workflow.

January 17, 2019

The Golden Spike

A Prefect Update

January 9, 2019

Positive and Negative Engineering

Don't panic.

November 1, 2018