Negative Engineering and the Art of Failing Successfully

A definition of negative engineering, and how to solve it.

May 19, 2022

Prefect + Gradient Flow Partner to Produce the State of Workflow Orchestration Report

For the first ever State of Workflow Orchestration Report, Prefect partnered with Gradient Flow to survey almost 600 data practitioners from a wide variety of industries. We explored the current and future state of workflow orchestration, focusing on the key uses and priority features of orchestration tools.

May 17, 2022

Prefect 2.0b4: A slew of updates

Community contributions, flow and task runner improvements, collections, QOL, bug squashing and more.

May 13, 2022

Telling Your Prefect Story

We want to help you tell your story about Prefect, including covering the cost of getting there.

May 10, 2022

Case Study: Rent the Runway + Prefect

How the innovative team at Rent the Runway builds a high-performing and fun tech stack, with Prefect at the core.

May 6, 2022

Extracting Youtube Data with Prefect

In part two of James Sopkin's social media series, we explore extracting and loading data from Youtube with Prefect.

May 5, 2022

Workflow Orchestration vs Data Orchestration

Are they different? Anna Geller disambiguates the terms!

May 4, 2022

It’s SNOWing in DC: Prefect and Snowflake Partner for a Better Modern Data Stack

April 28, 2022

Deploying Prefect flows with Github Actions

Store & register your prefect workflows with Github Actions

April 27, 2022

Why Prefect Invested in Textualize

Happy engineering, coming soon to a TUI near you!

April 25, 2022

A Brief History of Dataflow Automation

How did we get to modern dataflow automation? Bill Palombi takes us down memory lane, starting with cron.

April 21, 2022

Better monitoring with Prefect and Sentry

Read how Coveo integrated Sentry with Prefect to repeatably run and gain visibility into the performance of their dataflows.

April 21, 2022

How to Automate dbt Exposures Creation via Crawlers and Github Actions

This article demonstrates dbt exposures and how you can build them automatically in your CI pipeline using Github Actions.

April 19, 2022

Making ML Pipelines Invincible with Prefect

Mitali Sanwal of Saturn Cloud demonstrates how to make your data pipelines fault tolerant with Prefect in this short tutorial video.

April 18, 2022

Scaling screenscraping with Prefect and GCP

Ben Welsh, LA Times journalist and John S. Knight Fellow at Stanford University working on the Big Local News project, joins PrefectLive to demonstrate how he scales one of the largest screenscraping projects in the world.

April 14, 2022

Attending to Cloud Performance

We've heard that our UI was not as performant as users wanted when dealing with the scale of data our users regularly use in production workflows. In response to this feedback, my first objective as the new Senior Front-end Developer at Prefect is to discover and alleviate these issues in Cloud UI.

April 12, 2022

Expect Great Data: Using Great Expectations With Prefect to Ensure Data Quality

By adding Great Expectations quality checks into a Prefect workflow, data teams can increase overall data reliability and confidence.

April 8, 2022

Get Integrated to Google Cloud Platform With prefect-gcp

A new Prefect 2.0 Collection, prefect-gcp, was recently released. It consists of prebuilt Prefect integrations to easily interact with Google Cloud Platform within Prefect flows.

April 4, 2022

Video: Continuous Data Validation with Great Expectations and Prefect

This video is an introduction to Prefect and the Prefect Great Expectations integration.

March 31, 2022

Continuous Data Validation with Great Expectations and Prefect

In this article, we’ll walk through how Great Expectations can easily be incorporated into Prefect flows and the opportunities that unlocks.

March 31, 2022

From PDF to Snowflake

Today we will be extracting a financial table from a PDF and loading it into Snowflake using tabula-py for extracting tables, pandas to clean and shape our tables, and Prefect to run this flow for us.

March 29, 2022

Prefect Named a Top Orchestration Tool in the Data50

Last week Andreesen Horowitz (a16z) released their inaugural Data50 awards, a list of the world’s top data startups, and we’re proud to say we were included.

March 28, 2022

Announcing the Terraform Module to Deploy the Prefect Docker Agent on AWS EC2

This blog post will walk through how to deploy the Docker agent on AWS EC2 using Terraform.

March 24, 2022

Why 90% of Prefect job offers were accepted in 2021

COO Sarah Moses opens a window into our hiring process

March 23, 2022

Prefect is in the Data50

Prefect was named a top orchestration tool by a16z.

March 23, 2022

Launch Week Recap

That's a wrap!

March 22, 2022

Jeremiah on Datacast Podcast

Datacase episode 86: Risk management, open-source governance, and negative engineering with Jeremiah Lowin

March 21, 2022

Data-Driven AI

Launching Human-in-the-loop Processes on Toloka using Prefect.

March 21, 2022

Scaling up Prefect with GitStorage

Prefect Git Storage provides a workflow for developing and deploying data pipelines directly from git repositories, such as Gitlab or Github. The data pipeline code (python) is pulled from the git repository on each invocation with the ability to reference git branches and git tags.

March 21, 2022

Introducing Prefect Cloud 2.0

The next generation of orchestration is here.

March 18, 2022

Collections in Prefect 2.0

New with Prefect 2.0, we’re announcing Collections: a faster way to connect your data stack

March 17, 2022

SOC it 2 Us: We're Certified!

Prefect has attained SOC 2 Type II compliance!

March 17, 2022

Introducing Marvin's Challenge

I have decided to entertain myself by extending a monthly challenge to you, the Prefect community. Each month, I’ll announce my challenge, and you will all scramble to impress me in exchange for exclusive swag and recognition in the community.

March 16, 2022

Moving Your Prefect Flow From Local to the Cloud.

Learnings from moving testing locally and then to the cloud.

March 16, 2022

Join the Club: Introducing Prefect’s Club 42

Club 42 is the Prefect ambassador program, composed of our most passionate and enthusiastic community members.

March 16, 2022

Introducing Prefect 2.0

Love your workflows again.

March 15, 2022

Welcome to the first ever Prefect Launch Week!

We're making huge announcements every day, from March 14th to March 18th!

March 14, 2022

Jeremiah Lowin on Hashmap on Tap

Listen to hear how we got started and how we eliminate negative engineering for all!

March 14, 2022

Zero Outbound Marketing

Listen to our Head of Growth chat about our developer-friendly sales & marketing strategies.

March 10, 2022

Easy Event-Driven Workflows With AWS Lambda V2 Featuring Serverless Framework

Prefect makes it easy to do hard work.

March 8, 2022

Orchestrating ML with Prefect

Prefect's Kevin Kho appears on the MLOps Community Meetup.

March 4, 2022

Prefect Wins the MDS Rocketship Award 🚀 for Workflow Orchestration

March 3, 2022

Open-Source Spotlight: Kevin Kho

Prefect: The easiest way to automate your data. Our own Kevin Kho was the latest guest on Data Talks Club.

March 2, 2022

Prefect Core is Now Prefect 1.0

We’re excited to announce that Prefect Core is now officially Prefect 1.0! This release is the product of four years of invention and refinement. In recent months, we’ve been stabilizing interfaces, improving performance, and removing deprecated features. After rigorous internal testing, we published a 1.0 release candidate and invited feedback from our community a month ago. Prefect 1.0 is a rock solid, stable release.

February 23, 2022

We Didn't Say It, She Did!

"Airflow, there’s a new competitor in town." Make your data pipeline orchestration future-proof with Prefect.

February 18, 2022

Say Hi to Prefect Discourse — a New Forum for Data Engineers

Why we chose Discourse and how you can benefit from this new Prefect Community forum

February 16, 2022

Introducing Prefect-ML: Orchestrate a Distributed Hyperparameter Grid Search on Dask

Use Prefect as a machine learning experiment tracker by leveraging mapping and artifacts. By leveraging the built-in DaskExecutor, we can parallelize training ML models distributedly over a Dask cluster. This blog will show how Prefect’s mapping can be used as an interface for hyperparameter training.

February 14, 2022

My Experience with Prefect so Far (and a quick getting started guide)

Recently I’ve seen buzz rising about Prefect and wanted to check it out. So the past few weeks I’ve been learning, testing, and setting up Prefect and I wanted to share my thoughts so far.

February 11, 2022

Logs, the Prefect Way

Introducing Orion logging: A Pythonic logging system designed to maximize observability with a minimum of effort.

February 4, 2022

How to Deploy a Prefect Agent to Google Kubernetes Engine

This post contains code and commands you can use to deploy Prefect agents to Google Cloud’s Google Kubernetes Engine. The agents stand ready to execute workflows triggered by Prefect projects. One agent can run tasks from multiple projects.

February 4, 2022

Announcing Prefect integration with Airbyte to automate ELT pipelines

In a step to help data teams have better control over their data pipelines, Airbyte announces today its partnership and integration with Prefect. Prefect provides the leading dataflow automation platform that reduces engineering time and removes headaches for data engineers and data scientists by automating pipelines and workflows of any complexity. Prefect does so without ever seeing its customers' code to ensure privacy and security while providing visibility into what is running and where issues may arise.

February 4, 2022

How to Use Prefect and Monte Carlo to Achieve More Reliable Data Pipelines

As recently announced, Prefect has a brand-new integration with Monte Carlo—a leading platform that adds observability features across your data environments, from ingestion in the data warehouse or lake to your downstream dashboards and reports. This hands-on post will dive into what Monte Carlo is and how to use it to add even more observability to your Prefect flows. You’ll learn the similarities and differences between the tools and why using both in tandem can be beneficial.

February 1, 2022

Ingesting Twitter Data with Prefect

One powerful use-case for Prefect is building automated data pipelines to pull social media analytics data. This guide will demonstrate how to write a data flow that extracts and loads Twitter data into Snowflake. Follow along now.

January 25, 2022

Productionizing Prefect Flows With Docker and Azure

Kirk Quinbar wrote this piece to document the steps followed in order to get a Docker image into an Azure container registry and then the setup of an Azure VM for running flows.

January 24, 2022

How to Make Your Data Pipelines More Dynamic Using Parameters in Prefect

How to pass runtime-specific parameter values to your data pipelines

January 13, 2022

What data teams can learn from finance ft. Jeremiah Lowin, CEO of Prefect

Prefect CEO Jeremiah Lowin chats with Boris Jabes, CEO of Census, on The Sequel Show podcast to discuss what data teams can learn from finance.

January 13, 2022

A modern ETL pattern using Prefect’s S3 Storage and Kubernetes runtime

This post is a tutorial on patterns for Python ETLs leveraging Prefect workflow orchestration. Prefect makes your Python jobs easy to repeat, manage, scale, and understand.

January 10, 2022

The Prefect Way to Automate Your Data Pipelines With Fivetran

Dataflow automation platform Prefect brings Fivetran connectors to its open-source orchestration project

January 10, 2022

Built In Honors Prefect in Its Esteemed 2022 Best Places To Work Awards

Built In today announced that Prefect was honored in its 2022 Best Places to Work Awards. We’ve earned a place on the list of 100 Best Remote-First Companies.

January 5, 2022

Built In Honors Prefect in Its Esteemed 2022 Best Places To Work Awards

Prefect Earns Spot on Built In’s 100 Best Remote-First Companies List

January 5, 2022

Orchestrating ELT with Prefect, dbt Cloud, and Snowflake (Part 3)

How to use Prefect and dbt Cloud with a Snowflake data warehouse

January 4, 2022

2021 Growth Report

Check out how far the Prefect community has come in our 2021 Growth Report!

December 29, 2021

Prefect is a 2021 Pipeline Academy Award Winner

The 2021 Pipeline Academy Awards (The Pipies) are brought to you by Pipeline Academy. In their words, "it is our way of praising the teams that built meaningful software solutions that serve a real purpose, the ones that are most likely here to stay to make the lives of data engineers better."

December 23, 2021

Orchestrating ELT on Kubernetes with Prefect, dbt, & Snowflake (Part 2)

This article is the second in a series of tutorials about orchestrating ELT data pipelines. The first post demonstrated how to organize and orchestrate a variety of flows written by different teams and how to trigger those in the correct order using Prefect. This post builds on that by capturing more advanced use cases and showcases how to deploy the entire project to a Kubernetes cluster on AWS.

December 21, 2021

Getting Started With Flows

This guide is here to help you step through 7 simple steps to get started running a simple flow using Prefect Cloud and your local machine. Ready to get started?

December 16, 2021

TECHBLITZ: Prefect tackles negative engineering

Prefect CEO Jeremiah Lowin sits down with TECHBLITZ, a Japanese media outlet covering startups.

December 15, 2021

Prefect’s Next Stage Ecosystem: Premier Partnership Program and Verified by Prefect

Prefect announces our new Premier Partner program, featuring newly Verified by Prefect partners Fivetran, Monte Carlo Data, dbt Labs, PagerDuty, Great Expectations, Firebolt, and Airbyte.

December 7, 2021

Prefect Launches Premier Technology Partner Program

Partner Program expands Prefect ecosystem with new integrations for dbt, Firebolt, Fivetran, Monte Carlo and PagerDuty spanning support, app verification and roadmap sharing

December 7, 2021

How to Achieve More Trustworthy Data Pipelines with the Prefect Integration for Monte Carlo

December 7, 2021

You No Longer Need Two Separate Systems for Batch Processing and Streaming

For a long time, the inflexibility of batch orchestrators and schedulers kept forcing data teams to maintain a separate set of systems to satisfy the modern event-driven, real-time data workloads. But it doesn't have to be that way. Read for a new approach.

November 30, 2021

Scaling Out With Prefect

How to set up a production workflow on Prefect.

November 28, 2021

The Nice Way To Manage Your Data Science Workflow

Learn to build an ETL pipeline with Prefect in a beginner example.

November 28, 2021

30% of DataOps Organizations Use Prefect

DatOps in Practice: A survey.

November 28, 2021

Introducing Radar

Introducing Prefect Radar: A new way of visualizing workflows that elegantly clarifies their execution at any level of scale.

November 22, 2021

Backstage + Prefect

Box presents how they integrated Backstage with Prefect during Backstage's Community Sessions.

November 17, 2021

Flow of Flows - Orchestrating ELT with Prefect and dbt

November 16, 2021

Orchestrate a Data Science Project with Prefect

Optimize Your Workflow with a Few Lines of Code

November 12, 2021

Jeremiah Lowin on the Digital Anarchist Podcast

November 8, 2021

Jeremiah Lowin returns to the Data Sharks Podcast #19

November 8, 2021

Prefect on the OSS Startup Podcast

November 8, 2021

How to Cut Your AWS ECS Costs with Fargate Spot and Prefect

Creating serverless data engineering pipelines using Python.

November 1, 2021

The Prefect Way to Automate and Orchestrate Data Pipelines

Community member Thuwarakesh Murallie, a Data Scientist at Stax, Inc., presents a great beginners guide for getting started with Prefect.

October 28, 2021

Prefect Pipeline Code for Building a Basic Data Lake

The Prefect code you need to get started building a data lake on AWS.

October 25, 2021

Building a Basic Data Lake with Prefect, S3, Glue and Athena

The story of how Smooth Sensors built a data lake on AWS with Prefect, S3, Glue and Athena.

October 25, 2021

A Mathematician Dove Into Technology Head First

Hear Chris White cover his journey into data via mathematics on the Coruzant Technologies podcast.

October 15, 2021

When to start a formal product organization?

Prefect CTO Chris White covers the steps to building a product organization at a growing company.

October 15, 2021

Livestream - Future of Prefect

The return of Laura Lorenz! Join Laura for another spooky October livestream as she takes Orion for a spin. Laura will highlight the new API, subflows and caching, as well as the killer new UI and deployment details.

October 8, 2021

Prefect Orion: Our Second-Generation Workflow Engine

Love your workflows again.

October 5, 2021

A People's History of Kubernetes

The 30th anniversary of Linux gives us a chance to reflect on the evolution of open source and how it has transformed the corporate landscape of technology-makers.

October 5, 2021

Bootstrapping In America: Jeremiah Lowin of Prefect

Prefect is a workflow management system that helps to eliminate data flow problems for companies’ data teams. They provide a new dataflow automation platform that reduces engineering time by automating pipelines and workflows of any complexity.

October 4, 2021

Prefect Acquires SLATE Data

We've acquired SLATE Data, a premier data engineering company, to expand our service and support offerings and deliver even more value to our users.

October 4, 2021

A Demo of Prefect

A simple demo of getting started with Prefect, including code snippets, from @Creative-Force

October 1, 2021

Building a Competitive Data Architecture, One Technology at a Time

It's time to rethink data architecture. The architectures that have been built over the years were suitable for on-premise, legacy environments but tended to be static, inflexible, and accessible to only a few. Today's data-driven organizations demand capabilities that adapt to the enterprise and open new paths of innovation to business users.

September 27, 2021

How to make sure your design team gives good remote feedback

Asking everyone to keep their cameras on is a good start, but leaders need to do more than focus on the technical aspects when running remote critiques.

September 27, 2021

Scaling Your Prefect Workflow to the Cloud

When running Prefect out-of-the-box, orchestration of your workflow is done in the cloud, while the actual computation of your code is done locally on your machine. This means that your Prefect workflows are...

September 22, 2021

Big Data Workflow Automation with Prefect and Coiled

When running Prefect out-of-the-box, orchestration of your workflow is done in the cloud, while the actual computation of your code is done locally on your machine. This means that your Prefect workflows are limited by the resources of your machine. You can transcend these local machine limitations by delegating compute-intensive workloads to a Dask cluster on Coiled.

September 22, 2021

Applying Practical Mathematics with Chris White of Prefect

Our guest this week has a Ph.D. in Mathematics. Chris White was the first hire and is now CTO of the rapidly growing company, Prefect.

September 20, 2021

The Simple Guide to Productionizing Data Workflows with Docker

In this walkthrough, we’ll provide an end-to-end minimal example of how to package Python work into Docker containers for production.

September 9, 2021

Prefect: Zero to Hero

From no experience with data tooling to fully automated Prefect pipelines

August 27, 2021

Inside Prefect: Reflections on my Internship Experience

A peek behind the curtain from a software engineering intern

August 27, 2021

Prefect: Should you utilise the next generation of data pipelining software?

A mature market like data orchestration software requires something special to move a company from its current tooling. However, we believe there are several core features that showcase the benefit of using Prefect.

August 17, 2021

Prefect: How to Write and Schedule Your First ETL Pipeline with Python

July 22, 2021

Prefect x Kubernetes x Ephemeral Dask: Power, without responsibility

Prefect is rapidly becoming the standard in dataflow automation. In our last blog we briefly ran through using Prefect Executors to parallelize on a single node.

July 19, 2021

Workflow Orchestration with Prefect and Coiled

July 16, 2021

The Data Exchange Podcast: Chris White on building an engine that enables arbitrary proprietary workflows.

July 15, 2021

Parallel Prefect: A bit about Dask, real fast

The problem: I heard Prefect can run Python in parallel, what is the simplest way to achieve this?

July 8, 2021

Prefect 0.15.0: A New Flow Run Experience

An overhauled flow run CLI, sub-flow result passing, agentless execution, improved run observability, new auth patterns, and more!

July 1, 2021

Data Sharks: Jeremiah Lowin of Prefect

The Data Sharks discuss the future of Data Orchestration and how Prefect provides the easiest way to automate your data with incredible flexibility.

June 18, 2021

Escape Velocity: Prefect's Series B

Following our $32M Series B, we're scaling our team, building new products, and forming A More Prefect Union.

June 10, 2021

Prefect Announces $32M Series B Funding Led by Tiger Global to Deliver on Growing Demand for Dataflow Automation

"Any engineer delivering custom code for their organization can easily reap the benefits of Prefect's platform, creating a massive addressable market for the company to capture." Official press release

June 10, 2021

Data flow automation engine Prefect raises $32M

"Prefect’s success aligns with broader industry trends in digital transformation...data flows have raised the global GDP by at least 10% over a decade. The value totaled $7.8 trillion in 2014 alone, contributing to economic growth primarily by raising productivity." Coverage by VentureBeat

June 10, 2021

Prefect raises $32M for dataflow automation technology

"We see the data stack as a whole just exploding in terms of attention and the need for Prefect is growing exponentially." Coverage by TechTarget

June 10, 2021

Prefect Secures $32 Million for Its Dataflow Automation Platform

“Our mission is to help data engineers and data scientists eliminate negative engineering by delivering a world-class orchestration tool for today’s modern data stack" Coverage by StartUp Savant

June 10, 2021

D.C. startup raises tens of millions of dollars in second venture round of 2021

"The new capital will let Prefect hire up, grow its commercial business and further develop its platform" Coverage by Washington Business Journal

June 10, 2021

DC’s Prefect already raised two funding rounds in 2021. Here’s what’s next for the dataflow automation company

"We are chasing our own growth." Coverage by Technical.ly

June 10, 2021

Prefect Raises $32M To Be The ‘Toothpick’ In Companies’ Data Stack

“Suitors will always come, but Prefect has all of the elements to be an enduring business that forges its own path,” said Ethan Kurzweil of BVP. Coverage by Crunchbase News

June 10, 2021

Introducing: The Prefect KV Store

A flexible building block for workflow metadata

June 3, 2021

MLOps: Task and Workflow Orchestration Tools on Kubernetes

If you need a simple and clear answer―use Prefect.

May 28, 2021

Orchestrating Image Preprocessing Workflows for Label Studio with Prefect

Kevin Kho, Open Source Community Engineer at Prefect, showcases Prefect workflows with Label Studio

May 26, 2021

Prefect + Microsoft: Advancing Dataflow Automation Together

Unveiling the first details of our deepening relationship with Microsoft.

May 21, 2021

Enhance positive engineering with Prefect and Microsoft for Startups

We are delighted to welcome Prefect into the Microsoft for Startups portfolio. Working with Prefect will help our joint customers easily deploy on trusted infrastructure with the convenience of Prefect Cloud.

May 21, 2021

If you give a data engineer a job...

May 11, 2021

Don’t Panic! The Prefect Guide to Building a High-Performance Team

Building the startup is just as important as building the product. Here's how we establish a high-performance and collaborative culture.

April 28, 2021

Introducing a Dataflow Management System Backed Up by Prefect, AWS, and Github Actions

Take your local Prefect workflows to the next level by provisioning AWS & Prefect Cloud infrastructure, with Github Actions playing an important role.

April 27, 2021

Best Workflow and Pipeline Orchestration Tools – Machine Learning Guide

"I believe that [Prefect] is one of the best automated workflow management tools out there."

April 20, 2021

Data Engineering & Business Intelligence – An Exciting Quest

Prefect. dbt. Redshift. BI tools. Learn about Cloud Academy Staff Engineer Giuliano Sofi's (literally) epic journey to redefine and transform their data and reporting!

April 16, 2021

Building a Healthy Open Source Community with Jeremiah Lowin, Founder and CEO of Prefect

Join Jeremiah Lowin and Anu Sharma for a deep dive into open-source community building from 0 to 5,000.

April 9, 2021

Simple, Success-Based Pricing

Eliminate negative engineering with Prefect Cloud’s new success-based pricing, a generous free tier, and automatic discounts.

March 24, 2021

Liftoff: Prefect's Series A

Welcoming a new cohort of strategic investors.

February 25, 2021

GitHub CEO Joins Prefect’s $11.5 Million Funding Round for Dataflow Automation

Watch Jeremiah Lowin's interview about our Series A, future plans, and upcoming announcements!

February 17, 2021

Prefect Raises $11.5M in Series A Funding to Eliminate Negative Engineering in Dataflow Automation

February 9, 2021

Liftoff

A preview of our upcoming announcements, including our Series A, Microsoft partnership, updated product vision, and new community programs.

February 9, 2021

Serverless Data Pipelines Made Easy with Prefect and AWS ECS Fargate

Even though there are so many workflow orchestration solutions and cloud services for building data workloads, it’s hard to find one which is actually pleasant to use and allows you to get started quickly. One of my favorite tools for building data pipelines in Python is Prefect — a workflow management platform with a hybrid agent-based execution model.

February 8, 2021

Prefect Workflow Automation with Azure DevOps and AKS | Infinite Lambda

February 5, 2021

Prefect Workflow Automation with Azure DevOps and AKS

The team at Infinite Lambda shares how to automate Prefect flow deployments on Azure.

February 5, 2021

Understanding Great Expectations and How to Use It

Use Prefect and Great Expectations to load valid data into Snowflake.

February 4, 2021

Getting Started with Parallel Computation

Prefect provides many simple and expressive building blocks that users can snap together to add advanced features to their workflows. This post shows how easily Prefect tasks can be executed in parallel without requiring deep knowledge of distributed computing. In fact, it only requires two extra lines of code!

January 20, 2021

What We Love About Prefect

It’s open source, plays nicely with Kubernetes, and there’s a great community! We love how easy it was to set this up, and the clean UI and dashboards make it a pleasure to use.

January 15, 2021

Livestream - 0.14.0: your deploy just got easier

December 18, 2020

Prefect 0.14.0: Improved Flow Run Configuration

We’re excited to announce the release of Prefect 0.14.0

December 17, 2020

Introducing: The Artifacts API

Create native Prefect UI integrations with one line of code

December 8, 2020

Getting started with operationalizing your Python code

In the first part of this blog series we saw how quickly Prefect can help you run parallelized Python code from a fresh OS install

December 2, 2020

Scheduling & Optimizing RAPIDS Workflows with Dask and Prefect

Scheduling and optimizing accelerated data science workflows is really easy with Prefect, RAPIDS, and Dask. Get started with this tutorial.

November 9, 2020

Integrating Prefect & Databricks to Manage your Spark Jobs

This tutorial explains how to run Databricks notebooks and Spark jobs in Prefect flows.

November 3, 2020

Accelerating modelling with Frea: Paidy's in-house feature store with Prefect and PostgreSQL.

We’ve recently partnered with Prefect to make it easy for us to build scalable solutions for data science at Paidy that make sense for our team and ways of working. This blog post talks about the in-house Feature Store known as Frea which is built on top of the Prefect platform, and which reduced the model training pipeline time from weeks to hours.

October 26, 2020

How to Automate Python Workflows in Prefect (Step-by-Step Guide)

There was quite a steep learning curve for me to pick up Airflow. Thankfully, in the midst of my struggles, I discovered a library that would make the entire process ridiculously simple. Enter Prefect.

October 21, 2020

The Prefect Task Library

The Prefect community is a welcoming place for developers of all experience levels, and our task library is an excellent way to get involved with contributing to an open-source project.

October 8, 2020

How Prefect Kept my Family Cool in the Pool this Summer

The data workflow platform Prefect is used commercially for serious data engineering and machine learning pipelines. It's being rapidly adopted in industries like financial services, e-commerce, healthcare, and many others.

September 22, 2020

Slate Data Transforms its Business Model with Prefect Cloud

Slate Data provides data engineering services to reinvigorate data-driven business decisions by modernizing data tech stacks and empowering industry-specific data scientists.

September 17, 2020

Managing dependencies between data pipelines in Apache Airflow & Prefect

A simple approach to managing dependencies between your workflows.

September 4, 2020

Distributed data pipelines made easy with AWS EKS and Prefect

Anna shows you how you can obtain a highly available, scalable, distributed system that will make the orchestration of your data pipelines for ETL & ML much more enjoyable

August 25, 2020

The All-New Prefect Server and UI

Two key Prefect products are becoming open-source projects in our biggest release ever.

August 6, 2020

Workflow Automation: Empowering teams through our in-house, self-service framework

See how the Journal team used Prefect to build a self-serve automation framework for their data teams

July 20, 2020

Introducing: Prefect Universal Deploy

We’re excited to announce our best release yet: Prefect 0.8.0

December 11, 2019

Snowflake: the details of our first Data Warehousing project in the Cloud

Learn how Trade Me adopted Prefect to build a workflow for Snowflake.

July 28, 2019

Toward Dataflow Automation

Prefect Cloud is coming

June 10, 2019

Why Not Airflow?

An overview of the Prefect engine for Airflow users

May 1, 2019

Pipeline Pitfalls

Data engineers are often responsible for the planning, building and maintaining of data pipelines. During that process, they may face the challenging decision to either create a custom system or use an existing framework.

May 1, 2019

Prefect is Open-Source! 🎉

Today, on its two-year anniversary, we’re excited to announce that Prefect has launched as an open-source project with an Apache 2.0 license.

March 24, 2019

Prefect Runs on Prefect

Every morning, Marvin runs standup for our team. That’s not unusual for a tech startup, except that Marvin is actually a Prefect workflow.

January 17, 2019

The Golden Spike

A Prefect Update

January 9, 2019

Positive and Negative Engineering

Don't panic.

October 1, 2018