Data pipelines have become critical as data and machine learning increasingly become essential in most modern software applications and systems. In this episode we do a deep dive into what makes data pipelines challenging, as well as the essential components of modern workflow management and orchestration solutions. Chris explains some of the design choices they made with Prefect, the current status of the open source project, as well as the near-term roadmap. Anyone building data and machine learning products and services will want to listen to this episode: it is a great overview to important areas in modern data engineering and data infrastructure.

Jul. 15, 2021
The Data Exchange Podcast: Chris White on building an engine that enables arbitrary proprietary workflows
Chris White, CTO of Prefect, speaks on building an engine that enables arbitrary proprietary workflows.