It’s SNOWing in DC: Prefect and Snowflake Partner for a Better Modern Data Stack
Apr. 28, 2022

Prefect and Snowflake Partner for a Better Modern Data Stack

Andrew Black
Andrew BlackHead of Partnerships

Snowflake Technology Partner Select Logo

Snowflake needs no introduction; the revolutionary data warehouse has become the data cloud of choice for tens of thousands of data-driven organizations worldwide. It’s also among the most popular – and loved – data warehouses in our developer community. As announced in a press release today, we’re super excited to share that we have joined as official partners to make our products and their joint experience even better together.

For those who are not familiar with data warehouses or data workflow orchestration, it’s worth explaining why this pairing is so important and impactful. No matter what their data stack consists of, organizations have to have a way to move data around and perform actions on it (data workflow orchestration), and a place to store and retrieve it (data warehouse). As data stacks have become more complex with more platforms and tools, more use cases, and of course a lot more data, both workflow orchestration and data warehouses have become more vital.

With best-of-breed orchestration paired with super fast and reliable data access, data organizations can build and iterate faster while delivering insights more reliably and with greater confidence. While that sounds like a lot of marketing gobbledigook, it’s exactly what our users are telling us; in fact, 95% of our users report significant productivity gains by adopting Prefect and up to a 70% reduction in pipeline failures.

How do we do all this together? As the new standard in dataflow automation, Prefect enables data engineers to automate every data task and dataflow in their processes – most of which connect to or involve the data warehouse at some point. Built-in error handling means any issues are detected and automatically handled, and our Cloud UI makes it much easier to diagnose issues. In addition, both platforms are built for speed, Snowflake with auto-scaling across multiple Clouds, and Prefect with parallel processing and option for Dask execution.

But perhaps even more valuable, Prefect offers visibility into your data workflows, even those within Snowflake. This means that once you trigger a dataflow with Prefect, you know exactly the state of it, get warnings if there are errors, and easily restart in code or in the UI. It’s this comprehensive visibility that customers such as Alex Welch, VP of Enterprise Data and Analytics at TastyTrade, really set Prefect apart: “Prefect is the glue of our data stack. It gives us visibility into our critical pipelines that we never had before.”

Prefect and Snowflake are also aligned in our core value propositions for data teams: super fast, scalable, easy to use, and zero-maintenance.

In summary, using Prefect and Snowflake together, data teams can get more out of each product, and more out of their data. 

Snowflake Technology Partner Select Logo

Partnership announcements can sometimes seem bland, but deepening our relationship with Snowflake is really a big deal. We've been recognized by Snowflake as a "Select" partner, meaning we have passed technical, customer and use case validations. This means our users can be assured that Prefect has been validated to work with Snowflake, and continually updated to reach the latest versions. We’ll also have more access to each others’ technical resources to support customers. And for those customers new to adopting our products together, the experience will be streamlined – everything from technical validation through deployment.

We are already working on new and improved integrations for Prefect 2.0 to make our connections easier to set up and maintain by wrapping all available functions into a Collection.

We’re also looking forward to learning more from our joint users as we expand into more companies, industries, and use cases.

We'll also be extending Prefect capabilities to more Snowflake platforms... stay tuned.

If you’re a Prefect user and haven’t tried Snowflake yet, give it a shot. You can sign up for a free trial here, and get up and running quickly with our library of pre-built tasks. You can also, of course, roll your own if you prefer. I’d also recommend reading Anna Geller’s stellar blog series on using Prefect with Snowflake and other modern data stack tools, such as dbt.

If your organization is standardized on Snowflake but aren’t using Prefect, give it a try. Because it’s just Python, Prefect is super easy for any data engineer to pick up, and you can develop and test straight from your laptop. Our docs will walk you through it, or you can join our Slack with thousands of Prefect enthusiasts ready to assist and share ideas.

Happy Engineering!

- Andrew, Head of Partnerships

Posted on Apr 28, 2022
Blog Post

Love your workflows again

Orchestrate your stack to gain confidence in your data