The Golden Spike

Last month, Prefect had a “golden spike” moment. For the first time, we connected our open-source engine to our managed platform and let the entire system run on its own. Counting down “3… 2… 1…” and seeing a terminal blip “Hello, Prefect!” may not sound very exciting to you, but for us it was a major victory (and if it does sound exciting: careers@prefect.io). In a small way, it was the realization of something we’ve been working toward for months, and I’m proud to say it was just one of a number of milestones we achieved this quarter.

With the encouragement of our investors, we posted a version of a previous monthly update as the introductory post of our company blog. Coupled with a few other appearances, including Patrick O’Shaughnessy’s podcast and a presentation at the PyData DC conference, it helped jumpstart our company’s unveiling.

While we haven’t launched beyond our early partners, the response so far has been wonderful. Our Lighthouse Partner group has swelled to over 65 members, and we’ve started receiving inbound inquiries from large teams seeking data engineering relief. We are even in the early stages of exploring partnerships with a few enterprises. Personally, I was most excited when someone at PyData referred to Prefect as the “hot new workflow tool” — a sequence of words I’m confident has never been spoken before.

While it’s exciting that things are going very well, it’s important to benchmark our performance against the expectations I gave when we began this company in the spring.

Our overarching goal for 2018 was to quickly transition from “prototype to product:” getting the open-source framework (“Core”) and managed platform (“Cloud”) running autonomously and in tandem. As our “golden spike” moment demonstrates, we’re in good shape. We have already started using the system to execute some of our internal processes, fulfilling our internal mantra: “Prefect runs on Prefect.”

My original plan for 2018 was that we would develop the product by working closely with 3 interested parties. As I already mentioned, we ended up getting feedback from more than twenty times that number. We did not solicit many of those partners directly; they found us. As a result, they are motivated and engaged. Across all of them, we have only heard one use case that couldn’t be directly solved by Prefect in its current form (a specific type of machine learning called reinforcement learning). This gives us greater confidence that we’ve succeeded with one of our core (but controversial) bets: it really is possible to build a generalized data engineering framework.

We will spend the first quarter of 2019 intensively iterating with our preview partners. This second-stage preview will be more focused on user experience, aimed at reducing emergent frictions in our product. A critical part of that is our wonderful user interface, which is in heavy development right now. Our goal is to give Prefect a proper launch in the second quarter.

We finished 2018 ahead of schedule and (23%!) below budget. We achieved that by established a culture in which our employees are empowered to succeed. They are highly motivated and talented people with access to the resources they require. As a result, they achieve their goals quickly and efficiently.

I am deeply indebted to the small but mighty team that forms Prefect today. Their work has been extraordinary and their excitement drives the project forward. At the moment, 5 people call Prefect home; that number is about to start growing quickly.

From our whole team: we hope you have a wonderful new year and thank you for being the Prefect partners.

We are entering 2019 with great momentum and I look forward to reporting our successes.

Happy Engineering!