Introducing Prefect Cloud 2.0
Mar. 18, 2022

The next generation of orchestration is here.

Bill Palombi
Bill PalombiHead of Product

Today, we’re opening access to the first public beta of Prefect Cloud 2.0, the easiest way to get started with Prefect 2.0.

Prefect 2.0, also in beta, can be used to address all kinds of negative engineering challenges. Once you've gotten started with Prefect 2.0 and deployed a few mission critical flows, Prefect Cloud 2.0 can take your orchestration to the next level by:

  • Giving you powerful tools to securely share flow run metadata, like states, with colleagues.

  • Entirely removing the burden of infrastructure management and operations of Prefect 2.0

  • Enabling you to deeply, securely integrate Prefect 2.0 with other tools in your stack.

We’ve tried limited, waitlist-based beta access periods before, but we were overwhelmed by the volume of access requests and we hated turning people away. Instead, we’ve opened our beta to anyone and everyone who is willing to join us in our mission to eliminate negative engineering. There is a small price to pay: we want, no, we need your feedback to make Prefect Cloud 2.0 the best orchestration platform possible. During our beta period, we’re experimenting with new concepts and making changes to established concepts. These experiments are informed by years of experience working intimately with Prefect customers and community members, but it’s a continuous evolution that needs sustained feedback to progress. Join us in the never ending dialogue in Slack or Discourse.

Cloud 2.0 beta accounts are free, and will stay that way, even beyond the beta period. You can use your account as much as you’d like. They’re especially well suited to side projects and one person teams. To protect our nascent application as it takes its first steps, we’ve put a few common sense caps in place. Most users won’t have to worry about them - but if you bump into one, reach out to us - we may be able to remove it.

Blog Image - Cloud 2.0 Beta - 1

Use workspaces to securely share and monitor the state and history of dataflows

Cloud 1.0's rapid uptake among organizations of all shapes and sizes meant that, in some cases, the complexity of our user's organizations outstripped Cloud's capacity to model it. In particular, many organizations have several teams using Prefect, each with their own processes, permissions, and tools. Cloud 2.0 makes collaboration and access control a breeze with workspaces - collections of people, flows, work queues, and other resources - all within the same permission boundary.

Each workspace is backed by its own independent instance of Orion, the very same orchestration engine that powers Prefect 2.0, so working with one feels just like working with a local Orion instance. In fact, you can use our recently-introduced profiles to easily switch between running the same flow with a self hosted Orion instance and a Cloud 2.0 Orion instance, and expect the exact same behavior. You can take advantage of this for all kinds of needs, such as prototyping, local development, or test environments.

Our users when asked to choose between convenience and security:

Blog Image - Cloud 2.0 Beta - 2

Our users when asked to choose between convenience and security

Prefectionists, and their InfoSec colleagues, love Prefect's hybrid model. The hybrid model keeps your code and data private, while still taking full advantage of Prefect Cloud 2.0’s managed orchestration service. It offers all the convenience of a cloud service, and all the security guarantees of an on premise deployment. The hybrid model makes sense for most organizations, but one size does not fit all. Through the rest of the year, we’ll expand the options available to our customers. We’ll offer managed execution, giving you the simplicity of just running your flow code, without having to worry about how and where it runs. We’ll offer dedicated, single-tenant infrastructure, giving you the peace of mind of knowing that your orchestration environment and metadata are fully isolated from everyone else’s. And we may even offer some alternatives in between...

Blog Image - Cloud 2.0 Beta - 3

Cloud will reflect states and history from across your entire organization

The future of data engineering involves more tools, more environments, and more flexibility - most likely with a mix of self hosted and managed orchestration. Prefect Cloud 2.0’s fully open source Orion engine enables it to become a single, secure source of truth for the state and history of all dataflows across your organization, regardless of where the orchestration happens.

Cloud 2.0 supercharges Prefect 2.0’s collections, enabling you to utilize configurable, secure server-side, UI accessible access points to all of the components in your stack. We’ve started with storage, but we’ll expand this pattern to include every tool and service in the growing modern data stack and beyond. We’ll also enable you to define your own access points and expose them to the rest of your team via the Cloud UI and API.

Sure, an orchestration platform can make you more efficient, but can it make you happier? Apparently, it can. The most important feature of Prefect has always been how much people love to use it. Whether you look at our Slack’s #introductions channel or Cloud’s NPS score, we’re deeply privileged to work every day on a tool that has delivered positive value to so many people. Receiving and responding to your feedback is the reason that we pride ourselves on shipping early and shipping often — thank you for being part of our community.

Give Prefect 2.0 and Cloud 2.0 a try. You might just love your workflows again.

Happy engineering!

Posted on Mar 18, 2022
Blog Post

Love your workflows again

Orchestrate your stack to gain confidence in your data