Prefect October Updates: Hacktoberfest, Run Labels, Agent Tracking
Hello and welcome to the second edition of the Prefect Public Changelog! We've got exciting updates across the board.
Per-run Flow Run Labels 🏷️
Labels are a way of tracking and bucketing workflows, allowing you to determine where your flow runs are executed. Labels have existed on flows for some time but are now configurable on a per-run basis. Earmarking runs of the same flow for different environments enables things like faster testing iteration (only send runs with the testing label to a testing Agent, for example), environment and machine-specific execution, concurrency limiting certain run types, and more.
Moreover, now that labels can be attached to individual runs, the Prefect Scheduler can schedule your flows to run with certain labels at certain times! This is analogous to the long-standing ability to schedule changing Parameter values over time. Note that it's also now possible to schedule multiple runs of a flow to begin at the exact same time, as long as each run has either different parameter values or different labels.
Dynamic Task Run Names 🔖
Starting with Prefect Core 0.13.10, task runs can have names, which are stored and exposed in Server/Cloud. This improves task run tracking in the UI, particularly for large mapped pipelines. Since task runs can be dynamically configured at execution time, task runs can be named based on inputs... or you can just use emojis. 😎 🔥
Run it back - Restarting from Cancelled 🔄
This was a feature we saw crop up a lot in the community - flow runs in Cancelled states can now be restarted from the UI similarly to those in Failed states, giving finer runtime control and not cementing state unnecessarily.
Open Source Agent Tracking 🤖
Agents have received a permanent home as a data entity in Prefect! Until now, Agents lived as ephemeral processes only loosely recognized by the system outside of in-memory caches. Now that Agents have become full entities in the backend, they no longer require a cache layer to track, meaning Prefect Server users can now get quicker insight into one of the most critical pieces of infrastructure in their Prefect deployment. Starting from Prefect Core 0.13.0, the Server UI dashboard features an Agents tile and tab.
Marvin takes over the Prefect Docs 🔎
Life? Don't talk to me about life.
Our favorite pessimistic android is now part of the Prefect Docs! Marvin has begrudgingly agreed to assist with documentation search, improving results ranking, page indexing, and general discoverability. Search results now provide better context based on your query and are paginated to make sure you can always find what you're looking for.
Task Library Additions 🎃
We've had some great new additions to the Task Library:
Monday
Prefect now integrates with Monday: add items to your Monday boards without ever leaving your Flow!
You can see the Monday CreateItem task here
CancelFlowRun
Another popular request, the CancelFlowRunTask adds an easy interface to cancel flow run execution directly from your flow, whether that's cancelling the current flow if a condition isn't met, halting execution of another dependent flow, or for any other reason!
You can see the CancelFlowRunTask here.
The Task Library is a place for first-class integrations between Prefect and your favorite tools (including Prefect itself!). The library features a growing list of 30+ integrations with hundreds of ready-built tasks for you to drop into your flow with little to no configuration. It's also a great place to start contributing to Open Source (ahem, #hacktoberfest). If you're interested in contributing to the Task Library, we wrote a great blog post to help you get started!
Look out for another update near the end of the month! In the meantime there are lots of other places to keep up-to-date on what's going on in the Prefect ecosystem; follow @PrefectIO on Twitter, join us on Slack, or check out the Prefect Blog. You can also see realtime updates on the Prefect Core, Server, and UI repos to stay ahead of new releases!