Skip to main content

prefect.workers.base

Classes

BaseJobConfiguration

Methods:

from_template_and_values

Creates a valid worker configuration object from the provided base configuration and overrides. Important: this method expects that the base_job_template was already validated server-side.

is_using_a_runner

json_template

Returns a dict with job configuration as keys and the corresponding templates as values Defaults to using the job configuration parameter name as the template variable name. e.g.

prepare_for_flow_run

Prepare the job configuration for a flow run. This method is called by the worker before starting a flow run. It should be used to set any configuration values that are dependent on the flow run. Args:
  • flow_run: The flow run to be executed.
  • deployment: The deployment that the flow run is associated with.
  • flow: The flow that the flow run is associated with.
  • work_pool: The work pool that the flow run is running in.
  • worker_name: The name of the worker that is submitting the flow run.
  • worker_id: The backend ID of the worker that is submitting the flow run.

resolve_for_flow_run

Build a fully-prepared job configuration for an existing flow run. Reads the flow run’s deployment (when present) and flow, merges deployment- and flow-run-level job variables over the work pool’s base template, instantiates the configuration, and stamps attribution metadata via prepare_for_flow_run.

BaseVariables

Methods:

model_json_schema

TODO: stop overriding this method - use GenerateSchema in ConfigDict instead?

BaseWorkerResult

BaseWorker

Methods:

cleanup_handler_registry

client

get_all_available_worker_types

Returns all worker types available in the local registry.

get_and_submit_flow_runs

get_default_base_job_template

get_description

get_documentation_url

get_flow_run_logger

get_logo_url

get_name_slug

get_status

Retrieves the status of the current worker including its name, current worker pool, the work pool queues it is polling, and its local settings.

get_worker_class_from_type

Returns the worker class for a given worker type. If the worker type is not recognized, returns None.

handled_cleanup_kinds

is_worker_still_polling

This method is invoked by a webserver healthcheck handler and returns a boolean indicating if the worker has recorded a scheduled flow run poll within a variable amount of time. The query_interval_seconds is the same value that is used by the loop services - we will evaluate if the _last_polled_time was within that interval x 30 (so 10s -> 5m) The instance property self._last_polled_time is currently set/updated in get_and_submit_flow_runs()

kill_infrastructure

Kill infrastructure for a flow run. Override this method in subclasses to implement infrastructure-specific termination logic. Args:
  • infrastructure_pid: The infrastructure identifier from the flow run.
  • configuration: The job configuration for connecting to infrastructure.
  • grace_seconds: Time to allow for graceful shutdown before force killing.
Raises:
  • NotImplementedError: If the worker doesn’t support killing infrastructure.
  • InfrastructureNotFound: If the infrastructure doesn’t exist.
  • InfrastructureNotAvailable: If the infrastructure can’t be killed by this worker.

limiter

max_cleanup_concurrency

run

Runs a given flow run on the current worker.

setup

Prepares the worker to run.

start

Starts the worker and runs the main worker loops. By default, the worker will run loops to poll for scheduled/cancelled flow runs and sync with the Prefect API server. If run_once is set, the worker will only run each loop once and then return. If with_healthcheck is set, the worker will start a healthcheck server which can be used to determine if the worker is still polling for flow runs and restart the worker if necessary. Args:
  • run_once: If set, the worker will only run each loop once then return.
  • with_healthcheck: If set, the worker will start a healthcheck server.
  • printer: A print-like function where logs will be reported.

submit

EXPERIMENTAL: The interface for this method is subject to change. Submits a flow to run via the worker. Args:
  • flow: The flow to submit
  • parameters: The parameters to pass to the flow
  • job_variables: Job variables for infrastructure configuration
  • flow_run: Optional existing flow run to retry (reuses ID instead of creating new)
Returns:
  • A flow run future

sync_with_backend

Updates the worker’s local information about it’s current work pool and queues. Sends a worker heartbeat to the API.

teardown

Cleans up resources after the worker is stopped.

work_pool