Declarative Dataflow Deployments with Prefect Make CI/CD a Breeze

Command Line Interfaces and YAML are loved by DevOps engineers — use the same for your Dataflow

In Prefect, Python is the API — we believe that your code is the best representation of your workflow. But when it comes to deploying your flows, an imperative Python API can become cumbersome. This post will discuss how Prefect CLI and Blocks allow you to follow engineering best practices for reliable dataflow operations.

What is Prefect

Prefect is more than an orchestrator — it allows you to design, reliably execute and observe all dataflows regardless of the underlying execution environment. As long as your main workflow function is decorated with @flow, any run of such flow becomes observable from the Prefect UI.

Often you don’t want to be responsible for triggering those runs yourself. Instead, you may want to schedule your workflows, start them from an API call, or run those on remote compute clusters or cloud VMs by leveraging agents and work queues. In those scenarios, you need deployments.

What is Prefect Deployment

At its core, a flow deployment is a server-side workflow definition, allowing you to turn any flow…