Schedule & orchestrate dbt Cloud jobs with Prefect

Modular Data Stack with dbt Cloud Prefect block

Marvin orchestrating dbt Cloud jobs

This short post will walk you through how to set up dbt Cloud jobs and orchestrate those with Prefect. It assumes that you have already signed up for dbt Cloud and know how to use dbt.

dbt Cloud setup

First, you need to retrieve the dbt Cloud account ID and create an API key. To do that, go to Account Settings:

In the URL, you should now see the account ID. Copy it and paste it into DBT_CLOUD_ACCOUNT_ID in your .env file. Now from the same Account Settings page, go to the API Access section:

Create and copy the API key:

Programmatic block creation