Modular Data Stack — Build a Data Platform with Prefect, dbt and Snowflake (Part 6)

Coordinate ML, analytics & BI reporting data platform workflows with Prefect, dbt, Snowflake, and modular building blocks

Prefect as a blue bot coordinating ML, analytics & BI reporting workflows in the dataflow universe

Welcome to another article about building a data platform with Prefect, dbt, and Snowflake. If you’re new to this series, check out the summary linking to previous posts. This post will cover processes after data ingestion and transformation, including ML, analytics, and reporting flows. We’ll also demonstrate how you can manage dependencies across multiple teams using the GitHub block along with other modular building blocks available in Prefect.

To make the demo easy to follow, you’ll see this 🤖 emoji highlighting sections that prompt you to run or do something (rather than only explaining something). The code for the entire tutorial series is available in the prefect-dataplatform GitHub repository.

Table of contents
· Run code from other repositories
  ∘ Motivation
  ∘ dbt repository for analytics engineers
  ∘ 🤖 Creating a private dbt repository 
  ∘ 🤖 Creating a personal access token in GitHub
  ∘ 🤖 Creating a GitHub block
  ∘ Using the GitHub block with dbt
  ∘ 🤖 Triggering a dbt build using the GitHub block
  ∘ 📚…