Modular Data Stack — Build a Data Platform with Prefect, dbt and Snowflake (Part 5)

Failure handling with alert notifications and retries for data ingestion and dbt transformations in your Modern Data Stack

Marvin observing, orchestrating, and coordinating your dataflow universe and helping you build a reliable data platform through observability, alerting, and retries

This is a continuation of a series of articles about building a data platform with Prefect, dbt, and Snowflake. If you’re new to this series, have a look at the summary linking to previous posts. This demo will dive into handling failure, including alert notifications and retries, and orchestrating Snowflake data transformations implemented with dbt.

To make the demo easy to follow, you’ll see this 🤖 emoji highlighting sections that prompt you to run or do something (rather than only explaining something). The code for the entire tutorial series is available in the prefect-dataplatform GitHub repository.

Table of contents
· Failure handling: alert notifications & retries
  ∘ Alerting
  ∘ 🤖 Configure Slack alerts on failure
  ∘ 🤖 Failing successfully with automated retries
  ∘ 🤖 Trigger a flow run with a task that will fail and retry
  ∘ 📚 Summary of alerts & automated retries
· 🤖 Manual retries from the UI 
  ∘ 🤖 Important notes ❗️
  ∘ 🤖 Trigger a run from deployment to test alerts
  ∘ 🤖 Retry the flow run manually
  ∘…