How to Build a Modular Data Stack — Data Platform with Prefect, dbt and Snowflake
Build a data platform with Snowflake & dbt, and use Prefect to observe and coordinate your data stack
Data platform engineers are increasingly crossing the boundaries of teams, business domains, systems, repositories, and data pipelines. Running things on a schedule alone often doesn’t cut it anymore. Some dataflows are real-time, event-driven, or triggered ad-hoc. A reliable development lifecycle with CI/CD and flexible building blocks are now required to meet the demands of the rapidly changing world with a growing number of tools, data stores, and frequent migrations.
This series of articles covers possible ways of addressing some of these challenges. We discuss how you can build a scalable and painless data platform with Prefect, dbt, and Snowflake. It’s presented in a top-down approach, starting from a big-picture perspective and revealing increasingly more details. The code for the entire blog post series is included in the prefect-dataplatform repository.
If you found this repository helpful, feel free to give it a star! ⭐️
