The Unbundling vs Rebundling of a Data Stack Debate Missed The Point

Do you have to choose?

Photo by Wendy Wei from Pexels

Many tools in the contemporary data landscape started to become increasingly polarized. On the one hand, there are products highly specialized in one specific area, such as data ingestion, transformation, scheduling, cataloging, experiment tracking, alerting, etc. On the opposite side of the spectrum, there are tools that attempt to re-bundle all the pieces of the data stack as part of their single product.

Either-or decisions

Tools that fall into those extreme categories tend to see the world in black and white and force you to make either-or decisions. Either you choose a single product to manage the entire end-to-end lifecycle of your dataflow, or you lose data lineage and observability. Either you switch to a single product and vendor, or you end up with data stack fragmentation and chaos.

Coordination plane instead of a control plane

Full disclosure, I work at Prefect. Just yesterday, we launched Prefect 2.0 — a product that evolved, a.o., as a result of acknowledging that problem.

Prefect 2.0 provides an alternative to either-or decisions: a coordination plane that can simultaneously be used to orchestrate your…