Hi Jarek, thanks a lot for your update about 2.0 features, this sounds interesting. However, to me this article was not about just visually grouping tasks together but really about managing the execution order and dependencies between workflows. As long as my staging area ETLs are not finished, my data pipelines for business logic layer shouldn't start, because they need to have staging area tables up-to-date. This has influence on how we schedule those jobs. Ideally, the business logic layer can start directly after the staging area is finished. How is TaskGroup concept helpful in that? I would still have to schedule those task groups in such a way that my business layer TaskGroup starts after staging area TaskGroup is finished and only if it was successful. Did this new concept include how to manage state dependencies between TaskGroups?

Also when it comes to maintenance: if I introduced some changes to the staging area (child data pipeline), I want to be able to redeploy and rerun only this ETL without having any impact on other layers such as business logic and data marts. Again, this shows that it's not just about visualization in the UI but about separation of concerns and logically grouping co-dependent pipelines.

If this would be satisfied with the new concept of TaskGroup, I may later create a follow up article to explore the new functionality. But for now, I think that parent-child workflows work quite well in both Airflow and Prefect without introducing too much complexity other than adding Master data pipelines that coordinate the interaction between co-dependent child workflows and let us schedule co-dependent workflows much easier.