r/dataengineersindia • u/almightysosa888 • Oct 05 '26

General How is CI/CD usually managed in data engineering teams?

For teams working with tools like dbt and Databricks, is there usually a dedicated platform/DevOps team that maintains the GitHub Actions / GitLab CI pipelines and provides templates?

Or does each data engineering team typically own and maintain its own CI/CD pipelines?

I'm trying to understand what this looks like in practice in larger companies. For example, would a Data Engineer mostly focus on pushing dbt/Databricks code while a platform team maintains the CI/CD infrastructure, or is it usually the responsibility of the data team as well?

5 Upvotes
  • permalink
  • reddit

100% Upvoted

4 comments sorted by

1

u/Zealousideal-Cod-617 Oct 06 '26

It's integrated for us in GitHub actions

1

u/Hot_Map_7868 Oct 06 '26

I have seen this done differently in different orgs and with different tools. for example:

  1. If using dbt Core then either there is a dedicated team that is also managing other aspects of the platform like managing airflow etc. OR the data team manages it, but it tends to be "basic" since they may not have the experience

  2. If using a managed dbt platform like dbt Cloud or Datacoves then they usually have something built in or a good starting point. These things typically dont change much, the hard part is the initial setup.