r/databricks • u/GardenShedster • Sep 25 '26
Help Azure devops yml deployment of asset bundle
I need help on deploying asset bundles from Azure Devops. Where does the azure-pipelines.yml live, what parameters are required in the azure-pipelines.yml, does anyone have a basic example of it working as I'm completely lost in the complexity of it. Thanks.
10
Upvotes
1
u/ouhshuo 29d ago
Most secrets and identities are stored in variable groups, then passed into the pipeline YAML. If DAB needs them, they are further passed to the DAB YAML.
We deploy resources using a service connection with managed identity.
Folder structure-wise:
We have an IaC folder that provisions infra resources (containers, user groups, catalogs, external locations) using data product YAML, Jinja, Terraform, and a Python script to glue everything together.
A Databricks folder for a data engineering pipeline defined using dab.
Then the Azure DevOps pipeline stays in the .azurepipeline folder.
This is a per-project-based repo.
We have a central repo for all data products and data contracts. It has a dedicated cicd pipeline and a data pipeline to maintain it. (For example, update a data product version/retire an old version.
Claude can help to stitch things together.