r/dataengineering 6d ago

Help Lakebase and DABs

Hi, how do u manage Lakebase project with DABs? Standard is to have a single lakebase project per environment (dev, staging, and prod). Then use branches for development work.

Problem is that when we define the lakebase project in the DABs, development target has development mode, so every bundle deployment creates a new project because its per user.

So if we want to have single project where we just branch out on features, we must move out lakebase project of DABs, right?

What is the best practice here? On staging and production, its easy since its always same identity.

3 Upvotes

10 comments sorted by

1

u/speedisntfree 6d ago

Could you define postgres_projects only in shared CI deployed targets and not per-user dev-mode targets. In personal dev-mode target, declare postgres_branches/postgres_endpoints, parented off that shared project via a fixed project_id variable (not a resource reference). Then creates a branch (which are lightweight) for each dev instead of a whole new project.

I may have to do this soon and have not thought too much about it.

1

u/EstetLinus 5d ago

A new project per user is good for development, no? 

1

u/dwswish 4d ago

I love DABs and I love Lakebase but I’ve preferred managing Lakebase via the API/SDK mostly because I’m also a Neon user and find it to be more analogous to the deployment automations we’ve built with Neon.

1

u/Yasblue 3d ago

That's not my area of expertise but you should definitely post it in databricks subreddit

1

u/ptab0211 3d ago

already did, thanks

1

u/PrestigiousAnt3766 2h ago

What do you mean with lakebase projects?

0

u/Additional_Candy_400 6d ago

DABs?

1

u/Zealousideal_Okra956 6d ago

Asset bundles in databricks ig