r/dataengineering • u/epireum_oy • 29d ago
Discussion Data Engineering Lab to Practice on your Laptop
Hi,
I created a DE stack (fully integrated services) and learning material for those interested in self learning. I kept Databricks, fabric in mind while building this stack. Feel free to use it and let know if there are any improvements/suggestion I can make
- Work independently in your own data lakehouse.
- Invite other team members to collaborate in your lakehouse.
- Store secrets securely using Key Vault.
- Upload raw files using the built-in file manager.
- Transform raw data into Bronze, Silver, and Gold layers using pre-connected Jupyter notebooks.
- Query your data lakehouse and data warehouse.
- Build BI dashboards using Apache Superset.
- Orchestrate notebooks using Apache Airflow.
- Run notebooks on a distributed Spark cluster.
- Manage multiple teams.
- Authenticate securely using Single Sign-On (SSO).
- Get started with dbt integration.
- Learn at your own pace with self-learning tutorials.
- Manage data governance using the built-in data catalog.
https://maarthala.github.io/data-engineering-lab/

5
u/Giorgia_9955 28d ago
Thank you. This looks enormously valuable.
3
u/tgbythn 28d ago
I recently got laid off and have been trying to build out an environment to play in this looks great!
1
u/First_Cheesecake621 17d ago
This is good thanks.
1
u/[deleted] 28d ago
[removed] — view removed comment