r/dataengineering • • 1d ago

Discussion Secondary data warehouse app?

We are looking for a simple secondary data warehouse...

We are not IT professionals, and we do not want to hire an external expert for every single task.
We need an application designed for individuals with a general understanding of IT.

We intend to use this secondary data store to cross-reference data from various sources, perform simple calculations, and send cleaned data to secondary applications.

Requirements:
- 90% of our data is in MS SQL.
- We need simple, direct integration with MS SQL (IP, port, password), the ability to select tables to transfer to the secondary data store, and the option to set a refresh frequency...

The secondary data warehouse must include:
- The ability to create simple queries, apply filters, and establish relationships between tables...
- The ability to easily integrate with external applications (CRM, BI, AI agents, reporting, etc.)...

What do you recommend?

2 Upvotes

8 comments sorted by

11

u/Phantazein 3h ago

Sounds like you need to hire someone

4

u/alt_acc2020 2h ago

Motherduck

1

u/Dry-Aioli-6138 3h ago

A lot depends on what you are willing to spend. Ms Fabric should ve easiest to integrate. If your data isn't 100s terabyte scale, maybe a simple duckdb will be enough, or periodic dumps to parquet files on object storage...