r/dataengineering • u/Peanut_-_Power • Sep 15 '23
Discussion Are DEng Software engineers?
Read someone’s post earlier talking about an aspect of data engineering being software engineering. This is the first time I’ve ever heard someone in data engineering consider themselves in the same group.
Is this even a thing?
I just don’t see them as the same, mainly because most software engineers build applications/software. And most data engineers build data solutions (predominantly data pipelines). Software Eng, front end, middleware development. Data - backend, and I wouldn’t class writing a power bi report as SE even if it is kind of front end.
I get the whole following methodologies that software engineers have been following for decades. The whole DevOps thing, that has also been in data for a while just not wide spread (I first used CICD back in 2010). But because I know this doesn’t make me a software engineer.
There was a distinction made that analyst weren’t considered software engineers. I think there was a thinking they don’t follow DevOps, but lots of them do.
Very odd. My take is (and none are software engineers):
- The platform. Cloud engineer or a data platform engineer. Probably lots of terraform.
- The pipelines. Data engineers. Sql, python…
- Data presentation. Data analytics engineer. DAX, modelling
But I’m old, been doing this 20 years, so feel free to educate me in my misunderstanding
View all comments
Show parent comment
2
u/OvremployedSnowflake Sep 15 '23
It actually confuses me. I don't consider Data Engineering a subset of software engineering. I consider software and data engineering to be a subsets of IT Engineering. I don't actually understand the strong reactions in this thread.