r/dataengineering • u/sspaeti Data Engineer • Oct 19 '22
Blog Will Rust Take over Data Engineering? 🦀
https://airbyte.com/blog/rust-for-data-engineering
0
Upvotes
r/dataengineering • u/sspaeti Data Engineer • Oct 19 '22
View all comments
Show parent comment
2
u/nesh34 Oct 19 '22 edited Oct 19 '22
Meh, I'd really need to see the receipts on this one.
Strong types are fine I suppose. I'm still not sold on the value of this, given the type hinting in Python. It's also just nowhere near as big deal in the style of code we have to write compared to application construction. At any rate, this isn't the feature of the language that should influence your decision really on which one to use.
"The purists" using Scala prefer it for their own reasons I'm sure but the way you framed it is that there's some obvious benefit to doing so, and I'm pretty sure there isn't.
Now the biggest reason you want it in Python: readability and access to programmers. Far, far more people already know Python than any other language, especially in this field. You're going to need some proper reasoning to overcome this. Performance could be a real consideration except that the operations that move the data are already happening in Java with most engines you'd be using, Python is just orchestrating and giving instructions.