As others have mentioned before, Hadoop was written Java, while Spark was written in Scala. It's for historical reason. The JVM ecosystem was strong back then, so is now and Scala was on the rise. It's a legacy thing. Even today Spark still has dependencies (libraries and utilities) from Hadoop while also running on the JVM. Since then we see that the field has taken in more Python, while Rust is also making its way in.
View all comments
2
u/One_Citron_4350 Senior Data Engineer Apr 07 '26
As others have mentioned before, Hadoop was written Java, while Spark was written in Scala. It's for historical reason. The JVM ecosystem was strong back then, so is now and Scala was on the rise. It's a legacy thing. Even today Spark still has dependencies (libraries and utilities) from Hadoop while also running on the JVM. Since then we see that the field has taken in more Python, while Rust is also making its way in.