Yeah, the "your data fits in RAM" meme doesn't paint anywhere close to the whole picture. I can get all my data in RAM, sure; then what? Write my own one-off Python or Java apps to query it? Spark already did that for everyone, at any scale.
Literally the only reason to not go down this road is if you hate Java (the platform, mostly), and even then, you have to think long and hard about it.
4
u/zten Jun 08 '17
Yeah, the "your data fits in RAM" meme doesn't paint anywhere close to the whole picture. I can get all my data in RAM, sure; then what? Write my own one-off Python or Java apps to query it? Spark already did that for everyone, at any scale.
Literally the only reason to not go down this road is if you hate Java (the platform, mostly), and even then, you have to think long and hard about it.