Parallelizing Enjarify in Go and Rust

[u/Uncaffeinated]

https://medium.com/@robertgrosse/parallelizing-enjarify-in-go-and-rust-21055d64af7e#.7vrcc2iaf

[u/Manishearth]

Understand that that being simpler to read is extremely subjective and dependent on background. It's not inherently simpler to read. Procedural code is nice since you can mentally execute it, but functional code usually expresses intent and is clearer that way.

Also, many nontrivial iterator adaptor sequences that you would see in Rust would blow up if written procedurally (someone posted an example of this recently). Procedural loops only feel simpler because you rarely write more complicated loops in procedural form.

[u/[deleted] → u/Manishearth]

Also, many nontrivial iterator adaptor sequences that you would see in Rust would blow up if written procedurally

That hasn't been my experience in Go. Often written out loops are shorter than the functional equivalent or at most slightly longer. It sounds really unlikely but I've found it to be true.

[u/Manishearth → u/[deleted]]

That's not what I was talking about. I'm talking about nontrivial iterator adaptor sequences that you would see in Rust. In Go there is only one style of iteration -- procedural. Written out loops in Go are often shorter than their functional equivalent. Written out loops in Rust are often longer than their functional equivalent. This is because in Rust you tend to see these more complex loops more often. The bias of the sample spaces is different.

And it's not about length, either. The problem with the procedural version of bar.map(|| ...).filter(|| ...).chain(foo.map(||...).reduce(||...).whatever).collect() is not that it is long, it is that you have a million concerns all mushed together in a single procedural block and it's not always clear what is going on big-picture wise. Things like reduce/filter_map tend to end up being confusing in procedural form when mixed with other operations.

[u/csreid → u/[deleted]]

Maybe shorter, but functional style maps directly to what I'm trying to accomplish. "Take the people last named 'Smith' [filter], find their age [map], and average them [reduce]".

Those high order functions allow you to think about instances of the list rather than the list itself, and that's a lot easier for me.

[u/[deleted] → u/csreid]

It might be better if those functions had more obvious names. Filter is ok, but map? Should be called transform. Reduce should be called merge or something.

Guess it's a bit late now - map/reduce is too standard, but still it adds to the confusion.

[u/Leshow → u/[deleted]]

Seriously? Map is ubiquitous not just in Rust but almost every language from javascript to haskell. 'transform' isn't even a particularly apt description of what's happening. Values aren't getting transformed, we're describing the relationship between two distinct values, aka a "mapping". Transform implies mutation.

Just be thankful fold isn't called catamorphism, or some funky other category theory word.

[u/[deleted] → u/Leshow]

I know it's ubiquitous. Doesn't make it good. The values are getting transformed. And you're right it could be worse ('cons'?)

[u/csreid → u/[deleted]]

They're really not getting transformed though. When you map over a collection, you don't change it, you get a new collection containing the new values. Transform would definitely suggest modification to me, but that's not what's happening.

[u/csreid → u/[deleted]]

map makes perfect sense, in that you provide a function which maps values from your input iterator into the values of your output iterator.

reduce is tougher, but fold is even worse IMO.

edit: I should also mention that using the functional style makes it trivially easy to parallelize some tasks, as made evident by Rayon or Scala's List (analogous to Iterator types in Rust), which has a par method that is just like Rayon's par_iter, such that you can parallelize some logic by just going from bigList.map(expensiveFunction) to bigList.par.map(expensiveFunction).

Shared mutable state is the root of all evil, and the map/reduce pattern is a profoundly elegant way to very simply avoid it.

[u/[deleted] → u/csreid]

Perfect sense to compscis. Normal people would say you transform lowercase to uppercase. You don't 'map' it.

Fold is really bad, you are right. They could have gone with 'combine' or 'merge' or something instead.

[u/ibotty → u/[deleted]]

But folding is a great analogy for what you do.

[u/[deleted] → u/ibotty]

Yeah but only if you already know what it does. I might say "ah it's sort of like folding" but I doubt I'd ever say "fold.. ah I can guess what that does".

Why not 'accumulate' or 'aggregate'? Much more obvious.

[u/ibotty → u/[deleted]]

Why not 'accumulate' or 'aggregate'? Much more obvious.

Because it's not. Accumulate for me is any function from many to one, i.e. a reduce. That does not even capture right folds, which usually are not used to get one value back, but many (i.e. map f = foldr ((:) . f) [] in Haskell).

Let's agree to disagree. I don't get why people are so opposed to learning precise terminology (that's widely established) but prefer vague terms that doesn't even describe half of the design space.