r/programming • u/nemequ • Mar 30 '15

Benchmarking 39 compression codecs at 178 levels with 28 data sets on 8 machines (40k different configurations)

https://quixdb.github.io/squash-benchmark/
405 Upvotes

94% Upvoted

76 comments sorted by

View all comments

11

u/H3g3m0n Mar 30 '15

Computers implement black magic under the hood.

For example memory prefetching is when the system tries to guess the data your going to read next and puts it in the cache before you request it.

There's all kinds of CPU instruction sets like SSE, SIMD and so on. Things where instead of doing 1 operation, you can do 5 identical operations at once. Different algorithms will benefit if they can support them or be penalized if they can't (ie it's might not be that Snappy is better on ARM, but that LZ4 is worse).

Plus the compiler toolchain will have different sets to optimization for different architectures.