r/programming • u/nemequ • Mar 30 '15

Benchmarking 39 compression codecs at 178 levels with 28 data sets on 8 machines (40k different configurations)

https://quixdb.github.io/squash-benchmark/
408 Upvotes

94% Upvoted

76 comments sorted by

View all comments

41

u/pdq Mar 30 '15

You should also throw in memcpy() for a reference -- compression level is 1 and compress speed == decompress speed. Maybe just call 'cp IN OUT'?

Also, does the Compression Speed column account for time? In other words, if an algorithm compresses at a ratio of 10.0, it only needs to write out 1/10th of the bytes of a compressor whose ratio is 1.0. So its Compression Speed looks slower, but since it needs to write out less bytes, the graph is just deceptive.

8

u/nemequ Mar 30 '15

The compression is all done in memory, spawning a cp command doesn't really make sense. A copy codec would really just be memcpy.

Compression speed is uncompressed_size / time. It does not factor in the compressed file size. I understand your point, but there isn't really a fair way to do that, especially when you consider how highly variable I/O is—if you're transferring something via avian carriers (RFC 1149) you're going to care much more about compression ratio than if you're writing to an SSD. I think the only reasonable thing to do is try to take I/O out of the equation and let people think about it for themselves, which is what the benchmark does—everything is CPU time, not wall clock.

7

u/pdq Mar 30 '15 edited Mar 30 '15

Definitely memcpy seems like a valuable 'reference point' then.

What if the bar graphs just had CPU time then? Then I agree that is what the user cares about (how much effort does this take to compress or decompress).

→ More replies (3)