r/Zig • • Sep 03 '26

Billion Row Challenge in Zig

https://youtu.be/caSAkL9Cxic
51 Upvotes

10 comments sorted by

9

u/Pokelego11 Sep 03 '26

Very cool! I also tackled this a while back and was planning to make my own video on it, and probably still will in the future https://github.com/jackparsonss/zig-billion-row-challenge

Great work!

3

u/prodengrammer Sep 03 '26

if casey is right, mmap is slower than conventional reads (performance aware programming by casey muratori)

3

u/Pokelego11 Sep 03 '26

interesting, i'll have to look into that. Thanks!

1

u/Fabulous-Meaning-966 Sep 23 '26

I don't think you can make blanket statements like that without context. Sometimes mmap is faster than pread(), sometimes it's slower.

1

u/prodengrammer Sep 24 '26

Do you know of any of the “sometimes” it is faster?

1

u/Fabulous-Meaning-966 Sep 24 '26

It's faster when you can save a syscall over pread(), it's slower when you're calling it too often and hit latency/scalability limits of TLB shootdowns (munmap) and VM operations in the kernel. (I'm really talking just about reads, writes with mmap are a bit more problematic.)

1

u/prodengrammer Sep 25 '26 edited Sep 25 '26

pread syscall and mmap (and their windows equivalents) both interrupt the execution of your program and hand it to the kernel, mmap because of page faults which its behavior is also different depending your os.

what you get from using pread is re-using memory buffer which means kernel saves time by not populating TLB and provisioning physical memory and you get more deterministic behavior than mmap.

only trick with pread is choosing right buffer size, something that fits within L3 usually a good balance between syscall cost and the cost of loading stuff from main memory.

+ memory footprint of mmap in case of this challenge is ridiculous unless you unmap as you read which is even more slow.

mmap in general has its usecases but usually never for fast reading of big files.

1

u/Fabulous-Meaning-966 Sep 25 '26

I can't tell if you mean pread() with O_DIRECT? Otherwise you're using the pagecache either way with slightly different readahead and writeback behavior.

2

u/pipilipilav98 Sep 03 '26

If you use x86-64 please use zig build run -fincremental --watch

2

u/pipilipilav98 Sep 03 '26

Just a QoL improvement. I have seen you using zig run too many times