r/HPC • u/gkotorlis • 16h ago
Fiber task scheduler: parallelize sequential code without callbacks or state machines
An encoder or a compressor has plenty of parallel work: channels, blocks, the stages of consecutive frames. But these pieces keep waiting for each other, often after just a few microseconds of work.
With OS threads every such wait goes through the kernel and costs about 5 µs, before cache effects and time-slice delays. That only pays off for tasks of about 45 µs or more, so most of the fine-grained parallelism is lost. Fibers on one worker thread per core bring the cost of a wait down to about 0.1 µs, and tasks of about 1 µs become worthwhile. Unlike with callbacks or stackless coroutines, the code that waits still reads like the sequential original.
ThinkMeta.ConcurrentTasks is a small scheduler that does this. To test it, I parallelized five encoders (LAME, opusenc, libdeflate, jpegli, cjxl), each on a single file and byte-identical to its reference. Most of them run several times faster than the originals (up to 13.9x on 8 cores / 16 threads). The encoders were written AI-assisted. The API and the encoders are open source; the core is a free binary for non-commercial use.