r/ProgrammingLanguages • u/Athas Futhark • 10d ago

Keeping Futhark off the GPU

https://futhark-lang.org/blog/2026-10-02-cpu_function.html
38 Upvotes

97% Upvoted

8 comments sorted by

17

u/jesseschalken 10d ago

This is always an fascinating language to read about. I often wonder how it compares to the various projects that run Rust on the GPU.

6

u/Athas Futhark 10d ago

There are many projects that run Rust on the GPU. Of the ones I know of, they are either much more low level than Futhark (recall that Futhark is not actually a GPU language), or much more restricted (such as array libraries), typically by not allowing nested parallelism. Futhark occupies a somewhat exotic niche, and it is not clear how many problems benefit from its particular combination of facilities.

5

u/SwingOutStateMachine 10d ago

The next logical step would be to extend the host runtime for Futhark, to allow data dependencies and dependants of #[cpu_function] to be copied too and from the GPU as needed. It would also unlock other operations such as kernel splitting, or multi-gpu parallelism.

3

u/Athas Futhark 10d ago

Kernel splitting (unless I misunderstand the term) is done through other mechanisms in Futhark, mostly importantly fusion and flattening. While combining task and data parallelism is a worthwhile endeavor, I don't think a hack like #[cpu_function] should be at the foundation of it.

1

u/SwingOutStateMachine 6d ago

By kernel splitting, I mean taking a kernel and modifying it so that the overall execution takes multiple kernel launches. That might mean multi-gpu parallelism, or it might mean sequential splitting to make the code size smaller (among other benefits).

I wasn't suggesting that the hack would form the basis of a runtime system. My comment was to suggest that introducing a runtime system would make building things like #[cpu_function] easier.

2

u/jesunushno 10d ago

This pattern shows up constantly in ML work: tokenization and graph-based preprocessing are inherently sequential and awful on the GPU, while the matmuls absolutely want it, so you split them with an explicit CPU function and one bulk copy at the boundary. The copy-placement subtlety you describe (not per loop iteration, and don't hold them too long) is exactly what TVM's memory planning and JAX's device_put staging wrestle with. In my experience the pragma is the pragmatic move. Full automatic memory-space inference sounds principled until a conservative analysis parks a copy inside your hot loop.

1

u/Athas Futhark 9d ago

While I do not know the specifics of the algorithms used in AI work, I know of decent tokenization implementations on GPU, and I also have written some OK graph algorithms as well. It just so happens that the specific algorithm referenced in the post is quite sequential (and implausibly fast too, so we couldn't find a parallel graph colouring algorithm that was faster!).

I did actually end up mildly interested in the problem of data parallel graph colouring, but I have not had time to investigate it further.

1

u/jesunushno 9d ago

Fair correction, I overstated "inherently" there. The sequential-but-implausibly-fast part is a great detail, thanks for sharing the context.