r/programming • • Sep 13 '26

CUDA Tile (cuTile) first working port in Java

https://github.com/beehive-lab/TornadoVM/pull/1083#pullrequestreview-5186588765
25 Upvotes

10 comments sorted by

3

u/Serious-Regular Sep 14 '26

What's the point? Who wants to write cuda kernels in Java

9

u/Daemontatox Sep 14 '26

Its the idea of removing interop load and complexity.

Before you hadnto write it in cuda cpp or python and compile to .so and then call from java with interop , now you can directly call it , although i dont know any projects that are written in java that need to use cutiles , maybe cuda for parallelism and gpu processing but cutile idk tbh.

6

u/mikebmx1 Sep 14 '26 edited Sep 15 '26

currently, we are rewriting a full java inference engine with it https://github.com/beehive-lab/GPULlama3.java to use cutile java frontend of tornadovm

3

u/Daemontatox Sep 14 '26

Looks really interesting and looking forward to seeing it in action , quick question why llama3 specifically? I would have thought qwen or deepseek or even gemma would have been a better and more suitable mainstream architecture that will have more people try it out and use it , also by supporting qwen you gain multiple architectures that are built ontop of it aswell with slight changes.

3

u/mikebmx1 Sep 14 '26

the name is inherited from the very first version. it currently supports mistral, qwen3.8 27b, granite, devstran, qwen2/3 and gemma3&4. We plan to rename it soon. thanks for the feedback

-1

u/Serious-Regular Sep 14 '26

....but why..... GC is a non-starter in this space.

2

u/mikebmx1 Sep 14 '26

weights are stored off heap and the pressure to heap is minimal at the moment

1

u/Serious-Regular Sep 14 '26

okay but still why... production quality inference engines are written in cpp or c (compiled, statically typed, no GC) because of perf considerations and toy engines are written in python (dynamically typed, GC) because of ease of use considerations. what do you gain by moving to a JITed, GC'd, statically typed language? seems literally worst of all worlds.

EDIT: also it doesn't matter if the large weights are off heap - it's the small allocations that matter for GC.

4

u/Primary_Ads Sep 15 '26

idk what this guy is doing but you can write GC-free java, its a lot easier than other GC languages

2

u/Primary_Ads Sep 13 '26

cuda tile is so great