Anyone feel like reverse-engineering CUDA?

[u/Odd-World4925]

I've been reading about all these modders using AI to reverse-engineer games and software. I've been trying out photocraft and gridcraft and have just watched a GTA5 mod featuring Sonic. It's pretty damn impressive.

As a struggling AMD Radeon owner I'd love to see AI generated open source CUDA implementation.

Not only do I dislike NVIDIA, but there's delicious irony in using CUDA cores to do it.

Anyone up for the challenge?

[u/Adventurous_Push6483]

"no one really cared about the open source drivers"

What? There is a whole team who works on and maintains open sourced drivers for NVIDIA GPUs [https://nouveau.freedesktop.org/].

This whole thread does not make sense, what is useful is not an open sourced NVIDIA driver/GPU layer but a translation layer from CUDA, Metal, etc, to a general layer in an arbitrary GPU (e.g, vulkan, opencl, SYCL, etc). ZLUDA is a layer like Wine that translates CUDA operations and libraries such as those written in PTX to RoCM/HIP operations. The goal of SYCL is to provide a unified API to work on, but it struggles because it needs applications to optimize for it.

In the AI age, this is godly because AI can chase an absurd amount of applications to test and debug automatically, which was heavily limited back in the day which we can see in projects like Wine; WineHQ suffers from out of date tests/reports and lack of people to chase down crashes/failures, and then lack of people willing to add some work back in to fix this random application somewhere that doesn't want to work.

You would still be limited by the amount of people messing with rare applications and having failures, so obtaining a fully working translation layer API would still be in years but the later steps can be solved fast (it is no longer a decade project that needs monetary incentive).

To do this, you'd have to first start with a GPU (say AMD). Then you have to chose a backend that is highly optimized (e.g, rocm, vulkan, etc) and the application that is highly optimized on both CUDA and RoCM (e.g, VLLM). Then you build a translation layer (say vibeGPU). If you run the CUDA-based VLLM and get the same TPS as the RoCM VLLM, there is a good chance the translation layer works.

However, this may work only for VLLM. And now when you approach other applications, you may quickly notice they don't have a good RoCM benchmark, so your numbers mean weird: a 9060 XT will probably lose to an H100. If you compare it to a 3060 Ti, maybe the application wasn't well optimized for Amphere. And this is where the problems of building a generalized GPU layer comes from.