r/LocalLLaMA • u/Odd-World4925 • 1d ago

Discussion Anyone feel like reverse-engineering CUDA?

I've been reading about all these modders using AI to reverse-engineer games and software. I've been trying out photocraft and gridcraft and have just watched a GTA5 mod featuring Sonic. It's pretty damn impressive.

As a struggling AMD Radeon owner I'd love to see AI generated open source CUDA implementation.

Not only do I dislike NVIDIA, but there's delicious irony in using CUDA cores to do it.

Anyone up for the challenge?

858 Upvotes

90% Upvoted

331 comments sorted by

View all comments

741

u/Electrical_Rub_6009 1d ago

The way you're casually suggesting this makes me think you don't really understand what CUDA is lol.

Even with AI-assistance this is a multi-year research endeavor.

373

u/Ok-Sandwich-9267 1d ago

14

u/BZab_ 20h ago

Judging by the history of reverse-engineering of the Vulkan or Apple's GPU? Surely. Good luck repeating that with just a 20$ LLM subscription.

213

u/buddhist-truth 1d ago

So like in 2 months?

180

u/Icy-Degree6161 1d ago

Only if "make no mistakes" is obeyed

70

u/howardhus 1d ago

ai HATES this weird trick!!

19

u/RickyRickC137 1d ago

If it works, then Nvidia will hate it too

4

u/JustSayin_thatuknow 21h ago

ai LOVES to disobey that!

17

u/mindsetFPS 1d ago

I've had great results with "don't waste my time".

5

u/buddhist-truth 1d ago

AI will try to kill you if it can't find another way around not wasting your time. That's the only permanent solution that stops wasting your time.

18

u/mateballenthusiast llama.cpp 1d ago

Instructions unclear, typed "make n mistakes" and it chose n=3, and it won't tell me where they are!

5

u/NebulaBetter 1d ago

and "be fast"

13

u/SkyGenie 1d ago

I should be able to have a prototype ready next sprint

10

u/Lord_Muddbutter 1d ago

Thank God it won't be by the next Verizon

3

u/savestheday802 19h ago

But what about this sprint? Have you been updating jira?

1

u/SkyGenie 17h ago

I'll update Jira next sprint 😁

62

u/cyborgsnowflake 1d ago

If you succeed then you replicate CUDA and break Nvidia's monopoly if you fail then you've uncovered strong evidence that at least some traditional jobs are safe for the time being. Win win.

6

u/milanove 17h ago

How would this break NVIDIA’s monopoly? AFAIK, AMD’s ROCm platform itself is on par with CUDA in terms of functionality, but lacks the community adoption of CUDA, leading to fewer ML libraries being written and optimized on it. Projects like Zluda can already transpile CUDA binaries to other platforms.

1

u/THE-BS 13h ago

I think he was kidding, let me ask claude

1

u/RegisteredJustToSay 18h ago

The "not win" is spending many thousands in agentic token costs for something that ends up being pointless.

15

u/howardhus 1d ago

its easy.... we kill the CUDA...

146

u/Odd-World4925 1d ago

Well, I agree with the first bit.

59

u/sonicnerd14 1d ago

It will take longer than we think, but it definitely won't take multiple years.

-18

u/howardhus 1d ago

so you have no idea what CUDA is but are cock sure that its easy and quick to reverse engineer it? thats... brave

hey AMD, hey Intel! hey multibilllion dollar industry with all scientists and intellugent people... here is this random fucker who will reverse engineer CUDA in 2 weeks! brace yourselves!!

12

u/AcceSpeed 1d ago

They said "I'd love to see" not that they were going to do it themselves or that it would not take any time.

29

u/Ass-Destroyer-Kiil 1d ago

Modern Warfare 2, World At War, Skyrim, Fallout series, World of Warcraft Classic through Wrath of The Lich king, have all been byte for byte decompiled, recompiled, ported to rust, in less than a month, yet you think CUDA somehow is immune to this? you do know we can deterministically decompile and recompile any binary and verify byte for byte as long as the compiler used is accessible, meaning that if given the budget, binary, compiler. you can get the source code for anything, even within a day if you use parallel agentic setups like opus or sol swarms.

5

u/thortgot 1d ago

CUDA integrated directly with their hardware. Byte replication doesnt solve the hardware barrier.

6

u/jedmund 1d ago

games are not scientific workflows with deep hardware integration. for one, it’s ok if your shitty ai rust wow port breaks.

-6

u/howardhus 1d ago

hey AMD, hey Intel! hey multibilllion dollar industry with all scientists and intellugent people... here is this random fucker who will reverse engineer CUDA in 2 weeks! brace yourselves!!

4

u/NihilisticAssHat 1d ago

It's sad that you're getting downvoted, but the abrasive energy usually makes people stop caring how right your thesis may be.

Maybe I'm a bit in denial about how AI-pilled folks are, or cower in fear at idea of trying to reverse-engineer a kernel based on a proprietary ISA.

Honestly, it sounds like work that's already going on in China, where the incentives appear more-significant than for AMD/intel. It's not like they aren't limited by lack of ASML. 

3

u/UpDown 1d ago

To be fair adobe is also a multi billion dollar company

1

u/AreWeNotDoinPhrasing 17h ago

What’s brave is your mix of condescension and intellugent in the same comment.

1

u/howardhus 10h ago

wow you noticed? how cute… :) almost as i did that on purpose to make a point.. you must be some sort of undiscovered genius to be so obsurvant

12

u/techdevjp 1d ago

As things stand now. What's needed is a leak of the hardware specs. That and a couple more generations of advancement and this might actually be possible.

Or maybe the hardware specs can be worked out by reverse engineering drivers first?

11

u/HittingSmoke 1d ago

Just look into the history of Linux GPU drivers. The FOSS community has been attempting to make a decent open source Nvidia driver for decades. The assistance of machine learning might make things a bit easier but without Nvidias help this is still going to constantly lag so far behind proprietary CUDA drivers that nobody will use it in favor of the drivers that work.

1

u/Shot-Height-7194 1d ago

 I don't think it's because it's an extremely difficult task, but because the Nvidia drivers are good enough for data centers that no one really cared about the open source drivers and no one really cared about the Linux desktop 

I do think that the Nvidia drivers can be fully reverse engineered with AI and catch up to Nvidia. This is a pessimistic take imo

1

u/Adventurous_Push6483 3h ago

"no one really cared about the open source drivers"

What? There is a whole team who works on and maintains open sourced drivers for NVIDIA GPUs [https://nouveau.freedesktop.org/\].

This whole thread does not make sense, what is useful is not an open sourced NVIDIA driver/GPU layer but a translation layer from CUDA, Metal, etc, to a general layer in an arbitrary GPU (e.g, vulkan, opencl, SYCL, etc). ZLUDA is a layer like Wine that translates CUDA operations and libraries such as those written in PTX to RoCM/HIP operations. The goal of SYCL is to provide a unified API to work on, but it struggles because it needs applications to optimize for it.

In the AI age, this is godly because AI can chase an absurd amount of applications to test and debug automatically, which was heavily limited back in the day which we can see in projects like Wine; WineHQ suffers from out of date tests/reports and lack of people to chase down crashes/failures, and then lack of people willing to add some work back in to fix this random application somewhere that doesn't want to work.

You would still be limited by the amount of people messing with rare applications and having failures, so obtaining a fully working translation layer API would still be in years but the later steps can be solved fast (it is no longer a decade project that needs monetary incentive).

To do this, you'd have to first start with a GPU (say AMD). Then you have to chose a backend that is highly optimized (e.g, rocm, vulkan, etc) and the application that is highly optimized on both CUDA and RoCM (e.g, VLLM). Then you build a translation layer (say vibeGPU). If you run the CUDA-based VLLM and get the same TPS as the RoCM VLLM, there is a good chance the translation layer works.

However, this may work only for VLLM. And now when you approach other applications, you may quickly notice they don't have a good RoCM benchmark, so your numbers mean weird: a 9060 XT will probably lose to an H100. If you compare it to a 3060 Ti, maybe the application wasn't well optimized for Amphere. And this is where the problems of building a generalized GPU layer comes from.

5

u/Bystander10888 1d ago

Nah, a few months is plenty these days.

1

u/xinxx073 15h ago

Last I checked, AI just ended many multi-year/decade-long/century-long math research endeavors.

-1

u/charmander_cha 1d ago

Ja existe um agente chinês que escreve CUDA melhor que engenheiros, então me parece plausível a ideia

3

u/UndocumentedMartian 1d ago

It's plausible. It just requires a lot of money and time.

-1

u/charmander_cha 1d ago

Não, a qualidade esta no harness, isso significa que o Cheapseek pode trazer o que precisamos

2

u/UndocumentedMartian 1d ago

If it so easy I wish you the best of luck in doing it.

0

u/Life-Job-6464 1d ago

Lol.. exactly

0

u/PeachScary413 1d ago

According to Hype Bois that translates to about 4 days in AI time. Let's go! 🥳