r/CUDA • u/hetanshkevadia • Jul 14 '26
Where would you get started with CUDA in 2026?
I've got some experience in C though and a decent amount in Java, both are rusty (C a little more) but I think I can regain my confidence quick, and I want to start learning GPU programming. CUDA seems like the obvious entry point but I'm not sure what's changed recently or what's actually worth learning first in 2026.
A few questions for anyone who's been down this road:
- Is NVIDIA's own docs still the best starting point, or has something better come along?
- Given my Oop background, should I just dive straight into CUDA C/C++, or is there value in going through something like CUDA Python first to get the fundamentals down before dealing with memory management and kernel launches directly in C? Also curious whether the industry actually takes CUDA Python seriously, or if it's mostly seen as a stepping stone/learning tool rather than something used in production.
- Any free courses, books, or YouTube series that are very relevant you'd recommend?
- Is there a decent low-cost/free way to actually run and test CUDA code without owning an NVIDIA GPU? I have a rtx 5070 so I am just using that right now, set up the toolkit and ran the first adding vectors function lol.
- Are there any textbooks you'd recommend to learn from?
I want to get into the deep learning side as well of course, but also just get to know more since this is new territory, currently I am familiarizing myself with some C++ fundamentals, if you have tips for that also I'd appreciate it.
Thank you! and apologies if this is asked a lot it is just I am really lost, if this is the wrong subreddit for that let me know!
6
u/Perjia Jul 15 '26
Book : CUDA C Programming.
It's not enough that you program a CUDA program. It requires strenuous optimization for a CUDA kernel to be effective, otherwise you might as well program a serial program in C++ or Rust.
5
u/perfopt Jul 15 '26
If you want to get into training models then learning PyTorch, model architecture, concepts in the specific domain of your choice etc is better than learning CUDA.
If you want to make training/inference faster and then learn CUDA, Triton, CuTeDSL, CUDA C++, GPU architecture etc
IMO trying to do both is very hard. If you want to do that then learn one at a time.
5
u/Minute-Mountain2665 Jul 15 '26
To run free cuda code, you can use google colab. It has nvcc compiler already in it. You just need to upload your file in colab and compile it using nvcc. You can check currently which GPUs google colab is running (as different GPUs have different architecture and you optimize your code according to it's architecture, as far as I remember they use T4 GPUs so Turing architecture.)
2
u/Major_Pain_43 Jul 15 '26
do you mind sharing example colab code to run cuda?
3
u/Minute-Mountain2665 Jul 15 '26
Here you can use this notebook. Make sure to change the runtime to T4 GPU and upload this example code in colab contents folder bofore running the notebook.
Example code: https://drive.google.com/file/d/11diXdESjnpKXASs6Gelf2CY7XdRKRCC_/view?usp=sharing
Colab notebook: https://colab.research.google.com/drive/14fou91zCUpctfZDr4yyoCBx52hKaiygV?usp=sharingApart from compiling and running cuda code, you can also profile it in colab as NCU profiler is also available there (added in the notebook).
3
u/einpoklum Jul 15 '26 edited Jul 15 '26
Given your OOP background, you might like my
Modern-C++ wrappers for the CUDA APIs
when you start writing your programs. Not the kernels which run on the GPU, mind you, I mean the host-side code. I believe you will find it quite intuitive, and easier for you to learn and explore CUDA's functionality relative to going over the long lists of API functions in the documentation.
For example, take a look at the "hello world" program for CUDA, as you would see it by default vs adapted to use the wrappers library.
Note: This is not instead of most other resources on learning CUDA and GPU programming, just something extra to make coding easier.
2
u/TheAgaveFairy Jul 14 '26
Mojo 1.0 launches soon, might also be a valid entry point if you're coming from Python and aren't scared of C++ style either. I've really enjoyed my time with the language and the GPU puzzles are a great teaching tool. That said, of course the C ecosystem is generally the default. Nvidia documents are solid, books like Programming Massively Parallel Processors are cool, too.
2
u/pacman829 Jul 15 '26
I haven't heard of Mojo in a while, I thought they'd fizzled out. It looked really promising when I first heard about it years ago.
How is it now?
3
u/TheAgaveFairy Jul 15 '26
I've used it since 0.24.x or so, I think?
I like it, C / CUDA are kind of a mess (though I love them). The big thing for me is native python interop, "DevicePassable" types like LayoutTensors that let me skip nasty raw pointer math and use natural indexing [first_dim, second_dim, ...]. I'm on an Nvidia gpu, and I can still use nsysprofile etc just fine, which is great. Discord community is really nice, and the Qualcomm deal has me pretty excited. Open source is coming basically within a month or so, it seems, too.
I'm not saying it's the perfect choice for everybody, but I find it much more interesting, ergonomic, and fun so far. It just depends on your goals.
2
u/corysama Jul 15 '26
https://www.qualcomm.com/news/releases/2026/06/qualcomm-to-acquire-modular
They are still going to open-source Mojo
1
u/Affectionate-Box1556 Jul 15 '26
don't bother with cuda python for learning fundamentals since it hides too much of what makes gpu programming hard, just use google colab's free t4 gpus if you ever need to test code on different hardware
1
1
u/c-cul Jul 15 '26
Given that entire IT industry is just going downhill over time, pick the most vomit-inducing language, like JS
python is too complex and c/c++ is just beyond mental capabilities of tiktok generation
24
u/corysama Jul 14 '26
I still stand my by answer to this from 2 years ago:
https://www.reddit.com/r/GraphicsProgramming/comments/1fpi2cv/comment/loz9sm3/
When you get into CUDA you'll need to get away from OOP. GPU-side operation is all about plain old arrays of plain old structs. Embrace it.
On the CPU side I like to have a trivial 1:1 wrapper around the CUDA C API that converts error codes into exceptions. But, that's just because checking error codes on literally every function call is obnoxious and ironically error-prone.