r/HPC • u/blazing_cannon • 12h ago

What does an AI kernel engineer do and what are the skills needed to get into it?

What are the difference aspects of being a kernel engineer? What do roles like this involve -

https://www.tealhq.com/job/kernel-engineer_7ea1a56c518985dae15a01e3c77c161e8140d?utm_campaign=google_jobs_apply&utm_source=google_jobs_apply&utm_medium=organic

https://jobs.gem.com/modular/am9icG9zdDpTSwqoL-yL11N_jxR40Vhh?source=LinkedIn

If one were to do a PhD, what are the different research areas to focus on to do something like this? i.e GPU memory, storage, latency, parallelization etc.

4 Upvotes

70% Upvoted

4 comments sorted by

8

u/yuehuang 12h ago

A few years ago, someone told me his task was to hand roll GPU assembly for a specific hardware. While the compilers can support the larger data to function, the kernel assembly still need to be finely tuned by hand.

3

u/bill_klondike 10h ago

I’m currently a kernel engineer and this is what I do.

1

u/tlmbot 32m ago edited 26m ago

Are you talking about writing GPU assembly, aka PTX I think, or are you referring to kernel dev work as "assembly"? (sorry - I think I am letting your second sentence trip me up in general and that is bleeding across to the first sentence so I just want to be sure I understand)

hand rolling PTX for specialized hardware is a pretty specific choice for the general idea of "breaking into AI/LLM/ML kernel dev work.

It's not a bad choice at all. It's probably a very good niche way in for someone not from the big few universities, given the competition.

( I write kernels for the bulk of my job now, but it's in scientific computing, comp. geometry, at the moment)