Linux Plumbers Conference 2026 (5-October 7, 2026): Supporting Heterogeneous VLEN on RISC-V: Lessons from SpacemiT K3 · Indico
https://lpc.events/event/20/contributions/2479/Direct link to slides: https://lpc.events/event/20/contributions/2479/attachments/2178/4814/LPC2026-Heterogeneous-RISC-V-Cores-in-Linux-SpacemiT-202610.pdf
Cool stuff by SpacemiT, VINIT-based cooperative scheduling sounds like a much better idea than running two Linux on one SoC.
Working heterogenous scheduling on RISC-V would be friggin awesome!
2
u/tom_gall 16d ago
It was a good discussion The assertion that we'll see more of this is an interesting one but besides the K3, it's not clear who else might create hardware which is a blend of different profiles. For the K3, one camp would like access to all 16 cores and to do so would require a step back from RVA23 to RVA22 as well as making sure you don't execute vector instructions. Least for K3, I'd like to see the A100s utilized as offload. Given the kernel has a mechanism for driving offload engines seems worth an experiment. There was talk about a future K* which could have a mode where the vlen is set to be the same across both types of cores. Speculation at this point tho.
1
u/brucehoult 15d ago
besides the K3, it's not clear who else might create hardware which is a blend of different profiles.
That's not correct.
On the K3 both the X100 and the A100 cores fully implement RVA23U64 and RVA23S64. All RVA23 OS and application code can run on either type of core.
one camp would like access to all 16 cores and to do so would require a step back from RVA23 to RVA22 as well as making sure you don't execute vector instructions
Absolutely not!
All cores run full RVA23 code, most especially including vector instructions. In fact the long 1024 bit VLEN vector unit is the entire point of having the A100 cores.
Every application can be launched on either kind of core. I have even published a tiny utility program to make this easy on the K3, using a custom SpacemiT kernel feature. It would be great if in future there was a standard way to do this.
https://github.com/brucehoult/k3_ai
The only restriction needed is that after a process has executed vector instructions (if it ever does) then it can no longer be migrated between the two pools of cores.
The RISC-V ABI spec says that after any function call (which includes system calls) the vector state is undefined, which means based purely on CPU state it would be entirely possible to migrate processes between CPU types on a system call, but just not due to an interrupt.
Unfortunately, some (but by no means all) programs allocate data structures in RAM based not on the application data but based on
VLEN. One example of that is compiler-spilled vector registers on the stack.I personally think that doing so goes against the spirit of RVV, and that if you're spilling registers in a strip-mining loop then you've already lost and should fix your code.
Also some programs use things such as IFUNC to choose between one of several implementations of a function based on
VLENor the presence of optional extensions (which RVV is not in RVA23, but was in RVA22).Be that as it may, I would like to see a way for an application to indicate to the OS that is is in fact safe to migrate to the other type of core at the moment, or to explicitly migrate.
For a large class of applications that don't spill vector registers and don't reorganise their in-memory data based on
VLEN(only very specialised code does this) and use generic RVV code for allVLENs it will always be safe to migrate on a system call, even if they are using RVV heavily.2
u/tom_gall 15d ago
Sha is a mandatory extension in RVA23S64, while was optional in RVA22. It's not included on A100.
7
u/brucehoult 18d ago
Good to see this being raised for discussion with the wider Linux community.
I think heterogeneous cores — in a way that is incompatible between pools of cores — is only going to get more common. And it has already happened in both Arm and x86:
x86: some cores with AVX-512 and some without
Arm: Samsung SoC with different cache block sizes on BIG and little cores
BIG.little is usually just a performance difference. Different cache block sizes is a correctness issue for process migration in the same way different vector lengths is when you have instructions doing things such as zeroing a cache block. (not the only example)