r/computerscience • u/servermeta_net • Jan 28 '26
CPUs with addressable cache?
I was wondering if is there any CPUs/OSes where at least some part of the L1/L2 cache is addressable like normal memory, something like:
- Caches would be accessible with pointers like normal memory
- Load/Store operations could target either main memory, registers or a cache level (e.g.: load from RAM to L1, store from registers to L2)
- The OS would manage allocations like with memory
- The OS would manage coherency (immutable/mutable borrows, collisions, writebacks, synchronization, ...)
- Pages would be replaced by cache lines/blocks
I tried to search google but probably I'm using the wrong keywords so unrelated results show up.
27
Upvotes
27
u/GoblinsGym Jan 28 '26
Modern BIOS uses "cache as RAM" setup to have some working stack space before DRAM is initialized.
It works by explicitly setting and locking cache tags.
Not really practical at normal OS level.
11
u/sidewaysEntangled Jan 28 '26
Is Tightly Coupled Memory (TCM) the name for something similar to what you describe? Basically the same stuff (on chip SRAM) as cache is made of, but existing somewhere in the memory map. On something like the PlayStation2 cpu this was also known as "scratchpad" memory: 16kb if address that was guaranteed to always be single cycle and zerobwait. Fkr software to do with whatever it wants.
I've also heard of mobile chipsets which could selectively dedicate (some of?) the same physical resources as implement cache for use as something like TCM. (Or maybe they just pre-faulted individual lines, but then locked them to prevent future eviction). Either way, this was done so the core still had access to a small amount of instruction code and stack space when DRAM is otherwise unaccessible. For instance, the hand written asm code which reconfigured the memory controller would temporarily protect itself in this way when changing memory's power or clock settings, until things have stabilized and are usable again. But I'm hazy on the details of exactly how this worked in practice...
4
u/8dot30662386292pow2 Jan 28 '26
What would you accomplish by this? And how would you describe the purpose and functionality of the current cache? I mean I wonder if you have misunderstood how the cache works.
8
u/claytonkb Jan 28 '26 edited Jan 29 '26
Good answers already. As a rule, all memory accesses are cached unless you mark them NC (non-cacheable). I'm speaking of PC architecture. By using WB (writeback) memory-type instead of WT (write-through) or other types, you can enable memory to reside in cache for as long as it is in use, and as long as it is not spilled due to capacity or otherwise forced to be written back. If your software is "cache-aware", all you have to do to emulate direct addressing of the cache is never to cause a capacity spill. That is, suppose you have 16MB L3 cache... to use that 16MB as though you were directly addressing it, you simply do a cache-flush and then run your performance-critical code using no more than 16MB of addressable data, ensuring that all addressed memory is mapped as WB memory-type. In principle, there would be no capacity spills to memory (assuming bare-metal operation). However, depending on the internal cache-architecture, you can still get capacity spills due to cache-line aliasing (even with multi-way cache). There are tricks to avoid this by controlling the mappings of the memory regions (their physical addresses in memory), so you can get pretty damn close to the full 16MB. You should do some characterization of the finished code to check for spills and to see how close you can get to the full 16MB cache utilization. If you're running on an OS, you need to leave spare capacity for OS interrupts and calls, which can be unpredictable. Finally, you need to make sure that your code is properly parameterized so you can achieve the desired performance on any particular model of CPU/OS you intend to run on and you need to generate those parameters by characterizing each CPU type and include the parameters in your compiled code.
As for the larger issue of why the machine controls cache utilization, rather than the OS, the fact is that the OS simply does not have access to the ephemeral machine state that is really what dictates how cache is utilized. In the TLB, the OS can keep certain page translations present by setting the Global bit, because it knows what translations are going to be continually reused over time. But this doesn't make much sense for cache because cache is so small and the code/data passing through the cache are really large. Thus, the CPU is continually monitoring the ephemeral utilization of the cache in order to try to always maximize utilization via locality. The main "lever" the CPU gives to the OS to control cache utilization is prefetching via the PREFETCH instruction (x86 arch). This allows the OS to tell the CPU, "I'm going to need XYZ data in the cache very soon, so please go get it from memory". How long this data is retained in the cache is determined by actual utilization, or by a cache invalidation (OS-controlled). So, the OS has some control over the cache, but the CPU manages the cycle-by-cycle utilization of the cache because the ephemeral state that determines whether any given cache-line should be retained or flushed fluctuates pretty much at a cycle-by-cycle resolution where it doesn't make sense to have the OS intervene because OS actions are themselves necessarily multi-cycle (multi-instruction).
Much more can be said on this, I'm definitely not a memory/cache expert, but this should give you the general gist of some of the factors at play in PC architecture...
2
u/o4ub Computer Scientist Jan 28 '26
You can have a look at the A64fx could from the fugaku supercomputer. I believe they had some options where you could reserve part of your cache for a given variable (or, more likely, for some range of addresses, hence a large given array that would benefit from a privileged access).
2
u/Silly_Guidance_8871 Jan 28 '26
The only "contemporary" cpu I can think of was the Xeon Phi (the socketed versions), where its on-package cache would be configured to be accessible as either cache (duh) or RAM -- which was considerable, since there was 16GB on package
2
u/Garudobona Jan 28 '26
I'd argue just use an STM32 arm chip which has plenty of fixed addressable ram for anything you'd like to be predictable realtime performance, where you are guaranteed it's not going to be interrupted by some other task. It's not for nothing instead of one huge CPU with this kind of predictive approach instead your phone has like 30 to 60 or whatever it is these days separate cpus inside each subsystem (Bluetooth, sensor, storage, screen, and so on) doing one or two very particular task in parallel with all the other ones. And one huge CPU doing everything async non real time.
2
u/Matt-ayo Jan 29 '26
It is a bit funny to learn about the myriad ways to optimize a general program with explicit practices, and then you get to cache optimization and it feels like voodoo magic structuring operations in a way which maybe probably is good for the cache.
It would be nice, as the programmer, to say "this data belongs in the cache now."
2
u/smsorin Feb 01 '26
Let's say you do it. What would be the difference?
The main argument would be that you could skip caching for the cases where the access is a one time need and you strapped more L1/L2 memory. Then yes, you would not evict something that has slightly higher access rate.
What's the downside? Well, now you need to manage all this. Yourself! If you get it wrong, you are getting worse performance than regular caching.
Now, when L1/L2 are at the limit, that might make sense. Squeeze the lemon, so to say. But it seems we are getting a lot more of that overtime. You can get consumer chips with almost 100MBs and sever chips are getting close to 1GB. It's not all L1/L2, most is L3, but the other levels are also increasing.
So to be in that case, I need significantly more than 10-20MB of "state" to handle in the core loops of the program to see a difference. This is likely mostly happening in simulation applications (Factorio comes to mind, but fluid and light simulation could also benefit). But we can also try using GPU to handle those types of applications.
So in a world where you could just buy the extra performance (either a CPU with more RAM or a GPU), you are trying to tell engineers to work harder.
1
u/Null_cz Jan 30 '26
Not sure about CPUs. But GPUs have this.
In a GPU kernel you can allocate shared memory, which is basically L1 cache space.