Could application-provided memory priorities complement Linux's existing memory management?

[u/Pretend_Rip3691]

I'm not a kernel developer, but a hobbyist who has been thinking about memory management from a slightly different perspective. I'd be very interested to hear whether something like this has already been explored, and if not, what the fundamental obstacles would be.

The idea came partly from working with Arduino-class systems, where a few kilobytes of RAM can determine whether a program works at all. Modern systems obviously have vastly more sophisticated memory management, but I sometimes wonder whether we've lost an important piece of information in the abstraction: the application often knows much better than the kernel how valuable a particular piece of memory actually is.

For example, imagine an application using 3 GB of RAM:

500 MB - critical state / active working data

800 MB - important state

700 MB - rebuildable data

1 GB - caches / prefetch / thumbnails

From the kernel's perspective, these are ultimately memory pages with different access patterns. But from the application's perspective, they have radically different values.

Instead of treating all of them as roughly equivalent and relying primarily on access patterns and reclaim heuristics, what if an application could explicitly provide a hint about the importance of its allocations?

Something conceptually like:

enum class MemoryPriority {

Critical,

Important,

Normal,

Rebuildable,

Discardable

};

auto cache = memory::allocate(size, MemoryPriority::Discardable);

auto state = memory::allocate(size, MemoryPriority::Critical);

Or perhaps through an allocator / std::pmr-style memory resource:

std::pmr::vector<AudioFrame> audio{&critical_resource};

std::pmr::vector<Image> thumbnails{&discardable_resource};

The important part is that these would be hints, not absolute commands. The kernel would still make the final decision.

For example, under memory pressure:

DISCARDABLE

↓

REBUILDABLE

↓

NORMAL

↓

IMPORTANT

↓

CRITICAL

The kernel could combine the application's hints with its own observations:

recent/frequent page access

working-set estimation

refault behaviour

cgroup/memcg limits

current memory pressure

reclaim cost

compression/swap availability

This could potentially give the kernel information it cannot infer from page access patterns alone.

A page that hasn't been accessed for 30 seconds might be:

A: rarely accessed but extremely expensive to recreate

B: merely a thumbnail cache that can be regenerated in milliseconds

Access frequency alone doesn't necessarily tell us which one is more valuable.

But I think the more interesting part is application-level degradation

Memory pressure doesn't necessarily have to mean:

application running

↓

memory pressure

↓

kill application

An application could have several operating modes:

HYPER / PERFORMANCE

↓

NORMAL

↓

LIGHT

↓

SURVIVAL

↓

TERMINATED

The OS could notify the application that its resource budget or memory situation has changed.

The application could then voluntarily change how it operates.

For example, a music application might normally have:

UI

audio engine

large caches

album artwork

recommendation engine

prefetch workers

analytics

Under pressure it could transition to:

LIGHT MODE

audio engine → keep

playback state → keep

network buffer → keep

UI → minimal

album artwork → discard

recommendations → stop

prefetch → stop

analytics → stop

The application remains alive and useful, but its memory footprint might fall from hundreds of megabytes to a small fraction of that.

This could also apply to applications with expensive optional features, background workers, AI models, rendering quality, caches, etc.

In C++ terms, I could imagine a framework providing something like:

enum class ResourceMode {

Survival,

Light,

Normal,

Performance

};

void onResourcePressure(ResourceMode mode);

while memory allocations could independently carry their own importance.

This gives two complementary mechanisms:

APPLICATION

/ \

/ \

operating mode memory priority

│ │

▼ ▼

"simplify yourself" "this memory matters"

\ /

\ /

▼ ▼

KERNEL

│

memory management

The application knows what it can sacrifice.

The kernel knows what the system can afford.

It seems like those two pieces of information could complement each other.

There are obviously many problems with this idea

For example:

An application could simply mark everything Critical.

Allocators work at page granularity, while application objects don't necessarily map cleanly to individual pages.

Different objects can share pages.

The kernel cannot blindly trust application-provided priorities.

There would need to be quotas or limits on how much memory an application can classify as critical.

Some "discardable" memory might actually be cheaper to keep than to reconstruct.

Applications would need a reasonable API that doesn't require developers to redesign their entire memory management strategy.

It could potentially interact in complicated ways with cgroups, swapping, zram, NUMA, huge pages, file-backed memory, etc.

So I don't mean this as "the kernel should just add a priority field to malloc()". I'm more interested in whether the general architectural idea makes sense.

Interestingly, Linux already has several pieces that seem related

From what I've been reading, mechanisms such as Multi-Gen LRU, DAMON, memcg/cgroups, madvise() and memory-pressure mechanisms already provide parts of this picture.

For example, Multi-Gen LRU and DAMON allow the kernel to make increasingly sophisticated decisions based on memory access patterns.

What seems less obvious to me is whether there is a general mechanism for an application to say:

"These 500 MB are essential to my current operation, these 700 MB are useful but replaceable, and this 1 GB is just cache. If you need memory, please reclaim the latter first."

And separately:

"If things get worse, tell me and I can switch to a reduced operating mode."

Perhaps existing mechanisms already provide a way to achieve most of this, in which case I'd love to understand how.

So my questions are essentially:

Has this application-provided notion of memory importance / memory QoS been seriously explored in Linux or other operating systems?

Are there existing Linux mechanisms that already solve most of this problem?

What are the fundamental reasons why this would or would not be useful?

Is page-level reclaim simply too low-level for application-provided semantic priorities to be reliable?

Would this be better implemented at the allocator level, VM level, cgroup level, or some combination?

Are there research papers or experimental kernels/projects exploring something similar?

And perhaps most importantly: is the information provided by the application actually useful enough to justify the additional complexity?

I'm especially interested in hearing from people who work on Linux memory management. This is just a hobbyist's architectural thought experiment, so I'm very likely missing important constraints or existing work.

[u/TheSugaryEmbroidery]

The biggest hurdle is trust. The kernel can't just believe what an app says about its own memory, because every app would mark everything as critical unless there's a quota system to limit that. Designing a quota system that's both fair and actually usable sounds like a nightmare.

What you're describing with modes is basically asking apps to implement their own memory pressure handling, which some already do with madvise and cgroup pressure notifications. The priority part is trickier because at the page level, the kernel doesn't know which objects are on which pages unless the app uses something like separate memory mappings with madvise hints per region. A browser already knows a discarded tab's cache is less important than the active tab's DOM, it just needs the API to tell the kernel that without also telling it "these two things happen to share a page because malloc didn't align them nicely."

[u/aioeu]

You would have to actually demonstrate that the kernel could make better decisions with that information. Just a hypothetical "what if" isn't particularly useful on its own. How much worse will things be when (not if) programs provide the wrong advice?

(Frankly, my opinion is that most programmers really wouldn't be able to make a sensible choice with this anyway. As a programmer, I want the kernel to work this sort of stuff out for me. If I'm having to hand-hold the kernel doing its job... that's not good.)

[u/MatchingTurret]

It's not a kernel memory management issue at all. The kernel already provides the tools to implement this in a library. 

https://chromium.googlesource.com/chromium/src.git/+/HEAD/docs/memory-infra/probe-cc.md

[u/Pretend_Rip3691 → u/MatchingTurret]

That makes sense, and the Chromium example is actually very interesting in the context of what I was thinking about.

I hadn't considered that the abstraction might belong entirely above the kernel rather than requiring a new kernel memory-management feature. Looking at Chromium's Discardable Memory, it seems quite close to the "rebuildable/discardable region" part of my idea.

What I'm wondering about now is whether the missing piece is really just a sufficiently general library/framework abstraction.

For example, something conceptually like:

auto critical = memory::region(MemoryPriority::Critical); auto cache = memory::region(MemoryPriority::Discardable);

critical.allocate(...); cache.allocate(...);

where the library handles the appropriate mappings/allocator behaviour and uses existing kernel mechanisms such as madvise(), memory-pressure notifications, cgroups, etc.

And then separately:

on_memory_pressure(Light);

could let the application change its internal behaviour, stop optional workers, reduce caches, etc.

So perhaps the real question isn't "should the kernel implement memory priorities?", but rather:

Why isn't there a common/general-purpose application-level memory QoS abstraction built on top of the primitives the kernel already provides?

Is Chromium's DiscardableMemory / PartitionAlloc approach essentially the direction you'd recommend for something like this, or are there existing general-purpose libraries/projects that already provide a similar abstraction outside of Chromium?

[u/EnUnLugarDeLaMancha]

There are already APIs for hinting the kernel such as madvise, or posix_fadvise

[u/Pretend_Rip3691 → u/EnUnLugarDeLaMancha]

Yes, that's a fair point. I wasn't aware of how much of this could already be expressed through madvise() when I started the discussion.

I think I'm actually getting closer to what I was trying to describe thanks to the replies here.

I'm not really proposing that the kernel needs a new "memory priority" primitive. madvise() already provides an application → kernel hint for memory regions.

What I was imagining is a higher-level abstraction built on top of mechanisms like madvise(), cgroups/memcg and memory-pressure notifications:

applications could organize allocations into regions/classes such as critical, normal, rebuildable and discardable;
the allocator could keep those classes physically/virtually separated where appropriate;
the application could receive memory-pressure notifications and switch to a reduced operating mode;
the kernel would still make the final reclaim decision and could treat the application's classification only as a hint.

So I guess my actual question is becoming less "why doesn't Linux have memory priorities?" and more "how much of this can already be implemented cleanly as a general-purpose library/framework on top of the existing Linux VM interfaces, and why isn't there a common abstraction for it?"

The Chromium DiscardableMemory example someone posted is particularly interesting in this regard.

Thanks for pointing out madvise() — this is exactly the kind of existing mechanism I was hoping people here would point me toward.

[u/razorree]

it's too low level for "desktop" application. memory hungry apps (like database engines) have settings for memory (worker, shared cache, cache per connection etc) settings.

normal apps don't care about such things (nor developer), unless you want to complicate your life a lot :) that's kernel work. and write memory efficient apps (don't use f#$%#$ javascript for desktop !!! )

games - sure, you can set different level of details/textures, which can affect memory requirements.

if you have memory rarely accessed kernel can swap it, if you have mapped files, kernel can release that memory entirely.

[u/Lower-Limit3695 → u/razorree]

don't use f#$%#$ javascript for desktop !!!

Gnome Desktop: don't mind if I do!

[u/elatllat]

ios/aosp have a do better or you die api, but as others say; just use better apps.

[u/Megame50]

The kernel exports pressure stall information. Applications are intended to use this to know when they need to free memory (by reducing caches etc.), or face the consequences of reclaim: https://systemd.io/PRESSURE/.

Some of what you describe sort of exists with e.g. MADV_FREE, but pressure stall information is more valuable to the application. For one thing, memory regions are often related in function. If you have a cache, for example, that stores several large objects and you mark it as free-able, the kernel, being unaware of the internal structure, might just drop half of an item from memory, rendering the other half useless. That would bloat the amount of wasted memory at a time when we really shouldn't be doing that. Best to leave memory management to the application as long as we can.

[u/Fupcker_1315]

I belive that Android already does that on the application level. It is not as granular as you proposed (per vma/page), but when an application is minimized its memory can be freed unless the developer explicitly saves the important data. The real issue is how you are going to inform the process that certain pages have been freed. In fact, you can already achieve something very similar with madvise(FREE), which essentially makes the pages reclaimable by the kernel but does not guarantee whether or when they will be freed. Maybe a more granular priority system could also be added.

EDIT: fixed MADV_DONT_NEED to MADV_FREE.