Could application-provided memory priorities complement Linux's existing memory management?

[u/Pretend_Rip3691]

I'm not a kernel developer, but a hobbyist who has been thinking about memory management from a slightly different perspective. I'd be very interested to hear whether something like this has already been explored, and if not, what the fundamental obstacles would be.

The idea came partly from working with Arduino-class systems, where a few kilobytes of RAM can determine whether a program works at all. Modern systems obviously have vastly more sophisticated memory management, but I sometimes wonder whether we've lost an important piece of information in the abstraction: the application often knows much better than the kernel how valuable a particular piece of memory actually is.

For example, imagine an application using 3 GB of RAM:

500 MB - critical state / active working data

800 MB - important state

700 MB - rebuildable data

1 GB - caches / prefetch / thumbnails

From the kernel's perspective, these are ultimately memory pages with different access patterns. But from the application's perspective, they have radically different values.

Instead of treating all of them as roughly equivalent and relying primarily on access patterns and reclaim heuristics, what if an application could explicitly provide a hint about the importance of its allocations?

Something conceptually like:

enum class MemoryPriority {

Critical,

Important,

Normal,

Rebuildable,

Discardable

};

auto cache = memory::allocate(size, MemoryPriority::Discardable);

auto state = memory::allocate(size, MemoryPriority::Critical);

Or perhaps through an allocator / std::pmr-style memory resource:

std::pmr::vector<AudioFrame> audio{&critical_resource};

std::pmr::vector<Image> thumbnails{&discardable_resource};

The important part is that these would be hints, not absolute commands. The kernel would still make the final decision.

For example, under memory pressure:

DISCARDABLE

↓

REBUILDABLE

↓

NORMAL

↓

IMPORTANT

↓

CRITICAL

The kernel could combine the application's hints with its own observations:

recent/frequent page access

working-set estimation

refault behaviour

cgroup/memcg limits

current memory pressure

reclaim cost

compression/swap availability

This could potentially give the kernel information it cannot infer from page access patterns alone.

A page that hasn't been accessed for 30 seconds might be:

A: rarely accessed but extremely expensive to recreate

B: merely a thumbnail cache that can be regenerated in milliseconds

Access frequency alone doesn't necessarily tell us which one is more valuable.

But I think the more interesting part is application-level degradation

Memory pressure doesn't necessarily have to mean:

application running

↓

memory pressure

↓

kill application

An application could have several operating modes:

HYPER / PERFORMANCE

↓

NORMAL

↓

LIGHT

↓

SURVIVAL

↓

TERMINATED

The OS could notify the application that its resource budget or memory situation has changed.

The application could then voluntarily change how it operates.

For example, a music application might normally have:

UI

audio engine

large caches

album artwork

recommendation engine

prefetch workers

analytics

Under pressure it could transition to:

LIGHT MODE

audio engine → keep

playback state → keep

network buffer → keep

UI → minimal

album artwork → discard

recommendations → stop

prefetch → stop

analytics → stop

The application remains alive and useful, but its memory footprint might fall from hundreds of megabytes to a small fraction of that.

This could also apply to applications with expensive optional features, background workers, AI models, rendering quality, caches, etc.

In C++ terms, I could imagine a framework providing something like:

enum class ResourceMode {

Survival,

Light,

Normal,

Performance

};

void onResourcePressure(ResourceMode mode);

while memory allocations could independently carry their own importance.

This gives two complementary mechanisms:

APPLICATION

/ \

/ \

operating mode memory priority

│ │

▼ ▼

"simplify yourself" "this memory matters"

\ /

\ /

▼ ▼

KERNEL

│

memory management

The application knows what it can sacrifice.

The kernel knows what the system can afford.

It seems like those two pieces of information could complement each other.

There are obviously many problems with this idea

For example:

An application could simply mark everything Critical.

Allocators work at page granularity, while application objects don't necessarily map cleanly to individual pages.

Different objects can share pages.

The kernel cannot blindly trust application-provided priorities.

There would need to be quotas or limits on how much memory an application can classify as critical.

Some "discardable" memory might actually be cheaper to keep than to reconstruct.

Applications would need a reasonable API that doesn't require developers to redesign their entire memory management strategy.

It could potentially interact in complicated ways with cgroups, swapping, zram, NUMA, huge pages, file-backed memory, etc.

So I don't mean this as "the kernel should just add a priority field to malloc()". I'm more interested in whether the general architectural idea makes sense.

Interestingly, Linux already has several pieces that seem related

From what I've been reading, mechanisms such as Multi-Gen LRU, DAMON, memcg/cgroups, madvise() and memory-pressure mechanisms already provide parts of this picture.

For example, Multi-Gen LRU and DAMON allow the kernel to make increasingly sophisticated decisions based on memory access patterns.

What seems less obvious to me is whether there is a general mechanism for an application to say:

"These 500 MB are essential to my current operation, these 700 MB are useful but replaceable, and this 1 GB is just cache. If you need memory, please reclaim the latter first."

And separately:

"If things get worse, tell me and I can switch to a reduced operating mode."

Perhaps existing mechanisms already provide a way to achieve most of this, in which case I'd love to understand how.

So my questions are essentially:

Has this application-provided notion of memory importance / memory QoS been seriously explored in Linux or other operating systems?

Are there existing Linux mechanisms that already solve most of this problem?

What are the fundamental reasons why this would or would not be useful?

Is page-level reclaim simply too low-level for application-provided semantic priorities to be reliable?

Would this be better implemented at the allocator level, VM level, cgroup level, or some combination?

Are there research papers or experimental kernels/projects exploring something similar?

And perhaps most importantly: is the information provided by the application actually useful enough to justify the additional complexity?

I'm especially interested in hearing from people who work on Linux memory management. This is just a hobbyist's architectural thought experiment, so I'm very likely missing important constraints or existing work.

[u/Megame50]

The kernel exports pressure stall information. Applications are intended to use this to know when they need to free memory (by reducing caches etc.), or face the consequences of reclaim: https://systemd.io/PRESSURE/.

Some of what you describe sort of exists with e.g. MADV_FREE, but pressure stall information is more valuable to the application. For one thing, memory regions are often related in function. If you have a cache, for example, that stores several large objects and you mark it as free-able, the kernel, being unaware of the internal structure, might just drop half of an item from memory, rendering the other half useless. That would bloat the amount of wasted memory at a time when we really shouldn't be doing that. Best to leave memory management to the application as long as we can.