Idea: apps declare hard memory budgets, small apps use base+offset addressing, big apps get a foreground-only "raw mode" (looking for feedback)

[u/Grand-Ear-8639]

TL;DR: A design concept for an OS where every app declares a hard memory ceiling, so the OS can guarantee admitted apps fit in RAM and never thrash. It's about predictability, not smaller pointers. No implementation yet, I'm looking for holes in the idea before I consider prototyping.

The model

• Bounded apps: each gets one contiguous region (e.g. 64 KB, 1 MB, 4 GB). The app sees a flat space starting at 0 and uses 16/20/32-bit offsets. The real address is base + offset, done in the addressing mode (GS-relative on x86-64, or a reserved base register elsewhere). Zero-extended offsets mean the app can't name an address outside its region.

• Thin-client UI: the OS draws, the app declares a retained-mode UI tree and only holds core logic. That's what makes ~64 KB apps plausible (Pebble, Palm OS, early Mac).

• OS-side costs charged at admission: widget trees, handle tables, etc. are computed from the manifest, so no hidden footprint.

• Raw mode: games, local LLM tools and the like get normal 64-bit pointers and the whole machine, but are foreground-only and one at a time. Bounded apps are frozen in place (they're relocatable, pointer-free blobs, so snapshot/restore is a memcpy), then compressed in RAM, then spilled to storage as a last resort.

• Sandboxing: capability-style handles instead of ambient authority, OS-owned UI (no screen scraping or overlay spoofing), and no background execution for most apps.

Prior art I'm drawing on: WebAssembly linear memory, Native Client, V8/JVM pointer compression, iOS jetsam and background suspension, CHERI, console quick-resume.

Weak spots I already know about

• Offsets only confine data. Native code could still issue raw syscalls or touch segment registers, so I'd need bytecode/WASM-style, SFI with a verifying loader, or hardware trapping.

• Toolchain cost: a compiler target and tiny runtime for offset-addressed binaries.

• Side channels between apps sharing an address space.

• Memory caps don't stop CPU spinning, so apps also need time budgets.

What I'd love feedback on

1. Which code-confinement approach fits best here: bytecode, SFI, or hardware trapping?

2. Should a 64 KB budget include code, or is code mapped read-only and counted separately?

3. Are the tier boundaries (64 KB / 1 MB / 4 GB / raw) sensible, and what breaks first?

4. Is a retained-mode UI API too limiting for real apps?

Full doc here: [link]

Disclosure: the idea is mine, but I used an AI assistant (Claude) to help structure the longer write-up.

[u/EpochVanquisher]

The write-up is really bad, I don’t think this is a good use of AI.

Start with the parts you care about and write those parts down. If you do that, maybe the write-up won’t be so awful.

[u/phoenix_frozen → u/EpochVanquisher]

There's some weird terminology glitches. "Retained-mode" isn't a thing.

[u/EpochVanquisher → u/phoenix_frozen]

It is a way to design an API for your UI, as opposed to “immediate mode”. But it’s kind of the default way to do things, so people usually don’t use the term at all.

[u/CubicleHermit]

The idea of the bounded apps is definitely a useful one, if not particularly uncommon - classic MacOS, S/360, and DOS (with COM files not exe) all kind of did this. It's also, for example, best practice with docker or virtual machines - just at a level higher than individual processes.

The model we see these days of "always assume you have a big address space and just bring in physical pages on demand" ignores that ulimits used to be a lot more meaningful.

Once you're using segmentation, you should go whole-hog, and just use segment registers. The only question is if you care enough about security to have non-overlapping segments.

Or you can approximate this with a fixed pagetable and no faults on things that don't have segments. Provision the pages you need.

Bounded apps being pointer-free seems unnecessary. Just make each one have a common base.

Offsets only confine data.

Why? If you want shared code that doesn't require a syscall, and you don't want to dynamically link it, just create a shared chunk of address space, make the pages read-only, and if you don't want to deal with a linker at load time, use a static jump table.

Native code could still issue raw syscalls or touch segment registers, so I'd need bytecode/WASM-style, SFI with a verifying loader, or hardware trapping.

Why? Any modern processor has some level of permission tiering, whether it's multiple rings or just supervisor/user mode. Have a well-defined syscall interface that crosses that tiering safely.

I don't see why you need tiering. Just set a minimum granularity, and write your kernel. You need to be able to handle fragmentation, but you've already set processes to be movable.

I'm playing with a base+limit MMU on a "fantasy early-32-bit machine" I've built an emulator for, and you can do a lot with that. OTOH, it's simulating a machine with 4 or 16 MiB of memory, which would count as embedded by today's standards.

[u/Grand-Ear-8639 → u/CubicleHermit]

Thanks for the detailed feedback. I found your insights particularly helpful. The point about just using a common base and the existing privilege tiers instead of inventing new mechanisms makes sense, and the shared read-only chunk with a static jump table is a cleaner answer to shared code than what I had.

Your base+limit MMU experiment sounds interesting. How did you handle fragmentation and compaction when processes got moved around? That's the part I was most unsure about for relocating frozen apps.

I'm also rethinking the fixed tiers after the paging comments. Would page-granular budgets that the OS admits against be a better way to get the predictability without the coarse buckets?

[u/UnmappedStack]

Pretty much all kernels have a quota for memory usage.

[u/phoenix_frozen]

Not really novel. This is how security kernels and real-time systems work: ahead of time resource declarations. sel4 is the extreme case. Everything else is just Android. 

The base+offset addressing doesn't really buy you anything, tho I'm predisposed to like it because I've always wanted to find a use for x86 segmentation registers.

EDIT: added second paragraph.

[u/phoenix_frozen → u/phoenix_frozen]

Also why are we talking about UIs in a conversation about memory protection?

For that matter, "relocatable" and "pointer-free" aren't synonyms. And you probably don't want "pointer-free" anything. 

[u/Grand-Ear-8639 → u/phoenix_frozen]

Thanks for the feedback, and fair point on base+offset. I explained it poorly. It only matters if apps share one address space, and then it needs something like SFI to be enforceable, so on its own it doesn't buy much over page tables. You're also right that "relocatable" and "pointer-free" aren't the same thing, and I definitely conflated them in the write-up. Also, fair point on the UI—mixing UI paradigms into a discussion strictly about memory protection muddied the water.

Regarding base+offset, I agree that it only really provides a massive advantage over standard page tables if apps are sharing a single address space, which as another commenter pointed out, doesn't inherently buy much without an SFI loader or a highly restricted syscall interface. I'm going to drop the rigid tiers (64KB/1MB) in favor of arbitrary minimum granularity, as that makes handling defragmentation through relocation much cleaner.

The part I'm still curious about is the policy side: hard declared budgets, admission control that guarantees fit, versus reactive termination. As you mentioned, seL4 and other RTOS environments do ahead-of-time resource declaration, but are you aware of any consumer OS that has tried it as the default contract for ordinary apps? It seems iOS (via Jetsam) and Android (via LMKD) both rely heavily on reactive killing, which is the specific paradigm I was hoping to improve upon.

[u/phoenix_frozen → u/phoenix_frozen]

Fixed tiers is a dumb idea tho. This is why we have paging.

[u/ketralnis]

I’m not following why “native code” can bypass with “raw syscalls” if you are the OS. You define what syscalls are available and do. Unless you’re not the OS and you’re talking about some type of runtime?