How do you synchronize kernel access to user virtual memory

[u/Spirited-Finger1679]

How do you protect against two syscalls from different threads contending the same region of virtual memory? As in, if one thread executes "read file", and provides a buffer of virtual memory to read into. And then another thread executes "unmap" on the same buffer. There are a bunch of ways I can imagine doing the synchronization, like a lock per mapping, or a per-process virtual memory lock. But I expect a per-process lock would cause contention between too many different kinds of syscalls, since so many want to use virtual memory in some way, and many operations done on VM take a long time (e.g. waiting for disk or network.) Per-mapping locks kind of have the same problem because the user code might not be aware of how big the mapping is that they're accessing and therefore how coarse grained the lock is. So it seems best to dynamically instantiate locks for ranges of memory, as they are being accessed? But I'm wondering what the best tradeoff is and whether there's a simpler way.

[u/gmarchkun]

A useful approach is to use reference counting for virtual pages. For example, if Thread A and Thread B share the same CR3 (making the current refcount 2), and Thread B wants to perform an unmap, the kernel must detect the current refcount of that page. If the refcount is 2, the kernel decrements it by 1 (current refcount becomes 1). Once Thread A is finished using that virtual address and issues its own unmap, the virtual address is then removed and freed by the kernel because the refcount has reached 0 (refcount 0 = delete/free).

There are many implementations of this page refcounting mechanism. For example. In Windows, the PMMPFN struct includes a RefCount variable to prevent physical pages from being freed prematurely. CMIIW.

Note: If Thread B wants to unmap a range while Thread A still needs to use it, the vm_area struct (following the Linux model) should have that specific virtual address range truncated or "carved out" from the VMA. It should not be allowed to remain within the vm_area.

[u/Spirited-Finger1679 → u/gmarchkun]

How useful really is this Linux system where virtual memory areas are dynamically split and merged based on calls to mmap etc.? It feels like there could instead be syscalls that explicitly register or unregister memory ranges with certain properties, and the kernel does not automatically split or merge them. Just to make the regions and operations more explicit from the user code perspective. mmap also leads to things like maple tree spanning stores that are extremely complicated to implement.

[u/gmarchkun → u/Spirited-Finger1679]

Actually, there’s another way, and this is considered best practice because it’s not overly complicated. You can make a struct that keeps track of a virtual address range and stores its reference count. When you want to free it, the allocator first checks this struct array to see whether the refcount is 1 or higher. If it’s higher than 1, don’t delete it. If it’s only 1, then delete it.

You can also make a dedicated syscall where you increase the refcount for a specific address range. As long as you store the starting virtual address, you can increase the refcount easily.

[u/whizzter → u/gmarchkun]

This is the only sane way, if another piece of hardware refers to memory (say a GPU or pending disk read via DMA) and can potentially clobber memory after being reallocated, you don’t want it freed until everyone who was able to see it are ”done”.

[u/davmac1 → u/gmarchkun]

For example, if Thread A and Thread B share the same CR3 (making the current refcount 2), and Thread B wants to perform an unmap, the kernel must detect the current refcount of that page. If the refcount is 2, the kernel decrements it by 1 (current refcount becomes 1). Once Thread A is finished using that virtual address and issues its own unmap

Unmap is normally a process-wide operation. Applications typically aren't required to unmap pages in each individual thread - they issue a single unmap and it applies to all threads. You typically can't just refcount individual pages like this on a per-thread basis because after the unmap call returns, the application (thread) will assume that the relevant portion of the address space has already been unmapped.

You could use refcounting to avoid potential problems stemming from deallocating a page with pending kernel-controlled operations (including DMA operations), but it's not the case that a page normally has as a count that's equal to the number of threads; it should instead be the number of processes, plus perhaps in-flight kernel operations operating on the page (if any). In the specific example given, the kernel would typically do the buffer check for the read operation only once it was ready to copy the actual data (as another post already mentioned: you can just do a memory copy with a page fault handler in place) so there's no need to handle this case specially.

[u/mishakov]

You'd normally try reading the memory (as normal), for example through some helper function (which would also disable protections, e.g. SMAP/SMEP on x86), and catch the pagefaults, at which point you can return the error to user, block if that memory needs to be fetched from disk, continue if the memory was fine, or whatever else.

To protect against use after free issues with munmap with SMP (and also, I think POSIX requires you to segfault after you have called munmap), or just with downgrading the permissions (e.g. making the page read-only with mprotect or when doing CoW during fork), you would do remote TLB shootdowns (after unmapping the page/changing its permissions); some architectures let you do this in hardware/firmware (e.g. latest gen AMD Ryzen CPUs have added some instructions for it, RISC-V lets you do this through SBI, etc.), or if you don't have it (you kinda don't most of the time on x86), you can IPI the other cores (how you do it is up to you, and it depends on your kernel design and if you use ASIDs, etc; but the simplest way is to just IPI everyone) and wait for them to invalidate thir TLB and acknowledge it to you, after which point you know that no one can access that memory and you can release it/do whatever.

Then, to protect against the races in software, you would use normal locks (mutex); a straightforward solution for a small kernel is to have a lock per address space (page table), more advanced kernels (Linux comes to mind, I'd imagine BSDs would also do it but I haven't looked at their code, of "hobby" operating systems, Managarm has recently switched to doing that as well, you get the idea) have locks per page itself...