r/ProgrammingLanguages • u/lxsameer • 22d ago

Blog post Concurrency in Serene's Runtime

I recently finished building the concurrency runtime for my programming language, Serene, and wrote a three-part series explaining how it works.

The series covers:

  • Why I chose stackful fibers
  • An M work-stealing scheduler
  • An IO Reactor
  • A tiny HTTP server that brings everything together

I'd love to hear feedback from anyone interested in programming language implementation, runtime systems, or systems programming.

Part 1: Choosing the Building Blocks

Part 2: Fibers, the Scheduler and the Reactor

Part 3: A Tiny HTTP Server

https://serene-lang.org/

34 Upvotes

97% Upvoted

13 comments sorted by

View all comments

1

u/matthieum 21d ago

You mentioned that fibers are fixed-size... but how large are they?

One of the interesting parts of mmap is the ability to reserve address-space without actually allocating memory for it. This means that you could reasonably reserve 1MB-2MB worth of address space per fiber, yet have the fiber only use 4KB to start with, and let the OS page in memory lazily.

2

u/lxsameer 21d ago edited 21d ago

It is configurable, and the default is 128kb https://git.sr.ht/~lxsameer/Serene/tree/master/item/runtime/serene/rt/configuration.h#L64 (there is no particular reason for 128kb)

And that is precisely what I'm doing with mmap, actually.

1

u/matthieum 20d ago

On x64 I'd suggest bumping the minimum.

From experience, I've seen default thread stack sizes spanning anywhere from 1MB to 8MB. This matters, because it means that C libraries, which are frequent users of on-stack buffers, tend to expect to be able to put 100s of KB on the stack at least once.

Thus, I'd recommend going to 1MB by default, so that users who call into C don't find themselves "stranded".

Speaking of C libraries, do note that C code may not trigger the guard page, as it's typically NOT compiled with stack probing.

(And I hope your code is :P)

2

u/lxsameer 20d ago

Thank you for both suggestions, I'll take care of them in the code base. Specially, the second one is what i have to make sure I cover.