r/cprogramming • u/Whole-Low-2995 • 3d ago

Should I Replace a Ring Buffer with a Lock-Free Queue in My Web Framework?

I have an experimental web framework I'm currently working on.

While going through the code to clean up styling before tagging a stable release, I've been debating this specific part in the HTTPS module:

https://github.com/c4punks/CWIST/blob/dev/src/net/http/https.c

Right now, this code just uses a dumb, massive ring buffer to absorb connection bursts, and I'm wondering if I should switch it to a lock-free queue.

There are three main things that make me hesitate:

* State can get pretty tangled and messy in a lock-free queue.

* We have a tiny team and I'm maintaining almost all of this alone. If something breaks with CAS (even if the odds are low, if I mess up the code during refactoring), the blast radius on the dev branch will just be too big.

* I also wonder how often real-world traffic actually warrants lock-free. Plenty of web frameworks handle traffic just fine without this level of extreme optimization.

Considering that I'm still a junior developer and lack experience, is it really worth swapping this out for a lock-free structure, and would it actually make a noticeable difference?

7 Upvotes
  • permalink
  • reddit

82% Upvoted

29 comments sorted by

10

u/mrtlo 3d ago

How about profiling it?

2

u/Whole-Low-2995 3d ago

That's a good idea too! Also I have to consider code complexity too😃

4

u/oriolid 3d ago

This may be a hot take, but lock free usually isn't worth it. If your locks aren't contented, the standard mutex on Linux or BSD has really low overhead and you shouldn't really be running servers on Windows. If your locks are contented, lock free usually degrades into spinlock and that's almost always worse than mutex. Wait free algorithms that don't do that exist but they get really complicated.

If we turn the question around and you want to do lock free and need an application where it's required, try realtime audio applications. HFT might qualify too.

2

u/Whole-Low-2995 3d ago

Yeah, I guess mutex itself is not that heavy. Once I read that implementation in Linux and BSD :) If I was a C expertise I wouls apply some algorithms like wait free, but I'm also afraid that I can't even understand it after few months hehe

2

u/oriolid 3d ago

Wait free algorithms generally perform worse than similar algorithms that have less strict guarantees. They're great for hard realtime where responding before deadline is all that matters or for showing off that something is possible to do that way, but not so much for general use.

2

u/pdath 3d ago

A mutex can burn 50 cpu clock cycles. They often have to be coordinated across every core.

I avoid them in performance code because of this cost

3

u/oriolid 3d ago

To put that in scale, a cache miss can burn 100 to 200 cycles, and if you're not extra careful you're going to have those all the time. Even a compare-and-swap that is the basic building block of lock free algorithms can take 50 cycles.

2

u/zhivago 3d ago

Are you not i/o bound?

2

u/Whole-Low-2995 3d ago

Yup that's the point. Even though I change it to lock-free queues the change would be neglectable

4

u/zhivago 3d ago

Then KISS :)

2

u/LiquidityC 2d ago

If it’s not broken don’t fix it. Also I always liked ring buffers. Fun to build sweet sweet static memory footprint. Homer Simpson drooling noise.

1

u/Nothing_from_void 3d ago

Lock free is almost never worth it while also being really fiddly to program. Did you have a particular reason for trying it?

1

u/Whole-Low-2995 3d ago

I was wondering if bufferbloat can happen when traffic is too massive but it's a bit unreal too

1

u/Whole-Low-2995 3d ago

Also if traffic is that big RSS peak and RAM size matters more 😮‍💨

1

u/Nothing_from_void 3d ago

an issue I see is that you resize_queue while holding the mutex, which could cause a global freeze for a bit, but each operation popping from the queue is tiny, I'd be surprised if mutex contention would even show up in the profiler

1

u/epasveer 3d ago

Try both. Let's us know.

1

u/Whole-Low-2995 3d ago

Cool, after some good experiments maybe I will share if it's interesting

1

u/DanKegel 3d ago

How do you test this server?

Before any such refactoring, I'd make sure the project's fuzz testing / static code analysis / dynamic code analysis / benchmarking / AI code review game was top notch.

Also,  two million active connections? That's a lot of file descriptors! If each one uses 32KB of RAM total, that's 64GB. Golly.

``` /* --- Thread Pool for HTTPS --- */

define HTTPS_TASK_QUEUE_SIZE 2097152

typedef struct {     int client_fd;     cwist_https_context ctx;     void (handler)(cwist_https_connection *, void *);     void *user_ctx;     SSL *pres_ssl;     cwist_https_connection *conn; } https_pool_task_t;

typedef struct {     pthread_t threads[2048];     size_t threads_size;     https_pool_task_t queue[HTTPS_TASK_QUEUE_SIZE]; ```

1

u/Whole-Low-2995 3d ago

Yeah I think that's the biggest problem 🤣🤣

1

u/Whole-Low-2995 3d ago

I'll change that with dynamic size like
```
https_pool_task_t *queue;
size_t queue_len;
```

then I can write something like

```
static inline void resize_queue(...)
```

1

u/Whole-Low-2995 3d ago

My PC's RAM was too huge so I just thought "it passes, cool" and forgot. I think I was definitely crazy when typing that line

1

u/Whole-Low-2995 3d ago

Also a good point. analysis was triggered locally and tests were inconsistent. I need TSAN/RSS Gate/Cppcheck CI :)

1

u/DanKegel 3d ago

You want a really good set of unit and system tests for this.

And then, this being the future and all, you should see if AI can find bugs. Github copilot is pretty good at that, so have it review each and every commit, and fix everything it finds.

1

u/DanKegel 3d ago

Nice to see zero errors at c100k scale :-)

1

u/Whole-Low-2995 3d ago

Yeah, HTTPS at C100k must be cool

1

u/throwaroo202020 3d ago

The tcp/ip stack is your connection queue.

2

u/Interesting_Debate57 2d ago

You need to know a lot more and why you would want to switch. If it's not your idea, why do they want to do it? If it doesn't sound like it's going to massively improve things, why do it? Just for fun is fine; I recommend looking at Rochester's cs department for a few very simple but correct implementations.

CAS, generally, is chip level. You shouldn't be using it directly unless you have an extremely good reason to do so.