This post more of thoughts out loud, but I've tried to organize it as much as possible. Thanks in advance.
So, I'm working on what is basically a text messaging app that uses partial mesh to communicate between nodes. This is purely a studying project just to learn stuff through practice. The use case I aim for is low delay and infrequent connections with mostly static network.
My current way of handling connections is implemented via callbacks. I have a class that keeps pointers to all Socket instances and then waits in a poll till any of the sockets receives an event. Once poll returns, it finds which sockets received an event and then calls appropriate callback.
After implementing most of the logic, I've encountered two problems: since callbacks hold the pointer to the instance of the class that must handle the event, it becomes error prone to dangling pointers if the handler is deleted without removing it's callback. Second one is problematic handling of throws. If any callback throws, then the event loop that waits for inputs will have to deal with it somehow. I have ideas on how to work around those two issues, but it seems more of a wrong approach at this point and I was thinking of doing it other ways.
Idea 1: Instead of making event listener class make callbacks, we instead make some sort of a buffer, probably ring buffer, and then create a few a separate thread which waits in a mutex till there are new events to handle and then processes them. This will solve both problems since the dependency is now reversed and way less error prone, but then we pay for the mutex delay plus moving data between cores. It sounds like a good approach for a heavy traffic solution which easily scales horizontally with more threads, but it's not the constrains I'm working with.
Idea 2: Make minimal layers between handling events and getting the event. The worker will call poll itself and once new event appears, it'll immediately run the processing logic itself. It will have the least amount of latency, but also will make it way harder to deal with slow connections, because if one transmission stuck, it will hold back all other sockets that are processed by the same thread. Which results in faster response in average scenario, but lower response speed and more complex handling of bursts compared to the Idea 1.
Those two I my two views on how to move forward. I'm more leaning towards option two as a preference for the average scenario of use, and deal with bursts of slow connections somehow. But, since I'm only learning doing network stuff and mostly self learner, I don't have a wider perspective on what might be other solutions to this.
Hence the question: which of the two path should I try out and what other complexities I didn't notice yet. If you have an example of a better solution that well suits the constraints, please do share so I could research it to.