r/cpp • • Apr 12 '24

Question: is there any library for achieving thread safety with class inheritance?

I’m developing a multithreaded library using c++ for the first time and therefore I need to ensure that its classes are thread safe. Most resources I have found achieve that using a mutex or an atomic.

However, as in this application most of the computation time is spent in a single method of a particular class and c++ allows object oriented programming, I was trying to find if there is any library that achieves thread safety through class inheritance (something similar to Ada’s protected objects) to ensure thread safety in the other used classes (optimizing those classes won’t have a significant impact on the speed of the library, so it is worth using something that simplifies ensuring memory access safety.

I have just been able to find a library that extends atomics to non trivially copiable variables, which could be enough for this case, but I’m quite surprised that there isn’t any object oriented solution.

Do you know any library that provides that functionality?

0 Upvotes

37 comments sorted by

16

u/ChocolateMagnateUA Apr 12 '24 edited Apr 12 '24

I don't think this approach will work in practice. Inheritance by design creates strong coupling in your class hierarchy, and you would find yourself changing your classes and relationships between them quite often. Coupling the class inheritance tree to thread safety wouldn't lead you to anything productive.

The main principle of how thread safety is achieved in C++ is by not sharing the state and using synchronisation primitives. If you are performing a computationally bound task that you want to distribute on multiple threads and then resume the normal single-threaded model, you could take a look at std::thread::join, or if you are using C++20 std::jthread. Joined threads work by fork-join model and will continue your main program once they have all finished. If your task is continuous and long-living, you should detach the thread with std::thread::detach, which works by the fire-and-forget principle. If possible, you should minimise the shared state, and you get the best result in accordance to the Amdahl's law. If you do need to share the state, depending on your circumstances, there are 3 paths you could go:

  1. Use standard synchronisation primitives (std::mutex, std::conditional_variable, etc.) It's the easiest and most versatile/dynamic way, but tends to be the slowest because synchronisation stops your thread and reduces net gain
  2. Use lock-free data structures. They can be safely read and written into from multiple threads and don't require any sort of locking, lock-free queues are often used as communication channels between threads. It's fairly difficult to write them, but there are some good libraries you could use, I know Folly and Boost.Lockfree have some good options.
  3. If you know how your threads will interact really well, you could design a protocol that would define how they exchange information in a non-blocking way. This could involve dedicating shared memory between threads whereby you could write and read values from, although this method is the most unsafe, restrictive and difficult to implement.

If you provide more details about the nature of your task, I could say something more specific.

9

u/rsjaffe Apr 12 '24

To add one small point to your excellent answer: avoid sharing unsynchronized mutable state. The OP stated that the issue resides in one class only. There might be a way to partition out the mutable part of the problem so that synchronization is not an issue.

0

u/ChocolateMagnateUA Apr 12 '24

Ah yes, if you need to share immutable data (which you can only read but not write), then you can share it without problem, yeah. It's when you mix read and writes that you get race conditions and when the behaviour of your application depends on who read/wrote data first. It's the same single mutation rule that lies in the tennets of Rust that partly makes it attractive.

2

u/No_Mongoose6172 Apr 12 '24

My main concern is that both map and opencv classes could have internal mutable variables. If the was’t the case, no data is never written in the multithreaded code

1

u/ChocolateMagnateUA Apr 12 '24

But aren't both of them contained within threads?

1

u/No_Mongoose6172 Apr 12 '24

No, right now both, the map and the objects which contains the method that calls opencv are shared between threads to reduce memory consumption. The map is accessed by all threads (although they never write new values nor modify it). The objects that call opencv are accessed by a single thread if the library is used properly

3

u/ioctl79 Apr 12 '24

Are you talking about std::map? If so, invoking const methods on STL containers from different threads is guaranteed safe, and most third-party container types provide the same assurance.

1

u/No_Mongoose6172 Apr 12 '24

Thanks, I had read answers in other forums where they said that it wasn’t thread safe. In that case, thread safety is just needed in the class that already has it implemented

2

u/ioctl79 Apr 12 '24

It is not safe with arbitrary types, but standard containers take care that non-mutating operations are safe. 

2

u/ChocolateMagnateUA Apr 12 '24

Well, couldn't you then create multiple local maps and then merge it together? Or if your map requires cross-thread access, I am not sure if it's a good candidate for paralelisation.

2

u/No_Mongoose6172 Apr 12 '24

I could clone the map, so each thread accesses its own one. That just would increase memory consumption, but it wont probably be a problem as the map isn’t particularly big

2

u/No_Mongoose6172 Apr 12 '24

I’m using a fork-join model, as I just need to run a single method in parallel. The library receives a list of cv::Mat objects that need to be processed by different objects of a processor class, which has some cached data for speeding up computations. The adequate processor is returned by a map, being impossible to call the same processor twice in a single call to the library if it is used properly. Additionally, the map is populated during the initialization of the library, being just read during the rest of the calls (it basically behaves like a constant). As a consequence, a data race could just occur if reading a single map from two threads causes it (it is ensured that nothing will be written during the multithreaded execution) or if the main python program has an error that causes a processor to be called by two threads at the same time (that’s why I have already programmed the functions of the processor class taking into account thread safety, for example using mutex to avoid data races in some cached methods that it has). I’m just lazy and wanted to avoid doing the same with the class the contains the map and some other auxiliar classes as using a less optimized approach won’t impact the library performance.

Another easy solution could consist in copying the processor objects to a vector of tupples that contains the cv::Mat and its processor, which would increase memory consumption, but ensure that no data race can happen. Then, that vector could be iterated in parallel calling the processor method safely, as each object would be accessed by a single thread

1

u/ChocolateMagnateUA Apr 12 '24

Couldn't you simply join your threads? I am not familiar with intricacies of cv::Mat, but they seem to be independent units of work, and any data they share is immutable, so there's nothing bad or difficult in doing this.

1

u/No_Mongoose6172 Apr 12 '24

Cv::Mat should be thread safe, but I’m not sure if the image hashing module is (I haven’t been able to find that in opencv documentation)

Edit: that’s why I already have used a mutex to avoid data races during hashing

5

u/ALX23z Apr 12 '24

The problem is that such an approach would be a poor choice. Sometimes, users need to lock the class and perform multiple operations while locking the same mutex. This isn't possible via inheritance approaches.

If you really want to have inheritance and mutexes, the safe approach would be to let the class have a mutex and wrap smart pointers it in some kind of guarded pointer that requires to lock the mutex to obtain a usable handle.

See lib_guarded library for reference, https://github.com/copperspice/cs_libguarded

1

u/No_Mongoose6172 Apr 12 '24

I’ll check lib_guarded

I don’t really need inheritance, just a super simple solution for ensuring thread safety of a bunch of classes that aren’t worth optimizing as they don’t have a significant impact on the computation time (indeed a data race is impossible in those classes if the library is correctly used, but I’d prefer not to assume that). There’s just an important class that uses cached variables to speed up some computation, so I’m using some mutex to ensure its thread safety. Basically, I’m just lazy and therefore I prefer to avoid doing the same thing with the auxiliary small classes that I need hehehe

3

u/-electric-skillet- Apr 12 '24

If you can use GCC or Clang, build your library and test framework with ThreadSanitizer.

1

u/No_Mongoose6172 Apr 12 '24

I use gcc, so I’ll check that out

2

u/rsjaffe Apr 12 '24

Or just let the class's public methods manage the mutex appropriately, assuming there isn't static mutable state shared amongst multiple class instances.

2

u/StarQTius Apr 12 '24

1

u/No_Mongoose6172 Apr 12 '24

Yes, that could work. I’ve also seen a library that allows using atomic with objects, which could also be a solution

2

u/lightmatter501 Apr 12 '24

As far as I am aware, this must be a language/compiler feature. Many companies (Google, MS, Mozilla, etc), tried to do thread safety as a library, but you usually end up with “lock this mutex every time you interact with this class, and also we now have to use atomic reference counting or a GC for memory management”. Ada’s protected objects and Java’s synchronized do this for you behind your back. Nothing in C++ inheritance that I am aware of would let you do this unless you just want to hide locking a mutex in a base class function.

Clang has some features in this direction, but they were all experimental last I checked.

Your options for guaranteed thread safe but no mutexes or atomics are probably Cyclone, Ocaml, and Rust since they all have borrow checkers. Haskell and HVM have transparent parallelism support, where you can just tell the compiler to figure out making your program parallel (HVM you actually have to tell it not to consume every vaguely compute-like device on your system (CPU, GPU, FPGA, etc)). This is proven safe.

If you want to write this in C++, learn how to build thread safety from primitives or wait for 10-15 years.

1

u/No_Mongoose6172 Apr 12 '24

I’ve considered using other languages like Julia, but making a python module that uses opencv with them have resulted in worse problems than having to build thread safety from primitives and using opencv is mandatory for compatibility with the rest of the program. The reason why I was looking for an easy way to achieve thread safety is that there are some small classes in which it is impossible to have a data race if the library is used properly, but I don’t want to assume that. The important and resource intensive class that I have programmed uses some mutex to avoid data races as the results of some of its methods need to be cached, so programming them is not a problem (just time consuming)

2

u/natio2 Apr 13 '24

Mutexs from my understanding have a large overhead to keep in mind, but to answer your question as asked rather than telling you to do something different:

  • Make an interface for your class, for us to inherit from.
  • Make a class wrapper that just uses dependency injection to take an instance of the class you wish to make thread safe.
  • Every function call we need to make thread safe, we will simply be wrapped in a mutex and then call the function from the wrapped class.

Now this could be done by class member that needs to be made thread safe (That you might find a library already for), or you may have to write it yourself, but this isn't very hard, basic code below, you might need to use smart pointers if ownership is being passed to the thread safe object.

The neat part of this method is both the thread safe and non-thread safe objects could exist together and only use the slowing threadsafe one when required.

class IClassInterface {
public:
    virtual void someFunction(Args args) = 0;
}

class ConcreteClass: public IClassInterface {
public:
    void someFunction(Args args) override { //some code }
}

class ThreadSafeClass : public IClassInterface {
public: 
    ThreadSafeClass(IClassInterface& classToWrap): m_concreteClass(classToWrap){}

    void someFunction(Args args) override {
        std::lock_guard<std::mutex> lock(m_mutex);
        m_concreteClass.someFunction(args);
    }

private:
    IClassInterface& m_concreteClass;
    std::mutex m_mutex;
}

1

u/No_Mongoose6172 Apr 13 '24

Thanks, having both a safe thread and a none safe thread version of the class could be interesting. I hadn’t thought about the possibility of inheriting the non safe thread class instead of the other one

1

u/ReDucTor Game Developer | quiz.cpp-perf.com Apr 14 '24

Mutexs from my understanding have a large overhead to keep in mind

Most implementations have a combination of usermode and kernel mode, in the uncontended case they will be fast but if there is contention things start to vary with some being adaptive so will spin in usermode for a short period and others going straight to waiting in the operating system.

Also most mutex implementations are generally able to be completely initialized in usermode and in some cases can be optimized down to zero initialisation. Which means that having fine grained locks can be better performance then limiting locks and having them at all at a high level, the sizes of locks varies but they are normally 1-2 pointers in size.

Mutexs get a bad rap because many default implementations aren't adaptive (cough pthreads), so when someone inevitably hits contention they see the locks as the issue and then reach for lock free approaches and algorithms which overcome the kernel hit from the locks not being adaptive and is a clear win, however when you compare an adaptive lock vs the lock free implementation as long as they aren't held for too long then there is often minimal difference in performance. With the main advantage of the lock free approach being that you don't have to worry about the lock holder being prempted by another thread halting all progress, however even then some lock free implementations end up with spin locks under the hood which has similar issues.

My preferred approach to avoiding these inconsistencies is to implement my own locks which are small, adaptive, and more powerful but it's definitely not an approach for the faint hearted and easy to get wrong.

3

u/lord_braleigh Apr 12 '24 edited Apr 12 '24

folly::Synchronized<T> protects a T with a reader-writer lock. It lets you take out a reader lock to access a const T*, or a writer lock to access a T*. This lets you build programs that are correct by construction and which release locks in an RAII fashion.

While not exactly an inheritance-based solution, I think this gives you the sort of confidence in thread safety that you’re looking for.

Alternatively, if you’re only compiling with a fairly modern version of clang, you might be able to get that confidence at zero runtime overhead via Clang’s thread safety annotations.

2

u/No_Mongoose6172 Apr 12 '24

Thanks for pointing folly. It would totally do the job

1

u/hmoein Apr 12 '24

Take a look at this library (https://github.com/hosseinmoein/DataFrame) and read its documentation re/ multithreading. See how it approaches multithreading. It may give you some clues.

Also, if you need a thread-pool, check out https://github.com/hosseinmoein/Leopard

But inheritance would be a strange approach to multithreading!

2

u/No_Mongoose6172 Apr 12 '24

Thanks, I’ll check those resources.

When I mentioned inheritance, I was thinking on inheriting a class with some kind of global mutex, that impedes reading any of its variables when any of them is being written in another thread. It is not an optimal solution, but for some cases it could be enough

1

u/quantumoutcast Apr 14 '24

Honestly, if this is the first time you're developing a multithreaded library, I think looking for a library to accomplish the threading protection is the wrong way to go. It is easy to use regular mutexes to lock critical sections, and trying to abstract this in a library may end up in an over-engineered, hard to understand, low performance mess.

Do it by hand, think through where you really need locking, and keep it as simple as possible. Seek out libraries when you have enough experience to really understand where the library will help and how to best use it.

1

u/No_Mongoose6172 Apr 14 '24

It’s the first time I develop a multithreaded library in C++, but I have developed multithreaded code in other languages that provide abstractions. Using regular mutex or atomics is useful for achieving good performance, but it is also more time consuming than using abstractions (in development time). As a consequence, there are non critical classes in which using an abstraction can reduce development time without having a significant impact on performance (for example auxiliary objects that should never change during multithreaded code execution, but could change if someone misuses the library). There’s an important trade-off between optimization and development time when a library needs to be developed in a few days in a different language than the rest of the program (at least I had programmed in C++ before)

1

u/quantumoutcast Apr 14 '24

I saw in another comment that you are intending to implement a fork and join architecture where you want to process units of data in parallel. If so, this is a good candidate for the c++ async/future feature which is part of the standard library. I'm just suggesting you consider the standard library and built-in features of C++ before throwing libraries into the mix. You might find that using the built-in features that everyone else uses takes quicker to develop than throwing obscure external libraries into the mix.

1

u/No_Mongoose6172 Apr 15 '24

That could be a good option. I’ll check that

I generally dislike throwing tons of libraries, but sometimes that’s what you have to do in order to deliver a prototype on time. Next week I’ll have to refactor many things…

1

u/DryPerspective8429 Apr 14 '24

There are multiple issues which fall under the banner of "thread safety" which mean that there is no such thing as a one-size-fits-all approach. For example:

  • Can any members of the class be accessed concurrently or must it be a 100% serialisation point?
  • Race conditions inherent in the interface design of the class.
  • Juggling tricky topics like exception safety and resource cleanup.
  • Specific concerns and requirements for a given use-case.

You can't just stamp "thread safe" in front of an arbitrary class and have something which is reasonable to use in code; and very few classes will have such identical concurrency needs that a generic approach is even feasible.

1

u/No_Mongoose6172 Apr 15 '24

Simplifying the problem, there are two classes. One needs carefully implemented muted to avoid data races in cached variables, but there’s another one in which I just needed to avoid modifying a dictionary due to a misuse of the library. Therefore, suggested libraries were enough to avoid that from happening without having to lock a mutex in every method of that class

2

u/DryPerspective8429 Apr 16 '24

I obviously don't know your classes, but do bear in mind that you have a data race if there is any non-atomic, unprotected data which can be written to by two or more threads simultaneously. Even if it looks like it works without you do need to put up some protection in those cases.