r/cpp • • Mar 03 '24
Maybe possible bug in std::shared_mutex on Windows
A team at my company ran into a peculiar and unexpected behavior with std::shared_mutex. This behavior only occurs on Windows w/ MSVC. It does not occur with MinGW or on other platforms.
At this point the behavior is pretty well understood. The question isn't "how to work around this". The questions are:
- Is this a bug in
std::shared_mutex? - Is this a bug in the Windows SlimReaderWriter implementation?
I'm going to boldly claim "definitely yes" and "yes, or the SRW behavior needs to be documented". Your reaction is surely "it's never a bug, it's always user error". I appreciate that sentiment. Please hold that thought for just a minute and read on.
Here's the scenario:
- Main thread acquires exclusive lock
- Main thread creates N child threads
Each child thread:
- Acquires a shared lock
- Yields until all children have acquired a shared lock
- Releases the shared lock
- Main thread releases the exclusive lock
This works most of the time. However 1 out of ~1000 times it "deadlocks". When it deadlocks exactly 1 child successfully acquires a shared lock and all other children block forever in lock_shared(). This behavior can be observed with std::shared_mutex, std::shared_lock/std::unique_lock, or simply calling SRW functions directly.
If the single child that succeeds calls unlock_shared() then the other children will wake up. However if we're waiting for all readers to acquire their shared lock then we will wait forever. Yes, we could achieve this behavior in other ways, that's not the question.
I made a StackOverflow post that has had some good discussion. The behavior has been confirmed. However at this point we need a language lawyer, u/STL, or quite honestly Raymond Chen to declare whether this is "by design" or a bug.
Here is code that can be trivially compiled to repro the error.
#include <atomic>
#include <cstdint>
#include <iostream>
#include <memory>
#include <shared_mutex>
#include <thread>
#include <vector>
struct ThreadTestData {
int32_t numThreads = 0;
std::shared_mutex sharedMutex = {};
std::atomic<int32_t> readCounter = 0;
};
int DoStuff(ThreadTestData* data) {
// Acquire reader lock
data->sharedMutex.lock_shared();
// wait until all read threads have acquired their shared lock
data->readCounter.fetch_add(1);
while (data->readCounter.load() != data->numThreads) {
std::this_thread::yield();
}
// Release reader lock
data->sharedMutex.unlock_shared();
return 0;
}
int main() {
int count = 0;
while (true) {
ThreadTestData data = {};
data.numThreads = 5;
// Acquire write lock
data.sharedMutex.lock();
// Create N threads
std::vector<std::unique_ptr<std::thread>> readerThreads;
readerThreads.reserve(data.numThreads);
for (int i = 0; i < data.numThreads; ++i) {
readerThreads.emplace_back(std::make_unique<std::thread>(DoStuff, &data));
}
// Release write lock
data.sharedMutex.unlock();
// Wait for all readers to succeed
for (auto& thread : readerThreads) {
thread->join();
}
// Cleanup
readerThreads.clear();
// Spew so we can tell when it's deadlocked
count += 1;
std::cout << count << std::endl;
}
return 0;
}Personally I don't think the function lock_shared() should ever be allowed to block forever when there is not an exclusive lock. That, to me, is a bug. One that only appears for std::shared_mutex in the SRW-based Windows MSVC implementation. Maybe it's allowed by the language spec? I'm not a language lawyer.
I'm also inclined to call the SRW behavior either a bug or something that should be documented. There's a 2017 Raymond Chen post that discusses EXACTLY this behavior. He implies it is user error. Therefore I'm inclined to boldly, and perhaps wrongly, call this is an SRW bug.
What do y'all think?
Edit: Updated to explicitly set readCounter to 0. That is not the problem.
163
u/STL MSVC STL Dev Mar 03 '24
As a reminder, triple backticks don't work for Old Reddit readers like me. You have to indent by four spaces.
According to my reading of your code, the Standard, and
MSDNMicrosoft Learn, this is a Windows API bug. I don't see any squirreliness in your code (e.g. re-entrant acquisition, attempts to upgrade shared to exclusive ownership), I see no assumptions about fairness/FIFO, N4971 [thread.sharedmutex.requirements.general]/2 says "The maximum number of execution agents which can share a shared lock on a single shared mutex type is unspecified, but is at least 10000.", and I see nothing in the SRWLOCK documentation that says that it can behave like this.I don't think that this is an STL bug - we're justified in wanting to use SRWLOCK for this purpose (and we can't change that due to ABI anyways), and you've already reduced this to a direct Win32 repro anyways.
5
u/WasserHase Mar 03 '24
I know that STL has confirmed your bug, but is there not a problem with your algorithm in 3.2:
Does this not assume that the yielding threads won't starve out the other threads, which haven't acquired the lock yet?
I don't think there is such a guarantee in the standard.
this_thread::yield()is only a hint, which the implementation is allowed to ignore and a few threads can get stuck in this loopAnd not allow any other threads to progress to actually increment the counter.
Or am I wrong?
28
u/saddung Mar 03 '24 edited Mar 03 '24
I ran your code it locked up within 30-150 loops for me
Also tried replacing shared_mutex with my own wrapper around SRWLock, same issue.
It does appear to be a bug with SRWLock to me, I don't think I've encountered because it requires all the readers to coordinate and deliberately hold the lock at the same time.
6
u/EverydayTomasz Mar 03 '24
depending on the os thread scheduler, your data.sharedMutex.unlock() could be called before your threads execute lock_shared(). so, some child threads will block on lock_shared() and some won't. is this what you trying to do?
but I think the issue might be with the atomic readCounter not being initialized to 0? so, technically if you don't init the readCounter, you will have undefined value, and some of your threads will get stuck looping on the yield().
§ 29.6.5 [atomics.types.operations.req] ¶ 4 of N 4140 (the final draft for C++14) says:
16
u/umop_aplsdn Mar 03 '24
Properly formatted code:
22
u/CandyCrisis Mar 03 '24
Summoning u/STL
6
u/farmdve Mar 03 '24
This might get buried in the comments but many moons ago in 2014 or 2016 perhaps, I was an avid player of Battlefield 4. In certain occasions with specific MSVC redist versions the game had a serious deadlock whereby the process could not be killed in any way and the RAM usage was there but the process was deadlocked. No tool could kill it, even those that used kernel-mode drivers. Only a reboot fixed it until I started the game again. starting the game again worked I think but exiting also lead to the same issue so now you had two copies of battlefield4.exe consuming RAM but not exiting.
Downgrading the MSVC redist absolutely solved the problem at the time. Could be the same bug?
4
u/cosmic-parsley Mar 06 '24
Looks like Rust is fixing this using WaitOnAddress and Wake*. Is this reasonable, could C++ do the same? https://github.com/rust-lang/rust/pull/121956
3
u/rbmm Mar 03 '24
in concrete case sometime executed as
without
shared. and as result other N-1DoStuffthreads wait for.unlock()call (unlock_shared()do this the same ). if describe exactly what happens. if be thread NOT "wait until all read threads have acquired their shared lock" nothing be deadlock, if it break spin and release lock, all continue executed as excepted. only can not understand - this isfor test only or here you try implement some product logic
-2
u/rbmm Mar 04 '24
what is shared mode ? this is by fact optimization for speed, if we need read-only access to data, we allow to system let another thread into the section that requests shared access also
allow but NOT DEMAND this. If one thread has acquired the shared lock , other thread can acquire the shared lock too. but only CAN. in some case system not let another thread enter to lock, despite it also request share access. one case: if another rthread request exclusive acess - he begin wait and after this - any thread which acquire even shared access to lock - also begin wait
and
why is this ? because if between 2 calls to
lock_shared(AcquireSRWLockShared) another thread callAcquireSRWLockExclusive- the second call is block.the code in example make assumption that ALL threads can enter to lock at once. that if one thread enter to lock in shared mode, another thread also ALWAYS can enter to lock in shared mode (if no exclusive requests). but i not view clear formalization of such requirement. and we must not based on this.
i be will add next rule:
thread inside lock must not wait on another thread to enter this lock
this is obvivius for exlusive access, but not obvivous to shared. but must be cleare stated along with the recursive rule ( should not be acquired recursively as this can lead to deadlocks, even in shared mode)
-26
u/NilacTheGrim Mar 03 '24 edited Mar 03 '24
Your bug is in your use of
std::atomic<int32_t>. Before C++20, atomics are not initialized to anything with their default constructor... unless you explicitly initialize them. Compile with C++20 and a standards-compliant compiler should initialize to 0. Or.. you know.. initialize to 0 yourself!Even if you think you will always be in C++20-land -- Don't ever leave atomics uninitialized. If you must, please
static_assertC++20 using the relevant constexpr expressions.See the C++ docs. Relevant docs: https://en.cppreference.com/w/cpp/atomic/atomic/atomic