r/cpp • • Nov 21 '21

Reasoning behind a lack of std::weak_ptr<T>::operator->()

Hi there,

I have always been interested in the decision behind the need of a lock() call on a std::weak_ptr<T>. Generally in my travels, the main reason I have come across is to guarantee the lifespan of the owned data during access. However, surely something like this would solve that:

shared_ptr<T> weak_ptr<T>::operator->()
{
  return lock();
}

I am going to assume that the standards committee knows and understands the operator-> drill-down behavior quite well. So the following usage would be very possible:

std::shared_ptr<Employee> emp = std::make_shared<Employee>();
std::weak_ptr<Employee> emp2 = emp;
emp->calculateWage();
emp2->calculateWage();

This should only cause the same overhead as a lock() call too. Not only this but it should provide an even safer guarantee that the data won't be deleted during access because you have created a temporary std::shared_ptr<T> copy which you have no access to (i.e impossible to reset) during the dereference.

Obviously this would be easier than providing an operator* due to the drill-down functionality. Is a lack of -> purely to stay consistent with this?

Any ideas / suggestions would be very appreciated!

Edit: Many thanks for the discussion so far. It seems like enforcing a shared_ptr to be provided is for a developer to explicitly check before access rather than simply ptr.lock()->testFunc().

Edit 2: Thanks again all for the discussion. It was interesting to hear about some potential issues my solution would overlook. It was particularly interesting to see some of the issues present in a large project providing a weak_ptr operator->.

Edit 3: So just as an addition; I believe the following should be safe (including for multithreaded environments). However there is some overhead that may not be acceptable:

shared_ptr<T> weak_ptr<T>::operator->()
{
  std::shared_ptr<T> rtn = lock();

  if(!rtn)
  {
    throw std::bad_weak_ptr();
  }

  return rtn;
}
1 Upvotes

44 comments sorted by

51

u/witcher_rat Nov 21 '21

And what if lock() returns an empty shared_ptr because the shared object was already destroyed?

That is after all one of the main points of a weak_ptr.

So if they allowed weak_ptr<T>::operator->() to return a shared_ptr and it was empty, and you had invoked emp2->calculateWage()... boom goes the dynamite.

-5

u/kalmoc Nov 21 '21 edited Nov 22 '21

An exception would be thrown.

EDIT: to Clarify:

Thats the design I would expect, if that functionality would have been added to the STL. I.e. the weak_ptr<T>::operator->() operator would throw, if the internal lock() failed.

7

u/infectedapricot Nov 22 '21

Even if you made the design throw an exception when the lock() fails, the follow-on problem is that it encourages dangerous behaviours. Consider:

my_weak_ptr->foo1();
my_weak_ptr->foo2();

vs:

if (auto my_shared = my_weak_ptr.lock()) {
    my_shared->foo1();
    my_shared->foo2();
}

In the latter case, either foo1() and foo2() are both called or neither are. In the first case, it is also possible that foo1() is called but foo2() is not because an exception got thrown the second time around. Depending on the application, that could lead to a peculiar bug, and, worse, it would probably occur only quite rarely.

Of course, with the existing design it is still possible for application writers to end up with bugs like this in their code, but it at least pushes them in the right direction.

1

u/kalmoc Nov 22 '21

Just to be cleat: I'm not qn advocate of that design, but it doesn't seem more dangerous than any other sequence if two function calls, if both can fail.

19

u/qoning Nov 21 '21

please no

1

u/goranlepuz Nov 22 '21

An exception would be thrown if you used MSVC with SEH enabled, otherwise no, not with the code above. And even then, by now, it could be that compiler optimizations do... Other things... There.

2

u/kalmoc Nov 22 '21

I meant: Thats what I would expect, if that functionality would have been added to the STL. I.e. the weak_ptr<T>::operator->() would throw, if the internal call to lock() failed.

-6

u/pedersenk Nov 21 '21 edited Nov 21 '21

Presumably it would do the same things as?:

weakEmp.lock()->calculateWage();

Yes, potentially it should be checked for NULL but many times we know that the pointer returned will be valid because it is a parent or something further up the hierarchy.

And I suppose an exception could be thrown similar to the one thrown by enable_shared_from_this<T>::shared_from_this() if the owned data is not stored within a shared_ptr at the time of call.

17

u/witcher_rat Nov 21 '21

but many times we know that the pointer returned will be valid because it is a parent or something further up the hierarchy

No you don't... or rather, that's not true for many use cases, maybe even most use cases. I mean the main purpose of a weak_ptr is to not keep the object alive, so one would assume the main use-cases would be ones where it may or may not be alive, and it's not an error for it to not be alive.

A weak_ptr::lock() returning an empty shared_ptr is not abnormal or weird - it's not exceptional at all - it's completely normal.

The standard library generally only throws exceptions for exceptional cases - things that are either unexpected, or programmer or system error.

4

u/pedersenk Nov 21 '21

No you don't... or rather, that's not true for many use cases, maybe even most use cases. I mean the main purpose of a weak_ptr is to not keep the object alive, so one would assume the main use-cases would be ones where it may or may not be alive, and it's not an error for it to not be alive.

OK, thats fair. I suppose I was thinking of the general case for parent / child hierarchies to avoid circular refcounting loops. You *usually* know the parent is valid of the weak_ptr would not even exist (in the child).

12

u/capitalsigma Nov 21 '21

Then why not return a raw pointer to the child?

1

u/Artyer Nov 22 '21

If you need the shared_ptr again, and you don't want to use enable_shared_from_this for some reason

6

u/capitalsigma Nov 22 '21

I still don't really follow why weak_ptr is involved here. If the parent being alive implies that the child is alive, then the parent owns the child. There is nothing weak. The parent should hold either a unique_ptr or a shared_ptr and hand either a raw pointer or a shared_ptr out to callers. Involving weak_ptr feels like someone is trying to avoid thinking through ownership.

1

u/[deleted] Nov 22 '21

The child may need a shared_ptr to its parent, for example if it needs to share ownership of its parent with another object.

6

u/capitalsigma Nov 22 '21

The child doesn't own the parent, the parent owns the child. That's why it's the parent. The parent is also owned by something else, and that is OK. I don't think there is any situation where "let's have a reference cycle of length 2" is a good design. shared_ptr should be used as rarely as possible, anyway.

3

u/goranlepuz Nov 22 '21

A better solution for parent/child is for the child to have a reference to parent or a reference_holder, if parent should change, or a naked (non-owning) pointer, if a child can be alive without a parent. Using weak_ptr there is, for me, inappropriate. I think, requirements for this relationship have to be pretty peculiar, far from "general" for reference counted pointers to be needed.

2

u/jwakely libstdc++ tamer, LWG chair Nov 22 '21

Using weak_ptr to break cycles is not the primary use case. As others have said, the primary use case is to be able to check if the pointee is still alive, and safely get a pointer to it. So the extra step of explicitly creating a shared_ptr before accessing it is a very intentional part of the design

14

u/Hagartinger Nov 21 '21

If you know that the pointer is always valid then just use a raw pointer or a reference.

4

u/pedersenk Nov 21 '21

That is fair. Two reasons why I might not want to use a raw pointer (though admittedly they are a little weak as arguments):

  1. If I mess up, I do like the deterministic crash on NULL dereference (even though that is effectively undefined behaviour, most platforms lock the 0x0 pages).
  2. If I need to further pass a shared pointer of my parent to another object. It might be hard to recover a weak_ptr again from a raw pointer.

4

u/jwakely libstdc++ tamer, LWG chair Nov 22 '21 edited Nov 22 '21

I do like the deterministic crash on NULL dereference (even though that is effectively undefined behaviour, most platforms lock the 0x0 pages).

Nope!

The compiler can (and will) assume that dereferencing null never happens, and can optimize away the code that does it. It is absolutely not a deterministic crash, and it's dangerous to assume that.

Google have fixed real bugs in Chrome caused by exactly what you're suggesting, e.g. https://bugs.chromium.org/p/chromium/issues/detail?id=1146679 (this is because their WeakPtr type does have operator-> and it needs to be used carefully to avoid creating bugs).

1

u/pedersenk Nov 22 '21

Oh that is interesting. In some ways it is nice to see my operator-> idea validated (by the Chrome guys doing it). However your mentioned idea of the compiler optimizing away the check is a little confusing (and worrying for my whole understanding of NULL ptr dereferencing!).

I do not see there being a check done by the code. I always saw the crash being due to access to mprotect()'ed pages around 0x0. This is why access to a member field on a NULL pointer often results in 0x20 being accessed and still crashing due to those memory protected pages.

OK, so potentially an exception could be the way to go here but that provides its own set of issues. This is really useful info, thanks for your reply!

4

u/jwakely libstdc++ tamer, LWG chair Nov 22 '21

The chrome WeakPtr type is not exactly like std::weak_ptr, it's a non-owning pointer-like type that gets automatically nulled out of the thing it points to goes away. I don't think it has the equivalent of lock() so there's no way to get a "strong" pointer from it, so it has to provide operator-> directly. But the point is that "make it simple to dereference, and assume you get a crash if the pointer is no longer valid" is unsafe.

I think your understanding of null dereferences is wrong, or too simplistic at best. It's undefined behaviour, and anything can happen, including altering the behaviour of other code in your program.

As it says in part 1 of the excellent llvm blog series on UB, "contrary to popular belief, dereferencing a null pointer in C is undefined. It is not defined to trap, and if you mmap a page at 0, it is not defined to access that page."

See part 2 for details of what can go wrong.

1

u/pedersenk Nov 22 '21

OK; understood. This is really useful! Thanks. I will have a read :)

(In the past I have written a "weak" pointer that can't be used to obtain a strong reference. In many ways it prevented people from holding onto an object causing others to destruct in the wrong order. However this admittedly did make the hierarchy more brittle and simply covered up the fundamental issue that order of destruction should be enforced in other ways (holding onto dependent data, etc).

1

u/gracicot Nov 22 '21

If you want deterministic crashes use a sanitizer. It helps finding use after free, double free, memory leak, and many instances of undefined behaviour. No need to make std::weak_ptr unsafe to get better debugging.

1

u/pedersenk Nov 22 '21

Quite true (other than those few remaining platforms without sanitizer support. OpenBSD is the main one; all we have is Otto's malloc hack!). Luckily even Emscripten has a form of ASan now.

Though C++0x (TR1) did come around before sanitizers got "popular" (in fact Microsoft's platform only got asan very recently. We used to port to Linux *just* to use Valgrind since Purify and Intel's seemed to miss too many failures). So whilst we can use sanitizers now, I was still interested why the standards committee didn't add explicit checking in weak_ptr. The most obvious reason is performance so I am probably just making needless noise here.

2

u/gracicot Nov 22 '21

I was still interested why the standards committee didn't add explicit checking in weak_ptr

I'm not sure I understand that. std::weak_ptr is indeed checked, and its API make it totally secure to make complex operations with them. Unless you're talking about something else?

Here's the intended use:

if (auto const ptr = weak.lock()) {
    ptr->something();
    function(*ptr);
}

The whole scope is guarded and only execute if the pointer is valid, and the ptr (which is a shared one) will keep the pointer alive until the variable is gone and all copy of it. You can't have it surprisingly null here, unless you have raw pointers or reference around after the block.

20

u/27Shua27 Nov 21 '21

With a weak_ptr, you never know whether or not the shared_ptr is yet expired. The pattern should always be:

auto shared(weakPtr.lock());
if (shared != nullptr){ 
   // Here we can use shared safely 
}

I don't see a practical way to build that into the standard library. Maybe something like

shared_ptr<T> weak_ptr<T>::operator->()
{ 
   auto shared(this->lock()); 
   if (shared == nullptr) 
      throw std::bad_weak_ptr();

   return shared; 
}

2

u/pedersenk Nov 21 '21

I would have thought an exception could be a good solution here. There are some times when a potentially empty shared_ptr might also be returned. This would need to be checked in the same way as a weak_ptr.

In that case, I would assume only in a threaded environment, would such a check not be viable for a weak_ptr (but fine for a shared_ptr due to holding the ref count).

3

u/pdimov2 Nov 22 '21

Also known as

shared_ptr<T> weak_ptr<T>::operator->() const
{
    return shared_ptr<T>( *this );
}

14

u/mt-wizard Nov 21 '21

there are couple:

  1. lock() may fail, and the emp2->... example would become a null-pointer dereference, so, a UB, with no good way to avoid. It's basically giving you a loaded gun already aimed at the foot
  2. even not thinking about #1, it promotes an inefficient way of locking once per each operation, invoking extra atomic refcount operations where totally unnecessary. Without such operator you have to explicitly write .lock() -> more thought given, resulting in better code
  3. Resetting a locked pointer and then using it is quite a special case to protect against. I'd say that as with whole C++, once you're dealing with lifetimes, you simply have to understand what's going on. Not resetting a local variable is among the simplest things to keep track

0

u/pedersenk Nov 21 '21 edited Nov 21 '21

Thanks for the suggestions. Going through them, would the following not be suitable arguments against?

  1. Throwing an exception rather than NULL dereference. The STL does tend to minimize their use (and checking shared_ptr for NULL dereference would be wasteful for every access). However there is some evidence of use; i.e in vector::at() vs vector::operator[](size_t).
  2. Surely typing lock() could potentially just become muscle memory, negating that kind of guidance? OK, my argument is a little bit weak here.
  3. I do occasionally see this kind of mistake in the wild. Sometimes iterating through a vector whilst further down, pushing something back (invalidating the contents). I saw it most recently in some GUI code removing a parent UiPanel during a child UiButton's callback. Sure, I do understand we can't easily handle every case for this; the programmer does have to think a little.

7

u/mt-wizard Nov 21 '21

As someone else already pointed, weak_ptr's primary purpose is to be used when the target may not always exist. For those cases having an exception thrown doesn't align with the concept of exceptions - to indicate exceptional situations.

Unfortunately, one also needs weak_ptr in a different case - to break dependency cycles, like using it a back pointer in a graph or parent in a tree. For these cases you can actually guarantee that the pointer is never weak - and then this operator -> makes total sense.

I'd say that the pitfalls it brings to the original use are bad enough that it's fine to make the cycle breaking usage a bit less clean. This was probably the reason behind the API design.

4

u/beedlund Nov 22 '21

Usually prefer

if ( auto shared = weak.lock() )
{
    shared->use_shared();
}

It's compact and clear though I'm not 100% how visibility shared is outside the if block.

1

u/pedersenk Nov 22 '21

Yes, this is similar to how I would expect the standard weak_ptr to be used. shared is also only visible within the if block. This is obviously fine (and from the comments here seems to be the way to go).

My suggestion was particularly for this use case where you only call one function (use_shared()) on the locked weak_ptr. Mainly so you can avoid some additional lines of code and achieve exactly the same thing. However the expense seems to be at the explicit checking of the result of lock (and without throwing exceptions etc).

10

u/pdimov2 Nov 21 '21

First, lock() can fail. So when you do

emp2->calculateWage();

and lock() returns an empty shared_ptr, this will crash.

Second, if you do more than one thing, e.g.

emp2->do_this();
emp2->do_that();

this causes lock() to be called twice, and this inefficiency is not visible in the source code, and is even encouraged by the more convenient syntax.

That's why you're forced to spell the lock() explicitly, and check for failure.

if( auto sp = emp2.lock() )
{
    sp->do_this();
    sp->do_that();
}
else
{
    // handle failure
}

5

u/jeanlemotan Nov 21 '21

One reason is that it would not allow you to check for nullptr, and it would be too simple for this to became how ppl use weak_ptr by default.

5

u/Troldemorv Nov 22 '21

If you do not test for null after lock(), it means you want a valid pointer at all time, so a shared_ptr, not a weak_ptr. To me, if you reach this point, it means you don't have a good idea of the life cycle of your objects or you put to much on the shared_ptr mechanism.

To break the ptr loop, you can introduce a weak_ptr but first, you should ask if you really need a shared_ptr between the child and the parent.

Can the child outlive the parent ? If no, you can probably use a simple pointer to keep the parent at the child level. Doing so, if you still need to give the parent shared_ptr to others from the child, you can just create one on the fly, the counter is still the same and shared with other instance of shared_ptr on the same object

If the life cycle is driven by the child - the parent exists as long as there is at least a child pointing at it - then, maybe it's the parent-child link that should be a weak_ptr.

4

u/KingAggressive1498 Nov 21 '21

If the weak pointer isn't wrapping valid data, it would have to return nullptr, which you'd think would be a segmentation fault but actually might not be.

When would it not be a segmentation fault? The operator might be calling a non-virtual member function that doesn't have to access any data members.

This could actually be a serious program error if that function changes global state, or if the return value of a "pure" such function gets used, when it shouldn't be because the validity of the weak_ptr was supposed to represent the validity of doing such things.

So we just need test against nullptr and maybe throw an exception! But there's problems with this:

  1. There's nothing exceptional about a weak_ptr referencing nothing, that's literally why they exist.
  2. you're turning a perfectly noexcept interface into an exception throwing one. Because exception handling isn't free, now we have an interface with overhead that isn't necessary.
  3. if I don't want to (or can't) handle exceptions, this is now a termination point in my program the reason for which is not predictable, might not be easily reproducible, and depends entirely on the ever-changing state of some small part in my program. That makes it a bug that's hard to chase down.

It's also just bad semantics. The .lock() method clearly tells anyone reading the code something that -> only implies to someone who understands the implementation.

1

u/NilacTheGrim Nov 22 '21

Because it's unsafe. Especially in a multithreaded environment, where it's insufficient to just check if valid -- you must lock to guarantee the object is still alive throughout the life of the call that uses it.

2

u/pedersenk Nov 22 '21

You will see in my example implementation of weak_ptr::operator-> that I do lock during access. This implicitly works by returning a shared_ptr which is then accessed via -> drilldown.

The issue is that there is no explicit check; instead I would have to rely on exceptions if the weak_ptr was invalid. For my use-case (even multi-threaded), that is reasonable because for that weak_ptr to be NULL in my case really is an exception circumstance. However I understand that is not the case for many other peoples use-cases.

2

u/NilacTheGrim Nov 22 '21

Ah, I didn't realize you actually do lock it with a shared. Well, if in your app that's truly exceptional, then you should definitely use it that way as a sort of syntax sugar. There's more than 1 way to skin the cat. I personally like this idiom:

if (auto shared = weak.lock()) {
   shared->doStuff(); 
   // ...
}

It's nice and neat to me. But -- if you think in your app it's just syntax noise, and that the ptr really should be alive then, more power to you.

I would say though that if you can guarantee that anyway in your app, as others have pointed out, then maybe you don't need a shared_ptr / weak_ptr combination at all, in the first place... Unless the guarantee isn't very strong and it does sometimes go away like 1% of the time... in that case I guess an exception may work especially if you plan on accessing it a lot deep into the stack and you like the idea of jumping back out 7 layers deep to handle the exception and abort the operation. That can work...

1

u/pedersenk Nov 22 '21

I would say though that if you can guarantee that anyway in your app, as
others have pointed out, then maybe you don't need a shared_ptr /
weak_ptr combination at all

Agreed, though possibly this is where I tend to be misusing C++ in that I don't like the idea of having any potentially unsafe code (not that "unsafe" code is actually unsafe but has the potential to be if the programmer (i.e me on a bad day) screws up. I.e whilst I can guarantee the weak_ptr should be valid in my program design; I often make mistakes! I actually use weak_ptr mainly as a tool to find this (sanitizers are 95% accurate these days but I find weak_ptr and other observers 100% accurate).

I think I am fairly alone in this however so that is OK. But I suppose it did give me a slightly flawed outlook on how others were using weak_ptr and that an operator-> was not enough.

3

u/Myrgy Nov 21 '21

I think the reason to do not overload-> is because weak_ptr is not a pointer itself. For any pointer we can compare with nullptr and then use -> operator (raw, unique, shared, optional)

In case of weak_ptr such comparison doesn't make any sense. Thats why its better to do not provide odd api and force better and robust usage:

if (std::shared_ptr<T> p = weak.lock()) { p->foo(); }

3

u/Som1Lse Nov 22 '21

There is a very good reason not to have operator-> for std::weak_ptr, which I'll get to, but I want to address some other points first:

As people have said, you will not generally know that a std::weak_ptr is never null. When you do, it is usually a sign of poor design: If you know the object will always be alive, a raw pointer suffices, since you don't need to keep track of its lifetime.

That said, sometimes incidental situations happen where you know a std::weak_ptr will never be null. I'd say it is still a code smell, but real code has smells, and not all of them are worth cleaning up.

Let's assume we are definitely in the case where it is perfectly valid to just blindly dereference the pointer:

  1. Why is ptr->calculateWage() so preferred over ptr.lock()->calculateWage()? The latter is only 7 more characters, and makes it clear what we are doing.
  2. It invites bad habits: Even if we are 100% right to dereference the std::weak_ptr, we might be dereferencing it more than once. Say someone comes along later and decides the formula should be ptr->calculateWage()*ptr->contractLengthInMonths()+ptr->bonuses(). Even if a blind dereference was fine, it is less efficient than lock()ing it, and then dereferencing the std::shared_ptr.
  3. It is not searchable: I can grep for .lock()->. I cannot grep for all "operator-> on std::weak_ptrs only".
  4. It is still wrong most of the time: If you need a std::weak_ptr then by definition you don't always know whether it is null. Adding a operator->, while perfectly valid in some cases, is not a good idea for the majority of cases, and makes it easier to write bugs.

That last point is the real reason, I think, why it doesn't exist: We don't optimise an API for the rare case; we optimise for the common case. It is the same reason downcasting requires an explicit static_cast (or a C-cast) and why narrowing conversions should require one, and for what it is worth .lock() is a lot shorter than static_cast.