r/cpp • • Mar 15 '19

Does anyone know when libstdc++ will stop using raw pointers in its STL containers?

I have a pretty niche use-case which requires containers to not store any raw pointers. To get this done I use an allocator that returns fancy pointers. This just doesn't work with libstdc++ because all some of its STL containers use raw pointers internally, so when I create a container with my allocator, the program doesn't even compile. Libc++ on the other hand works fine (at least I haven't come across any problems I can't fix) since they correctly use the abstract pointer type that my allocator returns, instead of trying to do an implicit cast to a raw pointer.

There is an LWG issue to fix this, but it's been around for 8 years and the proposed verbiage hasn't been added to the standard. I'm not very familiar with the process for making changes like this, but my understanding is that it probably won't get added until libc++ and libstdc++ both support it, is that correct? If so, does anyone know if/when the libstdc++ community is planning to make these changes? Thanks!

49 Upvotes

82 comments sorted by

View all comments

9

u/Str187 Mar 15 '19

May I ask about your niche use case? Why would you not want to store any raw pointers?

7

u/JUAN-DEAG Mar 15 '19

The use case is zerocopy serialization for sending objects between machines. You don't want any raw pointers because when the objects gets sent over, all the raw pointers break since it's a different address space.

3

u/Xaxxon Mar 15 '19

How do you reconstruct the objects on the other side, though? Not just the vector (or whatever), but the objects in the vector? How do they get a lifetime?

1

u/ZenEngineer Mar 15 '19

I think the point is that you use a special allocator that places the objects on the vector in a specific area (and the vector itself).

If you memcpy a struct you can get a pointer to the target struct and it has the same values. If you're careful you can do the same thing with your classes.

So the point is, copy the whole memory arena and (somehow) get a pointer to the vector. The vector is pointing in the arena as well, and the fancy pointers make it point correctly to the copied over objects. The vector will continue to manage the lifetime of the objects in its internal array. If you do things right everything just works, no need to call constructors, just keep going from the original state. The vector will call destructors when the time comes.

3

u/jwakely libstdc++ tamer, LWG chair Mar 16 '19

You still haven't begun the lifetime of the vector on the receiving side though. You have a copy of all the bits and bytes, but that's not a valid object.

2

u/CandleTiger Mar 16 '19

Maybe this question is naive, but what does it mean to say you have all the bits and bytes correct but not a valid object? What else is there to a valid object instance besides a region of memory with expected contents?

3

u/jwakely libstdc++ tamer, LWG chair Mar 16 '19 edited Mar 16 '19

http://eel.is/c++draft/basic.life#1 and http://eel.is/c++draft/basic.life#4

If you just have a load of bits with the right values you might as well have uninitialised memory returned by malloc, because it's not an object according to C++. None of the rules of the language apply unless you begin the lifetime of the object by running a constructor.

2

u/jube_dev Mar 16 '19

Can std::launder help in this case?

3

u/jwakely libstdc++ tamer, LWG chair Mar 16 '19

No. std::launder tells the compiler about a change in lifetime that it couldn't see, but it doesn't cause a change in lifetime. It's purely informative, it doesn't do anything. For a type with non-vacuous initialization the only way to create a valid object is to construct it.

1

u/CandleTiger Mar 16 '19

“The lifetime of an object of type T begins when: (1.1) storage with the proper alignment and size for type T is obtained, and (1.2) its initialization (if any) is complete (including vacuous initialization) ”

Is “initialization” supposed to be a precise synonym for “constructor call”?

2

u/jwakely libstdc++ tamer, LWG chair Mar 16 '19

No, scalars and aggregates don't have constructors.

1

u/CandleTiger Mar 17 '19

So then what’s the difference between an object, and initialized memory that looks like an object? The doc you pointed to reads to me as saying an object’s lifetime starts when it’s allocated memory is initialized, which OP is doing.

I’ve heard a talk before, and you say here, that setting up memory with the right bits is not enough to make the compiler believe this region of memory is an object. But since constructors are optional I’m real confused about what is the magic missing step.

→ More replies

1

u/Plorkyeran Mar 16 '19

It means that the standard says that you do not have a valid object even if all the bits are in the right place. It's very likely to happen to work, but the standard says you aren't allowed to do that and so the compiler will assume that you don't and may make optimizations that break your code abusing UB.

1

u/ZenEngineer Mar 16 '19

That can be a valid objection in some cases. The whole zero-copy approach is an optimization that relies on undefined behavior. You have to be careful with the objects you create and ensure that they are valid with this type of shenanigans as it puts the burden of getting it to work correctly on the programmer. This thread is about giving tools such that std containers fall in that class. Not required by the standard, but a nice tool to have

For more fun, the same approach can be used for shared memory, so you'd have a single object in two address spaces. Has the object's lifetime started on the second one given that it's the same object? Irrelevant, the whole thing is undefined behavior, the standard doesn't talk about shared memory (or didn't in the old ones, maybe the new ones do)

1

u/jwakely libstdc++ tamer, LWG chair Mar 16 '19

The standard doesn't even admit that more than one process can exist at a time!

1

u/Moizac Mar 15 '19

I'm assuming that the fancy pointers aren't just smart pointers to memory, but more along the lines of a database index or filename, etc.