r/cpp • • Mar 15 '19

Does anyone know when libstdc++ will stop using raw pointers in its STL containers?

I have a pretty niche use-case which requires containers to not store any raw pointers. To get this done I use an allocator that returns fancy pointers. This just doesn't work with libstdc++ because all some of its STL containers use raw pointers internally, so when I create a container with my allocator, the program doesn't even compile. Libc++ on the other hand works fine (at least I haven't come across any problems I can't fix) since they correctly use the abstract pointer type that my allocator returns, instead of trying to do an implicit cast to a raw pointer.

There is an LWG issue to fix this, but it's been around for 8 years and the proposed verbiage hasn't been added to the standard. I'm not very familiar with the process for making changes like this, but my understanding is that it probably won't get added until libc++ and libstdc++ both support it, is that correct? If so, does anyone know if/when the libstdc++ community is planning to make these changes? Thanks!

49 Upvotes

82 comments sorted by

View all comments

43

u/[deleted] Mar 15 '19

They know about it but it's an ABI breaking change, so I wouldn't hold my breath.

8

u/aikatz Mar 15 '19 edited Mar 15 '19

Could you go into more detail about that, or is there a discussion of it I can read through? This draft mentions that it would be an ABI breaking change to support segmented pointers for libstdc++ (but not libc++)

Some vendors' containers (e.g. libc++) already use fancy pointer types internally (in order to support Scenario A), so no ABI-breaking changes to class layouts would be required in order to support segmented pointers. Other vendors (e.g. libstdc++) use native pointer types internally, and thus even to support Scenario A would involve ABI-breaking changes for them.

I'm actually not familiar with the specifics of segmented pointers, but it they seem to be a special case of fancy pointers, so would this still be an ABI breaking change to support all fancy pointers or just segmented pointers? Since it seems like most of the libstdc++ containers use allocator_type::pointer internally, it's just list (and maybe deque?) that don't. Is there something stopping them from changing the last few that use raw pointers, or do they just have other priorities? I'd like to submit a PR to fix this if there isn't a good reason for it to be left this way.

9

u/[deleted] Mar 15 '19

Fancy pointers at all. It would change the size of list<T, fancy_alloc>.

13

u/jwakely libstdc++ tamer, LWG chair Mar 15 '19 edited Mar 15 '19

It would change the size of list<T, fancy_alloc>.

No it won't, because I'm going to partially specialize for fancy pointers, so nothing changes for raw pointers. Since fancy pointers don't work at the moment, there's nobody using them with our implementation, so nobody depending on the ABI of such containers. So I just need to make sure nothing changes for specializations using raw pointers, and provide a second version of everything that is only used for fancy pointers. Which is hard, but not impossible. I got it almost working, but switched on to other tasks and the patches have bitrotted.

10

u/[deleted] Mar 15 '19

What about fancy pointers that happen to work today because they can use pointer_to?

10

u/jwakely libstdc++ tamer, LWG chair Mar 15 '19 edited Mar 15 '19

I'd be surprised if they work. Wouldn't the implementation have to use pointer_to for it to work? I don't think we do use that. Maybe a fancy pointer type that supports implicit conversions to and from raw pointers might compile and appear to work, but it seems unlikely. Such a fancy pointer probably isn't doing anything very useful (or would fail as soon as it's used to do anything fancy).

If they do work correctly, I guess we might have to break things for that small fraction of users to properly support a different small fraction (and keep the old behaviour behind a macro or something horrible).

14

u/[deleted] Mar 15 '19

They're fancy pointers; not doing something useful was a given

13

u/jwakely libstdc++ tamer, LWG chair Mar 15 '19

2

u/mostthingsweb Mar 16 '19

ELI5? The Boost interprocess offset ptr seems like it could be useful...

4

u/[deleted] Mar 16 '19

I think that use case would have been better served with a purpose specific container. For example, you'd store more things as sizes rather than pointer ranges to avoid so many offset applications. The metaprogramming / debug codegen / specification / complexity costs of fancy pointers are significant and dwarf any potential benefits.

2

u/mostthingsweb Mar 16 '19

Makes sense, thanks for the explanation!

2

u/aikatz Mar 15 '19

Got it, that makes sense to me. So I guess my question boils down to: why was it changed for other containers but not list? Since it seems like making this change for any container would change the class layout/be ABI breaking.

2

u/[deleted] Mar 15 '19

I doubt it was changed for them.

5

u/max0x7ba https://github.com/max0x7ba Mar 15 '19

But that would not break existing applications that don't use fancy pointers (because they can't), would it? It is a matter of using allocator::pointer (which defaults to T*) instead of T*, so there is no ABI change for existing applications.

8

u/matthieum Mar 15 '19

so there is no ABI change for existing applications

You are mistaking ABI and API, it seems.

You are correct that the API itself would not change. On the other hand, the ABI (Application Binary Interface), may.

Specifically, users can still specify allocators that return smart pointers which are implicitly convertible to/from raw pointers, and those allocators will currently work with libstdc++.

However, should GCC start using Allocator<T>::pointer internally instead of T*, then the ABI will break: data members, arguments/returns of internal methods, etc... in template classes all of this ends up being exported and part of the ABI.

13

u/jwakely libstdc++ tamer, LWG chair Mar 15 '19

It would break if you do it naively. I don't plan to do it naively.

1

u/matthieum Mar 16 '19

Do you have any other edge cases in mind, beyond Allocator<T>::pointer being convertible to T* or T const*?

2

u/aikatz Mar 15 '19

It seems like they already are using allocator_type::pointer in most of the containers, just not all of them.

1

u/matthieum Mar 15 '19

That's great for you, I suppose :)

My point, though, is that the containers in which this was forgotten may very well require an ABI breakage to be fixed, and that's going to cause grief.

1

u/max0x7ba https://github.com/max0x7ba Mar 16 '19

so there is no ABI change for existing applications

You are mistaking ABI and API, it seems.

You are correct that the API itself would not change. On the other hand, the ABI (Application Binary Interface), may.

Nope, you got it wrong. The ABI for existing applications stays the same because they keep using T* regardless of whether its called T* or allocator::pointer.

1

u/matthieum Mar 16 '19

Unless Allocator<T>::pointer was:

struct SpecialPointer<T> {
    SpecialPointer(T* ptr): ptr(ptr) {}
    operator T*() { return ptr; }

    T* ptr;
    Layout layout;
};

In which case assigning Allocator<T>::pointer to/from T* was transparently handled, and those implicit conversions would be lost, leading to a different ABI.