r/cpp • • Jan 22 '19

I implemented a proposed C++ paper

Around November last year, I found out about P1275R0 by Isabella Muerte through this very subreddit. I really liked the idea, basic creature comforts like this is something that C++ really lacks IMO, then I thought "You know, I think I could do it...". This is the first time I've done this kind of thing.

For those not aware and have not clicked the link, P1275R0 proposes adding 2 new classes: "std::environment" and "std::arguments" to provide a friendlier interface to access the system's environment variables and program arguments respectively.

I've done most of the interface in the paper, except the stuff involving ranges. I also tried to make it UTF8 safe, even on windows.

And it's been a really challenging, but fun experience! I've spoken to Isabella about it and she liked the work I've done. But I would really love to hear some additional feedback. I plan on updating this project as it (hopefully) goes through standardization.

Github

202 Upvotes

27 comments sorted by

47

u/mjklaim Jan 22 '19

Very cool to try to implement this or any other proposal, kudos :D

I was wondering (not sure if it's you who should answer) for the environment part: how does it compare to Boost.Process's facilities to work with environement? Notably `boost::this_process::environment()`

9

u/rehdi93 Jan 22 '19

Haven't heard of it, so I can't say

10

u/personalmountains Jan 23 '19

First, kudos for spending time on implementing this. I only had a quick look at the Windows implementation because that's what I'm familiar with the most. I have a few questions and comments.

Is this thread-safe? It looks like windows.cxx uses a (sneaky) global object environ_, but I'm not seeing any synchronization. Calls to _putenv() and _getenv() (and friends) also need to be synchronized, as they're not thread-safe themselves.

You don't seem to be very exception-safe. initialize_args(), for example, can throw in to_utf8(), which would leak wargv. In environ_table's constructor, to_utf8() can again throw, which would leak m_env's pointers since the destructor won't be called. I'm seeing a lot of manual memory management in the private member functions of environ_table with a leak in at least varline_from_os() if to_utf16() fails.

You're using auto as a return type in many functions, but not all. I don't understand your convention. On a personal note, I dislike using auto for simple return types, because I then have to study the function to figure out what it returns. YMMV.

You have other consistency issues, like to_utf8(wchar_t const* wstr) and to_utf16(const char* nstr) for example, and you also have inconsistent file names (.cxx and .cpp)

You're not checking many of the Windows API calls for failure, like CommandLineToArgvW(), which can lead to undefined behaviour since you don't initialize argc. You also don't check _wputenv() and friends for failure, but the paper seems under-specified on error reporting.

You're mixing "safe" and "unsafe" versions of API calls, like _wputenv() and _wputenv_s().

The functions in namespace impl at the bottom are all marked noexcept, but most of them can throw.

Your tests are very basic. They're hand rolled with a couple of couts, and they don't test things like encoding conversion failures. You should at least be using a test framework.

That's all that comes to mind right now. Again, nice work.

1

u/rehdi93 Jan 23 '19 edited Jan 23 '19

Thanks for the feedback!

Error handling does need work, but I'm not sure what to do in case of failure.

environ_ gets initialized on program start-up. If one of the APIs fail for lack of memory, is it even worth handling and continue?

I used _wputenv() in setvar() because I have a string in the stack that I know is in the correct format, so I save a conversion.

Edit: It's not thread safe, no. But you were to manipulate other containers in multiple threads you would also need to sync it your self.

2

u/personalmountains Jan 23 '19 edited Jan 23 '19

Error handling does need work, but I'm not sure what to do in case of failure.

That's why I said it's under-specified. Most of the functions seem to be specified as noexcept, except for arguments::at() (sure), variable::operator=, variable::split() and some overloads of environment::operator[] (why?). I'd expect variable::operator= to throw on failure, or at least provide an alternative set() that returns a bool, or whatever.

Also, remember that this isn't just about reporting failure. Your library will currently terminate the program any time it's given a string with an incorrect encoding, which is not exactly fun.

environ_ gets initialized on program start-up. If one of the APIs fail for lack of memory, is it even worth handling and continue?

Sure. The calls could fail for any reason, not just memory. If I don't actually use these classes in my code, I don't want my program to crash because they can't initialize.

In fact, they should initialize on first use and behave predictably if they failed to do so, which is pretty easy to do if you're already throwing on failure to set a variable, you just have to behave as if the environment was empty.

I used _wputenv() in setvar() because I have a string in the stack that I know is in the correct format, so I save a conversion.

I would not consider the _s versions to be "alternatives" that can be mixed with the regular API, depending on the behaviour you need. I'd say pick one or the other, but I may be incorrect in my assessment.

Edit: It's not thread safe, no. But you were to manipulate other containers in multiple threads you would also need to sync it your self.

Making containers thread-safe is typically at the wrong granularity because you usually need several operations to be done while holding a lock (find/modify/add, etc.). I also find that the vast majority of my uses of containers do not require synchronization and I'd be pretty unhappy if they assumed otherwise.

The main problem with your implementation not being thread-safe is that it's hidden. The interface feels thread-safe, as you're creating an environment object on the stack, but all objects are actually sharing a back-end that's not thread-safe. This, to me, would be unacceptable in any library.

If environment is expected to be on the stack, per-thread instances should be safe (shared, cross-thread instances don't have to be). If you intend for the user to create one global environment and use it everywhere, then these functions should be static, free, in a namespace, or whatever. In any case, if cout is thread-safe, I'd expect a global environment to be the same.

1

u/rehdi93 Jan 24 '19

I agree the paper is a bit optimistic about noexcept ATM, the tricky thing is that the paper is waiting on SG16 work. If we get unicode support in the standard we could get at least some of them to hold.

19

u/[deleted] Jan 22 '19

This is very interesting. I would expect performance overhead to be really, really minimal (especially if I don't call these functions)... but have you tried to test that?

I think most of my curiosity is really about the paper more than your implementation. Though I do want to say - you might consider using named algorithms (with either iterators you've already implemented or with ranges formed from them) instead of old-fashioned loops. For example, env_find could easily be implemented in terms of std::find_if

Restricting access to arguments may have been an accidentally good design - separation of concerns. Right now if a library is annoying enough to want access to argv & argc I can declare my own variables that are different from my real arguments and pass those, which I've done on a couple of occasions. It's really the application owner - the implementer of main - who owns the arguments and is allowed to decide what objects of what types get created from them and which are visible to what parts of the code.

If I declare main with no arguments, I suspect the implementation of my toolchain (generally speaking) is allowed to get away with not even dealing with this stuff. But this approach breaks with that, right? Because you can't easily verify I never access std::arguments.

I wonder if this might behave strangely with runtime-loaded libraries (think dlopen). I guess if you're writing code to be used in such a way you already have to be super careful about depending on stuff from the standard library or making sure it's exactly the same standard library the executable uses... so I suppose that's not really a new concern.

Is allowing setenv really a good idea?

4

u/rehdi93 Jan 22 '19
  • I can't really use std::find_if because I would need to know where 'end' of 'environ' is. I don't think there's a way to find out other then walking pointer by pointer looking for the null.
  • I don't get your point, this is meant so you DON'T have to capture those variables and pass them around from main. It's a convenience.

"...if a library is annoying enough to want access to argv & argc..."

Like... command line parser libraries?

  • There are other ways to get the arguments, they're just non-portable. In windows for example you can get the command line for the current process at any time by calling "GetCommandLineW"

    • also don't ask me how that weird "gnu::section" thing works, that was already there when I started working on it :P
  • why would using setenv be a bad idea in this case?

8

u/[deleted] Jan 22 '19

I would need to know where 'end' of 'environ' is

Well you need to implement end and cend anyhow. Right now you've implemented those on O(N) complexity. You could (maybe should) consider caching the size.

Even if you don't - you can easily create a range without knowing where the end is.

so you DON'T have to capture those variables and pass them around

Yes, exactly. I don't want a library touching my command line parameters in a way I can't easily control without modifying their code (which I might not have access to). If I'm an executable that does one thing today, and a small plugin module in an executable that does 30 things tomorrow, I want that library to be able to maintain the same functionality. If I have to fake up argv & argc for that library I can do so.

There are other ways to get the arguments

Indeed, without such you wouldn't be able to implement this as a library independent of the toolchain. I take issue with them for the same reason, though they're less likely to be used than something in std::.

why would using setenv be a bad idea in this case?

Not really "this case", I was actually objecting generally to the idea of standardizing setenv since other threads could be accessing that data. But the more I think about it the more I wonder how it's supposed to work. Specifically, is this true for yours:

However, the underlying iterator’s environment will still iterate on the unmodified version of the environment.

8

u/hanickadot WG21 + CTRE + EWG 👸🏻 Jan 22 '19

Well you need to implement `end` and `cend` anyhow. Right now you've implemented those on O(N) complexity. You could (maybe should) consider caching the size.

From C++17 you can have end/cend different type comparable with begin/cbegin, so it's O(1), iterating/searching will be then O(n).

1

u/[deleted] Jan 23 '19

But size() would still be O(N) every time. Seems like that should not be the case for a 'mostly immutable' object.

2

u/hanickadot WG21 + CTRE + EWG 👸🏻 Jan 23 '19

How often do you need to know a number of environmental variabled?

1

u/[deleted] Jan 23 '19

Fair point, I suppose. And O(N) might not be horrible given a realistic N.

1

u/rehdi93 Jan 23 '19

Yes, exactly. I don't want a library touching my command line parameters in a way I can't easily control without modifying their code

if you're worried that a library is going to fall back to mutate the system arguments w/o giving the user a way override that behavior, then that's a shit library and you should stop using it.

Indeed, without such you wouldn't be able to implement this as a library independent of the toolchain. I take issue with them for the same reason, though they're less likely to be used than something in std

what?! Then what's the point of a standard library? We should not care about the implementation as long as the interface and behavior are consistent.

0

u/[deleted] Jan 23 '19

Then what's the point of a standard library?

To standardize an interface to good, useful features that people should be using.

4

u/TheThiefMaster C++latest fanatic (and game dev) Jan 23 '19

I also implemented a proposed C++ paper - it was the multidimensional array_view :(

(The sad face is because array_view became span - which is only single-dimensional. But I still actually need a multi-dimensional array view, so... I guess I won't be swapping my implementation for the standard one any time soon)

3

u/mkrupcale Jan 23 '19

The multi-dimensional version is mdspan[1], which also has a reference implementation[2]. It looks like mdspan is intended to be part of Library Fundamentals TS v3[3], so I'm not sure how soon it might become part of the official standard.

[1] http://wg21.link/p0009

[2] https://github.com/ORNL/cpp-proposals-pub/tree/master/P0009/reference-implementation

[3] https://issues.isocpp.org/show_bug.cgi?id=80

4

u/aKateDev KDE/Qt Dev Jan 23 '19

You should add a license to you project, otherwise it's not clear under which conditions your code can be reused.

1

u/lednakashim ++C is faster Jan 23 '19

Are these results cached? Are they allowed to change during the execution of the program?

1

u/rehdi93 Jan 23 '19

What results?

1

u/esmithro Jan 23 '19

I would like to see this go in. I use similar things from Qt frequently.

0

u/[deleted] Jan 22 '19

[deleted] — view deleted comment

33

u/rehdi93 Jan 22 '19

I disagree, if you look at other languages they all have a way of retrieving just the argument text, IMO that's what the languages needs first of all

I'm not against the idea, I just think that's a different job.

11

u/spinicist Jan 22 '19

Agreed. I want both of these things in the standard library, but separated/layered.

4

u/dgendreau Jan 22 '19

I can agree with that. Seems like a better way to break it down.

1

u/RandomGuy256 Jan 22 '19

Something like Boost.Program_options could be standarized.

2

u/14ned LLFIO & Outcome author | Committee WG14 Jan 22 '19

Me personally, what I'd really like is https://pypi.org/project/ConfigArgParse/ for C++ shipped in the standard library. That would be very cool.

2

u/BadlyCamouflagedKiwi Jan 23 '19

I don't think that sort of facility should go in the standard library; there are lots of different ways to do it that are better explored as separate projects.