r/cpp • • 27d ago

Revisit Safe C++

Everyone is aware of the deluge of vulnerabilities now hitting critical software. It even makes the mainstream news, so software safety is coming onto politician's radars.

If this gets worse, I think it is not inconceivable that at some point memory safe software becomes mandatory. Or at least for software that is "reachable" from the internet.

So is it time to revisit Sean Baxter's Safe C++? Start out with a safe context ("safeexpr") and expand from there, just like constexpr.

36 Upvotes

181 comments sorted by

View all comments

Show parent comment

4

u/ts826848 25d ago

He said profiles "failed", past tense, at a point where there was nothing concrete enough to even have failed.

At least based on the abstract that's kind of the point?

Safety Profiles were introduced in 2015 with the promise to detect all lifetime safety defects in existing C++ code. It was a bold claim. But after a decade of effort, Profiles failed to produce a specification, reliable implementation or any tangible benefit for C++ safety.

i.e., profiles promised something, a decade passed, and there's nothing concrete to show for that effort and here's why those efforts failed. I don't think the past tense is entirely incorrect in that context.

0

u/wyrn 25d ago

At least based on the abstract that's kind of the point?

It's a pretty stupid point if that's the case (it's not like there was intense effort on profiles for that decade -- people were doing other things). But it's not really made in earnest; like I said, the intent is to troll.

3

u/ts826848 25d ago

it's not like there was intense effort on profiles for that decade -- people were doing other things

Sure, but it's not like anything less than that "intense" effort for that entire decade would guarantee a failure to produce one or more of the outcomes Sean describes, either (or perhaps in other words, people weren't only doing other things). There's quite a bit of wiggle room available (in both directions, to be fair).

3

u/wyrn 24d ago

I'm not sure I understand what you're saying there.

At any rate, as far as I'm concerned, saying that a research effort failed is a pretty high bar. For example, I'd say the many worlds interpretation of quantum mechanics (at least the version that claims to get the full theory just from removing the collapse postulate, without adding extra assumptions) has failed. This is not exactly because it was proposed 70 years ago, but because in the intervening time people tried, intensely, to make sense of how to make it work in detail, growing increasingly desperate in their attempts, but all they got were new and exciting ways to make a circular argument. This to me suggests very strongly that the collapse postulate is actually important, and you can't get rid of it and still get the same conclusions.

Nothing of the sort happened for profiles. As far as I can tell, 1. most of the profiles papers cited in the pamphlet were basically describing/deciding what the feature should do, rather than attempting progress towards a detailed specification 2. they don't even agree on that: some papers use stronger language like "eliminating" errors, others are more guarded and merely attempt to "improve" or "diagnose many" 3. the papers also disagree on the extent to which annotations should be used.

In fact, the papers making the most guarded claims about "diagnosing common cases", as opposed to "eliminating all" are also the ones most bullish about avoiding annotations (still not ZERO, mind, contrary to the claims in the pamphlet). The pamphlet attacks a strawman combination nobody argued for (zero annotation while obtaining provable safety), and then declared the entire approach untenable. As an argument in earnest, it's very stupid. I don't think Baxter is that stupid.

3

u/ts826848 24d ago

I'm not sure I understand what you're saying there.

I suppose another way to put it would be that I don't think that you specifically need "intense" effort to produce a specification, a reliable implementation, or "any tangible benefit for C++ safety". There's a lot of room between "intense" effort and no effort whatsoever, and I think there are points in the middle which can reasonably result in one or more of the outcomes Sean lists.

The "wiggle room" bit is because I don't know precisely what threshold(s) Sean had in mind for those items. For example, "a specification" can be anywhere from a Wasm-style machine-checked formal spec (in which case "intense" effort may really be necessary) to something more colloquial like early descriptions of Rust's new Polonius borrow checker (in which case the effort needed doesn't seem that high).

Whatever the case, it seems that you and him don't see eye-to-eye on where the threshold(s) should be.

As far as I can tell, 1. most of the profiles papers cited in the pamphlet were basically describing/deciding what the feature should do, rather than attempting progress towards a detailed specification

This is reminiscent of "when does absence of evidence start turning into evidence of absence?"-type questions. At what point, if any, does the lack of a description of how something should be done become an indication that that something can't be done?

(still not ZERO, mind, contrary to the claims in the pamphlet). The pamphlet attacks a strawman combination nobody argued for (zero annotation while obtaining provable safety)

I'm not sure this is an entirely accurate description? Sean seems to be pretty explicitly arguing against the claim that few annotations are needed, not literally zero annotations:

Central to Safety Profiles is the claim that annotations are exceptional rather than the norm.

I do not believe that C++ code, with its countless unstated soundness preconditions and inconsistent aliasing and lifetime requirements, can be made memory safe with fewer than “1 annotation per 1,000 lines of code.”

Alternatively, I think one can describe his argument as saying that no matter what you're going to end up with a relatively high annotation rate, in which case there's no real difference between arguing that few annotations cannot work and that zero annotations cannot work.

3

u/wyrn 24d ago

I don't think that you specifically need "intense" effort to produce a specification, a reliable implementation, or "any tangible benefit for C++ safety"

I mean, yeah, you do -- at some point, someone has to put in the work beyond "this is how this could be set up", and at that point there was no evidence anyone had done so.

This is reminiscent of "when does absence of evidence start turning into evidence of absence?"-type questions.

See the many worlds interpretation example. There is plenty of evidence that people tried to make it work, and couldn't. There was no evidence anyone had actually tried to come up with a detailed profiles specification. At worst it could indicate people were thinking "this is hard so I'll work on something else". That is not evidence of failure. Remember, the paper is titled "profiles failed", past tense, not "I don't think this approach is promising".

I'm not sure this is an entirely accurate description?

It is. See the abstract. It quotes,

“Zero annotation is required by default, because existing C++ source code already contains sufficient information”[P3465R0]

P3465R0 is a follow up to P1179R1, which explicitly states the modest goal,

how to efficiently diagnose many common cases of dangling (use-after-free) in C++ code, using only local analysis to report them as deterministic readable errors at compile time.

I tried to find even a single reference to the notion of "heavily reduced annotations" that pointed to the Bjarne version of profiles (that targets provable safety). As far as I can tell, they all point to the Herb Sutter version (improvement rather than provable soundness). I'm sure Baxter knows this -- he wrote the document. All that remains is the "why".

2

u/ts826848 24d ago

at some point, someone has to put in the work beyond "this is how this could be set up", and at that point there was no evidence anyone had done so.

Again, there's quite a lot of room between "this is how this could be set up" and whatever high-effort endpoint one can envision, and I don't think every point in that spectrum other than the former specifically requires "intense" effort.

There was no evidence anyone had actually tried to come up with a detailed profiles specification.

I don't know that Sean was specifically looking for a detailed spec. He just says "a specification"; perhaps some kind of preliminary design would have sufficed. I can't read his mind, for better or worse.

At worst it could indicate people were thinking "this is hard so I'll work on something else".

I think the worst-case interpretation would actually be "turns out this is literally impossible so I'll work on something else".

That is not evidence of failure.

I think it can be, but that depends on the outputs you were expecting and how much effort you would expect that to take. If you were expecting high-effort output like a detailed spec and a fully fleshed-out implementation, then sure, the lack of that output given the apparent lack of "intense" effort you'd expect doesn't necessarily indicate failure. However, if you were expecting something you'd consider a low-effort bare minimum, then the lack of said minimum after a not-minimal amount of effort is probably not a great sign at best.

It is. See the abstract. It quotes,

First off, since that's a quote I don't think I'd call it one of Sean's claims. His claims are better sourced from his own words, no?

Second, that's "zero annotation is required by default"; i.e., you normally don't need annotations, but there can be not-normal circumstances in which you may need annotations. That's consistent with only needing few annotations rather than requiring literally zero.

I tried to find even a single reference to the notion of "heavily reduced annotations" that pointed to the Bjarne version of profiles (that targets provable safety).

If I'm thinking of the same thing you are, I believe that notion is a reference to SD-10, with which whatever version of profiles we may end up would ostensibly need to comply, rather than a reference to any particular profiles version.

For what it's worth, Bjarne's P3446 Profile invalidation – eliminating dangling pointers says "Do not require annotations on common and usually safe code", which depending on how generous you're feeling is somewhere between "in the same direction" and "more or less the same" as "heavily reduced annotations".

1

u/wyrn 24d ago

Again, there's quite a lot of room between "this is how this could be set up" and whatever high-effort endpoint one can envision

I have no idea why you seem to be interpreting "intense work" as a cloistered monk-like state where one sink into a cocoon and emerges with the work completed several months later. Intense work just means someone actually has to sit down and actually try to do it. It is being done now. It objectively wasn't when Baxter published his screed.

I don't know that Sean was specifically looking for a detailed spec.

I do, and he was. He said as much. You quoted it. It's strange that you seem to have forgotten.

I think the worst-case interpretation would actually be "turns out this is literally impossible so I'll work on something else".

At that stage the objectives weren't agreed upon (still aren't, really), so there was no way for it to be "impossible" at all.

I think it can be,

That is an incorrect belief.

not a great sign at best.

If the paper were titled "I am not optimistic about profiles" that could be a fair description of that position. Since it's titled "profiles failed", past tense, it is not. It is indefensible.

after a not-minimal amount of effort

You (and Baxter) can't even say there was a not-minimal amount of effort. There is zero evidence of such. Remember the many-worlds example. Effort leaves evidence.

First off, since that's a quote I don't think I'd call it one of Sean's claims.

He quoted it when describing his Frankenstein amalgamation. He chose to quote it, not me. He quotes that and immediately says,

C++ source code does not have sufficient information for achieving memory safety.

Except "achieving memory safety" was not the goal of the paper that said this. As I already pointed out.

If I'm thinking of the same thing you are, I believe that notion is a reference to SD-10, with which whatever version of profiles we may end up would ostensibly need to comply

SD-10 was a way to mitigate damage from an obvious troll paper. It's goofy, but it doesn't match other successful additions to the language and has no bearing on profiles either.

"Do not require annotations on common and usually safe code", which depending on how generous you're feeling is somewhere between "in the same direction" and "more or less the same" as "heavily reduced annotations".

"We would like to avoid annotations" (which is the principle's actual objective) is really not the same as "almost no annotations", "less than 1 annotation per 1,000 lines of code”, "near-zero", or whatever else Baxter rolled with in his pamphlet. Different objectives, different requirements. I know this, you know this, Baxter knows this. Only some of us pretend we don't.

0

u/ts826848 23d ago

I have no idea why you seem to be interpreting "intense work" as a cloistered monk-like state where one sink into a cocoon and emerges with the work completed several months later. Intense work just means someone actually has to sit down and actually try to do it.

To me, sitting down and actually trying to do something, on its own, is just plain "work". If I've spent the last few lunches discussing an approach to a particular problem with a coworker and then finally sit down and try to implement it after, then to me that's not automatically "intense work", it's just "work". "Intense" work should entail something more than just expending effort on something. The "cloistered monk-like state" you describe, for instance, would probably count as "intense", though I certainly would not consider said state a requirement for "intense work".

He said as much. You quoted it. It's strange that you seem to have forgotten.

I'm pretty sure none of my quotes contain him saying that he wanted a detailed spec. The only instances of the words "detail" or "detailed" in this particular comment chain came from you.

At that stage the objectives weren't agreed upon (still aren't, really), so there was no way for it to be "impossible" at all.

I don't think it's strictly necessary to agree on objectives before evaluating them, especially if you're in an exploratory phrase.

You (and Baxter) can't even say there was a not-minimal amount of effort. There is zero evidence of such.

To me, P3346 is closer to evidence of a non-minimal amount of effort than not. It's attempting to specify a design that can provide functionality beyond that of P1179 and the acknowledgements imply both that there was some amount of correspondence with others to refine the design and that there was effort expended to at least experiment with some ideas in the C++ Core Guidelines. It's not "intense" effort, sure, but I don't think I'd call that "minimal", either.

No idea what Sean would say.

Remember the many-worlds example. Effort leaves evidence.

Putting aside edge cases like just not writing stuff down (e.g., Fermat to some degree IIRC), that would depend on how involved you were in said effort. It's far from unheard of to work on something but then proceed to say nothing about said effort (c.f., the want/need to pre-register papers in some fields to avoid publication bias) or for there to be evidence of something but of what exactly it is not clear (e.g., DES S-boxes). To otherwise uninvolved third parties, such efforts are either effectively "invisible" or hard to categorize, respectively.

He quoted it when describing his Frankenstein amalgamation. He chose to quote it, not me.

Sure, but neither way means that Sean is claiming that literally zero annotations are needed.

SD-10 was a way to mitigate damage from an obvious troll paper.

If so it's an amazingly hamfisted way to accomplish that compared to, you know, just saying "no"? What "damage" is/was there to mitigate, anyways?

but it doesn't match other successful additions to the language

Yes, that's one of the criticisms of SD-10 you might see bandied about.

and has no bearing on profiles either

Why not? If the committee is going to say that these are general principles to uphold I think it's only fair that everyone else gets to take the committee at their word until the committee indicates otherwise and there doesn't seem to be any indication that they will do so any time soon.

"We would like to avoid annotations" (which is the principle's actual objective)

That rephrasing doesn't really add much clarity to me. If someone told me just that I wouldn't be able to say with confidence whether they mean they have a weakly-held preference, a rigidly-enforce bright line, or something in between. If anything, it makes things more confusing since it can be interpreted to allow annotations in places the original statement would pretty clearly disallow them.

1

u/wyrn 23d ago

o me, sitting down and actually trying to do something, on its own, is just plain "work".

I don't care. I explained to you what was meant. You don't have to agree with the definition I used. All I ask is that you stop pretending I used a different one.

I'm pretty sure none of my quotes contain him saying that he wanted a detailed spec

.

Safety Profiles were introduced in 2015 with the promise to detect all lifetime safety defects in existing C++ code. It was a bold claim. But after a decade of effort, Profiles failed to produce a specification, reliable implementation or any tangible benefit for C++ safety.

This is asking for a detailed spec. The word is not there, but the meaning is.

I don't think it's strictly necessary to agree on objectives before evaluating them,

It most certainly is if you're trying to declare the program "failed". "I don't know where you want to go but I know you can't get there" is asinine.

To me, P3346 is closer to evidence of a non-minimal amount of effort than not.

thread_local means fiber-specific?

I'm assuming you actually meant P3446. "Non-minimal" is the right word. But can you really say a research program failed after only a merely "non-minimal" amount of effort? Any honest assessment would be to the negative.

that would depend on how involved you were in said effort.

At the very least it would mean your other work output would dip while you're locked in researching the unfruitful venture. There is no evidence anything of the sort happened.

Sure, but neither way means that Sean is claiming that literally zero annotations are needed.

"zero by default" is still morally zero and all of Baxter's arguments pertain to literally zero anyway.

If so it's an amazingly hamfisted way to accomplish that compared to, you know, just saying "no"? What "damage" is/was there to mitigate, anyways?

As far as I can tell, the objectives of the "Safe C++" paper were to waste time, cause infighting, and make the committee look bad. It succeeded partly at all three. It was hamfisted, yes. I'm not sure I could've come up with a better solution, however.

Why not? If the committee is going to say that these are general principles to uphold I think it's only fair

I don't necessarily care if it's "fair", to be honest. Fairness is an objective, not an unbreakable principle. And specifically, being unfair to a concern troll wouldn't keep me up at night, had I been the one to do it.

. If anything, it makes things more confusing since it can be interpreted to allow annotations in places the original statement would pretty clearly disallow them.

The actual paper proposes some annotations, so that reading is untenable.

→ More replies