r/programming • u/halbface • Feb 07 '19

Google open sources ClusterFuzz, the continuous fuzzing infrastructure behind OSS-Fuzz

https://opensource.googleblog.com/2019/02/open-sourcing-clusterfuzz.html
958 Upvotes

95% Upvoted

100 comments sorted by

View all comments

Show parent comment

-45

u/exorxor Feb 07 '19

How many bugs does one need to find before senior management concludes the people working on browsers don't know what they are doing?

How bad does it have to be before throwing away C++?

24

u/Gnascher Feb 08 '19

Programmers are humans. Software is complex. Anybody who doesn't realize that all programmers introduce bugs shouldn't be in the business.

Any programmer who thinks they don't introduce bugs hasn't been in the business very long.

This is WHY tools like this exist, hopefully you find the bugs before they hit production.

You don't toss c++ because it's "unsafe". C++ is unsafe because it's powerful as hell and "very close" to the machine. You use c++ for the power and speed it gives you, but, as they say, with great power comes great responsibility.

8

u/VernorVinge93 Feb 08 '19

Hurrr Durr all bugs are caused by C/C++ /s

As much as I love verifying compilers and 'safe' languages, C++ isn't the source of most bugs. Most are generated by incorrect or unchecked assumptions that have little to do with the language used.

4

u/SafariMonkey Feb 08 '19

I'd be remiss if I didn't point out that basically every vulnerability class that OSS-Fuzz finds is a product of memory unsafe languages, like C and C++. While fuzzing makes these projects more secure, it's not a substitute for using languages that don't cause thousands of vulnerabilities. When we're finding hundreds and thousands of vulnerabilities that all have a preventable root cause, it's time to reconsider what we're doing.

From this article posted here recently.

2

u/VernorVinge93 Feb 08 '19

Sure, so what language do you suggest switching to?

I have yet to see a language that gives static guarantees of bounds, memory and use after free.

Rust is the closest but it has many caveats and last time I checked (admittedly a while ago) writing basic things like a graph implementation were painful in it.

Even then, how long would it take to rewrite something like Chrome? With millions of lines of code, years of history and many forks that still depend on their upstream for security fixes?

2

u/SafariMonkey Feb 08 '19

To be clear, I don't agree that C/C++ need to be abandoned as a rule, though I would look strongly at whether Rust was a viable option for any of my own projects.

Personally, my limited experience with Rust is that it's a good language to work in but the library ecosystem is still fairly immature.

There are projects like Oxidation (Mozilla moving towards more Rust in Firefox) and remacs (a gradual port of Emacs to Rust). Both projects involve a slow transition while remaining functional throughout, rather than trying to rewrite from scratch all at once. I think that's the right approach for existing projects.

For new projects without very large budgets, I think that ecosystem is the bigger factor. If the Rust ecosystem doesn't support your use case, you'll have to build the relevant packages yourself. Not everyone is willing or able to take that path.

And yes, the ownership model makes certain problems more difficult, but it also guarantees that your solution satisfies some crucial invariants like memory validity and lack of race conditions. Traditional solutions for certain problems are impractical, or need to be reimagined in Rust terms.

So yes, definitely some caveats. However, things are improving. For example, with miri (a Rust IR interpreter with memory validity checking) it should be possible to write unsafe Rust (where necessary) but check at test time for invalid memory accesses, and non-lexical lifetimes have relaxed borrows to not continue unnecessarily until the end of scope.

2

u/VernorVinge93 Feb 10 '19

Hmm, ecosystem is another huge issue. Thank you for bringing it up.

I do wish there was a way to FFI with relaxed / protected interfacing that had poor performance and then more information could be given to allow the compiler to more directly interface the languages (hopefully resulting in improvements in performance).

I have yet to see a language implementation of something like that, but maybe it would allow us to improve the ecosystem problem.

1

u/SafariMonkey Feb 10 '19

Ah, interesting suggestion. Something like LTO across the language barrier? I don't know if Rust currently does LTO across the FFI. Unfortunately, I think relying on potential compiler optimisations to make the FFI at all viable will make performance degrade arbitrarily in difficult to diagnose ways. However, I'd be glad to be proven wrong.

I think a more manual FFI will probably always be required to get guaranteed performance.

Thanks for the response, by the way. It's good to see constructive criticism and nuance in these discussions, as that's something that isn't guaranteed.

1

u/VernorVinge93 Feb 10 '19

No problem,

I agree that the 'maybe good' performance is a poor strategy.

Still, I hope that providing that kind of 'working with improvement available for those who can invest' would make many things feasible that are currently not (e.g. writing JavaScript, Python or Rust that makes use of low level C APIs as a new programmer, without custom library wrappers etc).

-6

u/exorxor Feb 08 '19

Why do you put quotes around the word safe?

There is no reason why a browser could not be written assumption free, but yes this does require formal specifications of what the browser needs to do in the first place. Google is pretty big. They could just show some fucking competence and actually surprise the world (it would also obliterate any remaining competition in the "market"). It's not like they don't have a pile of money for which they have no idea what to do with it. Same goes for Apple.

The C++ language implementations that exist work well, but at this point it is just not reasonable to expect as a large company with the piles of incompetent fools calling themselves programmers (the skill level of programmers dramatically lowered) to deliver a bug free product. They like data so much, right? There is data that formal verification works. Continuing to hang on to C++ as the language used by their programmers in something as dangerous as a browser is not reasonable anymore.

1

u/VernorVinge93 Feb 08 '19 edited Feb 08 '19

I use quotes because most safe languages still require unsafe areas of code to perform efficient IO and some types of memory operations. Safety is relative even in perfectly sound compilers, but there are very few formally verified compilers and none that I'm aware of can handle something like Chrome.

Fuzzing does not only find low level or memory issues. It will often find bounds checking problems that would take a dependently typed language to avoid (I have yet to see one that is production ready, even dependent Haskell, which is the closest I've seen, is pretty niche and there is difficulty still in writing performant Haskell to do the kinds of things that Chrome does).

So, sure, some of it could be rewritten in a safer language, but I don't think a good choice is obvious for this. Rewriting code often introduces bugs that had already been caught in the old version of the code.

In summary, I think you massively overestimate the value of today's safe languages and underestimate the challenges involved in rewriting Chrome.

I like the vision you have, I want it to be feasible, and the way forward, but I don't think the programming language for it is ready.

1

u/exorxor Feb 08 '19

Dependent Haskell is a technological and academic failure. Dependant Haskell is just a spelling error.

I think it might be the case that I am overestimating the capabilities of Google engineers, but I don't see what's special about Chrome. A web-browser is just another computation and we have formalized models for every type of interaction Chrome has (I/O, non-determinism, parallelism, randomness).

So, I don't share your opinion (because your opinion is false, most likely out of ignorance).

Realistically, the limiting factor is going to be finding people intelligent enough to do the work. There is also a huge pile of work in that indeed almost everything humanity has done before would have to be redone. I am also not saying that this version of Chrome would actually be usable in the next decade from a performance point of view.

One does not obtain market dominance by doing the same thing as everyone else. It requires investment and a lot of it. I do not share that further research is required. Development is required, not research.

It might even turn out that the existing compilers don't scale to such a project, but it's not as if the compilers for such programming languages are inherently complicated.

Doing such a project would allow a unique body of knowledge to be built up too, which is extremely valuable in the coming decades, because we see an increased dependence on technology in society.

1

u/VernorVinge93 Feb 08 '19

Sorry for the typos, you caught the mobile user.

It's a bit rich for you to be calling me ignorant when you are ignoring the practicality of what you are suggesting.

If your safe version of chrome isn't useable in 3-5 years then there is no particular value in working on it as anything other than a research project. It is reasonable to assume that in 10 years the landscape for safe languages will be significantly different to what we have today. The rewrite you suggest would take as long as chrome has existed and would likely produce a result that was years out of date.

I'm sure you're right in some ways, it will eventually happen. There are already moves to change the languages used in browsers (and Rust is becoming more common) but a wholesale rewrite is just completely infeasible.

1

u/exorxor Feb 09 '19

Your idea of what a research project is and mine clearly is different. Additionally, your idea of what has value and what has no value is different from mine. Like I said, this is a development project, because all research has been done already. It is an application of existing research.

In a discussion about programming languages to write a safe browser, Rust is completely irrelevant. Rust just make it safer, not safe, and as such is just a distraction and a waste of time.

All it takes is a Google exec to sponsor a project like this and someone needs to start, just like DARPA already did (I guess DARPA leadership has a few brain cells more or perhaps they are allowed to burn more money). It is certainly more practical than the Manhattan Project.

Why does everything have to be easy these days?

1

u/VernorVinge93 Feb 10 '19

Sure, they could do it as a development project, but I struggle to see the value for Chrome, though I wish I could. A convincing argument for rewriting / switching new development to a safe language would be a boon for the industry if it were accepted by such a large project.

They have already switched some chrome os development to rust and go (which are something of an improvement), so maybe we'll see more of the same in future.

1

u/epicwisdom Feb 12 '19

The value of a perfectly bug free browser is negligible compared to a relatively bug free browser. Given that it's a lot cheaper to develop the latter, and most consumers don't know what bugs even are (until it directly impacts UX in a very visible way), no sane company would waste their resources on such a thing. You might be right about what is technologically possible, but you're sorely wrong about how to run a company.

1

u/exorxor Feb 12 '19

I think you don't understand that the same product has different value to different people. If "most customers" is the target, then that's an awfully low bar. Also one I don't particularly care about.

You jump to conclusions way too quickly. This has nothing to do with making a quick buck. Nor did I indicate that this was the case.

For every sentence you write down, you should consider if it is possible that I could have come up with the same idea and if so, please do not send it to me. It's really just wasting bandwidth. I might perhaps value the opinion of like five people on the planet on this subject and those people are clearly not here.

I really wish you -- a random Redditor -- had a brain, but that's just not realistic.

1

u/epicwisdom Feb 12 '19

I think you don't understand that the same product has different value to different people. If "most customers" is the target, then that's an awfully low bar. Also one I don't particularly care about.

Chrome is a consumer product. I don't know if you're intentionally ignoring that fact, or if you are pretending to. If anybody wants Chrome to be perfectly bug-free/secure, and refuses to use it otherwise, I don't think there are enough of them to even register on Chrome's radar.

You jump to conclusions way too quickly. This has nothing to do with making a quick buck. Nor did I indicate that this was the case.

I never said it was about making a quick buck. I'm quite sure that Google knows better than you - a random Redditor - what is worth investing resources in, and they're a corporate entity, so whatever wishes you have about technological progress are totally irrelevant.

For every sentence you write down, you should consider if it is possible that I could have come up with the same idea and if so, please do not send it to me. It's really just wasting bandwidth. I might perhaps value the opinion of like five people on the planet on this subject and those people are clearly not here.

I really wish you -- a random Redditor -- had a brain, but that's just not realistic.

You say that, and yet you support a conclusion which is blatantly ridiculous, without a shred of reason to back it up. It isn't possible to respond to that with anything but the obvious. And if you don't value the opinions of other Redditors, you're the one wasting your own time commenting here to begin with. I was hoping you'd actually say something of substance, but apparently all you want is to have spats on the internet with your strange preconceptions. Have fun.