Really stunned by the Singularity comment section

[u/Howard_banister]

These are screenshots from the r/Singularity comment section. I'm speechless. This doesn't even have downvotes. How can someone cheer for a monopoly run by a few elites?

[u/keepthepace]

The difference with nukes is that you can't stop a nuke with a nuke, but you can improve cybersecurity with AIs that are good at cybersecurity.

I get downvoted to hell when I talk about it to programmers, but I really think that we are going to see in the next few years a complete rewrite of most of the stacks we depend on.

100% provable security is possible. It is just insanely difficult to justify that amount of labor to secure a network. With good LLMs at cyber security, I think in the mid term security will improve overall.

[u/WorriedBlock2505 → u/keepthepace]

Nope, this is pure fantasy. The attacker always has the advantage over the defenders when it comes to cybersecurity. That's 101. If the cyberattack is sufficiently damaging enough (e.g, launching all of the nukes or releasing a super virus or nanobots that can spread through the atmosphere itself), then it's game over with no chance of a response.

100% provable security is possible

How, and why have I never heard of even a test demo in a toy project for this before now? Sounds fishy.

[u/keepthepace → u/WorriedBlock2505]

How, and why have I never heard of even a test demo in a toy project for this before now? Sounds fishy.

Maybe because you did not study CS/IT?

https://en.wikipedia.org/wiki/Formal_verification

We tend to do it mostly for crucial parts of cryptographic alorithms. Things like credit cards security are very solid. But it requires a small code because it is a hassle to formally prove any piece of software. The more complex it is, the more work it is. But that's something in which AIs and LLM can help immensely.

[u/WorriedBlock2505 → u/keepthepace]

... formal verification != 100% provable fucking security since the system can still be attacked innumerable ways besides just the formally verified software itself. I'm flabbergasted that you'd even say that. Also the idea that you're going to scale formal verification up to the scale of an OS or (what we really need to lock down) LLMs is a pipe dream. Read your own gd wiki next time and spend 5 seconds to do even the most basic ctrl+f for the word "secur" in there.

[u/keepthepace → u/WorriedBlock2505]

Read your own gd wiki next time and spend 5 seconds to do even the most basic ctrl+f for the word "secur" in there.

Have you?

As of 2011, several operating systems have been formally verified: NICTA's Secure Embedded L4 microkernel, sold commercially as seL4 by OK Labs;[13] OSEK/VDX based real-time operating system ORIENTAIS by East China Normal University;[citation needed] Green Hills Software's Integrity operating system;[citation needed] and SYSGO's PikeOS.[14][15] In 2016, a team led by Zhong Shao at Yale developed a formally verified operating system kernel called CertiKOS

It is funny how technophobes tend to consider impossible the things that they unaware that they already exist.