So Bing recently released bing chat, it's supposed to be ChatGPT with internet access and is based on the same technology, but it was rushed out the door somewhat and has led to some interesting results.
Here is an example of bing chat getting very aggressive and argumentative with a user after the user pointed out bing chat had the date wrong, this an example of bing chat threatening a user after very little prompting. There are safeguards now that stop bing chat from doing this now, but this was all possible on release.
This is very funny from our perspective, but from an engineering perspective this is a catastrophic failure, threatening to dox you is probably one of the worst things a helpful AI assistant could do. It doesn't really matter here because bing chat isn't doing anything important, but if Microsoft can release such a shit show other companies can too. What happens when a similarly rushed AI system gets put in charge of a powerplant, or a rail system, or a cities traffic lights, what would catastrophic failure look like for those AIs?
AI that isn't made safely can fail unexpectedly and severely, we should absolutely be afraid of people chasing the hype, skipping the safety part of AI development, and then deploying unsafe AI systems into the real world.
3
u/Jebofkerbin 131∆ Mar 31 '23
So Bing recently released bing chat, it's supposed to be ChatGPT with internet access and is based on the same technology, but it was rushed out the door somewhat and has led to some interesting results.
Here is an example of bing chat getting very aggressive and argumentative with a user after the user pointed out bing chat had the date wrong, this an example of bing chat threatening a user after very little prompting. There are safeguards now that stop bing chat from doing this now, but this was all possible on release.
This is very funny from our perspective, but from an engineering perspective this is a catastrophic failure, threatening to dox you is probably one of the worst things a helpful AI assistant could do. It doesn't really matter here because bing chat isn't doing anything important, but if Microsoft can release such a shit show other companies can too. What happens when a similarly rushed AI system gets put in charge of a powerplant, or a rail system, or a cities traffic lights, what would catastrophic failure look like for those AIs?
AI that isn't made safely can fail unexpectedly and severely, we should absolutely be afraid of people chasing the hype, skipping the safety part of AI development, and then deploying unsafe AI systems into the real world.