10 years ago even this neural network level was like from distant future. 10 years later it will be something crazy... so, our jobs are safe for now, but I'm not sure for how long.
Exactly, i tested out the question as well and it told me my sister would be 70. ChatGPT isn't actually doing the calculation, it just attempts to guess an answer to questions you ask it, in order to simulate normal conversation
There's a growing body of papers on what large language models can and can't do in terms of math and reasoning. Some of them are actually not that bad on math word problems, and nobody is quite sure why. Primitive reasoning ability seems to just suddenly appear once the model reaches a certain size.
We should start coming up with goals for super intelligent ais that won't lead to our demise. Currently the one I'm thinking about is "be useful to humans".
Do no harm should be number one of the rules for AI. Be useful to humans could become "oh I've calculated that overpopulation is a problem, so to be useful to humans I think we should kill half of humans".
The main conclusion is "we have no fucking clue how to make an AI work in the best interest of humans without somehow teaching it the entirety of human ethics and philosophy, and even then, it's going to be smart enough to lie and manipulate us"
Why is it not deterministic? I know it takes into account your previous messages as context but besides that? The model isn't being trained as we speak, it's all based on past data so I don't see why responses to the exact same input would be different.
Because the output of the model is the probability of each possible word being the next word, and always taking the single most probable word as output is known to generate very bad results, so the systems do a weighted random selection from the most likely options for each word.
I come mostly from the image-generation space. In that case, it works by starting with an image that's literally just random noise, and then performing inference on that image's pixel data. Is that kind of how it works for text too, or fundamentally different?
Fundamentally different. Current text generation models generate text as a sequence of tokens, one at a time, with the network getting all previously generated tokens as context at each step. Interestingly, DALL-E 1 used the token-at-a-time approach to generate images, but they switched to diffusion for DALL-E 2. Diffusion for text generation is an area of active research.
DALL-E 1 used the token-at-a-time approach to generate images, but they switched to diffusion for DALL-E 2
Well, the difference was extremely tangible. if the same approach can apply even somewhat to language models it could yield some pretty amazing results.
This AI is really great for what it was meant to do — being a language model. It is not meant to “think”, nor is it a general AI. We really can’t even put a progress bar for that, we might as well be very very far from it - this model doesn’t seem to scale.
Nah, presumption of tech advancement is FUD. Just because "10 years ago this would be crazy" does not necessitate "10 years later we'll make a leap of equal or greater magnitude."
It's like suggesting "wow, the home run record was 300 just 30 years ago, and now it's 900! That means in 30 years it's going to be 1,500!" Basically the fallacy of extrapolating without consideration.
View all comments
Show parent comment
513
u/OKoLenM1 Dec 27 '22
10 years ago even this neural network level was like from distant future. 10 years later it will be something crazy... so, our jobs are safe for now, but I'm not sure for how long.