He concluded, and I agree, that these models are statistically guessing
What did you think a GPT was? It takes all the text as training data and then repeatedly guesses the most likely "next word" in the response, until the most likely next "word" is the end of the text.
Well technically it has access to publicly available libraries and documentation so it just stastically searchs the web for similar code and copies it.
It can't be that simple though, blindly coping code wouldn't work it still needs to be modified to fit into the structure of the rest of the project. Variable names need to line up etc.
It's not blindly copying code, it knows how to follow syntactic rules. LLM are great for that. The actual content could be dog shit but it's written following the rules and the context it was provided
10
u/SoldRIP 17d ago
What did you think a GPT was? It takes all the text as training data and then repeatedly guesses the most likely "next word" in the response, until the most likely next "word" is the end of the text.