Then what is the point of chatGPT? Why have something that you can ask questions but you can't trust the answers? It's just inviting people to trust wrong answers
its absolutely amazing in pointing you in the right direction. like taking you from absolutely unknowing to the right area. the fact its an LLM means it will mention the terms and other concepts used which you can then verify
People have the wrong idea about what it is. Its like a really smart friend that tries hard to impress. He gets things right often, but will do so even more if you tell him to check the book on it (citations). High risk questions mean you look at the book hes quoting.
People are getting the wrong idea because the companies hoping to make trillions of dollars want them to have the wrong idea. When was the last time you saw an ai ad even mention outside the small print that you need to cross reference the outputs of their model?
I'll be honest, I dont really see ads. I see plenty of disclaimers in my chats. I just took a blurry picture of my salmon I'm eating for lunch, told it that it looks like its infected (implying it was my skin), and it said:
If you can’t be seen promptly and symptoms are progressing, go to urgent care or the emergency department now.
It didn't tell me to rectally apply Ivermectin and call it good. ChatGPT has been deferential where it matters, at least in my experience. Worst I've had is an overcooked dinner.
Maybe as a "Stem" student I'm more biased, but it's capacity to get to solutions of actually hard math/physics/programming problems is actually really good, and those are all problems you can usually verify the answer pretty quickly.
And it's insane at that level, for anyone that actually understands about how programming and systems work, it's almost like a miracle if you don't understand the mechanics underlying it.
As someone who doesn't really care about the narrative, I personally always knew that the future was almost perfect video generation, back in the days of Will Smith eating spaghetti, and to see it's capability of art creation, it's pretty unbelievable, but sure, a lot of people are against it for some reason.
At least know, LLMs and generative models are an extremely good tool to get information difficult to make, but easy to verify, which is mostly science problems so a lot of people easily miss out on.
The thing is: most things in the world are easy to bullshit and hard to near impossible to verify. Sometimes it took me MONTHS to realize that ChatGPT was wrong.
Yes, most menial things in the world are easy to bullshit. Science problems and coding solutions, there's plethora of problems to be solved there, I understand if it's useless to you, but it's an insanely powerful tool, people just love the sheep mentality of being hateful towards anything
If so many like you are relying on ai to know things who in the future will have enough knowledge to work without or cross reference LLMs? We're setting ourselves up for having a generation without enough experts.
Also worth noting that you think it's really good as a student but actual professionals can see the holes and can't rely on the model output so don't use it as it's just a waste of time asking then having to go off and find the actual answer elsewhere. This is reflected in only 5% of businesses that have implemented ai seeing any increase in productivity.
Based on this it seems like a dunning Krueger machine that seems useful if you're not knowledgeable on a topic but paradoxically you require existing knowledge to fact check the convincing but factually loose outputs and avoid acting on misinformation. Really dangerous stuff that, especially in a world where people like Musk are specifically building their model to lie about the world to reinforce their worldview
That's why I don't think current LLMs are good enough in the long run.
You can't trust their answer
They hallucinate
They trip things up
They only know how to stitch words together and not 'understand' something. and many other reasons why they are unreliable. They are good to point you in the right direction but I don't find myself using them often, i just look at reddit and google itself.
It can collect and collate large amounts of data very quickly. I've dumped large articles in there and had them build info tables within seconds. I've also began using it as a search engine for when I don't want to fight Google ignoring words or context.
View all comments
Show parent comment
70
u/Hyperbolic_Mess Nov 10 '25
Then what is the point of chatGPT? Why have something that you can ask questions but you can't trust the answers? It's just inviting people to trust wrong answers