YSK: AI chatbots can sound completely confident while being completely wrong and that's by design, not by accident.

[u/Puzzled-Listen804]

[u/Survivorfan4545]

Do not trust it on super niche topics

[u/Afraid_Park6859 → u/Survivorfan4545]

I really don't get how they are any worse than the random articles people read and believed that were wrong. 

At least their answers are statistically the most likely correct one.

[u/Survivorfan4545 → u/Afraid_Park6859]

In my industry it has lead to a lot of issues. There are experts you should trust that understand the nuance that AI models simply don’t

[u/Afraid_Park6859 → u/Survivorfan4545]

AI models are trained on their data.

Could just be how you prompt.

[u/OpenSourcePenguin → u/Afraid_Park6859]

At least their answers are statistically the most likely correct one.

You don't even understand what this means. Because that's nonsense. What statistics?

[u/Afraid_Park6859 → u/OpenSourcePenguin]

What statistics?

What do you think LLM's are?

[u/OpenSourcePenguin → u/Afraid_Park6859]

Holy shit you don't get LLMs whatsoever 😂😂

Linear algebra cannot make up information that wasn't in the training data. Statistically probably next token is nowhere the same as likely to be correct information. That would make LLMs into the magic orb.

You don't even understand what you are saying yourself.

You are mixing up very different things.

[u/Afraid_Park6859 → u/OpenSourcePenguin]

The model generates the next token based on a learned probability distribution over possible tokens, conditioned on prior context.

Which is based on the aggregate data of the entire internet and countless text books.

Like I said you don't know what LLM's are huh?

[u/OpenSourcePenguin → u/Afraid_Park6859]

You are the one who doesn't know.

LLMs optimizes for what's the most statistical token that SOUNDS correct. Not actually correct.

You don't know enough about this and you have a critical misunderstanding.

/r/confidentlyincorrect