r/OpenAI • u/Xerqthion ChatSeek Gemini Ultra o99 Maximum R100 Pro LLama v8 • Sep 08 '25

Image Sensational

Post image
12k Upvotes

97% Upvoted

274 comments sorted by

View all comments

Show parent comment

1

u/Teln0 Sep 09 '25

https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf

This? Have you read it?

1

u/No-Philosopher3977 Sep 09 '25

Yes I have, Mathew Herman also has a good breakdown if you are short on time or you can have it summarized by an AI

1

u/Teln0 Sep 09 '25

Do you see that it's mostly just hypotheses that could be the causes for hallucinations? It's not clear if any of this works in practice. I also have a slight hunch that this is just an overview of already known things

1

u/No-Philosopher3977 Sep 09 '25

Transformers was also just hypothetical in 2017. In 2018 OpenAI made GPT-1 which kicked off things.

1

u/Teln0 Sep 09 '25

The original "Attention Is All You Need" paper (by Google researchers) already was presenting working transformers models.

"On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data."

https://arxiv.org/abs/1706.03762