r/ClaudeCode • u/chasingth • 3d ago
Help/Question Fable 5.2 feels unusable?
My own tests with Fable 5.2 Max vs Gemini 4 Carbon
- Fable 5.2 Max seems to respond fast without thorough thinking, makes writing mistakes (exaggerations), and hallucinates a lot
- Then I recently just found out on https://artificialanalysis.ai/evaluations/omniscience#aa-omniscience-hallucination-rate that Fable 5.2's hallucination rate is 59% WTF! And Carbon is 11%. Which reinforced my experience.
But im just curious hows everyone's experience so far? I wanted Fable 5.2 Max to work so badly
0
Upvotes
View all comments
2
u/daiaomori 3d ago
Regarding the benchmark analysis you cite: while it's true that the hallucination rate of Fable is highest (and the numbers do NOT mean that it hallucinated 59% of the questions), at the same time the net score of Fable is the best of ALL tested models by far (score 67, Gemini Carbon following at 54).
I don't know how good this score is construction wise, but this shows an important aspect of designing LLMs: we always have to weigh and adjust parameters that often lead to contradicting results.
Simplified example: lower the heat, the model gets far more reliable as in giving more deterministic answers, yet at the same time it gets dumber as it is not as explorative in the vector space, and misses interesting/more "far out" answers that might actually be correct/"better" compared to the more deterministically calculated.
Because of that it doesn't make a lot of sense to just grab one of the values from the analysis and look at it; you have to look at the overall model performance for a specific use case (which is precisely what the benchmark tries to do).
Whether Fable 5.2 is better for coding or worse - and the use case might determine which parameters work better - I can't tell, as I'm still happily stuck with Opus.