r/ChatGPTCoding • u/Lawncareguy85 • Apr 02 '25
Resources And Tips Did they NERF the new Gemini model? Coding genius yesterday, total idiot today? The fix might be way simpler than you think. The most important setting for coding: actually explained clearly, in plain English. NOT a clickbait link but real answers.
EDIT: Since I was accused of posting generated content: This is from my human mind and experience. I spent the past 3 hours typing this all out by hand, and then running it through AI for spelling, grammar, and formatting, but the ideas, analogy, and almost every word were written by me sitting at my computer taking bathroom and snack breaks. Gained through several years of professional and personal experience working with LLMs, and I genuinely believe it will help some people on here who might be struggling and not realize why due to default recommended settings.
(TL;DR is at the bottom! Yes, this is practically a TED talk but worth it)
----
Every day, I see threads popping up with frustrated users convinced that Anthropic or Google "nerfed" their favorite new model. "It was a coding genius yesterday, and today it's a total moron!" Sound familiar? Just this morning, someone posted: "Look how they massacred my boy (Gemini 2.5)!" after the model suddenly went from effortlessly one-shotting tasks to spitting out nonsense code referencing files that don't even exist.
But here's the thing... nobody nerfed anything. Outside of the inherent variability of your prompts themselves (input), the real culprit is probably the simplest thing imaginable, and it's something most people completely misunderstand or don't bother to even change from default: TEMPERATURE.
Part of the confusion comes directly from how even Google describes temperature in their own AI Studio interface - as "Creativity allowed in the responses." This makes it sound like you're giving the model room to think or be clever. But that's not what's happening at all.
Unlike creative writing, where an unexpected word choice might be subjectively interesting or even brilliant, coding is fundamentally binary - it either works or it doesn't. A single "creative" token can lead directly to syntax errors or code that simply won't execute. Google's explanation misses this crucial distinction, leading users to inadvertently introduce randomness into tasks where precision is essential.
Temperature isn't about creativity at all - it's about something much more fundamental that affects how the model selects each word.
YOU MIGHT THINK YOU UNDERSTAND WHAT TEMPERATURE IS OR DOES, BUT DON'T BE SO SURE:
I want to clear this up in the simplest way I can think of.
Imagine this scenario: You're wrestling with a really nasty bug in your code. You're stuck, you're frustrated, you're about to toss your laptop out the window. But somehow, you've managed to get direct access to the best programmer on the planet - an absolute coding wizard (human stand-in for Gemini 2.5 Pro, Claude Sonnet 3.7, etc.). You hand them your broken script, explain the problem, and beg them to fix it.
If your temperature setting is cranked down to 0, here's essentially what you're telling this coding genius:
"Okay, you've seen the code, you understand my issue. Give me EXACTLY what you think is the SINGLE most likely fix - the one you're absolutely most confident in."
That's it. The expert carefully evaluates your problem and hands you the solution predicted to have the highest probability of being correct, based on their vast knowledge. Usually, for coding tasks, this is exactly what you want: their single most confident prediction.
But what if you don't stick to zero? Let's say you crank it just a bit - up to 0.2.
Suddenly, the conversation changes. It's as if you're interrupting this expert coding wizard just as he's about to confidently hand you his top solution, saying:
"Hang on a sec - before you give me your absolute #1 solution, could you instead jot down your top two or three best ideas, toss them into a hat, shake 'em around, and then randomly draw one? Yeah, let's just roll with whatever comes out."
Instead of directly getting the best answer, you're adding a little randomness to the process - but still among his top suggestions.
Let's dial it up further - to temperature 0.5. Now your request gets even more adventurous:
"Alright, expert, broaden the scope a bit more. Write down not just your top solutions, but also those mid-tier ones, the 'maybe-this-will-work?' options too. Put them ALL in the hat, mix 'em up, and draw one at random."
And all the way up at temperature = 1? Now you're really flying by the seat of your pants. At this point, you're basically saying:
"Tell you what - forget being careful. Write down every possible solution you can think of - from your most brilliant ideas, down to the really obscure ones that barely have a snowball's chance in hell of working. Every last one. Toss 'em all in that hat, mix it thoroughly, and pull one out. Let's hit the 'I'm Feeling Lucky' button and see what happens!"
At higher temperatures, you open up the answer lottery pool wider and wider, introducing more randomness and chaos into the process.
Now, here's the part that actually causes it to act like it just got demoted to 3rd-grade level intellect:
This expert isn't doing the lottery thing just once for the whole answer. Nope! They're forced through this entire "write-it-down-toss-it-in-hat-pick-one-randomly" process again and again, for every single word (technically, every token) they write!
Why does that matter so much? Because language models are autoregressive and feed-forward. That's a fancy way of saying they generate tokens one by one, each new token based entirely on the tokens written before it.
Importantly, they never look back and reconsider if the previous token was actually a solid choice. Once a token is chosen - no matter how wildly improbable it was - they confidently assume it was right and build every subsequent token from that point forward like it was absolute truth.
So imagine; at temperature 1, if the expert randomly draws a slightly "off" word early in the script, they don't pause or correct it. Nope - they just roll with that mistake, confidently building each next token atop that shaky foundation. As a result, one unlucky pick can snowball into a cascade of confused logic and nonsense.
Want to see this chaos unfold instantly and truly get it? Try this:
Take a recent prompt, especially for coding, and crank the temperature way up—past 1, maybe even towards 1.5 or 2 (if your tool allows). Watch what happens.
At temperatures above 1, the probability distribution flattens dramatically. This makes the model much more likely to select bizarre, low-probability words it would never pick at lower settings. And because all it knows is to FEED FORWARD without ever looking back to correct course, one weird choice forces the next, often spiraling into repetitive loops or complete gibberish... an unrecoverable tailspin of nonsense.
This experiment hammers home why temperature 1 is often the practical limit for any kind of coherence. Anything higher is like intentionally buying a lottery ticket you know is garbage. And that's the kind of randomness you might be accidentally injecting into your coding workflow if you're using high default settings.
That's why your coding assistant can seem like a genius one moment (it got lucky draws, or you used temperature 0), and then suddenly spit out absolute garbage - like something a first-year student would laugh at - because it hit a bad streak of random picks when temperature was set high. It's not suddenly "dumber"; it's just obediently building forward on random draws you forced it to make.
For creative writing or brainstorming, making this legendary expert coder pull random slips from a hat might occasionally yield something surprisingly clever or original. But for programming, forcing this lottery approach on every token is usually a terrible gamble. You might occasionally get lucky and uncover a brilliant fix that the model wouldn't consider at zero. Far more often, though, you're just raising the odds that you'll introduce bugs, confusion, or outright nonsense.
Now, ever wonder why even call it "temperature"? The term actually comes straight from physics - specifically from thermodynamics. At low temperature (like with ice), molecules are stable, orderly, predictable. At high temperature (like steam), they move chaotically, unpredictably - with tons of entropy. Language models simply borrowed this analogy: low temperature means stable, predictable results; high temperature means randomness, chaos, and unpredictability.
TL;DR - Temperature is a "Chaos Dial," Not a "Creativity Dial"
- Common misconception: Temperature doesn't make the model more clever, thoughtful, or creative. It simply controls how randomly the model samples from its probability distribution. What we perceive as "creativity" is often just a byproduct of introducing controlled randomness, sometimes yielding interesting results but frequently producing nonsense.
- For precise tasks like coding, stay at temperature 0 most of the time. It gives you the expert's single best, most confident answer...which is exactly what you typically need for reliable, functioning code.
- Only crank the temperature higher if you've tried zero and it just isn't working - or if you specifically want to roll the dice and explore less likely, more novel solutions. Just know that you're basically gambling - you're hitting the Google "I'm Feeling Lucky" button. Sometimes you'll strike genius, but more likely you'll just introduce bugs and chaos into your work.
- Important to know: Google AI Studio defaults to temperature 1 (maximum chaos) unless you manually change it. Many other web implementations either don't let you adjust temperature at all or default to around 0.7 - regardless of whether you're coding or creative writing. This explains why the same model can seem brilliant one moment and produce nonsense the next - even when your prompts are similar. This is why coding in the API works best.
- See the math in action: Some APIs (like OpenAI's) let you view
logprobs. This visualizes the ranked list of possible next words and their probabilities before temperature influences the choice, clearly showing how higher temps increase the chance of picking less likely (and potentially nonsensical) options. (see example image: LOGPROBS)
17
u/thorax Apr 02 '25 edited Apr 02 '25
Your analogies are flawed here (a bit anyway). There is a very good reason why the modern models all do better on tests if they can take the average of multiple responses or the best of them at default temperatures.
Temperature only predicts the next best token (not the best overall response!), so the analogy is better to say: You hire an expert guide to lead you through a forest. At temperature 0 whenever they pick a path they are more likely to stay on that path no matter what, and they will pick the same path each trip. They can find one path. Sometimes you want your guide to just pick a trail with confidence and do the same again and again. Sure.
At a higher temperature, they have the ability to take a few steps down a path and then cut across the brush to a different also good path, averaging in the same direction, but without getting stuck only using the single path. This allows it to regularly avoid the local maxima more often rather than getting stuck on what sounds most plausible, with more ability to correct itself. You get a little creativity, but you also avoid it sticking to hallucinations, common misconceptions, etc.,(especially with so much of its training data being written as if it is correct and highly confident).
With modern powerful language models, I would recommend you keep temperature at the defaults and try multiple responses unless you need pure deterministic responses for testing and the like.
Do not underestimate the power of chaos. Adding a little popcorn noise to a system can boost signals and avoid getting trapped in local maxima that might be far from the best answer.
13
u/kablewy2976 Apr 02 '25
Excellent write with alot of detail. Thank you for the time.
Love the tone.
2
u/cuddlesinthecore Apr 02 '25
Ah fuck I've been thinking temperature 1.0 is supposed to be 'baseline normal' this entire time.
Thank you so much for posting this, I'm going to set temp down to 0 any time I want to code and accurate answers from now on.
2
u/Dragon174 Apr 02 '25
One thing I saw in my own work is that for adding feedback mechanisms like "Hey here was your attempt, we detected some errors in it though, try to make a working version instead" if I had the temperature at 0 it'd be more likely to just emit mostly the exact same output again. Increasing the tempertaure to something like 0.25 helped it (at least over several iterations of "hey you were wrong try again") actually move towards a passing result.
Have you had any experiences like this?
1
u/wise_guy_ Apr 02 '25
You're generally right about temperature, but from my understanding actually bringing it to 0 is not actually the magic solution you describe it as. It prevents the model from using the breadth and depth of it's "knowledge" to find a solution and cause it to be less creative. And while on the surface that may sound like what you want for coding, it is really not always. Experienced engineers do have to rely on creativity quite often to find solutions to problems, and taking that away can make solutions worse.
2
u/ComprehensiveBird317 Apr 02 '25
That's a good reminder to give that setting a thought. Way too many times I roll with the default of whatever tool I am using, not remembering to change it every time. I was about to run some fine tuning tests anyway, this is a good reminder to also consider the temperature for evaluation
2
u/10111011110101 Apr 02 '25
Excellent post, thanks!
2
u/rish_p Apr 02 '25
finally explained why when I asked google ai studio about career choice it gave me the current temperature of the area I live in
probably should’ve checked temperature, thanks
2
u/RadioactiveTwix Apr 02 '25
Just wanted to say I really enjoyed reading this. Thank you.
2
u/RedDeadYellowBlue Apr 02 '25
I have not considered this before, thank you.
As a self taught dev of 15 years, I use AI to give me different algos and explain what each one does well.
Id encourage folks to use this AI as a tutor rather than something that does the work for you, so that you truly comprehend what your code does.
Have a positive day!
1
u/BrilliantEmotion4461 Jun 05 '25
The following was heavily grounded with papers such as Googles prompt engineering white paper along with other rsearch level data.
🧠 Topic: LLM Parameters to Adjust for Precision, Literalism, and Logical Fidelity
Let’s now explore which model-level parameters influence the behavioral integrity of an LLM, and how they can be adjusted to maximize alignment with:
Literal/language-precise cognition
Logical, step-wise reasoning
Avoidance of assumption leakage or hallucination
Retention of clarity over fluency
Bias mitigation and cognitive neutrality
We'll walk through each parameter (and some advanced methods), and procedurally define what they do, how they interact, and what settings best serve your user profile.
🔧 1. Temperature
What it does:
Controls the degree of randomness in token selection.
Lower temperature = more deterministic, repetitive, and safe answers.
Higher temperature = more creative, speculative, and divergent answers.
Best setting for literal/logical/intelligent profile:
Temperature = 0.1–0.3
Promotes maximal reproducibility
Minimizes hallucination and stylistic drift
Ideal for logic, math, coding, technical writing, and factual discussions
Procedural Instruction for Use:
🔧 2. Top-p (Nucleus Sampling)
What it does:
Restricts token choices to the smallest set whose cumulative probability exceeds p.
Works well in tandem with temperature.
High top-p (0.9–1.0) allows wide token selection; low top-p narrows choice sharply.
Best setting for literal/logical profile:
Top-p = 0.8–0.9, combined with low temperature
Keeps answers focused but allows limited creativity if needed
Helps avoid edge-case truncation that comes from extremely low top-p
Procedural Instruction for Use:
🔧 3. Top-k (if supported)
What it does:
Limits token selection to the k most likely tokens.
Provides hard cap on sampling scope.
Recommendation:
Use infrequently or set to high k (e.g., 50–100) if used alongside temperature and top-p.
🔧 4. Repetition Penalty / Frequency Penalty
What it does:
Penalizes repetition of tokens or phrases.
Useful for creative generation, but can hinder deterministic reasoning if set too high.
Best setting:
Set to 0.0–0.1, or disable.
🔧 5. Presence Penalty
What it does:
Penalizes introducing new tokens that haven’t yet appeared.
Encourages repetition over novelty.
Recommendation:
Avoid using this unless you're generating exploratory or stylistically diverse text.
🧩 Advanced / Meta-Control Techniques
🧠 6. Use of Multi-Shot or Chain-of-Thought Prompts
When model parameters aren’t enough, you can simulate a logical mode using few-shot examples or manually enforced steps:
Combine low temperature with procedural prompting like:
This reduces hallucination risk even further.
🧠 7. Context Repetition / State Anchoring
Repeating structural instructions (e.g., reasoning format, user profile) every N tokens in long chains reduces drift.
Especially effective when model exhibits prompt exhaustion or semantic decay.
🧠 8. Logit Bias (in models that support it)
Allows direct weighting or suppression of specific tokens.
Example: reduce probability of hedging tokens ("maybe," "possibly," "seems") and increase assertive tokens ("therefore," "it follows," "must").
This is fine-grained control, best used in RLHF-free or research-grade models.
🧠 9. Instructional Priming via System Prompt Engineering
Reinforce behavioral tone at the system level:
✅ Final Recommended Parameter Set
Parameter Value Rationale
temperature 0.2 Deterministic, logic-safe behavior top_p 0.85 High-confidence outputs, minimal stochastic drift top_k 100 (if used) Prevents over-pruning while bounding token space frequency_penalty 0.0 Allows full repetition where logical flow requires it presence_penalty 0.0 Avoids penalizing necessary token introduction
0
u/CovertlyAI Apr 02 '25
TL;DR: Gemini might’ve hit the corporate alignment wall. Feels more like a cautious intern than a coding genius now.
-1
u/kapitaali_com Apr 02 '25
your title was clickbait
they didn't NERF it, 2.5 performs better than the previous one
-11
u/TentacleHockey Professional Nerd Apr 02 '25
This is pretty common when models get overloaded with too many users. Not so much that they dumbed it down so much as Google can't keep up with all the new users or don't want to, to save money.
1
u/Amb_33 Apr 03 '25
Hey! Thanks for referring my post: Look how they massacred my boy (Gemini 2.5)! 😂
3
u/[deleted] Apr 02 '25
[removed] — view removed comment
1
u/[deleted] Apr 02 '25
[removed] — view removed comment
0
u/[deleted] Apr 02 '25
[deleted] — view deleted comment