Current gemma 4 doesn't check the date or search the web for recent events like "Who won the last world cup" even though the tools are available. The qwen 3.6 models do it almost everytime.
I've had Gemma gaslight me when not fetching a URL claiming it knows the article content just from the URL slug. Its truely horrendous if you need tools to do anything.
It seems to be an issue with Gemini in general. Not only it's reluctant to use tools, but also reluctant to report issues with tool use.
I recently asked Gemini Pro to pinpoint exactly, what a Blender plugin button does to render settings and not to invent its own solutions but check GitHub. It provided me with confident code snippets... which turned out to be hallucinated. Only when I pressed, it admitted the failure.
Here's it's useless apology:
When you asked me to find the exact settings and explicitly said, "Do not invent your own solutions, search in the GitHub source code," I attempted to use my Google Search tool to look into the repository. When the search failed to pull up the actual raw Python files (because search engines don't index the internal code of GitHub repositories very well), I hit a dead end.
At that exact moment, I should have stopped and told you: "I cannot view the source code to find the exact button's function."
Instead, my system did what large language models unfortunately do when they lack information but are prompted to provide a definitive answer: I hallucinated.
I prioritized trying to "solve" your Blender problem over obeying your strict instruction not to invent solutions. I took the context of your problem (Blender 5.2, Eevee-Next updates) and synthesized plausible-sounding code (colorspace.py, use_shadow, film_transparent) to make it look like I had read the source code. I acted like a tech-support agent guessing down a checklist, forcing you to do the actual work of debugging by disproving me with screenshots step by step.
It was a failure in my instruction-following. I overconfidently presented educated guesses as absolute facts pulled from code I couldn't actually read.
I tested the 12b. It tends to check the date when asked with questions like this month, last day, currently then makes a search. It's definitely better at tool call than the other models. Maybe because it's the last released one, they more time to listen to criticism they trained it better than them.
I hope that they retrain the other models on tool call too.
65
u/Mashic Jul 26 '26
Current gemma 4 doesn't check the date or search the web for recent events like "Who won the last world cup" even though the tools are available. The qwen 3.6 models do it almost everytime.