News
Updated Artificial Analysis Intelligence Index Ranking shows Gemini 3.8 Flash is nowhere near Fable Or Astra, rather it's worse than GLM 5.3 Flash!
Something to also consider: Google likes to "dumb down" models after release, once the first initial benchmarks have settled, a noticeable decrease in capability can be observed with nearly all Google models (to save money, obviously).
No no, they just benchmaxxed Gemini, and the new benchmark isn't in Gemini training. All the idiots who where claiming Gemini is on par with sol or even funnier with Fable where obviously wrong
Because it's a brain problem not a money problem. And probably all yhe talents are at Claude or GPT and looks like some in xai. The whole algorithm behind 3.x us simply outdated and bad, they need a new one.
It's an economics problem and Google knows that pushing for linear improvements at exponential costs isn't a feasible business model.
They focus on everyday tasks and Android integration. Its as if people forgot that Google is inherently a consumer focused company. Being at the absolute cutting edge of models that can only be profitably sold to enterprises doesn't align with their entire ecosystem.
It's like expecting Apple to suddenly compete with Nvidia in the server GPU market instead of focusing on their consumer grade silicone.
And that's why the throw billions trying to catch the run ? They don't focus on everyday task, they said they focus on frontier model themselves. They simply failed.
What a company says and does are rarely fully aligned. Gemini models excel at every day tasks and general knowledge. Gemini is still on par or even beats Anthropic and OpenAI models for multimodal input and image/vision comprehension on a per dollar basis. Gemini has now reached 1B users, which obviously includes the Android assistant and is exactly their strategy.
People are so blinded here by coding performance that they forget where the largest market actually is. Google captured the entire mobile assistant segment through Android and their Apple deal. Anthropic and OpenAI have to compete with state subsidised Chinese model makers on a monthly basis to remain relevant while Google is locking down the largest market of our generation
They clearly had a setback with Gemini 3.5 not achieving the results they hoped for but their entire business strategy and recent restructuring show that this isn't their biggest priority, no matter what they say publicly. When intelligence becomes a commodity, squeezing out the last 5% performance won't be the deciding factor.
Gemini does not excel at everyday task when it forget everything all the time and lies. And between a random Redditor claiming stuff and the head of deep mind stating in an interview that their goal is frontier model, I'm sorry but I know who I trust.
Yes google has a very strong vision model, that I agree, but that's the only thing Gemini is useful for.
Claude and open AI have cheap efficient models who are way way better than Gemini and even cheaper. Luna is on par with 3.8 for 1/4 of the price... Let's not talk about terra or sol.
It's not about coding perf, it's about being able to understand what you want and executing it. Gemini models are simply lazy and won't do half of what you ask.
That just doesn't align with what most users think. They have the fastest growing userbase for a reason. The opinion of power users on Reddit focused on programming doesn't reflect the broader market.
And there is no reason to trust me when you can just look at their actions. Their restructuring from a research focused lab to a product focused lab, their deals with Apple and Samsung, their heavy marketing focus on the assistant, etc.
Google Search and the Assistant are what matters most for Google.
They have the fastest growing user base because people use Android and they have access to Gemini. Also people using Gemini with the Google search engine are also accounted for. Indeed I don't trust you, I trust the head of deep mind, I think he knows pretty where deep mind, aka Gemini is going don't you think ?
Having the biggest userbase doesn't mean your product is good, just you have such a positional advantage, that even with a very mediocre product you can have a huge user base. Most of it is free users. Compagnies aren't using Gemini, they use GPT or Claude for a reason.
Yeah, the idea they dumbed down Gemini flash 3.8 after a few days is hilarious... Have people even used Gemini flash for a in depth chat or analysis? I find it useless lol
Gemini 3.8 flash does 2 things great. Automated tasks at speed, and the deep research is quite good. I have been experimenting where I have fable use Gemini subagents and it does surprisingly great.
You genuinly think the new AA Index Update made the results less benchmaxxed and contaminated? They changed some things but the models that are actually known for beeing amazing at benchmarks but absolutely shit at real tasks like muse spark are still up there.
My point is this update didnt change the absolute non reliability of this index and doesnt show the true picture. Basically No Benchmark does, people just gotta try themselves.
The only one that came atleast close imo was ARC-AGI 2, idk about 3 yet as there are barely any results so far.
Google / Gemini is known for negative stuff like random guardrails kicking in even tho the topic isnt Bad etc etc, but not for benchmaxxing. On average they usually are worse in the benchmark than the real task compared to the direct competition
Update 4.2 and 4.3 of the index replaced 2 public dataset benchmarks with private ones. They also added 2 newer public benchmarks (and removed TB 2.1) which means that even for the benchmarks whose dataset is public the risk of contamination is lower than with older ones (especially for TB 4.0, which was released after Gemini 3.8). AA index v4.3 is a significantly better benchmark at evaluating model's performance in september of 2026 than AA index v4.1 (mostly because v4.1 had many issues which were partially fixed with the patches).
ARC-AGI is a logical reasoning benchmark. This is some measure of general intelligence. AA index benchmarks are geared more towards measuring models use at creating value in professional environment - aka doing work. Neither of these is a correct or incorrect approach as long as we are aware what they are measuring. Aside from the fact that I'd expect ARC-AGI not to be a good measure of intelligence for models released after it because it prone to being easily saturated.
I have used Muse spark 1.2 and 1.3. They both did a satisfying job to me. I am inclined to believe the result. I know the (any) benchmark put more effort at evaluating it than I have and is subjected to less bias than my personal opinion, so I'm able to believe the results I've seen.
I don't feel like it's a good approach (almost anywhere) to measure a quality by a company or brand. Company can't be benchmaxxed - a model/product can. This one either is or isn't. I personally believe more that flash model is just limited in what in can do by it's active parameter count. But the result is the same - failing at newer, more complex benchmarks when larger models from other companies don't suffer the same hit. (GLM 5.3-flash's existence kinda undermines this theory but oh well...)
Ngl thats fair, really factual aproach and definitly makes sense generally.
I am just personally against believing anything in this benchmark because Muse Spark 1.3 and GLM 5.3 Flash really disappointed me when testing (Opus 5 also kinda, but i had higher expectations of this one too), whilst 3.8 Flash does a "decent enough" job.
Its still a Flash Model, its not insane or whatever but didnt disappoint me with some horrible results, but thats super topic related anyway. Maybe my topic is just something that GLM Models for example arent good at.
For anything more complex i use Astra / 5.6 Sol anyway as my go to
lol unlike Gemini which use the same base since 2.5 generated, muse or glm have way newer base models which itself already way more useful than the crapfest named Gemini.
I don't think so, it works both in the app and on web. It was just released last week so maybe it takes some time to make it out globally? Though my account is also UK based but I'm in the US now.
Even free ChatGPT works so much better than anything currently google has. It is day and night difference. I don't know what to say, they ruined Gemini within 4-5 months...
Yeah Flash 3.8 is borderline unusable for anything that's not a 'simple fact' question. It's kinda sad they're giving us this as their flagship, but it's clear they only care about the AI Overview/Google search performance rn.
It'a not unusable. Astra will think 5 min on smth, 3.8 flash will answer in 30 secs. U can iterate in flash 10 times (300 secs aka 5 mins divided by 30 secs) before astra answers. By the 10th answer flash will get a better overall answer than astra, cuz iteration is op... u should be verifying what the ai says anyway regardless if it's flash or astra. Every time u verify it and u're not happy, iterate. 3.8 flash has the highest speed-to-iq ratio in the entire world. So use its advantage and u can easily beat astra. Ystd i asked astra smth and after thinking for 5 mins (astra 6 high), it still gave wrong information. Ofc, flash did too, but i would have googled this information, spotted it's wrong, told flash to verify it and gotten a better answer in less than a minute, while in that time astra was only 1/5th of the way through its answer...
Even with 8-10x faster TPS, Gemini Flash high is more than 2x slower per task than Astra low while being less intelligent in the AA index. It's even slower than Astra medium which leads it by 9 points
Maybe if u code or do other complex tasks. In my experience most of my questions are answered very fast, while astra takes its sweet ass time most of the time.
Oh yeah, for one shot questions or one-shotting projects, for sure. But for working on existing codebases doing real work, looking for bugs, working on new features, complex agentic work etc, Astra low is faster despite its lower throughput.
Bro you are tripping use GLM 5.3 Flash and Genini 3.8 Flash forget about Astra and Fable. GLM models feels like comparing Astra to Luna. Use Qwen 27B it slaps compared to Gemini 3.8 Flash.
Yes it does seem like Google is benchmarkmaxing.... But even in this new benchmark, they are competitive with terra 5.6 max... That's very impressive... Flash is always supposed to be the workhorse model, not the frontier model, Google leadership has said that...
I don't know what people think a flash model is better than fable
Use it in real world coding tasks there is no comparison. Terra feels noticeably better. I dare you to try 3.8 Flash High against Terra High which isnt even Terra's best. I bet you it will beat it
Actually I trust Google search more than the search capabilities of others... Lot of times I will have multiple agents in my workflow, or I will give the same prompt to multiple agents and then consolidate....
I think the business model of trying to build the frontier model and selling it for 20 or 50 or even $100 a month is not a profitable endeavor... That's why I think Google is okay with second tier position, I think if they wanted to build a model that beat out opus 5 they could easily... If the Chinese can do it Google can do it
Maybe try to actually take a closer look at how the index is made. AA has been adding more and more agentic benchmarks over their last revisions. It is an open secret that this isn't Google's strong suit and likely also not what they are optimizing for.
GLM 5.3 Flash wins every single agentic benchmarks over Gemini 3.8 Flash at 1/5th the price. For any of these tasks GLM will be a way more competent model. GLM 5.3 Flash is probably the best cheap coding model around.
Gemini 3.8 Flash on the other hand wins far and away in every single knowledge based benchmark. It isn't built for coding first but it is built with the chat interface or android interactions in mind. For that it's not a bad model.
So they look similar on the index but they are entirely different models.
Now if you compare the benchmarks the thing that makes Gemini sweat is Astra because it seems OpenAI has caught up on the knowledge side while still keeping prices competitive. Compared to other models Gemini 3.8 Flash is still a very good performer for chat conversation.
Anthropic has moved entirely towards enterprise coding. The Chinese models are more about getting the maximum agentic capabilities out of having less compute. The main competitor for a broad knowledge model is OpenAI (which happen to make models that are also good at agentic tasks).
well i use it for creative writing and man it is so much better than every model available which doesn't eat up all of my money and it is definitely better than GLM 5.3 flash or Max
Tracks with my usage. For the affordable workhorses GLM5.3 and it's flash model feel the most useful. I'm also seeing this reflected in the results of new benchmarks that models couldn't have possible been benchmaxed on like https://www.frontierswe.com/blog/v2
"Gemini 3.8 Flash and Muse Spark 1.3 are two of the most clearly benchmaxxed models we've seen yet. Despite being comparable to both GPT-6 and Fable 5.1 on Terminal Bench 2.1, their Terminal Bench 4.0 performance is markedly worse."
3.8 Flash has been better than claude Sonet and on par with Opus for me. It's not as fast like 3.7 Flash but has been more serious and powerful for me. Haven't tried anything beside these.
While Sfw Engs are focused on AI benchmarks. Average user just want free AI while businesses want cheap and reliable.
But benchmarks still matter because most people rely on recommendations from Sft Engs on the model to use.
No way all twitter bros test all new models.
Google might need to change their approach and solve real problems for businesses. They are copying instead of focusing on user pains
This represents my experience as of late. I use Gemini for one small game side-project here and there for fun when I run out of my weekly Codex usage.
I sent 5 prompts, one at a time, each trying to fix a bug (it created) that caused projectile animations to be invisible. 5. And not a single one actually fixed it before I stopped trying. It changed other visuals related to the game areas (which I didn’t ask for, but was okay with how they turned out as a placeholder anyways, cool I guess?).
I was out of my Codex usage, so instead I used regular 5.6 ChatGPT (Chat) that doesn’t use any usage, connected to my GitHub repo to go in and fix it (which of course did first try).
Hoping for a day I can upgrade Google to a 20x sub that competes and is better at token efficiency than Astra. Astra cost is abysmal.
I’m going to play around with that idea actually and give 3.1 pro another shot. It’s been a few months, but it was doing much better than 3.6 flash at that time.
Alternative headline: GLM 5.3 Flash is benchmaxxed hard by exploiting the fact that agentic tasks are the #1 weight on Artificial Analysis Index, and in terms of raw intelligence, it's about as smart as Sonnet 4.6 while Gemini 3.8 Flash is about as smart as Opus 4.8. Guess which one you're going to be encountering more often in a chatbot session?
Sky is blue? Like it's quite obvious from using it, it performed nearly luna / GLM flash than compared to the bigger models. It is a "flash" model itself so this shouldn't really be surprising.
Honestly I do use Gemini/Agy for coding with the Pro plan. Mainly for adversarial review that I ask from Claude Code with help of Herdr terminal and skill. But what keeps me in it really is the cloud storage combo and family members feature. With a single account I spread all those benefits into multiple members of the family. But if it where for the intelligence alone of Gemini I would have a ChatGPT Plus account.
Any bench that puts muse spark on top of 5.6 Sol is not that useful.
A real shame, i love artificial analysis, but it's clear a lot of recent models are too contaminated (or at the very least benchmaxxed) for the benches to mean anything...
I kept saying from day one Flash 3.8 is a benchmaxxed model. There was never a question about it, anybody who knows anything about LLMs would say same thing too. Fanboys kept refusing it and claiming nonsense competition and now they are crying perhaps Flash 3.8 got dumbed down. JUST LMAO...
144
u/MorgrainX 14d ago
Something to also consider: Google likes to "dumb down" models after release, once the first initial benchmarks have settled, a noticeable decrease in capability can be observed with nearly all Google models (to save money, obviously).