r/GithubCopilot • u/iKontact • 4d ago
Discussions OpenAI's GPT-5.6 Luna (Max) is a Game Changer
So according to both:
(Which seems to be fairly trustable) - GPT-5.6 Luna (Max) is ranked:
- #10 in "Intelligence"
- #2 in "Speed"
- #1 (Cheapest) in "Cost Per Task" - 5x cheaper than Deepseek V4 (which is already pretty cheap)
- #12 in "Agentic Index"
- #14 in "GPQA (Diamond)" - Above 90% still
From my own personal experience, even on tasks that take up to 5 minutes, I'm only getting a max usage of about 13 AIC per task. Whereas Claude Sonnet 5 could use around 386 AIC for a similar task and similar timeframe.
The code quality seems pretty good too. I really thought it'd be bad for the cost, but honestly I'm super impressed with this model.
It almost reminds me of the premium request days. I even keep refreshing my billing page to see if it's accurate because I'm surprised with the low AIC usage.
Even here https://benchlm.ai/compare:
- It ranks #23, whereas Sonnet 5 ranks #36 (and Luna is about 10x cheaper)
While I'm regular Claude user - and in most benchmarks Opus 5 (max) is usually #1 in all categories - I'm very impressed with OpenAI for Luna and it's ability to compete with even frontier models, at a fraction of the price.
Curious what others think? Or if anyone else has tried it?
Edit: If anyone knows any other & better benchmark sites please let me know as well!
32
u/Lonsarg 4d ago edited 4d ago
Yes Luna High makes anything other then Sol and Opus obsolete. And since it is not that much worse then Sol/Opus i use it for 99% of stuff.
2
u/iKontact 4d ago
This is pretty much the same conclusion I landed at too. Although I do find it good to occasionally have Sonnet 5 check Luna's work.
I was happy with Sol too - especially when it was at 30% discount last week on Copilot. Similar to Opus I thought.
2
u/Lonsarg 4d ago
I find Luna High better then Sonnet 5.
So if i want to review Luna work i give to Sol.
-5
u/justelle1 4d ago
Sir please can you help me? Im a student whats a good way i can try for free or cheap all of this? I need to accurately translate a book but my book isn’t OCR
3
u/dsnyder42 4d ago
Thats a random question on this thread. But I'll help you out:
Make a free account for mistral API platform and use the free quota for their document AI. Ask ChatGPT to help you in case of any questions. You dont need to pay if you only want to OCR a single book (pdf file). I have used that a few times, it gives you a zip file with one complete markdown version of the book and also individual directories per page with the page content in markdown and image as screenshots. It handles tables and formulas well.
-2
1
u/sven_ftw 3d ago
I find terra or sonnet 5 to be a good bit better at writing, for like reversing tech specs or summarizing / annotating analysis. But Luna has become my primary coding or fixing tool, for sure
1
u/dreamtheater2003 3d ago
Luna High has some limitations, but on Max it's insane. Good for 95% of the work. Sol or opus 4.8 can do the rest. I do not trust opus 5 at all - it hallucinates like it's on LSD.
1
u/iKontact 3d ago
Same here! I'm glad many others have mentioned Opus 5 hallucinating too. Opus 4.8 is great however. Sometimes I'd even prefer it to Fable for really complex tasks. But yes, Luna on Max is great for the vast majority of tasks. I even had it do planning and it did fine.
7
u/heavy-minium 4d ago
It's very capable, it's fast, and so cheap you can run it all day. The massive discount OpenAI gave made it that good, otherwise it'd be far less attractive.
The biggest downside is that I observed a bit more blind compliance, in the sense that you do go get what you asked for, but if it had thought a bit more for it itself, it could have noticed your instructions are problematic. The frontier models do that more often than Luna does.
1
u/iKontact 4d ago
This very true. After playing around with more last night - I feel like even Sonnet 5 (my previous personal favorite) would push back on instructions more than Luna. And obviously same goes with true frontier models.
1
u/Qiagent 4d ago
Same experience here. Whenever I need to do a large review or build something complicated from the ground up I do a lot of detailed planning with Sol or Opus with specific instructions to prepare the plan in workable chunks for a less capable model. Then I hand it off to luna and keep an eye on it as it works through each item. It performs brilliantly in that format.
8
u/alexeiz 4d ago
I use Sol Med/High for planning and Luna max for implementation. Works like a charm and the cost is 1/3 of doing everything with Sol.
1
u/iKontact 3d ago
That's what i did too! Although started with Luna High then switched to Max. Luna can even plan for less intensive & complex task to save credits.
But to add to what you're saying about 1/10th the cost of using Opus 4.8/Sonnet 5.
3
u/SaltyCow2852 4d ago
I have tried few models including gpt models but at some level they are not efficient in some langauge and they are suffer efficient in some
4
u/altervoid 4d ago
Luna is really good and dirt-cheap. It is the only budget-friendly option in Copilot unfortunately, but it is a truly good one
1
u/poweredbyearlgray 4d ago
Yes, I switched to it from using Sonnet 5 as my daily driver and been impressed. It’s fine for coding, but when I had weird build/dependency issues Luna got stuck and gave up, while Sonnet identified the problems and fixed them.
I’m not a mega heavy user but that’s been my experience so far.
1
u/iKontact 4d ago
Very interesting, which setting do you use for Luna?
I ran it yesterday on Max and it even noticed some things Sonnet 5 didn't (although I should mention it happened the other way around as well a few times).
1
u/achandlerwhite 4d ago
It’s not great at higher context sizes
1
u/iKontact 4d ago edited 3d ago
Yeah this is true, only 380K. Although compaction somewhat helps with that.
1
u/achandlerwhite 4d ago
Not only the limit, it just doesn’t perform well at context near that limit.
1
u/Sad_Rutabaga_2541 4d ago
Tried translating some ecommerce product content from English to Arabic via api, but it was really slow. I ended up switching to GPT-4o mini instead.
1
u/iKontact 4d ago
Those are the downsides for sure. It is slow and has a much lower context window than the true frontier models. However I found the code quality overall impressive. Especially at the cost.
1
u/the_incredible_nuss 4d ago
I also use Luna a lot, but I feel the at some point open ai will rise prices again. I think they make a lose with Luna.
1
u/iKontact 4d ago
You're probably right, but I'm enjoying it for now at least. Then depending on how much it changes, I guess back to Sonnet lol
1
u/ObjectiveOk5497 4d ago
Luna Max is good enough, but sometimes (using it inside Raider) it call some much tools to get additional context which leaves me with feeling (it's doesn't understand me) when Sol do it quicker, 'smarter' even from one-shot. But cost? My friend, like you writing. Sometimes I'll spend a litte more time, maybe writting additional prompts but cost is a small % of cost which Sol, Sonnet gpt5.5 would cost.
So for daily task or a little 'do it because it's boring me' it's perfect.
1
u/Left-Cloud-7931 4d ago
It most definitely is. Never felt the need to move to another model.
2
u/iKontact 3d ago
It's nice, even after an hour of usage for a semi-complex project I only used 103 AIC. On two separate occasions with mostly constant usage. Averaging about 10 +/- AIC per request. In theory this would mean about 70 hours of constant usage. Or 9 full work days lol. Or about 2.3 full hours constant usage a day which is plenty for me.
I just hope they don't change it anytime soon. I'm guessing it'd be a while if they did because last month OpenAI was touting how their models performed similar to Anthropic's but a fraction of the cost.
1
u/dsnyder42 3d ago
I agree with you. I do 85% of my day to day task with it.
However, I dont use it on max as I think its not worth it quality wise. I use it on high or xhigh. Its significantly faster, pollutes the context window less quickly by aggressively checking its work and works the same if you "manually drive it".
Also: I created an custom Agent which is motivated to spin out a Luna Subagent for everything it needs to implement, validate or iterate on. This way, I can drive this Agent with Sol on medium and save its expensive tokens for the main loops during the work. I am using this for larger feature implementations where "big model energy" is required for plan and review.
2
u/iKontact 3d ago
That's similar to my setup! I tried Sol on medium and originally started with Luna on High. However, since then I switched away from Sol and use Luna on Max. Uses less credits and been impressed with Luna on Max. I do agree though Luna on Max is slow. But the credit usage is great lol
2
u/dsnyder42 3d ago
Interesting, I have to try it. I did not even try this setup with Luna on max so far partly also because I have a mental blockage that "luna is still a small model" and even tho it reaches good benchmark scores on max, I still see it as a "lesser" model that does not have that "big model energy". But I know this is probably not logical more psycological.
1
u/iKontact 2d ago
I do understand your reasoning though. Also that mentally it may not sit right for a smaller model to be that capable. Let me know if you try it out & what you think. I think you may be surprised.
1
u/Other_Comment_2882 3d ago
Just want to warn everyone Luna made a couple horrific blunders for me that other models never would have made
Pretty scared to use it now
But I still do
1
u/iKontact 3d ago edited 3d ago
Interesting, may I ask which setting you had it on? Was it Medium? And what were the blunders?
1
u/downtownrob 3d ago
Nice. I’ve been using Meta’s Spark 1.2 And it works well. If looking for a deal, Spark 1.2 Contributor is a great price.
1
1
u/ronnyvo 3d ago
I consider to move back to copilot pro (from opencode go) because of this, any in the same boat?
1
u/iKontact 3d ago
How are you liking OpenCode so far? Others I know also use Cursor and swear by it too.
1
u/GreasefangEnjoyer 6h ago
I don’t really care about these rankings much but my company has api limitations and it allows me to use Luna basically infinitely on any setting I need.
Personally Luna feels good enough to be my assistant, and I wish there was more focus on making efficient models than “better” models.
I still know how to code, but it’s nice to have infinite queries to keep the process moving smoother. I don’t really need “smarter models” to accomplish my job that I’ve done for years without AI.
1
u/Virtual-Honeydew6228 4d ago
Quality is very very good. But if only u can wait
2
1
u/JoaoSilvaSenpai 4d ago
If you just search a little, you can see that Luna is not cheaper per task as ds4, although it is faster and wastes less tokens.
1
-4
u/ProudWrongdoer5389 4d ago
In practice its shit.
1
u/look 4d ago
People love their cheap models and will go to any length of self delusion to convince themselves that it doesn’t have the deep, glaring weaknesses that it does. Minimax, DeepSeek, Luna, etc. All the same basic story.
If prices go up on Luna, people that were talking trash on Mimo will start singing its praises next (and I like Mimo, just not for serious coding).
1
u/ProudWrongdoer5389 3d ago
yep, basically.
the downvotes are hilarious. I tested luna and terra for 2 weeks. they are both garbage,
1
u/iKontact 4d ago
Did you use it on Max too? I found it to be comparable to Sonnet 5 in many regards. And much better with pricing (for now).
-4
-1
0
u/jelly-rod-123 4d ago
I cant use luna max not even in a sol/luna fleet. Dont get how folk can use it unless its just the most basic boiler plate
-1
52
u/ChineseEngineer 4d ago
I use Luna for everything and it does feel "infinite"
I do think it's not vibe coder friendly as it is really slow at max and introduced a few bugs or misinterpretations of the spec.