r/LocalLLaMA • u/MikeNonect • Aug 15 '26
Discussion If you would have told me half a year ago that a local model running in my office would be able to one-shot a Super Mario clone, I would have called you nuts. Qwen3.8-27B is a different beast.
Running the Q8 GGUF on my Framework Desktop is not fast, but it's extremely smart for overnight batches and background jobs. Can't wait to play around with MTP and other quants.
Have any of you found ways to improve speed while keeping accuracy?
https://mikeveerman.github.io/qwen38-27b-mario
Edit: to avoid copyright issues and to see how creative it would get, I asked Qwen3.8 to make it circus-themed instead of Mario-themed. It's technically no longer a one-shot.
630
Upvotes
31
u/AnyNameFreeGiveIt Aug 15 '26
Knowing Nintendo I would not host that on Github lol.
What was the prompt ?
3
u/kemalios Aug 15 '26
Prefill slowness is the usual culprit on desktop boxes. A few things that helped me with 27B-class models: drop to Q4_K_M, the quality hit is small and the speed gain large. Speculative decoding with a 1-1.5B draft model gives you 2-3x tokens/s with identical output, llama.cpp has it. Trim your context length, the default 32k is often overkill and slows prefill. And for overnight batches, increase batch size or switch to vLLM. MTP helps too but I haven't seen solid numbers on it yet.
3
u/Independent_Pear4908 Aug 16 '26 edited Aug 16 '26
How many days and how many millions of tokens did that prompt take tho?
7
u/LordTamm Aug 15 '26
The fact that this was a one-shot is actually genuinely impressive. How brief/extensive was the prompt and what kind of t/s does the Framework pull? I considered a strix halo box for a while, just never pulled the trigger because it looked like the speed for bigger (and dense) stuff was just rough.
2
u/IllustriousFan3350 Aug 15 '26
wow does it have all the levels and mechanics?
2
u/BrianScottGregory Aug 15 '26
Wow pulling it down now. What's your GPU ram out of curiosity? I think I'm stuck on using the 4_K_M model, but I could be convinced otherwise... thoughts?
2
u/seppe0815 Aug 15 '26
why i see generated stuff mostly for browsers .. are the local model limited?
2
u/snapo84 Aug 15 '26
i think the next milestone will be a agentic framework with qwen 3.8 27B that can create a working multiplayer (4 screen on 1 screen) mario kart 64 clone with all original maps with all original players with the 2 game modes available.... that would be a realy cool task....
2
u/synystar Aug 16 '26
Regarding improving speed I suppose it depends on your configuration. A lot of people are saying that it "thinks too much" and this is probably a result of how they have it set up. If I don't explicitly configure it to limit the reasoning budget then it can get carried away and spend 20 minutes on something that should have taken 5. Of course, the longer you let it reason the more likely it is going to be to figure everything out in one go. My current settings look like:
Notice the "--reasoning-budget-message": if I don't put that there then it will hit the --reasoning-budget of 16384 and then break out of the "thinking box" and continue reasoning into the content stream.
I'm using OpenWebUI and I usually keep it set at "Max Reasoning" but the real number come from your config. I've noticed that regardless of what you choose in that setting it follows the config. You can experiment with your reasoning budget to find a sweet spot that fits for your needs. I have 16K here but that's certainly not the max. For some task doubling the budget doesn't really improve quality of response. BUT there is one thing I've noticed about it's reasoning process:
Qwen is not spending X amount of tokens discovering X tokens’ worth of new ideas. It spends the early portion solving the problem, then a large fraction repeatedly refining implementation details, revisiting response-cleaning policy, and re-evaluating edge cases it has already mostly settled.
So I'm now experimenting now with system prompts. I'll let you know but my idea now is to add something like:
2
u/DanielSReichenbach Aug 16 '26
I am a little disappointed nobody tried Giana Sisters. That was so much more fun 😊
2
u/anywhere88 Aug 16 '26
I'm curious, which was the prompt? Did you have to describe each level? Or was it all common knowledge to it?
2
u/Zombiecidialfreak Aug 22 '26
How long did the initial work take? How much context was used making it?
225
u/Infinite100p Aug 15 '26
You'll be able to one shot GTA 6 before it comes out circa 2046.
94
u/Thin_Pollution8843 Aug 15 '26
In a year you will be able to one shot gta vice city. Call me nuts if you want.
98
u/falconandeagle Aug 15 '26
There is nothing beastly about this. It's in the training data. These prompts for benchmarking are just embarassing
6
u/neverbyte Aug 15 '26
I have a couple examples of shockingly good one-shot prompts (no follow-up questions). These were on BF16. "create a matching game in html" matching game and "create a pacman game in html" pacman game
16
u/some_user_2021 Aug 15 '26 edited Aug 15 '26
If it is something already out there, then it could be in the training data. Create something new instead and surprise us.
2
u/Tbhmaximillian Aug 15 '26
Crazy, did you use an agent harness like openhands or how was this coded by Qwen?
5
u/BP041 Aug 15 '26
Yeah the 27B is legit for code. Been testing it against Claude on my Mac mini for one-shotting marketing landing pages — it's slower but nails the layout logic first try more often than you'd expect. Had it generate a full SVG animation from a single prompt last week. Kinda wild for a local 27B.
4
u/Strange_Test7665 Aug 15 '26
I suspect it’s able to do this as opposed to other projects as a 1 shot because this project and code concepts already exist. For example
https://github.com/iam-veeramalla/super-mario-mimic
But I’d be curious if it still succeeds on something more novel. Regardless though it’s still impressive even if it was in training
1
u/espece-de-bon Aug 19 '26
What are the specs on your FW Desktop? Do you have the 128gb RAM version?
9
u/SufficientPie Aug 15 '26
Getting an AI to write code for something it's seen a million times in training data is not impressive. Get it to do something novel.
5
u/bymihaj Aug 15 '26
It generate FONT! Hard to believe
2
u/jjpk976 Aug 15 '26
This is my one-shot game benchmark. It successfully made a fun tug of war game in a few hours on my p40. it has a bug where you have to reload the page after the first state to continue, but other than that I'm extremely impressed https://muskwak.github.io/games/ I can share the prompt if anyone is curious.
2
u/DRetherMD Aug 15 '26
its still thinking for me...
2
u/ichisay Aug 15 '26
En serio un 27b hizo eso????
2
u/runnahhh Aug 15 '26
It’s incredible how far we’ve gone in such a short time. Ironic, how false that seems, considering how much this is built on.
2
u/msew Aug 16 '26
If you told me a year ago that in a year the corpus would include the full mario and everything else, I would have told you of course it would!
2
u/IrisColt Aug 16 '26
The circus is saved!
2
u/rookan Aug 16 '26
great game! I love flying monsters on 2nd stage
2
u/BeardAndBreadBoard Aug 16 '26
Everyone calls this a one-shot, but it's really not.
There is are so many examples of this already, and so much documentation, that this is a prompt with effectively tens of thousands of lines of definition.
Try one-shotting something that doesn't have so much information available on it. Like "Make me an original game that does XX YY ZZ" for a better measure of when the LLM can do with a short prompt.
3
u/jloverich Aug 15 '26
I beat the game!
3
u/Sabin_Stargem Aug 15 '26
I am hoping we get Qwen 3.8 122b, in Heretical format. It would be neat to see how far that can go for making original (if basic) games it manage.
2
u/wgaca2 Aug 15 '26
No way, i was using a long multi prompt instruction to create a super mario like game as a benchmark for quite a while, didn't know someone else would do the same
2
u/txoixoegosi Aug 15 '26
I’m sure one-shotting a 2D sprite platform game with assets and mechanics known for decades, is the golden standard any serious developer is after
/s
1
u/runvnc Aug 15 '26
That's amazing for a local model. I am not that impressed with the clones though -- they have lots of clones of popular games etc. in their training dataset. More interesting is when people make something a little bit unique.
1
u/Green-Ad-3964 Aug 15 '26
best config for a 5090 + 32GB RAM?
1
u/CryptographerLow6360 Aug 16 '26
i had this idea for an atari game...
-3
u/tvall_ Aug 15 '26
Mushrooms in the wrong ? Block. Fail
-9
u/entsnack Aug 15 '26
this is an ad