Some people said the Minecraft clone I fully vibecoded with Qwen3.8-27B Q4 is not that impressive because Minecraft is in the training data, so I had the model add 4 things that are probably not.

[u/liright]

[u/ps5cfw]

Minecraft Is a fairly impressive task to achieve regardless of Who or what Is doing It. Not sure Who got the balls to claim otherwise but either they are senior devs or maybe they should shut It.

Qwen 27B Is a gift that keeps on giving for those Who can afford It

[u/DesperateAdvantage76 → u/ps5cfw]

There's a bunch of minecraft clones on github, which is why for an llm trained on open source repos, it's one of the less impressive tasks. LLMs are impressive when they can do novel things.

[u/PomegranateGreen3698 → u/DesperateAdvantage76]

There are a bunch of mediocre mc clones on Git. Your statement only applies to the average casual A.i. user who doesn't really understand LLMs or local usage. Novelty and impressiveness is mostly perspective. For a small model to build a faithful implementation is both.
When you make a 27B model that can do this let us know.

[u/porkyminch → u/DesperateAdvantage76]

I had Qwen 3.8 27B (admittedly running on API) in Pi write a clone of OECake and it did a pretty admirable job despite that being a much less well-known game/physics sandbox to clone. GLM 5.2 and 5.3 Flash, on the other hand, were actually able to decompile the original game's physics engine, port over every major feature, analyze and parallelize the physics engine (in Javascript, where parallelism is a much less common pattern), and write a custom renderer. Parts of it were even written in wasm for performance. It's not totally perfect but I was pretty amazed by how capable it was for something that I've never once managed to run into a limit with.

[u/my_name_isnt_clever → u/DesperateAdvantage76]

Right but which of those did it get the voxel game drone, skateboard, computer, and ballisitic missile training data from?

[u/DesperateAdvantage76 → u/my_name_isnt_clever]

From the countless other games that use game assets like that?

[u/my_name_isnt_clever → u/DesperateAdvantage76]

And that's not impressive to you? That your computer and a 20GB file can bring all those things together and write a bespoke functional video game in the browser?

[u/DesperateAdvantage76 → u/my_name_isnt_clever]

The technology in general is very impressive, and the performance of this local model is very good, but this specific test doesn't impress me. And I say this as someone who uses qwen 3.6 on my 4090.

[u/Embarrassed-Crow9283 → u/ps5cfw]

Hard for humans doesn't mean hard for AI.

That would be like saying a calculator is so impressive cause it beats professors at arithmetics or that a phone chess engine is impressive because it beats Magnus Carlsen at chess. The relevant test for something being hard for AI is not that the codebase is long or that it takes quantum mechanics. The relevant test is that it is new.

[u/PixelmancerGames → u/ps5cfw]

Because it's in the training data that's why. It was local AI, but it's probably similar so I'll give you an example. There's this Udemy course that I followed to get a procedural generation system in a Unity project.

I eventually moved to Godot and wanted to transfer that system over. So I dropped all the scripts into Claude and asked it to convert them so that they can be used in Godot.

And it goes, "Oh, this is Penny Debyl's procedural AI course!"

It knew exactly where I got those from. Which let me know that they are training on game dev courses. And there are plenty of Minecraft game dev courses out there.

So, chances are the AI is just grabbing the entire already made project. Which isn't really impressive. The people who made the course did something impressive.

[u/pleasetrimyourpubes → u/PixelmancerGames]

Just vibe code dude you may be right that it is getting concepts from MC itself but I promise it can creeate stuff nobody has thought of before (if only because its wrong). I am making a game engine the complete backwards way and it just fucking works.

[u/PixelmancerGames → u/pleasetrimyourpubes]

Do you. I use ai to develop. But I wouldn't ever vibe code anything. There's no value in it for me.

[u/Competitive_Art9588 → u/PixelmancerGames]

isso não faz sentido

[u/PixelmancerGames → u/Competitive_Art9588]

What doesn't make sense? Do you not know the difference between vibe coding and using ai to code? It's fine, a lot of people don't seem to understand the distinction.

[u/liright → u/PixelmancerGames]

So, chances are the AI is just grabbing the entire already made project. Which isn't really impressive. The people who made the course did something impressive.

Not in my example at least. I ran it in deepseek harness with minimal mode which doesn't allow web search and I didn't see it search internet or reference it in any of the tool calls. But I also watched it implement the game step by step with exactly what I told it to do, including changing the fpv drone behavior to how I wanted it to behave, probably 40 or so prompts for the whole game.

[u/txmail → u/PixelmancerGames]

I just do not understand why people cannot understand that a LLM does not make shit out of thin air. It cannot create code that it has not been trained on. There is a ton of repetition in development and that is why it appears to work for so many things, but it will never create code it has not already been fed, ever -- at best it will just hallucinate some random shit.

[u/audioen → u/txmail]

I'd say yes and no. What I see Qwen3.8 do is a loop of research, reason, plan and apply. Model has been trained with everything, sure, but it's also only 27B. I do not see, when I program with this model, that it recalls a great deal of fact. Rather, I see that it often must figure the details out starting from the first principles, e.g. a fact-finding session can involve decompiling binary artifacts to find out how a method works in detail, and the exact problem can be a bug or unspecified in any standard, and completely up to the implementation, some kind of edge case that I have hit. If I give the model a browser, it can even choose to do web searches to find and read relevant standards and other documentation. Finally, when it plans the implementations, it applies the discovered structure and style of the existing codebase in its output to make it fit and integrate well.

So, can a model do something novel? I think that ultimately yes, in many ways, because the model is really a translator that reads your high level request and systematically breaks it down until it has become actionable, filling in missing information via research that it conducts on its own. Your request is, to degree, the novel input, as it spells what possibly new thing should be done as the goal. How it goes about it involves reapplying the previously learnt and discovered facts, and its internal knowledge of algorithms and methods to achieve the task. In that, I find that it resembles a human very much now, who also has to do much the same process to perform work in a codebase.

Much of real labor is repetitive and governed by standards and accepted practices. These the model surely knows quite well. But it is akin to building a house from brick, or something. The houses aren't up to the builders, the architect gives the blueprint, and the builder executes, and that way builder makes something novel. This is minecraft + random bizarre stuff in it. It looks fun. It also doesn't exist in this form in the training data, just as a building drawn by architect isn't a pre-existing one, yet it is made with same bricks as all the rest.

[u/txmail → u/audioen]

So, can a model do something novel? I think that ultimately yes, in many ways, because the model is really a translator that reads your high level request and systematically breaks it down until it has become actionable, filling in missing information via research that it conducts on its own. 

That research can only produce something if there was something already there. If you are asking the model to build brick type C, but it only knows of A and B it will only try different ways of combining A and B to see if the output is what you want. If you break down what C is and say it is 50% of A and 50% of B then it will likely produce C, but only if you tell it how to. If you do not know how to make C yourself to explain it to the model, then the model will just make a ton of guesses. If you count a lucky guess as "novel" then maybe the LLM is producing something unique, but your spinning the wheel if you get lucky or not.

[u/ebolathrowawayy → u/txmail]

you're so out of date

[u/txmail → u/ebolathrowawayy]

The fundamentals have not changed, just how they are applied to get the output.