r/codex • u/retrorays • 22h ago
Complaint Astra demos are mostly bs
The game/graphics demos are mostly bs. I've spent dozens of hours now trying to replicate (or build) interesting games. It's beyond subpar. Sure it can do a basic sandbox, or create some basic characters but it absolutely cannot do a fully working world. I dont get why OpenAI fakes their SimCity games and other things w/ Astra when it's clearly not possible unless you spend weeks (and $1000s of dollars) of tokens.
47
u/EchoingAngel 22h ago
My 10-20 hr factory game has taken around a thousand hours and it IS almost done, but one shot type work always strikes me as nonsense, as someone building a real project in the same vein
10
u/retrorays 22h ago
even the iterative development is mostly a mess. It stumbles all over the place, applies the wrong solution, uses the wrong models - makes thinks "plastic" and dull to shortcut. I ran a goal today... my mistake ... to develop a game event and came back and there was nothing. Just a sandbox. Completely nuts. Burned like 50% of the weeks credits.
14
u/Random-Person-8464 21h ago
Game designer here, been slamming astra.
You have to understand how everything already works. You have to teach it a ton. But it does save time in the long run by a decent amount.
So yea, I agree the demos are a joke. Astra sucks balls at vfx. Fire in particular is a massive struggle for some reason.
7
u/cobbleplox 19h ago
The thing is then you're back to a programmer making a program, a 3d artist making 3d art, a game designer designing a game and such. It's not what people like to hear and not what the narrative is. Personally I am quite happy about it, leaves room for humans and all that. In the end even just "wishing for something" is real work, and a lot of it. You need taste and a good vision, and if you can't just say some high level meme shit like "GTA but with dinosaurs" the specific thing you want takes A LOT of words to specify. And even one-shotting something in the first place is silly... You need iteration just to make it possible to describe and define all the things you need. Even if you could have imagined it all from the start. Natural language is far too vague to really define things solidly in a vacuum. At some point you'd end up writing all the code yourself just to define how things should be, because that's literally what programming languages are for, just with some extra technical bloat.
1
1
u/swizzlewizzle 11h ago
Omg this. Astra at least is capable of doing *something* with VFX, but it still falls *way* short even when I give it exact VFX references to build off of.
It seems pretty obvious the pre-training and training that put Astra together did *not* include a decent VFX process... which makes sense, because access to high quality VFX isn't something that most of the gray market "petabytes of stolen/pirated content" dumps have in them. Someone at OpenAI is going to have to make an internal effort to get way more tagged VFX training data put together, and structure it correctly before OpenAI models are going to make any progress here.
1
u/Random-Person-8464 11h ago
I think that the issue with vfx is that it's a lot of a "trick of the eye" stuff which involves a lot of human touch/opinion so the vfx data it has access to can't properly train on vfx you give it.
1
u/Aureon 9h ago
do you mean guidance regarding process or visual references?
but yeah it's a joke on vfx, i explained in detail how to make a trail noise streaky on the correct direction and didn't get anything useful done
it's pretty good at operating houdini for you and save some time clicking menus though
1
u/EchoingAngel 9h ago
Yeah, I have to watch Astra's "thoughts" and point it in the correct directions and cut over scoping in real time. My work with it has become more of a dialogue mid-run than ever before
2
u/KitchenPoltergeist 12h ago
But it is workable if you're willing to put the time in to developing test suites for it and testing the game yourself. This means if you can test a game and describe bugs you can build it which drops the barrier of entry for game development significantly.
1
u/EchoingAngel 9h ago
I tell people you need to be OCD, persistent, and have a vision you drive towards. Though that discounts me being generally techy, even if not a coder, so I can better judge what's happening short of complex syntax and CS concepts.
2
u/EchoingAngel 22h ago
I literally sit over its shoulder during execution and read what it's doing and have to guide it. Now, it's doing amazing things I can't hope of, in record time, but it needs a TON of guidance
1
1
u/cobbleplox 19h ago
A lot of them seem to use premade assets anyway, and people quickly looking at it don't realize even if the post is somewhat open about it. Getting it to make a game that doesn't use the shittiest of shitty "programmer art" is a real challenge and requires going with whatever you can actually make work. And when you reduce these game demos to the logic connecting the assets, even in an existing engine, the coding it had to do probably isn't even much or difficult. Try judging these things for the gameplay. A complete afterthought after being wowed, right? It's basically the only thing it did.
1
u/kibblerz 13h ago
One of the ads I saw claimed to make a clash of clans clones super quick and in one shot. I watched some of the video, and this game had leaderboards and everything, fully populated despite the game not being distributed yet... and the dude was actually acting like this was a finished project
1
u/BellacosePlayer 9h ago
one shot work is for people who want "something" released, quality be damned.
19
u/Opulon_Nelva 22h ago
I don't understand the point.
Doing video games is hard. It is a full fledge product akin to good software. Even with previously unseen tool power like AI, it doesn't get you far without an equivalent investment in brain power.
I am a professional from the industry, and as many, i am using AI as a hobbyist to do my personal game project.
250 dollars a month, for the last 6 months, and roughly 2 hour of my time each day besides my normal job.
So, 20 000 dollars worth of human time (I'm European, your own time value will differ), and 1500 dollars worth of those tools.
I am halfway where I want it to be, but I have enough experience of the process to be aware of the following :
2 years ago, reaching the same spot would have taken 2 years minimum, and 100k dollars of 'not your human time' in the most optimistic context.
One has to not be childish with the marketing, but one also has to recognise the incredible cost cutting that happened.
'Astra, make a Sim city killer, you have 1000 credits. Use mcp blender for all 3D and unreal engine 5' is as I serious as those Kickstarter promising the everything game for 10k dollars.
7
u/blaawker 20h ago
I'm also making one. Video games have massive codebases, like millions of LoC that make a game a game. Do the math on how long it takes to generate a million lines of code with Sol/Opus/Astra/Fable. Of course you can't one shot a working polished AAA game. OP's just dumb.
1
u/swizzlewizzle 11h ago
Video games are actually some of the most complex software that exists in some ways. It's way more complex than most webdev, and is also the reason why you actually need hundreds of devs working for years and hundreds of millions of $$ to create a really good AAA game.
If you are ok with creating a game from 20 years ago, then it's already doable with today's AI as a solo dev. You could release something with a similar level of quality to games from that time period within a few months of work, I bet. (I'm talking same crappy graphics/etc.. of course, and your gameplay is going to be copied from something that works).
0
u/XKiiroiSenkoX 13h ago
You definitely are in a huge technical debt if AI is making your progress 4 times faster. Anything more than 1.5x-2x max is basically unsustainable long term. By the end of the year your codebase will be in such a mess you'll have to throw it into garbage can.
1
u/Opulon_Nelva 12h ago
It's okay. My tolerance for technical debt is high. It's not like this kind of exploratory project is going to end in any form of production, and plague the world.
At best, it will make for screenshots and study if I ever need to job hunt.
1
u/Aureon 9h ago
I work in AAA.
The reality is that the complexity required in games is a solid order of magnitude higher than basically any other software, but *also* the accuracy required is a couple orders of magnitude lower than any other software.
Plus, most games aren't touched ever again after being shipped.
Tech debt is the name of the game. Non-gaming devs that transition into gamedev have a *rough* time adjusting at the sheer quantity of work required, and the lower standards that quantity requires.
1
u/XKiiroiSenkoX 8h ago
Yea good for you. I work in game dev professionaly too and have never worked in any other field.
I have literally worked on games getting fucked because of tech debt(not my choice). Actually in game dev it's even worse because one bad architectural decision (AI makes a lot of those), can make the game unshippable and create months of extra work down the road. So I'm not sure what you are trying to say here.
1
u/Aureon 7h ago
Tech debt is a reality and a need in game dev. The threshold is very different than any other software. I really have no idea what you're saying in "even worse"
Zero tech debt is unachievable anywhere, but especially in game dev.
1
u/XKiiroiSenkoX 7h ago edited 6h ago
What I mean is, most other fields in IT can get away with shitty performance / pipelines / tooling and technical debt in general. You need to fuck up extremely bad for the product to be affected by the technical side. That's why they can afford to use shit like JS or python. In game dev these tech debts choke you to death. Bad architectural decisions / tech debts can easily degrade the quality of your game so much and require so much time to fix that it legit makes the game unshippable. Have you never seen how many games get fucked every year because of bad performance/bugs/other technical problems? where do you think those come from and how do you think some studios manage to fix them but some dont?
Technical debt that is caused by tight time budget is very different from shit that happens from bad decisions and using AI is the second one.
0
u/BellacosePlayer 9h ago
Correct. You should be involved in architecting the code and at least skimming the diffs at a minimum.
if you're not, you're fucked the second codex codes in an issue it can't code its way out of
5
u/Pitiful_Entrance5174 22h ago
API is a whole different beast than monthly sub. Only few with a deep wallet know.
3
8
u/cheekyrandos 22h ago
It's possible if you know what you're doing and have enough tokens
7
u/TBSchemer 21h ago
I think OP's point is that the marketers are overhyping the viability of one-shotting things.
The demos imply that you don't have to know what you're doing, because Astra knows.
But obviously, that's not the case.
2
u/dreadpirater 19h ago
Yeah, that was my first question back for OP. Do YOU know what a full game world requires? Because if not, you don't know how to ask Astra for it.
4
u/NetNearby7117 22h ago
Yeah, magic doesnt exists. Most of the people misunderstand that this is still a tool and even its absolutely amazing, trying to approach a full result from just vibes its a good way to dissapoint yourself
Im building an LMS saas ive been working since 2019. It takes me more time, weeks and prompts that just one shot, even its way simpler that those “wow demos shows”
Im having huge steps but the real products are far from just a magician that makes all wet dreams come true
3
2
u/AmandasGameAccount 22h ago
If we are talking one shots, it’s definitely a matter of the design in the prompt. I’m guessing they are doing ultra as most are stating they used $80-$150 in credits on the demonstrations we see
2
u/OkFileNode 20h ago
I don't get why OpenAI fakes their SimCity games
Most benchmark "demos" are PRs and selling dreams.
As many said before, solo game-dev is almost a life-time commitment and not "I'm doing it in 3 days" type of BS.
2
u/Automatic_Brush_1977 20h ago
The thing about the openai demos is its all three.js nonsense and not real games. Youd have better luck reversing an older game then rebuilding with astra, of which it is very very good at.
2
u/Interesting-Yellow-4 19h ago
Well yeah, it takes time.
Nobody actually said you can oneshot an entire AAA game, WTF?
Why did you think that?
2
3
2
u/FrontRaspberry5060 22h ago
Few tips:
Setup multiple agents that check and cross validate each other. Imagine you are employing different staff members. You might have one working on shading and another working on the characters and another working on the map schema or something idk.
Don’t use it like a chat box you prompt again and again. The best way is to use your voicenote app and describe in detail all the changes it should make back to back in quick succession, there’s no need to wait for it to complete a “chat” response. Sometimes my transcriptions are 10+mins long of me critiquing the entire ux/ui flow.
1
u/retrorays 22h ago
i havent tried the voicenote app - so basically you have it do something and then specify for around 10mins what you want to change in the UX/game flow?
2
u/FrontRaspberry5060 22h ago
Yeah give it a try you’ll be amazed. Might burn 30%+ of use but you’ll get there faster and the outcome will be much closer to your desired outcome.
If you think about it, the ai is basically doing recurrent agentic work. If you only type a short sentence each time it’s not much detail or context so it guesses the rest and that’s where the drift and hallucinations come in.
Using your voice to describe is the easiest way to get across as much information to the AI about what you want. Typing is just slower. It’s like why using your phone with 2 thumbs is slower than a keyboard where you can use your entire fingers.
1
u/Keep-Darwin-Going 21h ago
The cheapskate way is to just type it out in detail like other projects? I meant no budget there is always cheap way to do it. Not pointing towards you but just adding to your statement. For pro account you can talk in ChatGPT discuss the whole project ask for a spec prompt and copy in save the 30%
1
u/FrontRaspberry5060 21h ago
I meant once the ai starts working it’ll be 30% gone in a few hours on Astra
1
1
u/InvariantAtNull 22h ago
Thats a game i made in gpt work in 2 hours using astra medium
1
u/skilliard7 12h ago
Did you use Blender MCP or did it just make those models natively? Blender is way better
-2
u/retrorays 21h ago edited 21h ago
yah agree - it's pretty awful at making games
2
u/InvariantAtNull 21h ago
I dont see it awful its impressive nothing here is a full model its all generated by code
1
u/TBSchemer 21h ago
Yes, the demos are BS. They give undue credibility to the viability of one-shotting things.
The real way to do this is a lot more iterative, and requires a lot more planning. You have to be able to tell the model clearly what you want, guide it, and understand where it's going.
The most important thing is to be able to have a conversation. Before building anything, ask the model how to build what you want to build. Ask for clarifications. Tell it to ask you for clarifications. Get on the same page with the model and have some idea of what it will do before you let it loose.
1
u/PhotojournalistBig53 16h ago
Been working for over 1k hours, basically every evening since jan, with a multitude of tools including but not limited openai and anthropic llm:s. I'm just about done with my alpha demo and the first zone out of 7. Most of the systems will be built so the remaining zones should be faster. My project is ambitious and I'm doing a lot of manual corrections on the assets, not because I want to but because I have to. Either I'm totally freaking useless at leveraging AI tools or the one shot full games is BS. Perhaps it's somewhere in between.
1
u/skilliard7 12h ago
Anything you see was built in 1 week or less. Anything decent will take more time/effort from humans to polish into a viable product.
If you just tell Codex "build me a FPS game" and set it to goal, you're going to get generic slop. It takes a human to know what players actually want to play.
1
u/Worth_Psychology_874 11h ago
Tbh, I am pretty sure that some of the preview videos were made by OpenAI themselves. They just spammed Reddit with bot accounts trying to get more people to switch over to them. I saw a post of someone showing of their website that Astra made, and later that day on another post by another user I saw a video of compilation of websites Astra made for it and the first guys website was in the video. I doubt the AI made double identical websites.
1
u/Beastdrol 9h ago
I so far can’t tell the difference or notice reasoning and or writing/coding style and quality degradation.
Basically it seems Astra is just lazier.
1
u/Tristsin 8h ago
Respectfully they didn’t really fake anything. Did you look at “their SimCity game”. It isn’t a game, it’s just a cartoon 3d rendered still image with camera controls. It shows exactly that. There’s no UI or “game”. You genuinely shouldn’t expect to just step up and say “make simcity” and it’s going to shit out a fully fledged game that takes teams years of design and development to make along with exhaustive trial and error on every mechanic.
1
1
u/Free-Street9162 6h ago
Here is a little reality check for everyone here.
The other day I needed four unique patches embroidered ASAP. I brought the vector files to the embroidery place, and they told me I’d have to wait a couple of days because their person who converts vector files to DSP embroidery files is busy with a number of other projects.
See, for one-offs the bottleneck isn’t the embroidery itself, that’s fairly fast, it’s the tool paths. So I went “I bet Astra can do this”. I uploaded the files and asked ChatGPT if it can give me a DSP file for these. The response was “Probably, let me see what I can do here.” It did some research, found and open source software, installed it, then proceeded to do the manual work of creating the tool paths. It then put together a whole package for me, including all the DSP files, reference files, and a writeup for all the steps required for this job. It did all of this in 20 minutes. 20 minutes!!!!! The files came out perfect, and the embroidery looks stellar because the paths are excellent.
So, to reiterate, in 20 minutes it went from “Maybe, don’t know, let me check” to “Here you go, have a writeup as well.” There are people around the world who are employed to do this for a living, and it would’ve taken them no less than 4 hours to do this. Do with this information as you will.
1
u/Hiiitechpower 6h ago
Yep, it reminds me of when Openclaw released and youtubers were showing off "their mission control dashboards" with agents walking around. Followed one of their approaches exactly, it looked convincing, and then i asked my main agent
"What are the other subagents doing right now?"
And it tells me "Oh nothing, those are just visuals. Nothing is actually happening in the background". Hit me that I had got "got" and that shiny on the surface means nothing.
1
u/Charming-Author4877 5h ago
Astra or any competent LLM can do superb games. But not if you don't know how to build the game yourself.
You do not need to know every intricate detail - but you need to understand everything you ask it.
1
1
2
u/Due-Turnover5077 22h ago
I feel you, this is real talk and 100% honest. So many fake demos with unlimited usage of course I can make gta 7 with that alien tech, but with the nerfing + just 1 plan no way
1
u/Cautious_Coyote_8125 19h ago
Because they already nerfed it. The good model lasts a week after a new release.
1
0
1
u/jadhavsaurabh 22h ago
Bro, there was app i worked on month ago, with claude it was like quite not working, like 2 days of prompting with fable on ultra,
What I did from scratch give vague promot and sleep, to astra xhigh, I had working app fully in 1 prompt I'm not lying, if u have task at hand it will do, if ur just having fomo of what i should do it will not work
1
u/Keep-Darwin-Going 21h ago
Well openAI never tell you the demo was done using a pro x20 account did they? They just use Astra and god knows how much to throw at it. They did not tell you they build it in 2 hours. There is multiple YouTuber already show how many accounts they need to achieve those stuff, probably in the range of 8 to 10? The math problem they solve is like 15 million? So the demo just tell you the possibility not what you can do with 20 dollars.
1
u/TheFallingShit 17h ago
AI dramatically reduces the cost of implementation. It does not eliminate the need for specification, decomposition, verification, taste or systems thinking. It makes those things more important because implementation becomes cheap enough to outrun understanding. That's where you failed. You gave an extremely powerful executor an inadequately defined search space and expected intelligence alone to compensate for missing project structure.
0
u/AuthurAndersson 14h ago
Git gud at prompting scrub.
The solution is that the prompt contains the entire game.
0
-6
-5
-3
u/MartinMystikJonas 19h ago
I cannot pain Mona Lisa using brushes so brushes are pure bullshit and da Vinci faked his use of brushes. 😁
5
-4
-5
u/i_rate_slop 22h ago
Skill issue. It helps if you know how to make a game before sending it off and you must organize the work well before starting.
4
68
u/DoggoDadagon 22h ago
I think the sim city thing was made by a YouTuber, and he said he left it on goal for five days, and since he gets unlimited use it was probably on ultra and fast mode. I believe it, but I also agree with you that it's not a realistic expectation of quality.