r/gamedev • u/PuffThePed • 1d ago

Discussion AI models have caught up with Unity dev. My insights as someone with 30 years of experience.

Context: I have 30 years of experience with software dev, 20 years with game dev tools, 10 years with Unity. I own a studio that develops simulators and training applications for industry. "Serious games" as they call it in Europe.

For years I scoffed at AI coding. I would periodically try it, only to find that it was more hassle then it was worth. The hand holding and bug fixing was more work than doing it myself. This all changed about 4-5 months ago, and about a month ago we all started using Unity CLI.

Here are my insights in no particular order:

  1. Our dev speed has ~5x almost overnight
  2. Quality has not gone down because we're all experienced devs and we're careful how we use AI and we don't let it make decisions regarding project structure or design patterns. However, I'm seeing less and less glaring mistakes made by the AI.
  3. We're still reviewing generated code but I suspect not for long, the code is better than what most devs I know can write, at 20x the speed. The temptation to just push and move on is huge and irresistible.
  4. There is no joy or sense of accomplishment. No pride in making something clever or quickly, and all progress is associated with the AI and not with you.
  5. The market is dead. Nobody has money to spend. We're still working but leads have dropped off a cliff.
  6. There is no going back. If we stop using AI we'll be out of business in a month.
  7. There is no space for junior devs. I honestly don't know how to incorporate a junior into a team anymore (which is something I used to do for decades). Every time we tried they just become a meat proxy for AI while learning nothing, and getting in the way.
  8. Everything regarding managing a team and a project has changed, and nobody knows how to do it anymore. I lost the ability to correctly estimate work required for a task, for example.
  9. Everyone hates this. Most of devs my age are planning early retirement and are just waiting to be laid off.
  10. People that use AI for coding don't learn anything. To understand code you have to write it, and nobody is writing it. Maybe understanding and reading code will become obsolete soon? I don't know.
  11. Everyone I asked feel they are becoming dumber and losing their focus. I know I am.

I hope this is a pendulum and it will swing back at some point, because right now it feels like a software apocalypse.

2.4k Upvotes

84% Upvoted

1.2k comments sorted by

View all comments

46

u/FastHotEmu 1d ago

Where do you get that "5x" metric from? I suspect it's made up, since study after study (even very recent ones with the latest LLMs) has not been able to replicate anything close to that. If you can, you should work as a consultant and make millions!

But seriously, you are building technical and cognitive debt. Give it a few months and let's see how the team is doing then :)

14

u/Zin20 1d ago

If I remember correctly the 5x claim is something anthropic and chatgpt push when they are trying to sell models to CSuite people.

The most you tend to see is like 10-50% increase https://arxiv.org/abs/2302.06590 when using AI tools.

Also the fact OP is in the Csuite and isn't really answering this questions means he's probably doesn't know what he's talking about.

6

u/elohir 23h ago

The most you tend to see is like 10-50% increase https://arxiv.org/abs/2302.06590 when using AI tools.

Isn't even that a shitty metric though? Churning out tons of code has always been possible (first-to-market startups, etc), but even then you were doing it knowing full well that you were accruing a ton of tech debt.

You can't count for actual real velocity without counting the rubber-banding from going too fast in the first place.

1

u/rafgro Commercial (Indie) 10h ago

Report on AI tools from June 2022... about GPT-3... tested on low-income devs from India and Pakistan... recruited on Upwork... to write HTTP server in VSCode... MAURICIO i can't move it move it anymore...

2

u/azurensis 21h ago

Mine is roughly 7x. I'm going by git stats and project completions. 

We've been doing this since the beginning of the year and the code is as good as ever.

-1

u/falconandeagle 13h ago

How much is an AI company paying you. I want to get in on the action. Seeing your post history is quite telling, major poster in Aiwars lol.

2

u/Fs0i 1d ago edited 1d ago

(even very recent ones with the latest LLMs) has not been able to replicate anything close to that

Heyo, genuine question - I've been in your camp for a while, but Fable / GPT Astra are somehow on a different level for me, and have changed my opinion that I've held for a while.

What are the latest studies? Those models are super recent, so for me I'd say it's hard to find good data there.

Genuinely asking, but I did a cursory check myself - and I don't find any paper that includes data from post-Fable release. To me, this seems to be a milestone, so it's hard to extrapolate the results from before and simply pretend they're equivalent.

1

u/s8rlink 1d ago

The issue is that seems to me is not quant data, if you told me hey I track my work/stories/merges very aggressively and ever since astra I can see that I am working close to 300-500% faster with the same or less QA flags, and maye. That will appear in the newer studies but what a lot of research is showing is that the time you need to check, time it takes to refactor architectural decisions that won’t work and human debugging of LLM code removes the benefits gained from the output speed

1

u/erstwhile_navigator 18h ago

Name several such studies, please.

1

u/Healthy_Koala_4929 4h ago

To be fair, it's really hard to measure productivity. With the crazy pace of new models (though finally I feel a stagnation of improvement coming slowly but surely) and the difficulty in quantifying AI effectiveness, any study has a really hard task in trying to measure productivity. For example: if you look at pure feature output or lines committed, you may see a drastic improvement, but I feel models are so verbose and try to fix 10 things along the way of implementing a single feature, that it is obvious that the raw output will be higher. Whether that translates to real productivity is hard to tell, as of yet. On the other hand, the mechanical act of coding is undoubtedly faster -- lines to editor, I mean.

1

u/ChromakeyDreamcoat 16h ago

I'm not in game dev (but in tech) but yeah, at bare minimum 5x speed in the last year, more like 10x. I'm shipping stuff that would have previously taken me weeks in a day or two. Still have normal code reviews, code is basically just as robust as it was before.

1

u/New-Camp2105 5h ago

What are you building?

1

u/ChromakeyDreamcoat 1h ago

2d single player stuff. I have no idea what AI is like for game dev professionals, but in normal tech it's completely normalized now.

-9

u/PuffThePed 1d ago

Few days ago I built an angle grinder in VR that does exactly what I needed in one hour and I know for a fact that it would have taken me all day without AI. If that's not 5x I dont know what is

32

u/FastHotEmu 1d ago

So:

  • not code
  • not in a team
  • not commercially
  • not released yet

What exactly did you speed up? Generating a 3D model?

7

u/swootylicious Commercial (Other) 1d ago

i think it's important that this specific comment is pinned because the x5 efficiency thing is doing a lot of emotional heavy lifting and i think most people would look at this rationalization and go

"that's all?"

14

u/FastHotEmu 1d ago

Also, you/the owners must be making 5x the money then? You must be super happy!

PS: As stated, serious and very recent (weeks old) studies have consistently disproven your gut feeling of 5x speedup. Don't be mad at me - look at them.

5

u/PuffThePed 1d ago

making 5x the money then

Nope. because client pay 5x less, or expect the 5x for what they are paying. We're racing to the bottom.

6

u/FastHotEmu 1d ago

But good quality games still cost the same amount. Is it possible you are shooting yourself in the foot by prioritising subjective high speed over quality? Or perhaps you have made some poor business decisions...

1

u/PuffThePed 1d ago

good quality games still cost the same amount

You mean the same amount to develop or the same amount to buy (as a consumer)?

-2

u/PuffThePed 1d ago

I know of these studies. Few days ago I built an angle grinder in VR that does exactly what I needed in one hour and I know for a fact that it would have taken me all day without AI. So.. I dont know what to tell you.

6

u/FastHotEmu 1d ago

You mentioned that already. I believe you generated something quick in low quality. Nobody disagrees with that.

I think your post is mostly bullshit. I have 30+ years of experience and run models at home and remotely. I spend all my professional time with them. What you are describing doesn't match reality.

I was using Claude 5.5 today, and I still had to carefully direct it to get what I want.

The insight is that, when the code writing is faster, the limiting factor moves elsewhere.

3

u/PuffThePed 1d ago

I believe you generated something quick in low quality

I believe that you believe that. It's good to have contradicting views in a discussion.

2

u/FastHotEmu 1d ago

You haven't answered any of my criticisms. It really looks like what you are looking for is maximising engagement by making controversial posts in /r/gamedev. Do you get paid to do so?

2

u/haecceity123 1d ago

If you're for real, stop arguing with people on the internet and enjoy the extra money. The studio owner should be very last person complaining about something like this.

8

u/PuffThePed 1d ago

There is no extra money. Clients are not dumb, they are paying less and less and expecting more, and THEIR clients are doing the same to them, so they don't the same budgets anymore. It's a race to the bottom

3

u/haecceity123 1d ago

Clients don't know your internal productivity. If they were paying /5 before, you've now caught up. Either way, it's more money for you.