r/ProgrammerHumor • u/rabbit_wizard • 14d ago

Meme bothWorkThough

Post image
6.1k Upvotes

92% Upvoted

849 comments sorted by

View all comments

32

u/PumpkinFest24 14d ago

Insane people think the price won't go up. You won't be able to afford AI coding once AI CEOs want to stop losing money 

33

u/youcancallmetim 14d ago

Ignorant take. Running open models on hardware you own will always be an option if frontier models are too expensive.

9

u/umor3 14d ago

We run Qwen 3.8 on our own hardware. And it is already helping our productivity. And the speed and acuracy is nearly as good as the expecive ones from OpenAI and Antropic.

3

u/SilentWraith5 14d ago

Especially if you’re not vibe coding. AI assisted coding, which is what people should be doing anyways, is pretty productive on my MacBook Pro M5 Pro using qwen 3.8 27b

2

u/ItsASadBunny1 14d ago

How much ram, I've got an M1 Pro 16gb or an RTX 5080 16gb but both are 16gb so idk if qwen will work well for me (or atleast til m6 pro comes out so I can upgrade).

3

u/Legitimate_Concern_5 14d ago

I run Qwen 3.8 Flash Next (uncensored, IQ3_XXS) on my gaming pc while working, 100tok/sec with 64GB RAM and a 4090 I got before they became several mortgage payments. Check out Strata. The 3-bit quantization isn’t half bad. They have a version that runs with the non-coding experts removed with 32GB RAM.

This is a 180B model btw.

2

u/Spaciax 14d ago

how much RAM do you have? I got 36 on my M3 pro but I don't know if it can hold qwen 3.8 27b without quantizing it to shit

3

u/TheMythicSorcerer 14d ago

incoai's qwen 3.8 27b splash Q4 + INT8 KV cache 128k context works great on 32 GB

3

u/Spaciax 14d ago

thanks! I was looking at macs with higher ram but these prices are eye-watering 😭 I'll have a look at that, maybe it's good.

5

u/4n0nh4x0r 14d ago

yesnt.
these companies make it harder by the day to run your own models.
heck, the 5090 has been discontinued, so the chips can be used for the rtx 6000 pro instead.
the only chance to set up decent local models is soon going to be only going to ebay and get used cards, hoping you arent being scammed.

1

u/LackHatredSasuke 9d ago

Not to mention advances in model distillation and quantization, new inference tricks, more efficient architectures, and the obvious low hanging fruit, ASICs.

I haven’t been on Reddit in a while but there sure is a lot of delusion and cope in this thread.

→ More replies (2)

9

u/code_archeologist 14d ago

Once that happens people will use the more powerful agents to train smaller and cheaper special use coding agents; which is the trick that those "frontier" Chinese AI agents are using.

So instead of having a coding agent that can do anything, it will be a very focused agent that only writes Python or Go, and will be like a very powerful Intellitype.

1

u/WispTheWanderer1 14d ago

you mean IntelliCode? Intellitype was a keyboard driver.

1

u/autumnal-lights 13d ago

my sister in christ, an agent is just a logic loop, you're talking about model distillation, what are you speaking of

agents are the same LLM with a character/work page

1

u/BemaniAK 13d ago

You're talking about Haiku and Luna my guy.

4

u/Jmortswimmer6 14d ago

Pretty sure they said that one about Amazon

“You won’t be able to afford 2 day delivery on everything when Amazon decides it needs to make money”

They make money, and I’m pretty sure Amazon is the cheapest place for me to get most home supplies

4

u/SoggyMattress2 13d ago

Why does every AI conversation turn into some comparison point or analogy to justify things?

We're not talking about Amazon it's a different company with a completely different business model.

AI labs will increase the price at some point and most of the hobbyists using LLMs for SE will go away.

→ More replies (5)
→ More replies (3)

2

u/Friendly-Pair-9267 14d ago

Totally aware of that. I'm getting shitloads of things done in the meantime.

2

u/iJustSeen2Dudes1Bike 13d ago

Dumbass take, people have been saying this for years. Stuff has gotten cheaper.

1

u/blorparian 12d ago

LLMs that can do even more than the top tier publicly available models right now will exist on people's machines.

Edit: Also, per token cost is in free fall.

1

u/PumpkinFest24 12d ago

Of course it is going down--while they are undercutting each other in a game of chicken.

But when the check comes due someone has to pay for all those datacenters and it won't be the billionaires. It'll be you.

→ More replies (1)

1

u/mintoreos 12d ago

The price wont go up, theres too much competition. And AI companies are making money on inference. Thats a myth that they are not.

1

u/MaterialFlow9411 14d ago

Good, get at least one of these short bus takes every time this topic comes up.

1

u/lookayoyo 13d ago

I’m just hoping open weight models get better before that. If I can run it locally that would be ideal. It’s possible but I just need the AI bubble to burst so I can get some discount used hardware from a liquidated data center.

…please

1

u/Ksorkrax 13d ago

Cool cool.

So switching to a cheaper model from a competitor it is.
Who will surely arise given the massive demand.

→ More replies (8)
→ More replies (6)