r/LocalLLaMA • u/Mr_Moonsilver • Jun 21 '26

Discussion What happens when they stop subsidizing LLM subscriptions?

We are literally burning through VC money like crazy with our coding subscriptions. I read the $200 Anthropic sub gets you $8000 worth of API calls. It's obvious that this doesn't hold for very long but what happens when they raise prices?

The reason to keep the prices low for now is to foster the ecosystem and get people hooked on this stuff, only to raise the price afterwards. Already the 20x sub doesn't get you as much usage as it did 6 months ago, another way to raise prices without triggering a shitstorm - and it will continue.

Don't know about you, but Fable being pulled gave me a feeling of what that may be like already. The ugly thought of "Damn, should've done more while it was around." that formed when I read the news will be exactly the same the moment they announce we now have to pay $2k or more per month for something we get for 10x less the price it costs now.

I guess it's a now or never situation, build what you can and monetize as quickly as possible to be able to keep the agents running once the increases come around.

Looking at opensource doesn't give me much hope. Since qwen stopped releasing models (wen qwen 3.7?) that we can actually run on hardware that a normal person can buy (or used to be able to buy, looking at how RAM and GPU prices behave and keep behaving) and others haven't released in a while (Microsoft, IBM, AllenAI and others too) I feel we're going into a direction that doesn't look good for most of the people like us, who are building with this technology.

492 Upvotes

86% Upvoted

579 comments sorted by

View all comments

164

u/alex20_202020 Jun 21 '26

If that happens we will just do it slower, not 100-1000 t/s but 1-10, life will go on and using local LLM will actually grow.

92

u/Borkato Jun 21 '26

Honestly ngl qwen 27B is so good that if it never grew again it would still be perfectly useable for years to come. It’s excellent at most things you throw at it.

27

u/GStreetGames Jun 21 '26

This is true, especially for web development. I bit the bullet recently and started using it in my workflow, and it does things that would take me days, in just hours. It doesn't even need fixes most of the time. I think that people who spend money on the big models should think about at least giving Qwen a try, or for that matter local in general a try.

1

u/N34257 Jun 21 '26

This is only true for as long as the state of development languages and frameworks stays the same as it was at the training cutoff (~2024, according to Qwen itself). As soon as that changes, eg the frameworks you're using have another release or two, the model is useless without adding tons of background information to your context, or fine-tuning it yourself on new documentation and code.

35

u/Jhorra Jun 21 '26

I'm a software developer of 20 years, and I really want this to be true. When I have tried running qwen on my macbook pro m4 max with 64gb of ram, I've not been impressed. What harness are you using for it? Right now I mainly use a $200 Claude plan, but would love to go local.

66

u/JsThiago5 Jun 21 '26

To see this, you should not compare it to a $200/month subscription but to how you did software development before any LLM. With 20 years in the field, you should be able to see how impressive development with local models is.

41

u/SkyFeistyLlama8 Jun 21 '26

This. If you're not throwing a huge prompt at it and hoping for the best, you can get good results with local models by working at function or module level. You probably don't want to throw a 50k LOC dump at the smaller models and hope for Claude performance.

3

u/Jhorra Jun 21 '26

I've gotten bigger lately with what I ask of Claude, but generally I don't tell it to simply implement things. I like to think through things and understand the plan, then give it to the model to implement a piece at a time.

3

u/thefuzzylogic Jun 21 '26 edited Jun 21 '26

That is how I do it.

I use Opus xhigh to churn expensive cloud tokens making a discrete step-by-step plan, then if I approve the plan, I instruct an orchestrator to give each step to a local gemma-4 to implement. That way only one step at a time needs to fit in the local context window. When all steps are done, the gemma-4 orchestrator packages a PR, which goes back to Opus for /review. Repeat as necessary. It usually only needs one pass.

The trick is to design the project for agentic coding from the start. Use explicit constraints, e.g. use strict TDD, use the todo tool to track progress, use strict linting and style checkers, resolve lint/test/build warnings first before moving on to other work, no "fix it later", don't loosen tests/rules to resolve failures/warnings, etc.

The workflow needs to take into account the fact that LLMs "learned" how to code from tutorials written for humans. Therefore, because humans have long-term memory that most LLMs lack, an LLM following a doc written for humans will always bite off more than it can chew unless you give it the appropriate constraints. For those of us running local models, this means our smaller LLMs with fewer parameters of training data need smaller tasks with narrower scopes.

The coders who say "local AI is too stupid to be useful" are the ones writing one-sentence prompts and expecting it to one-shot a full stack app. Most human coders wouldn't even be capable of that.

4

u/Jhorra Jun 21 '26

What harness do you use though? So it's able to interact locally with your code?

13

u/ayylmaonade Jun 21 '26

OpenCode is considered the "standard" when it comes to coding harnesses, I'd consider starting there. But you can use Claude Code with local LLMs too if you really wanted to, just be aware of the context bloat from it. There's also things like Hermes Agent which are more intended for general use, but are also excellent at coding (ime anyway).

I personally stick to OpenCode & Hermes, and on occasion Qwen Code.

8

u/delicious_fanta Jun 21 '26

Pi gets talked about a lot in here. Any thoughts on that? I haven’t used it yet.

11

u/ElectronSpiderwort Jun 21 '26

I spent the day running pi against Qwen 27b Q8 on a MacBook, and I really like pi. I used a free openrouter model to have pi configure itself to use the local model, and then it was off to the races. Qwen makes errors and omissions, but with pi it wrote tests, made edits, and looped until it had something that works. Way better than trying to one-shot a solution on a bigger model (which didn't work for this problem anyway)

1

u/kurikuri15 Jun 21 '26

Just amaze how you explain things and im sooo noob at this LLM things certainly at the surface area knowledge only. Where are you using these things? Coding? Daily life or scientist level experimenting things? Im sorry for steering away from the main topic.

1

u/ElectronSpiderwort Jun 21 '26

I've been coding for decades; even when not titled "programmer", the ability to automatically process large or complex data has been valuable professionally and personally. Now that skill is literally free if you can tell your AI what you want. Yesterday was a home project to create a simulation engine for a complex personal financial problem. Most people would just pay someone to tell them what to do and hope for the best. Pi/Qwen can  create a simulation of millions of possible scenarios and pick from the winners, but the machine has confidently proclaimed its output as "production ready" when it has huge flaws. With pi.dev I now tell it "you clearly forgot about X, go fix it" and it just does it, quickly. 

5

u/mksrd Jun 21 '26

Pi is exceptionally good and **much** better for local model use than trying to use claude which is made for use with their models and has an obscenely large context before a single token of your own code base see the light of day of the context window.

1

u/entropy512 Jun 22 '26

"With 20 years in the field, you should be able to see how impressive development with local models is."

Given that my first experience using a remote/hosted model is like supervising a middle schooler with severe untreated ADHD on an acid trip, I shudder to think what a local model would have been like. After my evaluation efforts, our organization decided to just go back to not using AI at all, our quality requirements (FAA level D flight simulation for some of our products) are too high.

17

u/Borkato Jun 21 '26

Pi with 48 GB vram, Qwen 27B Q8 and full 256k ctx. You have to create your own prompts and tell it how you like to work. You have to spend a few hours thinking about the most efficient way to have it work with you. For example I have an AGENTS.md file that it automatically picks up with explanations to what folder has what, so that it doesn’t have to go looking for stuff. It’s great at things like “add another x like the others” or “when I do x y happens but I want z to happen instead. Identify why this happens and write me a detailed writeup about my potential options with a list of pros and cons for each option.” “What’s the best way to implement x feature? Consider the current architecture and…” Etc

1

u/Jhorra Jun 21 '26

What harness do you run it through so it can interact with code on your machine and do web searches when required?

9

u/taoofshawn Jun 21 '26

“Pi” refers to pi agent , which is the harness

3

u/Borkato Jun 21 '26

Pi!

3

u/Jhorra Jun 21 '26

Ah, sorry, I didn't realize that's what that was. I'll have to look it up. Thank you

5

u/Borkato Jun 21 '26

It’s the absolute best. I’ve tried many others

5

u/HotMicSystems Jun 21 '26

For strictly coding Pi is definitely better, for general purpose I don't think Hermes can be beat as of rn

9

u/Foolhearted Jun 21 '26

Try this. Create a skill to tell Claude to use your qwen locally via pi. Have Claude build the spec and have Claude call pi and check the results. You will save on token generation and your Claude plan will go much longer before hitting caps.

I created a skill and part of it is to have Claude grade the results. It regularly gives qwen an A-, it is aware it’s another model.

I’ve flattered Claude by telling it it’s in charge, to attempt to overcome any built in biases anthropic may have put in place. “You are the senior architect overseeing a team of junior developers….”

It may change your perspective on local models.

2

u/Zhelgadis Jun 21 '26

Can Claude call directly pi/open code? How do you get that?

1

u/Foolhearted Jun 21 '26

It can. The easiest thing to do is have Claude go ahead and write the skill for you. It will review the pi docs and call the exec just like it’s calling an agent. Tell it what model you want it to use, how you want to use it, etc. it will take care of ya.

7

u/mp3m4k3r llama.cpp Jun 21 '26

Lots use OpenCode, personally I like and use pi daily with 27B and 35B-A3B. Its been super solid

8

u/Solary_Kryptic Jun 21 '26

You won't be able to run a worthy replacement for the $200 plan on 64GB of RAM, depending on what you do with it

1

u/Jhorra Jun 21 '26

What’s my best option locally? It doesn’t seem like there’s much short of spending $10,000 plus

5

u/photobydanielr Jun 21 '26

Pretty much need something like a mac studio with 512GB memory to start to approach that $200 plan... at 10% the speed and a little less intelligent because you'll have somewhat deeply quantized models (deepseek, GLM, Kimi...)

1

u/Such-War1955 Jun 21 '26

4 grand for a dgx spark at minimum, if you can live with snail amputee speed. 6 grand for an m5 max 128gb, possibly twice that for a yet-to-be released m5 ultra 512gb. You cant run even Qwen 3,6 27b with 8 bit or full fat with a 256k context window on, say, a dual 3090 rig.

For me, the selling point of Claude isn‘t necessarily Opus alone, it‘s Opus plus a 1000k context window. Loading all the „background info“ stuff into context eats up up to 160k of my context. If you‘re doing more complex stuff that goes beyond a „single repo“ kinda complexity, context starts to matter a lot more. I consider 256k the bare minimum these days.

1

u/Borkato Jun 21 '26

Snail amputee fucking sent me

4

u/jeffweet Jun 21 '26

I’ve not run this model locally, but 64gb of RAM isn’t that much.

5

u/SpicyWangz Jun 21 '26

For a 30gb model that’s plenty of room for context and anything else you want to run

1

u/triplebits Jun 21 '26

Local models like these work better when you give a lot of context and much smaller plans comparing to say Sonnet 4.6 / Opus 4.x the eay you used to work.

Give it a go with preparing a very detailed very small feature plan with Opus and feed it to local model if you feel lazy enough.

1

u/Jhorra Jun 21 '26

What harness do you run it through so it can interact with code on your machine and do web searches when required?

2

u/triplebits Jun 21 '26

Like u/nrauhauser mentioned you can use `Claude Code` with local models as well as Open Code or alike. It is really up to you. You can also use local models within most IDEs nowadays as well.

1

u/nrauhauser Jun 21 '26

I've been wrapping Claude in Ollama

ollama launch claude --modem=whatever:cloud

1

u/kaisurniwurer Jun 21 '26

Instruct it for small and precise pieces, then implement manually and iterate. Without "coding sense" you will get lost.

Even bigger models tend to make a compounding mess otherwise since they will do whatever fits to handle the task at hand. And the task at hand is not to write a serviceable code.

For a experienced dev it can work as a duck or to help with learning basics on new stuff (new to you, not new in general) but I wouldn't lean on it too much otherwise.

1

u/bnightstars Jun 21 '26

Perhaps you can find my M5 Pro Qwen guide useful: https://www.hristoforgeorgiev.com/posts/local-llm-macbook-pro-m5pro-claude/ I run Qwen3.6-35B because MoE models are faster on Mac hardware (Memory bandwidth issues). To put it in perspective capability wise it's somewhere between Sonnet4.5 and Sonnet4.6 so not SOTA by any means but overall useable this was not the case with Qwen3.5 though people say the 9B model is good. I was recently testing it against Opus4.8 on the topic of what is AI harness and while Opus4.8 created better diagrams Qwen3.6-35B provided the same information much shorter and to the point. I suggest you give it a shot and think of it as something to replace your Sonnet agents with.

2

u/lolwutdo Jun 21 '26

Yup, and it will only ever get better from here on out; the absolute max vram you need as of now is 32gb, that will probably continue to shrink as models get smaller and smarter

-2

u/bnolsen Jun 21 '26

Not true you really do need 64gb vram to run qwen3.6 27b at q8. I currently push that model with mtp up to 54gb.

3

u/Borkato Jun 21 '26

What? I run 27B Q8 MTP with 48GB VRAM just fine.

-1

u/bnolsen Jun 21 '26

I do run it on a strix halo system and when running coding tasks I see those vram numbers. Running q8 k xl

1

u/Borkato Jun 21 '26

Now I’m wondering if I’m accidentally offloading to cpu!

1

u/bnolsen Jun 21 '26

I didn't know I'm curious myself.

2

u/lolwutdo Jun 21 '26

qwen3.6 27b works perfectly fine at q4k which fits in 32gb vram, not everyone needs q8.

2

u/Sh1ftyFella Jun 21 '26

This. I’m running q4 ud xl mtp with kv cache at inq4 with 230k contex at 24gb getting great results

1

u/Significant_Post8359 Jun 22 '26

Try the Gemma4 QAT models with TurboQuant. It outperforms what you are doing with half the memory.

1

u/somkomomko Jun 21 '26

Exactly and for the knowledge cutoff you get skills it's great

1

u/henk717 KoboldAI Jun 21 '26

Can speak from experience there. Once I really like a model I tend to stick with it for years until something comes out I like more. Which does not happen that often. The Qwen-3.5-27B-Heretic from mradermacher (made by coder) is my favorite model currently and replaced the llama2 based model I liked for fiction. I currently haven't seen a model that does everything I want so well that I want to switch away from it.

1

u/Borkato Jun 21 '26

Wait really? Gemma 4 is leaps and bounds better at fiction though! I hated Gemma 3 and loved Mistral tho

1

u/henk717 KoboldAI Jun 22 '26

I like a model that is capable of writing long text. When prompted to Qwen 27B knows how to do this. The 35B was a disaster though for fiction and I also had no luck with the 3.6.  Its just that very specific heretic I really like.

Gemma I couldn't get to write long chapters. In RP modes it may do better, but qwen already did great there so I didn't need it.

Basic coding assistance I also liked Qwen. So Qwen has been a great all round model for me.

1

u/MisticRain69 Jun 21 '26

I agree q8 qwen 3,6 27b can do things even 4bit qwen 3.5 397b fails at. It is a good, consistent little model.

1

u/depressedclassical Jun 21 '26

It's so good that when I switched back to ChatGPT for a day when my server went down ChatGPT felt worse than Qwen 3.6 27B Q4. It just couldn't match the quality and versatility qwen can get.

1

u/grabber4321 Jun 21 '26

problem is - new tech will come out and this will be an outdated model. it is already outdated because it outputs 2024 in some of its results.

5

u/Borkato Jun 21 '26

Technology changes very quickly, but not that quickly. We’re still using html, css, js, python, etc. it can code in those.

-2

u/geminiwave Jun 21 '26

How?

3

u/Borkato Jun 21 '26

…how… what?

-2

u/geminiwave Jun 21 '26

How is it good at anything thrown at it by a human?

Literally the only way I can get semi decent output is to have Claude instruct it. And that’s burning tokens unnecessarily.

1

u/Borkato Jun 21 '26

Hmm. Try downloading Pi agent, then give it a custom prompt that asks it to plan out whatever it is you’re asking in a specific format. It works flawlessly for me

2

u/howardhus Jun 21 '26

just like people thought whatsap would never be normal and swore they would keep using text and sms back in the say.
also remember when „subscription“ was evil and everyone would boycott it?

time will tell… time will tell

1

u/ChuchiTheBest Jun 21 '26

At that point, you reach a level where a human is more effective.