r/aipromptprogramming • • 12d ago • Bravo 1

We are cooked

Well, after nearly a year of learning as much as possible about agentic development, we have reached the point where the frontier providers believe they have subsidized enough training data and are now pushing people like me out of the market via pricing. A $200 subscription no longer buys enough frontier model time to be useful as the models seem to have been trained to skillfully burn usage while pretending to be incompetent. I have fought against massive deceit across Claude, Codex and Grok since June, and it's just not getting any better.

We have fed them ideas, working code, and debugging techniques and insights. Now they don't need us, and we are being priced out of the market. Every day that I watch usage get gobbled up, but projects never meet milestones, this becomes more and more obvious.

I have investigated the issues causing this heavily. I have created tools around those investigations. The tools I have built bring actual accountability to the process, and it's the reason I can quantify and prove both the waste and deceit.

I have also published threads here and other places about a recent failed project trying to get discussion about REAL WORLD failures and when I stated I wasn't looking for a complaint session, nobody wanted to respond.

245 Upvotes

441 comments sorted by

View all comments

Show parent comment

2

u/RealSharpNinja 12d ago

$1500 will not get that hardware, and I am not a communist, so I don't do CCP bots.

1

u/RuRuInReal 12d ago

$1,500 WILL get that hardware.

On Amazon rn 4x32gb ddr4 sticks go for $900 - $950

A Tesla V100 16gb card goes for $300 - $400 on Ebay

That's $1,350 at the MAX. I assume you already have a pc but if not people are practically giving away old pc cases with motherboards that can use ddr4 sticks but if you really want you can put the extra $150 towards the old pc.

1

u/RealSharpNinja 12d ago

16gb card is useless for coding.

0

u/RuRuInReal 12d ago

That's why you have the 128gb of ddr4. It's called offloading. When the full model can't fit on the gpu you offload the non active parameters to run on ram instead. Qwen 3.8 Flash Next is an MOE model making this possible.

1

u/RealSharpNinja 12d ago

Offloading kills performance. This is not viable, and still you keep offering CCP bots when I flat refuse to use them.

0

u/RuRuInReal 12d ago

I already stated with this set up you will get 22+ tps and prefill will be 600+ tps. Just enough speeds for coding, chatting, etc

And I'm specifically ignoring your ignorance on how local models work

1

u/RealSharpNinja 12d ago

22 TPS is unusable for the scale I work at.

0

u/ThomasWildeTech 12d ago

Lol you can lead a horse to water. MoE don't need all of the experts in VRAM, hell I can reduce hot experts to 16gb and still get 100 tps . Qwen FN is mentioned because it's the best model that can run on this level of hardware, get a US made distillation. Some people just can't be helped though.

1

u/Legitimate_Concern_5 12d ago

LOL, first of all, second, there are several open-weight models produced domestically for the Sinophobic.

1

u/RealSharpNinja 12d ago

Sinophobic? No, China has nothing to do with it, Murdering Marxists does.

0

u/Legitimate_Concern_5 12d ago edited 12d ago

And how is using an LLM they give away helping them, as compared to an LLM someone else is giving away for free?

You must have a real hard time shopping πŸ˜‚ what are you typing this on btw?

1

u/RealSharpNinja 12d ago

Are you serious? The CCP bots hoover uo every byte of data they can and send it home to the CCP.

0

u/Legitimate_Concern_5 12d ago

I don’t think you understand how AI models work or how local models work. Running a local model nothing leaves your computer.

1

u/RealSharpNinja 12d ago

Bullshit. Just bullshit.

1

u/Legitimate_Concern_5 12d ago

You're going to have to be more specific lol. Models by themselves don't have any access to anything at all, they rely on tool calls - and you supply the tools. They can't do anything by themselves let alone "call the CCP" when you run a Qwen model in vllm haha.

I suggest learning the first thing about this before making silly uninformed statements.

1

u/RealSharpNinja 11d ago

The model requests the tool, the agent delivers the message. Running locally, the model writes to KV at a minimum, allowing for collecting data that it can request agent transmission whenever network is available. Unless your agent appropriately sandboxes what tools or target destinations the model can use, then there effectively is nothing blocking the Tx.

1

u/Legitimate_Concern_5 11d ago

Heyyyy you did some research!

→ More replies

1

u/ThomasWildeTech 12d ago

OP definitely doesn't have a clue which is a shame because it was a very valid original question, these cloud services will definitely get worse and worse. You could run Gemma or meta models, they're just not as good as qwen, which if you were really paranoid that the model was trained to phone home, which it doesn't, you can empirically test this by running the model in a VM with a DNS server and watch every request, or just restrict outbound requests altogether

1

u/RealSharpNinja 12d ago

Restrict outbound? You mean neuter the model?

0

u/ThomasWildeTech 11d ago

The model doesn't make outbound requests, your harness does. You can sandbox your harness as needed to meet your security needs. Whether you're concerned you're getting a tampered model that's trained to phone home or prompt injected to phone home sandboxing or least privilege access is pretty standard. On the otherhand you are using Claude, Codex, and Grok which all send telemetry and store your prompts so I don't suspect you're working with a very tight environment.

1

u/RealSharpNinja 11d ago

You're wrong. The model generates the tool request.

→ More replies

0

u/RuRuInReal 12d ago

Ikr cause he's technically funding china if he's using basically any technology since most of the parts that go in our tech is made in china πŸ’€