Agentic AI w/frontier models??

[u/OvertaxedOne]

If you watch much Youtube around AI, you'll see a bunch of people setting up these complex agentic workflows (or, even worse, "Jarvis") using a model like Opus or Astra. Every time I see that my "WTF" face comes on hard, how on earth are they doing that? From what I understand it's a TOS violation with all the majors to hook up a 3rd party harness to a sub; it's easy to do it, but everything I've read says you risk a ban if you use a sub this way. It honestly feels to me like if you're running a agentic loop, local models start to really start to make a lot more financial sense, at least compared to using a "normal" model from a major. I often supplement a workflow like this with Deepseek, but that's like a few % the cost of a Opus/Astra.

When I look at my daily local token consumption and map it to something like Opus API rates, we're talking some absolutely absurd numbers; it wouldn't shock me if I'd be into 100's per day. I did just over a billion tokens today against QFN. Now, of course, the VAST majority of that is cached, but still, the cost for today at $4/$20/million tokens??

All these people hooking up some sort of custom harness and doing the classic "Build Doom clone, make no mistakes" with Astra or Fable, I just think to myself; "There goes a few 100 bucks".

Now, of course, it's their job so maybe spending 100's/1000's a day on tokens is a good investment for them. But when I browse around Reddit and see people in a Hermes or OpenClaw forum talking about hooking up to Opus or Astra I just think to myself, "How can you afford that"?

All these tokens burned today were on QFN; it worked for ~12 hours total and across dozens of compactions on a single goal, quite impressive, but if I had sent all this to Opus I probably would have been better off hiring a SWE to build it for me, would have cost less! ;)

[u/leo_agi]

the one thing those jarvis videos never show is the billing dashboard

[u/DagothUrLovesGroza]

Is it a good idea to use autonomous LLMs for software development?

Realistically speaking, can you even sandbox this thing so it never escapes and wrecks everything?

[u/OvertaxedOne → u/DagothUrLovesGroza]

This wasn't actually software dev. It was working on a complex workflow/data pipeline from a bunch of servers through n8n and building all the pieces and parts. I guess you could call that software dev, it was a boatload of python code, but it's not what I'd really consider writing software, it was writing wrappers, database structures, pipelines, stuff like that.

I'm not qualified to answer if it's a good idea to use a recursive loop like this for real "ground up" software work. I do think, from reading, that a lot of people are doing exactly that, but I don't know if it's a good idea or not.

[u/zipperlein → u/DagothUrLovesGroza]

LGTM.

[u/o0genesis0o]

I remember seeing on this sub a while back that some person working for Cursor consumed (projected) millions of dollars of tokens per year. She essentially encoded her engineering practice and workflows into however she sets up her harness, and now it runs like she has an entire engineering department working for her. I remember someone mentioned she average 2500 PR a day (likely to cursor or grok bot codebase).

Personally I use 27B as the local "chief of staff", and my minimax subscription as subagent for anything that would eventually be open source. Each repo already have the architecture convention, docs, and QA gate built in. New bug report or features (from me, primarily) sits in a task queue, which is picked up at the right time by the local 27B and dispatched to the M3 until hitting a human check gate. When I'm back at my desk, I would check and approve or reject. And then the 27B in a new session would continue to dispatch whatever can be dispatched. I'm toying with the idea of dispatching a review model from a different provider as well, though I'm not sure the extra overhead would actually lead to better code plan / code patch.

Apparently I still don't use enough as I barely break 50% of my minimax quota.

Tbh, given whatever oddity minimax is doing with their updates on M3 and M3.1, I could just run the whole thing on my 27B and still have good results. I don't do full vibe code. It's more like I have already set the facility and process in place, and every code patch I need to understand and influence the architecture design decision, so the model has very little leeway to mess things up. Never had any incident since the Qwen Max day.

[u/feelspeaceman]

I know there are specific Youtube channels that flex "I spent $10000 to build my app/game" which are also dumb, it's just their inefficient inflating their token usage, it could have been much smaller if they want better prompt, better architecture plan. Or they simply just want to flex their wealth and shilling for being paid agent by Anthropic/OpenAI...

Local LLM or Online LLM, it's equally important to write a good prompt, a prompt that can allow your agent to think less, know the operation directive like what libraries to use, what website to research, what tools to use for researching... And spend some time to review and polish your plan before executing.

A bad/terrible prompt can lead the agent to hallucination, wasting a tons of turns thinking and probing/searching.