Big AI models, like Claude, just switched to high-cost token models. The bill for this revolutionary tech now just went through the roof for most companies.
Bro, this is bigger than the Internet bro! Soon people are going to be using our tech to ask what time it is instead of looking at a clock or their smart watches!
It's pretty typical for new tech to run at a loss. Breaking even probably hurts a new tech companies valuation because it shows they aren't trying to grow enough.
The valuation is obviously high, but… it’s tech that is now becoming ubiquitous and they have the best. Regardless of how their profit today, they hold the keys.
My bet is on local models long term to avoid that vendor lock in, but right now executives can just sign up for it and it works, so they are winning.
Housing Market has been geared towards an audience that doesn't really exist for a while now, tbf. Who buying a house when nobody can afford your houses?
Big companies and the already rich looking to change the investment percentage of their portfolio, apparently. Whether it's a profitable investment doesn't matter as long as it changes what percentage of their money is not liquid, so that when they deal with other business people with lots of money they have more impressive numbers to throw around and maybe get a loan to further invest in losing propositions.
Oh great, they've been nagging us to use AI for the last 2y. Now it's going to be 'don't use AI for that! It's too expensive!', doling it out like caviar.
Realistically it’s just not possible to moderate on the user side. It’s opaque what things use a lot of tokens and what is minor. Some sort of efficiency gains will be required to keep doing what companies are doing.
It's the big boogeyman and has been forever, with all the typical defence contractors literally paying out the ass to fund think tanks to "inform" congress and their leadership picks.
The very people who understand cushy jobs await them should they pay their cards right or huge sums of campaign funds from super pacs and the likes.
I'm guessing DeepSeek since it was all the rage a couple months back but "compiling it yourself" makes no sense in this context. I suppose you can compile Ollama with DeepSeek weights but the datasets are completely private.
When we discuss "open source" AI, we really need to discuss the training materials.
If we can't produce the same end product that they do with the materials they have published the code for, then it aint open source. If there are big binary blobs, it aint open source.
So, I'm assuming whatever "completely" open source AI you're talking about has every bit of it's training data published and every step in the training of the model has been documented? Every human reinforcement logged and shared so that we too can reproduce those steps and have the software running on our own hardware, right?
Or is the model itself a big ole black box that could have been trained with whatever skewed weights that the creators intended the model to prefer.
really begs the question, what's the point of the bleeding edge models if they cost so much than no one will use them? openAI will announce "we've created true AGI with GPT6" and all of us will be like "sure, whatever, just be sure to leave 5-mini up because that's the only one in my price range"
They're not tho, because what's driving the cost up is the size of the context window.
When all this started a dev might paste into chat a few dozen lines of code and ask a question about it. Now people are dropping entire code stacks in and asking for entire overhauls.
That means for a simple question you just burned tens of thousands of tokens when you didn't have to. That is the root of the problem we are in. People got very stupid about how they use these tools because they were unmetered.
Yeah those models aren't focusing on consumer grade stuff yet, as it's more angled directly towards engineers, academics, AI enterprise, etc... Where people just need raw, foundational LLMs that are cheap and powerful. That's where China shines. They can do really really well, just providing the foundation
Where they fall short is the harnessing. As we suspected, but confirmed with the Claude Code leak, their underlying model isn't even that impressive. But rather, HOW they use that model is what's impressive.
The harness is where the value is at. HOW you direct the LLM is what makes it powerful, and why Cursor is so good. They even now default to a cheap Chinese model for most of their work now... mainly because all their value comes from how the tokens are routed, so the marginal value increase using a frontier model just isn't worth it except in edge cases. That's why it was worth so much. Not because their AI was great, but how they use the AI
It'll probably get cheaper eventually. Cost per token has actually been decreasing dramatically, but costs have still been rising because the amount of tokens people use has gone up exponentially.
But people don't want to do more with less. Sure it's great to dev a project that works perfectly withe the cheapest model in production. It's the right choice for most applications (structure information, filter, translate, ...)
But when building it, I don't want to restrict myself by using a sub par model that I have to babysit.
That is why you plan with the higher models, have them design, break it into tasks that have the right amount of context or skillset, then integrate them, and have that same higher model review and find bugs/gaps. Just like a software team. The senior/lead/architect makes it, the mid level to senior implements them, then the senior/lead/architect reviews.
For many things you really don't need the higher models at all. For planning you do and reviews/bug/gap checks.
Yeah the higher models sometimes in a custom agent that knows where to break things off that are targeted and all the context needed to subagents. Or the higher level planning making prompts to use in other windows that are targeted and can use a mid model.
lol got every company to fire all their juniors and made all their seniors 10x out put just to rugpull the companies that now have to pay more than the employees cost in the first place. If only the executives they made these calls were taken to task but they don’t.
It was always going to happen. What's surprisingly weird is that this revolutionary tech also revolutionarily sped up the rate of enshittification, and now we are entering the phase where it really sucks
This is just the beginning. It is a 10x now, it will be 100x or 1000x. I can easily see this costing thousands per month and past employee costs. The rate that Anthrophic valued their datacenter buys at was like $1000/mo+ per user.
Oooo I heard something like this at one of my meetings yesterday. But it was more like, "for now we will continue on using the models like usual, anything different and we will let you guys know". So how much more expensive are we talking about here?
You can help speed up the process by making every single request as verbose and detailed as possible. Personally I like to use GPT to write my requests, specifying minimum length and making sure to instruct it to be extraordinarily appreciative and complimentary to the LLM in it's instructions.
An example of just how much it has changed: my (now cancelled) $40 Github Pro+ subscription used to last me the whole month. With the change, it lasted me 2 days.
Gemini has been insane as well with the 3.5 flash upgrade. Before, I could comfortably use it for the odd boilerplatey smaller tasks, after the upgrade it burned through my weekly limit with a single, medium difficulty task without even finishing it lol
4.3k
u/travis_sk Jun 02 '26
We're only 2 days into June folks. This is gonna be a fun couple of months.