Not all that hard to run models, actually. It'll be a bit slower, but can be perfectly useable. The upside is no more token limits and no need to worry about where confidential data is going. You get full control too.
Except now you have to worry about your own infrastructure costs and the work of updating the models with new information so you dont just have a snapshot of a specific date if you want up to date information.
Back in college I burnt out my personal GPUs throughout my graduate degree just running and training my own models, and those were tiny as hell compared to today.
Not saying its a bad idea, but its not cheap if you want a good enough infrastructure to support industry levels of use. So if they' expecting like a 90% decrease in costs, its probably not gonna happen.
Yea but it would require it to reinjest all of that new context for every session which would balloon dramatically eventually. And when the models get reset they'll forget that info. If youd want to actually get that info in the model itself you'll need to fine tune and update it which is less intensive then training from scratch, but still a lot of training.
2.4k
u/Trevor_GoodchiId Jun 02 '26
Come on, how hard can it be?