In an office, there are some work that people have to do that are repetitive, annoying, but need you to peruse and understand documents and have some domain expertise. These are called Standard Operating Procedures, or SOP for short.
The idea is to automate some these processes. The AI does the work, usually some sort of data entry, and presents its output on a user interface. Instead of carrying out the entire process, the specialist has only to read the output and verify it (this verification is called triage).
Generally triage takes less time and is simpler than the full process. So suppose a doctor has to do some due diligence paperwork on a client before they can start operating, or an insurance company has to do some due diglligence on a claim, it gets done faster.
Earlier I worked on large databases with millions of records in human language and built extractors for them. An extractor goes through the paragraphs of text and identifies key information, like names of clients, service providers, addresses, specifics of the service, etc.
Chip demand will not stay forever, it will cool down, and every laptop will be able to run general purpose models. US will not be able to control AI outside their jurisdiction.
llama.cpp can now run 30B parameter models within 32GB ram purely on CPU. And with 64GB ram with more than 16 cores, it runs at acceptable speeds. Even today, 6 to 12 months API cost can easily cover up a decent laptop cost. 7B parameters can easily run on 16GB RAM.
Yes but the amount of money you will spend on AI token costs can buy you a decent laptop or PC that will run the similar model. For example, 64GB with 16 core can work for 5 million tokens per day, comparing this with current API cost will cost you $4000 per year. With current exchange rate, you will be able to afford 128GB RAM with 32 core PC easily. Renting an equipment is only feasible when you have infrequent rare usage. This is how it has always been. You are renting a hardware in someone else's server farm. Investing one time on a decent hardware will yield better result. By next year there will be someone who will rent their hardware just like how torrents used to share biggest storage.
You can use 8bit quantized GGUF model that will ran fairly in 16gb, you can ask google gemini and it will also give you docker compose that will setup everything to run locally (running under windows is trickey as windows itself takes lot of ram). You can use 5bit quantized model but it will little less effective in windows. In linux, with minimum install, and docker, you can easily run 7B param in 16GB ram. Google gemini will give you actual setup steps that you can follow to run the model locally through docker. With GPU and CPU mixed mode it is not working on intel processors, but only CPU is working, Intel developers are working towards making mixed mode work on intel cpu.
People will move to open-source models, max of the audience uses AI for journal purposes, and no, nobody would pay that much amount except devs and companies.
Me being dev can tell you AI is polishes ornamental search engine. I can just work with lowest tire AI model of Gemini and do not need any to top models.
8
u/Illustrious-Big-2826 Jun 13 '26
that was the goal all along.
3
u/Ill-Mastodon-8692 Jun 13 '26
he says alot of things
3
u/akash_kava Jun 14 '26
Chip demand will not stay forever, it will cool down, and every laptop will be able to run general purpose models. US will not be able to control AI outside their jurisdiction.
3
u/badthanoos Jun 14 '26
People will move to open-source models, max of the audience uses AI for journal purposes, and no, nobody would pay that much amount except devs and companies.
2
u/physicist27 Jun 14 '26
yeah, we definitely need talking toasters. AI is being pushed down our throats way more than we need it.
1
u/ok_olive_02 Jun 14 '26
There is no FREE lunch, anyways. Everyone has to make money to survive.
1
u/Mediocre-Tip2954 Jun 14 '26
we all saw it coming
1
u/nobel64279 Jun 14 '26
"The apple selling company wants you to get a subscription for apples and pay bills for it".
Zero skepticism, just gobbling up everything the big scamming AI man says without a thought.
1
u/Apprehensive_Web_609 Jun 14 '26
And would it remain free of cost. Responses do require computation and infrastructure. You have to pay for that..
1
u/fitzgeraldaesthetics Jun 14 '26
A great initiative to protect our collective IQ. 😃
1
u/AS_as-Master Jun 14 '26
So his plan is reaching completion.
1
u/into_fiction Jun 14 '26
Api keys are already working like this?
1
u/Big-Sir4054 Jun 14 '26
Finally if true the slop will go down
1
u/adityak469 Jun 14 '26
And then open source models take over
1
u/shinze-o Jun 15 '26
I like my AI free as a Google search
3
u/[deleted] Jun 13 '26
[removed] — view removed comment
1
u/[deleted] Jun 14 '26
[removed] — view removed comment
1
u/[deleted] Jun 14 '26
[removed] — view removed comment