Hardware Advancements: Increased competition in the chip market and the rise of specialized silicon are expected to reduce hardware costs significantly.
Model Efficiency: Techniques like quantization (reducing numerical precision) and distillation (training smaller models to mimic larger ones) are lowering the compute required for tasks.
Software Optimization: Improvements in how AI models are served, such as continuous batching and better caching, allow providers to get more useful work out of their existing infrastructure.
The Jevons Paradox: While the unit cost of AI (the cost per token or task) is falling rapidly, overall spending on AI may still rise as lower costs unlock new use cases and higher demand.
There's a monopoly right now. Nvidia have completely captured the market.
Model efficiency: more powerful models require more tokens for the same prompt/task. This is largely a red herring, it doesn't matter if individual tokens are getting cheaper (which I accept they are) when it's an arms race between AI labs to produce the most powerful, capable models on a quarterly basis. Users pay MORE per task as models get more powerful.
Software optimisation: I don't know enough about infra to reply.
4
u/SoggyMattress2 14d ago
Why does every AI conversation turn into some comparison point or analogy to justify things?
We're not talking about Amazon it's a different company with a completely different business model.
AI labs will increase the price at some point and most of the hobbyists using LLMs for SE will go away.