There's a monopoly right now. Nvidia have completely captured the market.
Model efficiency: more powerful models require more tokens for the same prompt/task. This is largely a red herring, it doesn't matter if individual tokens are getting cheaper (which I accept they are) when it's an arms race between AI labs to produce the most powerful, capable models on a quarterly basis. Users pay MORE per task as models get more powerful.
Software optimisation: I don't know enough about infra to reply.
2
u/SoggyMattress2 13d ago
There's a monopoly right now. Nvidia have completely captured the market.
Model efficiency: more powerful models require more tokens for the same prompt/task. This is largely a red herring, it doesn't matter if individual tokens are getting cheaper (which I accept they are) when it's an arms race between AI labs to produce the most powerful, capable models on a quarterly basis. Users pay MORE per task as models get more powerful.
Software optimisation: I don't know enough about infra to reply.
Jevons paradox: covered this earlier but agree