Honestly trying to treat token like watt is pretty crazy. Like a watt of power truly is a scientific standard. A bit of information is closer but again pretty standard, you know what are sending when you choose to send or receive a bit of data. Trying to apply a standard utilization rate to a unit of information in exchange for a non-deterministic rate, size, or quality back is nuts.
I wonder how long they can keep the circus running though. They were obviously trying to vendor lock enterprise size businesses in before they eventually nuke the cost model, and Deepseek just preemptively slashed their cloud pricing probably knowing that everyone else will just migrate or even run on older cloud hardware which providers can sell by watt (and demand).
China might be behind on hardware, but their biggest wrench in the gears will probably just be popularizing their published LLMs which don’t require multiple nuclear powerplants to run lol.
There is an argument that token costs can be recorded as operational expenditure but it’s the same argument used for cloud infrastructure, so many companies are going to get hit with multimillion dollar bills for tokens, which will have the effect of reeling in and limiting token usage or moving away from it completely.
It’s the token model that will kill it
Honestly trying to treat token like watt is pretty crazy. Like a watt of power truly is a scientific standard. A bit of information is closer but again pretty standard, you know what are sending when you choose to send or receive a bit of data. Trying to apply a standard utilization rate to a unit of information in exchange for a non-deterministic rate, size, or quality back is nuts.
How do you meaningfully show an ROI on that?
Once those bills start hitting company profits, companies will pull back
IF they’rt are dependent on investor funding that demand signals of them using
They will pay the bills with someone elses money so as long as the checks cash
I wonder how long they can keep the circus running though. They were obviously trying to vendor lock enterprise size businesses in before they eventually nuke the cost model, and Deepseek just preemptively slashed their cloud pricing probably knowing that everyone else will just migrate or even run on older cloud hardware which providers can sell by watt (and demand).
China might be behind on hardware, but their biggest wrench in the gears will probably just be popularizing their published LLMs which don’t require multiple nuclear powerplants to run lol.
There is an argument that token costs can be recorded as operational expenditure but it’s the same argument used for cloud infrastructure, so many companies are going to get hit with multimillion dollar bills for tokens, which will have the effect of reeling in and limiting token usage or moving away from it completely.