The problem with tokens is that you need a very high amount of tokens in order to be able to actually do anything useful with an AI. See you all sat there with your 10,000 tokens thinking you’re rich and you have a 20 minute conversation with it and now you’re out of tokens.
AI tokens suffer from an inflation problem you have really big numbers but they don’t actually represent actually all that much capability.
Fortunately they are the companies token so I really don’t care, but I’ve been miffed if I was the one paying for them.
LLMs predict the next best “token”. So what’s a token? It can be a word, a simple punctuation mark, or sometimes even a phrase.
So basically, here’s what an LLM works like:
Basically, it feeds in tokens to itself to predict the next best token. Then, that entire chunk of tokens is fed back in to predict the next token.
As you can see, tokens kinda correlate well with energy consumption, hardware wear and tear and so on.
So, a low token request will consume less resources than a high token request. Hence, it makes sense to charge per token.
The problem with tokens is that you need a very high amount of tokens in order to be able to actually do anything useful with an AI. See you all sat there with your 10,000 tokens thinking you’re rich and you have a 20 minute conversation with it and now you’re out of tokens.
AI tokens suffer from an inflation problem you have really big numbers but they don’t actually represent actually all that much capability.
Fortunately they are the companies token so I really don’t care, but I’ve been miffed if I was the one paying for them.