頁面比較

Tokens are the basic unit that Azure OpenAI GPT models (including GPT3.5/4) use to compute the length of a text. They are groups of characters, which sometimes align with words, but not always. In particular, it depends on the number of characters and includes punctuation signs or emojis. This is why the tokens count is usually different from the word count.

In order to know exactly how many tokens spent in your text, you need to tokenize your text, which means running a tokenizer algorithm that will count all the tokens. Sounds difficult? It’s actually as easy as copy/pasting. You can use https://www.gptcalculator.xyz/  

For more information about Token.