In the context of OpenAI’s ChatGPT API, a token can refer to a chunk of text that the model reads. Each API call’s input message is broken down into tokens, which can be as short as one character or as long as one word (e.g., “a”, “apple”). The total number of tokens used in an API call affects the cost, speed, and whether the call works at all as it must be under the maximum limit (4096 tokens for gpt-3.5-turbo). Tokens are used in both the messages you write and the model’s responses in conversation.
