Extending chat context size
- Extending chat context size
- What is chat context size?
- How to Change the Chat Context Size
- Costs Associated with Chat Context Size
What is chat context size?
Chat context size determines how much information is sent to the AI chatbot every time user asks a question. It includes instructions (the prompt), current chat conversation, and relevant training data that helps the AI respond better. More context typically helps the AI provide more accurate and detailed answers.
Benefits of larger chat context:
- Improved accuracy and relevance of responses.
- The AI can understand and remember longer conversations.
- Enhanced ability to handle complex questions by referencing more training data.
- Allows inclusion of information from previous conversations with the same user, enabling more personalized and tailored interactions. *
* Note: Including context from previous conversations requires to enable chat summaries and memory. Check more details about this feature here.
How to Change the Chat Context Size
To adjust your chat context size:
- Navigate to the chatbot Advanced settings.
- Find the Context Size setting.
- Choose your desired context size from the available options:
- 8,000 tokens (default)
- 16,000 tokens
- 32,000 tokens
Increasing context size allows more information to be included in each interaction, enabling more detailed and accurate answers.
Costs Associated with Chat Context Size
The default context size is 8,000 tokens. Adjusting the context size affects how many message credits each interaction consumes:
Context Size | Cost (Message Credits per Message) |
8,000 | Default (1x message credits) |
16,000 | Double (2x message credits) |
32,000 | Quadruple (4x message credits) |
Choosing a larger context size provides better AI performance but consumes more message credits per message.