What is chat context size?
Chat context size determines how much information is sent to the AI every time a visitor asks a question. The context holds your chatbot's role instructions (the prompt), the conversation so far, and the most relevant pieces of training data retrieved from your knowledge base. A token is a small chunk of text the AI reads - on average 1 token is about 4 characters, or roughly 3/4 of a word.
Benefits of a larger context:
- More room for detailed role instructions without squeezing out your knowledge base
- Better understanding of longer conversations
- Ability to answer complex questions by referencing more training data at once
- When chat memory is enabled, more space for summaries and client profiles from previous conversations
How to change the context size
Select your chatbot, click the Settings tab, then select Model & Advanced in the left sidebar.
Find the Context size dropdown and choose your preferred option:
- 8000 tokens (default) - 1x message credits
- 16000 tokens (2x message credits/message)
- 32000 tokens (4x message credits/message)
The 16K and 32K options are available as premium features - they appear in the list only when your plan includes the larger chat context.
How the context window is shared
The context is a shared budget. Every message, it is divided between:
- Your prompt - the role and behavior instructions you wrote (only when you use Custom Role Definition)
- System instructions - a small amount ChatLab adds so the bot behaves correctly
- Tools - if your bot uses AI Actions or integrations, their instructions take a share
- Conversation memory - the recent conversation, plus summaries and client profiles when chat memory is enabled
- Left for your knowledge base - everything that remains is used for training data matched to the visitor's question
The key takeaway: the more space your instructions take, the less knowledge the AI can use to answer. Raising the context size grows the whole budget, so both your instructions and your knowledge get more room.
The context usage meter
When you define a custom role, ChatLab shows you live how much of the context your instructions consume. Select your chatbot and navigate to Settings > Role & Behavior, then switch to the Custom Role Definition tab. The meter appears directly under the instructions editor.
The colored bar is a miniature of your whole context window: each segment is one of the parts listed above, and the green segment on the right is what remains for your knowledge base. Next to the bar you see your prompt's token count against a recommended guideline (for example "~788 / 2,400 tokens · 8k") and a rating:
- Optimal - your prompt fits comfortably; plenty of room remains for knowledge.
- Getting large - the prompt is starting to crowd the context; consider trimming it.
- Too large - the prompt exceeds the recommended guideline or leaves too little room for knowledge; answers may get worse.
Click the bar to expand the full breakdown:
The expanded panel shows the approximate token count of each part and a Chat context size dropdown, so you can raise the context size right there without going to Model & Advanced. The exact split varies per conversation - the panel is an estimate, not an exact measurement.
Two limits are shown around the editor, and they work differently:
- Character limit (for example "2,882 / 8,000 characters") - a hard limit; the editor does not accept a longer prompt.
- Token guideline (the meter) - a soft guideline; exceeding it does not block saving, but it leaves less room for your knowledge base and conversation memory.
If the meter turns red, it suggests two fixes: increase the chat context size (messages then cost more credits - 16K is 2x, 32K is 4x, see message credits), or move step-by-step scenarios out of the prompt into Conversation Flow stages, explained next.
Context size and Conversation Flow
Conversation Flow lets you split your bot's behavior into stages, each with its own instructions. When the flow is enabled, the active stage's instructions are added to the prompt - so they share the same context budget. Long stage instructions on top of a long custom role leave less room for your knowledge base at the same context size.
Context size also controls how long each stage's instructions can be:
- 8,000-token context - up to 1,000 characters per stage
- 16,000 or 32,000-token context - up to 6,000 characters per stage
Raising the context size therefore does two things at once: it raises the per-stage instruction limit and restores room for knowledge. The reverse also works - if your custom role prompt is too large, moving step-by-step scenarios into flow stages keeps the core role short, because each stage's instructions are only loaded while the conversation is at that stage.
Credit cost per context size
The default context size is 8,000 tokens. Increasing it multiplies how many message credits each reply consumes:
- 8,000 tokens - 1x message credits (default)
- 16,000 tokens - 2x message credits per message
- 32,000 tokens - 4x message credits per message
The multiplier stacks with the credit cost of the AI model you select - a larger context makes every reply of that model proportionally more expensive.