Context management

Every request builds a fresh agent, so the model has no memory of what was said earlier in the session. Bi repacks the stored conversation into a model message sequence on the server, which is what makes multi-turn conversation work at all.

Cross-turn packing

There is one rule: the most recent turns are rebuilt in full, older turns are summarized.

• The last recent_turns_full turns (default 6) are rebuilt completely — user, assistant and tool messages alike. Tool blocks stored on an assistant row are restored into assistant.toolCalls plus the following role: "tool" messages, so tool_call / tool pairing stays intact. That pairing is a hard requirement of OpenAI-compatible APIs, and a break will get the turn rejected by the gateway.
• Older turns collapse into a summary inside <earlier_conversation_summary>, holding the user’s key points, the assistant’s conclusions and the list of changed files. The summary is assembled purely by rules, with no model call: no extra cost, no added latency, and reproducible output.
• History beyond 40 turns is dropped rather than reaching further back.

Compaction is counted in turns, not tokens. Many turns does not necessarily mean you will hit the ceiling, and a single very long turn can overflow early. To control it precisely, adjust recent_turns_full: raise it to keep more verbatim text, lower it to start summarizing sooner.

An interrupted or timed-out tool can produce an empty result string. In that case Bi substitutes a placeholder, (no output: tool interrupted or timed out). It tells the model the step produced nothing, and avoids an empty content tripping strict gateways.

Prefix stability

Summaries are fingerprinted on “count of older turns + id of the last message”. As long as the older history is unchanged — which is the normal case when appending to a conversation — the summary is frozen and reused, keeping the request prefix byte-for-byte identical.

There is a slack layer on top: the freeze point only advances once the fully-injected portion exceeds recent_turns_full + 2 turns. The purpose is to space out prefix invalidation. Without it, every added turn would shift the summary position and the cache would essentially never hit.

This exists to serve the model provider’s prompt cache. The cache lives on the provider’s side; Bi stores nothing. What Bi does is keep the head bytes of every request identical so the provider can keep hitting it.

Tuning

recent_turns_full lives in .bi/config.json. A value ≤ 0 or a missing key falls back to the default of 6.

• Short tasks, API calls, running scripts: lowering it, or leaving it alone, costs nothing — the summary barely participates.
• Long-document analysis, iterative work: raise it to 10–15 to keep more verbatim text and reduce the drift that aggressive summarization introduces.
• Expensive contexts: lower it to save tokens, accepting that older detail is summarized away sooner.