From bc56546b49b022be38b91cef52edb9126c24cc17 Mon Sep 17 00:00:00 2001 From: narawat Date: Sat, 25 Jul 2026 17:20:19 +0700 Subject: [PATCH] update --- packages/agent/learn/agent_workflow.md | 473 +++ packages/agent/learn/architecture.md | 4494 +++++++++++++++++++++ packages/agent/learn/memory_management.md | 624 +++ 3 files changed, 5591 insertions(+) create mode 100644 packages/agent/learn/agent_workflow.md create mode 100644 packages/agent/learn/architecture.md create mode 100644 packages/agent/learn/memory_management.md diff --git a/packages/agent/learn/agent_workflow.md b/packages/agent/learn/agent_workflow.md new file mode 100644 index 00000000..29dd3136 --- /dev/null +++ b/packages/agent/learn/agent_workflow.md @@ -0,0 +1,473 @@ +# Agent Workflow: Message Handling and Response Generation + +## Overview + +This document explains the complete step-by-step flow of how the agent processes a user message and generates a response, from the moment a user asks "what product do you have in stock" to when the agent responds with an answer. + +--- + +## Architecture Diagram + +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ USER INPUT │ +│ "what product do you have in stock?" │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 1. AGENT.PROMPT() ENTRY │ +│ File: packages/agent/src/agent.ts:339 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ • Validate no active run (throws if busy) │ │ +│ │ • normalizePromptInput() converts string to AgentMessage[] │ │ +│ │ • runPromptMessages() launches execution with lifecycle events │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 2. AGENT LIFECYCLE INITIALIZATION │ +│ File: packages/agent/src/agent.ts:398-412 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Events Emitted: │ │ +│ │ • agent_start │ │ +│ │ • turn_start │ │ +│ │ • message_start / message_end (for each prompt message) │ │ +│ │ │ │ +│ │ Context Snapshot Created: │ │ +│ │ • systemPrompt │ │ +│ │ • messages (copy) │ │ +│ │ • tools (copy) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 3. AGENT LOOP STARTS │ +│ File: packages/agent/src/agent-loop.ts:95 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ runPromptMessages() → runAgentLoop() │ │ +│ │ • Prompts added to context.messages │ │ +│ │ • Lifecycle events emitted for prompts │ │ +│ │ • Calls runLoop() (main processing loop) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 4. LLM CALL BOUNDARY - MESSAGE TRANSFORMATION │ +│ File: packages/agent/src/agent-loop.ts:281-372 (streamAssistantResponse) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Step 4.1: Context Transform (optional) │ │ +│ │ transformContext(messages) → transformed messages │ │ +│ │ (Used for context pruning/injection) │ │ +│ │ │ │ +│ │ Step 4.2: LLM Conversion │ │ +│ │ convertToLlm(messages) → Message[] │ │ +│ │ - Filters non-LLM messages (bashExecution, branchSummary, etc.) │ │ +│ │ - Converts: user → user, assistant → assistant, toolResult → toolResult │ │ +│ │ │ │ +│ │ Step 4.3: Build LLM Context │ │ +│ │ { │ │ +│ │ systemPrompt: context.systemPrompt, │ │ +│ │ messages: llmMessages, │ │ +│ │ tools: context.tools │ │ +│ │ } │ │ +│ │ │ │ +│ │ Step 4.4: Resolve API Key │ │ +│ │ getApiKey(model.provider) → apiKey │ │ +│ │ │ │ +│ │ Step 4.5: Stream Function Call │ │ +│ │ streamFunction(model, llmContext, options) │ │ +│ │ - Default: Models.streamSimple() from @earendil-works/pi-ai │ │ +│ │ - Makes actual LLM API call │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 5. LLM RESPONSE STREAMING │ +│ File: packages/agent/src/agent-loop.ts:317-371 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Stream yields events: │ │ +│ │ • start → creates partial AssistantMessage │ │ +│ │ • text_start → streaming text begins │ │ +│ │ • text_delta → text chunks arrive │ │ +│ │ • toolcall_start → tool call block begins │ │ +│ │ • toolcall_delta → tool call arguments arrive │ │ +│ │ • toolcall_end → tool call block complete │ │ +│ │ • text_end → text block complete │ │ +│ │ • done → final message complete │ │ +│ │ │ │ +│ │ State Updates: │ │ +│ │ • Partial message pushed to context.messages │ │ +│ │ • message_start event emitted │ │ +│ │ • message_update events emitted as text/tools stream in │ │ +│ │ • Final message committed to context.messages │ │ +│ │ • message_end event emitted │ │ +│ │ │ │ +│ │ Stop Reasons: │ │ +│ │ • stop - normal completion │ │ +│ │ • toolUse - model requested tool calls │ │ +│ │ • length - token limit reached │ │ +│ │ • error - failure │ │ +│ │ • aborted - operation aborted │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 6. TOOL CALL PARSING AND EXECUTION │ +│ File: packages/agent/src/agent-loop.ts:408-554 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Step 6.1: Extract Tool Calls │ │ +│ │ toolCalls = message.content.filter(c => c.type === "toolCall") │ │ +│ │ │ │ +│ │ Step 6.2: Determine Execution Mode │ │ +│ │ - Check config.toolExecution: "parallel" or "sequential" │ │ +│ │ - Check individual tool executionMode setting │ │ +│ │ - Decides how to execute tool batch │ │ +│ │ │ │ +│ │ Step 6.3: For Each Tool Call │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ 1. Tool Lookup │ │ │ +│ │ │ tool = context.tools.find(t => t.name === toolCall.name) │ │ │ +│ │ │ │ │ │ +│ │ │ 2. Argument Preparation │ │ │ +│ │ │ prepared = tool.prepareArguments?(toolCall.arguments) │ │ │ +│ │ │ │ │ │ +│ │ │ 3. Argument Validation │ │ │ +│ │ │ validateToolArguments(tool, preparedToolCall) │ │ │ +│ │ │ │ │ │ +│ │ │ 4. Before Tool Hook │ │ │ +│ │ │ beforeToolCall({ assistantMessage, toolCall, args, context }) │ │ │ +│ │ │ - Can block execution by returning { block: true, reason } │ │ │ +│ │ │ │ │ │ +│ │ │ 5. Execution │ │ │ +│ │ │ execute(toolCallId, params, signal, onUpdate, context) │ │ │ +│ │ │ │ │ │ +│ │ │ - Parallel Mode: Tools execute concurrently │ │ │ +│ │ │ - Sequential Mode: Tools execute one-by-one │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 7. TOOL EXECUTION EXAMPLE - READ TOOL │ +│ File: packages/agent/src/harness/tools/read.ts │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ User asks: "what product do you have in stock?" │ │ +│ │ │ │ +│ │ Agent decides to read catalog file "products.json" │ │ +│ │ │ │ +│ │ Input Arguments: │ │ +│ │ { │ │ +│ │ "path": "products.json", │ │ +│ │ "offset": 1, │ │ +│ │ "limit": 100 │ │ +│ │ } │ │ +│ │ │ │ +│ │ Execution Steps: │ │ +│ │ 1. resolveReadToolPath(env, path, signal) → absolutePath │ │ +│ │ 2. env.readBinaryFile(absolutePath, signal) → bytes │ │ +│ │ 3. Detect mimeType (check if image) │ │ +│ │ 4. For text files: │ │ +│ │ - Decode UTF-8 → textContent │ │ +│ │ - Split by lines → allLines │ │ +│ │ - Apply offset/limit slicing │ │ +│ │ - Truncate if exceeds DEFAULT_MAX_BYTES or DEFAULT_MAX_LINES │ │ +│ │ - Add truncation notice to output │ │ +│ │ 5. Return result: │ │ +│ │ { │ │ +│ │ content: [{ type: "text", text: output }], │ │ +│ │ details: { truncation: ... } │ │ +│ │ } │ │ +│ │ │ │ +│ │ Output Example: │ │ +│ │ "Showing lines 1-50 of 150. [Showing 50 lines of 150. Use offset=51 to │ │ +│ │ continue.]" │ │ +│ │ │ │ +│ │ Tool Result: │ │ +│ │ { │ │ +│ │ "role": "toolResult", │ │ +│ │ "toolCallId": "tool_abc123", │ │ +│ │ "toolName": "read", │ │ +│ │ "content": [{ "type": "text", "text": "..." }], │ │ +│ │ "isError": false, │ │ +│ │ "timestamp": 1721721600000 │ │ +│ │ } │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 8. TOOL RESULT HANDLING │ +│ File: packages/agent/src/agent-loop.ts:556-792 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Step 8.1: After Tool Hook │ │ +│ │ afterToolCall({ assistantMessage, toolCall, args, result, isError, ctx }) │ │ +│ │ - Can override result content, details, usage, terminate hint │ │ +│ │ │ │ +│ │ Step 8.2: Create Tool Result Message │ │ +│ │ { │ │ +│ │ role: "toolResult", │ │ +│ │ toolCallId: toolCall.id, │ │ +│ │ toolName: toolCall.name, │ │ +│ │ content: result.content ?? [], │ │ +│ │ details: result.details, │ │ +│ │ usage: result.usage, │ │ +│ │ isError: false, │ │ +│ │ timestamp: Date.now() │ │ +│ │ } │ │ +│ │ │ │ +│ │ Step 8.3: Emit Events │ │ +│ │ • tool_execution_start │ │ +│ │ • tool_execution_end │ │ +│ │ • message_start (toolResult message) │ │ +│ │ • message_end (toolResult message) │ │ +│ │ │ │ +│ │ Step 8.4: Update Context │ │ +│ │ • Push tool result message to currentContext.messages │ │ +│ │ • Push to newMessages array │ │ +│ │ │ │ +│ │ Step 8.5: Batch Termination Check │ │ +│ │ shouldTerminateToolBatch(finalizedCalls) │ │ +│ │ - Returns true if ALL tools set terminate: true │ │ +│ │ - If true, agent may stop after this batch │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 9. NEXT TURN PREPARATION │ +│ File: packages/agent/src/agent-loop.ts:224-257 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Step 9.1: Turn End Event │ │ +│ │ turn_end emitted with message and toolResults │ │ +│ │ │ │ +│ │ Step 9.2: prepareNextTurn Hook │ │ +│ │ prepareNextTurn({ message, toolResults, context, newMessages }) │ │ +│ │ - Can return updated context, model, or thinking level │ │ +│ │ - Used for dynamic context management │ │ +│ │ │ │ +│ │ Step 9.3: Queue Polling │ │ +│ │ getSteeringMessages() → inject messages for immediate processing │ │ +│ │ getFollowUpMessages() → check for queued follow-up messages │ │ +│ │ │ │ +│ │ Step 9.4: Loop Decision │ │ +│ │ • If steering messages exist → process them, continue loop │ │ +│ │ • If follow-up messages exist → process them, continue loop │ │ +│ │ • If tool calls remain in message → continue inner loop │ │ +│ │ • If no messages → emit agent_end, exit loop │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 10. AGENT RESPONSE GENERATION │ +│ File: packages/agent/src/agent-loop.ts:58-163 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ The agent loop continues until: │ │ +│ │ • No tool calls remain in assistant messages │ │ +│ │ • No steering/follow-up messages queued │ │ +│ │ • shouldStopAfterTurn() returns true (if configured) │ │ +│ │ │ │ +│ │ Final Response Generation: │ │ +│ │ 1. LLM streams text content blocks │ │ +│ │ 2. Message committed to context │ │ +│ │ 3. turn_end emitted │ │ +│ │ 4. agent_end emitted with all new messages │ │ +│ │ 5. Agent returns to idle state │ │ +│ │ │ │ +│ │ Final Agent Response: │ │ +│ │ "We have 15 products in stock:" │ │ +│ │ • Product A - $29.99 │ │ +│ │ • Product B - $49.99 │ │ +│ │ • Product C - $19.99 │ │ +│ │ (and 2 more products) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +``` + +--- + +## Complete Event Flow Diagram + +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ COMPLETE EVENT SEQUENCE FOR A TURN │ +│ WITH TOOL USE (Product Catalog Query) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +agent_start + ↓ +turn_start + ↓ +message_start (user message: "what product do you have in stock?") +message_end (user message) + ↓ +message_start (assistant message - streaming from LLM) +message_update (text delta: "We have") +message_update (toolcall delta: {"name":"read","arguments":{...}}) +message_update (toolcall end) +message_end (assistant message with tool calls) + ↓ +tool_execution_start (tool call: read products.json) +tool_execution_end (tool call: read complete) +message_start (toolResult message) +message_end (toolResult message) + ↓ +turn_end + ↓ +[Inner loop continues: send tool result to LLM] + ↓ +turn_start + ↓ +message_start (assistant message - streaming from LLM) +message_update (text delta: "We have 15 products in stock:") +message_update (text delta: "• Product A - $29.99") +message_update (text delta: "• Product B - $49.99") +message_end (final assistant message) + ↓ +turn_end + ↓ +agent_end +``` + +--- + +## Key Files Summary + +| Component | File | Purpose | +|-----------|------|---------| +| Agent class | `packages/agent/src/agent.ts` | Stateful wrapper, event emission, queue management | +| Agent loop | `packages/agent/src/agent-loop.ts` | Core loop, LLM calls, tool execution | +| Message types | `packages/agent/src/types.ts` | AgentMessage, AgentTool, AgentEvent definitions | +| Harness | `packages/agent/src/harness/agent-harness.ts` | Session integration, hooks, persistence | +| Read tool | `packages/agent/src/harness/tools/read.ts` | File reading implementation | +| Stream function | `packages/agent/src/stream-fn.ts` | Default stream function management | +| Types | `packages/agent/src/types.ts` | All type definitions | + +--- + +## Hook Points for Customization + +The agent supports multiple extension points: + +| Hook | Location | Purpose | +|------|----------|---------| +| `convertToLlm` | `agent.ts:99` | Transform messages before LLM call | +| `transformContext` | `agent.ts:100` | Modify context (pruning, injection) | +| `beforeToolCall` | `agent.ts:105` | Block or modify tool execution | +| `afterToolCall` | `agent.ts:106` | Override tool results | +| `prepareNextTurn` | `agent.ts:107` | Dynamic context/model updates | +| `shouldStopAfterTurn` | `agent-loop.ts` | Graceful termination | +| `getSteeringMessages` | `agent.ts:114` | Inject messages mid-turn | +| `getFollowUpMessages` | `agent.ts:115` | Queue follow-up messages | + +--- + +## Tool Execution Flow + +``` +Tool Call Received from LLM + ↓ +1. Tool Lookup (find by name) + ↓ +2. prepareArguments? (transform if defined) + ↓ +3. validateToolArguments (JSON Schema) + ↓ +4. beforeToolCall hook (can block) + ↓ +5. execute (parallel or sequential) + ↓ +6. onUpdate (stream partial results) + ↓ +7. afterToolCall hook (can override) + ↓ +8. Create toolResult message + ↓ +9. Emit events (start, end) + ↓ +10. Add to context.messages +``` + +--- + +## Example: "What product do you have in stock?" + +### Step-by-Step Execution: + +1. **User sends message** + ``` + "what product do you have in stock?" + ``` + +2. **Agent normalizes input** + ```typescript + [{ + role: "user", + content: [{ type: "text", text: "what product do you have in stock?" }], + timestamp: Date.now() + }] + ``` + +3. **LLM processes and decides to use `read` tool** + ```json + { + "role": "assistant", + "content": [{ + "type": "toolCall", + "name": "read", + "arguments": { + "path": "products.json", + "offset": 1, + "limit": 50 + }, + "id": "tool_abc123" + }] + } + ``` + +4. **Tool execution** + - Read `products.json` (150 lines total) + - Return lines 1-50 with truncation notice + - Add to context as toolResult + +5. **LLM generates final response** + ``` + We have 15 products in stock: + + • Product A - $29.99 + • Product B - $49.99 + • Product C - $19.99 + • Product D - $99.99 + • Product E - $149.99 + (and 10 more products) + + Use offset=51 to continue viewing. + ``` + +6. **Agent emits final response to user** + +--- + +## Summary + +The agent workflow follows a clear pattern: + +1. **Message Input** → Normalize and validate +2. **Context Setup** → Create snapshot with system prompt, messages, tools +3. **LLM Call** → Transform messages, resolve API key, stream response +4. **Tool Detection** → Check for tool calls in assistant message +5. **Tool Execution** → Validate, hook, execute, stream updates +6. **Result Handling** → Create toolResult message, emit events +7. **Next Turn** → Check for steering/follow-up messages, prepare context +8. **Response Generation** → Continue until no more tool calls needed +9. **Completion** → Emit final response to user + +The entire flow is event-driven, allowing for real-time updates and hook-based customization at every step. diff --git a/packages/agent/learn/architecture.md b/packages/agent/learn/architecture.md new file mode 100644 index 00000000..d29265ec --- /dev/null +++ b/packages/agent/learn/architecture.md @@ -0,0 +1,4494 @@ +# Agent Architecture: Pi Agent Core - A Comprehensive Guide for Julia Implementation + +## Table of Contents +1. [Overview](#overview) +2. [Architecture Layers](#architecture-layers) +3. [Core Components](#core-components) +4. [Message System](#message-system) +5. [Agent Loop](#agent-loop) +6. [Tool Execution](#tool-execution) +7. [Session Management](#session-management) +8. [Memory & Context Management](#memory--context-management) +9. [Event System](#event-system) +10. [Hook System](#hook-system) +11. [Implementation Guide for Julia](#implementation-guide-for-julia) +12. [Data Flow Diagrams](#data-flow-diagrams) +13. [Key Algorithms](#key-algorithms) + +--- + +## Complete System Architecture + +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ COMPLETE SYSTEM ARCHITECTURE DIAGRAM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ USER INPUT │ +│ "what product do you have in stock?" │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ AGENT CLASS (agent.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ State (_state: MutableAgentState) │ │ +│ │ • systemPrompt │ │ +│ │ • model │ │ +│ │ • thinkingLevel │ │ +│ │ • tools (accessor) │ │ +│ │ • messages (accessor) │ │ +│ │ • isStreaming │ │ +│ │ • streamingMessage │ │ +│ │ • pendingToolCalls │ │ +│ │ • errorMessage │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Queues │ │ +│ │ • steeringQueue (PendingMessageQueue) │ │ +│ │ • followUpQueue (PendingMessageQueue) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Hooks │ │ +│ │ • convertToLlm │ │ +│ │ • transformContext │ │ +│ │ • beforeToolCall │ │ +│ │ • afterToolCall │ │ +│ │ • prepareNextTurn │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Methods │ │ +│ │ • prompt(input) → normalizePromptInput() → runPromptMessages() │ │ +│ │ • continue() → runContinuation() │ │ +│ │ • reset() → clear state │ │ +│ │ • subscribe(listener) → unsubscribe() │ │ +│ │ • abort() → signal.abort() │ │ +│ │ • waitForIdle() → activeRun.promise │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ AGENT LOOP (agent_loop.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ runLoop() │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ Outer Loop: │ │ │ +│ │ │ while (true): │ │ │ +│ │ │ • Process pending messages (steering/follow-up) │ │ │ +│ │ │ • streamAssistantResponse() │ │ │ +│ │ │ • executeToolCalls() │ │ │ +│ │ │ • emit(turn_end) │ │ │ +│ │ │ • prepareNextTurn hook │ │ │ +│ │ │ • shouldStopAfterTurn hook │ │ │ +│ │ │ • Check steering/follow-up queues │ │ │ +│ │ │ → Continue or exit │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ streamAssistantResponse() │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ 1. transformContext() (optional) │ │ │ +│ │ │ 2. convertToLlm() │ │ │ +│ │ │ 3. Build LLM context │ │ │ +│ │ │ 4. Resolve API key │ │ │ +│ │ │ 5. streamFunction() → LLM API │ │ │ +│ │ │ 6. Stream events (start, delta*, done) │ │ │ +│ │ │ 7. Commit message to context │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ executeToolCalls() │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ if parallel: │ │ │ +│ │ │ • Preflight sequentially │ │ │ +│ │ │ • Execute concurrently (Promise.all) │ │ │ +│ │ │ │ │ │ +│ │ │ if sequential: │ │ │ +│ │ │ • Execute one-by-one │ │ │ +│ │ │ │ │ │ +│ │ │ For each tool: │ │ │ +│ │ │ • prepareToolCall() │ │ │ +│ │ │ • executePreparedToolCall() │ │ │ +│ │ │ • finalizeExecutedToolCall() │ │ │ +│ │ │ • createToolResultMessage() │ │ │ +│ │ │ • emit(tool_execution_start/end) │ │ │ +│ │ │ • emit(message_start/end) for toolResult │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ PROVIDER ABSTRACTION │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ streamFunction(model, llmContext, options) │ │ +│ │ • Models.streamSimple() from @earendil-works/pi-ai │ │ +│ │ • Makes actual LLM API call │ │ +│ │ • Returns AssistantMessageEventStream │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ SESSION PERSISTENCE (session.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Session │ │ +│ │ • storage: SessionStorage │ │ +│ │ • metadata: SessionMetadata │ │ +│ │ • entries: Dict{String, SessionEntry} │ │ +│ │ • leaf_id: Union{String, Nothing} │ │ +│ │ │ │ +│ │ Methods: │ │ +│ │ • get_branch() → SessionTreeEntry[] │ │ +│ │ • build_context() → SessionContext │ │ +│ │ • append_message() │ │ +│ │ • append_compaction() │ │ +│ │ • append_branch_summary() │ │ +│ │ • set_leaf_id() │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ SessionStorage (JSONL) │ │ +│ │ • file_path: String │ │ +│ │ • header: SessionMetadata │ │ +│ │ • entries: Vector{SessionEntry} │ │ +│ │ │ │ +│ │ Entry Types: │ │ +│ │ • message (user/assistant/toolResult) │ │ +│ │ • compaction (summary) │ │ +│ │ • branch_summary (divergence) │ │ +│ │ • leaf (current pointer) │ │ +│ │ • thinking_level_change │ │ +│ │ • model_change │ │ +│ │ • active_tools_change │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ MEMORY MANAGEMENT (compaction.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Token Estimation │ │ +│ │ • estimateContextTokens(messages) │ │ +│ │ • estimateTokens(message) (chars / 4 heuristic) │ │ +│ │ │ │ +│ │ Compaction Decision │ │ +│ │ • shouldCompact(contextTokens, contextWindow, settings) │ │ +│ │ • Trigger: contextTokens > contextWindow - reserveTokens │ │ +│ │ │ │ +│ │ Cut Point Finding │ │ +│ │ • findCutPoint(entries, keepRecentTokens) │ │ +│ │ • Walk backward, skip toolResults │ │ +│ │ │ │ +│ │ Summary Generation │ │ +│ │ • prepareCompaction(branchEntries, settings) │ │ +│ │ • generateSummary(messages, previousSummary?) │ │ +│ │ • compact(preparation, model, models) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ EVENT SYSTEM (events.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Event Types │ │ +│ │ • agent_start / agent_end │ │ +│ │ • turn_start / turn_end │ │ +│ │ • message_start / message_update / message_end │ │ +│ │ • tool_execution_start / update / end │ │ +│ │ │ │ +│ │ Agent.subscribe(listener) → unsubscribe() │ │ +│ │ processEvents(event) → await listeners │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ HOOK SYSTEM (hooks.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Hook Points │ │ +│ │ • convertToLlm (AgentMessage[] → Message[]) │ │ +│ │ • transformContext (optional pruning/injection) │ │ +│ │ • beforeToolCall (can block: { block: true }) │ │ +│ │ • afterToolCall (can override: { content, details, isError, ... }) │ │ +│ │ • prepareNextTurn (context/model/thinkingLevel update) │ │ +│ │ • shouldStopAfterTurn (graceful termination) │ │ +│ │ • getSteeringMessages (mid-turn injection) │ │ +│ │ • getFollowUpMessages (post-agent execution) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## Overview + +The Pi Agent Core is a sophisticated stateful agent system with the following characteristics: + +- **Stateful execution**: Maintains conversation history and context across multiple turns +- **Tool execution**: Supports LLM tool calling with parallel/sequential execution modes +- **Event streaming**: Real-time event system for UI updates +- **Session persistence**: JSONL-based persistent storage with tree-structured branching +- **Memory compaction**: Automatic context window management through LLM summarization +- **Flexible extension**: Hook-based customization at every system boundary + +### Key Design Principles + +1. **Separation of concerns**: Core agent logic is separated from storage and provider implementations +2. **Streaming first**: All operations are designed around async streams for responsiveness +3. **Type safety**: Strong TypeScript types for compile-time guarantees +4. **Extensibility**: Hooks at every major boundary allow customization +5. **Persistence**: Session history survives restarts through JSONL files + +--- + +## High-Level Architecture Overview + +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ USER INPUT │ +│ "what product do you have in stock?" │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 1. AGENT.PROMPT() ENTRY │ +│ File: packages/agent/src/agent.ts:339 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ • Validate no active run (throws if busy) │ │ +│ │ • normalizePromptInput() converts string to AgentMessage[] │ │ +│ │ • runPromptMessages() launches execution with lifecycle events │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 2. AGENT LIFECYCLE INITIALIZATION │ +│ File: packages/agent/src/agent.ts:398-412 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Events Emitted: │ │ +│ │ • agent_start │ │ +│ │ • turn_start │ │ +│ │ • message_start / message_end (for each prompt message) │ │ +│ │ │ │ +│ │ Context Snapshot Created: │ │ +│ │ • systemPrompt │ │ +│ │ • messages (copy) │ │ +│ │ • tools (copy) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 3. AGENT LOOP STARTS │ +│ File: packages/agent/src/agent-loop.ts:95 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ runPromptMessages() → runAgentLoop() │ │ +│ │ • Prompts added to context.messages │ │ +│ │ • Lifecycle events emitted for prompts │ │ +│ │ • Calls runLoop() (main processing loop) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 4. LLM CALL BOUNDARY - MESSAGE TRANSFORMATION │ +│ File: packages/agent/src/agent-loop.ts:281-372 (streamAssistantResponse) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Step 4.1: Context Transform (optional) │ │ +│ │ transformContext(messages) → transformed messages │ │ +│ │ (Used for context pruning/injection) │ │ +│ │ │ │ +│ │ Step 4.2: LLM Conversion │ │ +│ │ convertToLlm(messages) → Message[] │ │ +│ │ - Filters non-LLM messages (bashExecution, branchSummary, etc.) │ │ +│ │ - Converts: user → user, assistant → assistant, toolResult → toolResult │ │ +│ │ │ │ +│ │ Step 4.3: Build LLM Context │ │ +│ │ { │ │ +│ │ systemPrompt: context.systemPrompt, │ │ +│ │ messages: llmMessages, │ │ +│ │ tools: context.tools │ │ +│ │ } │ │ +│ │ │ │ +│ │ Step 4.4: Resolve API Key │ │ +│ │ getApiKey(model.provider) → apiKey │ │ +│ │ │ │ +│ │ Step 4.5: Stream Function Call │ │ +│ │ streamFunction(model, llmContext, options) │ │ +│ │ - Default: Models.streamSimple() from @earendil-works/pi-ai │ │ +│ │ - Makes actual LLM API call │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 5. LLM RESPONSE STREAMING │ +│ File: packages/agent/src/agent-loop.ts:317-371 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Stream yields events: │ │ +│ │ • start → creates partial AssistantMessage │ │ +│ │ • text_start → streaming text begins │ │ +│ │ • text_delta → text chunks arrive │ │ +│ │ • toolcall_start → tool call block begins │ │ +│ │ • toolcall_delta → tool call arguments arrive │ │ +│ │ • toolcall_end → tool call block complete │ │ +│ │ • text_end → text block complete │ │ +│ │ • done → final message complete │ │ +│ │ │ │ +│ │ State Updates: │ │ +│ │ • Partial message pushed to context.messages │ │ +│ │ • message_start event emitted │ │ +│ │ • message_update events emitted as text/tools stream in │ │ +│ │ • Final message committed to context.messages │ │ +│ │ • message_end event emitted │ │ +│ │ │ │ +│ │ Stop Reasons: │ │ +│ │ • stop - normal completion │ │ +│ │ • toolUse - model requested tool calls │ │ +│ │ • length - token limit reached │ │ +│ │ • error - failure │ │ +│ │ • aborted - operation aborted │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 6. TOOL CALL PARSING AND EXECUTION │ +│ File: packages/agent/src/agent-loop.ts:408-554 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Step 6.1: Extract Tool Calls │ │ +│ │ toolCalls = message.content.filter(c => c.type === "toolCall") │ │ +│ │ │ │ +│ │ Step 6.2: Determine Execution Mode │ │ +│ │ - Check config.toolExecution: "parallel" or "sequential" │ │ +│ │ - Check individual tool executionMode setting │ │ +│ │ - Decides how to execute tool batch │ │ +│ │ │ │ +│ │ Step 6.3: For Each Tool Call │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ 1. Tool Lookup │ │ │ +│ │ │ tool = context.tools.find(t => t.name === toolCall.name) │ │ │ +│ │ │ │ │ │ +│ │ │ 2. Argument Preparation │ │ │ +│ │ │ prepared = tool.prepareArguments?(toolCall.arguments) │ │ │ +│ │ │ │ │ │ +│ │ │ 3. Argument Validation │ │ │ +│ │ │ validateToolArguments(tool, preparedToolCall) │ │ │ +│ │ │ │ │ │ +│ │ │ 4. Before Tool Hook │ │ │ +│ │ │ beforeToolCall({ assistantMessage, toolCall, args, context }) │ │ │ +│ │ │ - Can block execution by returning { block: true, reason } │ │ │ +│ │ │ │ │ │ +│ │ │ 5. Execution │ │ │ +│ │ │ execute(toolCallId, params, signal, onUpdate, context) │ │ │ +│ │ │ │ │ │ +│ │ │ - Parallel Mode: Tools execute concurrently │ │ │ +│ │ │ - Sequential Mode: Tools execute one-by-one │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 7. TOOL EXECUTION EXAMPLE - READ TOOL │ +│ File: packages/agent/src/harness/tools/read.ts │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ User asks: "what product do you have in stock?" │ │ +│ │ │ │ +│ │ Agent decides to read catalog file "products.json" │ │ +│ │ │ │ +│ │ Input Arguments: │ │ +│ │ { │ │ +│ │ "path": "products.json", │ │ +│ │ "offset": 1, │ │ +│ │ "limit": 100 │ │ +│ │ } │ │ +│ │ │ │ +│ │ Execution Steps: │ │ +│ │ 1. resolveReadToolPath(env, path, signal) → absolutePath │ │ +│ │ 2. env.readBinaryFile(absolutePath, signal) → bytes │ │ +│ │ 3. Detect mimeType (check if image) │ │ +│ │ 4. For text files: │ │ +│ │ - Decode UTF-8 → textContent │ │ +│ │ - Split by lines → allLines │ │ +│ │ - Apply offset/limit slicing │ │ +│ │ - Truncate if exceeds DEFAULT_MAX_BYTES or DEFAULT_MAX_LINES │ │ +│ │ - Add truncation notice to output │ │ +│ │ 5. Return result: │ │ +│ │ { │ │ +│ │ content: [{ type: "text", text: output }], │ │ +│ │ details: { truncation: ... } │ │ +│ │ } │ │ +│ │ │ │ +│ │ Output Example: │ │ +│ │ "Showing lines 1-50 of 150. [Showing 50 lines of 150. Use offset=51 to │ │ +│ │ continue.]" │ │ +│ │ │ │ +│ │ Tool Result: │ │ +│ │ { │ │ +│ │ "role": "toolResult", │ │ +│ │ "toolCallId": "tool_abc123", │ │ +│ │ "toolName": "read", │ │ +│ │ "content": [{ "type": "text", "text": "..." }], │ │ +│ │ "isError": false, │ │ +│ │ "timestamp": 1721721600000 │ │ +│ │ } │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 8. TOOL RESULT HANDLING │ +│ File: packages/agent/src/agent-loop.ts:556-792 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Step 8.1: After Tool Hook │ │ +│ │ afterToolCall({ assistantMessage, toolCall, args, result, isError, ctx }) │ │ +│ │ - Can override result content, details, usage, terminate hint │ │ +│ │ │ │ +│ │ Step 8.2: Create Tool Result Message │ │ +│ │ { │ │ +│ │ role: "toolResult", │ │ +│ │ toolCallId: toolCall.id, │ │ +│ │ toolName: toolCall.name, │ │ +│ │ content: result.content ?? [], │ │ +│ │ details: result.details, │ │ +│ │ usage: result.usage, │ │ +│ │ isError: false, │ │ +│ │ timestamp: Date.now() │ │ +│ │ } │ │ +│ │ │ │ +│ │ Step 8.3: Emit Events │ │ +│ │ • tool_execution_start │ │ +│ │ • tool_execution_end │ │ +│ │ • message_start (toolResult message) │ │ +│ │ • message_end (toolResult message) │ │ +│ │ │ │ +│ │ Step 8.4: Update Context │ │ +│ │ • Push tool result message to currentContext.messages │ │ +│ │ • Push to newMessages array │ │ +│ │ │ │ +│ │ Step 8.5: Batch Termination Check │ │ +│ │ shouldTerminateToolBatch(finalizedCalls) │ │ +│ │ - Returns true if ALL tools set terminate: true │ │ +│ │ - If true, agent may stop after this batch │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 9. NEXT TURN PREPARATION │ +│ File: packages/agent/src/agent-loop.ts:224-257 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Step 9.1: Turn End Event │ │ +│ │ turn_end emitted with message and toolResults │ │ +│ │ │ │ +│ │ Step 9.2: prepareNextTurn Hook │ │ +│ │ prepareNextTurn({ message, toolResults, context, newMessages }) │ │ +│ │ - Can return updated context, model, or thinking level │ │ +│ │ - Used for dynamic context management │ │ +│ │ │ │ +│ │ Step 9.3: Queue Polling │ │ +│ │ getSteeringMessages() → inject messages for immediate processing │ │ +│ │ getFollowUpMessages() → check for queued follow-up messages │ │ +│ │ │ │ +│ │ Step 9.4: Loop Decision │ │ +│ │ • If steering messages exist → process them, continue loop │ │ +│ │ • If follow-up messages exist → process them, continue loop │ │ +│ │ • If tool calls remain in message → continue inner loop │ │ +│ │ • If no messages → emit agent_end, exit loop │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 10. AGENT RESPONSE GENERATION │ +│ File: packages/agent/src/agent-loop.ts:58-163 │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ The agent loop continues until: │ │ +│ │ • No tool calls remain in assistant messages │ │ +│ │ • No steering/follow-up messages queued │ │ +│ │ • shouldStopAfterTurn() returns true (if configured) │ │ +│ │ │ │ +│ │ Final Response Generation: │ │ +│ │ 1. LLM streams text content blocks │ │ +│ │ 2. Message committed to context │ │ +│ │ 3. turn_end emitted │ │ +│ │ 4. agent_end emitted with all new messages │ │ +│ │ 5. Agent returns to idle state │ │ +│ │ │ │ +│ │ Final Agent Response: │ │ +│ │ "We have 15 products in stock:" │ │ +│ │ • Product A - $29.99 │ │ +│ │ • Product B - $49.99 │ │ +│ │ • Product C - $19.99 │ │ +│ │ (and 2 more products) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +--- + +--- + +## Architecture Layers + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ User Interface │ +│ (VS Code Extension, CLI, etc.) │ +└─────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────┐ +│ Agent Harness (agent-harness.ts) │ +│ - Session integration │ +│ - Hook system │ +│ - Skill/prompt template management │ +│ - System prompt building │ +└─────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────┐ +│ Agent Class (agent.ts) │ +│ - State management (messages, tools, model) │ +│ - Event emission │ +│ - Steering/follow-up queues │ +│ - Active run management │ +└─────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────┐ +│ Agent Loop (agent-loop.ts) │ +│ - Main execution loop │ +│ - LLM call orchestration │ +│ - Tool execution │ +│ - Context transformation │ +└─────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────┐ +│ Provider Abstraction │ +│ (@earendil-works/pi-ai - external) │ +│ - LLM API calls │ +│ - Streaming interface │ +│ - Retry policies │ +└─────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────┐ +│ Session Storage (session/) │ +│ - JSONL file persistence │ +│ - Tree-structured history │ +│ - Compaction and summarization │ +└─────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## Core Components + +### 1. Agent Class (`agent.ts`) + +**Purpose**: Stateful wrapper around the low-level agent loop + +**Architecture Flow**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ AGENT CLASS ARCHITECTURE │ +├─────────────────────────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ State (_state: MutableAgentState) │ │ +│ │ • systemPrompt │ │ +│ │ • model │ │ +│ │ • thinkingLevel │ │ +│ │ • tools (accessor) │ │ +│ │ • messages (accessor) │ │ +│ │ • isStreaming │ │ +│ │ • streamingMessage │ │ +│ │ • pendingToolCalls │ │ +│ │ • errorMessage │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Queues (PendingMessageQueue) │ │ +│ │ • steeringQueue │ │ +│ │ └─ steer() → queue message for immediate injection │ │ +│ │ • followUpQueue │ │ +│ │ └─ followUp() → queue message for post-agent execution │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Hooks (optional customization points) │ │ +│ │ • convertToLlm │ │ +│ │ • transformContext │ │ +│ │ • beforeToolCall │ │ +│ │ • afterToolCall │ │ +│ │ • prepareNextTurn │ │ +│ │ • prepareNextTurnWithContext │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Active Run Management │ │ +│ │ • activeRun: { promise, resolve, abortController } │ │ +│ │ • abort() → sets signal.aborted │ │ +│ │ • waitForIdle() → returns promise that settles after all listeners │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Event Subscription │ │ +│ │ • listeners: Set │ │ +│ │ • subscribe(listener) → unsubscribe() │ │ +│ │ • processEvents(event) → await listeners for event │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +**Key responsibilities**: +- Maintain in-memory transcript (`_state.messages`) +- Manage tools, system prompt, model configuration +- Emit lifecycle events to subscribers +- Handle steering/follow-up message queues +- Prevent concurrent runs (single active run at a time) + +**State structure**: +```typescript +interface AgentState { + systemPrompt: string + model: Model + thinkingLevel: ThinkingLevel + tools: AgentTool[] + messages: AgentMessage[] + isStreaming: boolean + streamingMessage?: AgentMessage + pendingToolCalls: Set + errorMessage?: string +} +``` + +**Key methods**: +- `prompt(message)`: Start a new prompt from text, message, or array +- `continue()`: Continue from current context +- `steer(message)`: Queue steering message (interruption during tool execution) +- `followUp(message)`: Queue follow-up message (executes after agent stops) +- `reset()`: Clear all state +- `subscribe(listener)`: Register event listener +- `abort()`: Cancel current operation +- `waitForIdle()`: Wait for completion and event listeners + +### 2. Agent Loop (`agent-loop.ts`) + +**Purpose**: Core execution logic without state management + +**Outer Loop Flow**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ OUTER LOOP: TURN MANAGEMENT │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ while (true): │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Inner Loop (tool calls + steering messages) │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ while (hasMoreToolCalls || pendingMessages): │ │ │ +│ │ │ │ │ │ +│ │ │ 1. Process pending messages │ │ │ +│ │ │ (steering/follow-up injection) │ │ │ +│ │ │ │ │ │ +│ │ │ 2. Stream assistant response from LLM │ │ │ +│ │ │ (streamAssistantResponse) │ │ │ +│ │ │ │ │ │ +│ │ │ 3. Extract tool calls │ │ │ +│ │ │ (filter toolCall blocks from content) │ │ │ +│ │ │ │ │ │ +│ │ │ 4. Execute tool batch │ │ │ +│ │ │ (executeToolCalls: parallel or sequential) │ │ │ +│ │ │ │ │ │ +│ │ │ 5. Update context │ │ │ +│ │ │ (add tool results to messages) │ │ │ +│ │ │ │ │ │ +│ │ │ 6. Emit turn_end │ │ │ +│ │ │ │ │ │ +│ │ │ 7. Check prepareNextTurn hook │ │ │ +│ │ │ │ │ │ +│ │ │ 8. Check shouldStopAfterTurn hook │ │ │ +│ │ │ │ │ │ +│ │ │ 9. Get steering messages │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ │ │ │ +│ │ Check follow-up messages │ │ +│ │ └─ if follow-ups exist → continue outer loop │ │ +│ │ else → emit agent_end, exit │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +**Inner Loop: LLM Call Boundary**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ LLM CALL BOUNDARY: streamAssistantResponse() │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 1. Context Transform (optional) │ +│ transformContext(messages) → transformed messages │ +│ (pruning, context injection) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 2. Convert to LLM format │ +│ convertToLlm(messages) → Message[] │ +│ (filter non-LLM messages, convert custom types) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 3. Build LLM Context │ +│ { │ +│ systemPrompt: context.systemPrompt, │ +│ messages: llmMessages, │ +│ tools: context.tools │ +│ } │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 4. Resolve API Key │ +│ getApiKey(model.provider) → apiKey │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 5. Stream Function Call │ +│ streamFunction(model, llmContext, options) │ +│ (calls LLM provider API, returns AssistantMessageEventStream) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 6. Stream Events │ +│ for await (event of response) { │ +│ case "start": → partialMessage = event.partial │ +│ case "text_delta": → update partialMessage │ +│ case "toolcall_delta": → update partialMessage │ +│ case "done": → commit final message │ +│ case "error": → commit error message │ +│ } │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ 7. Commit Message │ +│ If partial existed: context.messages[.end] = final │ +│ Else: context.messages.push(final) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +**Tool Execution Flow**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ TOOL EXECUTION FLOW │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Assistant Message (with toolCall content blocks) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ extract tool calls from message.content │ +│ filter(c => c.type === "toolCall") │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ determine execution mode │ +│ • config.toolExecution: "parallel" or "sequential" │ +│ • per-tool executionMode override │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ PARALLEL MODE │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ 1. Preflight all tools sequentially │ │ │ +│ │ │ (prepareToolCall for each) │ │ │ +│ │ │ │ │ │ +│ │ │ 2. Queue async executions │ │ │ +│ │ │ (async () => execute + finalize) │ │ │ +│ │ │ │ │ │ +│ │ │ 3. Execute allowed tools concurrently │ │ │ +│ │ │ (Promise.all for queued functions) │ │ │ +│ │ │ │ │ │ +│ │ │ 4. Emit toolResult messages │ │ │ +│ │ │ (in assistant source order) │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ SEQUENTIAL MODE │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ For each tool call in order: │ │ │ +│ │ │ 1. prepareToolCall │ │ │ +│ │ │ 2. executePreparedToolCall │ │ │ +│ │ │ 3. finalizeExecutedToolCall │ │ │ +│ │ │ 4. Emit tool_result messages │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ check shouldTerminateToolBatch() │ +│ (returns true if ALL tools set terminate: true) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +**Main entry points**: +- `runAgentLoop(prompts, context, config, emit, signal, streamFn)`: Start new loop with prompts +- `runAgentLoopContinue(context, config, emit, signal, streamFn)`: Continue from existing context + +**Execution phases**: + +#### Phase 1: Outer Loop (Turn Management) +``` +while (true): + 1. Check steering messages (inject if any) + 2. Stream assistant response from LLM + 3. Extract tool calls from response + 4. Execute tool batch + 5. Emit turn_end event + 6. Check prepareNextTurn hook + 7. Check shouldStopAfterTurn hook + 8. Check follow-up messages + 9. If follow-up exists, continue outer loop + 10. If no follow-up, exit +``` + +#### Phase 2: Inner Loop (Tool Call Processing) +``` +while (hasMoreToolCalls || pendingMessages): + 1. Process pending messages + 2. Stream assistant response + 3. Extract tool calls + 4. Execute tool batch (parallel or sequential) + 5. Emit turn_end +``` + +#### Phase 3: LLM Call Boundary +``` +1. transformContext(messages) // Optional pruning/injection +2. convertToLlm(messages) // Filter/custom message conversion +3. Build LLM context +4. Resolve API key +5. Call streamFunction(model, context, options) +6. Stream events from LLM +7. Commit final message to context +``` + +**Tool execution modes**: +- **Parallel** (default): Preflight sequentially, execute allowed tools concurrently +- **Sequential**: Execute tools one-by-one + +### 3. Types System (`types.ts`) + +**Key types**: + +#### AgentMessage +```typescript +type AgentMessage = Message | CustomAgentMessages[keyof CustomAgentMessages] +``` +Extensible union of LLM messages and custom app-specific messages. + +#### AgentTool +```typescript +interface AgentTool extends Tool { + label: string // UI display name + prepareArguments?: (args) => Static // Argument transformation + execute: (toolCallId, params, signal, onUpdate) => Promise + executionMode?: "parallel" | "sequential" // Per-tool override +} +``` + +#### AgentContext +```typescript +interface AgentContext { + systemPrompt: string + messages: AgentMessage[] + tools?: AgentTool[] +} +``` + +#### AgentLoopConfig +Configuration object passed to low-level loop functions, including: +- Model specification +- convertToLlm transformation +- transformContext (optional) +- beforeToolCall hook +- afterToolCall hook +- prepareNextTurn hook +- shouldStopAfterTurn hook +- getSteeringMessages hook +- getFollowUpMessages hook + +--- + +## Message System + +### Message Types + +#### LLM Messages (standard) +- `user`: User input +- `assistant`: LLM response (streaming, contains tool calls) +- `toolResult`: Tool execution result + +#### Custom Messages (app-specific) +- `bashExecution`: Shell command execution result (hidden from LLM by default) +- `custom`: Custom app messages (visible to LLM if projected) +- `branchSummary`: Branch divergence summary +- `compactionSummary`: History compaction result + +**Message Type Hierarchy**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ MESSAGE TYPE HIERARCHY │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ AgentMessage (Union) │ +├─────────────────────────────────────────────────────────────────────────────────────┤ +│ │ │ +│ ├─ Message (LLM messages) │ +│ │ │ │ +│ │ ├─ UserMessage │ +│ │ │ role: "user" │ +│ │ │ content: TextContent[] │ +│ │ │ timestamp: number │ +│ │ │ │ +│ │ ├─ AssistantMessage │ +│ │ │ role: "assistant" │ +│ │ │ content: (TextContent | ToolCall | Thinking)[] │ +│ │ │ api, provider, model │ +│ │ │ usage: Usage │ +│ │ │ stopReason: string │ +│ │ │ errorMessage?: string │ +│ │ │ timestamp: number │ +│ │ │ │ +│ │ └─ ToolResultMessage │ +│ │ role: "toolResult" │ +│ │ toolCallId, toolName │ +│ │ content: TextContent[] │ +│ │ details, usage │ +│ │ isError: boolean │ +│ │ timestamp: number │ +│ │ │ +│ └─ CustomAgentMessages (app extensions) │ +│ │ │ +│ ├─ BashExecutionMessage │ +│ │ role: "bashExecution" │ +│ │ command, output, exitCode │ +│ │ excludeFromContext?: boolean │ +│ │ │ +│ ├─ CustomMessage │ +│ │ role: "custom" │ +│ │ customType, content, display, details │ +│ │ │ +│ ├─ BranchSummaryMessage │ +│ │ role: "branchSummary" │ +│ │ summary, fromId │ +│ │ │ +│ └─ CompactionSummaryMessage │ +│ role: "compactionSummary" │ +│ summary, tokensBefore │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ MESSAGE FLOW: AGENT → LLM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ AgentMessage[] (in-memory transcript) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ transformContext() (optional) │ +│ (pruning, context injection) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ convertToLlm() │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ for each message: │ │ +│ │ • user/assistant/toolResult → pass through │ │ +│ │ • bashExecution → convert to user message (with command/output) │ │ +│ │ • custom → convert to user message (text content) │ │ +│ │ • branchSummary → convert to user message │ │ +│ │ • compactionSummary → convert to user message │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Message[] (LLM-compatible) │ +│ (only user, assistant, toolResult) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ LLM provider API call │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +### Message Flow + +``` +User Input + ↓ +normalizePromptInput() → AgentMessage[] + ↓ +AgentContext.messages (in-memory) + ↓ +transformContext() (optional, for pruning/injection) + ↓ +convertToLlm() (filters/custom → LLM format) + ↓ +LLM provider (Message[]) + ↓ +AssistantMessage (streamed) + ↓ +AgentContext.messages (committed) +``` + +### Message Commitment + +Messages are committed to `context.messages` at specific points: +1. Assistant partial message: Added when `start` event arrives +2. Assistant final message: Replaces partial on `done`/`error` +3. Tool result message: Added after `tool_execution_end` + +--- + +## Agent Loop + +### Main Algorithm + +```typescript +async function runLoop(initialContext, newMessages, config, signal, emit, streamFn) { + let currentContext = initialContext + let pendingMessages = (await config.getSteeringMessages?.()) || [] + + while (true) { + let hasMoreToolCalls = true + + while (hasMoreToolCalls || pendingMessages.length > 0) { + // Process pending messages (steering/follow-up) + if (pendingMessages.length > 0) { + for (const msg of pendingMessages) { + await emit({ type: "message_start", message: msg }) + await emit({ type: "message_end", message: msg }) + currentContext.messages.push(msg) + newMessages.push(msg) + } + pendingMessages = [] + } + + // Stream assistant response + const message = await streamAssistantResponse( + currentContext, config, signal, emit, streamFn + ) + newMessages.push(message) + + // Check for errors + if (message.stopReason === "error" || message.stopReason === "aborted") { + await emit({ type: "turn_end", message, toolResults: [] }) + await emit({ type: "agent_end", messages: newMessages }) + return + } + + // Extract and execute tool calls + const toolCalls = message.content.filter(c => c.type === "toolCall") + const toolResults = [] + hasMoreToolCalls = false + + if (toolCalls.length > 0) { + const executedBatch = await executeToolCalls( + currentContext, message, config, signal, emit + ) + toolResults.push(...executedBatch.messages) + hasMoreToolCalls = !executedBatch.terminate + + for (const result of toolResults) { + currentContext.messages.push(result) + newMessages.push(result) + } + } + + await emit({ type: "turn_end", message, toolResults }) + + // Check prepareNextTurn hook + const nextTurnSnapshot = await config.prepareNextTurn?.({ + message, toolResults, context: currentContext, newMessages + }) + if (nextTurnSnapshot) { + currentContext = nextTurnSnapshot.context ?? currentContext + config = { ...config, model: nextTurnSnapshot.model ?? config.model } + } + + // Check shouldStopAfterTurn hook + if (await config.shouldStopAfterTurn?.({ ... })) { + await emit({ type: "agent_end", messages: newMessages }) + return + } + + // Get steering messages for next iteration + pendingMessages = (await config.getSteeringMessages?.()) || [] + } + + // Check follow-up messages + const followUpMessages = (await config.getFollowUpMessages?.()) || [] + if (followUpMessages.length > 0) { + pendingMessages = followUpMessages + continue + } + + // No more messages, exit + break + } + + await emit({ type: "agent_end", messages: newMessages }) +} +``` + +### Assistant Response Streaming + +```typescript +async function streamAssistantResponse(context, config, signal, emit, streamFn) { + // 1. Transform context (optional) + let messages = context.messages + if (config.transformContext) { + messages = await config.transformContext(messages, signal) + } + + // 2. Convert to LLM format + const llmMessages = await config.convertToLlm(messages) + + // 3. Build LLM context + const llmContext = { + systemPrompt: context.systemPrompt, + messages: llmMessages, + tools: context.tools, + } + + // 4. Resolve API key + const resolvedApiKey = await config.getApiKey?.(config.model.provider) || config.apiKey + + // 5. Call stream function + const response = await streamFn(config.model, llmContext, { + ...config, apiKey: resolvedApiKey, signal + }) + + // 6. Stream events + let partialMessage: AssistantMessage | null = null + let addedPartial = false + + for await (const event of response) { + switch (event.type) { + case "start": + partialMessage = event.partial + context.messages.push(partialMessage) + addedPartial = true + await emit({ type: "message_start", message: { ...partialMessage } }) + break + + case "text_start" | "text_delta" | "text_end" | + "thinking_start" | "thinking_delta" | "thinking_end" | + "toolcall_start" | "toolcall_delta" | "toolcall_end": + if (partialMessage) { + partialMessage = event.partial + context.messages[context.messages.length - 1] = partialMessage + await emit({ + type: "message_update", + assistantMessageEvent: event, + message: { ...partialMessage } + }) + } + break + + case "done" | "error": { + const finalMessage = await response.result() + if (addedPartial) { + context.messages[context.messages.length - 1] = finalMessage + } else { + context.messages.push(finalMessage) + await emit({ type: "message_start", message: { ...finalMessage } }) + } + await emit({ type: "message_end", message: finalMessage }) + return finalMessage + } + } + } +} +``` + +--- + +## Tool Execution + +### Tool Call Flow + +``` +Assistant Message (with toolCall content blocks) + ↓ +1. Extract tool calls from message.content +2. Determine execution mode (parallel/sequential) +3. For each tool: + - Look up tool by name + - Prepare arguments (prepareArguments hook) + - Validate arguments (JSON Schema) + - beforeToolCall hook (can block) + - Execute tool (parallel or sequential) + - onUpdate stream (optional) + - afterToolCall hook (can override result) + - Create toolResult message + - Emit events + ↓ +4. Check shouldTerminateToolBatch +5. Return toolResult messages +``` + +### Parallel vs Sequential Execution + +#### Parallel Execution +```typescript +async function executeToolCallsParallel(...) { + const finalizedCalls = [] + + // Preflight all tools sequentially + for (const toolCall of toolCalls) { + const preparation = await prepareToolCall(...) + + if (preparation.kind === "immediate") { + finalizedCalls.push(preparation) + } else { + // Queue async execution + finalizedCalls.push(async () => { + const executed = await executePreparedToolCall(preparation, signal, emit) + const finalized = await finalizeExecutedToolCall(...) + return finalized + }) + } + } + + // Execute allowed tools concurrently + const orderedFinalizedCalls = await Promise.all( + finalizedCalls.map(entry => typeof entry === "function" ? entry() : Promise.resolve(entry)) + ) + + // Emit toolResult messages in assistant source order + const messages = [] + for (const finalized of orderedFinalizedCalls) { + const toolResultMessage = createToolResultMessage(finalized) + await emitToolResultMessage(toolResultMessage, emit) + messages.push(toolResultMessage) + } + + return { messages, terminate: shouldTerminateToolBatch(orderedFinalizedCalls) } +} +``` + +#### Sequential Execution +```typescript +async function executeToolCallsSequential(...) { + const finalizedCalls = [] + const messages = [] + + for (const toolCall of toolCalls) { + const preparation = await prepareToolCall(...) + let finalized + + if (preparation.kind === "immediate") { + finalized = preparation + } else { + const executed = await executePreparedToolCall(preparation, signal, emit) + finalized = await finalizeExecutedToolCall(...) + } + + await emitToolExecutionEnd(finalized, emit) + const toolResultMessage = createToolResultMessage(finalized) + await emitToolResultMessage(toolResultMessage, emit) + finalizedCalls.push(finalized) + messages.push(toolResultMessage) + } + + return { messages, terminate: shouldTerminateToolBatch(finalizedCalls) } +} +``` + +### Tool Execution Stages + +1. **Preparation** (`prepareToolCall`) + - Look up tool by name + - Call `prepareArguments` if defined + - Validate with `validateToolArguments` + - Call `beforeToolCall` hook (can block with `{ block: true }`) + - Return `PreparedToolCall` or `ImmediateToolCallOutcome` + +2. **Execution** (`executePreparedToolCall`) + - Call `tool.execute(toolCallId, args, signal, onUpdate)` + - Stream partial results via `onUpdate` + - Handle errors, return `ExecutedToolCallOutcome` + +3. **Finalization** (`finalizeExecutedToolCall`) + - Call `afterToolCall` hook (can override result) + - Return `FinalizedToolCallOutcome` + +4. **Message Creation** (`createToolResultMessage`) + - Create `ToolResultMessage` + - Set `toolCallId`, `toolName`, `content`, `details`, `usage`, `isError`, `timestamp` + +--- + +## Session Management + +### JSONL Storage Format + +```jsonl +{"type":"session","version":3,"id":"abc123","timestamp":"2024-01-15T10:30:00.000Z", + "cwd":"/home/user/project","parentSession":"...","metadata":{}} +{"type":"message","id":"e001","parentId":null,"timestamp":"...","message":{...}} +{"type":"message","id":"e002","parentId":"e001","timestamp":"...","message":{...}} +{"type":"compaction","id":"e003","parentId":"e002","timestamp":"...", + "summary":"## Goal: ...\n...","firstKeptEntryId":"e001", + "tokensBefore":185000} +{"type":"leaf","id":"e004","parentId":"e003","timestamp":"...", + "targetId":"e002"} +``` + +**Session File Structure**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ SESSION FILE (.jsonl) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Line 1: Session Header (metadata) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ { │ │ +│ │ "type": "session", │ │ +│ │ "version": 3, │ │ +│ │ "id": "abc123", │ │ +│ │ "timestamp": "2024-01-15T10:30:00.000Z", │ │ +│ │ "cwd": "/home/user/project", │ │ +│ │ "parentSessionPath": "/path/to/parent.jsonl", │ │ +│ │ "metadata": {} │ │ +│ │ } │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Lines 2+: Entries (one per line) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ { │ │ +│ │ "type": "message", │ │ +│ │ "id": "e001", │ │ +│ │ "parentId": null, │ │ +│ │ "timestamp": "...", │ │ +│ │ "message": { /* message data */ } │ │ +│ │ } │ │ +│ │ │ │ +│ │ { │ │ +│ │ "type": "compaction", │ │ +│ │ "id": "e003", │ │ +│ │ "parentId": "e002", │ │ +│ │ "timestamp": "...", │ │ +│ │ "summary": "...", │ │ +│ │ "firstKeptEntryId": "e001", │ │ +│ │ "tokensBefore": 185000 │ │ +│ │ } │ │ +│ │ │ │ +│ │ { │ │ +│ │ "type": "leaf", │ │ +│ │ "id": "e004", │ │ +│ │ "parentId": "e003", │ │ +│ │ "timestamp": "...", │ │ +│ │ "targetId": "e002" │ │ +│ │ } │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +### Entry Types + +| Type | LLM Context? | Purpose | +|------|-------------|---------| +| `message` | Yes | User, assistant, toolResult | +| `compaction` | Yes (as summary message) | Replaces compacted history | +| `branch_summary` | Yes (as summary message) | Summary of diverged branch | +| `leaf` | No | Points to current tree leaf | +| `thinking_level_change` | No | Tracking thinking level changes | +| `model_change` | No | Tracking model changes | +| `active_tools_change` | No | Tracking tool enable/disable | +| `custom` | No (unless projected) | App-defined data | +| `custom_message` | Yes | App-defined messages | +| `label` | No | Human-readable labels | +| `session_info` | No | Session name history | + +**Session Tree Structure**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ SESSION TREE (Branch Navigation) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + + ┌───[e01]───┐ + │ user: "a" │ + └─────┬──────┘ + ▼ + ┌──────────┐ + │ assist 1 │ + └─────┬────┘ + ▼ + ┌──────────┐ + │ toolCall │ + └─────┬────┘ + ▼ + ┌──────────┐ + │ toolRes 1│ + └─────┬────┘ + ▼ + ┌──────────┐ + │ user: "b"│ ← user navigates here + └─────┬────┘ + │ + ┌─────┴─────┐ + │ │ + ┌──────────┐ ┌──────────┐ + │ user: "c" │ │ user: "d" │ ← branch point + └────┬─────┘ └────┬─────┘ + │ │ + ┌────▼─────┐ ┌────▼─────┐ + │ assist 2 │ │ assist 3 │ ← current leaf (d) + └──────────┘ └──────────┘ + +Context sent to LLM (when leaf is at "d"): + [compaction, user:b, user:d, assist:3] + +When user navigates to "b": + 1. Leaf moves from "d" back to "b" + 2. Branch summary generated for diverged work ("c" → "assist 2") + 3. Context: [compaction, branch_summary, turns 1-8] + 4. New branch grows from "b" +``` + +### Session Tree (Branching) + +Sessions form a **tree**, not a linear log: + +``` + ┌───[e01]───┐ + │ user: "a" │ + └─────┬──────┘ + ▼ + ┌──────────┐ + │ assist 1 │ + └─────┬────┘ + ▼ + ┌──────────┐ + │ toolCall │ + └─────┬────┘ + ▼ + ┌──────────┐ + │ toolRes 1│ + └─────┬────┘ + ▼ + ┌──────────┐ + │ user: "b"│ ← user goes back here + └─────┬────┘ + │ + ┌─────┴─────┐ + │ │ + ┌──────────┐ ┌──────────┐ + │ user: "c" │ │ user: "d" │ ← branch point + └────┬─────┘ └────┬─────┘ + │ │ + ┌────▼─────┐ ┌────▼─────┐ + │ assist 2 │ │ assist 3 │ ← current leaf (d) + └──────────┘ └──────────┘ +``` + +**Branch navigation**: +1. User navigates to a different point in history +2. Leaf moves to the selected entry +3. Branch summary generated for diverged work +4. New work branches from the selected point + +### Session API + +```typescript +class Session { + async getBranch(fromId?: string): Promise + async buildContext(options?: SessionContextBuildOptions): Promise + async getLeafId(): Promise + async setLeafId(id: string): Promise + async appendMessage(message: AgentMessage): Promise + async appendCompaction(summary: string, firstKeptEntryId: string, tokensBefore: number): Promise + async appendBranchSummary(summary: string, fromId: string): Promise + async appendLeaf(targetId: string): Promise +} +``` + +--- + +## Memory & Context Management + +### Token Estimation + +**Character-based heuristic** (4 chars ≈ 1 token): +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ TOKEN ESTIMATION ALGORITHM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ estimateTokens(message) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "user" │ +│ content.length / 4 │ +│ (images ≈ 4800 chars each) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "assistant" │ +│ sum of all content blocks: │ +│ • text blocks → text.length │ +│ • thinking blocks → thinking.length │ +│ • toolCall blocks → name.length + JSON.stringify(args).length │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "toolResult" / "custom" │ +│ content.length / 4 │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "bashExecution" │ +│ (command.length + output.length) / 4 │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "compactionSummary" / "branchSummary" │ +│ summary.length / 4 │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +### Compaction Strategy + +**Trigger condition**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ COMPACTION TRIGGER DECISION │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +contextTokens > contextWindow - reserveTokens + +Example (Claude with 200K context window): + Triggers when: contextTokens > 200000 - 16384 = 183616 + +Default Settings: + reserveTokens: 16384 (~16K for summary prompt + output) + keepRecentTokens: 20000 (~20K tokens of recent history to keep) +``` + +**Cut Point Finding Algorithm**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ CUT POINT SELECTION │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ findCutPoint(entries, startIndex, endIndex, keepRecentTokens) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + + Accumulate = 0 + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Walk BACKWARD from endIndex │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ for i from endIndex-1 down to startIndex: │ │ +│ │ │ │ +│ │ // Skip invalid cut points │ │ +│ │ if entry is toolResult message: │ │ +│ │ continue // tool results stay with their call │ │ +│ │ │ │ +│ │ tokens = estimateTokens(entry) │ │ +│ │ accumulated += tokens │ │ +│ │ │ │ +│ │ if accumulated >= keepRecentTokens: │ │ +│ │ return i + 1 // Snap to nearest valid cut point │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Valid Cut Points (safe to split): │ +│ • user message │ +│ • assistant message │ +│ • custom message │ +│ • branch_summary │ +│ │ +│ NOT Valid (tool results stay with their call): │ +│ • toolResult message (skipped) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +Example: + ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ + │ u1 │ │ a1 │ │ tr1│ │ u2 │ │ a2 │ │ tr2│ │ u3 │ + └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ + ▲ ▲ ▲ + │ │ │ + └── kept └── cut └── discarded + (~20K tokens) point history +``` + +**Compaction Preparation**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ COMPACTION PREPARATION │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ prepareCompaction(branchEntries, settings) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + + 1. Find previous compaction (if any) → previousSummary + │ + ▼ + 2. Estimate tokens of current context + │ + ▼ + 3. findCutPoint() → firstKeptEntryId + │ + ▼ + 4. Split into 3 groups: + │ + ├─ messagesToSummarize: entries BEFORE cut point + │ (these become the summary) + │ + ├─ retainedTail: entries AFTER cut point + │ (these stay verbatim) + │ + └─ turnPrefixMessages: if cut splits a turn + (the beginning of an interrupted turn) + │ + ▼ + 5. Extract file operations from messagesToSummarize: + └─ readFiles, modifiedFiles +``` + +**Summary Generation**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ SUMMARY GENERATION │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ generateSummary(messages, previousSummary?) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + + ┌──────────────────────────────────────────────────────────────────────────────┐ │ + │ Has previousSummary? │ │ + │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ + │ │ YES → UPDATE_SUMMARIZATION_PROMPT (iterative) │ │ │ + │ │ ┌────────────────────────────────────────────────────────────────┐ │ │ │ + │ │ │ │ │ │ │ + │ │ │ ${previousSummary} │ │ │ │ + │ │ │ │ │ │ │ + │ │ │ │ │ │ │ + │ │ │ │ │ │ │ + │ │ │ ${serializeConversation(messages)} │ │ │ │ + │ │ │ │ │ │ │ + │ │ │ │ │ │ │ + │ │ │ Update the previous summary with new progress: │ │ │ │ + │ │ │ - Add completed tasks │ │ │ │ + │ │ │ - Update progress │ │ │ │ + │ │ │ - Add new goals │ │ │ │ + │ │ │ - Keep existing information │ │ │ │ + │ │ └────────────────────────────────────────────────────────────────┘ │ │ │ + │ └──────────────────────────────────────────────────────────────────────┘ │ │ + │ │ │ + │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ + │ │ NO → FRESH_SUMMARIZATION_PROMPT (fresh) │ │ │ + │ │ ┌────────────────────────────────────────────────────────────────┐ │ │ │ + │ │ │ │ │ │ │ + │ │ │ ${serializeConversation(messages)} │ │ │ │ + │ │ │ │ │ │ │ + │ │ │ │ │ │ │ + │ │ │ Generate a summary: │ │ │ │ + │ │ │ ## Goal │ │ │ │ + │ │ │ ## Constraints & Preferences │ │ │ │ + │ │ │ ## Progress │ │ │ │ + │ │ │ ### Done │ │ │ │ + │ │ │ ### In Progress │ │ │ │ + │ │ │ ### Blocked │ │ │ │ + │ │ │ ## Key Decisions │ │ │ │ + │ │ │ ## Next Steps │ │ │ │ + │ │ │ ## Critical Context │ │ │ │ + │ │ │ Files read: [...] │ │ │ │ + │ │ │ Files modified: [...] │ │ │ │ + │ │ └────────────────────────────────────────────────────────────────┘ │ │ │ + │ └──────────────────────────────────────────────────────────────────────┘ │ │ + └──────────────────────────────────────────────────────────────────────────────┘ │ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ LLM generates structured summary: │ +│ │ +│ ## Goal │ +│ - [What is the user trying to accomplish?] │ +│ │ +│ ## Constraints & Preferences │ +│ - [Any constraints, preferences, or requirements] │ +│ │ +│ ## Progress │ +│ ### Done │ +│ - [x] [Completed tasks] │ +│ ### In Progress │ +│ - [ ] [Current work] │ +│ ### Blocked │ +│ - [Issues preventing progress] │ +│ │ +│ ## Key Decisions │ +│ - **[Decision]**: [Brief rationale] │ +│ │ +│ ## Next Steps │ +│ 1. [Ordered list of what should happen next] │ +│ │ +│ ## Critical Context │ +│ - [Any data, examples, or references needed to continue] │ +│ │ +│ Files read: [src/index.ts, package.json] │ +│ Files modified: [src/index.ts] │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +**Iterative Compaction**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ ITERATIVE COMPACTION │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +Compaction 1 (at ~185K tokens): + ┌──────────────────────────────────────────────────────────────────────────────┐ │ + │ Summary: "## Goal: Build a login page..." │ │ + │ firstKeptEntryId: "entry-003" │ │ + └──────────────────────────────────────────────────────────────────────────────┘ │ + +Compaction 2 (at ~185K tokens again): + ┌──────────────────────────────────────────────────────────────────────────────┐ │ + │ previousSummary: "## Goal: Build a login page..." │ │ + │ → UPDATE_SUMMARIZATION_PROMPT │ │ + │ → PRESERVES existing information │ │ + │ → ADDS new progress (move "In Progress" → "Done") │ │ + │ → NEW summary: "## Goal: Build a login page... Add OAuth..." │ │ + │ firstKeptEntryId: "entry-003" (same boundary) │ │ + └──────────────────────────────────────────────────────────────────────────────┘ │ + +Result: + • History before cut point replaced by summary + • Cut point stays at same location + • Summary grows incrementally with new progress + • Token budget maintained (~25K tokens after compaction) +``` + +--- + +## Event System + +### Event Types + +#### Agent Lifecycle +- `agent_start`: Agent begins processing +- `agent_end`: Final event for the run + +#### Turn Lifecycle +- `turn_start`: New turn begins +- `turn_end`: Turn completes with assistant message and tool results + +#### Message Lifecycle +- `message_start`: Any message begins +- `message_update`: **Assistant only**. Includes `assistantMessageEvent` with delta +- `message_end`: Message completes + +#### Tool Execution Lifecycle +- `tool_execution_start`: Tool begins +- `tool_execution_update`: Tool streams progress +- `tool_execution_end`: Tool completes + +**Complete Event Flow**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ COMPLETE EVENT SEQUENCE │ +│ (Agent with Tool Calls - Product Query) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +agent_start + ↓ +turn_start + ↓ +message_start (user message: "what product do you have in stock?") +message_end (user message) + ↓ +message_start (assistant message - streaming from LLM) +message_update (text delta: "We have") +message_update (toolcall delta: {"name":"read","arguments":{...}}) +message_update (toolcall end) +message_end (assistant message with tool calls) + ↓ +tool_execution_start (tool call: read products.json) +tool_execution_end (tool call: read complete) +message_start (toolResult message) +message_end (toolResult message) + ↓ +turn_end + ↓ +[Inner loop continues: send tool result to LLM] + ↓ +turn_start + ↓ +message_start (assistant message - streaming from LLM) +message_update (text delta: "We have 15 products in stock:") +message_update (text delta: "• Product A - $29.99") +message_update (text delta: "• Product B - $49.99") +message_end (final assistant message) + ↓ +turn_end + ↓ +agent_end +``` + +**Event Sequence Without Tools**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ EVENT FLOW (No Tool Calls) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +prompt("Hello!") + ↓ +agent_start + ↓ +turn_start + ↓ +message_start (user) +message_end (user) + ↓ +message_start (assistant - streaming) +message_update (text_delta) +message_update (text_delta) +message_end (assistant) + ↓ +turn_end + ↓ +agent_end +``` + +**Event Sequence with Steering Messages**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ EVENT FLOW (With Steering Messages) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +Turn 1: Agent executing tool calls... + ↓ +turn_end + ↓ +[Agent would stop, but...] + ↓ +steer({ role: "user", content: "Stop! Do this instead." }) + ↓ +turn_start (steering) + ↓ +message_start (steering message) +message_end (steering message) + ↓ +message_start (assistant response to steering) +message_update (text_delta) +message_end (assistant) + ↓ +turn_end + ↓ +agent_end +``` + +### Event Flow with Tools + +``` +agent_start + ↓ +turn_start + ↓ +message_start { user message } +message_end { user message } + ↓ +message_start { assistant message - streaming } +message_update { text_delta: "We have" } +message_update { toolcall_delta: {"name":"read","arguments":{...}} } +message_update { toolcall_end } +message_end { assistant message with tool calls } + ↓ +tool_execution_start { tool call: read products.json } +tool_execution_end { tool call: complete } +message_start { toolResult message } +message_end { toolResult message } + ↓ +turn_end { message, toolResults: [toolResult] } + ↓ +[Inner loop continues: send tool result to LLM] + ↓ +turn_start + ↓ +message_start { assistant message } +message_update { text_delta: "We have 15 products" } +message_end { final assistant message } + ↓ +turn_end + ↓ +agent_end +``` + +### Subscription Model + +```typescript +const unsubscribe = agent.subscribe(async (event, signal) => { + switch (event.type) { + case "message_update": + if (event.assistantMessageEvent.type === "text_delta") { + // Stream text to UI + process.stdout.write(event.assistantMessageEvent.delta) + } + break + + case "agent_end": + // Cleanup, save state, etc. + await flushSessionState(signal) + } +}) +``` + +**Subscription semantics**: +- Listeners are awaited in subscription order +- `agent_end` listeners are included in run settlement +- Agent becomes idle only after all awaited listeners finish +- All listeners receive the active abort signal + +--- + +## Hook System + +### Hook Points + +| Hook | Location | Purpose | +|------|----------|---------| +| `convertToLlm` | agent.ts:99 | Transform messages before LLM call | +| `transformContext` | agent.ts:100 | Modify context (pruning, injection) | +| `beforeToolCall` | agent.ts:105 | Block or modify tool execution | +| `afterToolCall` | agent.ts:106 | Override tool results | +| `prepareNextTurn` | agent.ts:107 | Dynamic context/model updates | +| `shouldStopAfterTurn` | agent-loop.ts | Graceful termination | +| `getSteeringMessages` | agent.ts:114 | Inject messages mid-turn | +| `getFollowUpMessages` | agent.ts:115 | Queue follow-up messages | + +**Hook Flow Diagram**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ HOOK EXECUTION FLOW │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Prompt Execution │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ agent.prompt("Hello!") │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Agent Class │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ runPromptMessages() │ │ +│ │ • normalizePromptInput() │ │ +│ │ • createContextSnapshot() │ │ +│ │ • createLoopConfig() │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Agent Loop │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ runLoop() │ │ +│ │ • streamAssistantResponse() │ │ +│ │ • executeToolCalls() │ │ +│ │ • turn_end │ │ +│ │ • prepareNextTurn hook │ │ +│ │ • shouldStopAfterTurn hook │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ LLM Call Boundary │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ streamAssistantResponse() │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ 1. transformContext hook (optional) │ │ +│ │ (pruning, context injection) │ │ +│ │ │ │ +│ │ 2. convertToLlm hook │ │ +│ │ (filter custom messages, convert to LLM format) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Tool Execution │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ executeToolCalls() │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ For each tool call: │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ prepareToolCall() │ │ │ +│ │ │ • validateToolArguments() │ │ │ +│ │ │ • beforeToolCall hook │ │ │ +│ │ │ (can block: return { block: true }) │ │ │ +│ │ │ │ │ │ +│ │ │ execute() │ │ │ +│ │ │ • onUpdate() (stream partial results) │ │ │ +│ │ │ │ │ │ +│ │ │ finalizeExecutedToolCall() │ │ │ +│ │ │ • afterToolCall hook │ │ │ +│ │ │ (can override: return { content: [...], details: {...} }) │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ prepareNextTurn │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ prepareNextTurn hook │ │ +│ │ • Can return updated context │ │ +│ │ • Can return updated model │ │ +│ │ • Can return updated thinkingLevel │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ shouldStopAfterTurn │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ shouldStopAfterTurn hook │ │ +│ │ • Returns true to stop agent gracefully │ │ +│ │ • Used for context management before compaction │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +**BeforeToolCall Hook**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ BEFORE_TOOL_CALL HOOK │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ beforeToolCall(context, signal) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + + ┌──────────────────────────────────────────────────────────────────────────────┐ │ + │ BeforeToolCallContext: │ │ + │ • assistantMessage: AssistantMessage │ │ + │ • toolCall: AgentToolCall │ │ + │ • args: unknown (validated) │ │ + │ • context: AgentContext │ │ + └──────────────────────────────────────────────────────────────────────────────┘ │ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Return Value │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ { block: true, reason?: string } │ │ +│ │ → Tool execution blocked, error result emitted │ │ +│ │ │ │ +│ │ undefined │ │ +│ │ → Tool execution proceeds │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +Example (blocking bash tool): + beforeToolCall: async ({ toolCall, context }) => { + if (toolCall.name === "bash") { + return { block: true, reason: "bash is disabled" }; + } + } +``` + +**AfterToolCall Hook**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ AFTER_TOOL_CALL HOOK │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ afterToolCall(context, signal) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + + ┌──────────────────────────────────────────────────────────────────────────────┐ │ + │ AfterToolCallContext: │ │ + │ • assistantMessage: AssistantMessage │ │ + │ • toolCall: AgentToolCall │ │ + │ • args: unknown (validated) │ │ + │ • result: AgentToolResult │ │ + │ • isError: boolean │ │ + │ • context: AgentContext │ │ + └──────────────────────────────────────────────────────────────────────────────┘ │ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Return Value (Partial Override) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ { │ │ +│ │ content?: (TextContent | ImageContent)[] │ │ +│ │ details?: unknown │ │ +│ │ isError?: boolean │ │ +│ │ usage?: Usage │ │ +│ │ terminate?: boolean // Hint to stop after batch │ │ +│ │ } │ │ +│ │ │ │ +│ │ Omitted fields keep original values │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +Example (terminate on success): + afterToolCall: async ({ toolCall, result, isError }) => { + if (toolCall.name === "notify_done" && !isError) { + return { terminate: true }; + } + } +``` + +**PrepareNextTurn Hook**: +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ PREPARE_NEXT_TURN HOOK │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ prepareNextTurn(context) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + + ┌──────────────────────────────────────────────────────────────────────────────┐ │ + │ PrepareNextTurnContext (extends ShouldStopAfterTurnContext): │ │ + │ • message: AssistantMessage │ │ + │ • toolResults: ToolResultMessage[] │ │ + │ • context: AgentContext │ │ + │ • newMessages: AgentMessage[] │ │ + └──────────────────────────────────────────────────────────────────────────────┘ │ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Return Value │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ { │ │ +│ │ context?: AgentContext │ │ +│ │ model?: Model │ │ +│ │ thinkingLevel?: ThinkingLevel │ │ +│ │ } │ │ +│ │ │ │ +│ │ undefined │ │ +│ │ → Keep current context/config │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +Example (switch to smaller model for next turn): + prepareNextTurn: async ({ message, context }) => { + if (context.messages.length > 100) { + return { model: getSmallerModel() }; + } + } +``` + +--- + +## Implementation Guide for Julia + +### Architecture Overview + +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ JULIA AGENT IMPLEMENTATION STRUCTURE │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +pi-agent/ +├── src/ +│ ├── agent.jl # Core Agent class (stateful wrapper) +│ │ ├── Agent structure +│ │ ├── State management (_state) +│ │ ├── Queue management (steering, follow-up) +│ │ ├── Event subscription +│ │ └── Methods: prompt(), continue(), reset(), subscribe() +│ │ +│ ├── agent_loop.jl # Low-level execution loop +│ │ ├── run_agent_loop() +│ │ ├── run_agent_loop_continue() +│ │ ├── run_loop() +│ │ ├── stream_assistant_response() +│ │ └── execute_tool_calls() (parallel/sequential) +│ │ +│ ├── messages.jl # Message types and conversion +│ │ ├── Message types (User, Assistant, ToolResult) +│ │ ├── Custom message types +│ │ ├── convert_to_llm() +│ │ └─ create_summary_messages() +│ │ +│ ├── tools.jl # Tool execution +│ │ ├── Tool structure +│ │ ├── prepare_tool_call() +│ │ ├── execute_prepared_tool_call() +│ │ ├── execute_tool_calls_parallel() +│ │ └── execute_tool_calls_sequential() +│ │ +│ ├── session.jl # Session persistence +│ │ ├── Session structure +│ │ ├── JSONL storage +│ │ ├── append_entry() +│ │ ├── get_branch() +│ │ └── build_context() +│ │ +│ ├── compaction.jl # Memory management +│ │ ├── estimate_tokens() +│ │ ├── should_compact() +│ │ ├── find_cut_point() +│ │ ├── prepare_compaction() +│ │ ├── generate_summary() +│ │ └── compact() +│ │ +│ ├── events.jl # Event system +│ │ ├── Event types (AgentStart, TurnEnd, etc.) +│ │ └── emit_event() +│ │ +│ ├── hooks.jl # Hook system +│ │ ├── Hook context types +│ │ ├── Hook return types +│ │ └── default hooks +│ │ +│ ├── types.jl # Type definitions +│ │ ├── ThinkingLevel enum +│ │ ├── Tool structure +│ │ ├── Message types +│ │ └── Context structures +│ │ +│ ├── stream.jl # Stream utilities +│ │ ├── Stream struct +│ │ ├── next_event() +│ │ └── stream_simple() +│ │ +│ └── utils/ +│ ├── truncate.jl # Text truncation +│ └── shell_output.jl # Shell command execution +│ +├── test/ +│ ├── agent_test.jl +│ ├── tools_test.jl +│ └── session_test.jl +│ +└── project.toml +``` + +### Step 1: Type Definitions (`types.jl`) + +```julia +# thinking_level.jl +@enum ThinkingLevel begin + THINKING_OFF + THINKING_MINIMAL + THINKING_LOW + THINKING_MEDIUM + THINKING_HIGH + THINKING_XHIGH + THINKING_MAX +end + +# tool.jl +mutable struct Tool{TParameters, TDetails} + name::String + label::String + description::String + parameters::TParameters + execute::Function + prepare_arguments::Union{Function, Nothing} + execution_mode::Symbol # :parallel or :sequential +end + +# message.jl +abstract type Message end + +struct UserMessage <: Message + content::Vector{Union{TextContent, ImageContent}} + timestamp::Int64 +end + +struct AssistantMessage <: Message + content::Vector{Union{TextContent, ToolCall, Thinking}} + api::String + provider::String + model::String + usage::Usage + stop_reason::String + error_message::Union{String, Nothing} + timestamp::Int64 +end + +struct ToolResultMessage <: Message + tool_call_id::String + tool_name::String + content::Vector{Union{TextContent, ImageContent}} + details::Any + usage::Union{Usage, Nothing} + is_error::Bool + timestamp::Int64 +end + +struct CustomMessage <: Message + custom_type::String + content::Union{String, Vector{Union{TextContent, ImageContent}}} + display::Bool + details::Any + timestamp::Int64 +end + +struct AgentMessage + message::Union{UserMessage, AssistantMessage, ToolResultMessage, CustomMessage} +end + +# context.jl +struct AgentContext + system_prompt::String + messages::Vector{AgentMessage} + tools::Union{Vector{Tool}, Nothing} +end +``` + +### Step 2: Message System (`messages.jl`) + +```julia +# messages.jl +const COMPACTION_SUMMARY_PREFIX = """ +The conversation history before this point was compacted into the following summary: + + +""" + +const COMPACTION_SUMMARY_SUFFIX = """ + +""" + +const BRANCH_SUMMARY_PREFIX = """ +The following is a summary of a branch that this conversation came back from: + + +""" + +const BRANCH_SUMMARY_SUFFIX = """ + +""" + +function convert_to_llm(messages::Vector{AgentMessage}) + llm_messages = Vector{Any}() + + for msg in messages + if msg.message isa UserMessage + push!(llm_messages, msg.message) + elseif msg.message isa AssistantMessage + push!(llm_messages, msg.message) + elseif msg.message isa ToolResultMessage + push!(llm_messages, msg.message) + elseif msg.message isa CustomMessage + # Convert custom to user message + content = if msg.message.content isa String + [TextContent(msg.message.content)] + else + msg.message.content + end + push!(llm_messages, UserMessage(content, msg.message.timestamp)) + end + end + + return llm_messages +end + +function create_branch_summary_message(summary::String, from_id::String, timestamp::String) + BranchSummaryMessage(summary, from_id, parse(Int64, timestamp)) +end + +function create_compaction_summary_message(summary::String, tokens_before::Int, timestamp::String) + CompactionSummaryMessage(summary, tokens_before, parse(Int64, timestamp)) +end +``` + +### Step 3: Tool Execution (`tools.jl`) + +```julia +# tools.jl +struct ToolExecutionResult{TDetails} + content::Vector{Union{TextContent, ImageContent}} + details::TDetails + usage::Union{Usage, Nothing} + added_tool_names::Vector{String} + terminate::Bool +end + +struct PreparedToolCall + tool_call::ToolCall + tool::Tool + args::Any +end + +struct ImmediateToolOutcome + result::ToolExecutionResult + is_error::Bool +end + +struct ExecutedToolOutcome + result::ToolExecutionResult + is_error::Bool +end + +struct FinalizedToolOutcome + tool_call::ToolCall + result::ToolExecutionResult + is_error::Bool +end + +function prepare_tool_call( + current_context::AgentContext, + assistant_message::AssistantMessage, + tool_call::ToolCall, + config::AgentLoopConfig, + signal::Union{AbortSignal, Nothing} +) + tool = find_tool(current_context.tools, tool_call.name) + + if tool === nothing + return ImmediateToolOutcome( + ToolExecutionResult([TextContent("Tool $(tool_call.name) not found")], Dict(), nothing, [], false), + true + ) + end + + try + prepared_tool_call = prepare_tool_call_arguments(tool, tool_call) + validated_args = validate_tool_arguments(tool, prepared_tool_call) + + if config.before_tool_call !== nothing + before_result = config.before_tool_call( + BeforeToolCallContext(assistant_message, tool_call, validated_args, current_context), + signal + ) + + if before_result.block + return ImmediateToolOutcome( + ToolExecutionResult([TextContent(before_result.reason)], Dict(), nothing, [], false), + true + ) + end + end + + return PreparedToolCall(tool_call, tool, validated_args) + catch error + return ImmediateToolOutcome( + ToolExecutionResult([TextContent(error.message)], Dict(), nothing, [], false), + true + ) + end +end + +function execute_prepared_tool_call( + prepared::PreparedToolCall, + signal::Union{AbortSignal, Nothing}, + emit::Function +) + update_events = Vector{Future}() + accepting_updates = true + + try + result = prepared.tool.execute( + prepared.tool_call.id, + prepared.args, + signal, + partial_result -> begin + if !accepting_updates + return + end + push!(update_events, Threads.@spawn begin + emit({ + type: "tool_execution_update", + tool_call_id: prepared.tool_call.id, + tool_name: prepared.tool_call.name, + args: prepared.tool_call.arguments, + partial_result: partial_result + }) + end) + end + ) + + accepting_updates = false + wait(update_events) + return ExecutedToolOutcome(result, false) + catch error + accepting_updates = false + wait(update_events) + return ExecutedToolOutcome( + ToolExecutionResult([TextContent(error.message)], Dict(), nothing, [], false), + true + ) + finally + accepting_updates = false + end +end + +function execute_tool_calls_parallel(...) + finalized_calls = Vector{Any}() + + # Preflight all tools sequentially + for tool_call in tool_calls + preparation = prepare_tool_call(...) + + if preparation isa ImmediateToolOutcome + push!(finalized_calls, preparation) + else + push!(finalized_calls, Threads.@async begin + executed = execute_prepared_tool_call(preparation, signal, emit) + finalize_executed_tool_call(...) + end) + end + end + + # Execute allowed tools concurrently + ordered_finalized = wait(finalized_calls) + + # Emit toolResult messages + messages = Vector{ToolResultMessage}() + for finalized in ordered_finalized + tool_result_message = create_tool_result_message(finalized) + emit({ type: "message_start", message: tool_result_message }) + emit({ type: "message_end", message: tool_result_message }) + push!(messages, tool_result_message) + end + + return { messages: messages, terminate: should_terminate_tool_batch(ordered_finalized) } +end +``` + +### Step 4: Agent Loop (`agent_loop.jl`) + +```julia +# agent_loop.jl +function run_agent_loop( + prompts::Vector{AgentMessage}, + context::AgentContext, + config::AgentLoopConfig, + emit::Function, + signal::Union{AbortSignal, Nothing}, + stream_fn::Function +) + new_messages = copy(prompts) + current_context = AgentContext( + context.system_prompt, + vcat(context.messages, prompts), + context.tools + ) + + emit({ type: "agent_start" }) + emit({ type: "turn_start" }) + + for prompt in prompts + emit({ type: "message_start", message: prompt }) + emit({ type: "message_end", message: prompt }) + end + + run_loop(current_context, new_messages, config, signal, emit, stream_fn) + + return new_messages +end + +function run_loop( + initial_context::AgentContext, + new_messages::Vector{AgentMessage}, + initial_config::AgentLoopConfig, + signal::Union{AbortSignal, Nothing}, + emit::Function, + stream_fn::Function +) + current_context = initial_context + config = initial_config + first_turn = true + pending_messages = config.get_steering_messages !== nothing ? config.get_steering_messages() : [] + + while true + has_more_tool_calls = true + + while has_more_tool_calls || !isempty(pending_messages) + if !first_turn + emit({ type: "turn_start" }) + else + first_turn = false + end + + # Process pending messages + if !isempty(pending_messages) + for msg in pending_messages + emit({ type: "message_start", message: msg }) + emit({ type: "message_end", message: msg }) + push!(current_context.messages, msg) + push!(new_messages, msg) + end + pending_messages = [] + end + + # Stream assistant response + message = stream_assistant_response(current_context, config, signal, emit, stream_fn) + push!(new_messages, message) + + # Check for errors + if message.stop_reason == "error" || message.stop_reason == "aborted" + emit({ type: "turn_end", message: message, tool_results: [] }) + emit({ type: "agent_end", messages: new_messages }) + return + end + + # Extract tool calls + tool_calls = filter(c -> c.type == "toolCall", message.content) + tool_results = [] + has_more_tool_calls = false + + if !isempty(tool_calls) + executed_batch = execute_tool_calls(current_context, message, config, signal, emit) + append!(tool_results, executed_batch.messages) + has_more_tool_calls = !executed_batch.terminate + + for result in tool_results + push!(current_context.messages, result) + push!(new_messages, result) + end + end + + emit({ type: "turn_end", message: message, tool_results: tool_results }) + + # Check prepareNextTurn hook + if config.prepare_next_turn !== nothing + next_turn_snapshot = config.prepare_next_turn({ + message: message, + tool_results: tool_results, + context: current_context, + new_messages: new_messages + }) + + if next_turn_snapshot !== nothing + current_context = next_turn_snapshot.context !== nothing ? next_turn_snapshot.context : current_context + config = merge(config, model = next_turn_snapshot.model !== nothing ? next_turn_snapshot.model : config.model) + end + end + + # Check shouldStopAfterTurn hook + if config.should_stop_after_turn !== nothing && config.should_stop_after_turn({ + message: message, + tool_results: tool_results, + context: current_context, + new_messages: new_messages + }) + emit({ type: "agent_end", messages: new_messages }) + return + end + + # Get steering messages + pending_messages = config.get_steering_messages !== nothing ? config.get_steering_messages() : [] + end + + # Check follow-up messages + follow_up_messages = config.get_follow_up_messages !== nothing ? config.get_follow_up_messages() : [] + + if !isempty(follow_up_messages) + pending_messages = follow_up_messages + continue + end + + break + end + + emit({ type: "agent_end", messages: new_messages }) +end + +function stream_assistant_response(context, config, signal, emit, stream_fn) + # 1. Transform context (optional) + messages = context.messages + if config.transform_context !== nothing + messages = config.transform_context(messages, signal) + end + + # 2. Convert to LLM format + llm_messages = config.convert_to_llm(messages) + + # 3. Build LLM context + llm_context = Context( + system_prompt = context.system_prompt, + messages = llm_messages, + tools = context.tools + ) + + # 4. Resolve API key + resolved_api_key = config.get_api_key !== nothing ? config.get_api_key(config.model.provider) : config.api_key + + # 5. Call stream function + response = stream_fn(config.model, llm_context, merge(config, api_key = resolved_api_key, signal = signal)) + + # 6. Stream events + partial_message = nothing + added_partial = false + + for event in response + if event.type == "start" + partial_message = event.partial + push!(context.messages, partial_message) + added_partial = true + emit({ type: "message_start", message: deepcopy(partial_message) }) + elseif event.type in ["text_start", "text_delta", "text_end", "thinking_start", "thinking_delta", "thinking_end", "toolcall_start", "toolcall_delta", "toolcall_end"] + if partial_message !== nothing + partial_message = event.partial + context.messages[end] = partial_message + emit({ + type: "message_update", + assistant_message_event: event, + message: deepcopy(partial_message) + }) + end + elseif event.type == "done" || event.type == "error" + final_message = response.result() + + if added_partial + context.messages[end] = final_message + else + push!(context.messages, final_message) + emit({ type: "message_start", message: deepcopy(final_message) }) + end + + emit({ type: "message_end", message: final_message }) + return final_message + end + end +end +``` + +### Step 5: Agent Class (`agent.jl`) + +```julia +# agent.jl +mutable struct ActiveRun + promise::Promise + resolve::Function + abort_controller::AbortController +end + +mutable struct Agent + _state::MutableAgentState + listeners::Set{Function} + steering_queue::PendingMessageQueue + follow_up_queue::PendingMessageQueue + convert_to_llm::Function + transform_context::Union{Function, Nothing} + stream_function::Function + get_api_key::Union{Function, Nothing} + on_payload::Union{Function, Nothing} + on_response::Union{Function, Nothing} + before_tool_call::Union{Function, Nothing} + after_tool_call::Union{Function, Nothing} + prepare_next_turn::Union{Function, Nothing} + prepare_next_turn_with_context::Union{Function, Nothing} + active_run::Union{ActiveRun, Nothing} + session_id::Union{String, Nothing} + thinking_budgets::Union{ThinkingBudgets, Nothing} + transport::Symbol + max_retry_delay_ms::Union{Int, Nothing} + tool_execution::Symbol +end + +struct MutableAgentState + system_prompt::String + model::Model + thinking_level::ThinkingLevel + tools::Vector{Tool} + messages::Vector{AgentMessage} + is_streaming::Bool + streaming_message::Union{AssistantMessage, Nothing} + pending_tool_calls::Set{String} + error_message::Union{String, Nothing} +end + +function Agent(; kwargs...) + state = MutableAgentState( + kwargs[:initial_state].system_prompt, + kwargs[:initial_state].model, + kwargs[:initial_state].thinking_level, + copy(kwargs[:initial_state].tools), + copy(kwargs[:initial_state].messages), + false, + nothing, + Set{String}(), + nothing + ) + + Agent( + state, + Set{Function}(), + PendingMessageQueue(kwargs[:steering_mode] === nothing ? "one-at-a-time" : kwargs[:steering_mode]), + PendingMessageQueue(kwargs[:follow_up_mode] === nothing ? "one-at-a-time" : kwargs[:follow_up_mode]), + kwargs[:convert_to_llm] !== nothing ? kwargs[:convert_to_llm] : default_convert_to_llm, + kwargs[:transform_context], + kwargs[:stream_fn], + kwargs[:get_api_key], + kwargs[:on_payload], + kwargs[:on_response], + kwargs[:before_tool_call], + kwargs[:after_tool_call], + kwargs[:prepare_next_turn], + kwargs[:prepare_next_turn_with_context], + nothing, + kwargs[:session_id], + kwargs[:thinking_budgets], + kwargs[:transport] === nothing ? :auto : kwargs[:transport], + kwargs[:max_retry_delay_ms], + kwargs[:tool_execution] === nothing ? :parallel : kwargs[:tool_execution] + ) +end + +function subscribe(agent::Agent, listener::Function) + push!(agent.listeners, listener) + return () -> delete!(agent.listeners, listener) +end + +function prompt(agent::Agent, input::String) + if agent.active_run !== nothing + throw(ErrorException("Agent is already processing.")) + end + + messages = normalize_prompt_input(input) + run_prompt_messages(agent, messages) +end + +function prompt(agent::Agent, messages::Vector{AgentMessage}) + if agent.active_run !== nothing + throw(ErrorException("Agent is already processing.")) + end + + run_prompt_messages(agent, messages) +end + +function continue(agent::Agent) + if agent.active_run !== nothing + throw(ErrorException("Agent is already processing.")) + end + + last_message = agent._state.messages[end] + + if last_message.message isa AssistantMessage + queued_steering = drain(agent.steering_queue) + + if !isempty(queued_steering) + run_prompt_messages(agent, queued_steering, skip_initial_steering_poll = true) + return + end + + queued_follow_ups = drain(agent.follow_up_queue) + + if !isempty(queued_follow_ups) + run_prompt_messages(agent, queued_follow_ups) + return + end + + throw(ErrorException("Cannot continue from message role: assistant")) + end + + run_continuation(agent) +end + +function run_prompt_messages(agent::Agent, messages::Vector{AgentMessage}, options = Dict()) + run_with_lifecycle(agent) do signal + run_agent_loop( + messages, + create_context_snapshot(agent), + create_loop_config(agent, options), + event -> process_events(agent, event), + signal, + agent.stream_function + ) + end +end + +function create_context_snapshot(agent::Agent) + AgentContext( + agent._state.system_prompt, + copy(agent._state.messages), + copy(agent._state.tools) + ) +end + +function create_loop_config(agent::Agent, options = Dict()) + skip_initial_steering_poll = get(options, :skip_initial_steering_poll, false) + + AgentLoopConfig( + model = agent._state.model, + reasoning = agent._state.thinking_level == THINKING_OFF ? nothing : agent._state.thinking_level, + session_id = agent.session_id, + on_payload = agent.on_payload, + on_response = agent.on_response, + transport = agent.transport, + thinking_budgets = agent.thinking_budgets, + max_retry_delay_ms = agent.max_retry_delay_ms, + tool_execution = agent.tool_execution, + before_tool_call = agent.before_tool_call, + after_tool_call = agent.after_tool_call, + prepare_next_turn = create_prepare_next_turn(agent), + convert_to_llm = agent.convert_to_llm, + transform_context = agent.transform_context, + get_api_key = agent.get_api_key, + get_steering_messages = () -> begin + if skip_initial_steering_poll + skip_initial_steering_poll = false + return [] + end + return drain(agent.steering_queue) + end, + get_follow_up_messages = () -> drain(agent.follow_up_queue) + ) +end + +function create_prepare_next_turn(agent::Agent) + function prepare(context, signal) + if agent.prepare_next_turn_with_context !== nothing + return agent.prepare_next_turn_with_context(context, signal) + end + + if agent.prepare_next_turn !== nothing + return agent.prepare_next_turn(signal) + end + + return nothing + end + + return prepare +end + +function run_with_lifecycle(agent::Agent, executor::Function) + abort_controller = AbortController() + promise = Promise{Void}() + resolve = () -> nothing + + function set_resolve(value) + resolve = value + if !isready(promise) + put!(promise, nothing) + end + end + + agent.active_run = ActiveRun(promise, set_resolve, abort_controller) + + agent._state.is_streaming = true + agent._state.streaming_message = nothing + agent._state.error_message = nothing + + try + executor(abort_controller.signal) + catch error + handle_run_failure(agent, error, abort_controller.signal.aborted) + finally + finish_run(agent) + end +end + +function handle_run_failure(agent::Agent, error, aborted) + failure_message = AssistantMessage( + [TextContent("")], + agent._state.model.api, + agent._state.model.provider, + agent._state.model.id, + EMPTY_USAGE, + aborted ? "aborted" : "error", + error isa ErrorException ? error.message : string(error), + Dates.datetime2timestamp(Dates.now()) + ) + + process_events(agent, { type: "message_start", message: failure_message }) + process_events(agent, { type: "message_end", message: failure_message }) + process_events(agent, { type: "turn_end", message: failure_message, tool_results: [] }) + process_events(agent, { type: "agent_end", messages: [failure_message] }) +end + +function finish_run(agent::Agent) + agent._state.is_streaming = false + agent._state.streaming_message = nothing + agent._state.pending_tool_calls = Set{String}() + + if agent.active_run !== nothing + agent.active_run.resolve() + agent.active_run = nothing + end +end + +function process_events(agent::Agent, event) + if event.type == "message_start" + agent._state.streaming_message = event.message + elseif event.type == "message_update" + agent._state.streaming_message = event.message + elseif event.type == "message_end" + agent._state.streaming_message = nothing + push!(agent._state.messages, event.message) + elseif event.type == "tool_execution_start" + push!(agent._state.pending_tool_calls, event.tool_call_id) + elseif event.type == "tool_execution_end" + delete!(agent._state.pending_tool_calls, event.tool_call_id) + elseif event.type == "turn_end" + if event.message.message isa AssistantMessage && event.message.message.error_message !== nothing + agent._state.error_message = event.message.message.error_message + end + elseif event.type == "agent_end" + agent._state.streaming_message = nothing + end + + signal = agent.active_run !== nothing ? agent.active_run.abort_controller.signal : nothing + + for listener in agent.listeners + Threads.@spawn listener(event, signal) + end +end + +function normalize_prompt_input(input::String) + content = [TextContent(input)] + return [AgentMessage(UserMessage(content, Dates.datetime2timestamp(Dates.now())))] +end + +function normalize_prompt_input(messages::Vector{AgentMessage}) + return messages +end +``` + +### Step 6: Session Persistence (`session.jl`) + +```julia +# session.jl +struct SessionEntry + type::String + id::String + parent_id::Union{String, Nothing} + timestamp::Int64 + # Dynamic fields based on type +end + +struct MessageEntry <: SessionEntry + message::AgentMessage +end + +struct CompactionEntry <: SessionEntry + summary::String + first_kept_entry_id::Union{String, Nothing} + tokens_before::Int +end + +struct BranchSummaryEntry <: SessionEntry + summary::String + from_id::String +end + +struct LeafEntry <: SessionEntry + target_id::String +end + +struct SessionStorage{TMetadata} + file_path::String + metadata::TMetadata + entries::Dict{String, SessionEntry} + leaf_id::Union{String, Nothing} +end + +function create_session(file_system::FileSystem, cwd::String, id::String, timestamp::DateTime) + session_dir = join_path(file_system.sessions_root, encode_cwd(cwd)) + create_dir(session_dir, recursive = true) + + file_path = join_path(session_dir, "$(replace(timestamp, r"[:\.]" => "-"))_$id.jsonl") + + metadata = SessionMetadata( + path = file_path, + cwd = cwd, + session_id = id, + parent_session_path = nothing, + created_at = timestamp + ) + + storage = SessionStorage(file_path, metadata, Dict{String, SessionEntry}(), nothing) + + # Write header + header = Dict( + "type" => "session", + "version" => 3, + "id" => id, + "timestamp" => string(timestamp), + "cwd" => cwd, + "metadata" => Dict{String, Any}() + ) + + open(file_path, "w") do f + write(f, JSON.json(header)) + write(f, "\n") + end + + return storage +end + +function append_message(storage::SessionStorage, message::AgentMessage) + entry_id = uuid7() + timestamp = Dates.datetime2timestamp(Dates.now()) + + entry = Dict( + "type" => "message", + "id" => entry_id, + "parentId" => storage.leaf_id, + "timestamp" => string(DateTime(timestamp)), + "message" => message_to_dict(message) + ) + + open(storage.file_path, "a") do f + write(f, JSON.json(entry)) + write(f, "\n") + end + + storage.entries[entry_id] = entry + storage.leaf_id = entry_id +end + +function append_compaction(storage::SessionStorage, summary::String, first_kept_entry_id::String, tokens_before::Int) + entry_id = uuid7() + timestamp = Dates.datetime2timestamp(Dates.now()) + + entry = Dict( + "type" => "compaction", + "id" => entry_id, + "parentId" => storage.leaf_id, + "timestamp" => string(DateTime(timestamp)), + "summary" => summary, + "firstKeptEntryId" => first_kept_entry_id, + "tokensBefore" => tokens_before + ) + + open(storage.file_path, "a") do f + write(f, JSON.json(entry)) + write(f, "\n") + end + + storage.entries[entry_id] = entry + storage.leaf_id = entry_id +end + +function get_branch(storage::SessionStorage, from_id::Union{String, Nothing} = nothing) + leaf_id = from_id !== nothing ? from_id : storage.leaf_id + + if leaf_id === nothing + return [] + end + + entries = Vector{SessionEntry}() + current_id = leaf_id + + while current_id !== nothing + entry = get(storage.entries, current_id, nothing) + + if entry === nothing + break + end + + push!(entries, entry) + + if entry.type == "compaction" && entry.first_kept_entry_id !== nothing + current_id = entry.first_kept_entry_id + else + current_id = entry.parent_id + end + end + + return reverse(entries) +end +``` + +### Step 7: Compaction (`compaction.jl`) + +```julia +# compaction.jl +const DEFAULT_COMPACTION_SETTINGS = Dict( + :enabled => true, + :reserve_tokens => 16384, + :keep_recent_tokens => 20000 +) + +function estimate_tokens(message::AgentMessage) + if message.message isa UserMessage + return length(join(message.message.content)) / 4 + elseif message.message isa AssistantMessage + total = 0 + for block in message.message.content + if block.type == "text" + total += length(block.text) + elseif block.type == "thinking" + total += length(block.thinking) + elseif block.type == "toolCall" + total += length(block.name) + length(JSON.json(block.arguments)) + end + end + return total + elseif message.message isa ToolResultMessage + return length(join(message.message.content)) / 4 + elseif message.message isa CustomMessage + if message.message.content isa String + return length(message.message.content) / 4 + else + return length(join(message.message.content)) / 4 + end + else + return 0 + end +end + +function estimate_context_tokens(messages::Vector{AgentMessage}) + total = 0 + for message in messages + total += estimate_tokens(message) + end + return total +end + +function should_compact(context_tokens::Int, context_window::Int, settings::Dict) + return context_tokens > context_window - get(settings, :reserve_tokens, 16384) +end + +function find_cut_point(entries::Vector{SessionEntry}, keep_recent_tokens::Int) + accumulated = 0 + + for i = length(entries):-1:1 + entry = entries[i] + + # Skip invalid cut points + if entry.type == "message" && entry.message.message isa ToolResultMessage + continue + end + + tokens = estimate_context_tokens([entry.message]) + accumulated += tokens + + if accumulated >= keep_recent_tokens + return i + 1 + end + end + + return 1 +end + +function prepare_compaction(branch_entries::Vector{SessionEntry}, settings::Dict) + # Find previous compaction + previous_compaction = nothing + for entry in branch_entries + if entry.type == "compaction" + previous_compaction = entry + end + end + + # Estimate tokens + context_tokens = estimate_context_tokens([e.message for e in branch_entries]) + + # Find cut point + cut_point = find_cut_point(branch_entries, get(settings, :keep_recent_tokens, 20000)) + + # Split into groups + messages_to_summarize = branch_entries[1:cut_point] + retained_tail = branch_entries[cut_point:end] + + # Extract file operations + file_ops = extract_file_operations(messages_to_summarize) + + return Dict( + :previous_compaction => previous_compaction, + :context_tokens => context_tokens, + :cut_point => cut_point, + :messages_to_summarize => messages_to_summarize, + :retained_tail => retained_tail, + :file_ops => file_ops + ) +end + +function generate_summary(messages::Vector{AgentMessage}, previous_summary::Union{String, Nothing} = nothing) + conversation = serialize_conversation(messages) + + if previous_summary !== nothing + # Update prompt + prompt = """ + + $previous_summary + + + + $conversation + + + Update the previous summary with new progress. + """ + else + # Fresh prompt + prompt = """ + + $conversation + + + Generate a summary: + ## Goal + ## Constraints & Preferences + ## Progress + ### Done + ### In Progress + ### Blocked + ## Key Decisions + ## Next Steps + ## Critical Context + ## Files read: [...] + ## Files modified: [...] + """ + end + + # Call LLM + response = models.complete_simple( + model, + Context( + system_prompt = "You are a helpful assistant that summarizes conversations.", + messages = [UserMessage([TextContent(prompt)], Dates.datetime2timestamp(Dates.now()))], + tools = nothing + ) + ) + + return response.choices[1].message.content +end + +function compact(preparation::Dict, model::Model, models::Models) + messages_to_summarize = preparation[:messages_to_summarize] + previous_compaction = preparation[:previous_compaction] + + if previous_compaction !== nothing + previous_summary = previous_compaction.summary + summary = generate_summary(messages_to_summarize, previous_summary) + else + summary = generate_summary(messages_to_summarize, nothing) + end + + # Extract file operations + file_ops = preparation[:file_ops] + summary *= "\n\nFiles read: $(file_ops.read_files)" + summary *= "\nFiles modified: $(file_ops.modified_files)" + + return Dict( + :summary => summary, + :first_kept_entry_id => messages_to_summarize[end].id, + :tokens_before => preparation[:context_tokens], + :retained_tail => preparation[:retained_tail] + ) +end +``` + +### Step 8: Event System (`events.jl`) + +```julia +# events.jl +struct AgentEvent + type::String + # Dynamic fields based on type +end + +struct AgentStart <: AgentEvent + type::String = "agent_start" +end + +struct AgentEnd <: AgentEvent + type::String = "agent_end" + messages::Vector{AgentMessage} +end + +struct TurnStart <: AgentEvent + type::String = "turn_start" +end + +struct TurnEnd <: AgentEvent + type::String = "turn_end" + message::AgentMessage + tool_results::Vector{ToolResultMessage} +end + +struct MessageStart <: AgentEvent + type::String = "message_start" + message::AgentMessage +end + +struct MessageUpdate <: AgentEvent + type::String = "message_update" + message::AgentMessage + assistant_message_event::Any +end + +struct MessageEnd <: AgentEvent + type::String = "message_end" + message::AgentMessage +end + +struct ToolExecutionStart <: AgentEvent + type::String = "tool_execution_start" + tool_call_id::String + tool_name::String + args::Any +end + +struct ToolExecutionUpdate <: AgentEvent + type::String = "tool_execution_update" + tool_call_id::String + tool_name::String + args::Any + partial_result::Any +end + +struct ToolExecutionEnd <: AgentEvent + type::String = "tool_execution_end" + tool_call_id::String + tool_name::String + result::Any + is_error::Bool +end + +function create_tool_result_message(finalized) + ToolResultMessage( + finalized.tool_call.id, + finalized.tool_call.name, + finalized.result.content, + finalized.result.details, + finalized.result.usage, + finalized.is_error, + Dates.datetime2timestamp(Dates.now()) + ) +end +``` + +### Step 9: Hook System (`hooks.jl`) + +```julia +# hooks.jl +struct BeforeToolCallContext + assistant_message::AssistantMessage + tool_call::ToolCall + args::Any + context::AgentContext +end + +struct BeforeToolCallResult + block::Bool + reason::Union{String, Nothing} +end + +struct AfterToolCallContext + assistant_message::AssistantMessage + tool_call::ToolCall + args::Any + result::ToolExecutionResult + is_error::Bool + context::AgentContext +end + +struct AfterToolCallResult + content::Union{Vector{Union{TextContent, ImageContent}}, Nothing} + details::Union{Any, Nothing} + is_error::Union{Bool, Nothing} + usage::Union{Usage, Nothing} + terminate::Union{Bool, Nothing} +end + +struct PrepareNextTurnContext + message::AssistantMessage + tool_results::Vector{ToolResultMessage} + context::AgentContext + new_messages::Vector{AgentMessage} +end + +struct AgentLoopTurnUpdate + context::Union{AgentContext, Nothing} + model::Union{Model, Nothing} + thinking_level::Union{ThinkingLevel, Nothing} +end + +function default_before_tool_call(context::BeforeToolCallContext, signal::Union{AbortSignal, Nothing}) + return nothing +end + +function default_after_tool_call(context::AfterToolCallContext, signal::Union{AbortSignal, Nothing}) + return AfterToolCallResult(nothing, nothing, nothing, nothing, nothing) +end + +function default_prepare_next_turn(context::PrepareNextTurnContext) + return nothing +end +``` + +### Step 10: Stream Utilities (`stream.jl`) + +```julia +# stream.jl +struct StreamEvent + type::String + # Dynamic fields based on type +end + +struct StartEvent <: StreamEvent + type::String = "start" + partial::AssistantMessage +end + +struct TextStartEvent <: StreamEvent + type::String = "text_start" +end + +struct TextDeltaEvent <: StreamEvent + type::String = "text_delta" + delta::String + partial::AssistantMessage +end + +struct TextEndEvent <: StreamEvent + type::String = "text_end" +end + +struct ThinkingStartEvent <: StreamEvent + type::String = "thinking_start" +end + +struct ThinkingDeltaEvent <: StreamEvent + type::String = "thinking_delta" + delta::String + partial::AssistantMessage +end + +struct ThinkingEndEvent <: StreamEvent + type::String = "thinking_end" +end + +struct ToolcallStartEvent <: StreamEvent + type::String = "toolcall_start" +end + +struct ToolcallDeltaEvent <: StreamEvent + type::String = "toolcall_delta" + delta::String + partial::AssistantMessage +end + +struct ToolcallEndEvent <: StreamEvent + type::String = "toolcall_end" +end + +struct DoneEvent <: StreamEvent + type::String = "done" +end + +struct ErrorEvent <: StreamEvent + type::String = "error" + error::String +end + +struct Stream + events::Channel{StreamEvent} +end + +function Stream() + return Stream(Channel{StreamEvent}(32)) +end + +function stream_simple(model::Model, context::Context, options::Dict) + stream = Stream() + + Threads.@spawn begin + # Call LLM provider + response = make_llm_call(model, context, options) + + # Stream events + for chunk in response + if chunk.delta !== nothing + push!(stream.events, TextDeltaEvent(chunk.delta, chunk.message)) + end + + if chunk.tool_calls !== nothing + for tool_call in chunk.tool_calls + push!(stream.events, ToolcallDeltaEvent(JSON.json(tool_call), chunk.message)) + end + end + end + + push!(stream.events, DoneEvent()) + close(stream.events) + end + + return stream +end + +function next_event(stream::Stream) + return take!(stream.events) +end + +function isdone(stream::Stream) + return isclosed(stream.events) +end +``` + +### Usage Example + +```julia +# main.jl +using PiAgent + +# Create models +models = create_models() +models.set_provider(anthropic_provider()) +model = models.get_model("anthropic", "claude-sonnet-4-6") + +# Create agent +agent = Agent( + initial_state = AgentState( + system_prompt = "You are a helpful assistant.", + model = model, + thinking_level = THINKING_OFF, + tools = [], + messages = [] + ), + stream_fn = models.stream_simple, + before_tool_call = default_before_tool_call, + after_tool_call = default_after_tool_call +) + +# Subscribe to events +unsubscribe = agent.subscribe() do event, signal + if event.type == "message_update" && event.assistant_message_event.type == "text_delta" + print(event.assistant_message_event.delta) + end +end + +# Run agent +prompt(agent, "Hello!") + +# Clean up +unsubscribe() +``` + +--- + +## Data Flow Diagrams + +### Complete Prompt Flow + +``` +User: "What product do you have in stock?" + +1. normalize_prompt_input() + ↓ + [{ role: "user", content: [{ type: "text", text: "..." }], timestamp: ... }] + +2. run_prompt_messages() + ↓ + create_context_snapshot() + create_loop_config() + +3. run_agent_loop() + ↓ + emit(agent_start) + emit(turn_start) + emit(message_start, message_end) for user prompt + +4. run_loop() + ↓ + stream_assistant_response() + ↓ + transform_context() (optional) + ↓ + convert_to_llm() + ↓ + Build LLM context + ↓ + stream_function() + ↓ + LLM API call + +5. Stream events from LLM + ↓ + start → text_delta* → done + +6. Commit message to context + ↓ + emit(message_start, message_update*, message_end) + +7. Extract tool calls + ↓ + execute_tool_calls() + ↓ + prepare_tool_call() for each + ↓ + execute_prepared_tool_call() + ↓ + finalize_executed_tool_call() + ↓ + create_tool_result_message() + ↓ + emit(tool_execution_start, tool_execution_end) + emit(message_start, message_end) for tool result + +8. emit(turn_end) + ↓ + prepareNextTurn hook + ↓ + shouldStopAfterTurn hook + +9. Check steering/follow-up queues + ↓ + Continue loop or exit + +10. emit(agent_end) +``` + +### Tool Execution Flow + +``` +Assistant Message (with toolCall blocks) + ↓ +extract tool calls + ↓ +determine execution mode + ↓ +for each tool_call: + ↓ + prepare_tool_call() + ├─ find_tool(tool_call.name) + ├─ prepare_tool_call_arguments() + ├─ validate_tool_arguments() + ├─ before_tool_call hook + │ └─ block? → error result + └─ return PreparedToolCall + ↓ + execute_prepared_tool_call() + ├─ tool.execute() + ├─ stream partial results via onUpdate() + └─ return ExecutedToolCallOutcome + ↓ + finalize_executed_tool_call() + ├─ after_tool_call hook + └─ return FinalizedToolCallOutcome + ↓ + create_tool_result_message() + ↓ + emit(tool_execution_start/end) + emit(message_start/end) + ↓ + add to context.messages +``` + +### Session Persistence Flow + +``` +AgentEvent + ↓ +handle_agent_event() + ↓ +pendingSessionWrites.push() + ↓ +flush_pending_session_writes() + ↓ +for each pending write: + ├─ message → session.append_message() + ├─ compaction → session.append_compaction() + ├─ branch_summary → session.append_branch_summary() + ├─ leaf → session.set_leaf_id() + └─ custom → session.append_custom_entry() + ↓ +JSONL file update +``` + +--- + +## Key Algorithms + +### 1. Context Token Estimation +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ CONTEXT TOKEN ESTIMATION ALGORITHM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ estimateContextTokens(messages) → Int │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ total = 0 │ +│ for each message in messages: │ +│ total += estimateTokens(message) │ +│ return total │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ estimateTokens(message) → Int │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "user" │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ content.length / 4 (4 chars ≈ 1 token) │ │ +│ │ (images ≈ 4800 chars each) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "assistant" │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ sum of all content blocks: │ │ +│ │ • text blocks → text.length │ │ +│ │ • thinking blocks → thinking.length │ │ +│ │ • toolCall blocks → name.length + JSON.stringify(args).length │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "toolResult" / "custom" │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ content.length / 4 │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "bashExecution" │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ (command.length + output.length) / 4 │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ role === "compactionSummary" / "branchSummary" │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ summary.length / 4 │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +```typescript +function estimateContextTokens(messages) { + let total = 0 + for (const message of messages) { + total += estimateTokens(message) + } + return total +} + +function estimateTokens(message) { + switch (message.role) { + case "user": + return message.content.reduce((sum, c) => sum + c.text.length, 0) / 4 + + case "assistant": + return message.content.reduce((sum, c) => { + if (c.type === "text") return sum + c.text.length + if (c.type === "thinking") return sum + c.thinking.length + if (c.type === "toolCall") return sum + c.name.length + JSON.stringify(c.arguments).length + return sum + }, 0) + + case "toolResult": + case "custom": + return message.content.reduce((sum, c) => sum + c.text.length, 0) / 4 + + case "bashExecution": + return (message.command.length + message.output.length) / 4 + + case "compactionSummary": + case "branchSummary": + return message.summary.length / 4 + } +} +``` + +### 2. Turn Start Index Detection +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ TURN START INDEX DETECTION ALGORITHM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ findTurnStartIndex(entries, startIndex, endIndex) → Int │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Walk backward from endIndex │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ for i from endIndex-1 down to startIndex: │ │ +│ │ │ │ +│ │ entry = entries[i] │ │ +│ │ │ │ +│ │ if entry.type == "message": │ │ +│ │ role = entry.message.role │ │ +│ │ if role == "user" || role == "assistant": │ │ +│ │ return i // Found turn start │ │ +│ │ │ │ +│ │ continue │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Return endIndex (no valid turn start found) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +### 3. File Operations Extraction +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ FILE OPERATIONS EXTRACTION ALGORITHM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ createFileOps() → FileOps │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ { │ │ +│ │ read: new Set(), │ │ +│ │ edited: new Set() │ │ +│ │ } │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ extractFileOpsFromMessage(message, fileOps) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ if message.role === "toolResult": │ +│ details = message.details │ +│ if details: │ +│ if details.readFiles: │ +│ for f of details.readFiles: │ +│ fileOps.read.add(f) │ +│ if details.modifiedFiles: │ +│ for f of details.modifiedFiles: │ +│ fileOps.edited.add(f) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ computeFileLists(fileOps, messages, entries, prevCompactionIndex) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ { │ +│ readFiles: [...fileOps.read], │ +│ modifiedFiles: [...fileOps.edited] │ +│ } │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +### 4. Branch Context Building +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ BRANCH CONTEXT BUILDING ALGORITHM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ buildContextEntries(pathEntries, options) → SessionTreeEntry[] │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ entries = defaultContextEntryTransform(pathEntries) │ +│ for transform of options.entryTransforms: │ +│ entries = transform(entries) │ +│ return entries │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ defaultContextEntryTransform(pathEntries) → SessionTreeEntry[] │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ Find latest compaction entry │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ for entry of pathEntries: │ │ +│ │ if entry.type == "compaction": │ │ +│ │ compaction = entry │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +│ │ +│ if !compaction: │ +│ return [...pathEntries] │ +│ │ +│ entries = [compaction] │ +│ compactionIdx = pathEntries.findIndex(e => e.id == compaction.id) │ +│ │ +│ if compaction.retainedTail: │ +│ for i from compactionIdx+1 to end: │ +│ entries.push(pathEntries[i]) │ +│ return entries │ +│ │ +│ if compaction.firstKeptEntryId: │ +│ foundFirstKept = false │ +│ for i from 0 to compactionIdx-1: │ +│ if pathEntries[i].id == compaction.firstKeptEntryId: │ +│ foundFirstKept = true │ +│ if foundFirstKept: │ +│ entries.push(pathEntries[i]) │ +│ │ +│ for i from compactionIdx+1 to end: │ +│ entries.push(pathEntries[i]) │ +│ │ +│ return entries │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ sessionEntryToContextMessages(entry, index, entries, options) → AgentMessage[] │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ if entry.type === "message": │ +│ return [entry.message] │ +│ │ +│ if entry.type === "compaction": │ +│ return [ │ +│ createCompactionSummaryMessage(...), │ +│ ...(entry.retainedTail ?? []) │ +│ ] │ +│ │ +│ if entry.type === "branchSummary" && entry.summary: │ +│ return [createBranchSummaryMessage(...)] │ +│ │ +│ if entry.type === "custom": │ +│ return [...(options.entryProjectors?.[entry.customType]?.(...) ?? [])] │ +│ │ +│ return [] │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +### 5. JSONL File Format + +```typescript +// Header +{ + type: "session", + version: 3, + id: sessionId, + timestamp: createdAt.toISOString(), + cwd: cwd, + parentSessionPath: parentSessionPath, + metadata: metadata +} + +// Entry types +{ + type: "message", + id: uuid7(), + parentId: previousLeafId, + timestamp: now.toISOString(), + message: { + role: "user" | "assistant" | "toolResult" | "custom", + // ... message fields + } +} + +{ + type: "compaction", + id: uuid7(), + parentId: previousLeafId, + timestamp: now.toISOString(), + summary: "...", + firstKeptEntryId: entryId, + tokensBefore: tokenCount, + details: { + readFiles: [...], + modifiedFiles: [...] + } +} + +{ + type: "branch_summary", + id: uuid7(), + parentId: previousLeafId, + timestamp: now.toISOString(), + summary: "...", + fromId: branchStartId +} + +{ + type: "leaf", + id: uuid7(), + parentId: previousLeafId, + timestamp: now.toISOString(), + targetId: newLeafId +} +``` + +### 6. Tool Call Argument Preparation +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ TOOL CALL ARGUMENT PREPARATION ALGORITHM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ prepareToolCallArguments(tool, toolCall) → ToolCall │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ if !tool.prepareArguments: │ +│ return toolCall │ +│ │ +│ preparedArguments = tool.prepareArguments(toolCall.arguments) │ +│ if preparedArguments === toolCall.arguments: │ +│ return toolCall │ +│ │ +│ return { │ +│ ...toolCall, │ +│ arguments: preparedArguments │ +│ } │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +### 7. Tool Batch Termination Check +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ TOOL BATCH TERMINATION CHECK ALGORITHM │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ shouldTerminateToolBatch(finalizedCalls) → Boolean │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ if finalizedCalls.length == 0: │ +│ return false │ +│ │ +│ return finalizedCalls.every(finalized => │ +│ finalized.result.terminate === true │ +│ ) │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +### 8. Message Normalization + +```typescript +function normalizePromptInput(input, images) { + if (Array.isArray(input)) { + return input + } + + if (typeof input !== "string") { + return [input] + } + + const content = [{ type: "text", text: input }] + if (images && images.length > 0) { + content.push(...images) + } + + return [{ role: "user", content, timestamp: Date.now() }] +} +``` + +--- + +## Summary + +The Pi Agent Core architecture is a sophisticated stateful agent system with: + +1. **Stateful execution**: Maintains conversation history across multiple turns +2. **Tool execution**: Supports LLM tool calling with parallel/sequential modes +3. **Event streaming**: Real-time event system for UI updates +4. **Session persistence**: JSONL-based persistent storage with tree-structured branching +5. **Memory compaction**: Automatic context window management through LLM summarization +6. **Flexible extension**: Hook-based customization at every system boundary + +The implementation follows these key principles: + +- **Separation of concerns**: Core agent logic separated from storage and provider implementations +- **Streaming first**: All operations designed around async streams for responsiveness +- **Type safety**: Strong types for compile-time guarantees +- **Extensibility**: Hooks at every major boundary allow customization +- **Persistence**: Session history survives restarts through JSONL files + +The Julia implementation should mirror this architecture, using Julia's type system for strong typing, async/await for streaming, and JSON for persistence. + +--- + +## Complete Architecture Summary + +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ COMPLETE SYSTEM ARCHITECTURE SUMMARY │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ USER INPUT │ +│ "what product do you have in stock?" │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ AGENT CLASS (agent.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ • prompt() → normalizePromptInput() → runPromptMessages() │ │ +│ │ • continue() → runContinuation() │ │ +│ │ • reset() → clear state │ │ +│ │ • subscribe(listener) → processEvents() │ │ +│ │ • abort() → signal.abort() │ │ +│ │ │ │ +│ │ State: systemPrompt, model, thinkingLevel, tools, messages │ │ +│ │ Queues: steeringQueue, followUpQueue │ │ +│ │ Hooks: convertToLlm, transformContext, beforeToolCall, │ │ +│ │ afterToolCall, prepareNextTurn │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ AGENT LOOP (agent_loop.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ runLoop() │ │ +│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │ +│ │ │ while (true): │ │ │ +│ │ │ • Process pending messages (steering/follow-up) │ │ │ +│ │ │ • streamAssistantResponse() → LLM API │ │ │ +│ │ │ • executeToolCalls() (parallel/sequential) │ │ │ +│ │ │ • emit(turn_end) │ │ │ +│ │ │ • prepareNextTurn hook │ │ │ +│ │ │ • shouldStopAfterTurn hook │ │ │ +│ │ │ → Check steering/follow-up queues │ │ │ +│ │ └──────────────────────────────────────────────────────────────────────┘ │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ LLM PROVIDER API CALL │ +│ Models.streamSimple(model, llmContext, options) │ +│ • transformContext() (optional) │ +│ • convertToLlm() │ +│ • streamFunction() → AssistantMessageEventStream │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ SESSION PERSISTENCE (session.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Session: storage, metadata, entries, leaf_id │ │ +│ │ • append_message() │ │ +│ │ • append_compaction() │ │ +│ │ • append_branch_summary() │ │ +│ │ • set_leaf_id() │ │ +│ │ │ │ +│ │ JSONL format: │ │ +│ │ { type: "message", ... } │ │ +│ │ { type: "compaction", summary: "...", ... } │ │ +│ │ { type: "branch_summary", summary: "...", ... } │ │ +│ │ { type: "leaf", targetId: "..." } │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ MEMORY MANAGEMENT (compaction.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ estimateContextTokens(messages) → Int │ │ +│ │ │ │ +│ │ shouldCompact(contextTokens, contextWindow, settings) → Boolean │ │ +│ │ Trigger: contextTokens > contextWindow - reserveTokens │ │ +│ │ │ │ +│ │ findCutPoint(entries, keepRecentTokens) → Int │ │ +│ │ Walk backward, skip toolResults │ │ +│ │ │ │ +│ │ prepareCompaction(branchEntries, settings) → Preparation │ │ +│ │ generateSummary(messages, previousSummary?) → String │ │ +│ │ compact(preparation, model, models) → Summary │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ EVENT SYSTEM (events.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Events: │ │ +│ │ • agent_start / agent_end │ │ +│ │ • turn_start / turn_end │ │ +│ │ • message_start / message_update / message_end │ │ +│ │ • tool_execution_start / update / end │ │ +│ │ │ │ +│ │ subscribe(listener) → processEvents(event) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ HOOK SYSTEM (hooks.jl) │ +│ ┌──────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Hooks: │ │ +│ │ • convertToLlm (AgentMessage[] → Message[]) │ │ +│ │ • transformContext (pruning/injection) │ │ +│ │ • beforeToolCall (block: { block: true }) │ │ +│ │ • afterToolCall (override: { content, details, isError, ... }) │ │ +│ │ • prepareNextTurn (context/model/thinkingLevel update) │ │ +│ │ • shouldStopAfterTurn (graceful termination) │ │ +│ │ • getSteeringMessages (mid-turn injection) │ │ +│ │ • getFollowUpMessages (post-agent execution) │ │ +│ └──────────────────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## Quick Reference Guide + +### Architecture Flow + +``` +User Input → Agent Class → Agent Loop → LLM API → Session → Compaction + │ │ │ │ │ + └── Events ──┘ └── Hooks ┘ └── Tree +``` + +### Key Components + +| Component | File | Purpose | +|-----------|------|---------| +| Agent | `agent.jl` | Stateful wrapper, event emission, queues | +| Agent Loop | `agent_loop.jl` | Core execution, LLM calls, tool execution | +| Messages | `messages.jl` | Message types, conversion, summaries | +| Tools | `tools.jl` | Tool execution, validation, hooks | +| Session | `session.jl` | JSONL persistence, tree structure | +| Compaction | `compaction.jl` | Token estimation, summarization | +| Events | `events.jl` | Event types, subscription | +| Hooks | `hooks.jl` | Hook contexts, return types | +| Types | `types.jl` | All type definitions | +| Stream | `stream.jl` | Stream utilities | + +### Event Flow (With Tools) + +``` +agent_start + ↓ +turn_start + ↓ +message_start (user) + ↓ +message_end (user) + ↓ +message_start (assistant - streaming) + ↓ +message_update (text_delta, toolcall_delta) + ↓ +message_end (assistant with tool calls) + ↓ +tool_execution_start (read tool) + ↓ +tool_execution_end (read complete) + ↓ +message_start (toolResult) + ↓ +message_end (toolResult) + ↓ +turn_end + ↓ +[Inner loop continues] + ↓ +turn_start + ↓ +message_start (assistant - streaming) + ↓ +message_end (final response) + ↓ +turn_end + ↓ +agent_end +``` + +### Tool Execution Modes + +**Parallel (default)**: +``` +1. Preflight all tools sequentially +2. Queue async executions +3. Execute concurrent (Promise.all) +4. Emit results in assistant source order +``` + +**Sequential**: +``` +1. Execute one-by-one +2. Wait for each to complete +3. Emit in execution order +``` + +### Memory Management + +``` +contextTokens > contextWindow - reserveTokens → Trigger Compaction + +findCutPoint: Walk backward, skip toolResults +generateSummary: LLM creates structured summary +compact: Replace history with summary, keep tail verbatim + +Result: ~185K tokens → ~25K tokens (saved ~160K tokens) +``` + +### Hook Points + +| Hook | Purpose | +|------|---------| +| convertToLlm | Transform messages for LLM | +| transformContext | Pruning/injection (optional) | +| beforeToolCall | Block execution (can return { block: true }) | +| afterToolCall | Override result (can return { content, details, ... }) | +| prepareNextTurn | Update context/model/thinkingLevel | +| shouldStopAfterTurn | Graceful termination | +| getSteeringMessages | Inject messages mid-turn | +| getFollowUpMessages | Queue messages for post-agent | diff --git a/packages/agent/learn/memory_management.md b/packages/agent/learn/memory_management.md new file mode 100644 index 00000000..6c34b2f3 --- /dev/null +++ b/packages/agent/learn/memory_management.md @@ -0,0 +1,624 @@ +# Memory & Context Management in Pi Agent + +## 1. Architecture Overview + +The agent manages memory at **two layers**: + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Agent Harness │ +│ ┌──────────────┐ ┌──────────────┐ ┌─────────────────────┐ │ +│ │ Session │ │ Compaction │ │ Branch Navigation │ │ +│ │ (JSONL tree)│ │ (summarize) │ │ (reset + summarize)│ │ +│ └──────┬───────┘ └──────────────┘ └─────────────────────┘ │ +└─────────┼───────────────────────────────────────────────────────┘ + │ builds context +┌─────────▼───────────────────────────────────────────────────────┐ +│ Agent Class │ +│ ┌───────────────────────────────────────────────────────────┐ │ +│ │ _state.messages: AgentMessage[] (linear transcript) │ │ +│ │ _state.tools, systemPrompt, model │ │ +│ └───────────────────────────────────────────────────────────┘ │ +│ subscribe() → events → UI updates │ +└─────────────────────────────────────────────────────────────────┘ +``` + +**Key separation:** +- **In-memory** (`Agent`): linear transcript for the current run. Cleared on `reset()`. +- **On-disk** (`Session`): persistent tree of entries in JSONL files. Survives restarts. +- **Compaction**: replaces old on-disk history with an LLM-generated summary, controlling context window usage. + +--- + +## 2. Data Flow: From Prompt to LLM Call + +``` +agent.prompt("Read README.md") + │ + ▼ +┌──────────────────────────┐ +│ normalizePromptInput() │ → { role: "user", content: "..." } +└──────────┬───────────────┘ + │ + ▼ +┌──────────────────────────┐ +│ runWithLifecycle() │ → sets isStreaming=true +│ runAgentLoop() │ +└──────────┬───────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ runLoop() — the main while(true) loop │ +│ │ +│ Inner loop: │ +│ 1. Inject steering/follow-up messages │ +│ 2. streamAssistantResponse() │ +│ ┌─────────────────────────────────────────────┐ │ +│ │ transformContext() │ │ +│ │ AgentMessage[] → AgentMessage[] │ │ +│ │ (prune, inject external context) │ │ +│ └──────────────┬──────────────────────────────┘ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────┐ │ +│ │ convertToLlm() │ │ +│ │ AgentMessage[] → Message[] │ │ +│ │ (filter to user/assistant/toolResult only) │ │ +│ └──────────────┬──────────────────────────────┘ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────┐ │ +│ │ streamFunction() → LLM provider │ │ +│ │ { systemPrompt, messages, tools } │ │ +│ └──────────────┬──────────────────────────────┘ │ +│ ▼ │ +│ Stream events: start → delta* → done │ +│ ▼ │ +│ Return AssistantMessage │ +│ 3. Extract toolCall blocks │ +│ 4. If toolCalls: executeToolCalls() │ +│ → create toolResult messages │ +│ → append to context │ +│ 5. Check shouldStopAfterTurn / prepareNextTurn │ +│ 6. Check steering/follow-up queues │ +│ → Loop if more tool calls or queued messages │ +│ │ +│ Outer loop: │ +│ → Check follow-up queue for messages after agent would │ +│ stop │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 3. Context Building (On-Disk → In-Memory) + +The `AgentHarness` bridges on-disk session data to the in-memory agent loop. + +``` +session.buildContext() + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ getBranch() — walk from leaf → root via parentId │ +│ │ +│ Tree structure: │ +│ │ +│ ┌──────┐ ┌──────┐ ┌──────────┐ ┌──────────┐ │ +│ │msg 1 │───▶│msg 2 │───▶│ msg 3 │───▶│ msg 4 │ │ +│ │user │ │assist│ │ toolCall │ │ user │ │ +│ └──────┘ └──────┘ └──────────┘ └──────────┘ │ +│ │ +│ Path to root: [msg1, msg2, msg3, msg4] │ +└──────────┬────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ defaultContextEntryTransform() — THE KEY STEP │ +│ │ +│ Finds latest "compaction" entry in path: │ +│ │ +│ ┌──────┐ ┌──────────┐ ┌──────┐ ┌──────┐ │ +│ │msg 1 │ │compaction│ │msg 3 │ │msg 4 │ │ +│ │user │ │summary X │ │msg 2 │ │assist│ │ +│ └──────┘ └──────────┘ └──────┘ └──────┘ │ +│ │ │ │ │ │ +│ ├──────────────┤ │ │ │ +│ │ SKIPPED │ │ │ │ +│ │ (summarized)│ │ │ │ +│ └──────────────┼───────────┘ │ │ +│ ▼ ▼ │ +│ Include compaction entry Include entries after │ +│ + firstKeptEntryId the compaction point │ +│ │ +│ Result: [compaction, msg3, msg4] │ +│ → compaction entry becomes a "compactionSummary" message │ +└──────────┬────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ sessionEntryToContextMessages() │ +│ │ +│ For each entry: │ +│ message → [message] │ +│ compaction → [compactionSummary, ...retainedTail] │ +│ branch_summary → [branchSummaryMessage] │ +│ custom_message → [customMessage] │ +│ other → [] (omitted from LLM context) │ +│ │ +│ Flat result: [compactionSummary, msg3, msg4] │ +└──────────┬────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ deriveSessionContextState() │ +│ │ +│ Extracts from entries: │ +│ thinkingLevel ← latest thinking_level_change or assistant │ +│ model ← latest model_change or assistant │ +│ activeTools ← latest active_tools_change │ +└─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 4. Token Estimation + +Before compaction can decide whether to trigger, it needs to know how many tokens the context uses. + +``` +estimateContextTokens(messages) + │ + ├── Has provider-reported usage on last assistant msg? + │ ├── YES → use actual usage + estimate tail + │ │ (accurate — avoids compounding error) + │ │ + │ └── NO → estimate all messages from scratch + │ + ▼ +estimateTokens(message) [character heuristic: chars / 4] + │ + ├── role === "user" + │ content.length / 4 + │ (images ≈ 4800 chars each) + │ + ├── role === "assistant" + │ sum of all content blocks: + │ text blocks → text.length + │ thinking blocks → thinking.length + │ toolCall blocks → name.length + JSON.stringify(args).length + │ + ├── role === "toolResult" / "custom" + │ content.length / 4 + │ + ├── role === "bashExecution" + │ (command.length + output.length) / 4 + │ + └── role === "compactionSummary" / "branchSummary" + summary.length / 4 +``` + +**Why `chars / 4`?** Rough heuristic: ~4 ASCII characters ≈ 1 token. Conservative estimate to avoid under-counting. + +--- + +## 5. Compaction — The Core Memory Management + +### 5.1 Trigger Condition + +``` +shouldCompact(contextTokens, contextWindow, settings) + → contextTokens > contextWindow - reserveTokens + +Defaults: + reserveTokens: 16384 (~16K tokens for summary prompt + output) + keepRecentTokens: 20000 (~20K tokens of recent history to keep) + +Example (Claude with 200K context window): + Triggers when: contextTokens > 200000 - 16384 = 183616 +``` + +### 5.2 Finding the Cut Point + +``` +findCutPoint(entries, startIndex, endIndex, keepRecentTokens) + │ + │ Walk BACKWARD from endIndex + │ + ├── Accumulate estimated tokens per message + ├── Stop when accumulated ≥ keepRecentTokens + ├── Snap to nearest valid cut point + │ + ▼ +┌─────────────────────────────────────────────────────────────┐ +│ Valid cut points (safe to split): │ +│ - user message │ +│ - assistant message │ +│ - custom message │ +│ - branch_summary │ +│ │ +│ NOT valid (tool results stay with their call): │ +│ - toolResult message (skipped) │ +│ │ +│ Example: │ +│ │ +│ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ │ +│ │ u1 │ │ a1 │ │ tr1│ │ u2 │ │ a2 │ │ tr2│ │ u3 │ │ +│ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ │ +│ ▲ ▲ ▲ │ +│ │ │ │ │ +│ └── kept └── cut └── discarded │ +│ (~20K tokens) point history │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 5.3 The Compaction Process + +``` +prepareCompaction(branchEntries, settings) + │ + ├── Find previous compaction (if any) → previousSummary + ├── Estimate tokens of current context + ├── findCutPoint() → firstKeptEntryId + │ + ├── Split into 3 groups: + │ │ + │ ├── messagesToSummarize: entries BEFORE cut point + │ │ (these become the summary) + │ │ + │ ├── retainedTail: entries AFTER cut point + │ │ (these stay verbatim) + │ │ + │ └── turnPrefixMessages: if cut splits a turn + │ (the beginning of an interrupted turn) + │ + └── Extract file operations from messagesToSummarize: + → readFiles, modifiedFiles + +compact(preparation, model, models) + │ + ├── If isSplitTurn: + │ ├── generateSummary(messagesToSummarize) → history summary + │ ├── generateTurnPrefixSummary(turnPrefixMessages) → turn context + │ └── Combine: history + "---" + turn prefix + │ + ├── Else (normal): + │ ├── Has previousSummary? + │ │ ├── YES → UPDATE_SUMMARIZATION_PROMPT (iterative) + │ │ └── NO → SUMMARIZATION_PROMPT (fresh) + │ └── Call LLM with conversation text + prompt + │ + ├── Append file operations: + │ "Files read: [...]\nFiles modified: [...]" + │ + └── Return: + { + summary: "## Goal...\n## Progress...\n...", + firstKeptEntryId: "entry-uuid", + tokensBefore: 185000, + retainedTail: [msg3, msg4, ...], + details: { readFiles: [...], modifiedFiles: [...] } + } +``` + +### 5.4 Summary Format + +``` +The LLM generates a structured summary: + +## Goal +- [What is the user trying to accomplish?] + +## Constraints & Preferences +- [Any constraints, preferences, or requirements] + +## Progress +### Done +- [x] [Completed tasks] +### In Progress +- [ ] [Current work] +### Blocked +- [Issues preventing progress] + +## Key Decisions +- **[Decision]**: [Brief rationale] + +## Next Steps +1. [Ordered list of what should happen next] + +## Critical Context +- [Any data, examples, or references needed to continue] + +Files read: [src/index.ts, package.json] +Files modified: [src/index.ts] +``` + +### 5.5 Iterative Compaction + +Successive compact calls **update** the existing summary rather than replacing it: + +``` +Compaction 1 (at ~185K tokens): + Summary: "## Goal: Build a login page..." + firstKeptEntryId: "entry-003" + +Compaction 2 (at ~185K tokens again): + previousSummary: "## Goal: Build a login page..." + → UPDATE_SUMMARIZATION_PROMPT + → PRESERVES existing information + → ADDS new progress (move "In Progress" → "Done") + → NEW summary: "## Goal: Build a login page... Add OAuth..." + firstKeptEntryId: "entry-003" (same boundary) +``` + +--- + +## 6. Session Storage — JSONL Format + +``` +Session file: .pi/sessions/--home-user--/2024-01-15T10-30-00_abc123.jsonl + +Line 1 (header): + {"type":"session","version":3,"id":"abc123","timestamp":"2024-01-15T10:30:00.000Z", + "cwd":"/home/user/project","parentSession":"...","metadata":{}} + +Line 2+ (entries, one per line): + {"type":"message","id":"e001","parentId":null,"timestamp":"...","message":{...}} + {"type":"message","id":"e002","parentId":"e001","timestamp":"...","message":{...}} + {"type":"compaction","id":"e003","parentId":"e002","timestamp":"...", + "summary":"## Goal: ...\n...","firstKeptEntryId":"e001", + "tokensBefore":185000} + {"type":"leaf","id":"e004","parentId":"e003","timestamp":"...", + "targetId":"e002"} +``` + +### Entry Types + +| Type | LLM Context? | Purpose | +|------|-------------|---------| +| `message` | Yes | User, assistant, toolResult | +| `compaction` | Yes (as summary message) | Replaces compacted history | +| `branch_summary` | Yes (as summary message) | Summary of diverged branch | +| `leaf` | No | Points to current tree leaf | +| `thinking_level_change` | No | Tracking thinking level changes | +| `model_change` | No | Tracking model changes | +| `active_tools_change` | No | Tracking tool enable/disable | +| `custom` | No (unless projector configured) | App-defined data | +| `custom_message` | Yes | App-defined messages | +| `label` | No | Human-readable labels | +| `session_info` | No | Session name history | + +--- + +## 7. Session Tree (Branching) + +Sessions form a **tree**, not a linear log. This lets users "go back" and try a different approach. + +``` +Session tree: + + ┌───[e01]───┐ + │ user: "a" │ + └─────┬──────┘ + ▼ + ┌──────────┐ + │ assist 1 │ + └─────┬────┘ + ▼ + ┌──────────┐ + │ toolCall │ + └─────┬────┘ + ▼ + ┌──────────┐ + │ toolRes 1│ + └─────┬────┘ + ▼ + ┌──────────┐ + │ user: "b"│ ← user goes back here + └─────┬────┘ + │ + ┌─────┴─────┐ + │ │ + ┌──────────┐ ┌──────────┐ + │ user: "c" │ │ user: "d" │ ← branch point + └────┬─────┘ └────┬─────┘ + │ │ + ┌────▼─────┐ ┌────▼─────┐ + │ assist 2 │ │ assist 3 │ ← current leaf (d) + └──────────┘ └──────────┘ + +When user navigates to "user: b": + - Leaf moves from "d" back to "b" + - Branch summary generated for diverged work ("c" → "assist 2") + - New work branches from "b": + ┌─────┐ + │ user: "e" │ ← new branch + └─────┬─────┘ + ▼ + ┌──────────┐ + │ assist 4 │ + └──────────┘ + +Context sent to LLM: + [compaction summary, user:b, user:e, assist:4] + → The old "c"/"assist 2" branch is replaced by its summary +``` + +--- + +## 8. Pending Writes — Batching Session Persistence + +To avoid writing every message individually during a run: + +``` +Agent loop events + │ + ▼ +handleAgentEvent(event) + │ + ├── message_end → pendingSessionWrites.push({ type: "message", message }) + ├── turn_end → flushPendingSessionWrites() (save_point) + ├── agent_end → flushPendingSessionWrites() + │ + ▼ +flushPendingSessionWrites() + │ + ├── Iterate pendingSessionWrites[] + │ ├── message → session.appendMessage(msg) + │ ├── model_change → session.appendModelChange(provider, id) + │ ├── thinking_level_change → session.appendThinkingLevelChange(level) + │ ├── active_tools_change → session.appendActiveToolsChange(names) + │ ├── custom → session.appendCustomEntry(type, data) + │ ├── custom_message → session.appendCustomMessageEntry(...) + │ ├── label → session.appendLabel(targetId, label) + │ ├── session_info → session.appendSessionName(name) + │ └── leaf → session.getStorage().setLeafId(targetId) + │ + └── Shift all writes → empty pending list +``` + +During the run, messages are accumulated in `pendingSessionWrites` and only flushed to disk at `save_point` (end of each turn) or `agent_end`. + +--- + +## 9. Hooks & Extensibility — Context Control Points + +The harness exposes hooks at every memory management boundary: + +``` +┌────────────────────────────────────────────────────────────────┐ +│ Hooks │ +├────────────────────────┬─────────────────────────────────────┤ +│ Hook │ When │ +├────────────────────────┼─────────────────────────────────────┤ +│ before_agent_start │ Before each prompt, can add │ +│ │ messages or modify system prompt │ +├────────────────────────┼─────────────────────────────────────┤ +│ context │ Before each LLM call, can prune/ │ +│ │ modify AgentMessage[] │ +├────────────────────────┼─────────────────────────────────────┤ +│ before_provider_request│ Before each provider API call, can │ +│ │ modify headers, retries, timeout │ +├────────────────────────┼─────────────────────────────────────┤ +│ before_provider_payload│ Before sending payload to provider,│ +│ │ can modify the request body │ +├────────────────────────┼─────────────────────────────────────┤ +│ after_provider_response│ After receiving response, for │ +│ │ logging/metrics │ +├────────────────────────┼─────────────────────────────────────┤ +│ tool_call │ Before tool execution, can │ +│ │ return { block: true } │ +├────────────────────────┼─────────────────────────────────────┤ +│ tool_result │ After tool execution, can override │ +│ │ content, details, isError, terminate│ +├────────────────────────┼─────────────────────────────────────┤ +│ session_before_compact│ Before compaction, can cancel or │ +│ │ provide custom compaction result │ +├────────────────────────┼─────────────────────────────────────┤ +│ session_before_tree │ Before branch navigation, can │ +│ │ cancel or provide custom summary │ +├────────────────────────┼─────────────────────────────────────┤ +│ prepareNextTurn │ Between turns, can replace context, │ +│ │ model, or thinkingLevel │ +├────────────────────────┼─────────────────────────────────────┤ +│ shouldStopAfterTurn │ After a turn, if true the loop │ +│ │ exits (agent_end, no more LLM calls)│ +└────────────────────────┴─────────────────────────────────────┘ +``` + +--- + +## 10. Complete Lifecycle: Long Session + +``` +Session starts empty + │ + ▼ +Turn 1: "Create a React component" + Context: [compaction summary (empty)] + Messages exchanged: ~2K tokens + └─ Session: [user1, assist1, toolCall, toolRes1, assist2] + │ + ▼ +Turn 2-10: Iterative development + Context: growing with each turn + Total context: ~50K tokens + └─ Session: [user1..assist2, user2..assist20] + │ + ▼ +Turn 15: Context approaching limit (~170K tokens) + shouldCompact() → true + └─ Compaction 1: + - Summarizes turns 1-12 + - Keeps turns 13-15 verbatim + - Summary: "## Goal: React component ## Progress: built X, Y" + │ + ▼ +Turn 20: Context ~180K tokens + shouldCompact() → true + └─ Compaction 2 (iterative update): + - Updates existing summary with new progress + - "## Goal: React component ## Done: built X,Y ## New: added auth" + │ + ▼ +Turn 25: Context ~186K tokens → triggers compaction + └─ Compaction 3: + - Summary now covers ~22 turns of history + - Retained tail: last 20K tokens (~5 turns) + - Context window freed: ~186K → ~25K tokens + │ + ▼ +User navigates to Turn 8: + - Branch summary generated for Turns 9-25 + - Leaf moves back to Turn 8 + - Context: [compaction, branch_summary, turns 1-8] + │ + ▼ +User continues from Turn 8: + - New branch grows from Turn 8 + - Old branch (9-25) replaced by branch_summary + │ + ▼ +Session ends, JSONL file persists on disk + Next session: loads from JSONL, rebuilds context +``` + +--- + +## 11. Token Budget Summary + +``` +Example: Claude Sonnet (200K context window) + + ┌─────────────────────────────────────────────────────────────┐ + │ Context Window: 200,000 tokens │ + ├─────────────────────────────────────────────────────────────┤ + │ Reserved for summary: 16,384 tokens │ + ├─────────────────────────────────────────────────────────────┤ + │ Keep recent: 20,000 tokens │ + ├─────────────────────────────────────────────────────────────┤ + │ Max context before compaction: 183,616 tokens │ + │ (= 200000 - 16384) │ + ├─────────────────────────────────────────────────────────────┤ + │ After compaction: ~25,000 tokens │ + │ (20,000 tail + ~5,000 summary) │ + │ → ~158,616 tokens freed │ + └─────────────────────────────────────────────────────────────┘ +``` + +--- + +## 12. Key Files Reference + +| File | Responsibility | +|------|---------------| +| `agent-loop.ts` | Core loop, tool execution, streaming | +| `agent.ts` | Stateful `Agent` class, event system | +| `agent-harness.ts` | High-level harness, hooks, session management | +| `compaction/compaction.ts` | Token estimation, cut point, LLM summarization | +| `compaction/branch-summarization.ts` | Branch divergence summarization | +| `session/session.ts` | Session tree, entry appending, context building | +| `session/jsonl-storage.ts` | JSONL file read/write | +| `session/jsonl-repo.ts` | Session repo: create/open/list/delete/fork | +| `types.ts` | All type definitions | +| `messages.ts` | `convertToLlm()`, custom message helpers | +| `system-prompt.ts` | System prompt building | +| `skills.ts` | Skill management |