ton/YiemAgent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

update

Browse Source
This commit is contained in:
narawat
2026-07-26 22:21:10 +07:00
parent abfe6f45fb
commit 597319a147
4 changed files with 9 additions and 3904 deletions
Download Patch File Download Diff File Expand all files Collapse all files
learn_pi_agent/agent_workflow.md
-473
View File
@@ -1,473 +0,0 @@
# Agent Workflow: Message Handling and Response Generation
## Overview
This document explains the complete step-by-step flow of how the agent processes a user message and generates a response, from the moment a user asks "what product do you have in stock" to when the agent responds with an answer.
---
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ USER INPUT │
│ "what product do you have in stock?" │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 1. AGENT.PROMPT() ENTRY │
│ File: packages/agent/src/agent.ts:339 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ • Validate no active run (throws if busy) │ │
│ │ • normalizePromptInput() converts string to AgentMessage[] │ │
│ │ • runPromptMessages() launches execution with lifecycle events │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 2. AGENT LIFECYCLE INITIALIZATION │
│ File: packages/agent/src/agent.ts:398-412 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Events Emitted: │ │
│ │ • agent_start │ │
│ │ • turn_start │ │
│ │ • message_start / message_end (for each prompt message) │ │
│ │ │ │
│ │ Context Snapshot Created: │ │
│ │ • systemPrompt │ │
│ │ • messages (copy) │ │
│ │ • tools (copy) │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 3. AGENT LOOP STARTS │
│ File: packages/agent/src/agent-loop.ts:95 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ runPromptMessages() → runAgentLoop() │ │
│ │ • Prompts added to context.messages │ │
│ │ • Lifecycle events emitted for prompts │ │
│ │ • Calls runLoop() (main processing loop) │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 4. LLM CALL BOUNDARY - MESSAGE TRANSFORMATION │
│ File: packages/agent/src/agent-loop.ts:281-372 (streamAssistantResponse) │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Step 4.1: Context Transform (optional) │ │
│ │ transformContext(messages) → transformed messages │ │
│ │ (Used for context pruning/injection) │ │
│ │ │ │
│ │ Step 4.2: LLM Conversion │ │
│ │ convertToLlm(messages) → Message[] │ │
│ │ - Filters non-LLM messages (bashExecution, branchSummary, etc.) │ │
│ │ - Converts: user → user, assistant → assistant, toolResult → toolResult │ │
│ │ │ │
│ │ Step 4.3: Build LLM Context │ │
│ │ { │ │
│ │ systemPrompt: context.systemPrompt, │ │
│ │ messages: llmMessages, │ │
│ │ tools: context.tools │ │
│ │ } │ │
│ │ │ │
│ │ Step 4.4: Resolve API Key │ │
│ │ getApiKey(model.provider) → apiKey │ │
│ │ │ │
│ │ Step 4.5: Stream Function Call │ │
│ │ streamFunction(model, llmContext, options) │ │
│ │ - Default: Models.streamSimple() from @earendil-works/pi-ai │ │
│ │ - Makes actual LLM API call │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 5. LLM RESPONSE STREAMING │
│ File: packages/agent/src/agent-loop.ts:317-371 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Stream yields events: │ │
│ │ • start → creates partial AssistantMessage │ │
│ │ • text_start → streaming text begins │ │
│ │ • text_delta → text chunks arrive │ │
│ │ • toolcall_start → tool call block begins │ │
│ │ • toolcall_delta → tool call arguments arrive │ │
│ │ • toolcall_end → tool call block complete │ │
│ │ • text_end → text block complete │ │
│ │ • done → final message complete │ │
│ │ │ │
│ │ State Updates: │ │
│ │ • Partial message pushed to context.messages │ │
│ │ • message_start event emitted │ │
│ │ • message_update events emitted as text/tools stream in │ │
│ │ • Final message committed to context.messages │ │
│ │ • message_end event emitted │ │
│ │ │ │
│ │ Stop Reasons: │ │
│ │ • stop - normal completion │ │
│ │ • toolUse - model requested tool calls │ │
│ │ • length - token limit reached │ │
│ │ • error - failure │ │
│ │ • aborted - operation aborted │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 6. TOOL CALL PARSING AND EXECUTION │
│ File: packages/agent/src/agent-loop.ts:408-554 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Step 6.1: Extract Tool Calls │ │
│ │ toolCalls = message.content.filter(c => c.type === "toolCall") │ │
│ │ │ │
│ │ Step 6.2: Determine Execution Mode │ │
│ │ - Check config.toolExecution: "parallel" or "sequential" │ │
│ │ - Check individual tool executionMode setting │ │
│ │ - Decides how to execute tool batch │ │
│ │ │ │
│ │ Step 6.3: For Each Tool Call │ │
│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │
│ │ │ 1. Tool Lookup │ │ │
│ │ │ tool = context.tools.find(t => t.name === toolCall.name) │ │ │
│ │ │ │ │ │
│ │ │ 2. Argument Preparation │ │ │
│ │ │ prepared = tool.prepareArguments?(toolCall.arguments) │ │ │
│ │ │ │ │ │
│ │ │ 3. Argument Validation │ │ │
│ │ │ validateToolArguments(tool, preparedToolCall) │ │ │
│ │ │ │ │ │
│ │ │ 4. Before Tool Hook │ │ │
│ │ │ beforeToolCall({ assistantMessage, toolCall, args, context }) │ │ │
│ │ │ - Can block execution by returning { block: true, reason } │ │ │
│ │ │ │ │ │
│ │ │ 5. Execution │ │ │
│ │ │ execute(toolCallId, params, signal, onUpdate, context) │ │ │
│ │ │ │ │ │
│ │ │ - Parallel Mode: Tools execute concurrently │ │ │
│ │ │ - Sequential Mode: Tools execute one-by-one │ │ │
│ │ └──────────────────────────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 7. TOOL EXECUTION EXAMPLE - READ TOOL │
│ File: packages/agent/src/harness/tools/read.ts │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ User asks: "what product do you have in stock?" │ │
│ │ │ │
│ │ Agent decides to read catalog file "products.json" │ │
│ │ │ │
│ │ Input Arguments: │ │
│ │ { │ │
│ │ "path": "products.json", │ │
│ │ "offset": 1, │ │
│ │ "limit": 100 │ │
│ │ } │ │
│ │ │ │
│ │ Execution Steps: │ │
│ │ 1. resolveReadToolPath(env, path, signal) → absolutePath │ │
│ │ 2. env.readBinaryFile(absolutePath, signal) → bytes │ │
│ │ 3. Detect mimeType (check if image) │ │
│ │ 4. For text files: │ │
│ │ - Decode UTF-8 → textContent │ │
│ │ - Split by lines → allLines │ │
│ │ - Apply offset/limit slicing │ │
│ │ - Truncate if exceeds DEFAULT_MAX_BYTES or DEFAULT_MAX_LINES │ │
│ │ - Add truncation notice to output │ │
│ │ 5. Return result: │ │
│ │ { │ │
│ │ content: [{ type: "text", text: output }], │ │
│ │ details: { truncation: ... } │ │
│ │ } │ │
│ │ │ │
│ │ Output Example: │ │
│ │ "Showing lines 1-50 of 150. [Showing 50 lines of 150. Use offset=51 to │ │
│ │ continue.]" │ │
│ │ │ │
│ │ Tool Result: │ │
│ │ { │ │
│ │ "role": "toolResult", │ │
│ │ "toolCallId": "tool_abc123", │ │
│ │ "toolName": "read", │ │
│ │ "content": [{ "type": "text", "text": "..." }], │ │
│ │ "isError": false, │ │
│ │ "timestamp": 1721721600000 │ │
│ │ } │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 8. TOOL RESULT HANDLING │
│ File: packages/agent/src/agent-loop.ts:556-792 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Step 8.1: After Tool Hook │ │
│ │ afterToolCall({ assistantMessage, toolCall, args, result, isError, ctx }) │ │
│ │ - Can override result content, details, usage, terminate hint │ │
│ │ │ │
│ │ Step 8.2: Create Tool Result Message │ │
│ │ { │ │
│ │ role: "toolResult", │ │
│ │ toolCallId: toolCall.id, │ │
│ │ toolName: toolCall.name, │ │
│ │ content: result.content ?? [], │ │
│ │ details: result.details, │ │
│ │ usage: result.usage, │ │
│ │ isError: false, │ │
│ │ timestamp: Date.now() │ │
│ │ } │ │
│ │ │ │
│ │ Step 8.3: Emit Events │ │
│ │ • tool_execution_start │ │
│ │ • tool_execution_end │ │
│ │ • message_start (toolResult message) │ │
│ │ • message_end (toolResult message) │ │
│ │ │ │
│ │ Step 8.4: Update Context │ │
│ │ • Push tool result message to currentContext.messages │ │
│ │ • Push to newMessages array │ │
│ │ │ │
│ │ Step 8.5: Batch Termination Check │ │
│ │ shouldTerminateToolBatch(finalizedCalls) │ │
│ │ - Returns true if ALL tools set terminate: true │ │
│ │ - If true, agent may stop after this batch │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 9. NEXT TURN PREPARATION │
│ File: packages/agent/src/agent-loop.ts:224-257 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Step 9.1: Turn End Event │ │
│ │ turn_end emitted with message and toolResults │ │
│ │ │ │
│ │ Step 9.2: prepareNextTurn Hook │ │
│ │ prepareNextTurn({ message, toolResults, context, newMessages }) │ │
│ │ - Can return updated context, model, or thinking level │ │
│ │ - Used for dynamic context management │ │
│ │ │ │
│ │ Step 9.3: Queue Polling │ │
│ │ getSteeringMessages() → inject messages for immediate processing │ │
│ │ getFollowUpMessages() → check for queued follow-up messages │ │
│ │ │ │
│ │ Step 9.4: Loop Decision │ │
│ │ • If steering messages exist → process them, continue loop │ │
│ │ • If follow-up messages exist → process them, continue loop │ │
│ │ • If tool calls remain in message → continue inner loop │ │
│ │ • If no messages → emit agent_end, exit loop │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 10. AGENT RESPONSE GENERATION │
│ File: packages/agent/src/agent-loop.ts:58-163 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ The agent loop continues until: │ │
│ │ • No tool calls remain in assistant messages │ │
│ │ • No steering/follow-up messages queued │ │
│ │ • shouldStopAfterTurn() returns true (if configured) │ │
│ │ │ │
│ │ Final Response Generation: │ │
│ │ 1. LLM streams text content blocks │ │
│ │ 2. Message committed to context │ │
│ │ 3. turn_end emitted │ │
│ │ 4. agent_end emitted with all new messages │ │
│ │ 5. Agent returns to idle state │ │
│ │ │ │
│ │ Final Agent Response: │ │
│ │ "We have 15 products in stock:" │ │
│ │ • Product A - $29.99 │ │
│ │ • Product B - $49.99 │ │
│ │ • Product C - $19.99 │ │
│ │ (and 2 more products) │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
```
---
## Complete Event Flow Diagram
```
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ COMPLETE EVENT SEQUENCE FOR A TURN │
│ WITH TOOL USE (Product Catalog Query) │
└─────────────────────────────────────────────────────────────────────────────────────┘
agent_start
↓
turn_start
↓
message_start (user message: "what product do you have in stock?")
message_end (user message)
↓
message_start (assistant message - streaming from LLM)
message_update (text delta: "We have")
message_update (toolcall delta: {"name":"read","arguments":{...}})
message_update (toolcall end)
message_end (assistant message with tool calls)
↓
tool_execution_start (tool call: read products.json)
tool_execution_end (tool call: read complete)
message_start (toolResult message)
message_end (toolResult message)
↓
turn_end
↓
[Inner loop continues: send tool result to LLM]
↓
turn_start
↓
message_start (assistant message - streaming from LLM)
message_update (text delta: "We have 15 products in stock:")
message_update (text delta: "• Product A - $29.99")
message_update (text delta: "• Product B - $49.99")
message_end (final assistant message)
↓
turn_end
↓
agent_end
```
---
## Key Files Summary
| Component | File | Purpose |
|-----------|------|---------|
| Agent class | `packages/agent/src/agent.ts` | Stateful wrapper, event emission, queue management |
| Agent loop | `packages/agent/src/agent-loop.ts` | Core loop, LLM calls, tool execution |
| Message types | `packages/agent/src/types.ts` | AgentMessage, AgentTool, AgentEvent definitions |
| Harness | `packages/agent/src/harness/agent-harness.ts` | Session integration, hooks, persistence |
| Read tool | `packages/agent/src/harness/tools/read.ts` | File reading implementation |
| Stream function | `packages/agent/src/stream-fn.ts` | Default stream function management |
| Types | `packages/agent/src/types.ts` | All type definitions |
---
## Hook Points for Customization
The agent supports multiple extension points:
| Hook | Location | Purpose |
|------|----------|---------|
| `convertToLlm` | `agent.ts:99` | Transform messages before LLM call |
| `transformContext` | `agent.ts:100` | Modify context (pruning, injection) |
| `beforeToolCall` | `agent.ts:105` | Block or modify tool execution |
| `afterToolCall` | `agent.ts:106` | Override tool results |
| `prepareNextTurn` | `agent.ts:107` | Dynamic context/model updates |
| `shouldStopAfterTurn` | `agent-loop.ts` | Graceful termination |
| `getSteeringMessages` | `agent.ts:114` | Inject messages mid-turn |
| `getFollowUpMessages` | `agent.ts:115` | Queue follow-up messages |
---
## Tool Execution Flow
```
Tool Call Received from LLM
↓
1. Tool Lookup (find by name)
↓
2. prepareArguments? (transform if defined)
↓
3. validateToolArguments (JSON Schema)
↓
4. beforeToolCall hook (can block)
↓
5. execute (parallel or sequential)
↓
6. onUpdate (stream partial results)
↓
7. afterToolCall hook (can override)
↓
8. Create toolResult message
↓
9. Emit events (start, end)
↓
10. Add to context.messages
```
---
## Example: "What product do you have in stock?"
### Step-by-Step Execution:
1. **User sends message**
```
"what product do you have in stock?"
```
2. **Agent normalizes input**
```typescript
[{
role: "user",
content: [{ type: "text", text: "what product do you have in stock?" }],
timestamp: Date.now()
}]
```
3. **LLM processes and decides to use `read` tool**
```json
{
"role": "assistant",
"content": [{
"type": "toolCall",
"name": "read",
"arguments": {
"path": "products.json",
"offset": 1,
"limit": 50
},
"id": "tool_abc123"
}]
}
```
4. **Tool execution**
- Read `products.json` (150 lines total)
- Return lines 1-50 with truncation notice
- Add to context as toolResult
5. **LLM generates final response**
```
We have 15 products in stock:
• Product A - $29.99
• Product B - $49.99
• Product C - $19.99
• Product D - $99.99
• Product E - $149.99
(and 10 more products)
Use offset=51 to continue viewing.
```
6. **Agent emits final response to user**
---
## Summary
The agent workflow follows a clear pattern:
1. **Message Input** → Normalize and validate
2. **Context Setup** → Create snapshot with system prompt, messages, tools
3. **LLM Call** → Transform messages, resolve API key, stream response
4. **Tool Detection** → Check for tool calls in assistant message
5. **Tool Execution** → Validate, hook, execute, stream updates
6. **Result Handling** → Create toolResult message, emit events
7. **Next Turn** → Check for steering/follow-up messages, prepare context
8. **Response Generation** → Continue until no more tool calls needed
9. **Completion** → Emit final response to user
The entire flow is event-driven, allowing for real-time updates and hook-based customization at every step.
learn_pi_agent/architecture.md
-2798
View File
File diff suppressed because it is too large Load Diff
learn_pi_agent/memory_management.md
-624
View File
@@ -1,624 +0,0 @@
# Memory & Context Management in Pi Agent
## 1. Architecture Overview
The agent manages memory at **two layers**:
```
┌─────────────────────────────────────────────────────────────────┐
│ Agent Harness │
│ ┌──────────────┐ ┌──────────────┐ ┌─────────────────────┐ │
│ │ Session │ │ Compaction │ │ Branch Navigation │ │
│ │ (JSONL tree)│ │ (summarize) │ │ (reset + summarize)│ │
│ └──────┬───────┘ └──────────────┘ └─────────────────────┘ │
└─────────┼───────────────────────────────────────────────────────┘
│ builds context
┌─────────▼───────────────────────────────────────────────────────┐
│ Agent Class │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ _state.messages: AgentMessage[] (linear transcript) │ │
│ │ _state.tools, systemPrompt, model │ │
│ └───────────────────────────────────────────────────────────┘ │
│ subscribe() → events → UI updates │
└─────────────────────────────────────────────────────────────────┘
```
**Key separation:**
- **In-memory** (`Agent`): linear transcript for the current run. Cleared on `reset()`.
- **On-disk** (`Session`): persistent tree of entries in JSONL files. Survives restarts.
- **Compaction**: replaces old on-disk history with an LLM-generated summary, controlling context window usage.
---
## 2. Data Flow: From Prompt to LLM Call
```
agent.prompt("Read README.md")
│
▼
┌──────────────────────────┐
│ normalizePromptInput() │ → { role: "user", content: "..." }
└──────────┬───────────────┘
│
▼
┌──────────────────────────┐
│ runWithLifecycle() │ → sets isStreaming=true
│ runAgentLoop() │
└──────────┬───────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ runLoop() — the main while(true) loop │
│ │
│ Inner loop: │
│ 1. Inject steering/follow-up messages │
│ 2. streamAssistantResponse() │
│ ┌─────────────────────────────────────────────┐ │
│ │ transformContext() │ │
│ │ AgentMessage[] → AgentMessage[] │ │
│ │ (prune, inject external context) │ │
│ └──────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────┐ │
│ │ convertToLlm() │ │
│ │ AgentMessage[] → Message[] │ │
│ │ (filter to user/assistant/toolResult only) │ │
│ └──────────────┬──────────────────────────────┘ │
│ ▼ │
│ ┌─────────────────────────────────────────────┐ │
│ │ streamFunction() → LLM provider │ │
│ │ { systemPrompt, messages, tools } │ │
│ └──────────────┬──────────────────────────────┘ │
│ ▼ │
│ Stream events: start → delta* → done │
│ ▼ │
│ Return AssistantMessage │
│ 3. Extract toolCall blocks │
│ 4. If toolCalls: executeToolCalls() │
│ → create toolResult messages │
│ → append to context │
│ 5. Check shouldStopAfterTurn / prepareNextTurn │
│ 6. Check steering/follow-up queues │
│ → Loop if more tool calls or queued messages │
│ │
│ Outer loop: │
│ → Check follow-up queue for messages after agent would │
│ stop │
└─────────────────────────────────────────────────────────────┘
```
---
## 3. Context Building (On-Disk → In-Memory)
The `AgentHarness` bridges on-disk session data to the in-memory agent loop.
```
session.buildContext()
│
▼
┌─────────────────────────────────────────────────────────────┐
│ getBranch() — walk from leaf → root via parentId │
│ │
│ Tree structure: │
│ │
│ ┌──────┐ ┌──────┐ ┌──────────┐ ┌──────────┐ │
│ │msg 1 │───▶│msg 2 │───▶│ msg 3 │───▶│ msg 4 │ │
│ │user │ │assist│ │ toolCall │ │ user │ │
│ └──────┘ └──────┘ └──────────┘ └──────────┘ │
│ │
│ Path to root: [msg1, msg2, msg3, msg4] │
└──────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ defaultContextEntryTransform() — THE KEY STEP │
│ │
│ Finds latest "compaction" entry in path: │
│ │
│ ┌──────┐ ┌──────────┐ ┌──────┐ ┌──────┐ │
│ │msg 1 │ │compaction│ │msg 3 │ │msg 4 │ │
│ │user │ │summary X │ │msg 2 │ │assist│ │
│ └──────┘ └──────────┘ └──────┘ └──────┘ │
│ │ │ │ │ │
│ ├──────────────┤ │ │ │
│ │ SKIPPED │ │ │ │
│ │ (summarized)│ │ │ │
│ └──────────────┼───────────┘ │ │
│ ▼ ▼ │
│ Include compaction entry Include entries after │
│ + firstKeptEntryId the compaction point │
│ │
│ Result: [compaction, msg3, msg4] │
│ → compaction entry becomes a "compactionSummary" message │
└──────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ sessionEntryToContextMessages() │
│ │
│ For each entry: │
│ message → [message] │
│ compaction → [compactionSummary, ...retainedTail] │
│ branch_summary → [branchSummaryMessage] │
│ custom_message → [customMessage] │
│ other → [] (omitted from LLM context) │
│ │
│ Flat result: [compactionSummary, msg3, msg4] │
└──────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ deriveSessionContextState() │
│ │
│ Extracts from entries: │
│ thinkingLevel ← latest thinking_level_change or assistant │
│ model ← latest model_change or assistant │
│ activeTools ← latest active_tools_change │
└─────────────────────────────────────────────────────────────┘
```
---
## 4. Token Estimation
Before compaction can decide whether to trigger, it needs to know how many tokens the context uses.
```
estimateContextTokens(messages)
│
├── Has provider-reported usage on last assistant msg?
│ ├── YES → use actual usage + estimate tail
│ │ (accurate — avoids compounding error)
│ │
│ └── NO → estimate all messages from scratch
│
▼
estimateTokens(message) [character heuristic: chars / 4]
│
├── role === "user"
│ content.length / 4
│ (images ≈ 4800 chars each)
│
├── role === "assistant"
│ sum of all content blocks:
│ text blocks → text.length
│ thinking blocks → thinking.length
│ toolCall blocks → name.length + JSON.stringify(args).length
│
├── role === "toolResult" / "custom"
│ content.length / 4
│
├── role === "bashExecution"
│ (command.length + output.length) / 4
│
└── role === "compactionSummary" / "branchSummary"
summary.length / 4
```
**Why `chars / 4`?** Rough heuristic: ~4 ASCII characters ≈ 1 token. Conservative estimate to avoid under-counting.
---
## 5. Compaction — The Core Memory Management
### 5.1 Trigger Condition
```
shouldCompact(contextTokens, contextWindow, settings)
→ contextTokens > contextWindow - reserveTokens
Defaults:
reserveTokens: 16384 (~16K tokens for summary prompt + output)
keepRecentTokens: 20000 (~20K tokens of recent history to keep)
Example (Claude with 200K context window):
Triggers when: contextTokens > 200000 - 16384 = 183616
```
### 5.2 Finding the Cut Point
```
findCutPoint(entries, startIndex, endIndex, keepRecentTokens)
│
│ Walk BACKWARD from endIndex
│
├── Accumulate estimated tokens per message
├── Stop when accumulated ≥ keepRecentTokens
├── Snap to nearest valid cut point
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Valid cut points (safe to split): │
│ - user message │
│ - assistant message │
│ - custom message │
│ - branch_summary │
│ │
│ NOT valid (tool results stay with their call): │
│ - toolResult message (skipped) │
│ │
│ Example: │
│ │
│ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────┐ │
│ │ u1 │ │ a1 │ │ tr1│ │ u2 │ │ a2 │ │ tr2│ │ u3 │ │
│ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ └────┘ │
│ ▲ ▲ ▲ │
│ │ │ │ │
│ └── kept └── cut └── discarded │
│ (~20K tokens) point history │
└─────────────────────────────────────────────────────────────┘
```
### 5.3 The Compaction Process
```
prepareCompaction(branchEntries, settings)
│
├── Find previous compaction (if any) → previousSummary
├── Estimate tokens of current context
├── findCutPoint() → firstKeptEntryId
│
├── Split into 3 groups:
│ │
│ ├── messagesToSummarize: entries BEFORE cut point
│ │ (these become the summary)
│ │
│ ├── retainedTail: entries AFTER cut point
│ │ (these stay verbatim)
│ │
│ └── turnPrefixMessages: if cut splits a turn
│ (the beginning of an interrupted turn)
│
└── Extract file operations from messagesToSummarize:
→ readFiles, modifiedFiles
compact(preparation, model, models)
│
├── If isSplitTurn:
│ ├── generateSummary(messagesToSummarize) → history summary
│ ├── generateTurnPrefixSummary(turnPrefixMessages) → turn context
│ └── Combine: history + "---" + turn prefix
│
├── Else (normal):
│ ├── Has previousSummary?
│ │ ├── YES → UPDATE_SUMMARIZATION_PROMPT (iterative)
│ │ └── NO → SUMMARIZATION_PROMPT (fresh)
│ └── Call LLM with conversation text + prompt
│
├── Append file operations:
│ "Files read: [...]\nFiles modified: [...]"
│
└── Return:
{
summary: "## Goal...\n## Progress...\n...",
firstKeptEntryId: "entry-uuid",
tokensBefore: 185000,
retainedTail: [msg3, msg4, ...],
details: { readFiles: [...], modifiedFiles: [...] }
}
```
### 5.4 Summary Format
```
The LLM generates a structured summary:
## Goal
- [What is the user trying to accomplish?]
## Constraints & Preferences
- [Any constraints, preferences, or requirements]
## Progress
### Done
- [x] [Completed tasks]
### In Progress
- [ ] [Current work]
### Blocked
- [Issues preventing progress]
## Key Decisions
- **[Decision]**: [Brief rationale]
## Next Steps
1. [Ordered list of what should happen next]
## Critical Context
- [Any data, examples, or references needed to continue]
Files read: [src/index.ts, package.json]
Files modified: [src/index.ts]
```
### 5.5 Iterative Compaction
Successive compact calls **update** the existing summary rather than replacing it:
```
Compaction 1 (at ~185K tokens):
Summary: "## Goal: Build a login page..."
firstKeptEntryId: "entry-003"
Compaction 2 (at ~185K tokens again):
previousSummary: "## Goal: Build a login page..."
→ UPDATE_SUMMARIZATION_PROMPT
→ PRESERVES existing information
→ ADDS new progress (move "In Progress" → "Done")
→ NEW summary: "## Goal: Build a login page... Add OAuth..."
firstKeptEntryId: "entry-003" (same boundary)
```
---
## 6. Session Storage — JSONL Format
```
Session file: .pi/sessions/--home-user--/2024-01-15T10-30-00_abc123.jsonl
Line 1 (header):
{"type":"session","version":3,"id":"abc123","timestamp":"2024-01-15T10:30:00.000Z",
"cwd":"/home/user/project","parentSession":"...","metadata":{}}
Line 2+ (entries, one per line):
{"type":"message","id":"e001","parentId":null,"timestamp":"...","message":{...}}
{"type":"message","id":"e002","parentId":"e001","timestamp":"...","message":{...}}
{"type":"compaction","id":"e003","parentId":"e002","timestamp":"...",
"summary":"## Goal: ...\n...","firstKeptEntryId":"e001",
"tokensBefore":185000}
{"type":"leaf","id":"e004","parentId":"e003","timestamp":"...",
"targetId":"e002"}
```
### Entry Types
| Type | LLM Context? | Purpose |
|------|-------------|---------|
| `message` | Yes | User, assistant, toolResult |
| `compaction` | Yes (as summary message) | Replaces compacted history |
| `branch_summary` | Yes (as summary message) | Summary of diverged branch |
| `leaf` | No | Points to current tree leaf |
| `thinking_level_change` | No | Tracking thinking level changes |
| `model_change` | No | Tracking model changes |
| `active_tools_change` | No | Tracking tool enable/disable |
| `custom` | No (unless projector configured) | App-defined data |
| `custom_message` | Yes | App-defined messages |
| `label` | No | Human-readable labels |
| `session_info` | No | Session name history |
---
## 7. Session Tree (Branching)
Sessions form a **tree**, not a linear log. This lets users "go back" and try a different approach.
```
Session tree:
┌───[e01]───┐
│ user: "a" │
└─────┬──────┘
▼
┌──────────┐
│ assist 1 │
└─────┬────┘
▼
┌──────────┐
│ toolCall │
└─────┬────┘
▼
┌──────────┐
│ toolRes 1│
└─────┬────┘
▼
┌──────────┐
│ user: "b"│ ← user goes back here
└─────┬────┘
│
┌─────┴─────┐
│ │
┌──────────┐ ┌──────────┐
│ user: "c" │ │ user: "d" │ ← branch point
└────┬─────┘ └────┬─────┘
│ │
┌────▼─────┐ ┌────▼─────┐
│ assist 2 │ │ assist 3 │ ← current leaf (d)
└──────────┘ └──────────┘
When user navigates to "user: b":
- Leaf moves from "d" back to "b"
- Branch summary generated for diverged work ("c" → "assist 2")
- New work branches from "b":
┌─────┐
│ user: "e" │ ← new branch
└─────┬─────┘
▼
┌──────────┐
│ assist 4 │
└──────────┘
Context sent to LLM:
[compaction summary, user:b, user:e, assist:4]
→ The old "c"/"assist 2" branch is replaced by its summary
```
---
## 8. Pending Writes — Batching Session Persistence
To avoid writing every message individually during a run:
```
Agent loop events
│
▼
handleAgentEvent(event)
│
├── message_end → pendingSessionWrites.push({ type: "message", message })
├── turn_end → flushPendingSessionWrites() (save_point)
├── agent_end → flushPendingSessionWrites()
│
▼
flushPendingSessionWrites()
│
├── Iterate pendingSessionWrites[]
│ ├── message → session.appendMessage(msg)
│ ├── model_change → session.appendModelChange(provider, id)
│ ├── thinking_level_change → session.appendThinkingLevelChange(level)
│ ├── active_tools_change → session.appendActiveToolsChange(names)
│ ├── custom → session.appendCustomEntry(type, data)
│ ├── custom_message → session.appendCustomMessageEntry(...)
│ ├── label → session.appendLabel(targetId, label)
│ ├── session_info → session.appendSessionName(name)
│ └── leaf → session.getStorage().setLeafId(targetId)
│
└── Shift all writes → empty pending list
```
During the run, messages are accumulated in `pendingSessionWrites` and only flushed to disk at `save_point` (end of each turn) or `agent_end`.
---
## 9. Hooks & Extensibility — Context Control Points
The harness exposes hooks at every memory management boundary:
```
┌────────────────────────────────────────────────────────────────┐
│ Hooks │
├────────────────────────┬─────────────────────────────────────┤
│ Hook │ When │
├────────────────────────┼─────────────────────────────────────┤
│ before_agent_start │ Before each prompt, can add │
│ │ messages or modify system prompt │
├────────────────────────┼─────────────────────────────────────┤
│ context │ Before each LLM call, can prune/ │
│ │ modify AgentMessage[] │
├────────────────────────┼─────────────────────────────────────┤
│ before_provider_request│ Before each provider API call, can │
│ │ modify headers, retries, timeout │
├────────────────────────┼─────────────────────────────────────┤
│ before_provider_payload│ Before sending payload to provider,│
│ │ can modify the request body │
├────────────────────────┼─────────────────────────────────────┤
│ after_provider_response│ After receiving response, for │
│ │ logging/metrics │
├────────────────────────┼─────────────────────────────────────┤
│ tool_call │ Before tool execution, can │
│ │ return { block: true } │
├────────────────────────┼─────────────────────────────────────┤
│ tool_result │ After tool execution, can override │
│ │ content, details, isError, terminate│
├────────────────────────┼─────────────────────────────────────┤
│ session_before_compact│ Before compaction, can cancel or │
│ │ provide custom compaction result │
├────────────────────────┼─────────────────────────────────────┤
│ session_before_tree │ Before branch navigation, can │
│ │ cancel or provide custom summary │
├────────────────────────┼─────────────────────────────────────┤
│ prepareNextTurn │ Between turns, can replace context, │
│ │ model, or thinkingLevel │
├────────────────────────┼─────────────────────────────────────┤
│ shouldStopAfterTurn │ After a turn, if true the loop │
│ │ exits (agent_end, no more LLM calls)│
└────────────────────────┴─────────────────────────────────────┘
```
---
## 10. Complete Lifecycle: Long Session
```
Session starts empty
│
▼
Turn 1: "Create a React component"
Context: [compaction summary (empty)]
Messages exchanged: ~2K tokens
└─ Session: [user1, assist1, toolCall, toolRes1, assist2]
│
▼
Turn 2-10: Iterative development
Context: growing with each turn
Total context: ~50K tokens
└─ Session: [user1..assist2, user2..assist20]
│
▼
Turn 15: Context approaching limit (~170K tokens)
shouldCompact() → true
└─ Compaction 1:
- Summarizes turns 1-12
- Keeps turns 13-15 verbatim
- Summary: "## Goal: React component ## Progress: built X, Y"
│
▼
Turn 20: Context ~180K tokens
shouldCompact() → true
└─ Compaction 2 (iterative update):
- Updates existing summary with new progress
- "## Goal: React component ## Done: built X,Y ## New: added auth"
│
▼
Turn 25: Context ~186K tokens → triggers compaction
└─ Compaction 3:
- Summary now covers ~22 turns of history
- Retained tail: last 20K tokens (~5 turns)
- Context window freed: ~186K → ~25K tokens
│
▼
User navigates to Turn 8:
- Branch summary generated for Turns 9-25
- Leaf moves back to Turn 8
- Context: [compaction, branch_summary, turns 1-8]
│
▼
User continues from Turn 8:
- New branch grows from Turn 8
- Old branch (9-25) replaced by branch_summary
│
▼
Session ends, JSONL file persists on disk
Next session: loads from JSONL, rebuilds context
```
---
## 11. Token Budget Summary
```
Example: Claude Sonnet (200K context window)
┌─────────────────────────────────────────────────────────────┐
│ Context Window: 200,000 tokens │
├─────────────────────────────────────────────────────────────┤
│ Reserved for summary: 16,384 tokens │
├─────────────────────────────────────────────────────────────┤
│ Keep recent: 20,000 tokens │
├─────────────────────────────────────────────────────────────┤
│ Max context before compaction: 183,616 tokens │
│ (= 200000 - 16384) │
├─────────────────────────────────────────────────────────────┤
│ After compaction: ~25,000 tokens │
│ (20,000 tail + ~5,000 summary) │
│ → ~158,616 tokens freed │
└─────────────────────────────────────────────────────────────┘
```
---
## 12. Key Files Reference
| File | Responsibility |
|------|---------------|
| `agent-loop.ts` | Core loop, tool execution, streaming |
| `agent.ts` | Stateful `Agent` class, event system |
| `agent-harness.ts` | High-level harness, hooks, session management |
| `compaction/compaction.ts` | Token estimation, cut point, LLM summarization |
| `compaction/branch-summarization.ts` | Branch divergence summarization |
| `session/session.ts` | Session tree, entry appending, context building |
| `session/jsonl-storage.ts` | JSONL file read/write |
| `session/jsonl-repo.ts` | Session repo: create/open/list/delete/fork |
| `types.ts` | All type definitions |
| `messages.ts` | `convertToLlm()`, custom message helpers |
| `system-prompt.ts` | System prompt building |
| `skills.ts` | Skill management |
src/llmfunction.jl
+9 -9
View File
@@ -330,17 +330,17 @@ function search_wine_database!(a::T, thoughtdict::AbstractDict; useSQLLLM::Bool=
pg_conn_str = "host=$host_url port=$port dbname=$dbname user=$user password=$password" pg_conn_str = "host=$host_url port=$port dbname=$dbname user=$user password=$password"
#WORKING #WORKING
df = GeneralUtils.find_text_vector_similarity( # df = GeneralUtils.find_text_vector_similarity(
vector_search_str, # vector_search_str,
"wine", # "wine",
"tasting_notes_embedding", # "tasting_notes_embedding",
GeneralUtils.execute_postgres_sql(pg_conn_str, sql), #BUG input pair (F, arg) # GeneralUtils.execute_postgres_sql(pg_conn_str, sql), #BUG input pair (F, arg)
a.context.getTextEmbedding([vector_search_str]) #BUG input pair (F, arg) # a.context.getTextEmbedding([vector_search_str]) #BUG input pair (F, arg)
) # )
@show df # @show df
error(888888) # error(888888)
items = nothing items = nothing