38 KiB
38 KiB
Agent Workflow: Message Handling and Response Generation
Overview
This document explains the complete step-by-step flow of how the agent processes a user message and generates a response, from the moment a user asks "what product do you have in stock" to when the agent responds with an answer.
Architecture Diagram
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ USER INPUT │
│ "what product do you have in stock?" │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 1. AGENT.PROMPT() ENTRY │
│ File: packages/agent/src/agent.ts:339 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ • Validate no active run (throws if busy) │ │
│ │ • normalizePromptInput() converts string to AgentMessage[] │ │
│ │ • runPromptMessages() launches execution with lifecycle events │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 2. AGENT LIFECYCLE INITIALIZATION │
│ File: packages/agent/src/agent.ts:398-412 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Events Emitted: │ │
│ │ • agent_start │ │
│ │ • turn_start │ │
│ │ • message_start / message_end (for each prompt message) │ │
│ │ │ │
│ │ Context Snapshot Created: │ │
│ │ • systemPrompt │ │
│ │ • messages (copy) │ │
│ │ • tools (copy) │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 3. AGENT LOOP STARTS │
│ File: packages/agent/src/agent-loop.ts:95 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ runPromptMessages() → runAgentLoop() │ │
│ │ • Prompts added to context.messages │ │
│ │ • Lifecycle events emitted for prompts │ │
│ │ • Calls runLoop() (main processing loop) │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 4. LLM CALL BOUNDARY - MESSAGE TRANSFORMATION │
│ File: packages/agent/src/agent-loop.ts:281-372 (streamAssistantResponse) │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Step 4.1: Context Transform (optional) │ │
│ │ transformContext(messages) → transformed messages │ │
│ │ (Used for context pruning/injection) │ │
│ │ │ │
│ │ Step 4.2: LLM Conversion │ │
│ │ convertToLlm(messages) → Message[] │ │
│ │ - Filters non-LLM messages (bashExecution, branchSummary, etc.) │ │
│ │ - Converts: user → user, assistant → assistant, toolResult → toolResult │ │
│ │ │ │
│ │ Step 4.3: Build LLM Context │ │
│ │ { │ │
│ │ systemPrompt: context.systemPrompt, │ │
│ │ messages: llmMessages, │ │
│ │ tools: context.tools │ │
│ │ } │ │
│ │ │ │
│ │ Step 4.4: Resolve API Key │ │
│ │ getApiKey(model.provider) → apiKey │ │
│ │ │ │
│ │ Step 4.5: Stream Function Call │ │
│ │ streamFunction(model, llmContext, options) │ │
│ │ - Default: Models.streamSimple() from @earendil-works/pi-ai │ │
│ │ - Makes actual LLM API call │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 5. LLM RESPONSE STREAMING │
│ File: packages/agent/src/agent-loop.ts:317-371 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Stream yields events: │ │
│ │ • start → creates partial AssistantMessage │ │
│ │ • text_start → streaming text begins │ │
│ │ • text_delta → text chunks arrive │ │
│ │ • toolcall_start → tool call block begins │ │
│ │ • toolcall_delta → tool call arguments arrive │ │
│ │ • toolcall_end → tool call block complete │ │
│ │ • text_end → text block complete │ │
│ │ • done → final message complete │ │
│ │ │ │
│ │ State Updates: │ │
│ │ • Partial message pushed to context.messages │ │
│ │ • message_start event emitted │ │
│ │ • message_update events emitted as text/tools stream in │ │
│ │ • Final message committed to context.messages │ │
│ │ • message_end event emitted │ │
│ │ │ │
│ │ Stop Reasons: │ │
│ │ • stop - normal completion │ │
│ │ • toolUse - model requested tool calls │ │
│ │ • length - token limit reached │ │
│ │ • error - failure │ │
│ │ • aborted - operation aborted │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 6. TOOL CALL PARSING AND EXECUTION │
│ File: packages/agent/src/agent-loop.ts:408-554 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Step 6.1: Extract Tool Calls │ │
│ │ toolCalls = message.content.filter(c => c.type === "toolCall") │ │
│ │ │ │
│ │ Step 6.2: Determine Execution Mode │ │
│ │ - Check config.toolExecution: "parallel" or "sequential" │ │
│ │ - Check individual tool executionMode setting │ │
│ │ - Decides how to execute tool batch │ │
│ │ │ │
│ │ Step 6.3: For Each Tool Call │ │
│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │
│ │ │ 1. Tool Lookup │ │ │
│ │ │ tool = context.tools.find(t => t.name === toolCall.name) │ │ │
│ │ │ │ │ │
│ │ │ 2. Argument Preparation │ │ │
│ │ │ prepared = tool.prepareArguments?(toolCall.arguments) │ │ │
│ │ │ │ │ │
│ │ │ 3. Argument Validation │ │ │
│ │ │ validateToolArguments(tool, preparedToolCall) │ │ │
│ │ │ │ │ │
│ │ │ 4. Before Tool Hook │ │ │
│ │ │ beforeToolCall({ assistantMessage, toolCall, args, context }) │ │ │
│ │ │ - Can block execution by returning { block: true, reason } │ │ │
│ │ │ │ │ │
│ │ │ 5. Execution │ │ │
│ │ │ execute(toolCallId, params, signal, onUpdate, context) │ │ │
│ │ │ │ │ │
│ │ │ - Parallel Mode: Tools execute concurrently │ │ │
│ │ │ - Sequential Mode: Tools execute one-by-one │ │ │
│ │ └──────────────────────────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 7. TOOL EXECUTION EXAMPLE - READ TOOL │
│ File: packages/agent/src/harness/tools/read.ts │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ User asks: "what product do you have in stock?" │ │
│ │ │ │
│ │ Agent decides to read catalog file "products.json" │ │
│ │ │ │
│ │ Input Arguments: │ │
│ │ { │ │
│ │ "path": "products.json", │ │
│ │ "offset": 1, │ │
│ │ "limit": 100 │ │
│ │ } │ │
│ │ │ │
│ │ Execution Steps: │ │
│ │ 1. resolveReadToolPath(env, path, signal) → absolutePath │ │
│ │ 2. env.readBinaryFile(absolutePath, signal) → bytes │ │
│ │ 3. Detect mimeType (check if image) │ │
│ │ 4. For text files: │ │
│ │ - Decode UTF-8 → textContent │ │
│ │ - Split by lines → allLines │ │
│ │ - Apply offset/limit slicing │ │
│ │ - Truncate if exceeds DEFAULT_MAX_BYTES or DEFAULT_MAX_LINES │ │
│ │ - Add truncation notice to output │ │
│ │ 5. Return result: │ │
│ │ { │ │
│ │ content: [{ type: "text", text: output }], │ │
│ │ details: { truncation: ... } │ │
│ │ } │ │
│ │ │ │
│ │ Output Example: │ │
│ │ "Showing lines 1-50 of 150. [Showing 50 lines of 150. Use offset=51 to │ │
│ │ continue.]" │ │
│ │ │ │
│ │ Tool Result: │ │
│ │ { │ │
│ │ "role": "toolResult", │ │
│ │ "toolCallId": "tool_abc123", │ │
│ │ "toolName": "read", │ │
│ │ "content": [{ "type": "text", "text": "..." }], │ │
│ │ "isError": false, │ │
│ │ "timestamp": 1721721600000 │ │
│ │ } │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 8. TOOL RESULT HANDLING │
│ File: packages/agent/src/agent-loop.ts:556-792 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Step 8.1: After Tool Hook │ │
│ │ afterToolCall({ assistantMessage, toolCall, args, result, isError, ctx }) │ │
│ │ - Can override result content, details, usage, terminate hint │ │
│ │ │ │
│ │ Step 8.2: Create Tool Result Message │ │
│ │ { │ │
│ │ role: "toolResult", │ │
│ │ toolCallId: toolCall.id, │ │
│ │ toolName: toolCall.name, │ │
│ │ content: result.content ?? [], │ │
│ │ details: result.details, │ │
│ │ usage: result.usage, │ │
│ │ isError: false, │ │
│ │ timestamp: Date.now() │ │
│ │ } │ │
│ │ │ │
│ │ Step 8.3: Emit Events │ │
│ │ • tool_execution_start │ │
│ │ • tool_execution_end │ │
│ │ • message_start (toolResult message) │ │
│ │ • message_end (toolResult message) │ │
│ │ │ │
│ │ Step 8.4: Update Context │ │
│ │ • Push tool result message to currentContext.messages │ │
│ │ • Push to newMessages array │ │
│ │ │ │
│ │ Step 8.5: Batch Termination Check │ │
│ │ shouldTerminateToolBatch(finalizedCalls) │ │
│ │ - Returns true if ALL tools set terminate: true │ │
│ │ - If true, agent may stop after this batch │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 9. NEXT TURN PREPARATION │
│ File: packages/agent/src/agent-loop.ts:224-257 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ Step 9.1: Turn End Event │ │
│ │ turn_end emitted with message and toolResults │ │
│ │ │ │
│ │ Step 9.2: prepareNextTurn Hook │ │
│ │ prepareNextTurn({ message, toolResults, context, newMessages }) │ │
│ │ - Can return updated context, model, or thinking level │ │
│ │ - Used for dynamic context management │ │
│ │ │ │
│ │ Step 9.3: Queue Polling │ │
│ │ getSteeringMessages() → inject messages for immediate processing │ │
│ │ getFollowUpMessages() → check for queued follow-up messages │ │
│ │ │ │
│ │ Step 9.4: Loop Decision │ │
│ │ • If steering messages exist → process them, continue loop │ │
│ │ • If follow-up messages exist → process them, continue loop │ │
│ │ • If tool calls remain in message → continue inner loop │ │
│ │ • If no messages → emit agent_end, exit loop │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ 10. AGENT RESPONSE GENERATION │
│ File: packages/agent/src/agent-loop.ts:58-163 │
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
│ │ The agent loop continues until: │ │
│ │ • No tool calls remain in assistant messages │ │
│ │ • No steering/follow-up messages queued │ │
│ │ • shouldStopAfterTurn() returns true (if configured) │ │
│ │ │ │
│ │ Final Response Generation: │ │
│ │ 1. LLM streams text content blocks │ │
│ │ 2. Message committed to context │ │
│ │ 3. turn_end emitted │ │
│ │ 4. agent_end emitted with all new messages │ │
│ │ 5. Agent returns to idle state │ │
│ │ │ │
│ │ Final Agent Response: │ │
│ │ "We have 15 products in stock:" │ │
│ │ • Product A - $29.99 │ │
│ │ • Product B - $49.99 │ │
│ │ • Product C - $19.99 │ │
│ │ (and 2 more products) │ │
│ └──────────────────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────────────┘
Complete Event Flow Diagram
┌─────────────────────────────────────────────────────────────────────────────────────┐
│ COMPLETE EVENT SEQUENCE FOR A TURN │
│ WITH TOOL USE (Product Catalog Query) │
└─────────────────────────────────────────────────────────────────────────────────────┘
agent_start
↓
turn_start
↓
message_start (user message: "what product do you have in stock?")
message_end (user message)
↓
message_start (assistant message - streaming from LLM)
message_update (text delta: "We have")
message_update (toolcall delta: {"name":"read","arguments":{...}})
message_update (toolcall end)
message_end (assistant message with tool calls)
↓
tool_execution_start (tool call: read products.json)
tool_execution_end (tool call: read complete)
message_start (toolResult message)
message_end (toolResult message)
↓
turn_end
↓
[Inner loop continues: send tool result to LLM]
↓
turn_start
↓
message_start (assistant message - streaming from LLM)
message_update (text delta: "We have 15 products in stock:")
message_update (text delta: "• Product A - $29.99")
message_update (text delta: "• Product B - $49.99")
message_end (final assistant message)
↓
turn_end
↓
agent_end
Key Files Summary
| Component | File | Purpose |
|---|---|---|
| Agent class | packages/agent/src/agent.ts |
Stateful wrapper, event emission, queue management |
| Agent loop | packages/agent/src/agent-loop.ts |
Core loop, LLM calls, tool execution |
| Message types | packages/agent/src/types.ts |
AgentMessage, AgentTool, AgentEvent definitions |
| Harness | packages/agent/src/harness/agent-harness.ts |
Session integration, hooks, persistence |
| Read tool | packages/agent/src/harness/tools/read.ts |
File reading implementation |
| Stream function | packages/agent/src/stream-fn.ts |
Default stream function management |
| Types | packages/agent/src/types.ts |
All type definitions |
Hook Points for Customization
The agent supports multiple extension points:
| Hook | Location | Purpose |
|---|---|---|
convertToLlm |
agent.ts:99 |
Transform messages before LLM call |
transformContext |
agent.ts:100 |
Modify context (pruning, injection) |
beforeToolCall |
agent.ts:105 |
Block or modify tool execution |
afterToolCall |
agent.ts:106 |
Override tool results |
prepareNextTurn |
agent.ts:107 |
Dynamic context/model updates |
shouldStopAfterTurn |
agent-loop.ts |
Graceful termination |
getSteeringMessages |
agent.ts:114 |
Inject messages mid-turn |
getFollowUpMessages |
agent.ts:115 |
Queue follow-up messages |
Tool Execution Flow
Tool Call Received from LLM
↓
1. Tool Lookup (find by name)
↓
2. prepareArguments? (transform if defined)
↓
3. validateToolArguments (JSON Schema)
↓
4. beforeToolCall hook (can block)
↓
5. execute (parallel or sequential)
↓
6. onUpdate (stream partial results)
↓
7. afterToolCall hook (can override)
↓
8. Create toolResult message
↓
9. Emit events (start, end)
↓
10. Add to context.messages
Example: "What product do you have in stock?"
Step-by-Step Execution:
-
User sends message
"what product do you have in stock?" -
Agent normalizes input
[{ role: "user", content: [{ type: "text", text: "what product do you have in stock?" }], timestamp: Date.now() }] -
LLM processes and decides to use
readtool{ "role": "assistant", "content": [{ "type": "toolCall", "name": "read", "arguments": { "path": "products.json", "offset": 1, "limit": 50 }, "id": "tool_abc123" }] } -
Tool execution
- Read
products.json(150 lines total) - Return lines 1-50 with truncation notice
- Add to context as toolResult
- Read
-
LLM generates final response
We have 15 products in stock: • Product A - $29.99 • Product B - $49.99 • Product C - $19.99 • Product D - $99.99 • Product E - $149.99 (and 10 more products) Use offset=51 to continue viewing. -
Agent emits final response to user
Summary
The agent workflow follows a clear pattern:
- Message Input → Normalize and validate
- Context Setup → Create snapshot with system prompt, messages, tools
- LLM Call → Transform messages, resolve API key, stream response
- Tool Detection → Check for tool calls in assistant message
- Tool Execution → Validate, hook, execute, stream updates
- Result Handling → Create toolResult message, emit events
- Next Turn → Check for steering/follow-up messages, prepare context
- Response Generation → Continue until no more tool calls needed
- Completion → Emit final response to user
The entire flow is event-driven, allowing for real-time updates and hook-based customization at every step.