474 lines
38 KiB
Markdown
474 lines
38 KiB
Markdown
# Agent Workflow: Message Handling and Response Generation
|
|
|
|
## Overview
|
|
|
|
This document explains the complete step-by-step flow of how the agent processes a user message and generates a response, from the moment a user asks "what product do you have in stock" to when the agent responds with an answer.
|
|
|
|
---
|
|
|
|
## Architecture Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ USER INPUT │
|
|
│ "what product do you have in stock?" │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 1. AGENT.PROMPT() ENTRY │
|
|
│ File: packages/agent/src/agent.ts:339 │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ • Validate no active run (throws if busy) │ │
|
|
│ │ • normalizePromptInput() converts string to AgentMessage[] │ │
|
|
│ │ • runPromptMessages() launches execution with lifecycle events │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 2. AGENT LIFECYCLE INITIALIZATION │
|
|
│ File: packages/agent/src/agent.ts:398-412 │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Events Emitted: │ │
|
|
│ │ • agent_start │ │
|
|
│ │ • turn_start │ │
|
|
│ │ • message_start / message_end (for each prompt message) │ │
|
|
│ │ │ │
|
|
│ │ Context Snapshot Created: │ │
|
|
│ │ • systemPrompt │ │
|
|
│ │ • messages (copy) │ │
|
|
│ │ • tools (copy) │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 3. AGENT LOOP STARTS │
|
|
│ File: packages/agent/src/agent-loop.ts:95 │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ runPromptMessages() → runAgentLoop() │ │
|
|
│ │ • Prompts added to context.messages │ │
|
|
│ │ • Lifecycle events emitted for prompts │ │
|
|
│ │ • Calls runLoop() (main processing loop) │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 4. LLM CALL BOUNDARY - MESSAGE TRANSFORMATION │
|
|
│ File: packages/agent/src/agent-loop.ts:281-372 (streamAssistantResponse) │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Step 4.1: Context Transform (optional) │ │
|
|
│ │ transformContext(messages) → transformed messages │ │
|
|
│ │ (Used for context pruning/injection) │ │
|
|
│ │ │ │
|
|
│ │ Step 4.2: LLM Conversion │ │
|
|
│ │ convertToLlm(messages) → Message[] │ │
|
|
│ │ - Filters non-LLM messages (bashExecution, branchSummary, etc.) │ │
|
|
│ │ - Converts: user → user, assistant → assistant, toolResult → toolResult │ │
|
|
│ │ │ │
|
|
│ │ Step 4.3: Build LLM Context │ │
|
|
│ │ { │ │
|
|
│ │ systemPrompt: context.systemPrompt, │ │
|
|
│ │ messages: llmMessages, │ │
|
|
│ │ tools: context.tools │ │
|
|
│ │ } │ │
|
|
│ │ │ │
|
|
│ │ Step 4.4: Resolve API Key │ │
|
|
│ │ getApiKey(model.provider) → apiKey │ │
|
|
│ │ │ │
|
|
│ │ Step 4.5: Stream Function Call │ │
|
|
│ │ streamFunction(model, llmContext, options) │ │
|
|
│ │ - Default: Models.streamSimple() from @earendil-works/pi-ai │ │
|
|
│ │ - Makes actual LLM API call │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 5. LLM RESPONSE STREAMING │
|
|
│ File: packages/agent/src/agent-loop.ts:317-371 │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Stream yields events: │ │
|
|
│ │ • start → creates partial AssistantMessage │ │
|
|
│ │ • text_start → streaming text begins │ │
|
|
│ │ • text_delta → text chunks arrive │ │
|
|
│ │ • toolcall_start → tool call block begins │ │
|
|
│ │ • toolcall_delta → tool call arguments arrive │ │
|
|
│ │ • toolcall_end → tool call block complete │ │
|
|
│ │ • text_end → text block complete │ │
|
|
│ │ • done → final message complete │ │
|
|
│ │ │ │
|
|
│ │ State Updates: │ │
|
|
│ │ • Partial message pushed to context.messages │ │
|
|
│ │ • message_start event emitted │ │
|
|
│ │ • message_update events emitted as text/tools stream in │ │
|
|
│ │ • Final message committed to context.messages │ │
|
|
│ │ • message_end event emitted │ │
|
|
│ │ │ │
|
|
│ │ Stop Reasons: │ │
|
|
│ │ • stop - normal completion │ │
|
|
│ │ • toolUse - model requested tool calls │ │
|
|
│ │ • length - token limit reached │ │
|
|
│ │ • error - failure │ │
|
|
│ │ • aborted - operation aborted │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 6. TOOL CALL PARSING AND EXECUTION │
|
|
│ File: packages/agent/src/agent-loop.ts:408-554 │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Step 6.1: Extract Tool Calls │ │
|
|
│ │ toolCalls = message.content.filter(c => c.type === "toolCall") │ │
|
|
│ │ │ │
|
|
│ │ Step 6.2: Determine Execution Mode │ │
|
|
│ │ - Check config.toolExecution: "parallel" or "sequential" │ │
|
|
│ │ - Check individual tool executionMode setting │ │
|
|
│ │ - Decides how to execute tool batch │ │
|
|
│ │ │ │
|
|
│ │ Step 6.3: For Each Tool Call │ │
|
|
│ │ ┌──────────────────────────────────────────────────────────────────────┐ │ │
|
|
│ │ │ 1. Tool Lookup │ │ │
|
|
│ │ │ tool = context.tools.find(t => t.name === toolCall.name) │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ 2. Argument Preparation │ │ │
|
|
│ │ │ prepared = tool.prepareArguments?(toolCall.arguments) │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ 3. Argument Validation │ │ │
|
|
│ │ │ validateToolArguments(tool, preparedToolCall) │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ 4. Before Tool Hook │ │ │
|
|
│ │ │ beforeToolCall({ assistantMessage, toolCall, args, context }) │ │ │
|
|
│ │ │ - Can block execution by returning { block: true, reason } │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ 5. Execution │ │ │
|
|
│ │ │ execute(toolCallId, params, signal, onUpdate, context) │ │ │
|
|
│ │ │ │ │ │
|
|
│ │ │ - Parallel Mode: Tools execute concurrently │ │ │
|
|
│ │ │ - Sequential Mode: Tools execute one-by-one │ │ │
|
|
│ │ └──────────────────────────────────────────────────────────────────────┘ │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 7. TOOL EXECUTION EXAMPLE - READ TOOL │
|
|
│ File: packages/agent/src/harness/tools/read.ts │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ User asks: "what product do you have in stock?" │ │
|
|
│ │ │ │
|
|
│ │ Agent decides to read catalog file "products.json" │ │
|
|
│ │ │ │
|
|
│ │ Input Arguments: │ │
|
|
│ │ { │ │
|
|
│ │ "path": "products.json", │ │
|
|
│ │ "offset": 1, │ │
|
|
│ │ "limit": 100 │ │
|
|
│ │ } │ │
|
|
│ │ │ │
|
|
│ │ Execution Steps: │ │
|
|
│ │ 1. resolveReadToolPath(env, path, signal) → absolutePath │ │
|
|
│ │ 2. env.readBinaryFile(absolutePath, signal) → bytes │ │
|
|
│ │ 3. Detect mimeType (check if image) │ │
|
|
│ │ 4. For text files: │ │
|
|
│ │ - Decode UTF-8 → textContent │ │
|
|
│ │ - Split by lines → allLines │ │
|
|
│ │ - Apply offset/limit slicing │ │
|
|
│ │ - Truncate if exceeds DEFAULT_MAX_BYTES or DEFAULT_MAX_LINES │ │
|
|
│ │ - Add truncation notice to output │ │
|
|
│ │ 5. Return result: │ │
|
|
│ │ { │ │
|
|
│ │ content: [{ type: "text", text: output }], │ │
|
|
│ │ details: { truncation: ... } │ │
|
|
│ │ } │ │
|
|
│ │ │ │
|
|
│ │ Output Example: │ │
|
|
│ │ "Showing lines 1-50 of 150. [Showing 50 lines of 150. Use offset=51 to │ │
|
|
│ │ continue.]" │ │
|
|
│ │ │ │
|
|
│ │ Tool Result: │ │
|
|
│ │ { │ │
|
|
│ │ "role": "toolResult", │ │
|
|
│ │ "toolCallId": "tool_abc123", │ │
|
|
│ │ "toolName": "read", │ │
|
|
│ │ "content": [{ "type": "text", "text": "..." }], │ │
|
|
│ │ "isError": false, │ │
|
|
│ │ "timestamp": 1721721600000 │ │
|
|
│ │ } │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 8. TOOL RESULT HANDLING │
|
|
│ File: packages/agent/src/agent-loop.ts:556-792 │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Step 8.1: After Tool Hook │ │
|
|
│ │ afterToolCall({ assistantMessage, toolCall, args, result, isError, ctx }) │ │
|
|
│ │ - Can override result content, details, usage, terminate hint │ │
|
|
│ │ │ │
|
|
│ │ Step 8.2: Create Tool Result Message │ │
|
|
│ │ { │ │
|
|
│ │ role: "toolResult", │ │
|
|
│ │ toolCallId: toolCall.id, │ │
|
|
│ │ toolName: toolCall.name, │ │
|
|
│ │ content: result.content ?? [], │ │
|
|
│ │ details: result.details, │ │
|
|
│ │ usage: result.usage, │ │
|
|
│ │ isError: false, │ │
|
|
│ │ timestamp: Date.now() │ │
|
|
│ │ } │ │
|
|
│ │ │ │
|
|
│ │ Step 8.3: Emit Events │ │
|
|
│ │ • tool_execution_start │ │
|
|
│ │ • tool_execution_end │ │
|
|
│ │ • message_start (toolResult message) │ │
|
|
│ │ • message_end (toolResult message) │ │
|
|
│ │ │ │
|
|
│ │ Step 8.4: Update Context │ │
|
|
│ │ • Push tool result message to currentContext.messages │ │
|
|
│ │ • Push to newMessages array │ │
|
|
│ │ │ │
|
|
│ │ Step 8.5: Batch Termination Check │ │
|
|
│ │ shouldTerminateToolBatch(finalizedCalls) │ │
|
|
│ │ - Returns true if ALL tools set terminate: true │ │
|
|
│ │ - If true, agent may stop after this batch │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 9. NEXT TURN PREPARATION │
|
|
│ File: packages/agent/src/agent-loop.ts:224-257 │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ Step 9.1: Turn End Event │ │
|
|
│ │ turn_end emitted with message and toolResults │ │
|
|
│ │ │ │
|
|
│ │ Step 9.2: prepareNextTurn Hook │ │
|
|
│ │ prepareNextTurn({ message, toolResults, context, newMessages }) │ │
|
|
│ │ - Can return updated context, model, or thinking level │ │
|
|
│ │ - Used for dynamic context management │ │
|
|
│ │ │ │
|
|
│ │ Step 9.3: Queue Polling │ │
|
|
│ │ getSteeringMessages() → inject messages for immediate processing │ │
|
|
│ │ getFollowUpMessages() → check for queued follow-up messages │ │
|
|
│ │ │ │
|
|
│ │ Step 9.4: Loop Decision │ │
|
|
│ │ • If steering messages exist → process them, continue loop │ │
|
|
│ │ • If follow-up messages exist → process them, continue loop │ │
|
|
│ │ • If tool calls remain in message → continue inner loop │ │
|
|
│ │ • If no messages → emit agent_end, exit loop │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ 10. AGENT RESPONSE GENERATION │
|
|
│ File: packages/agent/src/agent-loop.ts:58-163 │
|
|
│ ┌──────────────────────────────────────────────────────────────────────────────┐ │
|
|
│ │ The agent loop continues until: │ │
|
|
│ │ • No tool calls remain in assistant messages │ │
|
|
│ │ • No steering/follow-up messages queued │ │
|
|
│ │ • shouldStopAfterTurn() returns true (if configured) │ │
|
|
│ │ │ │
|
|
│ │ Final Response Generation: │ │
|
|
│ │ 1. LLM streams text content blocks │ │
|
|
│ │ 2. Message committed to context │ │
|
|
│ │ 3. turn_end emitted │ │
|
|
│ │ 4. agent_end emitted with all new messages │ │
|
|
│ │ 5. Agent returns to idle state │ │
|
|
│ │ │ │
|
|
│ │ Final Agent Response: │ │
|
|
│ │ "We have 15 products in stock:" │ │
|
|
│ │ • Product A - $29.99 │ │
|
|
│ │ • Product B - $49.99 │ │
|
|
│ │ • Product C - $19.99 │ │
|
|
│ │ (and 2 more products) │ │
|
|
│ └──────────────────────────────────────────────────────────────────────────────┘ │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
```
|
|
|
|
---
|
|
|
|
## Complete Event Flow Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
|
│ COMPLETE EVENT SEQUENCE FOR A TURN │
|
|
│ WITH TOOL USE (Product Catalog Query) │
|
|
└─────────────────────────────────────────────────────────────────────────────────────┘
|
|
|
|
agent_start
|
|
↓
|
|
turn_start
|
|
↓
|
|
message_start (user message: "what product do you have in stock?")
|
|
message_end (user message)
|
|
↓
|
|
message_start (assistant message - streaming from LLM)
|
|
message_update (text delta: "We have")
|
|
message_update (toolcall delta: {"name":"read","arguments":{...}})
|
|
message_update (toolcall end)
|
|
message_end (assistant message with tool calls)
|
|
↓
|
|
tool_execution_start (tool call: read products.json)
|
|
tool_execution_end (tool call: read complete)
|
|
message_start (toolResult message)
|
|
message_end (toolResult message)
|
|
↓
|
|
turn_end
|
|
↓
|
|
[Inner loop continues: send tool result to LLM]
|
|
↓
|
|
turn_start
|
|
↓
|
|
message_start (assistant message - streaming from LLM)
|
|
message_update (text delta: "We have 15 products in stock:")
|
|
message_update (text delta: "• Product A - $29.99")
|
|
message_update (text delta: "• Product B - $49.99")
|
|
message_end (final assistant message)
|
|
↓
|
|
turn_end
|
|
↓
|
|
agent_end
|
|
```
|
|
|
|
---
|
|
|
|
## Key Files Summary
|
|
|
|
| Component | File | Purpose |
|
|
|-----------|------|---------|
|
|
| Agent class | `packages/agent/src/agent.ts` | Stateful wrapper, event emission, queue management |
|
|
| Agent loop | `packages/agent/src/agent-loop.ts` | Core loop, LLM calls, tool execution |
|
|
| Message types | `packages/agent/src/types.ts` | AgentMessage, AgentTool, AgentEvent definitions |
|
|
| Harness | `packages/agent/src/harness/agent-harness.ts` | Session integration, hooks, persistence |
|
|
| Read tool | `packages/agent/src/harness/tools/read.ts` | File reading implementation |
|
|
| Stream function | `packages/agent/src/stream-fn.ts` | Default stream function management |
|
|
| Types | `packages/agent/src/types.ts` | All type definitions |
|
|
|
|
---
|
|
|
|
## Hook Points for Customization
|
|
|
|
The agent supports multiple extension points:
|
|
|
|
| Hook | Location | Purpose |
|
|
|------|----------|---------|
|
|
| `convertToLlm` | `agent.ts:99` | Transform messages before LLM call |
|
|
| `transformContext` | `agent.ts:100` | Modify context (pruning, injection) |
|
|
| `beforeToolCall` | `agent.ts:105` | Block or modify tool execution |
|
|
| `afterToolCall` | `agent.ts:106` | Override tool results |
|
|
| `prepareNextTurn` | `agent.ts:107` | Dynamic context/model updates |
|
|
| `shouldStopAfterTurn` | `agent-loop.ts` | Graceful termination |
|
|
| `getSteeringMessages` | `agent.ts:114` | Inject messages mid-turn |
|
|
| `getFollowUpMessages` | `agent.ts:115` | Queue follow-up messages |
|
|
|
|
---
|
|
|
|
## Tool Execution Flow
|
|
|
|
```
|
|
Tool Call Received from LLM
|
|
↓
|
|
1. Tool Lookup (find by name)
|
|
↓
|
|
2. prepareArguments? (transform if defined)
|
|
↓
|
|
3. validateToolArguments (JSON Schema)
|
|
↓
|
|
4. beforeToolCall hook (can block)
|
|
↓
|
|
5. execute (parallel or sequential)
|
|
↓
|
|
6. onUpdate (stream partial results)
|
|
↓
|
|
7. afterToolCall hook (can override)
|
|
↓
|
|
8. Create toolResult message
|
|
↓
|
|
9. Emit events (start, end)
|
|
↓
|
|
10. Add to context.messages
|
|
```
|
|
|
|
---
|
|
|
|
## Example: "What product do you have in stock?"
|
|
|
|
### Step-by-Step Execution:
|
|
|
|
1. **User sends message**
|
|
```
|
|
"what product do you have in stock?"
|
|
```
|
|
|
|
2. **Agent normalizes input**
|
|
```typescript
|
|
[{
|
|
role: "user",
|
|
content: [{ type: "text", text: "what product do you have in stock?" }],
|
|
timestamp: Date.now()
|
|
}]
|
|
```
|
|
|
|
3. **LLM processes and decides to use `read` tool**
|
|
```json
|
|
{
|
|
"role": "assistant",
|
|
"content": [{
|
|
"type": "toolCall",
|
|
"name": "read",
|
|
"arguments": {
|
|
"path": "products.json",
|
|
"offset": 1,
|
|
"limit": 50
|
|
},
|
|
"id": "tool_abc123"
|
|
}]
|
|
}
|
|
```
|
|
|
|
4. **Tool execution**
|
|
- Read `products.json` (150 lines total)
|
|
- Return lines 1-50 with truncation notice
|
|
- Add to context as toolResult
|
|
|
|
5. **LLM generates final response**
|
|
```
|
|
We have 15 products in stock:
|
|
|
|
• Product A - $29.99
|
|
• Product B - $49.99
|
|
• Product C - $19.99
|
|
• Product D - $99.99
|
|
• Product E - $149.99
|
|
(and 10 more products)
|
|
|
|
Use offset=51 to continue viewing.
|
|
```
|
|
|
|
6. **Agent emits final response to user**
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
The agent workflow follows a clear pattern:
|
|
|
|
1. **Message Input** → Normalize and validate
|
|
2. **Context Setup** → Create snapshot with system prompt, messages, tools
|
|
3. **LLM Call** → Transform messages, resolve API key, stream response
|
|
4. **Tool Detection** → Check for tool calls in assistant message
|
|
5. **Tool Execution** → Validate, hook, execute, stream updates
|
|
6. **Result Handling** → Create toolResult message, emit events
|
|
7. **Next Turn** → Check for steering/follow-up messages, prepare context
|
|
8. **Response Generation** → Continue until no more tool calls needed
|
|
9. **Completion** → Emit final response to user
|
|
|
|
The entire flow is event-driven, allowing for real-time updates and hook-based customization at every step.
|