update

2026-08-10 19:19:58 +07:00
parent c5cb18f0f1
commit a9fa23f01b
1 changed files with 202 additions and 94 deletions
@@ -6,26 +6,135 @@ This document describes the complete tool lifecycle in the YiemAgent framework,

 ## Table of Contents

-1. [Overview](#1-overview)
-2. [Tool Definition — The `agentTool` Struct](#2-tool-definition--the-agenttool-struct)
-3. [Tool Registration — Per-Agent Tool Stores](#3-tool-registration--per-agent-tool-stores)
-4. [The Agent Loop — High-Level Flow](#4-the-agent-loop--high-level-flow)
-5. [Message Processing Pipeline](#5-message-processing-pipeline)
-6. [Tool Call Extraction from LLM Response](#6-tool-call-extraction-from-llm-response)
-7. [The Per-Call Pipeline — Prepare, Execute, Finalize](#7-the-per-call-pipeline--prepare-execute-finalize)
-8. [Execution Modes — Sequential vs Parallel](#8-execution-modes--sequential-vs-parallel)
-9. [Tool Call Batches & Termination Logic](#9-tool-call-batches--termination-logic)
-10. [Tool Result Message Creation](#10-tool-result-message-creation)
-11. [Error Handling & Recovery Pattern](#11-error-handling--recovery-pattern)
-12. [Event System — Tool Lifecycle Events](#12-event-system--tool-lifecycle-events)
-13. [Agent Lifecycle Hooks](#13-agent-lifecycle-hooks)
-14. [Self-Modifying Tools](#14-self-modifying-tools)
-15. [Complete End-to-End Example](#15-complete-end-to-end-example)
-16. [Tool File Contract](#16-tool-file-contract)
+1. [Quick Start: Four-Step Tool Lifecycle](#1-quick-start-four-step-tool-lifecycle)
+2. [Overview](#2-overview)
+3. [Tool Definition — The `agentTool` Struct](#3-tool-definition--the-agenttool-struct)
+4. [Tool Registration — Per-Agent Tool Stores](#4-tool-registration--per-agent-tool-stores)
+5. [The Agent Loop — High-Level Flow](#5-the-agent-loop--high-level-flow)
+6. [Message Processing Pipeline](#6-message-processing-pipeline)
+7. [Tool Call Extraction from LLM Response](#7-tool-call-extraction-from-llm-response)
+8. [The Per-Call Pipeline — Prepare, Execute, Finalize](#8-the-per-call-pipeline--prepare-execute-finalize)
+9. [Execution Modes — Sequential vs Parallel](#9-execution-modes--sequential-vs-parallel)
+10. [Tool Call Batches & Termination Logic](#10-tool-call-batches--termination-logic)
+11. [Tool Result Message Creation](#11-tool-result-message-creation)
+12. [Error Handling & Recovery Pattern](#12-error-handling--recovery-pattern)
+13. [Event System — Tool Lifecycle Events](#13-event-system--tool-lifecycle-events)
+14. [Agent Lifecycle Hooks](#14-agent-lifecycle-hooks)
+15. [Self-Modifying Tools](#15-self-modifying-tools)
+16. [Complete End-to-End Example](#16-complete-end-to-end-example)
+17. [Tool File Contract](#17-tool-file-contract)
+18. [Appendix: Type Reference](#18-appendix-type-reference)

 ---

-## 1. Overview
+## 1. Quick Start: Four-Step Tool Lifecycle
+
+This section shows the complete lifecycle from discovery to result extraction. Each step maps to the detailed sections below.
+
+### Step 1: Discover — `listTools`
+
+The agent calls the `listTools` tool to see available tools and detect name collisions before creating new ones.
+
+```julia
+# The listTools tool is auto-injected via listTool(store) — no manual registration needed
+tool = listTool(store)  # Returns an agentTool that, when executed, lists all tools in the store
+```
+
+**Result extraction:**
+```julia
+result = tool.execute("call-1", Dict{String,Any}(), nothing, x->x)
+# result.content[1].text => "Available tools:\n- getTime: Time Lookup — Get current local time...\n- getWeather: Weather Lookup — Fetch current weather..."
+```
+
+**Source:** `toolRegistry.jl:54-82`
+
+---
+
+### Step 2: Load — `loadTools()`
+
+Load all tool modules from a directory into a `ToolStore`. Each `.jl` file must define `getTool()::agentTool`.
+
+```julia
+using YiemAgent, YiemAgent.toolRegistry
+
+store = ToolStore(name="myAgent")
+tools = loadTools(store, "src/tools")
+# Scans src/tools/ for .jl files, wraps each in a submodule, calls getTool(), registers in store.tools
+```
+
+**Result extraction:**
+```julia
+all_tools = getTools(store)  # OrderedDict{String, agentTool}
+# Keys: "getTime", "getWeather", "writeTool"
+getTime_tool = all_tools["getTime"]
+
+# Manual registration (alternative to loadTools)
+registerTool(store, my_tool)
+clearTools(store)  # Clear all tools from store
+```
+
+**Source:** `toolRegistry.jl:113-172`
+
+---
+
+### Step 3: Use — Tool Execution
+
+Tools can be used in two ways:
+
+**Direct execution (testing / standalone):**
+```julia
+using YiemAgent.type
+
+sig = nothing
+op = x -> x  # no-op partial result callback
+
+# Execute a loaded tool directly
+result = getTime_tool.execute("call-1", Dict("city" => "Tokyo"), sig, op)
+```
+
+**Via agent loop (production):**
+```
+user message → run_agent(agent, Dict("role"=>"user", "content"=>...))
+  → _agent_loop detects message → @spawn _process_message(agent)
+    → prepareContext → formatMsgForLLM → llmCall
+      → LLM returns tool_calls
+        → executeToolCalls(context, response, tool_call_list, config, signal, emit)
+          → prepareToolCall → executePreparedToolCall → finalizeExecutedToolCall
+            → createToolResultMessage → batch.messages (toolResultMessage[])
+```
+
+**Source:** Direct: `test/toolTest.jl:81-99` | Agent: `agentCore.jl:35-311`
+
+---
+
+### Step 4: Extract Result
+
+**`agentToolResult`** (raw tool output, `type.jl:429-434`):
+```julia
+result = getTime_tool.execute("call-1", Dict("city" => "Tokyo"), nothing, x->x)
+
+result.content[1]      # textContent("Current time in Tokyo: ...")
+result.content[1].text # "Current time in Tokyo: 2026-08-10T..."
+result.details         # Dict{Any,Any}() — tool-specific metadata
+result.usage           # nothing — llmUsage tracking (optional)
+result.terminate       # false — signals loop termination
+```
+
+**`toolResultMessage`** (wrapped for conversation history, `type.jl:152-191`):
+```julia
+msg = batch.messages[1]  # toolResultMessage
+
+msg.toolCallId   # "call-1"
+msg.toolName     # "getTime"
+msg.content      # Vector{messageContent}
+msg.isError      # false
+msg.details      # tool-specific metadata
+msg.timestamp    # DateTime
+```
+
+---
+
+## 2. Overview

 The tool system follows a **three-phase pipeline** per tool call:

@@ -45,7 +154,7 @@ The pipeline ensures that **every tool call produces a result**, even on failure

 ---

-## 2. Tool Definition — The `agentTool` Struct
+## 3. Tool Definition — The `agentTool` Struct

 **Source:** `type.jl:261-281`

@@ -106,11 +215,11 @@ struct agentToolResult
 end
 ```

-The `terminate` flag is checked at the batch level. See [Section 9](#9-tool-call-batches--termination-logic) for details.
+The `terminate` flag is checked at the batch level. See [Section 10](#10-tool-call-batches--termination-logic) for details.

 ---

-## 3. Tool Registration — Per-Agent Tool Stores
+## 4. Tool Registration — Per-Agent Tool Stores

 **Source:** `toolRegistry.jl`

@@ -133,7 +242,7 @@ end
 function loadTools(store::ToolStore, dir::String)::OrderedDict{String, agentTool}
 ```

-**Source:** `toolRegistry.jl:113-177`
+**Source:** `toolRegistry.jl:113-172`

 1. **Scans** `dir` for `.jl` files (excluding files matching `registry` in name)
 2. **Sorts** filenames alphabetically for deterministic registration order
@@ -205,7 +314,7 @@ This ensures that `yiemAgent` instances with different `tool_store` references o

 ---

-## 4. The Agent Loop — High-Level Flow
+## 5. The Agent Loop — High-Level Flow

 **Source:** `agentCore.jl:35-145`

@@ -274,7 +383,7 @@ end

 ---

-## 5. Message Processing Pipeline
+## 6. Message Processing Pipeline

 **Source:** `agentCore.jl:175-311`

@@ -356,7 +465,7 @@ There is a deliberate `error(5555555)` at `agentCore.jl:214` that halts executio

 ---

-## 6. Tool Call Extraction from LLM Response
+## 7. Tool Call Extraction from LLM Response

 **Source:** `agentCore.jl:217-245`

@@ -432,11 +541,11 @@ end

 ---

-## 7. The Per-Call Pipeline — Prepare, Execute, Finalize
+## 8. The Per-Call Pipeline — Prepare, Execute, Finalize

 This is the core of the tool execution system. Each tool call (whether part of a batch or standalone) goes through exactly three phases.

-### 7.1 Phase 1: Prepare — `prepareToolCall()`
+### 8.1 Phase 1: Prepare — `prepareToolCall()`

 **Source:** `agentCore.jl:511-547`

@@ -477,7 +586,7 @@ function prepareToolCall(

 **Key design principle:** Preparation **never throws**. Every failure path returns an `immediateOutcome` with `isError=true`, ensuring the agent loop always has a valid result to feed back to the LLM.

-### 7.2 Phase 2: Execute — `executePreparedToolCall()`
+### 8.2 Phase 2: Execute — `executePreparedToolCall()`

 **Source:** `agentCore.jl:589-617`

@@ -492,46 +601,46 @@ function executePreparedToolCall(
 **Steps:**

 1. **Initialize streaming state:**
-   ```julia
-   updateEvents = promise[]  # vector to collect update event handles
-   accepting = true          # guard to prevent duplicate emissions
-   ```
+    ```julia
+    updateEvents = promise[]  # vector to collect update event handles
+    accepting = true          # guard to prevent duplicate emissions
+    ```

 2. **Call `tool.execute()`:**
-   ```julia
-   result = prep.tool.execute(
-       prep.toolCall.id,
-       prep.args,
-       signal,
-       partialResult -> begin
-           if accepting
-               push!(updateEvents,
-                     emit(toolExecUpdateEvent(prep.toolCall.id, prep.toolCall.name,
-                                              prep.toolCall.arguments, partialResult)))
-           end
-       end
-   )
-   ```
+    ```julia
+    result = prep.tool.execute(
+        prep.toolCall.id,
+        prep.args,
+        signal,
+        partialResult -> begin
+            if accepting
+                push!(updateEvents,
+                      emit(toolExecUpdateEvent(prep.toolCall.id, prep.toolCall.name,
+                                               prep.toolCall.arguments, partialResult)))
+            end
+        end
+    )
+    ```

 3. **Wait for streaming to settle:**
-   ```julia
-   accepting = false
-   wait.(updateEvents)  # wait for all pending update event handlers
-   return executedOutcome(result, false)
-   ```
+    ```julia
+    accepting = false
+    wait.(updateEvents)  # wait for all pending update event handlers
+    return executedOutcome(result, false)
+    ```

 4. **On error:**
-   ```julia
-   catch err
-       accepting = false
-       wait.(updateEvents)
-       return executedOutcome(createErrorToolResult(sprint(showerror, err)), true)
-   end
-   ```
+    ```julia
+    catch err
+        accepting = false
+        wait.(updateEvents)
+        return executedOutcome(createErrorToolResult(sprint(showerror, err)), true)
+    end
+    ```

 **Streaming design:** The `accepting` guard prevents emitting updates after the call completes. If the tool's `execute` function yields after emitting updates but before returning, no duplicate or stale updates are emitted.

-### 7.3 Phase 3: Finalize — `finalizeExecutedToolCall()`
+### 8.3 Phase 3: Finalize — `finalizeExecutedToolCall()`

 **Source:** `agentCore.jl:675-706`

@@ -542,44 +651,44 @@ function finalizeExecutedToolCall(
    prep::preparedToolCall,
    executed::executedOutcome,
    config::agentLoopConfig,
-    signal::Union{Nothing, abortSignal},
+    signal::Union{Nothing,abortSignal},
 )::finalizedOutcome
 ```

 **Steps:**

 1. **Extract execution result:**
-   ```julia
-   result = executed.result
-   isError = executed.isError
-   ```
+    ```julia
+    result = executed.result
+    isError = executed.isError
+    ```

 2. **Run `afterToolCall` hook** — if `config.afterToolCall !== nothing`:
-   - Passes `afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context)` and `signal`
-   - Hook can mutate the result:
-     ```julia
-     after = config.afterToolCall(afterCtx(...))
-     if after !== nothing
-         result = merge(result, dict(
-             :content => get(after, :content, result.content),
-             :details => get(after, :details, result.details),
-             :usage => get(after, :usage, result.usage),
-             :terminate => get(after, :terminate, result.terminate)
-         ))
-         isError = get(after, :isError, isError)
-     end
-     ```
-   - Common use cases: mask sensitive data, normalize usage, flip `terminate` based on business logic
-   - On error: `result = createErrorToolResult(sprint(showerror, err)); isError = true`
+    - Passes `afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context)` and `signal`
+    - Hook can mutate the result:
+      ```julia
+      after = config.afterToolCall(afterCtx(...))
+      if after !== nothing
+          result = merge(result, dict(
+              :content => get(after, :content, result.content),
+              :details => get(after, :details, result.details),
+              :usage => get(after, :usage, result.usage),
+              :terminate => get(after, :terminate, result.terminate)
+          ))
+          isError = get(after, :isError, isError)
+      end
+      ```
+    - Common use cases: mask sensitive data, normalize usage, flip `terminate` based on business logic
+    - On error: `result = createErrorToolResult(sprint(showerror, err)); isError = true`

 3. **Return:**
-   ```julia
-   return finalizedOutcome(prep.toolCall, result, isError)
-   ```
+    ```julia
+    return finalizedOutcome(prep.toolCall, result, isError)
+    ```

 **Source:** `type.jl:787-791` — `finalizedOutcome` holds the original tool call reference, final result (post-hook), and error status.

-### 7.4 Emission — `emitToolExecutionEnd()`
+### 8.4 Emission — `emitToolExecutionEnd()`

 **Source:** `agentCore.jl:736-739`

@@ -594,7 +703,7 @@ This is called immediately after finalization, before building the `toolResultMe

 ---

-## 8. Execution Modes — Sequential vs Parallel
+## 9. Execution Modes — Sequential vs Parallel

 **Source:** `agentCore.jl:795-936, 988-1011`

@@ -721,7 +830,7 @@ end

 ---

-## 9. Tool Call Batches & Termination Logic
+## 10. Tool Call Batches & Termination Logic

 **Source:** `type.jl:793-838`

@@ -803,7 +912,7 @@ end

 ---

-## 10. Tool Result Message Creation
+## 11. Tool Result Message Creation

 **Source:** `agentCore.jl:373-379`

@@ -864,7 +973,7 @@ Dict(

 ---

-## 11. Error Handling & Recovery Pattern
+## 12. Error Handling & Recovery Pattern

 The framework uses a **result-based error handling** pattern instead of exceptions for tool call failures. This ensures the LLM always receives a tool result message, giving it the information to recover.

@@ -912,7 +1021,7 @@ Returns an `agentToolResult` with a single `textContent` block containing the er

 ---

-## 12. Event System — Tool Lifecycle Events
+## 13. Event System — Tool Lifecycle Events

 **Source:** `type.jl:480-516`

@@ -970,7 +1079,7 @@ The `agentEventSink` function is a user-provided callback that receives all even

 ---

-## 13. Agent Lifecycle Hooks
+## 14. Agent Lifecycle Hooks

 ### Hook Types

@@ -1101,7 +1210,7 @@ Override this to produce custom LLM message formats for different APIs/providers

 ---

-## 14. Self-Modifying Tools
+## 15. Self-Modifying Tools

 The framework supports tools that modify the tool system itself at runtime.

@@ -1110,7 +1219,6 @@ The framework supports tools that modify the tool system itself at runtime.
 **Source:** `tools/writeTool.jl`

 `writeTool` is a **file writer**, not a code generator. The LLM provides the tool logic as `executeCode` (Julia code body), and `writeTool` wraps it in Julia boilerplate:
-
 1. Converts `inputSchema` Dict into `Dict{String,Any}(...)` string literal
 2. Indents `executeCode` with 4 spaces
 3. Wraps it inside `function executeTool(...)::agentToolResult ... end`
@@ -1155,7 +1263,7 @@ Each `ToolStore` gets its own `listTool` instance bound to that store via `listT

 ---

-## 15. Complete End-to-End Example
+## 16. Complete End-to-End Example

 ### Full Lifecycle: User Message to Tool Result

@@ -1269,7 +1377,7 @@ AGENT LOOP: SEND RESPONSE TO USER

 ---

-## 16. Tool File Contract
+## 17. Tool File Contract

 Each `.jl` file in `src/tools/` must conform to the following contract:

@@ -1400,7 +1508,7 @@ All functions in the file are scoped under `_tool_myTool`, preventing name colli

 ---

-## Appendix: Type Reference
+## 18. Appendix: Type Reference

 ### Message Types