diff --git a/src/tools/README.md b/src/tools/README.md index 9e0c73c..bfe4569 100644 --- a/src/tools/README.md +++ b/src/tools/README.md @@ -1,211 +1,378 @@ -# Tools +# Tools — Complete End-to-End Lifecycle -Tools allow the agent to perform actions and fetch data. Each tool defines a **schema** (what arguments it accepts) and an **execution function** (what it does). +This document describes the complete tool lifecycle in the YiemAgent framework, from definition through execution, for framework authors and maintainers who need a thorough understanding of the architecture. -## Tool Anatomy +--- -Each tool has 3 main parts: +## Table of Contents -### 1. Schema (`inputSchema`) +1. [Overview](#1-overview) +2. [Tool Definition — The `agentTool` Struct](#2-tool-definition--the-agenttool-struct) +3. [Tool Registration — The Global Registry](#3-tool-registration--the-global-registry) +4. [The Agent Loop — High-Level Flow](#4-the-agent-loop--high-level-flow) +5. [Message Processing Pipeline](#5-message-processing-pipeline) +6. [Tool Call Extraction from LLM Response](#6-tool-call-extraction-from-llm-response) +7. [The Per-Call Pipeline — Prepare, Execute, Finalize](#7-the-per-call-pipeline--prepare-execute-finalize) +8. [Execution Modes — Sequential vs Parallel](#8-execution-modes--sequential-vs-parallel) +9. [Tool Call Batches & Termination Logic](#9-tool-call-batches--termination-logic) +10. [Tool Result Message Creation](#10-tool-result-message-creation) +11. [Error Handling & Recovery Pattern](#11-error-handling--recovery-pattern) +12. [Event System — Tool Lifecycle Events](#12-event-system--tool-lifecycle-events) +13. [Agent Lifecycle Hooks](#13-agent-lifecycle-hooks) +14. [Self-Modifying Tools](#14-self-modifying-tools) +15. [Complete End-to-End Example](#15-complete-end-to-end-example) +16. [Tool File Contract](#16-tool-file-contract) -JSON Schema (MCP format) describing the tool's arguments. The `"required"` array lists mandatory fields: +--- -```julia -inputSchema = Dict{String,Any}( - "type" => "object", - "properties" => Dict( - "city" => Dict("type" => "string", "description" => "City name"), - "units" => Dict("type" => "string", "enum" => ["celsius", "fahrenheit"], "default" => "celsius") - ), - "required" => ["city"] -) +## 1. Overview + +The tool system follows a **three-phase pipeline** per tool call: + +``` +PREPARE → EXECUTE → FINALIZE ``` -### 2. Execution Function (`execute`) +Each phase has a single responsibility and produces an intermediate result: -A function with the signature: +| Phase | Function | Input | Output | Purpose | +|-------|----------|-------|--------|---------| +| Prepare | `prepareToolCall()` | `agentContext`, `assistantMessage`, `agentToolCall`, `agentLoopConfig`, `abortSignal` | `preparedToolCall` or `immediateOutcome` | Resolve tool, validate args, run pre-hook | +| Execute | `executePreparedToolCall()` | `preparedToolCall`, `abortSignal`, `emit` | `executedOutcome` | Call `tool.execute()`, stream partial results | +| Finalize | `finalizeExecutedToolCall()` | `agentContext`, `assistantMessage`, `preparedToolCall`, `executedOutcome`, `agentLoopConfig`, `abortSignal` | `finalizedOutcome` | Run post-hook, emit end event | + +The pipeline ensures that **every tool call produces a result**, even on failure. Errors are captured as `immediateOutcome`, `executedOutcome`, or `finalizedOutcome` with `isError=true`, then converted to `toolResultMessage` objects that are fed back to the LLM conversation history. + +--- + +## 2. Tool Definition — The `agentTool` Struct + +**Source:** `type.jl:261-281` + +Every tool is an `agentTool` struct with the following fields: ```julia -execute(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult -``` - -- **`toolCallId`** — unique ID for this invocation (from the LLM's tool call) -- **`args`** — validated arguments provided by the LLM -- **`signal`** — abort signal for cancellable operations -- **`onPartialResult`** — callback for streaming progress updates -- **Returns** — `agentToolResult` with content, details, usage, and termination flag - -```julia -function executeTool(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult - # Optional: stream progress updates - onPartialResult(Dict("status" => "Fetching data...")) - - # Do work - result = "Weather in $(args["city"]): Sunny, 22°C" - - # Return result - return agentToolResult( - [textContent(result)], - Dict{Any,Any}(), # details - nothing, # usage - false # terminate (true to stop agent loop) - ) +struct agentTool + name::String # Unique identifier (e.g. "getWeather") + label::String # Human-readable name (e.g. "Weather Lookup") + description::String # What the tool does (shown to the LLM for selection) + inputSchema::Any # JSON Schema (MCP format) describing parameters + execute::Function # Core execution: (toolCallId, args, signal, onPartialResult) -> agentToolResult + prepareArguments::Union{Function, Nothing} # Optional: (args) -> modified_args (before validation) + validateRequiredArgs::Union{Function, Nothing} # Optional: (args) -> Union{Nothing, String} error + parallelToolExecute::Bool # Override: run in parallel with other tools end ``` -### 3. Tool Definition (`getTool()`) +### Field Details -Returns an `agentTool` struct: +| Field | Required | Signature | Purpose | +|-------|----------|-----------|---------| +| `name` | Yes | `String` | Unique key for tool lookup in `context.tools[name]` | +| `label` | Yes | `String` | Human-readable name for display | +| `description` | Yes | `String` | Shown to LLM for tool selection decisions | +| `inputSchema` | Yes | `Dict{String,Any}` | JSON Schema (MCP format) with `type`, `properties`, `required` | +| `execute` | Yes | `Function` | The actual tool logic (see signature below) | +| `prepareArguments` | No | `Function` | Transforms args before validation; `(args::Dict) -> Dict` | +| `validateRequiredArgs` | No | `Function` | Custom validation; `(args::Dict) -> Union{Nothing, String}` | +| `parallelToolExecute` | No | `Bool` | Default `false`. When `true`, allows parallel execution | -| Field | Type | Description | -|---|---|---| -| `name` | `String` | Unique identifier (e.g. `"getWeather"`) | -| `label` | `String` | Human-readable name (e.g. `"Weather Lookup"`) | -| `description` | `String` | What the tool does (shown to the LLM) | -| `inputSchema` | `Any` | JSON Schema (MCP format) | -| `execute` | `Function` | The execution function | -| `prepareArguments` | `Union{Function,Nothing}` | Optional argument transform before validation | -| `validateRequiredArgs` | `Union{Function,Nothing}` | Optional custom validation | -| `parallelToolExecute` | `Bool` | Run this tool in parallel with others | - -## Argument Validation - -Validation happens **before** tool execution, in the `prepareToolCall` phase. Invalid calls return an error immediately without invoking `execute`, `beforeToolCall`, or logging `toolExecutionStart`. - -### Default: JSON Schema Required Fields - -Set `validateRequiredArgs = nothing` to use the default validator, which checks that all fields in `inputSchema["required"]` are present: +### Execute Function Signature ```julia -# src/tools/getWeather.jl — uses default validation -function getTool()::agentTool - return agentTool( - name = "getWeather", - # ... - validateRequiredArgs = nothing, # uses default - ) +execute(toolCallId::String, + args::Dict{String,Any}, + signal::Union{Nothing,abortSignal}, + onPartialResult::Function)::agentToolResult +``` + +| Parameter | Description | +|-----------|-------------| +| `toolCallId` | Unique ID from the LLM's tool call (e.g., `"call_abc123"`) | +| `args` | Validated arguments from the LLM, already passed through `prepareArguments` and `validateRequiredArgs` | +| `signal` | Optional `abortSignal` for cancellable operations. Check `signal.aborted` to abort early. | +| `onPartialResult` | Callback for streaming progress: `onPartialResult(partial_data)` emits `toolExecUpdateEvent` | + +### Returns — `agentToolResult` + +**Source:** `type.jl:429-434` + +```julia +struct agentToolResult + content::Vector{messageContent} # Output content (textContent, etc.) + details::Dict{Any,Any} # Tool-specific metadata (e.g., counts, IDs) + usage::Union{llmUsage, Nothing} # Token usage tracking (optional) + terminate::Bool # If true, signals the tool requested loop termination end ``` -### Custom Validation Hook +The `terminate` flag is checked at the batch level. See [Section 9](#9-tool-call-batches--termination-logic) for details. -Override `validateRequiredArgs` when you need: -- **Cross-field constraints** (e.g. "at least one of X or Y") -- **Format validation** (e.g. regex patterns, date parsing) -- **Domain rules** (e.g. value ranges, business logic) +--- -The hook signature takes only `args`: +## 3. Tool Registration — The Global Registry + +**Source:** `tools/registry.jl` + +### How `loadTools()` Works ```julia -function validateRequiredArgs(args::Dict{String,Any})::Union{Nothing,String} - tz = get(args, "timezone", nothing) - city = get(args, "city", "") - - if !haskey(args, "timezone") && isempty(city) - return "Missing required argument: provide at least one of 'timezone' or 'city'" - end - - if tz !== nothing - tz_str = string(tz) - if !occursin(r"^[A-Za-z]+\/[A-Za-z]+(/[A-Za-z]+)*$", tz_str) - return "Invalid timezone format: '$tz_str'. Use IANA format, e.g. 'America/New_York'" +function loadTools(dir::String)::OrderedDict{String, agentTool} +``` + +**Source:** `tools/registry.jl:95-160` + +1. **Scans** `dir` for `.jl` files (excluding files matching `registry` in name) +2. **Sorts** filenames alphabetically for deterministic registration order +3. **Wraps** each file in a dynamically created submodule: + ```julia + # For "getWeather.jl" → module _tool_getWeather + module _tool_getWeather + using ..type + using Dates, UUIDs, DataStructures, JSON + # (file contents here) + end + ``` +4. **Evaluates** `getTool()` within the submodule scope using `Core.eval(mod, :(getTool()))` — this avoids world-age issues +5. **Validates** the return value is an `agentTool` instance +6. **Stores** the module reference in `_tool_modules` to prevent GC of closures +7. **Registers** the tool in `_registry` and returns an `OrderedDict{String, agentTool}` + +### Why Submodules? + +Each tool file is loaded into its own **namespaced submodule**. This means: +- `validateRequiredArgs`, `prepareArguments`, `executeTool`, and helper functions defined in `getTime.jl` are scoped under `_tool_getTime` +- No name collisions between tools — `getTime.validateRequiredArgs` is distinct from `getWeather.validateRequiredArgs` +- The module reference is kept alive in `_tool_modules` so closures (in `execute`, `validateRequiredArgs`, `prepareArguments`) don't get garbage collected + +### Auto-Registration + +The `_listTool()` is auto-registered in `__init__()` (line 16-18), so `listTools` is always available without explicit loading. + +### Registration API + +```julia +# Auto-load from directory +tools = loadTools("src/tools") # OrderedDict{String, agentTool} + +# Manual registration +registerTool(my_tool) # Adds to global _registry + +# Query +all_tools = getTools() # Vector{agentTool} (deep copy) + +# Clear +clearTools() # Empties _registry +``` + +--- + +## 4. The Agent Loop — High-Level Flow + +**Source:** `agentCore.jl:35-145` + +The `_agent_loop()` function runs as a background `@spawn` task, created when `yiemAgent` is constructed. + +### Channel Architecture + +``` +yiemAgent struct contains: + - inputChannel (Channel, capacity 16) ← user sends messages here via run_agent() + - followUpChannel (Channel, capacity 32) ← user sends follow-ups here via follow_up() + - outputChannel (Channel, capacity 16) ← agent sends responses here via take_response() +``` + +### Loop States + +The loop tracks 6 states (documented at `agentCore.jl:39-75`): + +| State | `processingTask` | `activeRun` | `inputChannel` | `followUpChannel` | Behavior | +|-------|-----------------|-------------|----------------|-------------------|----------| +| 1 | `nothing` | `false` | empty | empty | Idle, waiting | +| 2 | `nothing` | `false` | has msg | empty | New message → spawn `_process_message` | +| 3 | running | `true` | empty | empty | Processing, no new input | +| 4 | running | `true` | has msg | empty | New message while busy → queued | +| 5 | running | `true` | empty | has msg | Follow-up while busy → queued | +| 6 | done | `false` | empty | empty | Task completed → send result, reset | + +### Loop Logic (simplified) + +```julia +function _agent_loop(agent::yiemAgent) + while true + # 1. Wait for message from inputChannel (blocking poll) + msg = fetch!(agent.inputChannel) # agentCore.jl:84 + + # 2. Handle shutdown signal + if msg === :shutdown + drain both channels, break loop + end + + # 3. If agent is idle, spawn _process_message + if agent._state.activeRun == false + processingTask = Threads.@spawn _process_message(agent) + agent._state.activeRun = true + end + + # 4. While processing: check for followUp messages + if istaskdone(processingTask) == false && isready(agent.followUpChannel) + drain followUpChannel → push to inputChannel + continue # wait for current processing to finish + end + + # 5. When processing completes + if istaskdone(processingTask) == true + result = fetch(processingTask) + put!(agent.outputChannel, result) + agent._state.activeRun = false + processingTask = nothing end end - - return nothing end ``` -Return `nothing` to pass, or an error `String` to fail. The error is fed back to the LLM so it can retry with corrected arguments. +**Key design:** The loop always checks `inputChannel` before `followUpChannel`. Follow-up messages are merged into `inputChannel` only when the current processing task is active, ensuring they are processed after the primary message completes but before new input arrives. -## Tool Discovery and Lifecycle +--- -The agent iterates through tools via a **discover → execute → loop** cycle. Here is the complete flow from the framework author's perspective: +## 5. Message Processing Pipeline -### The Agent Loop +**Source:** `agentCore.jl:175-311` + +`_process_message(agent)` is the core function that processes a batch of user messages through the LLM pipeline. + +### Pipeline Steps ```julia -# agentCore.jl:175 - _process_message() -while true - # 1. Drain messages from inputChannel - while isready(agent.inputChannel) - raw_msg = take!(agent.inputChannel) - user_msg = OpenAiToUserMessage(raw_msg) - push!(agent._state.messages, user_msg) +function _process_message(agent::yiemAgent)::assistantMessage + final_response = nothing + + while true # Loop until LLM returns response without tool calls + # ── Step 1: Drain inputChannel ────────────────────────────── + while isready(agent.inputChannel) + raw_msg = take!(agent.inputChannel) + if raw_msg === :shutdown + put!(agent.inputChannel, :shutdown) + break + end + user_msg = OpenAiToUserMessage(raw_msg) # Convert Dict → userMessage + push!(agent._state.messages, user_msg) + end + + # ── Step 2: Prepare context ───────────────────────────────── + preparedContext = agent.prepareContext(agent._state) + # Default: deep copies systemPrompt, messages, tools from agentState → agentContext + # Override point: filter tools, inject context, modify system prompt + + # ── Step 3: Format for LLM ────────────────────────────────── + formatted_messages = agent.formatMsgForLLM(preparedContext) + # Converts agentContext → Dict("messages" => [...]) in OpenAI format + # Wraps systemPrompt as system role, converts each messageContent block + + # ── Step 4: Call LLM ──────────────────────────────────────── + response = agent.llmCall(formatted_messages) + # Returns assistantMessage with content::Vector{messageContent} + # Each content block has a type: "text", "thinking", or "tool_call" + + # ── Step 5: Extract tool calls ────────────────────────────── + has_tool_calls, tool_call_list = extract_tool_calls(response.content) + # Inspects content blocks for "tool_calls" or "tool_call" Dict entries + + # ── Step 6: Execute tool calls or return ──────────────────── + if has_tool_calls && !isempty(tool_call_list) + # Build context and config + context = agentContext(agent._state.systemPrompt, agent._state.messages, agent._state.tools) + config = agentLoopConfig(agent._state.tools, agent.beforeToolCall, agent.afterToolCall, ...) + signal = nothing + emit = agent.agentEventSink + + # Execute tool calls (sequential or parallel) + batch = executeToolCalls(context, response, tool_call_list, config, signal, emit) + + # Save results to conversation history + for tool_result in batch.messages + push!(agent._state.messages, tool_result) + end + + # Check termination + if batch.terminate + final_response = build_final_response(batch) + break + end + # Otherwise, loop back to Step 1 (drain any new input) and call LLM again + else + # No tool calls — this is the final response + final_response = response + break + end end - # 2. Format messages for LLM - ctx = agent.prepareContext(agent._state) - formatted = agent.formatMsgForLLM(ctx) - - # 3. Call LLM - response = agent.llmCall(formatted) - - # 4. Check if LLM used tool calls - if has_tool_calls(response.content) - # 5. Execute tools, feed results back to LLM, loop - else - # 6. No tool calls — return final response - break - end + return final_response end ``` -### Step 1: Tool Discovery +### Debug Note -Tools are discovered from `agent._state.tools`, which is a `Vector{agentTool}` populated during agent creation: +There is a deliberate `error(5555555)` at `agentCore.jl:214` that halts execution after the LLM call. This appears to be a debugging/staging marker. Remove or replace it before production use. + +--- + +## 6. Tool Call Extraction from LLM Response + +**Source:** `agentCore.jl:217-245` + +After the LLM call, the agent inspects `response.content` (a `Vector{messageContent}`) for tool call blocks. Two formats are supported: + +### Format 1: OpenAI `tool_calls` array ```julia -# Loading tools -tools = loadTools("src/tools") # returns Vector{agentTool} - -# Passing to agent -agent = yiemAgent( - systemPrompt = "...", - tools = tools, # ← tools stored in agent._state.tools - llmCall = my_llm_call, - agentEventSink = my_event_sink, +# Response content block: +Dict( + :type => "tool_calls", + :tool_calls => [ + Dict(:id => "call_1", :name => "getWeather", :arguments => Dict("city" => "Tokyo")), + Dict(:id => "call_2", :name => "getTime", :arguments => Dict("timezone" => "Asia/Tokyo")), + ] ) ``` -When the LLM response contains tool calls, the agent builds an `agentContext` with those tools: +### Format 2: Single `tool_call` block ```julia -context = agentContext( - agent._state.systemPrompt, - agent._state.messages, - agent._state.tools, # ← tools available for discovery +Dict( + :type => "tool_call", + :id => "call_1", + :name => "getWeather", + :arguments => Dict("city" => "Tokyo") ) ``` -### Step 2: Extract Tool Calls from LLM Response - -The agent inspects the `response.content` blocks for `tool_calls`: +### Extraction Logic ```julia -# agentCore.jl:217-245 tool_call_list = agentToolCall[] for content_block in response.content if content_block isa Dict + # OpenAI format: array of tool calls if get(content_block, :type, "") == "tool_calls" - # OpenAI format: {"type": "tool_calls", "tool_calls": [...]} for tc_data in get(content_block, :tool_calls, []) tc = agentToolCall( type = "function", - id = get(tc_data, :id, string(uuid4())), + id = get(tc_data, :id, string(uuid4())), # fallback UUID name = get(tc_data, :function, Dict())[:name], arguments = get(tc_data, :function, Dict())[:arguments], ) push!(tool_call_list, tc) end + # Alternative format: single tool call elseif get(content_block, :type, "") == "tool_call" - # Alternative format: single tool_call block + tc_data = content_block tc = agentToolCall( type = "function", - id = get(content_block, :id, string(uuid4())), - name = get(content_block, :name, ""), - arguments = get(content_block, :arguments, Dict()), + id = get(tc_data, :id, string(uuid4())), + name = get(tc_data, :name, ""), + arguments = get(tc_data, :arguments, Dict{String,Any}()), ) push!(tool_call_list, tc) end @@ -213,313 +380,1013 @@ for content_block in response.content end ``` -### Step 3: Execute Each Tool Call - -For each tool call, the agent runs through the **prepare → execute → finalize** pipeline: +The `agentToolCall` struct is defined at `type.jl:362-367`: ```julia -# agentCore.jl:247-302 -if has_tool_calls && length(tool_call_list) > 0 - context = agentContext(agent._state.systemPrompt, agent._state.messages, agent._state.tools) - config = agentLoopConfig(agent._state.tools, agent.beforeToolCall, agent.afterToolCall, execution_mode) - signal = nothing - emit = agent.agentEventSink - - # Execute all tool calls (sequential or parallel) - batch = executeToolCalls(context, response, tool_call_list, config, signal, emit) - - # Save results to conversation history - for tool_result in batch.messages - push!(agent._state.messages, tool_result) - end - - # If any tool requested termination, break the loop - if batch.terminate - final_response = build_final_response(batch) - break - end - # Otherwise, loop back to step 2 (format + call LLM again) +struct agentToolCall + type::String # Always "function" + id::String # Unique tool call identifier + name::String # Tool name (matches context.tools keys) + arguments::Dict{String, Any} # Parsed tool arguments end ``` -### Step 4: The Per-Call Pipeline +--- -Each tool call goes through three phases: +## 7. The Per-Call Pipeline — Prepare, Execute, Finalize -``` -┌─────────────────────────────────────────────────────────────────┐ -│ PREPARE → prepareToolCall() │ -│ │ -│ 1. Find tool by name in context.tools │ -│ 2. Transform args via tool.prepareArguments (if defined) │ -│ 3. Validate via tool.validateRequiredArgs (or default) │ -│ 4. Run beforeToolCall hook (if defined) │ -│ └── on any failure → return immediateOutcome (skip execution) │ -│ └── success → return preparedToolCall │ -├─────────────────────────────────────────────────────────────────┤ -│ EXECUTE → executePreparedToolCall() │ -│ │ -│ 1. emit toolExecutionStart event │ -│ 2. call tool.execute(toolCallId, args, signal, onPartialResult)│ -│ 3. wait for all pending update events │ -│ └── on error → return executedOutcome(isError=true) │ -│ └── success → return executedOutcome(isError=false) │ -├─────────────────────────────────────────────────────────────────┤ -│ FINALIZE → finalizeExecutedToolCall() │ -│ │ -│ 1. Run afterToolCall hook (if defined) │ -│ - can mutate content, details, usage, terminate, isError │ -│ 2. emit toolExecutionEnd event │ -│ 3. createToolResultMessage → adds to conversation history │ -│ └── return finalizedOutcome │ -└─────────────────────────────────────────────────────────────────┘ -``` +This is the core of the tool execution system. Each tool call (whether part of a batch or standalone) goes through exactly three phases. -### Step 5: Feed Results Back to LLM +### 7.1 Phase 1: Prepare — `prepareToolCall()` -Tool results are added to `agent._state.messages` as `toolResultMessage` objects. On the next loop iteration, `formatMsgForLLM()` converts them to OpenAI format and the LLM receives the results: - -``` -Conversation history after tool execution: - [system] "You are a helpful assistant." - [user] "What's the weather in Tokyo?" - [assistant] (tool_calls: getWeather(city="Tokyo")) - [tool] tool_call_id="call_1", tool_name="getWeather", content="Weather in Tokyo: Sunny, 22°C" -``` - -The LLM then decides: call another tool, or return a final text answer. - -## Execution Modes - -### Sequential - -Tools execute one at a time in order. Required when: -- Tools have implicit dependencies -- Tools share state (e.g. writing to the same file) -- Tools have `parallelToolExecute = false` - -Set globally via `agentLoopConfig.toolExecution = "sequential"`, or per-tool via `parallelToolExecute = false`. - -### Parallel - -Tools execute concurrently when all are independent. Reduces wall-clock time. Set `parallelToolExecute = true` on individual tools, or set `agentLoopConfig.toolExecution = "parallel"`. - -## Streaming Partial Results - -For long-running tools (API calls, file uploads, training), use `onPartialResult` to stream progress: +**Source:** `agentCore.jl:511-547` ```julia -function executeTool(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult - onPartialResult(Dict("status" => "Step 1: Fetching data...")) - sleep(1) - - onPartialResult(Dict("status" => "Step 2: Processing...")) - sleep(1) - - return agentToolResult( - [textContent("Done!")], - Dict{Any,Any}(), nothing, false +function prepareToolCall( + context::agentContext, + assistantMsg::assistantMessage, + toolCall::agentToolCall, + config::agentLoopConfig, + signal::Union{Nothing, abortSignal}, +)::Union{preparedToolCall, immediateOutcome} +``` + +**Steps:** + +1. **Resolve tool by name** — `get(context.tools, toolCall.name, nothing)` + - If `nothing` → `immediateOutcome(createErrorToolResult("Tool X not found"), true)` + +2. **Prepare arguments** — `prepareToolCallArguments(tool, toolCall)` + - Calls `tool.prepareArguments(toolCall.arguments)` if defined + - Returns the toolCall with transformed arguments + - If no hook or no change, returns original `toolCall` + +3. **Validate arguments** — `validateToolArguments(tool, prepared)` + - Calls `tool.validateRequiredArgs(prepared.arguments)` if defined, otherwise uses default `validateRequiredArgs(prepared.arguments, tool.inputSchema)` + - Default validator checks `inputSchema["required"]` array + - On failure: throws `ArgumentError(error_string)`, caught by the try-catch below + +4. **Run `beforeToolCall` hook** — if `config.beforeToolCall !== nothing` + - Passes `assistantMsgCtx(assistantMsg, toolCall, validatedArgs, context)` and `signal` + - Hook can return `nothing` (proceed), or `Dict(:block => true, :reason => "...")` (reject) + - If `signal.aborted == true` → `immediateOutcome(createErrorToolResult("Operation aborted"), true)` + - If `before.block == true` → `immediateOutcome(createErrorToolResult(get(before, :reason, "blocked")), true)` + +5. **Return success** → `preparedToolCall(tool, toolCall, validatedArgs)` + +**Source:** `type.jl:681-685` — `preparedToolCall` holds the resolved tool, original call metadata, and validated arguments together. + +**Key design principle:** Preparation **never throws**. Every failure path returns an `immediateOutcome` with `isError=true`, ensuring the agent loop always has a valid result to feed back to the LLM. + +### 7.2 Phase 2: Execute — `executePreparedToolCall()` + +**Source:** `agentCore.jl:589-617` + +```julia +function executePreparedToolCall( + prep::preparedToolCall, + signal::Union{Nothing, abortSignal}, + emit::Function, +)::executedOutcome +``` + +**Steps:** + +1. **Initialize streaming state:** + ```julia + updateEvents = promise[] # vector to collect update event handles + accepting = true # guard to prevent duplicate emissions + ``` + +2. **Call `tool.execute()`:** + ```julia + result = prep.tool.execute( + prep.toolCall.id, + prep.args, + signal, + partialResult -> begin + if accepting + push!(updateEvents, + emit(toolExecUpdateEvent(prep.toolCall.id, prep.toolCall.name, + prep.toolCall.arguments, partialResult))) + end + end + ) + ``` + +3. **Wait for streaming to settle:** + ```julia + accepting = false + wait.(updateEvents) # wait for all pending update event handlers + return executedOutcome(result, false) + ``` + +4. **On error:** + ```julia + catch err + accepting = false + wait.(updateEvents) + return executedOutcome(createErrorToolResult(sprint(showerror, err)), true) + end + ``` + +**Streaming design:** The `accepting` guard prevents emitting updates after the call completes. If the tool's `execute` function yields after emitting updates but before returning, no duplicate or stale updates are emitted. + +### 7.3 Phase 3: Finalize — `finalizeExecutedToolCall()` + +**Source:** `agentCore.jl:675-706` + +```julia +function finalizeExecutedToolCall( + context::agentContext, + assistantMsg::assistantMessage, + prep::preparedToolCall, + executed::executedOutcome, + config::agentLoopConfig, + signal::Union{Nothing, abortSignal}, +)::finalizedOutcome +``` + +**Steps:** + +1. **Extract execution result:** + ```julia + result = executed.result + isError = executed.isError + ``` + +2. **Run `afterToolCall` hook** — if `config.afterToolCall !== nothing`: + - Passes `afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context)` and `signal` + - Hook can mutate the result: + ```julia + after = config.afterToolCall(afterCtx(...)) + if after !== nothing + result = merge(result, dict( + :content => get(after, :content, result.content), + :details => get(after, :details, result.details), + :usage => get(after, :usage, result.usage), + :terminate => get(after, :terminate, result.terminate) + )) + isError = get(after, :isError, isError) + end + ``` + - Common use cases: mask sensitive data, normalize usage, flip `terminate` based on business logic + - On error: `result = createErrorToolResult(sprint(showerror, err)); isError = true` + +3. **Return:** + ```julia + return finalizedOutcome(prep.toolCall, result, isError) + ``` + +**Source:** `type.jl:787-791` — `finalizedOutcome` holds the original tool call reference, final result (post-hook), and error status. + +### 7.4 Emission — `emitToolExecutionEnd()` + +**Source:** `agentCore.jl:736-739` + +```julia +function emitToolExecutionEnd(finalized::finalizedOutcome, emit::Function) + emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name, + finalized.result, finalized.isError)) +end +``` + +This is called immediately after finalization, before building the `toolResultMessage`. + +--- + +## 8. Execution Modes — Sequential vs Parallel + +**Source:** `agentCore.jl:795-936, 988-1011` + +### Dispatcher — `executeToolCalls()` + +```julia +function executeToolCalls( + context::agentContext, + assistantMsg::assistantMessage, + toolCalls::Vector{agentToolCall}, + config::agentLoopConfig, + signal::Union{Nothing, abortSignal}, + emit::Function, +)::agentToolCallBatch +``` + +**Decision logic** (`agentCore.jl:997-1010`): + +```julia +hasSequential = false +for tc in toolCalls + t = get(context.tools, tc.name, nothing) + if t !== nothing && !t.parallelToolExecute + hasSequential = true + break + end +end + +if config.toolExecution == "sequential" || hasSequential + return executeToolCallsSequential(...) +else + return executeToolCallsParallel(...) +end +``` + +**Rule:** If the global config is `"sequential"` **OR** any tool in the batch has `parallelToolExecute = false`, the entire batch runs sequentially. Sequential is the safe default. + +### Sequential Execution — `executeToolCallsSequential()` + +**Source:** `agentCore.jl:795-829` + +```julia +function executeToolCallsSequential(...)::agentToolCallBatch + finalizedCalls = finalizedOutcome[] + messages = toolResultMessage[] + + for tc in toolCalls + emit(toolExecStartEvent(tc.id, tc.name, tc.arguments)) + + prep = prepareToolCall(context, assistantMsg, tc, config, signal) + + if prep isa immediateOutcome + finalized = finalizedOutcome(tc, prep.result, prep.isError) + else + executed = executePreparedToolCall(prep, signal, emit) + finalized = finalizeExecutedToolCall(context, assistantMsg, prep, executed, config, signal) + end + + emitToolExecutionEnd(finalized, emit) + push!(messages, createToolResultMessage(finalized)) + push!(finalizedCalls, finalized) + + if signal !== nothing && signal.aborted + break # abort: skip remaining calls + end + end + + return agentToolCallBatch(messages, shouldTerminate(finalizedCalls)) +end +``` + +### Parallel Execution — `executeToolCallsParallel()` + +**Source:** `agentCore.jl:888-936` + +```julia +function executeToolCallsParallel(...)::agentToolCallBatch + entries = union{finalizedOutcome, task{finalizedOutcome}}[] + + for tc in toolCalls + emit(toolExecStartEvent(tc.id, tc.name, tc.arguments)) + + prep = prepareToolCall(context, assistantMsg, tc, config, signal) + + if prep isa immediateOutcome + finalized = finalizedOutcome(tc, prep.result, prep.isError) + emitToolExecutionEnd(finalized, emit) + push!(entries, finalized) # immediate outcome — no task + else + task = task() do + executed = executePreparedToolCall(prep, signal, emit) + finalized = finalizeExecutedToolCall(context, assistantMsg, prep, executed, config, signal) + emitToolExecutionEnd(finalized, emit) + return finalized + end + schedule(task) + push!(entries, task) # pending task + end + + if signal !== nothing && signal.aborted + break # abort during preparation — skip remaining + end + end + + # Collect results in original order + finalizedCalls = finalizedOutcome[] + for entry in entries + outcome = entry isa task ? fetch(entry) : entry + push!(finalizedCalls, outcome) + end + + messages = toolResultMessage[] + for f in finalizedCalls + push!(messages, createToolResultMessage(f)) + end + + return agentToolCallBatch(messages, shouldTerminate(finalizedCalls)) +end +``` + +**Key design:** All tool calls are **prepared** sequentially (validation, hooks), then prepared calls are **executed** concurrently as separate tasks. Results are collected in the original call order via `fetch()`. + +**Trade-off:** Parallel execution reduces wall-clock time for independent tools but can overwhelm external resources (rate limits, connection pools, disk I/O). + +--- + +## 9. Tool Call Batches & Termination Logic + +**Source:** `type.jl:793-838` + +### `agentToolCallBatch` + +```julia +struct agentToolCallBatch + messages::Vector{toolResultMessage} # Tool result messages for this batch + terminate::Bool # Whether the batch should terminate the loop +end +``` + +### Termination Logic — `shouldTerminate()` + +**Source:** `agentCore.jl:409-411` + +```julia +function shouldTerminate(batches::Vector{finalizedOutcome})::Bool + return !isempty(batches) && all(b -> b.result.terminate, batches) +end +``` + +**Rule:** `terminate` is `true` **only when every tool in the batch** has `result.terminate == true`. This prevents a single tool that sets `terminate: true` (e.g., for metadata purposes) from accidentally stopping the agent when other tools did not intend to terminate. + +### When to Set `terminate = true` + +From the type documentation (`type.jl:803-815`): + +| Use Case | Description | +|----------|-------------| +| Task completion | A tool like `deploy` or `submit` finishes its work and signals the agent to stop | +| Unrecoverable error | A tool hits a fatal condition (auth token expired, database connection lost) | +| Async handoff | A tool triggers a long-running external operation; the external system will later resume via `continue()` | + +### Batch Processing in `_process_message()` + +**Source:** `agentCore.jl:266-307` + +```julia +batch = executeToolCalls(context, response, tool_call_list, config, signal, emit) + +# Save results to conversation history +for tool_result in batch.messages + push!(agent._state.messages, tool_result) +end + +if batch.terminate + # Build final response from tool results + final_content = [textContent("Tool execution completed.")] + for tool_result in batch.messages + for content_block in tool_result.content + if content_block isa textContent + append!(final_content, [content_block]) + elseif content_block isa Dict + if haskey(content_block, :text) + push!(final_content, textContent(content_block[:text])) + end + end + end + end + final_response = assistantMessage( + role = "assistant", + content = final_content, + api = response.api, + model = response.model, + usage = response.usage, + stopReason = "tool_use_terminated", + errorMessage = if any(x -> x.isError, batch.messages) + "One or more tool calls failed" + else + nothing + end, + timestamp = now(), + ) + break # exit the while loop +end +# If batch.terminate == false, loop back to call LLM again with tool results +``` + +--- + +## 10. Tool Result Message Creation + +**Source:** `agentCore.jl:373-379` + +### `createToolResultMessage()` + +```julia +function createToolResultMessage(f::finalizedOutcome)::toolResultMessage + return toolResultMessage( + "toolResult", # role + f.toolCall.id, # toolCallId + f.toolCall.name, # toolName + f.result.content, # content (Vector{messageContent}) + f.result.details, # details + f.result.usage, # usage + get(f.result, :addedToolNames, string[]), # addedToolNames (for dynamic tools) + f.isError, # isError + nowMillis(), # timestamp ) end ``` -UI listeners and the TUI consume these events in real time via `toolExecutionUpdate`. +### `toolResultMessage` Struct -## Loading Tools - -### Auto-load from Directory +**Source:** `type.jl:152-191` ```julia -using .toolRegistry - -tools = loadTools("src/tools") # scans for *.jl files with getTool() +struct toolResultMessage <: agentMessage + role::String # Always "tool" + toolCallId::String # ID matching the original tool call + toolName::String # Name of the executed tool + content::Vector{messageContent} # Tool output content + details::Any # Additional tool-specific details + usage::Union{llmUsage, Nothing} # Token usage if applicable + addedToolNames::Union{Vector{String}, Nothing} # Tools added during execution + isError::Bool # Whether the tool call resulted in an error + timestamp::Timestamp # When the result was recorded +end ``` -Files are loaded alphabetically for deterministic registration order. - -### Manual Registration - -```julia -tool = getTool() # from your tool module -registerTool(tool) -``` - -## Complete Lifecycle Example - -```julia -# ─── USER SENDS MESSAGE ─────────────────────────────────────────── -run_agent(agent, "What's the weather in Tokyo?") - -# ─── LOOP ITERATION 1 ───────────────────────────────────────────── -# Agent formats messages and calls LLM -formatted = agent.formatMsgForLLM(agent.prepareContext(agent._state)) -response = agent.llmCall(formatted) -# LLM returns: {"content": [{"type": "tool_calls", "tool_calls": [{"name": "getWeather", "arguments": {"city": "Tokyo"}}]}]} - -# Agent extracts tool call, builds context -context = agentContext(systemPrompt, messages, agent._state.tools) -tool_call_list = [agentToolCall("call_1", "getWeather", Dict("city" => "Tokyo"))] - -# PREPARE: find tool, validate args -tool = find(t -> t.name == "getWeather", context.tools) # found! -validateRequiredArgs(Dict("city" => "Tokyo"), tool.inputSchema) # passes -beforeToolCall_hook(agentMsgCtx, nothing) # nil, skipped - -# EXECUTE: call tool.execute() -result = tool.execute("call_1", Dict("city" => "Tokyo"), nothing, onPartialResult) -# Returns: agentToolResult([textContent("Weather in Tokyo: Sunny, 22°C")], Dict(), nothing, false) - -# FINALIZE: afterToolCall hook, emit events -finalized = finalizedOutcome(tc, result, false) -emit(toolExecEndEvent("call_1", "getWeather", result, false)) -msg = createToolResultMessage(finalized) # toolResultMessage for conversation history - -# Add result to conversation -push!(agent._state.messages, msg) -# Messages now: [user: "What's the weather?", assistant: {tool_calls: getWeather}, tool: "Sunny, 22°C"] - -# ─── LOOP ITERATION 2 ───────────────────────────────────────────── -# LLM called again with tool result included -formatted = agent.formatMsgForLLM(agent.prepareContext(agent._state)) -response = agent.llmCall(formatted) -# LLM returns: {"content": [{"type": "text", "text": "The weather in Tokyo is sunny, 22°C."}]} - -# No tool calls detected → break loop, return final response -return assistantMessage(content=[textContent("The weather in Tokyo is sunny, 22°C.")], ...) - -# ─── USER RECEIVES RESPONSE ─────────────────────────────────────── -response = take_response(agent) -println(response.content) -# => "[textContent(\"The weather in Tokyo is sunny, 22°C.\")]" -``` - -## Self-Modifying Tools - -The framework includes tools that allow the agent to create new tools at runtime. - -### `writeTool` — Create New Tool Files - -`writeTool` is a **file writer**, not a code generator. The LLM provides the tool logic as `executeCode` (the actual Julia code), and `writeTool` wraps it in the required boilerplate. - -**How it works:** - -The LLM constructs `writeTool` with: -- **`executeCode`** — the actual tool logic (Julia code body, NOT wrapped in a function) -- **`name`, `label`, `description`** — tool metadata -- **`inputSchema`** — parameter schema in MCP format -- **`validateCode`, `prepareCode`** (optional) — custom validation/preparation logic - -`writeTool` produces `src/tools/.jl` by: -1. Converting the `inputSchema` Dict into a Julia `Dict{String,Any}(...)` string literal -2. Indenting `executeCode` with 4 spaces -3. Wrapping it inside a `function executeTool(...)::agentToolResult ... end` template -4. Appending the `getTool()` definition that returns an `agentTool` struct -5. Writing the combined string to disk - -**Workflow:** +### Conversation History After Tool Execution ``` -LLM decides: "Need a searchWine tool. I'll provide the logic." - -LLM calls writeTool: - name: "searchWine" - executeCode: "query = args[\"query\"]\nresult = search(query)\nreturn ..." - -writeTool wraps it → src/tools/searchWine.jl: - function executeTool(...)::agentToolResult - query = args["query"] ← LLM code (indented 4 spaces) - result = search(query) - return agentToolResult(...) - end - - function getTool()::agentTool - return agentTool(name="searchWine", ...) - end - -Restart → loadTools("src/tools") loads searchWine.jl +[system] "You are a helpful assistant." +[user] "What's the weather in Tokyo?" +[assistant] {tool_calls: [getWeather(city="Tokyo")]} +[tool] tool_call_id="call_1", tool_name="getWeather", content="Weather in Tokyo: Sunny, 22°C" ``` -**Example specification:** +On the next loop iteration, `formatMsgForLLM()` converts `toolResultMessage` to OpenAI format: ```julia Dict( - "name" => "searchWine", - "label" => "Wine Search", - "description" => "Search a wine database by name, region, or variety", - "inputSchema" => Dict( - "type" => "object", - "properties" => Dict( - "query" => Dict("type" => "string", "description" => "Search query"), - "maxResults" => Dict("type" => "integer", "default" => 10) - ), - "required" => ["query"] - ), - "executeCode" => """ - query = args["query"] - max_results = get(args, "maxResults", 10) - # Perform search logic here - result = "Found 3 wines matching: $query" - return agentToolResult([textContent(result)], Dict{Any,Any}(), nothing, false) - """, - "parallel" => false + "role" => "tool", + "tool_call_id" => "call_1", + "content" => [Dict("type" => "text", "text" => "Weather in Tokyo: Sunny, 22°C")] ) ``` -**Optional hooks:** +--- -| Field | Description | -|---|---| -| `validateCode` | Custom validation Julia code (runs before execute). Return `nothing` to pass, or an error `String` to fail. | -| `prepareCode` | Argument preparation code (runs before validation). Return modified args dict. | +## 11. Error Handling & Recovery Pattern + +The framework uses a **result-based error handling** pattern instead of exceptions for tool call failures. This ensures the LLM always receives a tool result message, giving it the information to recover. + +### Error Flow + +``` +Tool call fails at any phase + │ + ▼ +┌─────────────────────────┐ +│ Phase: Prepare │ → immediateOutcome(error_result, true) +│ Phase: Execute │ → executedOutcome(error_result, true) +│ Phase: Finalize (hook) │ → finalizedOutcome(tc, error_result, true) +└─────────────────────────┘ + │ + ▼ +emitToolExecutionEnd(finalized, emit) + │ + ▼ +createToolResultMessage(finalized) + │ + ▼ +push!(agent._state.messages, toolResultMessage) + │ + ▼ +formatMsgForLLM() → LLM receives error as tool result + │ + ▼ +LLM can: retry with corrected args, report failure, or ask user for clarification +``` + +### `createErrorToolResult()` + +**Source:** `agentCore.jl:339-341` + +```julia +function createErrorToolResult(msg::String)::agentToolResult + return agentToolResult([textContent("text", msg)], Dict{Any,Any}()) +end +``` + +Returns an `agentToolResult` with a single `textContent` block containing the error message. This is wrapped in an `immediateOutcome`, `executedOutcome`, or `finalizedOutcome` depending on where the error occurred, then converted to `toolResultMessage` for the LLM. + +**Key design:** Returning a result instead of throwing allows the LLM to see the error and decide whether to retry, re-issue the call with different arguments, or report failure to the user. + +--- + +## 12. Event System — Tool Lifecycle Events + +**Source:** `type.jl:480-516` + +Each tool call emits a three-event lifecycle: + +``` +toolExecStartEvent → [zero or more toolExecUpdateEvent] → toolExecEndEvent +``` + +### Event Types + +```julia +struct toolExecStartEvent + toolCallId::String # ID of the tool call + toolName::String # Name of the tool + arguments::Dict{String, Any} # Tool arguments +end + +struct toolExecUpdateEvent + toolCallId::String # ID of the tool call + toolName::String # Name of the tool + arguments::Dict{String, Any} # Tool arguments + partialResult::Any # The partial result data +end + +struct toolExecEndEvent + toolCallId::String # ID of the tool call + toolName::String # Name of the tool + result::agentToolResult # The final tool result + isError::Bool # Whether execution resulted in an error +end +``` + +### Event Emission Points + +| Event | Emitted From | When | +|-------|-------------|------| +| `toolExecStartEvent` | `executeToolCalls*()` loop | Before `prepareToolCall()` for each tool call | +| `toolExecUpdateEvent` | `executePreparedToolCall()` | Inside `onPartialResult` callback during `tool.execute()` | +| `toolExecEndEvent` | `emitToolExecutionEnd()` | After `finalizeExecutedToolCall()` for each tool call | + +### Event Sink + +The `emit` function is passed through the entire call chain: + +```julia +emit = agent.agentEventSink # set during yiemAgent construction +``` + +The `agentEventSink` function is a user-provided callback that receives all events. This is typically used by: +- **TUI (Terminal UI):** Display real-time progress, tool names, results +- **Logging systems:** Record tool execution history +- **Monitoring:** Track tool usage, execution times, error rates +- **Audit trails:** Log all tool calls with arguments and results + +--- + +## 13. Agent Lifecycle Hooks + +### Hook Types + +| Hook | Signature | Called | Purpose | +|------|-----------|--------|---------| +| `prepareContext` | `(state::agentState) -> agentContext` | Before each LLM call | Filter tools, inject context, modify system prompt | +| `formatMsgForLLM` | `(ctx::agentContext) -> Dict` | After `prepareContext` | Convert to LLM-specific format | +| `llmCall` | `(messages::Dict) -> assistantMessage` | After formatting | Actually invoke the LLM API | +| `beforeToolCall` | `(msgCtx::assistantMsgCtx, signal) -> Union{Nothing, Dict}` | In `prepareToolCall` | Ask for user permission, block execution, abort | +| `afterToolCall` | `(afterCtx::afterCtx, signal) -> Union{Nothing, Dict}` | In `finalizeExecutedToolCall` | Mutate result, mask data, flip `terminate` | +| `agentEventSink` | `(event) -> nothing` | Throughout lifecycle | Emit events for TUI, logging, monitoring | + +### `beforeToolCall` Hook + +**Source:** `agentCore.jl:530-541` + +```julia +if config.beforeToolCall !== nothing + before = config.beforeToolCall( + assistantMsgCtx(assistantMsg, toolCall, validatedArgs, context), signal + ) + if signal !== nothing && signal.aborted + return immediateOutcome(createErrorToolResult("Operation aborted"), true) + end + if before !== nothing && before.block + return immediateOutcome( + createErrorToolResult(get(before, :reason, "Tool execution was blocked")), true) + end +end +``` + +**Return values:** +- `nothing` — proceed with execution +- `Dict(:block => true, :reason => "...")` — block execution, error fed back to LLM +- `Dict(:block => false)` — proceed (explicit allow) + +### `afterToolCall` Hook + +**Source:** `agentCore.jl:687-703` + +```julia +if config.afterToolCall !== nothing + try + after = config.afterToolCall( + afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal + ) + if after !== nothing + result = merge(result, dict( + :content => get(after, :content, result.content), + :details => get(after, :details, result.details), + :usage => get(after, :usage, result.usage), + :terminate => get(after, :terminate, result.terminate) + )) + isError = get(after, :isError, isError) + end + catch err + result = createErrorToolResult(sprint(showerror, err)) + isError = true + end +end +``` + +**Common use cases:** +- Mask sensitive data from result content before the LLM sees it (e.g., removing API keys from error messages) +- Normalize usage tracking data into a consistent format +- Inspect the result and decide to flip `terminate: true` based on business logic +- Wrap an error result in a friendlier message for the LLM to understand + +### `prepareContext` Hook + +**Source:** `utils.jl:111-125` + +```julia +function prepareContext(state::agentState)::agentContext + # TODO: filter tools from state.tools based on user intent + filteredTools = state.tools + + # TODO: add filtered tools to the current system prompt / modify systemPrompt + preparedSystemPrompt = state.systemPrompt + + # TODO: add system prompt, adjust/modify and inject additional context into messages + preparedMessages = deepcopy(state.messages) + + return agentContext(preparedSystemPrompt, preparedMessages, filteredTools) +end +``` + +**Override points for customization:** +- Filter tools based on user intent (e.g., only show "wine" tools when user asks about wine) +- Modify the system prompt dynamically (e.g., inject current time, user preferences) +- Inject additional context (e.g., retrieved documents, current user state) +- Prune or reorder messages before formatting for the LLM + +### `formatMsgForLLM` Hook + +**Source:** `utils.jl:158-219` + +Default implementation converts `agentContext` to OpenAI-compatible format: + +```julia +function formatMsgForLLM(ctx::agentContext)::Dict{String, Any} + messages = Vector{Dict{String, Any}}() + + # System prompt as system message + if !isempty(ctx.systemPrompt) + push!(messages, Dict( + "role" => "system", + "content" => [Dict("type" => "text", "text" => ctx.systemPrompt)] + )) + end + + # Conversation messages + for msg in ctx.messages + if msg isa userMessage + push!(messages, _userMessageToOpenAI(msg)) + elseif msg isa assistantMessage + push!(messages, _assistantMessageToOpenAI(msg)) + elseif msg isa toolResultMessage + push!(messages, _toolResultMessageToOpenAI(msg)) + end + end + + return Dict("messages" => messages) +end +``` + +Override this to produce custom LLM message formats for different APIs/providers (e.g., Anthropic, Google, Ollama). + +--- + +## 14. Self-Modifying Tools + +The framework supports tools that modify the tool system itself at runtime. + +### `writeTool` — Create New Tool Files + +**Source:** `tools/writeTool.jl` + +`writeTool` is a **file writer**, not a code generator. The LLM provides the tool logic as `executeCode` (Julia code body), and `writeTool` wraps it in Julia boilerplate: + +1. Converts `inputSchema` Dict into `Dict{String,Any}(...)` string literal +2. Indents `executeCode` with 4 spaces +3. Wraps it inside `function executeTool(...)::agentToolResult ... end` +4. Appends `getTool()` returning an `agentTool` struct +5. Writes the combined string to `src/tools/.jl` ### `listTools` — Discover Available Tools -Returns all registered tools. Primarily useful for **collision detection** before creating a new tool via `writeTool` — the LLM checks existing names before picking a unique one. +**Source:** `tools/registry.jl:24-52` + +Returns all registered tools. Primarily useful for **collision detection** before creating a new tool via `writeTool`. + +### Self-Tooling Workflow + +``` +1. Agent detects no existing tool handles the user's request +2. Agent calls writeTool with: + - name: "searchWine" + - label: "Wine Search" + - description: "Search a wine database..." + - inputSchema: { ... } + - executeCode: "query = args[\"query\"]\nresult = search(query)\n..." + - (optional) validateCode, prepareCode +3. writeTool generates src/tools/searchWine.jl +4. Agent restarts (or hot-reloads) → loadTools("src/tools") picks up the new file +5. Agent calls searchWine(query="cabernet") +6. Result: "Found 5 cabernet wines..." +``` + +### `writeTool` Input Schema + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | `String` | Yes | Valid Julia identifier (letters, digits, underscores) | +| `label` | `String` | Yes | Human-readable tool name | +| `description` | `String` | Yes | What the tool does | +| `inputSchema` | `Dict` | Yes | JSON Schema in MCP format | +| `executeCode` | `String` | Yes | Julia code for `executeTool` body (NOT wrapped in function) | +| `validateCode` | `String` | No | Custom validation Julia code | +| `prepareCode` | `String` | No | Argument preparation code | +| `parallel` | `Bool` | No | Whether the tool can run in parallel (default: `false`) | + +--- + +## 15. Complete End-to-End Example + +### Full Lifecycle: User Message to Tool Result + +``` +USER SENDS MESSAGE + └─> run_agent(agent, "What's the weather in Tokyo?") + └─> put!(agent.inputChannel, Dict("role" => "user", "content" => [...])) + + +LOOP ITERATION 1 — LLM DECIDES TO USE A TOOL + └─> _agent_loop: detects msg in inputChannel + └─> Threads.@spawn _process_message(agent) + + ── _process_message ────────────────────────────────────────────── + │ + │ Step 1: Drain inputChannel + │ raw_msg = Dict("role" => "user", "content" => [...]) + │ user_msg = OpenAiToUserMessage(raw_msg) + │ push!(agent._state.messages, user_msg) + │ + │ Step 2: prepareContext + │ ctx = agent.prepareContext(agent._state) + │ → agentContext(systemPrompt, messages, tools) + │ + │ Step 3: formatMsgForLLM + │ formatted = agent.formatMsgForLLM(ctx) + │ → Dict("messages" => [ + │ Dict("role" => "system", "content" => [...]), + │ Dict("role" => "user", "content" => [...]), + │ ]) + │ + │ Step 4: llmCall + │ response = agent.llmCall(formatted) + │ → assistantMessage(content = [ + │ Dict(:type => "tool_calls", :tool_calls => [ + │ Dict(:id => "call_1", :name => "getWeather", + │ :arguments => Dict("city" => "Tokyo")) + │ ]) + │ ]) + │ + │ Step 5: Extract tool calls + │ tool_call_list = [agentToolCall("function", "call_1", "getWeather", ...)] + │ + │ Step 6: Execute tool calls + │ context = agentContext(systemPrompt, messages, tools) + │ config = agentLoopConfig(tools, beforeToolCall, afterToolCall, "sequential") + │ batch = executeToolCalls(context, response, tool_call_list, config, nothing, emit) + │ + │ ── executeToolCallsSequential ────────────────────────────── + │ │ + │ │ For tc = agentToolCall("call_1", "getWeather", ...): + │ │ + │ │ emit(toolExecStartEvent("call_1", "getWeather", {"city": "Tokyo"})) + │ │ + │ │ PREPARE: + │ │ tool = context.tools["getWeather"] → found! + │ │ validatedArgs = validateToolArguments(tool, tc) + │ │ → validateRequiredArgs(Dict("city" => "Tokyo"), inputSchema) → passes + │ │ beforeToolCall_hook(...) → nothing (skipped) + │ │ → preparedToolCall(tool, tc, {"city" => "Tokyo"}) + │ │ + │ │ EXECUTE: + │ │ result = tool.execute("call_1", {"city" => "Tokyo"}, nothing, onPartialResult) + │ │ → agentToolResult([textContent("Weather in Tokyo: Sunny, 22°C")], {}, nothing, false) + │ │ → executedOutcome(result, false) + │ │ + │ │ FINALIZE: + │ │ afterToolCall_hook(...) → nothing (skipped) + │ │ → finalizedOutcome(tc, result, false) + │ │ + │ │ emit(toolExecEndEvent("call_1", "getWeather", result, false)) + │ │ msg = createToolResultMessage(finalized) + │ │ → toolResultMessage("tool", "call_1", "getWeather", [...], {}, nothing, [], false, ts) + │ │ + │ └─> agentToolCallBatch([msg], false) + │ + │ Save results: + │ for tool_result in batch.messages + │ push!(agent._state.messages, tool_result) + │ end + │ + │ batch.terminate == false → loop back to Step 1 + │ + + +LOOP ITERATION 2 — LLM RETURNS FINAL TEXT RESPONSE + ── _process_message (second iteration) ─────────────────────────── + │ + │ Step 1: Drain inputChannel → empty + │ + │ Step 2-3: prepareContext → formatMsgForLLM + │ → messages now include: + │ [system] "You are a helpful assistant." + │ [user] "What's the weather in Tokyo?" + │ [tool] tool_call_id="call_1", content="Weather in Tokyo: Sunny, 22°C" + │ + │ Step 4: llmCall + │ → assistantMessage(content = [Dict(:type => "text", :text => "The weather in Tokyo is sunny, 22°C.")]) + │ + │ Step 5: Extract tool calls → none + │ + │ Step 6: has_tool_calls == false → break, return final_response + │ + └─> return final_response + + +AGENT LOOP: SEND RESPONSE TO USER + └─> put!(agent.outputChannel, final_response) + └─> take_response(agent) → assistantMessage("The weather in Tokyo is sunny, 22°C.") +``` + +--- + +## 16. Tool File Contract + +Each `.jl` file in `src/tools/` must conform to the following contract: + +### Required Function ```julia -# Result from listTools: -# Available tools: -# - getWeather: Weather Lookup — Fetch current weather and forecast for a given city. -# - getTime: Time Lookup — Get current local time for a timezone or city. -# - writeTool: Create Tool — Generate new tool files... -# - listTools: List Tools — List all available tools with their names and labels... +function getTool()::agentTool + # Must return an agentTool instance +end ``` -### Complete Self-Tooling Example +### Optional Functions -``` -User: "I need to search for wines. Do you have a tool for that?" +```julia +# Argument preparation (before validation) +function prepareArguments(args::Dict{String,Any})::Dict{String,Any} + # Return modified args, or args unchanged + return args +end -# ─── LOOP: Agent realizes no wine search tool exists ───────────────── +# Custom validation (before execution) +function validateRequiredArgs(args::Dict{String,Any})::Union{Nothing,String} + # Return nothing to pass, or error string to fail + return nothing +end -# LLM generates the tool logic and calls writeTool to write it to disk - -[Tool Call] writeTool(name="searchWine", label="Wine Search", - description="Search a wine database by name, region, or variety", - inputSchema={...}, - executeCode="query = args[\"query\"]\nresult = \"Found wines...\"\nreturn agentToolResult([textContent(result)], ...)") - -# writeTool generates src/tools/searchWine.jl - -# ─── SYSTEM RESTARTS ───────────────────────────────────────────────── - -# loadTools("src/tools") loads searchWine.jl alongside all other tools - -# ─── Agent calls the new tool ───────────────────────────────────────── - -[Tool Call] searchWine(query="cabernet", maxResults=5) -# Result: "Found 5 cabernet wines..." - -# ─── Final response ────────────────────────────────────────────────── - -"The search found 5 cabernet wines: ..." +# Core execution +function executeTool(toolCallId::String, + args::Dict{String,Any}, + signal::Union{Nothing,abortSignal}, + onPartialResult::Function)::agentToolResult + # Return agentToolResult with content, details, usage, terminate + return agentToolResult([textContent("result")], Dict{Any,Any}(), nothing, false) +end ``` -## Available Tools +### File Structure -| Tool | Description | Validation | -|---|---|---| -| `getWeather` | Fetch weather for a city | Default (JSON Schema required) | -| `getTime` | Get current time for a timezone or city | Custom (cross-field + format) | -| `writeTool` | Create a new Julia tool module at runtime | Built-in (name + schema validation) | -| `listTools` | List all available tools with descriptions | None (no arguments) | +```julia +# src/tools/myTool.jl + +# Optional: helper functions +function helper_function(...) + ... +end + +# Optional: prepareArguments +function prepareArguments(args::Dict{String,Any})::Dict{String,Any} + return args +end + +# Optional: validateRequiredArgs +function validateRequiredArgs(args::Dict{String,Any})::Union{Nothing,String} + return nothing +end + +# Required: executeTool +function executeTool(toolCallId::String, args::Dict{String,Any}, + signal::Union{Nothing,abortSignal}, + onPartialResult::Function)::agentToolResult + ... +end + +# Required: getTool +function getTool()::agentTool + return agentTool( + name = "myTool", + label = "My Tool", + description = "What this tool does", + inputSchema = Dict{String,Any}( + "type" => "object", + "properties" => Dict(...), + "required" => [...] + ), + execute = executeTool, + prepareArguments = prepareArguments, + validateRequiredArgs = validateRequiredArgs, + parallelToolExecute = false + ) +end +``` + +### Module Isolation + +When `loadTools()` loads a file, it wraps it in a dynamically created submodule: + +```julia +# User writes in src/tools/myTool.jl: +function getTool()::agentTool ... end + +# loadTools() creates: +module _tool_myTool + using ..type + using Dates, UUIDs, DataStructures, JSON + # (user's code here) +end +``` + +All functions in the file are scoped under `_tool_myTool`, preventing name collisions with other tools. The module reference is kept alive in `_tool_modules` to prevent garbage collection of closures. + +--- + +## Appendix: Type Reference + +### Message Types + +| Type | Source | Description | +|------|--------|-------------| +| `messageContent` | `type.jl:67` | Abstract base for message content | +| `textContent` | `type.jl:69` | Plain text content (`text::String`) | +| `imageContent` | `type.jl:73` | Image content (`data::String`, `mimeType::String`) | +| `agentMessage` | `type.jl:82` | Abstract base for all messages | +| `userMessage` | `type.jl:84` | User message (`role`, `content`, `timestamp`) | +| `assistantMessage` | `type.jl:111` | LLM response (`role`, `content`, `api`, `provider`, `model`, `usage`, `stopReason`, `errorMessage`, `timestamp`) | +| `toolResultMessage` | `type.jl:152` | Tool result (`role`, `toolCallId`, `toolName`, `content`, `details`, `usage`, `addedToolNames`, `isError`, `timestamp`) | + +### Tool Types + +| Type | Source | Description | +|------|--------|-------------| +| `agentTool` | `type.jl:261` | Tool definition (name, label, description, schema, execute, hooks) | +| `agentToolCall` | `type.jl:362` | Tool call from LLM (type, id, name, arguments) | +| `agentToolResult` | `type.jl:429` | Tool execution result (content, details, usage, terminate) | +| `agentToolCallBatch` | `type.jl:835` | Batch of tool results (messages, terminate) | + +### Lifecycle Outcome Types + +| Type | Source | Description | +|------|--------|-------------| +| `preparedToolCall` | `type.jl:681` | After prepare: (tool, toolCall, args) | +| `immediateOutcome` | `type.jl:719` | Failed before execution: (result, isError) | +| `executedOutcome` | `type.jl:753` | After execute, before finalize: (result, isError) | +| `finalizedOutcome` | `type.jl:787` | After all phases: (toolCall, result, isError) | + +### Context & Config Types + +| Type | Source | Description | +|------|--------|-------------| +| `agentContext` | `type.jl:299` | Conversation snapshot (systemPrompt, messages, tools) | +| `agentState` | `type.jl:310` | Mutable runtime state (systemPrompt, model, tools, messages, pendingToolCalls, activeRun, errorMessage) | +| `agentLoopConfig` | `type.jl:403` | Loop config (tools, beforeToolCall, afterToolCall, toolExecution) | +| `abortSignal` | `type.jl:416` | Abort flag (`aborted::Bool`) | +| `assistantMsgCtx` | `type.jl:445` | Context for beforeToolCall (message, toolCall, args, context) | +| `afterCtx` | `type.jl:463` | Context for afterToolCall (message, toolCall, args, result, isError, context) | + +### Event Types + +| Type | Source | Description | +|------|--------|-------------| +| `toolExecStartEvent` | `type.jl:480` | (toolCallId, toolName, arguments) | +| `toolExecUpdateEvent` | `type.jl:495` | (toolCallId, toolName, arguments, partialResult) | +| `toolExecEndEvent` | `type.jl:511` | (toolCallId, toolName, result, isError) | + +### Agent Types + +| Type | Source | Description | +|------|--------|-------------| +| `agent` | `type.jl:522` | Abstract base type | +| `yiemAgent` | `type.jl:527` | High-level agent wrapper (state, channels, callbacks, task) |