526 lines
20 KiB
Markdown
526 lines
20 KiB
Markdown
# Tools
|
|
|
|
Tools allow the agent to perform actions and fetch data. Each tool defines a **schema** (what arguments it accepts) and an **execution function** (what it does).
|
|
|
|
## Tool Anatomy
|
|
|
|
Each tool has 3 main parts:
|
|
|
|
### 1. Schema (`inputSchema`)
|
|
|
|
JSON Schema (MCP format) describing the tool's arguments. The `"required"` array lists mandatory fields:
|
|
|
|
```julia
|
|
inputSchema = Dict{String,Any}(
|
|
"type" => "object",
|
|
"properties" => Dict(
|
|
"city" => Dict("type" => "string", "description" => "City name"),
|
|
"units" => Dict("type" => "string", "enum" => ["celsius", "fahrenheit"], "default" => "celsius")
|
|
),
|
|
"required" => ["city"]
|
|
)
|
|
```
|
|
|
|
### 2. Execution Function (`execute`)
|
|
|
|
A function with the signature:
|
|
|
|
```julia
|
|
execute(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult
|
|
```
|
|
|
|
- **`toolCallId`** — unique ID for this invocation (from the LLM's tool call)
|
|
- **`args`** — validated arguments provided by the LLM
|
|
- **`signal`** — abort signal for cancellable operations
|
|
- **`onPartialResult`** — callback for streaming progress updates
|
|
- **Returns** — `agentToolResult` with content, details, usage, and termination flag
|
|
|
|
```julia
|
|
function executeTool(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult
|
|
# Optional: stream progress updates
|
|
onPartialResult(Dict("status" => "Fetching data..."))
|
|
|
|
# Do work
|
|
result = "Weather in $(args["city"]): Sunny, 22°C"
|
|
|
|
# Return result
|
|
return agentToolResult(
|
|
[textContent(result)],
|
|
Dict{Any,Any}(), # details
|
|
nothing, # usage
|
|
false # terminate (true to stop agent loop)
|
|
)
|
|
end
|
|
```
|
|
|
|
### 3. Tool Definition (`getTool()`)
|
|
|
|
Returns an `agentTool` struct:
|
|
|
|
| Field | Type | Description |
|
|
|---|---|---|
|
|
| `name` | `String` | Unique identifier (e.g. `"getWeather"`) |
|
|
| `label` | `String` | Human-readable name (e.g. `"Weather Lookup"`) |
|
|
| `description` | `String` | What the tool does (shown to the LLM) |
|
|
| `inputSchema` | `Any` | JSON Schema (MCP format) |
|
|
| `execute` | `Function` | The execution function |
|
|
| `prepareArguments` | `Union{Function,Nothing}` | Optional argument transform before validation |
|
|
| `validateRequiredArgs` | `Union{Function,Nothing}` | Optional custom validation |
|
|
| `parallelToolExecute` | `Bool` | Run this tool in parallel with others |
|
|
|
|
## Argument Validation
|
|
|
|
Validation happens **before** tool execution, in the `prepareToolCall` phase. Invalid calls return an error immediately without invoking `execute`, `beforeToolCall`, or logging `toolExecutionStart`.
|
|
|
|
### Default: JSON Schema Required Fields
|
|
|
|
Set `validateRequiredArgs = nothing` to use the default validator, which checks that all fields in `inputSchema["required"]` are present:
|
|
|
|
```julia
|
|
# src/tools/getWeather.jl — uses default validation
|
|
function getTool()::agentTool
|
|
return agentTool(
|
|
name = "getWeather",
|
|
# ...
|
|
validateRequiredArgs = nothing, # uses default
|
|
)
|
|
end
|
|
```
|
|
|
|
### Custom Validation Hook
|
|
|
|
Override `validateRequiredArgs` when you need:
|
|
- **Cross-field constraints** (e.g. "at least one of X or Y")
|
|
- **Format validation** (e.g. regex patterns, date parsing)
|
|
- **Domain rules** (e.g. value ranges, business logic)
|
|
|
|
The hook signature takes only `args`:
|
|
|
|
```julia
|
|
function validateRequiredArgs(args::Dict{String,Any})::Union{Nothing,String}
|
|
tz = get(args, "timezone", nothing)
|
|
city = get(args, "city", "")
|
|
|
|
if !haskey(args, "timezone") && isempty(city)
|
|
return "Missing required argument: provide at least one of 'timezone' or 'city'"
|
|
end
|
|
|
|
if tz !== nothing
|
|
tz_str = string(tz)
|
|
if !occursin(r"^[A-Za-z]+\/[A-Za-z]+(/[A-Za-z]+)*$", tz_str)
|
|
return "Invalid timezone format: '$tz_str'. Use IANA format, e.g. 'America/New_York'"
|
|
end
|
|
end
|
|
|
|
return nothing
|
|
end
|
|
```
|
|
|
|
Return `nothing` to pass, or an error `String` to fail. The error is fed back to the LLM so it can retry with corrected arguments.
|
|
|
|
## Tool Discovery and Lifecycle
|
|
|
|
The agent iterates through tools via a **discover → execute → loop** cycle. Here is the complete flow from the framework author's perspective:
|
|
|
|
### The Agent Loop
|
|
|
|
```julia
|
|
# agentCore.jl:175 - _process_message()
|
|
while true
|
|
# 1. Drain messages from inputChannel
|
|
while isready(agent.inputChannel)
|
|
raw_msg = take!(agent.inputChannel)
|
|
user_msg = OpenAiToUserMessage(raw_msg)
|
|
push!(agent._state.messages, user_msg)
|
|
end
|
|
|
|
# 2. Format messages for LLM
|
|
ctx = agent.prepareContext(agent._state)
|
|
formatted = agent.formatMsgForLLM(ctx)
|
|
|
|
# 3. Call LLM
|
|
response = agent.llmCall(formatted)
|
|
|
|
# 4. Check if LLM used tool calls
|
|
if has_tool_calls(response.content)
|
|
# 5. Execute tools, feed results back to LLM, loop
|
|
else
|
|
# 6. No tool calls — return final response
|
|
break
|
|
end
|
|
end
|
|
```
|
|
|
|
### Step 1: Tool Discovery
|
|
|
|
Tools are discovered from `agent._state.tools`, which is a `Vector{agentTool}` populated during agent creation:
|
|
|
|
```julia
|
|
# Loading tools
|
|
tools = loadTools("src/tools") # returns Vector{agentTool}
|
|
|
|
# Passing to agent
|
|
agent = yiemAgent(
|
|
systemPrompt = "...",
|
|
tools = tools, # ← tools stored in agent._state.tools
|
|
llmCall = my_llm_call,
|
|
agentEventSink = my_event_sink,
|
|
)
|
|
```
|
|
|
|
When the LLM response contains tool calls, the agent builds an `agentContext` with those tools:
|
|
|
|
```julia
|
|
context = agentContext(
|
|
agent._state.systemPrompt,
|
|
agent._state.messages,
|
|
agent._state.tools, # ← tools available for discovery
|
|
)
|
|
```
|
|
|
|
### Step 2: Extract Tool Calls from LLM Response
|
|
|
|
The agent inspects the `response.content` blocks for `tool_calls`:
|
|
|
|
```julia
|
|
# agentCore.jl:217-245
|
|
tool_call_list = agentToolCall[]
|
|
|
|
for content_block in response.content
|
|
if content_block isa Dict
|
|
if get(content_block, :type, "") == "tool_calls"
|
|
# OpenAI format: {"type": "tool_calls", "tool_calls": [...]}
|
|
for tc_data in get(content_block, :tool_calls, [])
|
|
tc = agentToolCall(
|
|
type = "function",
|
|
id = get(tc_data, :id, string(uuid4())),
|
|
name = get(tc_data, :function, Dict())[:name],
|
|
arguments = get(tc_data, :function, Dict())[:arguments],
|
|
)
|
|
push!(tool_call_list, tc)
|
|
end
|
|
elseif get(content_block, :type, "") == "tool_call"
|
|
# Alternative format: single tool_call block
|
|
tc = agentToolCall(
|
|
type = "function",
|
|
id = get(content_block, :id, string(uuid4())),
|
|
name = get(content_block, :name, ""),
|
|
arguments = get(content_block, :arguments, Dict()),
|
|
)
|
|
push!(tool_call_list, tc)
|
|
end
|
|
end
|
|
end
|
|
```
|
|
|
|
### Step 3: Execute Each Tool Call
|
|
|
|
For each tool call, the agent runs through the **prepare → execute → finalize** pipeline:
|
|
|
|
```julia
|
|
# agentCore.jl:247-302
|
|
if has_tool_calls && length(tool_call_list) > 0
|
|
context = agentContext(agent._state.systemPrompt, agent._state.messages, agent._state.tools)
|
|
config = agentLoopConfig(agent._state.tools, agent.beforeToolCall, agent.afterToolCall, execution_mode)
|
|
signal = nothing
|
|
emit = agent.agentEventSink
|
|
|
|
# Execute all tool calls (sequential or parallel)
|
|
batch = executeToolCalls(context, response, tool_call_list, config, signal, emit)
|
|
|
|
# Save results to conversation history
|
|
for tool_result in batch.messages
|
|
push!(agent._state.messages, tool_result)
|
|
end
|
|
|
|
# If any tool requested termination, break the loop
|
|
if batch.terminate
|
|
final_response = build_final_response(batch)
|
|
break
|
|
end
|
|
# Otherwise, loop back to step 2 (format + call LLM again)
|
|
end
|
|
```
|
|
|
|
### Step 4: The Per-Call Pipeline
|
|
|
|
Each tool call goes through three phases:
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ PREPARE → prepareToolCall() │
|
|
│ │
|
|
│ 1. Find tool by name in context.tools │
|
|
│ 2. Transform args via tool.prepareArguments (if defined) │
|
|
│ 3. Validate via tool.validateRequiredArgs (or default) │
|
|
│ 4. Run beforeToolCall hook (if defined) │
|
|
│ └── on any failure → return immediateOutcome (skip execution) │
|
|
│ └── success → return preparedToolCall │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ EXECUTE → executePreparedToolCall() │
|
|
│ │
|
|
│ 1. emit toolExecutionStart event │
|
|
│ 2. call tool.execute(toolCallId, args, signal, onPartialResult)│
|
|
│ 3. wait for all pending update events │
|
|
│ └── on error → return executedOutcome(isError=true) │
|
|
│ └── success → return executedOutcome(isError=false) │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ FINALIZE → finalizeExecutedToolCall() │
|
|
│ │
|
|
│ 1. Run afterToolCall hook (if defined) │
|
|
│ - can mutate content, details, usage, terminate, isError │
|
|
│ 2. emit toolExecutionEnd event │
|
|
│ 3. createToolResultMessage → adds to conversation history │
|
|
│ └── return finalizedOutcome │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Step 5: Feed Results Back to LLM
|
|
|
|
Tool results are added to `agent._state.messages` as `toolResultMessage` objects. On the next loop iteration, `formatMsgForLLM()` converts them to OpenAI format and the LLM receives the results:
|
|
|
|
```
|
|
Conversation history after tool execution:
|
|
[system] "You are a helpful assistant."
|
|
[user] "What's the weather in Tokyo?"
|
|
[assistant] (tool_calls: getWeather(city="Tokyo"))
|
|
[tool] tool_call_id="call_1", tool_name="getWeather", content="Weather in Tokyo: Sunny, 22°C"
|
|
```
|
|
|
|
The LLM then decides: call another tool, or return a final text answer.
|
|
|
|
## Execution Modes
|
|
|
|
### Sequential
|
|
|
|
Tools execute one at a time in order. Required when:
|
|
- Tools have implicit dependencies
|
|
- Tools share state (e.g. writing to the same file)
|
|
- Tools have `parallelToolExecute = false`
|
|
|
|
Set globally via `agentLoopConfig.toolExecution = "sequential"`, or per-tool via `parallelToolExecute = false`.
|
|
|
|
### Parallel
|
|
|
|
Tools execute concurrently when all are independent. Reduces wall-clock time. Set `parallelToolExecute = true` on individual tools, or set `agentLoopConfig.toolExecution = "parallel"`.
|
|
|
|
## Streaming Partial Results
|
|
|
|
For long-running tools (API calls, file uploads, training), use `onPartialResult` to stream progress:
|
|
|
|
```julia
|
|
function executeTool(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult
|
|
onPartialResult(Dict("status" => "Step 1: Fetching data..."))
|
|
sleep(1)
|
|
|
|
onPartialResult(Dict("status" => "Step 2: Processing..."))
|
|
sleep(1)
|
|
|
|
return agentToolResult(
|
|
[textContent("Done!")],
|
|
Dict{Any,Any}(), nothing, false
|
|
)
|
|
end
|
|
```
|
|
|
|
UI listeners and the TUI consume these events in real time via `toolExecutionUpdate`.
|
|
|
|
## Loading Tools
|
|
|
|
### Auto-load from Directory
|
|
|
|
```julia
|
|
using .toolRegistry
|
|
|
|
tools = loadTools("src/tools") # scans for *.jl files with getTool()
|
|
```
|
|
|
|
Files are loaded alphabetically for deterministic registration order.
|
|
|
|
### Manual Registration
|
|
|
|
```julia
|
|
tool = getTool() # from your tool module
|
|
registerTool(tool)
|
|
```
|
|
|
|
## Complete Lifecycle Example
|
|
|
|
```julia
|
|
# ─── USER SENDS MESSAGE ───────────────────────────────────────────
|
|
run_agent(agent, "What's the weather in Tokyo?")
|
|
|
|
# ─── LOOP ITERATION 1 ─────────────────────────────────────────────
|
|
# Agent formats messages and calls LLM
|
|
formatted = agent.formatMsgForLLM(agent.prepareContext(agent._state))
|
|
response = agent.llmCall(formatted)
|
|
# LLM returns: {"content": [{"type": "tool_calls", "tool_calls": [{"name": "getWeather", "arguments": {"city": "Tokyo"}}]}]}
|
|
|
|
# Agent extracts tool call, builds context
|
|
context = agentContext(systemPrompt, messages, agent._state.tools)
|
|
tool_call_list = [agentToolCall("call_1", "getWeather", Dict("city" => "Tokyo"))]
|
|
|
|
# PREPARE: find tool, validate args
|
|
tool = find(t -> t.name == "getWeather", context.tools) # found!
|
|
validateRequiredArgs(Dict("city" => "Tokyo"), tool.inputSchema) # passes
|
|
beforeToolCall_hook(agentMsgCtx, nothing) # nil, skipped
|
|
|
|
# EXECUTE: call tool.execute()
|
|
result = tool.execute("call_1", Dict("city" => "Tokyo"), nothing, onPartialResult)
|
|
# Returns: agentToolResult([textContent("Weather in Tokyo: Sunny, 22°C")], Dict(), nothing, false)
|
|
|
|
# FINALIZE: afterToolCall hook, emit events
|
|
finalized = finalizedOutcome(tc, result, false)
|
|
emit(toolExecEndEvent("call_1", "getWeather", result, false))
|
|
msg = createToolResultMessage(finalized) # toolResultMessage for conversation history
|
|
|
|
# Add result to conversation
|
|
push!(agent._state.messages, msg)
|
|
# Messages now: [user: "What's the weather?", assistant: {tool_calls: getWeather}, tool: "Sunny, 22°C"]
|
|
|
|
# ─── LOOP ITERATION 2 ─────────────────────────────────────────────
|
|
# LLM called again with tool result included
|
|
formatted = agent.formatMsgForLLM(agent.prepareContext(agent._state))
|
|
response = agent.llmCall(formatted)
|
|
# LLM returns: {"content": [{"type": "text", "text": "The weather in Tokyo is sunny, 22°C."}]}
|
|
|
|
# No tool calls detected → break loop, return final response
|
|
return assistantMessage(content=[textContent("The weather in Tokyo is sunny, 22°C.")], ...)
|
|
|
|
# ─── USER RECEIVES RESPONSE ───────────────────────────────────────
|
|
response = take_response(agent)
|
|
println(response.content)
|
|
# => "[textContent(\"The weather in Tokyo is sunny, 22°C.\")]"
|
|
```
|
|
|
|
## Self-Modifying Tools
|
|
|
|
The framework includes tools that allow the agent to create new tools at runtime.
|
|
|
|
### `writeTool` — Create New Tool Files
|
|
|
|
`writeTool` is a **file writer**, not a code generator. The LLM provides the tool logic as `executeCode` (the actual Julia code), and `writeTool` wraps it in the required boilerplate.
|
|
|
|
**How it works:**
|
|
|
|
The LLM constructs `writeTool` with:
|
|
- **`executeCode`** — the actual tool logic (Julia code body, NOT wrapped in a function)
|
|
- **`name`, `label`, `description`** — tool metadata
|
|
- **`inputSchema`** — parameter schema in MCP format
|
|
- **`validateCode`, `prepareCode`** (optional) — custom validation/preparation logic
|
|
|
|
`writeTool` produces `src/tools/<name>.jl` by:
|
|
1. Converting the `inputSchema` Dict into a Julia `Dict{String,Any}(...)` string literal
|
|
2. Indenting `executeCode` with 4 spaces
|
|
3. Wrapping it inside a `function executeTool(...)::agentToolResult ... end` template
|
|
4. Appending the `getTool()` definition that returns an `agentTool` struct
|
|
5. Writing the combined string to disk
|
|
|
|
**Workflow:**
|
|
|
|
```
|
|
LLM decides: "Need a searchWine tool. I'll provide the logic."
|
|
|
|
LLM calls writeTool:
|
|
name: "searchWine"
|
|
executeCode: "query = args[\"query\"]\nresult = search(query)\nreturn ..."
|
|
|
|
writeTool wraps it → src/tools/searchWine.jl:
|
|
function executeTool(...)::agentToolResult
|
|
query = args["query"] ← LLM code (indented 4 spaces)
|
|
result = search(query)
|
|
return agentToolResult(...)
|
|
end
|
|
|
|
function getTool()::agentTool
|
|
return agentTool(name="searchWine", ...)
|
|
end
|
|
|
|
Restart → loadTools("src/tools") loads searchWine.jl
|
|
```
|
|
|
|
**Example specification:**
|
|
|
|
```julia
|
|
Dict(
|
|
"name" => "searchWine",
|
|
"label" => "Wine Search",
|
|
"description" => "Search a wine database by name, region, or variety",
|
|
"inputSchema" => Dict(
|
|
"type" => "object",
|
|
"properties" => Dict(
|
|
"query" => Dict("type" => "string", "description" => "Search query"),
|
|
"maxResults" => Dict("type" => "integer", "default" => 10)
|
|
),
|
|
"required" => ["query"]
|
|
),
|
|
"executeCode" => """
|
|
query = args["query"]
|
|
max_results = get(args, "maxResults", 10)
|
|
# Perform search logic here
|
|
result = "Found 3 wines matching: $query"
|
|
return agentToolResult([textContent(result)], Dict{Any,Any}(), nothing, false)
|
|
""",
|
|
"parallel" => false
|
|
)
|
|
```
|
|
|
|
**Optional hooks:**
|
|
|
|
| Field | Description |
|
|
|---|---|
|
|
| `validateCode` | Custom validation Julia code (runs before execute). Return `nothing` to pass, or an error `String` to fail. |
|
|
| `prepareCode` | Argument preparation code (runs before validation). Return modified args dict. |
|
|
|
|
### `listTools` — Discover Available Tools
|
|
|
|
Returns all registered tools. Primarily useful for **collision detection** before creating a new tool via `writeTool` — the LLM checks existing names before picking a unique one.
|
|
|
|
```julia
|
|
# Result from listTools:
|
|
# Available tools:
|
|
# - getWeather: Weather Lookup — Fetch current weather and forecast for a given city.
|
|
# - getTime: Time Lookup — Get current local time for a timezone or city.
|
|
# - writeTool: Create Tool — Generate new tool files...
|
|
# - listTools: List Tools — List all available tools with their names and labels...
|
|
```
|
|
|
|
### Complete Self-Tooling Example
|
|
|
|
```
|
|
User: "I need to search for wines. Do you have a tool for that?"
|
|
|
|
# ─── LOOP: Agent realizes no wine search tool exists ─────────────────
|
|
|
|
# LLM generates the tool logic and calls writeTool to write it to disk
|
|
|
|
[Tool Call] writeTool(name="searchWine", label="Wine Search",
|
|
description="Search a wine database by name, region, or variety",
|
|
inputSchema={...},
|
|
executeCode="query = args[\"query\"]\nresult = \"Found wines...\"\nreturn agentToolResult([textContent(result)], ...)")
|
|
|
|
# writeTool generates src/tools/searchWine.jl
|
|
|
|
# ─── SYSTEM RESTARTS ─────────────────────────────────────────────────
|
|
|
|
# loadTools("src/tools") loads searchWine.jl alongside all other tools
|
|
|
|
# ─── Agent calls the new tool ─────────────────────────────────────────
|
|
|
|
[Tool Call] searchWine(query="cabernet", maxResults=5)
|
|
# Result: "Found 5 cabernet wines..."
|
|
|
|
# ─── Final response ──────────────────────────────────────────────────
|
|
|
|
"The search found 5 cabernet wines: ..."
|
|
```
|
|
|
|
## Available Tools
|
|
|
|
| Tool | Description | Validation |
|
|
|---|---|---|
|
|
| `getWeather` | Fetch weather for a city | Default (JSON Schema required) |
|
|
| `getTime` | Get current time for a timezone or city | Custom (cross-field + format) |
|
|
| `writeTool` | Create a new Julia tool module at runtime | Built-in (name + schema validation) |
|
|
| `listTools` | List all available tools with descriptions | None (no arguments) |
|