# Tools

Tools allow the agent to perform actions and fetch data. Each tool defines a **schema** (what arguments it accepts) and an **execution function** (what it does).

## Tool Anatomy

Each tool has 3 main parts:

### 1. Schema (`inputSchema`)

JSON Schema (MCP format) describing the tool's arguments. The `"required"` array lists mandatory fields:

```julia
inputSchema = Dict{String,Any}(
    "type" => "object",
    "properties" => Dict(
        "city" => Dict("type" => "string", "description" => "City name"),
        "units" => Dict("type" => "string", "enum" => ["celsius", "fahrenheit"], "default" => "celsius")
    ),
    "required" => ["city"]
)
```

### 2. Execution Function (`execute`)

A function with the signature:

```julia
execute(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult
```

- **`toolCallId`** — unique ID for this invocation (from the LLM's tool call)
- **`args`** — validated arguments provided by the LLM
- **`signal`** — abort signal for cancellable operations
- **`onPartialResult`** — callback for streaming progress updates
- **Returns** — `agentToolResult` with content, details, usage, and termination flag

```julia
function executeTool(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult
    # Optional: stream progress updates
    onPartialResult(Dict("status" => "Fetching data..."))
    
    # Do work
    result = "Weather in $(args["city"]): Sunny, 22°C"
    
    # Return result
    return agentToolResult(
        [textContent(result)],
        Dict{Any,Any}(),  # details
        nothing,           # usage
        false              # terminate (true to stop agent loop)
    )
end
```

### 3. Tool Definition (`getTool()`)

Returns an `agentTool` struct:

| Field | Type | Description |
|---|---|---|
| `name` | `String` | Unique identifier (e.g. `"getWeather"`) |
| `label` | `String` | Human-readable name (e.g. `"Weather Lookup"`) |
| `description` | `String` | What the tool does (shown to the LLM) |
| `inputSchema` | `Any` | JSON Schema (MCP format) |
| `execute` | `Function` | The execution function |
| `prepareArguments` | `Union{Function,Nothing}` | Optional argument transform before validation |
| `validateRequiredArgs` | `Union{Function,Nothing}` | Optional custom validation |
| `parallelToolExecute` | `Bool` | Run this tool in parallel with others |

## Argument Validation

Validation happens **before** tool execution, in the `prepareToolCall` phase. Invalid calls return an error immediately without invoking `execute`, `beforeToolCall`, or logging `toolExecutionStart`.

### Default: JSON Schema Required Fields

Set `validateRequiredArgs = nothing` to use the default validator, which checks that all fields in `inputSchema["required"]` are present:

```julia
# src/tools/getWeather.jl — uses default validation
function getTool()::agentTool
    return agentTool(
        name = "getWeather",
        # ...
        validateRequiredArgs = nothing,  # uses default
    )
end
```

### Custom Validation Hook

Override `validateRequiredArgs` when you need:
- **Cross-field constraints** (e.g. "at least one of X or Y")
- **Format validation** (e.g. regex patterns, date parsing)
- **Domain rules** (e.g. value ranges, business logic)

The hook signature takes only `args`:

```julia
function validateRequiredArgs(args::Dict{String,Any})::Union{Nothing,String}
    tz = get(args, "timezone", nothing)
    city = get(args, "city", "")
    
    if !haskey(args, "timezone") && isempty(city)
        return "Missing required argument: provide at least one of 'timezone' or 'city'"
    end
    
    if tz !== nothing
        tz_str = string(tz)
        if !occursin(r"^[A-Za-z]+\/[A-Za-z]+(/[A-Za-z]+)*$", tz_str)
            return "Invalid timezone format: '$tz_str'. Use IANA format, e.g. 'America/New_York'"
        end
    end
    
    return nothing
end
```

Return `nothing` to pass, or an error `String` to fail. The error is fed back to the LLM so it can retry with corrected arguments.

## Tool Discovery and Lifecycle

The agent iterates through tools via a **discover → execute → loop** cycle. Here is the complete flow from the framework author's perspective:

### The Agent Loop

```julia
# agentCore.jl:175 - _process_message()
while true
    # 1. Drain messages from inputChannel
    while isready(agent.inputChannel)
        raw_msg = take!(agent.inputChannel)
        user_msg = OpenAiToUserMessage(raw_msg)
        push!(agent._state.messages, user_msg)
    end

    # 2. Format messages for LLM
    ctx = agent.prepareContext(agent._state)
    formatted = agent.formatMsgForLLM(ctx)

    # 3. Call LLM
    response = agent.llmCall(formatted)

    # 4. Check if LLM used tool calls
    if has_tool_calls(response.content)
        # 5. Execute tools, feed results back to LLM, loop
    else
        # 6. No tool calls — return final response
        break
    end
end
```

### Step 1: Tool Discovery

Tools are discovered from `agent._state.tools`, which is a `Vector{agentTool}` populated during agent creation:

```julia
# Loading tools
tools = loadTools("src/tools")  # returns Vector{agentTool}

# Passing to agent
agent = yiemAgent(
    systemPrompt = "...",
    tools = tools,  # ← tools stored in agent._state.tools
    llmCall = my_llm_call,
    agentEventSink = my_event_sink,
)
```

When the LLM response contains tool calls, the agent builds an `agentContext` with those tools:

```julia
context = agentContext(
    agent._state.systemPrompt,
    agent._state.messages,
    agent._state.tools,  # ← tools available for discovery
)
```

### Step 2: Extract Tool Calls from LLM Response

The agent inspects the `response.content` blocks for `tool_calls`:

```julia
# agentCore.jl:217-245
tool_call_list = agentToolCall[]

for content_block in response.content
    if content_block isa Dict
        if get(content_block, :type, "") == "tool_calls"
            # OpenAI format: {"type": "tool_calls", "tool_calls": [...]}
            for tc_data in get(content_block, :tool_calls, [])
                tc = agentToolCall(
                    type = "function",
                    id = get(tc_data, :id, string(uuid4())),
                    name = get(tc_data, :function, Dict())[:name],
                    arguments = get(tc_data, :function, Dict())[:arguments],
                )
                push!(tool_call_list, tc)
            end
        elseif get(content_block, :type, "") == "tool_call"
            # Alternative format: single tool_call block
            tc = agentToolCall(
                type = "function",
                id = get(content_block, :id, string(uuid4())),
                name = get(content_block, :name, ""),
                arguments = get(content_block, :arguments, Dict()),
            )
            push!(tool_call_list, tc)
        end
    end
end
```

### Step 3: Execute Each Tool Call

For each tool call, the agent runs through the **prepare → execute → finalize** pipeline:

```julia
# agentCore.jl:247-302
if has_tool_calls && length(tool_call_list) > 0
    context = agentContext(agent._state.systemPrompt, agent._state.messages, agent._state.tools)
    config = agentLoopConfig(agent._state.tools, agent.beforeToolCall, agent.afterToolCall, execution_mode)
    signal = nothing
    emit = agent.agentEventSink

    # Execute all tool calls (sequential or parallel)
    batch = executeToolCalls(context, response, tool_call_list, config, signal, emit)

    # Save results to conversation history
    for tool_result in batch.messages
        push!(agent._state.messages, tool_result)
    end

    # If any tool requested termination, break the loop
    if batch.terminate
        final_response = build_final_response(batch)
        break
    end
    # Otherwise, loop back to step 2 (format + call LLM again)
end
```

### Step 4: The Per-Call Pipeline

Each tool call goes through three phases:

```
┌─────────────────────────────────────────────────────────────────┐
│ PREPARE  →  prepareToolCall()                                  │
│                                                                 │
│   1. Find tool by name in context.tools                         │
│   2. Transform args via tool.prepareArguments (if defined)      │
│   3. Validate via tool.validateRequiredArgs (or default)        │
│   4. Run beforeToolCall hook (if defined)                       │
│   └── on any failure → return immediateOutcome (skip execution) │
│   └── success → return preparedToolCall                         │
├─────────────────────────────────────────────────────────────────┤
│ EXECUTE  →  executePreparedToolCall()                           │
│                                                                 │
│   1. emit toolExecutionStart event                              │
│   2. call tool.execute(toolCallId, args, signal, onPartialResult)│
│   3. wait for all pending update events                         │
│   └── on error → return executedOutcome(isError=true)           │
│   └── success → return executedOutcome(isError=false)           │
├─────────────────────────────────────────────────────────────────┤
│ FINALIZE  →  finalizeExecutedToolCall()                         │
│                                                                 │
│   1. Run afterToolCall hook (if defined)                        │
│      - can mutate content, details, usage, terminate, isError   │
│   2. emit toolExecutionEnd event                                │
│   3. createToolResultMessage → adds to conversation history     │
│   └── return finalizedOutcome                                   │
└─────────────────────────────────────────────────────────────────┘
```

### Step 5: Feed Results Back to LLM

Tool results are added to `agent._state.messages` as `toolResultMessage` objects. On the next loop iteration, `formatMsgForLLM()` converts them to OpenAI format and the LLM receives the results:

```
Conversation history after tool execution:
  [system] "You are a helpful assistant."
  [user] "What's the weather in Tokyo?"
  [assistant] (tool_calls: getWeather(city="Tokyo"))
  [tool] tool_call_id="call_1", tool_name="getWeather", content="Weather in Tokyo: Sunny, 22°C"
```

The LLM then decides: call another tool, or return a final text answer.

## Execution Modes

### Sequential

Tools execute one at a time in order. Required when:
- Tools have implicit dependencies
- Tools share state (e.g. writing to the same file)
- Tools have `parallelToolExecute = false`

Set globally via `agentLoopConfig.toolExecution = "sequential"`, or per-tool via `parallelToolExecute = false`.

### Parallel

Tools execute concurrently when all are independent. Reduces wall-clock time. Set `parallelToolExecute = true` on individual tools, or set `agentLoopConfig.toolExecution = "parallel"`.

## Streaming Partial Results

For long-running tools (API calls, file uploads, training), use `onPartialResult` to stream progress:

```julia
function executeTool(toolCallId::String, args::Dict{String,Any}, signal::Union{Nothing,abortSignal}, onPartialResult::Function)::agentToolResult
    onPartialResult(Dict("status" => "Step 1: Fetching data..."))
    sleep(1)
    
    onPartialResult(Dict("status" => "Step 2: Processing..."))
    sleep(1)
    
    return agentToolResult(
        [textContent("Done!")],
        Dict{Any,Any}(), nothing, false
    )
end
```

UI listeners and the TUI consume these events in real time via `toolExecutionUpdate`.

## Loading Tools

### Auto-load from Directory

```julia
using .toolRegistry

tools = loadTools("src/tools")  # scans for *.jl files with getTool()
```

Files are loaded alphabetically for deterministic registration order.

### Manual Registration

```julia
tool = getTool()  # from your tool module
registerTool(tool)
```

## Complete Lifecycle Example

```julia
# ─── USER SENDS MESSAGE ───────────────────────────────────────────
run_agent(agent, "What's the weather in Tokyo?")

# ─── LOOP ITERATION 1 ─────────────────────────────────────────────
# Agent formats messages and calls LLM
formatted = agent.formatMsgForLLM(agent.prepareContext(agent._state))
response = agent.llmCall(formatted)
# LLM returns: {"content": [{"type": "tool_calls", "tool_calls": [{"name": "getWeather", "arguments": {"city": "Tokyo"}}]}]}

# Agent extracts tool call, builds context
context = agentContext(systemPrompt, messages, agent._state.tools)
tool_call_list = [agentToolCall("call_1", "getWeather", Dict("city" => "Tokyo"))]

# PREPARE: find tool, validate args
tool = find(t -> t.name == "getWeather", context.tools)  # found!
validateRequiredArgs(Dict("city" => "Tokyo"), tool.inputSchema)  # passes
beforeToolCall_hook(agentMsgCtx, nothing)  # nil, skipped

# EXECUTE: call tool.execute()
result = tool.execute("call_1", Dict("city" => "Tokyo"), nothing, onPartialResult)
# Returns: agentToolResult([textContent("Weather in Tokyo: Sunny, 22°C")], Dict(), nothing, false)

# FINALIZE: afterToolCall hook, emit events
finalized = finalizedOutcome(tc, result, false)
emit(toolExecEndEvent("call_1", "getWeather", result, false))
msg = createToolResultMessage(finalized)  # toolResultMessage for conversation history

# Add result to conversation
push!(agent._state.messages, msg)
# Messages now: [user: "What's the weather?", assistant: {tool_calls: getWeather}, tool: "Sunny, 22°C"]

# ─── LOOP ITERATION 2 ─────────────────────────────────────────────
# LLM called again with tool result included
formatted = agent.formatMsgForLLM(agent.prepareContext(agent._state))
response = agent.llmCall(formatted)
# LLM returns: {"content": [{"type": "text", "text": "The weather in Tokyo is sunny, 22°C."}]}

# No tool calls detected → break loop, return final response
return assistantMessage(content=[textContent("The weather in Tokyo is sunny, 22°C.")], ...)

# ─── USER RECEIVES RESPONSE ───────────────────────────────────────
response = take_response(agent)
println(response.content)
# => "[textContent(\"The weather in Tokyo is sunny, 22°C.\")]"
```

## Self-Modifying Tools

The framework includes tools that allow the agent to create new tools at runtime.

### `writeTool` — Create New Tool Files

`writeTool` is a **file writer**, not a code generator. The LLM provides the tool logic as `executeCode` (the actual Julia code), and `writeTool` wraps it in the required boilerplate.

**How it works:**

The LLM constructs `writeTool` with:
- **`executeCode`** — the actual tool logic (Julia code body, NOT wrapped in a function)
- **`name`, `label`, `description`** — tool metadata
- **`inputSchema`** — parameter schema in MCP format
- **`validateCode`, `prepareCode`** (optional) — custom validation/preparation logic

`writeTool` produces `src/tools/<name>.jl` by:
1. Converting the `inputSchema` Dict into a Julia `Dict{String,Any}(...)` string literal
2. Indenting `executeCode` with 4 spaces
3. Wrapping it inside a `function executeTool(...)::agentToolResult ... end` template
4. Appending the `getTool()` definition that returns an `agentTool` struct
5. Writing the combined string to disk

**Workflow:**

```
LLM decides: "Need a searchWine tool. I'll provide the logic."

LLM calls writeTool:
  name: "searchWine"
  executeCode: "query = args[\"query\"]\nresult = search(query)\nreturn ..."

writeTool wraps it → src/tools/searchWine.jl:
  function executeTool(...)::agentToolResult
      query = args["query"]          ← LLM code (indented 4 spaces)
      result = search(query)
      return agentToolResult(...)
  end

  function getTool()::agentTool
      return agentTool(name="searchWine", ...)
  end

Restart → loadTools("src/tools") loads searchWine.jl
```

**Example specification:**

```julia
Dict(
    "name" => "searchWine",
    "label" => "Wine Search",
    "description" => "Search a wine database by name, region, or variety",
    "inputSchema" => Dict(
        "type" => "object",
        "properties" => Dict(
            "query" => Dict("type" => "string", "description" => "Search query"),
            "maxResults" => Dict("type" => "integer", "default" => 10)
        ),
        "required" => ["query"]
    ),
    "executeCode" => """
        query = args["query"]
        max_results = get(args, "maxResults", 10)
        # Perform search logic here
        result = "Found 3 wines matching: $query"
        return agentToolResult([textContent(result)], Dict{Any,Any}(), nothing, false)
    """,
    "parallel" => false
)
```

**Optional hooks:**

| Field | Description |
|---|---|
| `validateCode` | Custom validation Julia code (runs before execute). Return `nothing` to pass, or an error `String` to fail. |
| `prepareCode` | Argument preparation code (runs before validation). Return modified args dict. |

### `listTools` — Discover Available Tools

Returns all registered tools. Primarily useful for **collision detection** before creating a new tool via `writeTool` — the LLM checks existing names before picking a unique one.

```julia
# Result from listTools:
# Available tools:
# - getWeather: Weather Lookup — Fetch current weather and forecast for a given city.
# - getTime: Time Lookup — Get current local time for a timezone or city.
# - writeTool: Create Tool — Generate new tool files...
# - listTools: List Tools — List all available tools with their names and labels...
```

### Complete Self-Tooling Example

```
User: "I need to search for wines. Do you have a tool for that?"

# ─── LOOP: Agent realizes no wine search tool exists ─────────────────

# LLM generates the tool logic and calls writeTool to write it to disk

[Tool Call] writeTool(name="searchWine", label="Wine Search",
    description="Search a wine database by name, region, or variety",
    inputSchema={...},
    executeCode="query = args[\"query\"]\nresult = \"Found wines...\"\nreturn agentToolResult([textContent(result)], ...)")

# writeTool generates src/tools/searchWine.jl

# ─── SYSTEM RESTARTS ─────────────────────────────────────────────────

# loadTools("src/tools") loads searchWine.jl alongside all other tools

# ─── Agent calls the new tool ─────────────────────────────────────────

[Tool Call] searchWine(query="cabernet", maxResults=5)
# Result: "Found 5 cabernet wines..."

# ─── Final response ──────────────────────────────────────────────────

"The search found 5 cabernet wines: ..."
```

## Available Tools

| Tool | Description | Validation |
|---|---|---|
| `getWeather` | Fetch weather for a city | Default (JSON Schema required) |
| `getTime` | Get current time for a timezone or city | Custom (cross-field + format) |
| `writeTool` | Create a new Julia tool module at runtime | Built-in (name + schema validation) |
| `listTools` | List all available tools with descriptions | None (no arguments) |