YiemAgent/README.md

# YiemAgent

Julia framework for building agents with tool use and MCP (Model Context Protocol) support.

## Quick Start

```julia
using YiemAgent

# 1. Create an LLM callable (see "Custom LLM Callable" below)
llmCall = ...

# 2. Create a callable MCP server (see "MCP Server Integration" below)
mcpServer = ...

# 3. Create the agent
agent = yiemAgent(llmCall; mcpServer=mcpServer, eventSink=println)

# 4. Interact
promptAgent(agent, "What tools do you have?")
response = takeResponse(agent)

# 5. Stop when done
stopAgent(agent)
```

## Agent API

| Function | Description |
|----------|-------------|
| `promptAgent(agent, text)` | Send plain text to the agent's input channel |
| `promptAgent(agent, msg)` | Send a structured message (OpenAI format dict) to the input channel |
| `takeResponse(agent)` | Block and take the agent's response from output channel |
| `followUp(agent, text)` | Send plain text follow-up while agent is still processing |
| `followUp(agent, msg)` | Send a structured follow-up (OpenAI format dict) while agent is processing |
| `stopAgent(agent)` | Gracefully stop the agent and close channels |

## Creating a yiemAgent

```julia
yiemAgent(llmCall;
    systemPrompt::String = "You are a helpful assistant.",
    model = nothing,
    messages::Vector{agentMessage} = agentMessage[],
    prepareContext::Function = prepareContext,
    formatMsgForLLM::Function = formatMsgForLLM,
    beforeToolCall::Function = beforeToolCall,
    afterToolCall::Function = afterToolCall,
    sessionId::Union{String, Nothing} = nothing,
    maxRetryDelayMs::Union{Int64, Nothing} = nothing,
    parallelToolExecute::Bool = false,
    eventSink::Function = eventSink,
    mcpServer = nothing
)
```

| Parameter | Description |
|-----------|-------------|
| `llmCall` | **(required)** Function that receives formatted messages and returns an LLM response |
| `systemPrompt` | System prompt for the agent |
| `model` | LLM model configuration (`llmModel` struct) |
| `messages` | Initial conversation messages |
| `prepareContext` | Hook to transform messages/context before each LLM call |
| `formatMsgForLLM` | Hook to convert agent context to LLM API format |
| `beforeToolCall` | Hook to block/allow tool execution |
| `afterToolCall` | Hook to post-process tool results |
| `sessionId` | Optional session identifier for MCP server |
| `maxRetryDelayMs` | Maximum retry delay in milliseconds |
| `parallelToolExecute` | Run multiple tool calls in parallel |
| `eventSink` | Callback for agent events/debug messages |
| `mcpServer` | Callable struct for MCP server communication |

## Usage Examples

### Text Messages

```julia
# Send plain text
promptAgent(agent, "What tools do you have?")
response = takeResponse(agent)

# Follow-up while agent is processing
promptAgent(agent, "Tell me about wine")
followUp(agent, "Also include prices")
response = takeResponse(agent)
```

### Structured Messages (OpenAI Format)

```julia
promptAgent(agent, Dict("role" => "user", "content" => "Hello"))
response = takeResponse(agent)
```

### Output Format

`takeResponse()` returns a `Dict{String, Any}` in OpenAI message format:

```julia
Dict(
    "role" => "assistant",
    "content" => [
        Dict("type" => "text", "text" => "The weather in Tokyo is Sunny, 22°C."),
        # ... more content blocks
    ]
)
```

Content block types:

| Block Type | Fields | Description |
|------------|--------|-------------|
| `text` | `"type" => "text"`, `"text" => "..."` | Plain text response from the LLM |
| `image_url` | `"type" => "image_url"`, `"image_url" => Dict("url" => "...")` | Image content (rare, used by multimodal models) |

Example — extracting the text response:

```julia
response = takeResponse(agent)
text_response = first(c["text"] for c in response["content"] if c["type"] == "text")
```

### Multimodal Messages (Text + Image)

```julia
# Encode images to data URIs
image_path = "test/image.png"
image_bytes = read(image_path)
base64_string = base64encode(image_bytes)
data_uri = "data:image/png;base64,$(base64_string)"

# Build message with text and image content blocks
message = Dict(
    "role" => "user",
    "content" => [
        Dict("type" => "text", "text" => "Do you have Brunello di Montalcino?"),
        Dict("type" => "image_url", "image_url" => Dict("url" => data_uri))
    ]
)
promptAgent(agent, message)
response = takeResponse(agent)
```

### Custom Hooks

```julia
# beforeToolCall — request user approval for destructive tools
function my_before_tool_call(context::beforeToolCallContext, signal::abortSignal)::beforeToolCallResult
    if context.toolCall.name == "deleteFile"
        return beforeToolCallResult(true, "User must approve file deletion")
    end
    return beforeToolCallResult(false, "N/A")
end

# afterToolCall — mask sensitive data from results
function my_after_tool_call(context::afterToolCallContext, signal::abortSignal)::Union{agentToolResult, Nothing}
    if context.isError
        return agentToolResult(
            [textContent("The tool encountered an error. Please try again.")],
            context.result.details, context.result.usage, context.result.terminate
        )
    end
    return nothing
end

agent = yiemAgent(llmCall;
    mcpServer=mcpServer,
    beforeToolCall=my_before_tool_call,
    afterToolCall=my_after_tool_call,
    eventSink=println
)
```

### Custom LLM Callable

```julia
function my_llm_call(messages_dict::Dict{String, Any})
    # messages_dict contains "messages" key with OpenAI-format message array,
    # plus optional "tools", "temperature", etc.

    # Call your LLM provider (OpenAI, Anthropic, local model, etc.)
    raw_response = call_my_llm_provider(messages_dict)

    # Return a dict with these fields:
    return Dict(
        "message" => Dict("role" => "assistant", "content" => raw_response["content"]),
        "finish_reason" => raw_response["finish_reason"],
        "api" => "my_provider",
        "provider" => "my_provider",
        "model" => "my-model-name",
        "usage" => llmUsage(100, 50)
    )
end
```

### Custom Context Preparation

```julia
function my_prepare_context(state::agentState, eventSink)::agentContext
    sysPrompt = state.systemPrompt * "\nCurrent time: $(now())"
    msgs = deepcopy(state.messages)

    # Prune old messages to fit context window
    max_messages = 20
    if length(msgs) > max_messages
        msgs = msgs[end-max_messages+1:end]
    end

    return agentContext(sysPrompt, msgs, state.tools)
end

agent = yiemAgent(llmCall; prepareContext=my_prepare_context, ...)
```

## MCP Server Integration

YiemAgent uses JSON-RPC 2.0 for MCP communication. The `mcpServer` callable struct must implement `tools/list` and `tools/call` methods.

### Protocol

```julia
# tools/list — returns tool definitions
mcpServer("tools/list")
# → Dict("jsonrpc"=>"2.0", "id"=>1, "result"=>Dict("tools"=>[...], "nextCursor"=>...))

# tools/call — executes a tool
mcpServer("tools/call", toolName, arguments)
# → Dict("jsonrpc"=>"2.0", "id"=>2, "result"=>Dict("content"=>[...], "isError"=>...))

# Protocol error responses include "error" instead of "result":
Dict("jsonrpc"=>"2.0", "id"=>2, "error"=>Dict("code"=>-32602, "message"=>"..."))
```

### Example: NATS-based MCP Server Client

```julia
struct mcpServer
    natsConn::NATS.Connection
    topic::String
    senderID::String
    fileserver_url::String
end

# tools/list implementation
function (c::mcpServer)(method::String)
    payload = Dict("jsonrpc" => "2.0", "id" => 1, "method" => method, "params" => Dict{String, Any}())
    payloads = [("payload", payload, "dictionary")]
    _, msg_envelope = msghandler.smartpack(c.topic, payloads;
        sender_id=c.senderID, msg_purpose="mcp_tools_list", fileserver_url=c.fileserver_url)
    reply = NATS.request(c.natsConn, c.topic, msg_envelope, timeout=180)
    incoming = msghandler.smartunpack(String(reply.payload))
    return incoming["payloads"][1][2]
end

# tools/call implementation
function (c::mcpServer)(method::String, toolName::String, arguments::Dict{String, Any})
    payload = Dict("jsonrpc" => "2.0", "id" => 2, "method" => method,
        "params" => Dict("name" => toolName, "arguments" => arguments))
    payloads = [("payload", payload, "dictionary")]
    _, msg_envelope = msghandler.smartpack(c.topic, payloads;
        sender_id=c.senderID, msg_purpose="mcp_tool_call", fileserver_url=c.fileserver_url)
    reply = NATS.request(c.natsConn, c.topic, msg_envelope, timeout=180)
    incoming = msghandler.smartunpack(String(reply.payload))
    return incoming["payloads"][1][2]
end
```

## Tool Discovery

Tools are discovered dynamically via the MCP server:

1. `listTools` is the only pre-registered tool at agent creation
2. When the LLM calls `listTools()`, tools are discovered from the MCP server and registered at runtime
3. Supports pagination via `nextCursor` for large tool sets
4. New tools become immediately available without restarting the agent

## Hooks

Hooks allow customization of the agent's behavior at key points in the execution pipeline.

| Hook | Signature | Purpose |
|------|-----------|---------|
| `prepareContext` | `(state::agentState, eventSink) -> agentContext` | Transform messages/context before LLM call (filter tools, inject context, prune messages) |
| `formatMsgForLLM` | `(ctx::agentContext, eventSink) -> Dict` | Convert agent context to LLM API message format |
| `beforeToolCall` | `(context::beforeToolCallContext, signal::abortSignal) -> beforeToolCallResult` | Block/allow tool execution; request user approval |
| `afterToolCall` | `(context::afterToolCallContext, signal::abortSignal) -> Union{agentToolResult, Nothing}` | Post-process tool results; mask sensitive data; set `terminate` flag |
| `eventSink` | `(msg) -> nothing` | Callback for agent events/debug messages throughout the pipeline |

## Agent Lifecycle

1. **Creation** — `yiemAgent()` spawns a background task (`_agentLoop`) listening on `inputChannel` and `followUpChannel`
2. **Input** — User messages enter via `promptAgent()` (immediate) or `followUp()` (queued while agent is busy)
3. **Processing** — `_processMessage()` handles LLM calls, tool execution, and conversation history in a loop
4. **Tool Execution Pipeline** — each tool call follows three phases:
   - `prepareToolCall` — resolve tool, validate arguments, run `beforeToolCall` hook
   - `executePreparedToolCall` — invoke `tool.execute()`
   - `finalizeExecutedToolCall` — run `afterToolCall` hook, construct `toolResultMessage`
5. **Termination** — tool `execute` functions can set `terminate=true` to stop the agent loop
6. **Shutdown** — `stopAgent()` sends a `:shutdown` signal, drains channels, and closes the agent

## Architecture

```
src/
├── YiemAgent.jl          # Module entry point
├── type.jl               # Core types (messages, tools, agent state, events)
├── utils.jl              # Message formatting, context preparation, validation
├── toolRegistry.jl       # Tool store, MCP discovery, listTools registration
├── agentCore.jl          # Agent loop, tool execution pipeline, yiemAgent struct
└── api.jl                # Public API (promptAgent, takeResponse, followUp, stopAgent)
```