YiemAgent

Julia framework for building agents with tool use and MCP (Model Context Protocol) support.

Quick Start

using YiemAgent

# 1. Create an LLM callable (see "Custom LLM Callable" below)
llmCall = ...

# 2. Create a callable MCP server (see "MCP Server Integration" below)
mcpServer = ...

# 3. Create the agent
agent = yiemAgent(llmCall; mcpServer=mcpServer, eventSink=println)

# 4. Interact
promptAgent(agent, "What tools do you have?")
response = takeResponse(agent)

# 5. Stop when done
stopAgent(agent)

Agent API

Function	Description
promptAgent(agent, text)	Send plain text to the agent's input channel
promptAgent(agent, msg)	Send a structured message (OpenAI format dict) to the input channel
takeResponse(agent)	Block and take the agent's response from output channel
followUp(agent, text)	Send plain text follow-up while agent is still processing
followUp(agent, msg)	Send a structured follow-up (OpenAI format dict) while agent is processing
stopAgent(agent)	Gracefully stop the agent and close channels

Creating a yiemAgent

yiemAgent(llmCall;
    systemPrompt::String = "You are a helpful assistant.",
    model = nothing,
    messages::Vector{agentMessage} = agentMessage[],
    prepareContext::Function = prepareContext,
    formatMsgForLLM::Function = formatMsgForLLM,
    beforeToolCall::Function = beforeToolCall,
    afterToolCall::Function = afterToolCall,
    sessionId::Union{String, Nothing} = nothing,
    maxRetryDelayMs::Union{Int64, Nothing} = nothing,
    parallelToolExecute::Bool = false,
    eventSink::Function = eventSink,
    mcpServer = nothing
)

Parameter	Description
llmCall	(required) Function that receives formatted messages and returns an LLM response
systemPrompt	System prompt for the agent
model	LLM model configuration (llmModel struct)
messages	Initial conversation messages
prepareContext	Hook to transform messages/context before each LLM call
formatMsgForLLM	Hook to convert agent context to LLM API format
beforeToolCall	Hook to block/allow tool execution
afterToolCall	Hook to post-process tool results
sessionId	Optional session identifier for MCP server
maxRetryDelayMs	Maximum retry delay in milliseconds
parallelToolExecute	Run multiple tool calls in parallel
eventSink	Callback for agent events/debug messages
mcpServer	Callable struct for MCP server communication

Usage Examples

Text Messages

# Send plain text
promptAgent(agent, "What tools do you have?")
response = takeResponse(agent)

# Follow-up while agent is processing
promptAgent(agent, "Tell me about wine")
followUp(agent, "Also include prices")
response = takeResponse(agent)

Structured Messages (OpenAI Format)

promptAgent(agent, Dict("role" => "user", "content" => "Hello"))
response = takeResponse(agent)

Output Format

takeResponse() returns a Dict{String, Any} in OpenAI message format:

Dict(
    "role" => "assistant",
    "content" => [
        Dict("type" => "text", "text" => "The weather in Tokyo is Sunny, 22°C."),
        # ... more content blocks
    ]
)

Content block types:

Block Type	Fields	Description
text	"type" => "text", "text" => "..."	Plain text response from the LLM
image_url	"type" => "image_url", "image_url" => Dict("url" => "...")	Image content (rare, used by multimodal models)

Example — extracting the text response:

response = takeResponse(agent)
text_response = first(c["text"] for c in response["content"] if c["type"] == "text")

Multimodal Messages (Text + Image)

# Encode images to data URIs
image_path = "test/image.png"
image_bytes = read(image_path)
base64_string = base64encode(image_bytes)
data_uri = "data:image/png;base64,$(base64_string)"

# Build message with text and image content blocks
message = Dict(
    "role" => "user",
    "content" => [
        Dict("type" => "text", "text" => "Do you have Brunello di Montalcino?"),
        Dict("type" => "image_url", "image_url" => Dict("url" => data_uri))
    ]
)
promptAgent(agent, message)
response = takeResponse(agent)

Custom Hooks

# beforeToolCall — request user approval for destructive tools
function my_before_tool_call(context::beforeToolCallContext, signal::abortSignal)::beforeToolCallResult
    if context.toolCall.name == "deleteFile"
        return beforeToolCallResult(true, "User must approve file deletion")
    end
    return beforeToolCallResult(false, "N/A")
end

# afterToolCall — mask sensitive data from results
function my_after_tool_call(context::afterToolCallContext, signal::abortSignal)::Union{agentToolResult, Nothing}
    if context.isError
        return agentToolResult(
            [textContent("The tool encountered an error. Please try again.")],
            context.result.details, context.result.usage, context.result.terminate
        )
    end
    return nothing
end

agent = yiemAgent(llmCall;
    mcpServer=mcpServer,
    beforeToolCall=my_before_tool_call,
    afterToolCall=my_after_tool_call,
    eventSink=println
)

Custom LLM Callable

function my_llm_call(messages_dict::Dict{String, Any})
    # messages_dict contains "messages" key with OpenAI-format message array,
    # plus optional "tools", "temperature", etc.

    # Call your LLM provider (OpenAI, Anthropic, local model, etc.)
    raw_response = call_my_llm_provider(messages_dict)

    # Return a dict with these fields:
    return Dict(
        "message" => Dict("role" => "assistant", "content" => raw_response["content"]),
        "finish_reason" => raw_response["finish_reason"],
        "api" => "my_provider",
        "provider" => "my_provider",
        "model" => "my-model-name",
        "usage" => llmUsage(100, 50)
    )
end

Custom Context Preparation

function my_prepare_context(state::agentState, eventSink)::agentContext
    sysPrompt = state.systemPrompt * "\nCurrent time: $(now())"
    msgs = deepcopy(state.messages)

    # Prune old messages to fit context window
    max_messages = 20
    if length(msgs) > max_messages
        msgs = msgs[end-max_messages+1:end]
    end

    return agentContext(sysPrompt, msgs, state.tools)
end

agent = yiemAgent(llmCall; prepareContext=my_prepare_context, ...)

MCP Server Integration

YiemAgent uses JSON-RPC 2.0 for MCP communication. The mcpServer callable struct must implement tools/list and tools/call methods.

Protocol

# tools/list — returns tool definitions
mcpServer("tools/list")
# → Dict("jsonrpc"=>"2.0", "id"=>1, "result"=>Dict("tools"=>[...], "nextCursor"=>...))

# tools/call — executes a tool
mcpServer("tools/call", toolName, arguments)
# → Dict("jsonrpc"=>"2.0", "id"=>2, "result"=>Dict("content"=>[...], "isError"=>...))

# Protocol error responses include "error" instead of "result":
Dict("jsonrpc"=>"2.0", "id"=>2, "error"=>Dict("code"=>-32602, "message"=>"..."))

Example: NATS-based MCP Server Client

struct mcpServer
    natsConn::NATS.Connection
    topic::String
    senderID::String
    fileserver_url::String
end

# tools/list implementation
function (c::mcpServer)(method::String)
    payload = Dict("jsonrpc" => "2.0", "id" => 1, "method" => method, "params" => Dict{String, Any}())
    payloads = [("payload", payload, "dictionary")]
    _, msg_envelope = msghandler.smartpack(c.topic, payloads;
        sender_id=c.senderID, msg_purpose="mcp_tools_list", fileserver_url=c.fileserver_url)
    reply = NATS.request(c.natsConn, c.topic, msg_envelope, timeout=180)
    incoming = msghandler.smartunpack(String(reply.payload))
    return incoming["payloads"][1][2]
end

# tools/call implementation
function (c::mcpServer)(method::String, toolName::String, arguments::Dict{String, Any})
    payload = Dict("jsonrpc" => "2.0", "id" => 2, "method" => method,
        "params" => Dict("name" => toolName, "arguments" => arguments))
    payloads = [("payload", payload, "dictionary")]
    _, msg_envelope = msghandler.smartpack(c.topic, payloads;
        sender_id=c.senderID, msg_purpose="mcp_tool_call", fileserver_url=c.fileserver_url)
    reply = NATS.request(c.natsConn, c.topic, msg_envelope, timeout=180)
    incoming = msghandler.smartunpack(String(reply.payload))
    return incoming["payloads"][1][2]
end

Tool Discovery

Tools are discovered dynamically via the MCP server:

1. listTools is the only pre-registered tool at agent creation
2. When the LLM calls listTools(), tools are discovered from the MCP server and registered at runtime
3. Supports pagination via nextCursor for large tool sets
4. New tools become immediately available without restarting the agent

Hooks

Hooks allow customization of the agent's behavior at key points in the execution pipeline.

Hook	Signature	Purpose
prepareContext	(state::agentState, eventSink) -> agentContext	Transform messages/context before LLM call (filter tools, inject context, prune messages)
formatMsgForLLM	(ctx::agentContext, eventSink) -> Dict	Convert agent context to LLM API message format
beforeToolCall	(context::beforeToolCallContext, signal::abortSignal) -> beforeToolCallResult	Block/allow tool execution; request user approval
afterToolCall	(context::afterToolCallContext, signal::abortSignal) -> Union{agentToolResult, Nothing}	Post-process tool results; mask sensitive data; set terminate flag
eventSink	(msg) -> nothing	Callback for agent events/debug messages throughout the pipeline

Agent Lifecycle

1. Creation — yiemAgent() spawns a background task (_agentLoop) listening on inputChannel and followUpChannel
2. Input — User messages enter via promptAgent() (immediate) or followUp() (queued while agent is busy)
3. Processing — _processMessage() handles LLM calls, tool execution, and conversation history in a loop
4. Tool Execution Pipeline — each tool call follows three phases:
   • prepareToolCall — resolve tool, validate arguments, run beforeToolCall hook
   • executePreparedToolCall — invoke tool.execute()
   • finalizeExecutedToolCall — run afterToolCall hook, construct toolResultMessage
5. Termination — tool execute functions can set terminate=true to stop the agent loop
6. Shutdown — stopAgent() sends a :shutdown signal, drains channels, and closes the agent

Architecture

src/
├── YiemAgent.jl          # Module entry point
├── type.jl               # Core types (messages, tools, agent state, events)
├── utils.jl              # Message formatting, context preparation, validation
├── toolRegistry.jl       # Tool store, MCP discovery, listTools registration
├── agentCore.jl          # Agent loop, tool execution pipeline, yiemAgent struct
└── api.jl                # Public API (promptAgent, takeResponse, followUp, stopAgent)