update
This commit is contained in:
@@ -2,138 +2,310 @@
|
||||
|
||||
Julia framework for building agents with tool use and MCP (Model Context Protocol) support.
|
||||
|
||||
## Getting Started
|
||||
## Quick Start
|
||||
|
||||
1. Install dependencies: `]add JSON, DataStructures, UUIDs, Dates, NATS, DataFrames`
|
||||
2. Create a callable `mcpServer` struct that communicates with an MCP server via NATS
|
||||
3. Create a `yiemAgent` with an LLM callable and MCP server:
|
||||
```julia
|
||||
agent = YiemAgent.yiemAgent(llmCall; mcpServer=mcpServer, eventSink=yourSink)
|
||||
```
|
||||
4. Call `promptAgent(agent, message)` then `takeResponse(agent)`
|
||||
```julia
|
||||
using YiemAgent
|
||||
|
||||
# 1. Create an LLM callable (see "Custom LLM Callable" below)
|
||||
llmCall = ...
|
||||
|
||||
# 2. Create a callable MCP server (see "MCP Server Integration" below)
|
||||
mcpServer = ...
|
||||
|
||||
# 3. Create the agent
|
||||
agent = yiemAgent(llmCall; mcpServer=mcpServer, eventSink=println)
|
||||
|
||||
# 4. Interact
|
||||
promptAgent(agent, "What tools do you have?")
|
||||
response = takeResponse(agent)
|
||||
|
||||
# 5. Stop when done
|
||||
stopAgent(agent)
|
||||
```
|
||||
|
||||
## Agent API
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `promptAgent(agent, text)` | Send plain text to the agent's input channel |
|
||||
| `promptAgent(agent, msg)` | Send a structured message (OpenAI format dict) to the input channel |
|
||||
| `takeResponse(agent)` | Block and take the agent's response from output channel |
|
||||
| `followUp(agent, text)` | Send plain text follow-up while agent is still processing |
|
||||
| `followUp(agent, msg)` | Send a structured follow-up (OpenAI format dict) while agent is processing |
|
||||
| `stopAgent(agent)` | Gracefully stop the agent and close channels |
|
||||
|
||||
## Creating a yiemAgent
|
||||
|
||||
```julia
|
||||
yiemAgent(llmCall;
|
||||
systemPrompt::String = "You are a helpful assistant.",
|
||||
model = nothing,
|
||||
messages::Vector{agentMessage} = agentMessage[],
|
||||
prepareContext::Function = prepareContext,
|
||||
formatMsgForLLM::Function = formatMsgForLLM,
|
||||
beforeToolCall::Function = beforeToolCall,
|
||||
afterToolCall::Function = afterToolCall,
|
||||
sessionId::Union{String, Nothing} = nothing,
|
||||
maxRetryDelayMs::Union{Int64, Nothing} = nothing,
|
||||
parallelToolExecute::Bool = false,
|
||||
eventSink::Function = eventSink,
|
||||
mcpServer = nothing
|
||||
)
|
||||
```
|
||||
|
||||
| Parameter | Description |
|
||||
|-----------|-------------|
|
||||
| `llmCall` | **(required)** Function that receives formatted messages and returns an LLM response |
|
||||
| `systemPrompt` | System prompt for the agent |
|
||||
| `model` | LLM model configuration (`llmModel` struct) |
|
||||
| `messages` | Initial conversation messages |
|
||||
| `prepareContext` | Hook to transform messages/context before each LLM call |
|
||||
| `formatMsgForLLM` | Hook to convert agent context to LLM API format |
|
||||
| `beforeToolCall` | Hook to block/allow tool execution |
|
||||
| `afterToolCall` | Hook to post-process tool results |
|
||||
| `sessionId` | Optional session identifier for MCP server |
|
||||
| `maxRetryDelayMs` | Maximum retry delay in milliseconds |
|
||||
| `parallelToolExecute` | Run multiple tool calls in parallel |
|
||||
| `eventSink` | Callback for agent events/debug messages |
|
||||
| `mcpServer` | Callable struct for MCP server communication |
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Text Messages
|
||||
|
||||
```julia
|
||||
# Send plain text
|
||||
promptAgent(agent, "What tools do you have?")
|
||||
response = takeResponse(agent)
|
||||
|
||||
# Follow-up while agent is processing
|
||||
promptAgent(agent, "Tell me about wine")
|
||||
followUp(agent, "Also include prices")
|
||||
response = takeResponse(agent)
|
||||
```
|
||||
|
||||
### Structured Messages (OpenAI Format)
|
||||
|
||||
```julia
|
||||
promptAgent(agent, Dict("role" => "user", "content" => "Hello"))
|
||||
response = takeResponse(agent)
|
||||
```
|
||||
|
||||
### Output Format
|
||||
|
||||
`takeResponse()` returns a `Dict{String, Any}` in OpenAI message format:
|
||||
|
||||
```julia
|
||||
Dict(
|
||||
"role" => "assistant",
|
||||
"content" => [
|
||||
Dict("type" => "text", "text" => "The weather in Tokyo is Sunny, 22°C."),
|
||||
# ... more content blocks
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
Content block types:
|
||||
|
||||
| Block Type | Fields | Description |
|
||||
|------------|--------|-------------|
|
||||
| `text` | `"type" => "text"`, `"text" => "..."` | Plain text response from the LLM |
|
||||
| `image_url` | `"type" => "image_url"`, `"image_url" => Dict("url" => "...")` | Image content (rare, used by multimodal models) |
|
||||
|
||||
Example — extracting the text response:
|
||||
|
||||
```julia
|
||||
response = takeResponse(agent)
|
||||
text_response = first(c["text"] for c in response["content"] if c["type"] == "text")
|
||||
```
|
||||
|
||||
### Multimodal Messages (Text + Image)
|
||||
|
||||
```julia
|
||||
# Encode images to data URIs
|
||||
image_path = "test/image.png"
|
||||
image_bytes = read(image_path)
|
||||
base64_string = base64encode(image_bytes)
|
||||
data_uri = "data:image/png;base64,$(base64_string)"
|
||||
|
||||
# Build message with text and image content blocks
|
||||
message = Dict(
|
||||
"role" => "user",
|
||||
"content" => [
|
||||
Dict("type" => "text", "text" => "Do you have Brunello di Montalcino?"),
|
||||
Dict("type" => "image_url", "image_url" => Dict("url" => data_uri))
|
||||
]
|
||||
)
|
||||
promptAgent(agent, message)
|
||||
response = takeResponse(agent)
|
||||
```
|
||||
|
||||
### Custom Hooks
|
||||
|
||||
```julia
|
||||
# beforeToolCall — request user approval for destructive tools
|
||||
function my_before_tool_call(context::beforeToolCallContext, signal::abortSignal)::beforeToolCallResult
|
||||
if context.toolCall.name == "deleteFile"
|
||||
return beforeToolCallResult(true, "User must approve file deletion")
|
||||
end
|
||||
return beforeToolCallResult(false, "N/A")
|
||||
end
|
||||
|
||||
# afterToolCall — mask sensitive data from results
|
||||
function my_after_tool_call(context::afterToolCallContext, signal::abortSignal)::Union{agentToolResult, Nothing}
|
||||
if context.isError
|
||||
return agentToolResult(
|
||||
[textContent("The tool encountered an error. Please try again.")],
|
||||
context.result.details, context.result.usage, context.result.terminate
|
||||
)
|
||||
end
|
||||
return nothing
|
||||
end
|
||||
|
||||
agent = yiemAgent(llmCall;
|
||||
mcpServer=mcpServer,
|
||||
beforeToolCall=my_before_tool_call,
|
||||
afterToolCall=my_after_tool_call,
|
||||
eventSink=println
|
||||
)
|
||||
```
|
||||
|
||||
### Custom LLM Callable
|
||||
|
||||
```julia
|
||||
function my_llm_call(messages_dict::Dict{String, Any})
|
||||
# messages_dict contains "messages" key with OpenAI-format message array,
|
||||
# plus optional "tools", "temperature", etc.
|
||||
|
||||
# Call your LLM provider (OpenAI, Anthropic, local model, etc.)
|
||||
raw_response = call_my_llm_provider(messages_dict)
|
||||
|
||||
# Return a dict with these fields:
|
||||
return Dict(
|
||||
"message" => Dict("role" => "assistant", "content" => raw_response["content"]),
|
||||
"finish_reason" => raw_response["finish_reason"],
|
||||
"api" => "my_provider",
|
||||
"provider" => "my_provider",
|
||||
"model" => "my-model-name",
|
||||
"usage" => llmUsage(100, 50)
|
||||
)
|
||||
end
|
||||
```
|
||||
|
||||
### Custom Context Preparation
|
||||
|
||||
```julia
|
||||
function my_prepare_context(state::agentState, eventSink)::agentContext
|
||||
sysPrompt = state.systemPrompt * "\nCurrent time: $(now())"
|
||||
msgs = deepcopy(state.messages)
|
||||
|
||||
# Prune old messages to fit context window
|
||||
max_messages = 20
|
||||
if length(msgs) > max_messages
|
||||
msgs = msgs[end-max_messages+1:end]
|
||||
end
|
||||
|
||||
return agentContext(sysPrompt, msgs, state.tools)
|
||||
end
|
||||
|
||||
agent = yiemAgent(llmCall; prepareContext=my_prepare_context, ...)
|
||||
```
|
||||
|
||||
## MCP Server Integration
|
||||
|
||||
YiemAgent uses JSON-RPC 2.0 for MCP communication. The `mcpServer` callable struct must implement `tools/list` and `tools/call` methods.
|
||||
|
||||
### Protocol
|
||||
|
||||
```julia
|
||||
# tools/list — returns tool definitions
|
||||
mcpServer("tools/list")
|
||||
# → Dict("jsonrpc"=>"2.0", "id"=>1, "result"=>Dict("tools"=>[...], "nextCursor"=>...))
|
||||
|
||||
# tools/call — executes a tool
|
||||
mcpServer("tools/call", toolName, arguments)
|
||||
# → Dict("jsonrpc"=>"2.0", "id"=>2, "result"=>Dict("content"=>[...], "isError"=>...))
|
||||
|
||||
# Protocol error responses include "error" instead of "result":
|
||||
Dict("jsonrpc"=>"2.0", "id"=>2, "error"=>Dict("code"=>-32602, "message"=>"..."))
|
||||
```
|
||||
|
||||
### Example: NATS-based MCP Server Client
|
||||
|
||||
```julia
|
||||
struct mcpServer
|
||||
natsConn::NATS.Connection
|
||||
topic::String
|
||||
senderID::String
|
||||
fileserver_url::String
|
||||
end
|
||||
|
||||
# tools/list implementation
|
||||
function (c::mcpServer)(method::String)
|
||||
payload = Dict("jsonrpc" => "2.0", "id" => 1, "method" => method, "params" => Dict{String, Any}())
|
||||
payloads = [("payload", payload, "dictionary")]
|
||||
_, msg_envelope = msghandler.smartpack(c.topic, payloads;
|
||||
sender_id=c.senderID, msg_purpose="mcp_tools_list", fileserver_url=c.fileserver_url)
|
||||
reply = NATS.request(c.natsConn, c.topic, msg_envelope, timeout=180)
|
||||
incoming = msghandler.smartunpack(String(reply.payload))
|
||||
return incoming["payloads"][1][2]
|
||||
end
|
||||
|
||||
# tools/call implementation
|
||||
function (c::mcpServer)(method::String, toolName::String, arguments::Dict{String, Any})
|
||||
payload = Dict("jsonrpc" => "2.0", "id" => 2, "method" => method,
|
||||
"params" => Dict("name" => toolName, "arguments" => arguments))
|
||||
payloads = [("payload", payload, "dictionary")]
|
||||
_, msg_envelope = msghandler.smartpack(c.topic, payloads;
|
||||
sender_id=c.senderID, msg_purpose="mcp_tool_call", fileserver_url=c.fileserver_url)
|
||||
reply = NATS.request(c.natsConn, c.topic, msg_envelope, timeout=180)
|
||||
incoming = msghandler.smartunpack(String(reply.payload))
|
||||
return incoming["payloads"][1][2]
|
||||
end
|
||||
```
|
||||
|
||||
## Tool Discovery
|
||||
|
||||
Tools are discovered dynamically via the MCP server:
|
||||
|
||||
1. `listTools` is the only pre-registered tool at agent creation
|
||||
2. When the LLM calls `listTools()`, tools are discovered from the MCP server and registered at runtime
|
||||
3. Supports pagination via `nextCursor` for large tool sets
|
||||
4. New tools become immediately available without restarting the agent
|
||||
|
||||
## Hooks
|
||||
|
||||
Hooks allow customization of the agent's behavior at key points in the execution pipeline.
|
||||
|
||||
| Hook | Signature | Purpose |
|
||||
|------|-----------|---------|
|
||||
| `prepareContext` | `(state::agentState, eventSink) -> agentContext` | Transform messages/context before LLM call (filter tools, inject context, prune messages) |
|
||||
| `formatMsgForLLM` | `(ctx::agentContext, eventSink) -> Dict` | Convert agent context to LLM API message format |
|
||||
| `beforeToolCall` | `(context::beforeToolCallContext, signal::abortSignal) -> beforeToolCallResult` | Block/allow tool execution; request user approval |
|
||||
| `afterToolCall` | `(context::afterToolCallContext, signal::abortSignal) -> Union{agentToolResult, Nothing}` | Post-process tool results; mask sensitive data; set `terminate` flag |
|
||||
| `eventSink` | `(msg) -> nothing` | Callback for agent events/debug messages throughout the pipeline |
|
||||
|
||||
## Agent Lifecycle
|
||||
|
||||
1. **Creation** — `yiemAgent()` spawns a background task (`_agentLoop`) listening on `inputChannel` and `followUpChannel`
|
||||
2. **Input** — User messages enter via `promptAgent()` (immediate) or `followUp()` (queued while agent is busy)
|
||||
3. **Processing** — `_processMessage()` handles LLM calls, tool execution, and conversation history in a loop
|
||||
4. **Tool Execution Pipeline** — each tool call follows three phases:
|
||||
- `prepareToolCall` — resolve tool, validate arguments, run `beforeToolCall` hook
|
||||
- `executePreparedToolCall` — invoke `tool.execute()`
|
||||
- `finalizeExecutedToolCall` — run `afterToolCall` hook, construct `toolResultMessage`
|
||||
5. **Termination** — tool `execute` functions can set `terminate=true` to stop the agent loop
|
||||
6. **Shutdown** — `stopAgent()` sends a `:shutdown` signal, drains channels, and closes the agent
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
src/
|
||||
├── YiemAgent.jl # Module entry point
|
||||
├── type.jl # Core types (messages, tools, agent state)
|
||||
├── type.jl # Core types (messages, tools, agent state, events)
|
||||
├── utils.jl # Message formatting, context preparation, validation
|
||||
├── agentCore.jl # Agent loop, tool execution pipeline
|
||||
├── api.jl # Public API (promptAgent, takeResponse, followUp, stopAgent)
|
||||
└── toolRegistry.jl # Tool store, MCP discovery, listTools registration
|
||||
├── toolRegistry.jl # Tool store, MCP discovery, listTools registration
|
||||
├── agentCore.jl # Agent loop, tool execution pipeline, yiemAgent struct
|
||||
└── api.jl # Public API (promptAgent, takeResponse, followUp, stopAgent)
|
||||
```
|
||||
|
||||
## MCP Protocol
|
||||
|
||||
YiemAgent uses JSON-RPC 2.0 for MCP communication. The `mcpServer` callable struct must implement:
|
||||
|
||||
```julia
|
||||
# tools/list — returns tool definitions
|
||||
mcpServer("tools/list") # → Dict("jsonrpc"=>"2.0", "id"=>1, "result"=>Dict("tools"=>[...], "nextCursor"=>...))
|
||||
|
||||
# tools/call — executes a tool
|
||||
mcpServer("tools/call", toolName, arguments) # → Dict("jsonrpc"=>"2.0", "id"=>2, "result"=>Dict("content"=>[...], "isError"=>...))
|
||||
```
|
||||
|
||||
Protocol error responses include `"error"` instead of `"result"`:
|
||||
```julia
|
||||
Dict("jsonrpc"=>"2.0", "id"=>2, "error"=>Dict("code"=>-32602, "message"=>"..."))
|
||||
```
|
||||
|
||||
### Implementing the MCP Server Client
|
||||
|
||||
You must provide a callable struct that communicates with your MCP server. Example using NATS:
|
||||
|
||||
```julia
|
||||
struct mcpServer
|
||||
natsConn::NATS.Connection
|
||||
topic::String
|
||||
senderID::String
|
||||
fileserver_url::String
|
||||
end
|
||||
|
||||
# tools/list implementation
|
||||
function (c::mcpServer)(method::String)
|
||||
if method != "tools/list"
|
||||
error("mcpServer: unexpected method '$method' (expected 'tools/list')")
|
||||
end
|
||||
|
||||
payload = Dict("jsonrpc" => "2.0", "id" => 1, "method" => method, "params" => Dict{String, Any}())
|
||||
payloads = [("payload", payload, "dictionary")]
|
||||
_, msg_envelope_json_str = msghandler.smartpack(
|
||||
c.topic, payloads;
|
||||
sender_id=c.senderID,
|
||||
msg_purpose="mcp_tools_list",
|
||||
fileserver_url=c.fileserver_url)
|
||||
|
||||
reply = NATS.request(c.natsConn, c.topic, msg_envelope_json_str, timeout=180)
|
||||
incoming_env = msghandler.smartunpack(String(reply.payload))
|
||||
return incoming_env["payloads"][1][2]
|
||||
end
|
||||
|
||||
# tools/call implementation
|
||||
function (c::mcpServer)(method::String, toolName::String, arguments::Dict{String, Any})
|
||||
if method != "tools/call"
|
||||
error("mcpServer: unexpected method '$method' (expected 'tools/call')")
|
||||
end
|
||||
|
||||
payload = Dict(
|
||||
"jsonrpc" => "2.0",
|
||||
"id" => 2,
|
||||
"method" => method,
|
||||
"params" => Dict(
|
||||
"name" => toolName,
|
||||
"arguments" => arguments
|
||||
)
|
||||
)
|
||||
payloads = [("payload", payload, "dictionary")]
|
||||
_, msg_envelope_json_str = msghandler.smartpack(
|
||||
c.topic, payloads;
|
||||
sender_id=c.senderID,
|
||||
msg_purpose="mcp_tool_call",
|
||||
fileserver_url=c.fileserver_url)
|
||||
|
||||
reply = NATS.request(c.natsConn, c.topic, msg_envelope_json_str, timeout=180)
|
||||
incoming_env = msghandler.smartunpack(String(reply.payload))
|
||||
return incoming_env["payloads"][1][2]
|
||||
end
|
||||
```
|
||||
|
||||
See `test/runtest.jl` for the complete working example.
|
||||
|
||||
## Tool Discovery
|
||||
|
||||
Tools are discovered dynamically via MCP server:
|
||||
|
||||
1. `listTool` is the only pre-registered tool
|
||||
2. When the LLM calls `listTools()`, tools are discovered from the MCP server and registered at runtime
|
||||
3. Supports pagination via `nextCursor` for large tool sets
|
||||
|
||||
## Agent API
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `promptAgent(agent, msg)` | Send a message to the agent's input channel |
|
||||
| `takeResponse(agent)` | Block and take the agent's response from output channel |
|
||||
| `followUp(agent, msg)` | Send a follow-up message while agent is still processing |
|
||||
| `stopAgent(agent)` | Gracefully stop the agent and close channels |
|
||||
|
||||
## Agent Lifecycle
|
||||
|
||||
1. `yiemAgent()` spawns a background task (`_agentLoop`) listening on `inputChannel` and `followUpChannel`
|
||||
2. User messages enter via `promptAgent()` or `followUp()`
|
||||
3. `_processMessage()` handles LLM calls, tool execution, and conversation history
|
||||
4. Tool execution follows three phases: `prepareToolCall` → `executePreparedToolCall` → `finalizeExecutedToolCall`
|
||||
5. `beforeToolCall`/`afterToolCall` hooks allow pre/post-processing of tool calls
|
||||
6. Tool `execute` functions can set `terminate=true` to stop the agent loop
|
||||
|
||||
## Hooks
|
||||
|
||||
| Hook | Signature | Purpose |
|
||||
|------|-----------|---------|
|
||||
| `prepareContext` | `(state, sink, llmCall) -> ctx` | Transform messages/context before LLM call |
|
||||
| `formatMsgForLLM` | `(ctx, sink) -> dict` | Convert agent context to LLM API format |
|
||||
| `beforeToolCall` | `(context, signal) -> result` | Block/allow tool execution |
|
||||
| `afterToolCall` | `(context, signal) -> result` | Post-process tool results |
|
||||
| `eventSink` | `(msg) -> nothing` | Callback for agent events/debug messages |
|
||||
|
||||
Reference in New Issue
Block a user