YiemAgent/docs/2-solution_design.md

Solution Design: YiemAgent Agent Framework

1. Problem Decomposition

Problem	Description	User Impact
P-001	Developers need a chatbot framework in Julia that handles LLM calls and tool calls without writing boilerplate	Must write separate request/response handling, tool call parsing, and loop management code for each agent
P-002	Tool discovery is manual and repetitive	Must register every tool individually; no automatic discovery from MCP-compatible servers
P-003	Multiple LLM providers have different response formats	Must handle message.tool_calls (LMStudio/vLLM) vs content blocks (OpenAI API) separately
P-004	Tool execution needs lifecycle hooks for observability and control	No built-in before/after hooks for authorization, logging, or result sanitization
P-005	Agent state management is error-prone	Must handle message history, tool registration, and pending tool call tracking manually

2. Solution Approach

Approach: "Build a channel-based agent system where the LLM talks to tools automatically, tools talk to MCP servers automatically, and you compose agents by injecting llmCall and mcpServer functions — the framework owns the loop, the state, and the format conversion."

Key Principles:

• Channel-based async processing: Each agent owns three channels (inputChannel, followUpChannel, outputChannel); a background @spawn task runs the agent loop, processing messages in priority order
• Tool call loop: The agent calls llmCall(), extracts tool calls, executes them (sequentially or in parallel), appends results, and calls llmCall() again — repeating until no tool calls remain
• Pluggable hooks: prepareContext, formatMsgForLLM, beforeToolCall, afterToolCall, and llmCall are all injectable functions — the framework provides sensible defaults, but you override what you need
• MCP-native tool discovery: Tools are discovered at startup via MCP tools/list and can be discovered at runtime via the listTools builtin tool (idempotent registration with pagination support)
• Format-agnostic LLM responses: _extractToolCalls() handles both OpenAI content blocks and LMStudio/vLLM nested message format, allowing the same agent to work with any model provider

3. Decision Rationale

Decision ID	Decision	Rationale	Alternatives Rejected
SD-001	Use channel-based async processing with @spawn task	Provides non-blocking message queues; inputChannel blocks if full (backpressure); followUpChannel allows async follow-ups while agent is busy	Threading with shared state — race conditions; callback-based async — hard to compose
SD-002	Background _agentLoop task runs message dispatch loop	Isolates message processing from construction; agent can be created and used concurrently	Synchronous agent.run(message) — blocks caller; manual task management — boilerplate
SD-003	_processMessage runs llmCall() in an inner loop (while loop) until no tool calls remain	Handles multi-turn tool use transparently — agent decides when to stop based on LLM response	External loop in caller code — duplicates agent loop logic; single-turn only — no multi-step tool use
SD-004	MCP server integration via callable struct mcpServer(method, args...)	MCP is model-agnostic protocol; callable struct allows type-stable single dispatch; supports NATS transport (struct has natsConn, topic, senderID fields)	Ad-hoc HTTP client for MCP — breaks NATS-only transports; tight coupling to specific transport
SD-005	Tool call lifecycle: prepareToolCall() → executePreparedToolCall() → finalizeExecutedToolCall() — three phases, never throw errors	Preparation validates args and runs beforeToolCall hook (can block); execution calls tool.execute(); finalization runs afterToolCall hook. Each phase returns a result type, not exceptions — agent loop catches all and builds error tool result	try/catch in loop — loses context; throwing errors breaks tool result message contract
SD-006	Message types are Julia structs with keyword constructors (userMessage(content=[...]))	Type stable; keyword constructors provide defaults (role="user", timestamp=now()); supports both positional and keyword usage	Dict{String, Any} for messages — lose type safety, docgen, IDE support
SD-007	eventSink function passed to all deep lifecycle calls	Provides hooks for external monitoring (logging, metrics, NATS pub/sub) without modifying core logic	Built-in println — can't disable or redirect; no metrics integration
SD-008	abortSignal struct with .aborted::Bool field passed to prepareToolCall() and hooks	Simple mutable state for interrupting long-running tool calls (no async cancellation — just flag check)	Threading with Task.cancel() — Julia thread model doesn't support it; @async spawn — complex cleanup
SD-009	_extractToolCalls() supports both message.tool_calls and content[{type="tool_calls"}] formats	LMStudio.jl/vLLM vs OpenAI API have different response shapes; framework must work with both	Separate parser per provider — maintenance burden; require provider wrapper — adds indirection

4. High-Level Component Diagram

flowchart TD
    User[External Caller] -->|promptAgent / followUp| InputCh[inputChannel]
    
    subagent AgentLoop [Agent Loop Task]
        InputCh -->|take!| Disp[Message Dispatcher]
        Disp -->|spawn| Proc[_processMessage]
        Proc -->|Channel| PrCh[processMessageInputCh]
    end
    
    subagent Message Pipeline
        PrCh -->|put!| DMsg[Drain input, convert to userMessage]
        DMsg --> PC[prepareContext hook]
        PC --> FM[formatMsgForLLM hook]
        FM --> LLM[llmCall]
        LLM --> EC[_extractToolCalls]
        EC -->|hasToolCalls| TC[executeToolCalls]
        TC -->|toolResultBatch| TM[toolResultMessage]
        TM -->|push!| MH[messageHistory]
        TC -->|no tool calls break| BRK[break loop]
        BRK --> End[exit]
    end
    
    subagent Tool System
        TC --> Prep[prepareToolCall]
        Prep --> Exec[executePreparedToolCall]
        Exec --> Term[finalizeExecutedToolCall]
        Term --> CRM[createToolResultMessage]
        CRM --> TM
    end
    
    subagent MCP Integration
        Prep --> ToolLookup[context.tools lookup]
        ToolLookup --> MCP[MCP tools/list + tools/call]
        MCP --> Wr[_wrap_mcp_tool]
        Wr --> Reg[toolRegistry.registerAllTools]
    end
    
    OutputCh[outputChannel] --> User
    Proc -->|final response| OutputCh

Component Descriptions:

• Agent Loop Task (SD-001, SD-002): Runs @spawn _agentLoop(); manages three channels; dispatches messages to _processMessage
• Message Pipeline (SD-003, SD-004): prepareContext → formatMsgForLLM → llmCall → extract tool calls → loop
• Tool System (SD-005): Three-phase lifecycle (prepare → execute → finalize); never throws; returns error tool result on failure
• MCP Integration (SD-004): tools/list discovery with pagination; _wrap_mcp_tool creates executable agentTool with MCP JSON-RPC calls

5. Alternatives Considered

Alternative	Pros	Cons	Decision
Synchronous agent.process(message)	Simpler API; no channels or tasks	Blocks caller; no follow-up support; harder to test concurrency	Rejected — async channels needed for real-time chat UX
JSON dicts for messages everywhere	Flexible; no type definitions	No static checking; no docgen; no IDE autocomplete; harder to serialize/deserialize	Rejected — Julia advantages of structs (type-stable, documented) outweigh cost
Single onToolCall hook (pre-execute only)	Simpler hook interface	Can't sanitize results post-execution; can't set terminate flag after tool completes	Rejected — afterToolCall needed for masking secrets, normalizing usage, inspecting error status
Direct HTTP calls to MCP servers (no NATS)	Simpler dependency (no NATS)	Breaks if MCP uses NATS (common in microservices); no single transport	Rejected — callable struct abstracts transport; NATS is one implementation option
External orchestrator runs tool call loop	Caller has full control	Duplicates logic; every caller must implement same loop; harder to maintain	Rejected — loop is agent framework responsibility (single source of truth)

6. Risk Assessment

Risk	Impact	Probability	Mitigation
llmCall throws (network timeout, API error)	Agent loop exits with error in eventSink	Medium	try/catch in _processMessage and _agentLoop; eventSink receives error string
Tool prepareArguments or validateRequiredArgs hook throws	prepareToolCall catches and returns immediateOutcome error	Low	Hook errors caught; error tool result sent to LLM for recovery
MCP server unreachable	Tool discovery returns 0 tools; listTools still available	Medium	register_mcp_tools returns 0 on error; tool execution calls catch errors
Long-running tool call hangs	Caller blocks on outputChannel; no built-in timeout	Medium	eventSink passes abort signal; caller sets signal.aborted = true externally
Multiple agents share same mcpServer struct	mcpServer must be thread-safe for concurrent tool calls	Low	Each thread gets own mcpServer instance; NATS handles concurrency
Message history grows unbounded	Memory exhaustion in long sessions	Medium	No built-in pruning; prepareContext can filter/prune messages (TODO in code)

7. Requirements Traceability

Solution Component	Requirement ID	Decision ID	Description
Channel-based async processing	FR-001	SD-001, SD-002	inputChannel, followUpChannel, outputChannel with priority dispatch
LLM call loop	FR-003	SD-003	_processMessage while loop; llmCall() → extract → execute → append → repeat
MCP tool discovery	FR-004	SD-004, SD-009	tools/list with pagination; tools/call execution
Lifecycle hooks	FR-005	SD-005	beforeToolCall can block; afterToolCall transforms; never throw
Message type conversion	FR-006	SD-006	Julia structs with keyword constructors; _userMessageToOpenAI, _messageContentToBlocks
Format-agnostic extraction	FR-002	SD-009	_extractToolCalls handles both message.tool_calls and content blocks
Event sink observability	NFR-201, NFR-103	SD-007	eventSink passed to all lifecycle steps; logs state machine transitions