# Solution Design: YiemAgent Agent Framework

## 1. Problem Decomposition

| Problem | Description | User Impact |
|---------|-------------|-------------|
| P-001 | Developers need a chatbot framework in Julia that handles LLM calls and tool calls without writing boilerplate | Must write separate request/response handling, tool call parsing, and loop management code for each agent |
| P-002 | Tool discovery is manual and repetitive | Must register every tool individually; no automatic discovery from MCP-compatible servers |
| P-003 | Multiple LLM providers have different response formats | Must handle `message.tool_calls` (LMStudio/vLLM) vs `content` blocks (OpenAI API) separately |
| P-004 | Tool execution needs lifecycle hooks for observability and control | No built-in before/after hooks for authorization, logging, or result sanitization |
| P-005 | Agent state management is error-prone | Must handle message history, tool registration, and pending tool call tracking manually |

## 2. Solution Approach

**Approach**: "Build a channel-based agent system where the LLM talks to tools automatically, tools talk to MCP servers automatically, and you compose agents by injecting `llmCall` and `mcpServer` functions — the framework owns the loop, the state, and the format conversion."

**Key Principles**:
- **Channel-based async processing**: Each agent owns three channels (`inputChannel`, `followUpChannel`, `outputChannel`); a background `@spawn` task runs the agent loop, processing messages in priority order
- **Tool call loop**: The agent calls `llmCall()`, extracts tool calls, executes them (sequentially or in parallel), appends results, and calls `llmCall()` again — repeating until no tool calls remain
- **Pluggable hooks**: `prepareContext`, `formatMsgForLLM`, `beforeToolCall`, `afterToolCall`, and `llmCall` are all injectable functions — the framework provides sensible defaults, but you override what you need
- **MCP-native tool discovery**: Tools are discovered at startup via MCP `tools/list` and can be discovered at runtime via the `listTools` builtin tool (idempotent registration with pagination support)
- **Format-agnostic LLM responses**: `_extractToolCalls()` handles both OpenAI content blocks and LMStudio/vLLM nested message format, allowing the same agent to work with any model provider

## 3. Decision Rationale

| Decision ID | Decision | Rationale | Alternatives Rejected |
|-------------|----------|-----------|----------------------|
| SD-001 | Use channel-based async processing with `@spawn` task | Provides non-blocking message queues; `inputChannel` blocks if full (backpressure); `followUpChannel` allows async follow-ups while agent is busy | Threading with shared state — race conditions; callback-based async — hard to compose |
| SD-002 | Background `_agentLoop` task runs message dispatch loop | Isolates message processing from construction; agent can be created and used concurrently | Synchronous agent.run(message) — blocks caller; manual task management — boilerplate |
| SD-003 | `_processMessage` runs `llmCall()` in an inner loop (while loop) until no tool calls remain | Handles multi-turn tool use transparently — agent decides when to stop based on LLM response | External loop in caller code — duplicates agent loop logic; single-turn only — no multi-step tool use |
| SD-004 | MCP server integration via callable struct `mcpServer(method, args...)` | MCP is model-agnostic protocol; callable struct allows type-stable single dispatch; supports NATS transport (struct has `natsConn`, `topic`, `senderID` fields) | Ad-hoc HTTP client for MCP — breaks NATS-only transports; tight coupling to specific transport |
| SD-005 | Tool call lifecycle: `prepareToolCall()` → `executePreparedToolCall()` → `finalizeExecutedToolCall()` — three phases, never throw errors | Preparation validates args and runs `beforeToolCall` hook (can block); execution calls `tool.execute()`; finalization runs `afterToolCall` hook. Each phase returns a result type, not exceptions — agent loop catches all and builds error tool result | `try/catch` in loop — loses context; throwing errors breaks tool result message contract |
| SD-006 | Message types are Julia structs with keyword constructors (`userMessage(content=[...])`) | Type stable; keyword constructors provide defaults (`role="user"`, `timestamp=now()`); supports both positional and keyword usage | `Dict{String, Any}` for messages — lose type safety, docgen, IDE support |
| SD-007 | `eventSink` function passed to all deep lifecycle calls | Provides hooks for external monitoring (logging, metrics, NATS pub/sub) without modifying core logic | Built-in `println` — can't disable or redirect; no metrics integration |
| SD-008 | `abortSignal` struct with `.aborted::Bool` field passed to `prepareToolCall()` and hooks | Simple mutable state for interrupting long-running tool calls (no async cancellation — just flag check) | Threading with `Task.cancel()` — Julia thread model doesn't support it; `@async` spawn — complex cleanup |
| SD-009 | `_extractToolCalls()` supports both `message.tool_calls` and `content[{type="tool_calls"}]` formats | LMStudio.jl/vLLM vs OpenAI API have different response shapes; framework must work with both | Separate parser per provider — maintenance burden; require provider wrapper — adds indirection |

## 4. High-Level Component Diagram

```mermaid
flowchart TD
    User[External Caller] -->|promptAgent / followUp| InputCh[inputChannel]
    
    subagent AgentLoop [Agent Loop Task]
        InputCh -->|take!| Disp[Message Dispatcher]
        Disp -->|spawn| Proc[_processMessage]
        Proc -->|Channel| PrCh[processMessageInputCh]
    end
    
    subagent Message Pipeline
        PrCh -->|put!| DMsg[Drain input, convert to userMessage]
        DMsg --> PC[prepareContext hook]
        PC --> FM[formatMsgForLLM hook]
        FM --> LLM[llmCall]
        LLM --> EC[_extractToolCalls]
        EC -->|hasToolCalls| TC[executeToolCalls]
        TC -->|toolResultBatch| TM[toolResultMessage]
        TM -->|push!| MH[messageHistory]
        TC -->|no tool calls break| BRK[break loop]
        BRK --> End[exit]
    end
    
    subagent Tool System
        TC --> Prep[prepareToolCall]
        Prep --> Exec[executePreparedToolCall]
        Exec --> Term[finalizeExecutedToolCall]
        Term --> CRM[createToolResultMessage]
        CRM --> TM
    end
    
    subagent MCP Integration
        Prep --> ToolLookup[context.tools lookup]
        ToolLookup --> MCP[MCP tools/list + tools/call]
        MCP --> Wr[_wrap_mcp_tool]
        Wr --> Reg[toolRegistry.registerAllTools]
    end
    
    OutputCh[outputChannel] --> User
    Proc -->|final response| OutputCh
```

**Component Descriptions**:
- **Agent Loop Task** (`SD-001`, `SD-002`): Runs `@spawn _agentLoop()`; manages three channels; dispatches messages to `_processMessage`
- **Message Pipeline** (`SD-003`, `SD-004`): `prepareContext` → `formatMsgForLLM` → `llmCall` → extract tool calls → loop
- **Tool System** (`SD-005`): Three-phase lifecycle (prepare → execute → finalize); never throws; returns error tool result on failure
- **MCP Integration** (`SD-004`): `tools/list` discovery with pagination; `_wrap_mcp_tool` creates executable `agentTool` with MCP JSON-RPC calls

## 5. Alternatives Considered

| Alternative | Pros | Cons | Decision |
|-------------|------|------|----------|
| Synchronous `agent.process(message)` | Simpler API; no channels or tasks | Blocks caller; no follow-up support; harder to test concurrency | **Rejected** — async channels needed for real-time chat UX |
| JSON dicts for messages everywhere | Flexible; no type definitions | No static checking; no docgen; no IDE autocomplete; harder to serialize/deserialize | **Rejected** — Julia advantages of structs (type-stable, documented) outweigh cost |
| Single `onToolCall` hook (pre-execute only) | Simpler hook interface | Can't sanitize results post-execution; can't set terminate flag after tool completes | **Rejected** — `afterToolCall` needed for masking secrets, normalizing usage, inspecting error status |
| Direct HTTP calls to MCP servers (no NATS) | Simpler dependency (no NATS) | Breaks if MCP uses NATS (common in microservices); no single transport | **Rejected** — callable struct abstracts transport; NATS is one implementation option |
| External orchestrator runs tool call loop | Caller has full control | Duplicates logic; every caller must implement same loop; harder to maintain | **Rejected** — loop is agent framework responsibility (single source of truth) |

## 6. Risk Assessment

| Risk | Impact | Probability | Mitigation |
|------|--------|-------------|------------|
| `llmCall` throws (network timeout, API error) | Agent loop exits with error in `eventSink` | Medium | `try/catch` in `_processMessage` and `_agentLoop`; eventSink receives error string |
| Tool `prepareArguments` or `validateRequiredArgs` hook throws | `prepareToolCall` catches and returns `immediateOutcome` error | Low | Hook errors caught; error tool result sent to LLM for recovery |
| MCP server unreachable | Tool discovery returns 0 tools; `listTools` still available | Medium | `register_mcp_tools` returns 0 on error; tool execution calls catch errors |
| Long-running tool call hangs | Caller blocks on `outputChannel`; no built-in timeout | Medium | `eventSink` passes abort signal; caller sets `signal.aborted = true` externally |
| Multiple agents share same `mcpServer` struct | `mcpServer` must be thread-safe for concurrent tool calls | Low | Each thread gets own `mcpServer` instance; NATS handles concurrency |
| Message history grows unbounded | Memory exhaustion in long sessions | Medium | No built-in pruning; `prepareContext` can filter/prune messages (TODO in code) |

## 7. Requirements Traceability

| Solution Component | Requirement ID | Decision ID | Description |
|-------------------|----------------|-------------|-------------|
| Channel-based async processing | FR-001 | SD-001, SD-002 | `inputChannel`, `followUpChannel`, `outputChannel` with priority dispatch |
| LLM call loop | FR-003 | SD-003 | `_processMessage` while loop; `llmCall()` → extract → execute → append → repeat |
| MCP tool discovery | FR-004 | SD-004, SD-009 | `tools/list` with pagination; `tools/call` execution |
| Lifecycle hooks | FR-005 | SD-005 | `beforeToolCall` can block; `afterToolCall` transforms; never throw |
| Message type conversion | FR-006 | SD-006 | Julia structs with keyword constructors; `_userMessageToOpenAI`, `_messageContentToBlocks` |
| Format-agnostic extraction | FR-002 | SD-009 | `_extractToolCalls` handles both `message.tool_calls` and `content` blocks |
| Event sink observability | NFR-201, NFR-103 | SD-007 | `eventSink` passed to all lifecycle steps; logs state machine transitions |