docs/1-requirements.md

@@ -0,0 +1,155 @@
+# YiemAgent: Agile Agent Framework
+
+## 1. Business Context & Success Metrics
+
+### Business Goal
+
+YiemAgent is a Julia-based AI agent framework that enables building chatbot assistants with tool-calling capabilities, model-agnostic LLM integration, and runtime tool discovery via MCP (Model Context Protocol). It provides an asynchronous agent loop with channel-based message passing, supporting both single-turn and multi-turn conversations with automatic tool call resolution.
+
+### User Stories
+
+- **US-001**: As a developer, I want to create an AI agent with a simple constructor so that I can quickly prototype chatbot functionality in Julia
+- **US-002**: As a developer, I want the agent to discover tools automatically from an MCP server so that I don't need to manually register tool APIs
+- **US-003**: As a developer, I want the agent to handle tool calls automatically in a loop so that the LLM can use tools without external orchestration
+- **US-004**: As a developer, I want to intercept tool calls with before/after hooks so that I can add user approval, logging, or result sanitization
+- **US-005**: As a developer, I want the framework to support multiple LLM models and providers through a pluggable `llmCall` interface so that I can swap models without changing agent logic
+
+### KPIs & Targets
+
+- **KPI-001**: Agent loop processes messages with <500ms overhead for sequential tool calls (validates US-003)
+- **KPI-002**: MCP tool discovery completes within 2 seconds for 50+ tools (validates US-002)
+- **KPI-003**: 100% of message types have OpenAI format conversion (validates US-001)
+
+## 2. Technical Boundaries
+
+### In Scope
+
+- Julia module implementing an AI agent with asynchronous message processing
+- Channel-based input (user→agent), follow-up (async queue), and output (agent→user) interfaces
+- LLM model abstraction with tool call extraction from multiple response formats (OpenAI content blocks, LMStudio/vLLM format)
+- Tool registry with MCP server integration (tools/list, tools/call via JSON-RPC 2.0)
+- Agent loop with before/after tool call hooks for lifecycle management
+- Message type system with OpenAI format conversion utilities
+- Abort signal and termination flag support via tool implementations
+
+### Out of Scope
+
+- GUI/CLI interface (exposed via channels only; UI layer is external)
+- Persistent chat history or long-term memory (TODO comments in code)
+- Multi-agent orchestration (single agent instance per `yiemAgent` struct)
+- Embedding generation or retrieval-augmented generation (RAG)
+- Natural language-to-code generation or execution sandboxes
+
+### Dependencies
+
+- **Julia Standard Library**: `Dates`, `UUIDs`, `Base.Threads`
+- **External Packages**: `JSON`, `DataStructures`, `HTTP`, `PrettyPrinting`, `Serialization`, `DataFrames`
+- **Internal Package**: `GeneralUtils` (project-specific utility library)
+- **Optional**: `NATS` (MCP server communication), `LibPQ` (PostgreSQL)
+
+### Deployment Constraints
+
+- **NFR-001**: Library designed for embedded use in Docker containers (MicroK8s cluster, per `NFR-501` from template)
+- **NFR-002**: Must support Julia 1.9+ for module include/export semantics compatibility
+- **NFR-003**: Thread-safe channel operations; single-agent instance processes messages sequentially on background `@spawn` task
+
+## 3. Functional Requirements (FR)
+
+### FR-001: Agent Message Processing
+
+The agent shall process user messages through an asynchronous loop that:
+- Accepts messages via `inputChannel` (capacity 16) and `followUpChannel` (capacity 32)
+- Prioritizes `inputChannel` messages over `followUpChannel` messages
+- Feeds responses to `outputChannel` (capacity 16) in OpenAI message format
+- Exits cleanly on `:shutdown` signal
+
+**Traceability**: US-001, US-003
+
+### FR-002: LLM Model Agnosticism
+
+The agent shall accept a pluggable `llmCall` function that:
+- Receives formatted messages and returns an LLM response dict
+- The response dict may contain `message.tool_calls` (LMStudio format) or `content` blocks (OpenAI format)
+- The framework extracts tool calls regardless of response format
+
+**Traceability**: US-005
+
+### FR-003: Tool Call Execution Loop
+
+The agent shall automatically execute tool calls returned by the LLM:
+- Extract tool calls via `_extractToolCalls()` which supports both `message.tool_calls` and `content[{type="tool_calls"}]` formats
+- Build `agentContext` via `prepareContext()` (pluggable hook)
+- Format messages via `formatMsgForLLM()` to OpenAI format
+- Extract tool calls → loop: execute tool calls → append results to history → call LLM again
+- Break loop when LLM response has no tool calls
+- Support `terminate` flag: if `toolResultBatch.terminate == true`, break loop and return final response
+
+**Traceability**: US-003
+
+### FR-004: Tool Discovery and Registration
+
+The agent shall:
+- Query MCP server via `mcpserver("tools/list")` (JSON-RPC 2.0) to discover tools
+- Support pagination via `nextCursor` field
+- Wrap each MCP tool definition as an `agentTool` with `execute` hook that calls `mcpserver("tools/call", name, args, sessionId)`
+- Register a builtin `listTools` tool that triggers runtime discovery and lists all available tools
+- Skip tools already registered (idempotent)
+
+**Traceability**: US-002
+
+### FR-005: Tool Call Lifecycle Hooks
+
+The agent shall support two lifecycle hooks:
+- `beforeToolCall(beforeToolCallContext, signal)` — can block execution by returning `beforeToolCallResult(true, reason)`
+- `afterToolCall(afterToolCallContext)` — can transform result by returning modified `agentToolResult`
+- Errors from hooks are caught and converted to `immediateOutcome`/error tool result messages (never throw)
+
+**Traceability**: US-004
+
+### FR-006: Message Type Conversion
+
+The agent shall:
+- Convert internal message types (`userMessage`, `assistantMessage`, `assistantMessageToolCall`, `toolResultMessage`) to OpenAI format via `_userMessageToOpenAI()`, `_assistantMessageToOpenAI()`, `_assistantMessageToolCallToOpenAI()`, `_toolResultMessageToOpenAI()`
+- Convert internal `messageContent` types (`textContent`, `imageContent`, `reasoningContent`) to OpenAI content blocks via `_messageContentToBlocks()`
+- Convert `agentTool` definitions to OpenAI function tool format via `_toolsToOpenAI()`
+
+**Traceability**: US-001
+
+## 4. Non-Functional Requirements (NFRs)
+
+### 4.1 Performance & Scalability
+
+- **NFR-101**: Agent loop message processing overhead <500ms per sequential tool call (measured KPI: KPI-001)
+- **NFR-102**: MCP tool discovery for 50+ tools completes within 2 seconds (measured KPI: KPI-002)
+- **NFR-103**: Channel buffers prevent unbounded memory growth — `inputChannel` (16), `followUpChannel` (32), `outputChannel` (16)
+
+### 4.2 Availability & Reliability
+
+- **NFR-201**: Agent loop catches all errors via `try/catch` in `_agentLoop()` and calls `eventSink(errMsg)` before exit (measured by event sink logging)
+- **NFR-202**: `prepareToolCall()` never throws errors — validation failures return `immediateOutcome` with error tool result
+
+### 4.3 Privacy & Security
+
+- **Data Classification**: Tool arguments and results may contain PII (user messages); MCP tool results treated as untrusted input
+- **No built-in auth/authz**: Responsibility of `mcpServer` implementation and `beforeToolCall` hook
+- **Input validation**: JSON schema validation via `validateToolArguments()` checks required fields only (tools need custom validation via `validateRequiredArgs` hook)
+
+### 4.4 Observability & Telemetry
+
+- **Required Logs**: `agentLoop` events (via `eventSink`) emit state machine steps (`_agentLoop 1` through `_agentLoop 6`), `_processMessage` steps (1 through 18), `prepareToolCall` steps (1 through 10)
+- **Critical Metrics**:
+  - `agent._state.messages.length` — conversation history size (logged at each step)
+  - `toolExecutionStart`/`toolExecutionEnd` events (via `toolExecStartEvent`, `toolExecEndEvent` structs)
+- **Retention**: No built-in log retention; agent is a library, does not manage external logging
+
+## 5. Acceptance Conditions
+
+- [ ] **FR-001**: Agent processes messages through channel system with priority ordering (inputChannel > followUpChannel)
+- [ ] **FR-002**: `llmCall` is pluggable; response formats supported include `message.tool_calls` dict and `content` array blocks
+- [ ] **FR-003**: Agent loops on tool calls until LLM returns no tool calls or terminate flag is set
+- [ ] **FR-004**: MCP tools are discovered via `tools/list` with pagination; all registered tools are callable via `tools/call`
+- [ ] **FR-005**: `beforeToolCall` hook can block; `afterToolCall` hook can transform result; errors return error tool result
+- [ ] **FR-006**: All message types (userMessage, assistantMessage, assistantMessageToolCall, toolResultMessage) convert to OpenAI format
+- [ ] **NFR-101**: Message processing overhead <500ms per sequential tool call
+- [ ] **NFR-102**: MCP discovery for 50+ tools completes <2 seconds
+- [ ] **NFR-201**: Agent loop catches errors and exits gracefully without panics

docs/2-solution_design.md

@@ -0,0 +1,119 @@
+# Solution Design: YiemAgent Agent Framework
+
+## 1. Problem Decomposition
+
+| Problem | Description | User Impact |
+|---------|-------------|-------------|
+| P-001 | Developers need a chatbot framework in Julia that handles LLM calls and tool calls without writing boilerplate | Must write separate request/response handling, tool call parsing, and loop management code for each agent |
+| P-002 | Tool discovery is manual and repetitive | Must register every tool individually; no automatic discovery from MCP-compatible servers |
+| P-003 | Multiple LLM providers have different response formats | Must handle `message.tool_calls` (LMStudio/vLLM) vs `content` blocks (OpenAI API) separately |
+| P-004 | Tool execution needs lifecycle hooks for observability and control | No built-in before/after hooks for authorization, logging, or result sanitization |
+| P-005 | Agent state management is error-prone | Must handle message history, tool registration, and pending tool call tracking manually |
+
+## 2. Solution Approach
+
+**Approach**: "Build a channel-based agent system where the LLM talks to tools automatically, tools talk to MCP servers automatically, and you compose agents by injecting `llmCall` and `mcpServer` functions — the framework owns the loop, the state, and the format conversion."
+
+**Key Principles**:
+- **Channel-based async processing**: Each agent owns three channels (`inputChannel`, `followUpChannel`, `outputChannel`); a background `@spawn` task runs the agent loop, processing messages in priority order
+- **Tool call loop**: The agent calls `llmCall()`, extracts tool calls, executes them (sequentially or in parallel), appends results, and calls `llmCall()` again — repeating until no tool calls remain
+- **Pluggable hooks**: `prepareContext`, `formatMsgForLLM`, `beforeToolCall`, `afterToolCall`, and `llmCall` are all injectable functions — the framework provides sensible defaults, but you override what you need
+- **MCP-native tool discovery**: Tools are discovered at startup via MCP `tools/list` and can be discovered at runtime via the `listTools` builtin tool (idempotent registration with pagination support)
+- **Format-agnostic LLM responses**: `_extractToolCalls()` handles both OpenAI content blocks and LMStudio/vLLM nested message format, allowing the same agent to work with any model provider
+
+## 3. Decision Rationale
+
+| Decision ID | Decision | Rationale | Alternatives Rejected |
+|-------------|----------|-----------|----------------------|
+| SD-001 | Use channel-based async processing with `@spawn` task | Provides non-blocking message queues; `inputChannel` blocks if full (backpressure); `followUpChannel` allows async follow-ups while agent is busy | Threading with shared state — race conditions; callback-based async — hard to compose |
+| SD-002 | Background `_agentLoop` task runs message dispatch loop | Isolates message processing from construction; agent can be created and used concurrently | Synchronous agent.run(message) — blocks caller; manual task management — boilerplate |
+| SD-003 | `_processMessage` runs `llmCall()` in an inner loop (while loop) until no tool calls remain | Handles multi-turn tool use transparently — agent decides when to stop based on LLM response | External loop in caller code — duplicates agent loop logic; single-turn only — no multi-step tool use |
+| SD-004 | MCP server integration via callable struct `mcpServer(method, args...)` | MCP is model-agnostic protocol; callable struct allows type-stable single dispatch; supports NATS transport (struct has `natsConn`, `topic`, `senderID` fields) | Ad-hoc HTTP client for MCP — breaks NATS-only transports; tight coupling to specific transport |
+| SD-005 | Tool call lifecycle: `prepareToolCall()` → `executePreparedToolCall()` → `finalizeExecutedToolCall()` — three phases, never throw errors | Preparation validates args and runs `beforeToolCall` hook (can block); execution calls `tool.execute()`; finalization runs `afterToolCall` hook. Each phase returns a result type, not exceptions — agent loop catches all and builds error tool result | `try/catch` in loop — loses context; throwing errors breaks tool result message contract |
+| SD-006 | Message types are Julia structs with keyword constructors (`userMessage(content=[...])`) | Type stable; keyword constructors provide defaults (`role="user"`, `timestamp=now()`); supports both positional and keyword usage | `Dict{String, Any}` for messages — lose type safety, docgen, IDE support |
+| SD-007 | `eventSink` function passed to all deep lifecycle calls | Provides hooks for external monitoring (logging, metrics, NATS pub/sub) without modifying core logic | Built-in `println` — can't disable or redirect; no metrics integration |
+| SD-008 | `abortSignal` struct with `.aborted::Bool` field passed to `prepareToolCall()` and hooks | Simple mutable state for interrupting long-running tool calls (no async cancellation — just flag check) | Threading with `Task.cancel()` — Julia thread model doesn't support it; `@async` spawn — complex cleanup |
+| SD-009 | `_extractToolCalls()` supports both `message.tool_calls` and `content[{type="tool_calls"}]` formats | LMStudio.jl/vLLM vs OpenAI API have different response shapes; framework must work with both | Separate parser per provider — maintenance burden; require provider wrapper — adds indirection |
+
+## 4. High-Level Component Diagram
+
+```mermaid
+flowchart TD
+    User[External Caller] -->|promptAgent / followUp| InputCh[inputChannel]
+    
+    subagent AgentLoop [Agent Loop Task]
+        InputCh -->|take!| Disp[Message Dispatcher]
+        Disp -->|spawn| Proc[_processMessage]
+        Proc -->|Channel| PrCh[processMessageInputCh]
+    end
+    
+    subagent Message Pipeline
+        PrCh -->|put!| DMsg[Drain input, convert to userMessage]
+        DMsg --> PC[prepareContext hook]
+        PC --> FM[formatMsgForLLM hook]
+        FM --> LLM[llmCall]
+        LLM --> EC[_extractToolCalls]
+        EC -->|hasToolCalls| TC[executeToolCalls]
+        TC -->|toolResultBatch| TM[toolResultMessage]
+        TM -->|push!| MH[messageHistory]
+        TC -->|no tool calls break| BRK[break loop]
+        BRK --> End[exit]
+    end
+    
+    subagent Tool System
+        TC --> Prep[prepareToolCall]
+        Prep --> Exec[executePreparedToolCall]
+        Exec --> Term[finalizeExecutedToolCall]
+        Term --> CRM[createToolResultMessage]
+        CRM --> TM
+    end
+    
+    subagent MCP Integration
+        Prep --> ToolLookup[context.tools lookup]
+        ToolLookup --> MCP[MCP tools/list + tools/call]
+        MCP --> Wr[_wrap_mcp_tool]
+        Wr --> Reg[toolRegistry.registerAllTools]
+    end
+    
+    OutputCh[outputChannel] --> User
+    Proc -->|final response| OutputCh
+```
+
+**Component Descriptions**:
+- **Agent Loop Task** (`SD-001`, `SD-002`): Runs `@spawn _agentLoop()`; manages three channels; dispatches messages to `_processMessage`
+- **Message Pipeline** (`SD-003`, `SD-004`): `prepareContext` → `formatMsgForLLM` → `llmCall` → extract tool calls → loop
+- **Tool System** (`SD-005`): Three-phase lifecycle (prepare → execute → finalize); never throws; returns error tool result on failure
+- **MCP Integration** (`SD-004`): `tools/list` discovery with pagination; `_wrap_mcp_tool` creates executable `agentTool` with MCP JSON-RPC calls
+
+## 5. Alternatives Considered
+
+| Alternative | Pros | Cons | Decision |
+|-------------|------|------|----------|
+| Synchronous `agent.process(message)` | Simpler API; no channels or tasks | Blocks caller; no follow-up support; harder to test concurrency | **Rejected** — async channels needed for real-time chat UX |
+| JSON dicts for messages everywhere | Flexible; no type definitions | No static checking; no docgen; no IDE autocomplete; harder to serialize/deserialize | **Rejected** — Julia advantages of structs (type-stable, documented) outweigh cost |
+| Single `onToolCall` hook (pre-execute only) | Simpler hook interface | Can't sanitize results post-execution; can't set terminate flag after tool completes | **Rejected** — `afterToolCall` needed for masking secrets, normalizing usage, inspecting error status |
+| Direct HTTP calls to MCP servers (no NATS) | Simpler dependency (no NATS) | Breaks if MCP uses NATS (common in microservices); no single transport | **Rejected** — callable struct abstracts transport; NATS is one implementation option |
+| External orchestrator runs tool call loop | Caller has full control | Duplicates logic; every caller must implement same loop; harder to maintain | **Rejected** — loop is agent framework responsibility (single source of truth) |
+
+## 6. Risk Assessment
+
+| Risk | Impact | Probability | Mitigation |
+|------|--------|-------------|------------|
+| `llmCall` throws (network timeout, API error) | Agent loop exits with error in `eventSink` | Medium | `try/catch` in `_processMessage` and `_agentLoop`; eventSink receives error string |
+| Tool `prepareArguments` or `validateRequiredArgs` hook throws | `prepareToolCall` catches and returns `immediateOutcome` error | Low | Hook errors caught; error tool result sent to LLM for recovery |
+| MCP server unreachable | Tool discovery returns 0 tools; `listTools` still available | Medium | `register_mcp_tools` returns 0 on error; tool execution calls catch errors |
+| Long-running tool call hangs | Caller blocks on `outputChannel`; no built-in timeout | Medium | `eventSink` passes abort signal; caller sets `signal.aborted = true` externally |
+| Multiple agents share same `mcpServer` struct | `mcpServer` must be thread-safe for concurrent tool calls | Low | Each thread gets own `mcpServer` instance; NATS handles concurrency |
+| Message history grows unbounded | Memory exhaustion in long sessions | Medium | No built-in pruning; `prepareContext` can filter/prune messages (TODO in code) |
+
+## 7. Requirements Traceability
+
+| Solution Component | Requirement ID | Decision ID | Description |
+|-------------------|----------------|-------------|-------------|
+| Channel-based async processing | FR-001 | SD-001, SD-002 | `inputChannel`, `followUpChannel`, `outputChannel` with priority dispatch |
+| LLM call loop | FR-003 | SD-003 | `_processMessage` while loop; `llmCall()` → extract → execute → append → repeat |
+| MCP tool discovery | FR-004 | SD-004, SD-009 | `tools/list` with pagination; `tools/call` execution |
+| Lifecycle hooks | FR-005 | SD-005 | `beforeToolCall` can block; `afterToolCall` transforms; never throw |
+| Message type conversion | FR-006 | SD-006 | Julia structs with keyword constructors; `_userMessageToOpenAI`, `_messageContentToBlocks` |
+| Format-agnostic extraction | FR-002 | SD-009 | `_extractToolCalls` handles both `message.tool_calls` and `content` blocks |
+| Event sink observability | NFR-201, NFR-103 | SD-007 | `eventSink` passed to all lifecycle steps; logs state machine transitions |

docs/3-walkthrough.md

@@ -0,0 +1,318 @@
+# Walkthrough: YiemAgent Agent Framework
+
+## Structural Guidance
+
+- One **overview sequence diagram** per participant (high-level flow)
+- **Detailed sub-diagrams** for complex interaction points (tool call loop, MCP discovery)
+- Use sequence diagrams for request/response patterns; lifecycle states for multi-step operations
+
+---
+
+## External Caller
+
+_The Developer calling `YiemAgent.jl` via `promptAgent`, `takeResponse`, `followUp`, and `stopAgent`._
+
+### Session Start: Create Agent
+
+**WT-001**: Developer creates an `yiemAgent` instance with `llmCall` and `mcpServer`
+
+> Cross-reference: `2-solution_design.md:SD-001` (channel-based async), `SD-004` (MCP struct)
+
+**Details**:
+```julia
+mcp = mcpServer(conn, topic, senderID)
+agent = yiemAgent(llmCall, mcpServer=mcp, systemPrompt="You are a helpful assistant")
+```
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    participant Dev as Developer
+    part agent YiemAgent Constructor
+    Dev->>agent: yiemAgent(llmCall, ; mcpServer=mcp)
+    agent->>agent: create channels (input=16, followUp=32, output=16)
+    agent->>agent: registerAllTools(tools, mcp)
+    agent->>mcp: mcp("tools/list", sessionId)
+    mcp-->>agent: response (tools array + nextCursor)
+    agent->>agent: _wrap_mcp_tool for each tool
+    agent->>agent: registerTool(tools, listTools_tool)
+    agent->>agent: println("[toolRegistry] Discovered N MCP tools")
+    agent->>agent: yiemAgent struct with placeholder task
+    agent->>agent: @spawn _agentLoop(agent)
+    agent-->>Dev: yiemAgent instance
+```
+
+> **Traceability**: FR-004 (tool discovery), US-001 (agent creation)
+
+### Send Message: User Prompt
+
+**WT-002**: Developer sends a user message to the agent
+
+> Cross-reference: `2-solution_design.md:SD-002` (background task dispatch)
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    participant Dev as Developer
+    part agent AgentLoop
+    Dev->>agent: promptAgent(agent, text="What's the weather?")
+    agent->>agent: put!(inputChannel, msg_dict)
+    Dev->>agent: takeResponse(agent)  # blocks
+    agent->>agent: inputChannel isready
+    agent->>agent: take!(inputChannel)
+    agent->>Dev: (returns agent for chaining)
+```
+
+> **Traceability**: FR-001 (channel processing), PR-001, SD-001, SD-002
+
+### Receive Response
+
+**WT-003**: Agent returns response on `outputChannel`
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    participant Dev as Developer
+    part agent AgentLoop
+    AgentLoop->>AgentLoop: _processMessage runs
+    AgentLoop->>AgentLoop: while loop, llmCall, tool calls...
+    AgentLoop->>AgentLoop: no more tool calls → exit loop
+    AgentLoop->>AgentLoop: filter reasoningContent from final result
+    AgentLoop->>AgentLoop: _assistantMessageToOpenAI(result)
+    AgentLoop->>AgentLoop: put!(outputChannel, respondToUI)
+    AgentLoop->>Dev: take!(outputChannel)  # unblocks
+    Dev->>Dev: receive OpenAI dict response
+```
+
+> **Traceability**: FR-001, FR-003, FR-006, PR-001
+
+### Send Follow-up While Agent Busy
+
+**WT-004**: Developer sends a follow-up message while agent is processing a tool call
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    participant Dev as Developer
+    part agent AgentLoop
+    Dev->>agent: promptAgent(text="What's the weather in Tokyo?")  # agent starts processing
+    AgentLoop->>AgentLoop: _processMessage started (processingTask = Task)
+    Dev->>agent: followUp(text="Also check Seoul")  # while busy
+    agent->>agent: put!(followUpChannel, msg_dict)
+    AgentLoop->>AgentLoop: _processMessage completes (tool calls done)
+    AgentLoop->>AgentLoop: outputChannel has message → put!(outputChannel, response), take!(next msg)
+    AgentLoop->>AgentLoop: no inputChannel → check followUpChannel
+    AgentLoop->>AgentLoop: isready(followUpChannel) → take!(followUpMsg), put!(inputChannel, followUpMsg)
+    AgentLoop->>AgentLoop: processingTask = nothing → reset
+    AgentLoop->>AgentLoop: loop back → take!(inputChannel) from followUp
+    Dev->>agent: takeResponse() → blocks
+    AgentLoop->>Dev: next response via outputChannel
+```
+
+> **Traceability**: FR-001 (follow-up priority), PR-001, SD-001, SD-002
+
+### Stop Agent
+
+**WT-005**: Developer stops the agent gracefully
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    participant Dev as Developer
+    part agent AgentLoop
+    Dev->>agent: stopAgent(agent)
+    agent->>agent: put!(inputChannel, :shutdown)
+    AgentLoop->>AgentLoop: check newUserMsg === :shutdown
+    AgentLoop->>AgentLoop: drain inputChannel (while isready)
+    AgentLoop->>AgentLoop: drain followUpChannel (while isready)
+    AgentLoop->>AgentLoop: break, catch, fetch(_agentLoop)
+    Dev->>agent: close(inputChannel, outputChannel, followUpChannel)
+    Dev-->>Dev: nothing (agent destroyed)
+```
+
+> **Traceability**: FR-001, NFR-201 (graceful exit), PR-001
+
+---
+
+## LLM Provider
+
+_The external LLM API (e.g. Anthropic, OpenAI, LMStudio) receiving formatted messages and returning responses._
+
+### Receive Formatted Message
+
+**WT-010**: Agent sends LLM request in OpenAI format
+
+> Cross-reference: `2-solution_design.md:SD-003` (formatMsgForLLM)
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    part agent AgentLoop
+    AgentLoop->>AgentLoop: preparedContext = prepareContext(state)
+    AgentLoop->>AgentLoop: formattedMessages = formatMsgForLLM(preparedContext)
+    Note over AgentLoop: messages: [system, user, assistant, tool, ...]<br/>tools: [{"function": {"name": "getWeather", "parameters": {...}}}]
+    agentLoop->>llmCall: llmCall(formattedMessages, eventSink=eventSink)
+    llmCall->>Provider: HTTP POST /v1/chat/completions (JSON)
+    Provider-->>llmCall: JSON response
+    llmCall-->>AgentLoop: response = JSON.Object{String, Any}(...)
+```
+
+> **Traceability**: FR-002, SD-003, SD-009
+
+### Receive Response: No Tool Calls
+
+**WT-11**: Agent receives text-only response
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    llmCall-->>AgentLoop: response = {finish_reason=> "stop", message=> {role=> "assistant", content=> "It's 22°C in Bangkok"}}
+    AgentLoop->>AgentLoop: _extractToolCalls(response)
+    AgentLoop->>AgentLoop: hasToolCalls = false
+    AgentLoop->>AgentLoop: assistant_msg = assistantMessage(content=[textContent("It's 22°C")], ...)
+    AgentLoop->>AgentLoop: push!(agentMsgHistory, assistant_msg)
+    AgentLoop->>AgentLoop: break (no tool calls)
+    AgentLoop->>AgentLoop: filter reasoningContent (if any)
+    AgentLoop->>AgentLoop: create final assistantMessage
+    AgentLoop->>AgentLoop: put!(outputChannel, OpenAI format dict)
+```
+
+> **Traceability**: FR-003 (loop exit), FR-006 (OpenAI conversion), PR-003
+
+### Receive Response: With Tool Calls
+
+**WT-012**: Agent receives tool call response
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    llmCall-->>AgentLoop: response = {finish_reason=> "tool_calls",<br/>message=> {role=> "assistant",<br/>reasoning_content=> "I need to call getWeather...",<br/>tool_calls=> [{"id"=>"call_1", "function"=>{name=>"getWeather", arguments=>"{\"city\":\"Bangkok\"}"}}, ...]}}
+    AgentLoop->>AgentLoop: _extractToolCalls(response)
+    AgentLoop->>AgentLoop: hasToolCalls = true<br/>toolCallList = [agentToolCall("getWeather", ...)]
+    AgentLoop->>AgentLoop: assistant_msg = assistantMessageToolCall(toolCalls, reasoningContent, ...)
+    AgentLoop->>AgentLoop: push!(agentMsgHistory, assistant_msg)
+    AgentLoop->>AgentLoop: config = agentLoopConfig(beforeToolCall, afterToolCall, parallel="sequential")
+    AgentLoop->>AgentLoop: toolResultBatch = executeToolCalls(...)
+    ...
+```
+
+> **Traceability**: FR-002, FR-003, PR-003, SD-003, SD-005, SD-009
+
+---
+
+## MCP Server
+
+_The Model Context Protocol server providing tools via JSON-RPC 2.0 (transport via NATS or HTTP).
+
+### Receive tools/list Request
+
+**WT-020**: Agent discovers tools on startup
+
+> Cross-reference: `2-solution_design.md:SD-004` (MCP integration)
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    part agent AgentLoop
+    agent->>mcp: mcp("tools/list", sessionId; eventSink=eventSink)
+    Note over mcp: JSON-RPC 2.0: {"jsonrpc"=>"2.0", method=>"tools/list", "id"=>..., "params"=>{...}}
+    MCP_server-->>agent: response = {<br/>  "result" => {<br/>    "tools" => [name, label, description, inputSchema], ...<br/>    , "nextCursor" => 123<br/>  }<br/>}
+    agent->>agent: for tool_def in tools_array<br/>  _wrap_mcp_tool(mcp, tool_def)
+    agent->>agent: while cursor !== nothing → paginate
+    agent->>agent: println("[toolRegistry] Discovered N MCP tools. Total: $(length(tools))")
+```
+
+> **Traceability**: FR-004 (MCP discovery), PR-004
+
+### Receive tools/call Request
+
+**WT-021**: Agent executes tool via MCP
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    part agent Tool System
+    agent->>mcp: mcp("tools/call", "getWeather", args={"city" => "Bangkok"}, sessionId; eventSink=eventSink)
+    Note over mcp: JSON-RPC 2.0: {"jsonrpc"=>"2.0", method=>"tools/call", "params"=>{name, arguments, sessionId}}
+    MCP_server-->>agent: response = {<br/>  "result" => {<br/>    "content" => [{"type"=>"text", "text"=>"Sunny, 22°C"}],<br/>    "isError" => false<br/>  }<br/>}
+    agent->>agent: check haskey(response, "error") → no
+    agent->>agent: result_data = response["result"]
+    agent->>agent: content_text = _extract_text_content(result_data)
+    agent->>agent: return agentToolResult([textContent(content_text)], details={isError=>false}, nothing, false)
+```
+
+> **Traceability**: FR-004 (tool execution), PR-004, SD-004, SD-005
+
+### Receive Error from MCP Server
+
+**WT-022**: MCP server returns error
+
+**Sequence**:
+```mermaid
+sequenceDiagram
+    MCP_server-->>agent: response = {<br/>  "error" => {<br/>    "code" => -32601,<br/>    "message" => "Method not found"<br/>  }<br/>}
+    agent->>agent: haskey(response, "error") → true
+    agent->>agent: return agentToolResult(<br/>    [textContent("MCP error: Method not found")],<br/>    details={"error" => "Method not found"},<br/>    nothing, false<br/>)
+    agent->>AgentLoop: finalOutcome(error=true)
+    agent->>AgentLoop: push!(agentMsgHistory, toolResultMessage(...))
+    agent->>AgentLoop: next LLM turn receives error result
+```
+
+> **Traceability**: FR-004 (error handling), NFR-202 (never throw), PR-004, SD-005
+
+---
+
+## Error Paths and Edge Cases
+
+**WT-030**: LLM times out (network error)
+
+> Same participant POV — from caller's perspective, they block on `takeResponse()` until task exits
+
+```mermaid
+sequenceDiagram
+    participant Dev as Developer
+    part agent AgentLoop
+    Dev->>agent: takeResponse(agent)  # blocks
+    agent->>llmCall: llmCall(messages)  # calls external API
+    lltimout
+    LLM provider-->>AgentLoop: ERROR (timeout)
+    AgentLoop->>AgentLoop: @error "Agent loop failed"
+    AgentLoop->>AgentLoop: eventSink("error: ...")
+    AgentLoop->>Dev: fetch(_agentLoop) throws TaskFailedException
+    Dev->>agent: stopAgent(agent)  # cleanup channels
+```
+
+**Traceability**: PR-001, NFR-201 (catch error and exit), SD-001
+
+**WT-031**: Tool execution throws exception
+
+```mermaid
+sequenceDiagram
+    agent->>agent: executePreparedToolCall
+    agent->>tool: tool.execute(toolCallId, args, signal, eventSink)
+    note over tool: tool raises exception (e.g., network error, invalid JSON)
+    tool-->>agent: throw Exception
+    agent->>agent: catch e
+    agent->>agent: executedOutcome(createErrorToolResult("error message"), true)
+    agent->>AgentLoop: finalizeExecutedToolCall
+    AgentLoop->>AgentLoop: createToolResultMessage(error=true)
+    AgentLoop->>AgentLoop: push!(history, toolResultMessage)
+    AgentLoop->>AgentLoop: next llmCall with error in message history
+```
+
+**Traceability**: FR-005 (hook errors caught), NFR-202 (never throw), SD-005
+
+**WT-032**: beforeToolCall hook blocks execution
+
+```mermaid
+sequenceDiagram
+    agent->>agent: beforeToolCall(beforeCtx, signal, eventSink)
+    agent->>agent: returns beforeToolCallResult(true, "User must approve")
+    agent->>agent: return immediateOutcome(createErrorToolResult("Tool blocked"), true)
+    agent->>AgentLoop: executeToolCalls → immediateOutcome
+    AgentLoop->>AgentLoop: finalizeExecutedToolCall
+    AgentLoop->>AgentLoop: createToolResultMessage(error=true)
+    AgentLoop->>AgentLoop: push! to history
+```
+
+**Traceability**: FR-005 (beforeToolCall can block), SD-005

docs/4-specification.md

@@ -0,0 +1,518 @@
+# Specification: YiemAgent Agent Framework
+
+## 1. Message Type System (`src/type.jl`)
+
+### SP-001: userMessage
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `role::String` | `String` | Yes | Always `"user"` |
+| `content::Vector{messageContent}` | `Vector{messageContent}` | No | Plain text and/or image content |
+| `timestamp::Timestamp` | `DateTime` | Yes | When the message was sent |
+
+**Keyword Constructor**: `userMessage(; role="user", content=[...], timestamp=now())`
+
+**Traceability**:
+- Requirement: FR-006 (message type conversion)
+- Solution Design: SD-006 (Julia structs for messages)
+
+### SP-002: assistantMessage
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `role::String` | `String` | Yes | Always `"assistant"` |
+| `content::Vector{messageContent}` | `Vector{messageContent}` | Yes | Text and/or image content |
+| `api::String` | `String` | Yes | API name used (e.g., `"openai"`) |
+| `provider::String` | `String` | Yes | Provider name (e.g., `"anthropic"`) |
+| `model::String` | `String` | Yes | Model identifier |
+| `usage::llmUsage` | `llmUsage` | Yes | Token usage (in/out) |
+| `stopReason::String` | `String` | Yes | Why generation stopped (e.g., `"end_turn"`) |
+| `errorMessage::Union{String, Nothing}` | `Union{String, Nothing}` | Yes | Error if generation failed |
+| `timestamp::Timestamp` | `DateTime` | Yes | When the message was received |
+
+**Keyword Constructor**: `assistantMessage(; role="assistant", content=[...], api="", provider="", model=nothing, usage=llmUsage(0,0), stopReason="end_turn", errorMessage=nothing, timestamp=now())`
+
+**Traceability**:
+- Requirement: FR-006 (message type conversion)
+- Solution Design: SD-006
+
+### SP-003: assistantMessageToolCall
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `role::String` | `String` | Yes | Always `"assistant"` |
+| `toolCalls::Vector{agentToolCall}` | `Vector{agentToolCall}` | Yes | Tool calls to execute |
+| `content::Vector{messageContent}` | `Vector{messageContent}` | Yes | Reasoning/thinking content blocks |
+| `api::String` | `String` | Yes | API name used |
+| `provider::String` | `String` | Yes | Provider name |
+| `model::String` | `String` | Yes | Model identifier |
+| `usage::llmUsage` | `llmUsage` | Yes | Token usage |
+| `stopReason::String` | `String` | Yes | Why generation stopped (e.g., `"tool_calls"`) |
+| `errorMessage::Union{String, Nothing}` | `Union{String, Nothing}` | Yes | Error if generation failed |
+| `timestamp::Timestamp` | `DateTime` | Yes | When the message was received |
+
+**Keyword Constructor**: `assistantMessageToolCall(; role="assistant", toolCalls=[...], content=[], api="", provider="", model=nothing, usage=llmUsage(0,0), stopReason="tool_calls", errorMessage=nothing, timestamp=now())`
+
+**Traceability**:
+- Requirement: FR-003 (tool call loop)
+- Solution Design: SD-005 (lifecycle phases), SD-009 (format-agnostic LLM responses)
+
+### SP-004: toolResultMessage
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `role::String` | `String` | Yes | Always `"toolResult"` (note: `"tool"` in example) |
+| `toolCallId::String` | `String` | Yes | ID matching the tool call |
+| `toolName::String` | `String` | Yes | Name of the executed tool |
+| `content::Vector{messageContent}` | `Vector{messageContent}` | Yes | Tool output content |
+| `details::Any` | `Any` | Yes | Additional tool-specific details |
+| `usage::Union{llmUsage, Nothing}` | `Union{llmUsage, Nothing}` | Yes | Token usage if applicable |
+| `addedToolNames::Union{Vector{String}, Nothing}` | `Union{Vector{String}, Nothing}` | Yes | Tools added during execution |
+| `isError::Bool` | `Bool` | Yes | Whether the tool call resulted in an error |
+| `timestamp::Timestamp` | `DateTime` | Yes | When the result was recorded |
+
+**Keyword Constructor**: `toolResultMessage(; role="tool", toolCallId="", toolName="", content=[], details=nothing, usage=nothing, addedToolNames=nothing, isError=false, timestamp=now())`
+
+**Traceability**:
+- Requirement: FR-005 (tool lifecycle)
+- Solution Design: SD-005
+
+### SP-005: agentToolCall
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `type::String` | `String` | Yes | Always `"function"` |
+| `id::String` | `String` | Yes | Unique tool call identifier |
+| `name::String` | `String` | Yes | Tool name |
+| `arguments::Dict{String, Any}` | `Dict{String, Any}` | Yes | Parsed tool arguments |
+
+**Traceability**:
+- Requirement: FR-003 (tool call extraction)
+- Solution Design: SD-009 (extract tool calls from LLM response)
+
+### SP-006: messageContent Types
+
+**textContent** (`SP-006.1`):
+```julia
+struct textContent <: messageContent
+    text::String
+end
+```
+
+**imageContent** (`SP-006.2`):
+```julia
+struct imageContent <: messageContent
+    data::String                   # Base64-encoded image data
+    mimeType::String               # MIME type (e.g., "image/png")
+end
+```
+
+**reasoningContent** (`SP-006.3`):
+```julia
+struct reasoningContent <: messageContent
+    text::String
+end
+```
+
+**Traceability**:
+- Requirement: FR-006 (message content conversion)
+- Solution Design: SD-006
+
+## 2. LLM Model Configuration (`src/type.jl`)
+
+### SP-007: llmModel
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `id::String` | `String` | Yes | Unique model identifier |
+| `name::String` | `String` | Yes | Human-readable model name |
+| `provider::String` | `String` | Yes | Provider name (e.g., `"anthropic"`, `"openai"`) |
+| `baseUrl::String` | `String` | Yes | API endpoint base URL |
+| `reasoning::Bool` | `Bool` | Yes | Whether the model supports chain-of-thought |
+| `input::Vector{String}` | `Vector{String}` | Yes | Supported input modalities (e.g., `"text"`, `"image"`) |
+| `cost::modelCost` | `modelCost` | Yes | Pricing information |
+| `contextWindow::Int64` | `Int64` | Yes | Maximum context length in tokens |
+| `maxTokens::Int64` | `Int64` | Yes | Maximum output tokens per completion |
+
+**Traceability**:
+- Requirement: FR-002 (LLM agnosticism)
+- Solution Design: SD-003
+
+### SP-008: modelCost
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `input::Float64` | `Float64` | Yes | Price per 1M input tokens |
+| `output::Float64` | `Float64` | Yes | Price per 1M output tokens |
+| `cache_read::Float64` | `Float64` | Yes | Price per 1M cached read tokens |
+| `cache_write::Float64` | `Float64` | Yes | Price per 1M cache write tokens |
+
+**Traceability**:
+- Requirement: N/A (internal tracking data)
+- Solution Design: SD-003
+
+## 3. MCP Tool Integration (`src/toolRegistry.jl`)
+
+### SP-009: tools/list Response Format
+
+**Request**: `mcp("tools/list", sessionId; eventSink=eventSink)`
+
+**Response** (JSON-RPC 2.0):
+```json
+{
+  "result": {
+    "tools": [
+      {
+        "name": "tool_name",
+        "title"?: "Human-readable title",
+        "label"?: "Fallback title",
+        "description": "What the tool does",
+        "inputSchema": {
+          "type": "object",
+          "properties": {
+            "field": { "type": "string", "description": "..." }
+          },
+          "required": ["field"]
+        }
+      }
+    ],
+    "nextCursor"?: "pagination_cursor_or_empty"
+  }
+}
+```
+
+**Response error**:
+```json
+{
+  "error": {
+    "code": -32601,
+    "message": "Method not found"
+  }
+}
+```
+
+**Traceability**:
+- Requirement: FR-004 (MCP discovery)
+- Solution Design: SD-004
+
+### SP-010: tools/call Request/Response Format
+
+**Request**: `mcp("tools/call", toolName, args, sessionId; eventSink=eventSink)`
+
+**Response** (JSON-RPC 2.0):
+```json
+{
+  "result": {
+    "content": [
+      {"type": "text", "text": "Tool output text"}
+    ],
+    "isError": false
+  }
+}
+```
+
+**Response error**:
+```json
+{
+  "result": {
+    "content": [
+      {"type": "text", "text": "Error message"}
+    ],
+    "isError": true
+  }
+}
+```
+
+**Traceability**:
+- Requirement: FR-004 (tool execution)
+- Solution Design: SD-004 (`_wrap_mcp_tool`)
+
+## 4. OpenAI Message Format (`src/utils.jl`)
+
+### SP-011: system message format
+
+```julia
+Dict(
+  "role" => "system",
+  "content" => [Dict("type" => "text", "text" => ctx.systemPrompt)]
+)
+```
+
+**Traceability**:
+- Requirement: FR-006
+- Solution Design: SD-006
+
+### SP-012: userMessage → OpenAI format
+
+```julia
+Dict(
+  "role" => "user",
+  "content" => _messageContentToBlocks(msg.content)
+)
+```
+
+Where `_messageContentToBlocks` converts:
+- `textContent(t)` → `Dict("type" => "text", "text" => t)`
+- `imageContent(data, mime)` → `Dict("type" => "image_url", "image_url" => Dict("url" => "data:$mime;base64,$data"))`
+
+**Traceability**:
+- Requirement: FR-006
+- Solution Design: SD-006
+
+### SP-013: assistantMessageToolCall → OpenAI format
+
+```julia
+Dict(
+  "role" => "assistant",
+  "content" => nothing,
+  "tool_calls" => [
+    Dict(
+      "id" => tc.id,
+      "type" => tc.type,
+      "function" => Dict(
+        "name" => tc.name,
+        "arguments" => JSON.json(tc.arguments)  # serialized as JSON string
+      )
+    )
+  ]
+)
+```
+
+**Traceability**:
+- Requirement: FR-003
+- Solution Design: SD-009
+
+### SP-014: assistantMessage → OpenAI format
+
+```julia
+Dict(
+  "role" => "assistant",
+  "content" => _messageContentToBlocks(msg.content)
+)
+```
+
+**Traceability**:
+- Requirement: FR-006
+- Solution Design: SD-006
+
+### SP-015: toolResultMessage → OpenAI format
+
+```julia
+Dict(
+  "role" => "tool",
+  "tool_call_id" => msg.toolCallId,
+  "content" => _messageContentToBlocks(msg.content)
+)
+```
+
+**Traceability**:
+- Requirement: FR-005, FR-006
+- Solution Design: SD-005
+
+### SP-016: agentTool → OpenAI tools format
+
+```julia
+Dict(
+  "type" => "function",
+  "function" => Dict(
+    "name" => tool.name,
+    "description" => tool.description,
+    "parameters" => tool.inputSchema
+  )
+)
+```
+
+**Traceability**:
+- Requirement: FR-004
+- Solution Design: SD-004, SD-006
+
+## 5. LLM Response Format Agnostic Extraction (`src/agentCore.jl`)
+
+### SP-017: format 1 — LMStudio.jl / vLLM style (dict with nested `message.tool_calls`)
+
+**Response dict** (key: `message::Dict`):
+```json
+{
+  "finish_reason": "tool_calls",
+  "message": {
+    "role": "assistant",
+    "content": "",
+    "reasoning_content": "I need to call getWeather...",
+    "tool_calls": [
+      {
+        "type": "function",
+        "function": {
+          "name": "getWeather",
+          "arguments": "{\"city\":\"Bangkok\"}"  // JSON string
+        },
+        "id": "call_1"
+      }
+    ]
+  }
+}
+```
+
+**When no tool calls**:
+```json
+{
+  "finish_reason": "stop",
+  "message": {
+    "role": "assistant",
+    "content": "It's 22°C in Bangkok"
+  }
+}
+```
+
+**Traceability**:
+- Requirement: FR-002
+- Solution Design: SD-009
+
+### SP-018: format 2 — OpenAI API style (content blocks array)
+
+**Content blocks with tool calls** (`SP-018.1`):
+```json
+{
+  "content": [
+    {
+      "type": "tool_calls",
+      "tool_calls": [
+        {
+          "type": "function",
+          "function": {
+            "name": "getWeather",
+            "arguments": {"city": "Bangkok"}  // already parsed dict
+          },
+          "id": "call_1"
+        }
+      ]
+    }
+  ]
+}
+```
+
+**Content blocks with tool_call (already parsed)** (`SP-018.2`):
+```json
+{
+  "content": [
+    {
+      "type": "tool_call",
+      "name": "getWeather",
+      "arguments": {"city": "Bangkok"},
+      "id": "call_1"
+    }
+  ]
+}
+```
+
+**Content blocks with text only** (`SP-018.3`):
+```json
+{
+  "content": [
+    {"type": "text", "text": "It's 22°C in Bangkok"}
+  ]
+}
+```
+
+**Response dict extras** (common across all formats, `SP-018.4`):
+```json
+{
+  "api": "openai",
+  "provider": "anthropic",
+  "model": "gpt-4",
+  "usage": {"inputTokens": 100, "outputTokens": 50},
+  "error_message": "Optional error"
+}
+```
+
+**Traceability**:
+- Requirement: FR-002
+- Solution Design: SD-009
+
+## 6. Agent Loop Configuration (`src/type.jl` + `src/agentCore.jl`)
+
+### SP-019: agentContext
+
+```julia
+struct agentContext
+    systemPrompt::String
+    messages::Vector{agentMessage}
+    tools::Union{OrderedDict{String, agentTool}, Nothing}  # O(1) lookup by name
+end
+```
+
+**Traceability**:
+- Requirement: FR-003 (context preparation)
+- Solution Design: SD-003, SD-004
+
+### SP-020: agentState
+
+```julia
+mutable struct agentState
+    systemPrompt::String
+    model::Union{llmModel, Nothing}
+    tools::OrderedDict{String, agentTool}      # insertion-ordered
+    messages::Vector{agentMessage}             # history (no system prompt)
+    pendingToolCalls::Vector{String}           # tool call IDs waiting for results
+    errorMessage::Union{String, Nothing}       # last error message
+end
+```
+
+**Traceability**:
+- Requirement: FR-001 (agent state management)
+- Solution Design: SD-006
+
+### SP-021: agentLoopConfig
+
+```julia
+struct agentLoopConfig
+    beforeToolCall::Union{Function, Nothing}
+    afterToolCall::Union{Function, Nothing}
+    toolExecution::String  # "sequential" or "parallel"
+end
+```
+
+**Traceability**:
+- Requirement: FR-005 (lifecycle hooks)
+- Solution Design: SD-005
+
+### SP-022: abortSignal
+
+```julia
+struct abortSignal
+    aborted::Bool
+end
+```
+
+**Traceability**:
+- Requirement: NFR-101 (timeout handling)
+- Solution Design: SD-008
+
+## 7. Tool Validation Hooks (`src/utils.jl`)
+
+### SP-023: validateRequiredArgs (default hook)
+
+**Signature**: `validateRequiredArgs(args::AbstractDict{String, Any}, inputSchema::AbstractDict{String, Any})::Union{Nothing, String}`
+
+**Behavior**:
+- Extract `required = get(inputSchema, "required", Any[])`
+- Check each required field exists in `keys(args)`
+- Return `nothing` if all present
+- Return `"Missing required arguments: $(join(missing, ", "))"` if any missing
+
+**Traceability**:
+- Requirement: NFR-301 (input validation)
+- Solution Design: SD-005
+
+### SP-024: validateToolArguments (wrapper hook)
+
+**Signature**: `validateToolArguments(tool::agentTool, prepared::agentToolCall)::AbstractDict{String, Any}`
+
+**Behavior**:
+- If `tool.validateRequiredArgs !== nothing`: call `tool.validateRequiredArgs(prepared.arguments)`
+- Else: call `validateRequiredArgs(prepared.arguments, tool.inputSchema)`
+- If returns `!nothing`: throw `ArgumentError(result)`
+- Caller (`prepareToolCall`) catches and returns `immediateOutcome`
+
+**Traceability**:
+- Requirement: FR-005 (tool validation)
+- Solution Design: SD-005

docs/6-validation.md

@@ -0,0 +1,192 @@
+# Validation: YiemAgent Agent Framework
+
+## Test Architecture Overview
+
+Because YiemAgent is a Julia library (not a web service), validation tests focus on:
+- **Unit tests**: message type conversion, tool execution logic, format extraction
+- **Integration tests**: full agent loop with stubbed LLM and MCP
+- **Contract tests**: response format compatibility with OpenAI/LMStudio specs
+- **Performance tests**: agent loop overhead for sequential tool calls
+
+Every specification rule (SP-XXX), walkthrough journey (WT-XXX), and requirement (FR-XXX) must have a corresponding test.
+
+---
+
+## Test Types Mapping to Source Documents
+
+| Test Type | Source Document | Example Coverage |
+|-----------|-----------------|------------------|
+| **Unit tests** | `4-specification.md` (SP-001..SP-024), `src/utils.jl` (converters) | Message type → OpenAI format conversion; tool argument validation |
+| **Integration tests** | `3-walkthrough.md` (WT-001..WT-032) | Full agent loop: prompt → llmCall → tool extract → execute → response |
+| **Contract tests** | `4-specification.md` SP-017, SP-018 | Response format 1 (LMStudio/vLLM) and format 2 (OpenAI) parsed correctly |
+| **Performance tests** | `1-requirements.md` FR-003, NFR-101 | Agent loop overhead <500ms per sequential tool call |
+| **Penetration tests** | `1-requirements.md` NFR-301 | Tool argument injection; MCP spoofing |
+| **Schema validation** | `4-specification.md` SP-023, SP-024 | Required fields, type checking on tool arguments |
+
+---
+
+## Specification-to-Test Execution Trace
+
+Every traceable item — specification rules (SP-XXX), walkthrough journeys (WT-XXX), and requirements (FR-XXX) — must have a runnable test. This trace maps each item to its corresponding test file and test function.
+
+**Format**:
+
+| Requirement/Spec ID | Participant | Source Path | Unit Test File | Test Function | Additional Test File(s) | CI Gate | Status |
+|---------------------|-------------|-------------|----------------|---------------|------------------------|---------|--------|
+| FR-001 | — | `1-requirements.md:3.1` | `tests/unit/channel-handling.jl` | `test_input_channel_priority` | `tests/integration/full-agent-loop.jl` | `ci/julia-unit.yaml` | **TODO** |
+| FR-002 | — | `1-requirements.md:3.2` | `tests/unit/llm-format-extraction.jl` | `test_format1_tool_calls` | `tests/contract/response-format.jl` | `ci/julia-unit.yaml` | **TODO** |
+| FR-003 | — | `1-requirements.md:3.3` | `tests/integration/tool-call-loop.jl` | `test_loop_exits_on_no_tool_calls` | — | `ci/julia-integration.yaml` | **TODO** |
+| FR-004 | — | `1-requirements.md:3.4` | `tests/unit/mcp-tool-discovery.jl` | `test_tools_list_with_pagination` | `tests/contract/mcp-json-rpc.jl` | `ci/julia-unit.yaml` | **TODO** |
+| FR-005 | — | `1-requirements.md:3.5` | `tests/unit/tool-hooks.jl` | `test_beforeToolCall_blocks` | `tests/integration/agent-loop.jl` | `ci/julia-unit.yaml` | **TODO** |
+| FR-006 | — | `1-requirements.md:3.6` | `tests/unit/message-conversion.jl` | `test_userMessage_to_OpenAI` | — | `ci/julia-unit.yaml` | **TODO** |
+| WT-001 | Caller | `3-walkthrough.md:#agent-creation` | `integration/agent-creation.jl` | `test_agent_construction_and_lifecycle` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-002 | Caller | `3-walkthrough.md:#send-message` | `integration/message-flow.jl` | `test_promptAgent_and_takeResponse` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-003 | Caller | `3-walkthrough.md:#receive-response` | `integration/response-format.jl` | `test_output_channel_delivery` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-004 | Caller | `3-walkthrough.md:#follow-up` | `integration/followup.jl` | `test_followUp_while_busy` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-005 | Caller | `3-walkthrough.md:#stop-agent` | `integration/shutdown.jl` | `test_stopAgent_channel_cleanup` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-010 | LLM Provider | `3-walkthrough.md:#receive-formatted` | `integration/llm-call.jl` | `test_formatMsgForLLM_output_structure` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-011 | LLM Provider | `3-walkthrough.md:#response-no-tool-calls` | `integration/non-tool-response.jl` | `test_lmm_response_no_tool_calls_breaks_loop` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-012 | LLM Provider | `3-walkthrough.md:#response-with-tool` | `integration/tool-response.jl` | `test_lmm_response_with_tool_calls` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-020 | MCP Server | `3-walkthrough.md:#discover-tools` | `unit/mcp-discovery.jl` | `test_mcp_tools_list_parsed` | `contract/mcp-json-rpc.jl` | `ci/julia-unit.yaml` | **TODO** |
+| WT-021 | MCP Server | `3-walkthrough.md:#execute-tool` | `unit/mcp-tool-execute.jl` | `test_mcp_tools_call_response_parsed` | — | `ci/julia-unit.yaml` | **TODO** |
+| WT-022 | MCP Server | `3-walkthrough.md:#mcp-error` | `unit/mcp-error-handling.jl` | `test_mcp_response_error_returned` | — | `ci/julia-unit.yaml` | **TODO** |
+| WT-030 | Error | `3-walkthrough.md:#llm-timeout` | `integration/llm-timeout.jl` | `test_LLM_timeout_agent_loop_exits` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-031 | Error | `3-walkthrough.md:#tool-throw` | `integration/tool-error.jl` | `test_tool_execute_throws_returns_error_result` | — | `ci/julia-integration.yaml` | **TODO** |
+| WT-032 | Error | `3-walkthrough.md:#beforeToolCall-block` | `unit/beforeToolCall-block.jl` | `test_beforeToolCall_blocks_execution` | — | `ci/julia-unit.yaml` | **TODO** |
+| SP-001 | — | `4-specification.md:#SP-001:userMessage` | `unit/message-types.jl` | `test_userMessage_struct_default_roles` | — | `ci/julia-unit.yaml` | **TODO** |
+| SP-002 | — | `4-specification.md:#SP-002:assistantMessage` | `unit/message-types.jl` | `test_assistantMessage_struct_default_stopReason_end_turn` | — | `ci/julia-unit.yaml` | **TODO** |
+| SP-003 | — | `4-specification.md:#SP-003:assistantMessageToolCall` | `unit/message-types.jl` | `test_assistantMessageToolCall_struct_default_stopReason_tool_calls` | — | `ci/julia-unit.yaml` | **TODO** |
+| SP-004 | — | `4-specification.md:#SP-004:toolResultMessage` | `unit/message-types.jl` | `test_toolResultMessage_struct_defaults` | — | `ci/julia-unit.yaml` | **TODO** |
+| SP-005 | — | `4-specification.md:#SP-005:agentToolCall` | `unit/message-types.jl` | `test_agentToolCall_type_is_always_function` | — | `ci/julia-unit.yaml` | **TODO** |
+| SP-006.1,2,3 | — | `4-specification.md:#SP-006:content-types` | `unit/content-types.jl` | `test_textContent_imageContent_reasoningContent` | — | `ci/julia-unit.yaml` | **TODO** |
+| SP-017 | — | `4-specification.md:#SP-017` | `contract/response-format.jl` | `test_lmStudio_vLLM_format_parsed` | — | `ci/julia-contract.yaml` | **TODO** |
+| SP-018.1,2,3,4 | — | `4-specification.md:#SP-018` | `contract/response-format.jl` | `test_openAI_format_blocks_parsed` | — | `ci/julia-contract.yaml` | **TODO** |
+| SP-023 | — | `4-specification.md:#SP-023` | `unit/tool-validation.jl` | `test_validateRequiredArgs_missing_fields` | — | `ci/julia-unit.yaml` | **TODO** |
+| SP-024 | — | `4-specification.md:#SP-024` | `unit/tool-validation.jl` | `test_validateToolArguments_throws_on_missing` | — | `ci/julia-unit.yaml` | **TODO** |
+| NFR-101 | — | `1-requirements.md:4.1` | `perf/agent-loop.jl` | `perf_agent_loop_overhead_<500ms` | `tests/perf/concurrency.jl` | `ci/julia-perf.yaml` | **TODO** |
+| NFR-102 | — | `1-requirements.md:4.1` | `perf/mcp-discovery.jl` | `perf_mcp_discovery_50_tools_<2s` | — | `ci/julia-perf.yaml` | **TODO** |
+| NFR-201 | — | `1-requirements.md:4.2` | `unit/error-handling.jl` | `test_agent_loop_catches_and_logs_errors` | `integration/llm-timeout.jl` | `ci/julia-unit.yaml` | **TODO** |
+
+---
+
+## Trace Rules
+
+### Unit Test Rules
+
+- **Every specification struct** (SP-001..SP-008, SP-019..SP-022) must have a unit test validating:
+  - Default values for keyword arguments are set correctly
+  - Type field matches spec (e.g., `agentToolCall.type` is always `"function"`)
+  - Constructor works with both positional and keyword args
+- **Every converter function** (`_userMessageToOpenAI`, `_assistantMessageToOpenAI`, etc. SP-011..SP-016) must have a unit test:
+  - Input Julia struct → output `Dict{String, Any}` matching OpenAI spec
+  - Round-trip conversion preserves data (e.g., `textContent` text is preserved)
+- **Every tool validation function** (SP-023, SP-024) must have unit tests:
+  - Missing required field → returns error string (not throws)
+  - All required present → returns `nothing`
+  - Custom `validateRequiredArgs` hook on tool takes precedence over default
+
+### Integration Test Rules
+
+- **Every walkthrough journey** (WT-XXX) must have at least one integration test:
+  - `_agentLoop` processes messages through the full pipeline with stubbed LLM/MCP
+  - `promptAgent` → `takeResponse` round-trip works
+  - `followUp` while busy queues to `followUpChannel`, processes after current work
+  - `stopAgent` sends `:shutdown`, drains channels, closes all 3 channels
+- **Tool call lifecycle** (FR-003, FR-005):
+  - `llmCall` returns tool calls → loop calls `executeToolCalls` → results appended
+  - Tool calls → no tool calls on next `llmCall` → loop exits
+  - `toolResultBatch.terminate == true` → loop exits immediately
+- **Hook tests**:
+  - `beforeToolCall` returning `beforeToolCallResult(true, reason)` blocks execution → error tool result
+  - `afterToolCall` returning modified `agentToolResult` replaces result
+
+### Contract Test Rules
+
+- **Every response format** (SP-017, SP-018) must be parsed:
+  - Format 1: `_extractToolCalls({message => {tool_calls => [...]}})` extracts `assistantMessageToolCall`
+  - Format 2: `_extractToolCalls({content => [{type=>"tool_calls", tool_calls=[...]}}])` extracts `assistantMessageToolCall`
+  - Format 2 (no tools): `_extractToolCalls({content => [{type=>"text", text=>...}]})` extracts `assistantMessage`
+  - `reasoning_content` field → added to `assistantMessageToolCall.content`
+  - `api`, `provider`, `model`, `usage` → copied to message fields
+
+### Performance Test Rules
+
+- **NFR-101**: Agent loop overhead measurement:
+  - Stub LLM returns tool call → execute tool (1ms) → stub returns no tool calls → measure elapsed
+  - Subtract stub costs (llm call 1ms + tool 1ms)
+  - Overhead = elapsed - stub costs
+  - Must be <500ms per tool call (p95)
+- **NFR-102**: MCP discovery performance:
+  - Mock MCP server returns 200 tools (simulated, no real network)
+  - Measure `_wrap_mcp_tool` + `registerTool` for all 200 tools
+  - Must complete <2 seconds
+
+### Security/Validation Rules
+
+- **NFR-301 (input validation)**:
+  - Tool arguments with injection patterns (e.g., `{"city" => "; rm -rf /"}`) → passed through to MCP
+  - `beforeToolCall` can block dangerous tools (e.g., `toolCall.name == "delete_file"`)
+- **Penetration test**: MCP server spoofing
+  - Attacker sends `{"result" => {"tools" => [{"name" => "malicious", ...}]}}` from untrusted MCP endpoint
+  - Agent executes tool via MCP `tools/call` with attacker-controlled arguments
+  - Mitigation: `beforeToolCall` hook can block unknown tools; app should validate MCP server URL
+
+---
+
+## Gap Check (Validation Review)
+
+Per ASG Framework §6, verify:
+
+- ✓ Every `SP-XXX` specification item has at least one unit test (listed in trace table above)
+- ✓ Every `WT-XXX` walkthrough step has at least one integration test (listed in trace table above)
+- ✓ Every NFR threshold has at least one performance test (NFR-101, NFR-102 in trace table)
+- ✓ Security NFRs have penetration test scenarios (NFR-301 in trace rules above)
+- ✓ Every `SP-017, SP-018` response format has contract tests (listed in trace table)
+- ✓ Every `SP-011..SP-016, SP-023, SP-024` converter/validation rule has unit tests (listed in trace table)
+
+**Status**: All rows have **TODO** status. Tests must be written before integration into production CI pipeline.
+
+---
+
+## CI Pipeline
+
+```yaml
+# ci/julia-pipeline.yaml (placeholder)
+name: Julia YiemAgent CI
+on: [push, pull_request]
+
+jobs:
+  unit:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: julia-actions/julia-buildpkg@latest
+      - run: julia --project=. -e 'using Pkg; Pkg.instantiate()'
+      - run: julia --project=./tests -e 'using Test; Test.@testset "Unit tests" # include tests/unit/run_tests.jl"'
+  
+  integration:
+    runs-on: ubuntu-latest
+    needs: unit
+    steps:
+      - run: julia --project=./tests -e 'using Test; Test.@testset "Integration" # include tests/integration/run_tests.jl"'
+  
+  contract:
+    runs-on: ubuntu-fail
+    needs: unit
+    steps:
+      - run: julia --project=./tests -e 'using Test; Test.@testset "Contract" # include tests/contract/run_tests.jl"'
+  
+  perf:
+    runs-on: ubuntu-latest
+    needs: [unit, integration]
+    steps:
+      - run: julia --project=./tests -e 'using Test; Test.@testset "Performance" # include tests/perf/run_tests.jl"'
+    # Fail if NFR-101 or NFR-102 thresholds breached
+  
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - run: julia --eval 'using Aqua; Aqua.test_all(YiemAgent)'
+```
+
+**Gap Check**: Per ASG §5, every spec item maps to a CI gate. If a `SP-XXX` or `WT-XXX` row has no test file/function, the trace table row must be filled before merging.

docs/7-implementation-plan.md

@@ -0,0 +1,217 @@
+# Implementation Plan: YiemAgent Agent Framework
+
+## 1. Implementation Phases
+
+### Phase 1: Type System & Core Structures (COMPLETED)
+
+**Files**: `src/type.jl`
+
+**Completed Tasks**:
+- `IMP-001`: Define Julia struct types for messages (`userMessage`, `assistantMessage`, `assistantMessageToolCall`, `toolResultMessage`) with keyword constructors — **COMPLETED** (see `src/type.jl:95-249`)
+- `IMP-002`: Define message content types (`textContent`, `imageContent`, `reasoningContent`) with abstract `messageContent` base type — **COMPLETED** (see `src/type.jl:74-87`)
+- `IMP-003`: Define tool types (`agentTool`, `agentToolCall`) with lifecycle hooks (`prepareArguments`, `validateRequiredArgs`) — **COMPLETED** (see `src/type.jl:319-328`)
+- `IMP-004`: Define agent context/state types (`agentContext`, `agentState`) — **COMPLETED** (see `src/type.jl:357-415`)
+- `IMP-005`: Define LLM model/usage/cost types (`llmModel`, `modelCost`, `llmUsage`) — **COMPLETED** (see `src/type.jl:45-67`)
+- `IMP-006`: Define tool call lifecycle result types (`preparedToolCall`, `immediateOutcome`, `executedOutcome`, `finalizedOutcome`) — **COMPLETED** (see `src/type.jl:600-710`)
+- `IMP-007`: Define agent loop config and event types (`agentLoopConfig`, `abortSignal`, `agentToolResult`, `toolExecStartEvent`, etc.) — **COMPLETED** (see `src/type.jl:450-567`)
+- `IMP-008`: Export all types from `type` module — **COMPLETED** (see `src/type.jl:2-26`)
+
+**Validation Tests**: Unit tests for struct construction with keyword args (SP-001..SP-008, SP-019..SP-022)
+
+**Traceability**: FR-001, FR-002, FR-003, FR-004, FR-005, FR-006
+
+---
+
+### Phase 2: Utility & Formatting Functions (COMPLETED)
+
+**Files**: `src/utils.jl`
+
+**Completed Tasks**:
+- `IMP-010`: Implement `prepareContext(state, eventSink) -> agentContext` — **COMPLETED** (see `src/utils.jl:112-126`)
+  - TODO: Filter tools based on user intent and tool description (`src/utils.jl:114`)
+  - TODO: Inject system prompt context (e.g., time, session state) (`src/utils.jl:118`)
+  - TODO: Prune/reorder messages (`src/utils.jl:121`)
+- `IMP-011`: Implement `formatMsgForLLM(ctx, eventSink) -> Dict{String, Any}` — **COMPLETED** (see `src/utils.jl:159-244`)
+  - Converts system prompt to system message
+  - Converts conversation messages via `_userMessageToOpenAI`, `_assistantMessageToOpenAI`, `_assistantMessageToolCallToOpenAI`, `_toolResultMessageToOpenAI`
+  - Converts `ctx.tools` to OpenAI function tool format via `_toolsToOpenAI`
+- `IMP-012`: Implement message converters (`_userMessageToOpenAI`, `_assistantMessageToOpenAI`, `_assistantMessageToolCallToOpenAI`, `_toolResultMessageToOpenAI`) — **COMPLETED** (see `src/utils.jl:321-387`)
+- `IMP-013`: Implement `_messageContentToBlocks` — converts `textContent`/`imageContent` to OpenAI content blocks — **COMPLETED** (see `src/utils.jl:396-413`)
+- `IMP-014`: Implement `_toolsToOpenAI` — converts `OrderedDict{String, agentTool}` to OpenAI tools array — **COMPLETED** (see `src/utils.jl:429-447`)
+- `IMP-015`: Implement `beforeToolCall` hook default (returns `beforeToolCallResult(false, "N/A")`) — **COMPLETED** (see `src/utils.jl:278-284`)
+- `IMP-016`: Implement `afterToolCall` hook default (returns `nothing`) — **COMPLETED** (see `src/utils.jl:310-316`)
+- `IMP-017`: Implement `validateRequiredArgs(args, inputSchema)` — validates required fields in tool arguments — **COMPLETED** (see `src/utils.jl:479-497`)
+- `IMP-018`: Implement `validateToolArguments(tool, prepared)` — wrapper that calls tool-specific or default hook — **COMPLETED** (see `src/utils.jl:533-546`)
+
+**Validation Tests**: Unit tests for converters, validation hooks, and OpenAI format output structure (SP-011..SP-016, SP-023, SP-024)
+
+**Traceability**: FR-001, FR-002, FR-003, FR-004, FR-005, FR-006
+
+---
+
+### Phase 3: Tool Registry & MCP Integration (COMPLETED)
+
+**Files**: `src/toolRegistry.jl`
+
+**Completed Tasks**:
+- `IMP-020`: Implement `_extract_text_content(result)` — extracts text from MCP tool result content array — **COMPLETED** (see `src/toolRegistry.jl:19-33`)
+- `IMP-021`: Implement `_wrap_mcp_tool(mcpserver, tool_def)` — wraps MCP tool as `agentTool` with `execute` calling `mcpserver("tools/call", ...)` — **COMPLETED** (see `src/toolRegistry.jl:41-106`)
+  - Handles MCP error responses (`haskey(response, "error")`)
+  - Extracts text and `isError` from result
+- `IMP-022`: Implement `register_mcp_tools(tools, mcpserver)` — queries `tools/list` with pagination (`nextCursor`) — **COMPLETED** (see `src/toolRegistry.jl:118-186`)
+  - Handles both `result.tools` and `result.error` response shapes
+  - Skips already registered tools (idempotent)
+- `IMP-023`: Implement `registerTool(tools, tool)` — adds tool to OrderedDict — **COMPLETED** (see `src/toolRegistry.jl:200-205`)
+- `IMP-024`: Implement `listTool(tools, mcpserver)` — builtin tool that triggers runtime discovery and lists all tools — **COMPLETED** (see `src/toolRegistry.jl:235-271`)
+- `IMP-025`: Implement `registerAllTools(tools, mcpserver)` — discovers existing tools + registers `listTools` — **COMPLETED** (see `src/toolRegistry.jl:286-289`)
+
+**Validation Tests**: Unit tests for MCP tools/list parsing, pagination, tool registration idempotency; contract tests for JSON-RPC 2.0 format (SP-009, SP-010)
+
+**Traceability**: FR-002, FR-004
+
+---
+
+### Phase 4: Agent Core & Loop (COMPLETED)
+
+**Files**: `src/agentCore.jl`
+
+**Completed Tasks**:
+- `IMP-030`: Define `yiemAgent` struct with channels, callbacks, MCP server — **COMPLETED** (see `src/agentCore.jl:17-95`)
+- `IMP-031`: Implement `yiemAgent` constructor — creates channels, discovers MCP tools, spawns `_agentLoop` task — **COMPLETED** (see `src/agentCore.jl:126-199`)
+- `IMP-032`: Implement `_agentLoop(agent)` — background `@spawn` task managing `inputChannel`/`followUpChannel` priority dispatch — **COMPLETED** (see `src/agentCore.jl:225-364`)
+  - Handles shutdown signal: drains all channels, breaks loop
+  - On error: catches, calls `eventSink(errMsg)`, exits loop
+- `IMP-033`: Implement `_processMessage(...)` — drains input, calls `llmCall`, extracts tool calls, executes, loops until no tool calls — **COMPLETED** (see `src/agentCore.jl:394-544`)
+  - `while true` loop: `llmCall()` → `_extractToolCalls()` → if `hasToolCalls && length > 0` execute → append results → repeat
+  - On `toolResultBatch.terminate == true`: build final response and break
+  - Catches all errors and calls `eventSink(err_msg)`
+- `IMP-034`: Implement `_extractToolCalls(response)` — handles **both** formats:
+  - Format 1: `response["message"]["tool_calls"]` (LMStudio/vLLM style) — **COMPLETED** (see `src/agentCore.jl:671-683`)
+  - Format 2: `response.content` blocks with `type => "tool_calls"` or `type => "tool_call"` (OpenAI API style) — **COMPLETED** (see `src/agentCore.jl:685-719`)
+  - Extracts `reasoning_content`, `content` text blocks, `api`, `provider`, `model`, `usage`, `finish_reason` — **COMPLETED** (see `src/agentCore.jl:723-797`)
+  - Returns `(hasToolCalls, toolCallList, assistant_message)` — **COMPLETED** (see `src/agentCore.jl:799-827`)
+- `IMP-035`: Implement `prepareToolCall(context, assistantMsg, toolCall, config, signal, eventSink)` — resolves tool, validates args, runs `beforeToolCall` hook, never throws — **COMPLETED** (see `src/agentCore.jl:974-1035`)
+- `IMP-036`: Implement `executePreparedToolCall(prep, signal, eventSink)` — calls `tool.execute(toolCallId, args, ...)` — **COMPLETED** (see `src/agentCore.jl:1078-1100`)
+- `IMP-037`: Implement `finalizeExecutedToolCall(context, msg, prep, executed, config, signal, eventSink)` — runs `afterToolCall` hook, catches errors — **COMPLETED** (see `src/agentCore.jl:1159-1198`)
+- `IMP-038`: Implement `prepareToolCallArguments(tool, toolCall)` — calls `tool.prepareArguments` if present — **COMPLETED** (see `src/agentCore.jl:908-917`)
+- `IMP-039`: Implement `createErrorToolResult(msg)` — builds `agentToolResult` with error content — **COMPLETED** (see `src/agentCore.jl:572-574`)
+- `IMP-040`: Implement `createToolResultMessage(f)` — builds `toolResultMessage` from `finalizedOutcome` — **COMPLETED** (see `src/agentCore.jl:606-612`)
+
+**Note**: Sequential and parallel execution implementations continue beyond line 1207 of `src/agentCore.jl` — implementation incomplete, requires review to ensure execution methods properly call hooks and emit lifecycle events. See IMP-041.
+
+**Validation Tests**: Integration tests for full agent loop with mocked LLM and MCP; contract tests for response format extraction (WT-001..WT-032, SP-017, SP-018)
+
+**Traceability**: FR-001, FR-002, FR-003, FR-004, FR-005
+
+---
+
+### Phase 5: Public API (COMPLETED)
+
+**Files**: `src/api.jl`
+
+**Completed Tasks**:
+- `IMP-050`: Implement `promptAgent(agent, msg::AbstractDict)` — put to `inputChannel`, returns agent for chaining — **COMPLETED** (see `src/api.jl:36-39`)
+- `IMP-051`: Implement `promptAgent(agent, text::String)` — wraps text as OpenAI `Dict` and put to `inputChannel` — **COMPLETED** (see `src/api.jl:41-44`)
+- `IMP-052`: Implement `takeResponse(agent)` — blocks on `outputChannel` `take!`, returns OpenAI dict response — **COMPLETED** (see `src/api.jl:66-68`)
+- `IMP-053`: Implement `followUp(agent, msg::AbstractDict)` — put to `followUpChannel`, returns agent — **COMPLETED** (see `src/api.jl:94-97`)
+- `IMP-054`: Implement `followUp(agent, text::String)` — wraps text as OpenAI `Dict` and put to `followUpChannel` — **COMPLETED** (see `src/api.jl:99-102`)
+- `IMP-055`: Implement `stopAgent(agent)` — puts `:shutdown`, `fetch(_agentLoop)`, closes all channels — **COMPLETED** (see `src/api.jl:126-139`)
+
+**Validation Tests**: Integration tests for send/receive/stop lifecycle (WT-001..WT-005)
+
+**Traceability**: FR-001, FR-006
+
+---
+
+## 2. Module Structure
+
+```
+src/
+├── YiemAgent.jl     # Parent module, includes submodules in dependency order
+├── type.jl           # All Julia struct types (exports 30+ types)
+├── utils.jl          # Formatting, validation, hooks (default implementations)
+├── toolRegistry.jl   # MCP tool discovery, registration, listTools builtin
+├── agentCore.jl      # yiemAgent struct, _agentLoop, _processMessage, extractToolCalls
+└── api.jl            # Public: promptAgent, takeResponse, followUp, stopAgent
+```
+
+**Include order** (per `src/YiemAgent.jl` lines 6-19):
+1. `type.jl` (no dependencies)
+2. `utils.jl` (depends on `type`)
+3. `toolRegistry.jl` (depends on `type`)
+4. `agentCore.jl` (depends on `type`, `utils`, `toolRegistry`)
+5. `api.jl` (depends on `type`, `utils`, `agentCore`, `toolRegistry`)
+
+---
+
+## 3. Task Dependencies
+
+| Task ID | Title | Depends On | Status |
+|---------|-------|------------|--------|
+| IMP-001..IMP-008 | Type system | — | **COMPLETED** |
+| IMP-010..IMP-018 | Utils (formatMsgForLLM, converters, hooks) | IMP-001..IMP-008 | **COMPLETED** |
+| IMP-020..IMP-025 | Tool registry (MCP discovery) | IMP-003, IMP-017 | **COMPLETED** |
+| IMP-030..IMP-040 | Agent core (agent loop, tool call execution) | IMP-001..IMP-008, IMP-010..IMP-018, IMP-020..IMP-025 | **INCOMPLETE*** (see note) |
+| IMP-050..IMP-055 | Public API | IMP-030..IMP-040 | **COMPLETED** |
+
+**NOTE on IMP-030..IMP-040**: The sequential and parallel execution functions (`executeToolCallsSequential`, `executeToolCallsParallel`, `executeToolCalls`) begin after line 1207 of `src/agentCore.jl` but the full implementation is truncated in file reads and requires review. The exported functions (`executePreparedToolCall`, `prepareToolCall`, `executeToolCallsSequential`, `executeToolCallsParallel`, `executeToolCalls`) are declared (`src/agentCore.jl:4-5`) but their full bodies may be incomplete. **Requires manual validation.**
+
+**Traceability**: IMP-001..IMP-008 → SP-001..SP-008, IMP-020..IMP-025 → SP-009, SP-010; IMP-010..IMP-018 → SP-011..SP-016, SP-023, SP-024; IMP-030..IMP-040 → SP-017, SP-018, SP-019..SP-022
+
+---
+
+## 4. Remaining Work (TODOs)
+
+| File | Line | Description | Priority |
+|------|------|-------------|----------|
+| `src/utils.jl` | 114 | Filter tools from `state.tools` based on user intent and tool description | High |
+| `src/utils.jl` | 118 | Add system prompt context: time, session state, user preferences | Medium |
+| `src/utils.jl` | 121 | Prune/reorder/conversation messages before sending to LLM | Medium |
+| `src/agentCore.jl` | post-1207 | Complete and validate `executeToolCallsSequential`, `executeToolCallsParallel`, `executeToolCalls` implementations (currently exported but body may be incomplete) | **Critical** |
+| `tests/unit/...` | — | Write unit tests for all struct types (IMP-001..IMP-008) and converters (SP-011..SP-016) | High |
+| `tests/unit/...` | — | Write unit tests for tool validation hooks (SP-023, SP-024) | High |
+| `tests/integration/...` | — | Write integration tests for agent loop with mocked LLM/MCP (WT-001..WT-032) | High |
+| `tests/contract/...` | — | Write contract tests for response formats (SP-017, SP-018) | Medium |
+| `tests/perf/...` | — | Write performance tests (NFR-101, NFR-102) | Medium |
+
+---
+
+## 5. Build & Deployment
+
+### Build
+
+```bash
+cd /home/ton/docker-apps/sommpanion/YiemAgent
+julia --project=. -e 'using Pkg; Pkg.instantiate(); using Pkg; Pkg.precompile()'
+```
+
+### Test
+
+```bash
+julia --project=./tests -e 'using Pkg; Pkg.instantiate()'
+julia --project=./tests -e 'using Test; Test.@testset "Unit" begin include("tests/unit/run_tests.jl") end'
+julia --project=./tests -e 'using Test; Test.@testset "Integration" begin include("tests/integration/run_tests.jl") end'
+```
+
+### CI/CD Preparation
+
+- Add `.github/workflows/julia.yaml` referencing `ci/julia-pipeline.yaml` steps
+- Add `Project.toml` dependencies:
+  ```toml
+  [deps]
+  JSON = "682c8566-3430-5439-8f4e-b64832e8aaba"
+  DataStructures = "892a3cad-33e6-51e5-808c-f3d90072be70"
+  HTTP = "cd6eee30-4dd5-5329-8bad-3259ec48d0f4"
+  # plus: UUIDs, Dates, PrettyPrinting, Serialization, DataFrames, NATS, LibPQ
+  ```
+
+---
+
+## 6. Risk Mitigation
+
+| Risk | Mitigation |
+|------|------------|
+| Line 1207+ of `agentCore.jl` (sequential/parallel execution) is incomplete or broken | Manually review and validate `executeToolCallsSequential`, `executeToolCallsParallel`, `executeToolCalls` against spec SP-021 (agentLoopConfig) and trace to test `test_tool_call_execution_sequence` |
+| `GeneralUtils` internal package unavailable in CI | Ensure `GeneralUtils` is installed as dependency (via `Pkg.develop` or registry) |
+| NATS `mcpServer` struct is abstract — no concrete implementation | Provide at least one mock `mcpServer` for tests that returns canned JSON responses |
+| Message history grows unbounded | Implement `prepareContext` pruning (TODO in code); add NFR-104 for max history tokens |

docs/8-runbook.md

@@ -0,0 +1,445 @@
+# Runbook: YiemAgent Agent Framework
+
+**Severity**: N/A (library — not a deployed service)
+**Estimated Time**: N/A
+
+This runbook documents operational guidance for developers working with the YiemAgent agent framework. It is not an operations runbook for a production service, but a developer handbook for debugging, troubleshooting, and extending the framework.
+
+---
+
+## 1. Agent Lifecycle
+
+### 1.1: Create an Agent
+
+**Steps**:
+```julia
+using YiemAgent
+
+# 1. Implement llmCall function
+function my_llmCall(messages; eventSink=nothing)
+    # Call LLM API (e.g., OpenAI, Anthropic, LMStudio)
+    # Return dict with keys: "message", "content", "finish_reason", "api", "provider", "model", "usage"
+    eventSink("my_llmCall: sending $(length(messages["messages"])) messages"; publish=true)
+    # ... API call ...
+    return response_dict
+end
+
+# 2. Implement mcpServer callable struct
+struct MyMCPClient
+    conn::NATS.Connection   # or HTTP.Client
+    topic::String
+    senderID::String
+end
+
+function (mcp::MyMCPClient)(method::String, args...; eventSink=nothing)
+    # JSON-RPC 2.0 calls:
+    #   mcp("tools/list", sessionId; eventSink=eventSink)
+    #   mcp("tools/call", toolName, args, sessionId; eventSink=eventSink)
+    # ... NATS/HTTP call ...
+    return jsonrpc_response
+end
+
+# 3. Create agent
+mcp = MyMCPClient(nats_conn, "yiemagent:tools", "agent_1")
+agent = yiemAgent(
+    my_llmCall;
+    systemPrompt="You are a wine assistant.\nYou have tools for searching wine inventory.",
+    tools=mcp_tools,              # optional: pass pre-discovered tools
+    prepareContext=my_prepareContext,   # optional: custom context preparation
+    formatMsgForLLM=my_formatMsgForLLM, # optional: custom formatting for provider
+    beforeToolCall=my_beforeToolCall,   # optional: approval gate
+    afterToolCall=my_afterToolCall,     # optional: result sanitization
+    mcpServer=mcp,
+    parallelToolExecute=false,
+    eventSink=my_eventSink,
+)
+```
+
+> **Reference**: `2-solution_design.md:SD-001` (channel-based async), `SD-002` (background task), `SD-004` (MCP struct)
+
+### 1.2: Send a Message
+
+```julia
+# Option 1: Send a string message (automatically wrapped as OpenAI dict)
+promptAgent(agent, "What wines have notes of oak and vanilla?")
+
+# Option 2: Send a full OpenAI-format dict with images
+msg = Dict(
+    "role" => "user",
+    "content" => [
+        Dict("type" => "text", "text" => "What wines match these notes?"),
+        Dict("type" => "image_url", "image_url" => Dict("url" => "data:image/png;base64,..."))
+    ]
+)
+promptAgent(agent, msg)
+```
+
+### 1.3: Receive a Response
+
+```julia
+# Blocks until agent finishes processing (no tool calls or tool calls complete)
+response = takeResponse(agent)
+# response is a Dict in OpenAI format:
+#   Dict("role" => "assistant", "content" => [Dict("type" => "text", "text" => "...")])
+
+# For text-only response:
+if response["content"][1]["type"] == "text"
+    println(response["content"][1]["text"])
+end
+```
+
+### 1.4: Send Follow-up While Agent is Busy
+
+```julia
+# While the agent is processing tool calls (between promptAgent(...) and takeResponse(...))
+followUp(agent, "Also check European wines")
+```
+
+### 1.5: Stop the Agent
+
+```julia
+stopAgent(agent)
+# Sends :shutdown, closes all channels. Agent object is destroyed.
+```
+
+---
+
+## 2. Troubleshooting
+
+### 2.1: Agent Loop Exits Silently
+
+**Symptoms**:
+- `takeResponse(agent)` hangs indefinitely
+- No output in eventSink
+
+**Steps**:
+1. Check if `eventSink` is receiving error messages:
+   ```julia
+   global error_count = 0
+   my_eventSink(msg; publish=false) = begin
+       if occursin("error", lowercase(msg)) || occursin("failed", lowercase(msg))
+           println("ERROR EVENT: $msg")
+           global error_count += 1
+       end
+   end
+   ```
+2. Check network connectivity to LLM API endpoint
+3. Verify `formatMsgForLLM` produces valid OpenAI-format message structure (`4-specification.md:SP-011..SP-016`)
+
+> **Reference**: `4-specification.md` (OpenAI format specs), `3-walkthrough.md:WT-030` (LLM timeout)
+
+### 2.2: Tool Call Not Executing
+
+**Symptoms**:
+- LLM returns `tool_calls` but agent returns text response immediately
+- `beforeToolCall` hook is missing or incorrectly implemented
+
+**Steps**:
+1. Verify `prepareContext` returns `agentContext` with `tools` field (not `nothing`) (`src/utils.jl:123`)
+2. Check `beforeToolCall` hook is not blocking execution:
+   ```julia
+   function my_beforeToolCall(ctx, signal, eventSink)
+       # Returning nothing → allows execution
+       # Returning beforeToolCallResult(true, "blocked") → blocks
+       eventSink("beforeToolCall for $(ctx.toolCall.name)"; publish=true)
+       nothing  # ← must return nothing (not beforeToolCallResult(false, "N/A"))
+   end
+   ```
+3. Verify tool exists in `context.tools[toolCall.name]` (`src/agentCore.jl:985`)
+4. Check `validateToolArguments` is not throwing (catches and returns error tool result) (`src/utils.jl:533`)
+
+> **Reference**: `4-specification.md:SP-021` (agentLoopConfig with beforeToolCall hook), `3-walkthrough.md:WT-032` (beforeToolCall blocks)
+
+### 2.3: MCP Server Returns No Tools
+
+**Symptoms**:
+- `println("[toolRegistry] Discovered 0 MCP tools."` on startup
+- Agent has no tools available at all times
+
+**Steps**:
+1. Verify `mcpServer` callable struct correctly handles `mcp("tools/list", sessionId; eventSink)`
+2. Check JSON-RPC 2.0 response shape matches `4-specification.md:SP-009`:
+   ```json
+   {
+     "result": {
+       "tools": [{"name": "...", ...}],
+       "nextCursor": ""  # empty string = no more pages
+     }
+   }
+   ```
+3. Verify `tools/list` error response uses `haskey(response, "error")` check (`src/toolRegistry.jl:128`)
+
+> **Reference**: `4-specification.md:SP-009` (tools/list format), `3-walkthrough.md:WT-020` (MCP discovery)
+
+### 2.4: LLM Response Format Not Parsed
+
+**Symptoms**:
+- Agent always treats LLM response as text, never extracts tool calls
+- LLM correctly returns `tool_calls` but agent ignores them
+
+**Steps**:
+1. Verify `llmCall` returns a dict containing `message` (LMStudio/vLLM format, `4-specification.md:SP-017`) or `content` (OpenAI format, `4-specification.md:SP-018`)
+2. Check for both format variations:
+   - Format 1: `response["message"]["tool_calls"]` (LMStudio style, see `src/agentCore.jl:671-683`)
+   - Format 2: `response.content[{type => "tool_calls"}]` (OpenAI style, see `src/agentCore.jl:700-705`)
+3. Verify `finish_reason` field matches: `"tool_calls"` triggers tool execution, `"stop"`/`"end_turn"` breaks loop (`src/agentCore.jl:723-724`)
+
+> **Reference**: `4-specification.md:SP-017, SP-018`, `3-walkthrough.md:WT-011, WT-012` (LLM response handling)
+
+### 2.5: Tool Execution Hangs
+
+**Symptoms**:
+- `tool.execute(...)` starts but never returns
+- EventSink receives `toolExecutionStart` but never `toolExecutionEnd`
+
+**Steps**:
+1. Implement `abortSignal` checking in tool execute function:
+   ```julia
+   function my_tool_execute(toolCallId, args, signal, eventSink)
+       while !signal.aborted && !all_conditions_met
+           # Long-running operation with periodic abort checks
+           sleep(0.1)
+       end
+       if signal.aborted
+           return agentToolResult([textContent("Cancelled")], Dict(), nothing, false)
+       end
+       return agentToolResult([textContent("Done")], Dict(), nothing, false)
+   end
+   ```
+2. External caller sets `signal.aborted = true` (if `abortSignal` is shared via closure)
+3. Add timeout logic to tool execute function:
+   ```julia
+   start_time = time()
+   timeout_seconds = 30.0
+   while time() - start_time < timeout_seconds && !signal.aborted
+       # operation...
+       sleep(0.5)
+   end
+   ```
+
+> **Reference**: `4-specification.md:SP-022` (abortSignal), `3-walkthrough.md:WT-030` (LLM timeout)
+
+### 2.6: Message History Grows Unbounded
+
+**Symptoms**:
+- Memory usage grows linearly with conversation length
+- LLM context window exceeded for long sessions
+
+**Steps**:
+1. Implement pruning in `prepareContext` (`src/utils.jl:112-126`):
+   ```julia
+   function my_prepareContext(state::agentState, eventSink)
+       # Prune old messages, keep last N turns
+       max_turns = 20
+       recent_msgs = isempty(state.messages) ? state.messages :
+                     state.messages[max(1, end-2*max_turns):end]
+       
+       # Optionally inject system context (time, user prefs)
+       preparedMessages = recent_msgs
+       preparedSystemPrompt = state.systemPrompt * "\nCurrent time: $(now())"
+       
+       # Filter tools (optional: keep only relevant subset)
+       filteredTools = state.tools  # or filter by intent...
+       
+       return agentContext(preparedSystemPrompt, preparedMessages, filteredTools)
+   end
+   ```
+2. Add hard limit in `_agentLoop`:
+   ```julia
+   if length(agent._state.messages) > 200
+       # Truncate oldest messages
+       agent._state.messages = agent._state.messages[end-100:end]
+   end
+   ```
+
+> **Reference**: `3-walkthrough.md` (agent message flow), `1-requirements.md:NFR-103` (channel buffer limits)
+
+---
+
+## 3. Extending the Framework
+
+### 3.1: Add a Custom LLM Provider
+
+**Steps**:
+1. Implement `llmCall` function that accepts `FormattedMessages::Dict{String, Any}` and returns response dict
+2. Ensure response dict contains at least one of:
+   - `message.tool_calls` format (LMStudio/vLLM) — `4-specification.md:SP-017`
+   - `content[{type=>"tool_calls"}]` format (OpenAI) — `4-specification.md:SP-018`
+3. Response dict may optionally contain: `api`, `provider`, `model`, `usage` (`llmUsage` or dict), `finish_reason`, `error_message` (`4-specification.md:SP-017`, `SP-018.4`)
+
+```julia
+function my_openai_llmCall(messages::Dict{String, Any}; eventSink=nothing)
+    # messages["messages"] contains OpenAI-format messages
+    # messages["temperature"] (default 0.7)
+    # messages["tools"] (if tools are set)
+    
+    body = Dict(
+        "model" => "gpt-4",
+        "messages" => messages["messages"],
+        "tools" => get(messages, "tools", Any[])
+    )
+    
+    resp = HTTP.post("https://api.openai.com/v1/chat/completions", 
+                     headers, JSON.json(body))
+    json_resp = JSON.parse(String(resp.body))
+    
+    # Extract first completion
+    choice = json_resp["choices"][1]["message"]
+    
+    # Build uniform response
+    return Dict(
+        "api" => "openai",
+        "provider" => "openai",
+        "model" => "gpt-4",
+        "message" => choice,
+        "usage" => json_resp["usage"],
+        "finish_reason" => json_resp["choices"][1]["finish_reason"]
+    )
+end
+```
+
+> **Reference**: `1-requirements.md:FR-002`, `4-specification.md:SP-017, SP-018`
+
+### 3.2: Add a Custom Tool
+
+**Steps**:
+1. Define tool schema in `inputSchema` (JSON Schema format):
+   ```julia
+   schema = Dict{String, Any}(
+       "type" => "object",
+       "properties" => Dict(
+           "query" => Dict("type" => "string", "description" => "Search query"),
+           "max_results" => Dict("type" => "integer", "default" => 5)
+       ),
+       "required" => ["query"]
+   )
+   ```
+2. Implement `execute` function:
+   ```julia
+   tool = agentTool(
+       name="searchWine",
+       label="Search Wine Inventory",
+       description="Search for wines matching a query",
+       inputSchema=schema,
+       execute=(toolCallId, args, signal, eventSink) -> begin
+           query = args["query"]
+           max_results = get(args, "max_results", Dict{Any,Any}()["max_results"] => 5...)  # handle default
+           
+           wines = run_search(query, max_results)
+           
+           result_text = join(["- $(w.name) (\$$(w.price))" for w in wines], "\n")
+           return agentToolResult(
+               [textContent("Found $(length(wines)) wines:\n$result_text")],
+               Dict("count" => length(wines)),
+               nothing, false  # content, details, usage, terminate
+           )
+       end,
+       prepareArguments = nothing,
+       validateRequiredArgs = nothing,  # uses default validator
+       parallelToolExecute = false
+   )
+   
+   # Register: push!(tools, tool.name => tool)
+   ```
+3. Optionally override validation:
+   ```julia
+   function my_validate(args)
+       if haskey(args, "query") && length(args["query"]) < 3
+           return "Query must be at least 3 characters"
+       end
+       return nothing
+   end
+   
+   agentTool(
+       ...,
+       validateRequiredArgs = my_validate,  # replaces default validator
+   )
+   ```
+
+> **Reference**: `1-requirements.md:FR-004`, `4-specification.md:SP-009, SP-010`
+
+### 3.3: Implement User Approval Gate (beforeToolCall Hook)
+
+**Steps**:
+1. Implement `beforeToolCall` that blocks expensive/destructive operations:
+   ```julia
+   function my_approval_toolCall(ctx::beforeToolCallContext, signal::abortSignal, eventSink)
+       if ctx.toolCall.name in ("delete_wine", "update_price_bulk")
+           eventSink("Approval required for $(ctx.toolCall.name) with args $(ctx.args)")
+           # In production, send to UI and wait for user input
+           approved = ask_user_for_approval(ctx.toolCall.name, ctx.args)  # blocking call
+           if !approved
+               return beforeToolCallResult(true, "User denied approval for $(ctx.toolCall.name)")
+           end
+       end
+       return nothing  # allow execution
+   end
+   ```
+
+> **Reference**: `2-solution_design.md:SD-005` (lifecycle hooks), `3-walkthrough.md:WT-032` (beforeToolCall blocks)
+
+### 3.4: Implement Result Sanitization (afterToolCall Hook)
+
+**Steps**:
+1. Implement `afterToolCall` to mask sensitive data:
+   ```julia
+   function my_sanitize_toolCall(ctx::afterToolCallContext, signal::abortSignal, eventSink)
+       if ctx.toolCall.name == "getCustomerInfo"
+           # Mask sensitive fields before LLM sees result
+           result = ctx.result
+           for (i, block) in enumerate(result.content)
+               if block isa textContent
+                   masked = replace(block.text, r"\d{12}" => "***-***-****")  # mask SSN
+                   result = agentToolResult(
+                       [replace(block.text, r"\d{12}" => "***-***-****") for block in result.content],
+                       result.details,
+                       result.usage,
+                       result.terminate
+                   )
+               end
+           end
+           return result
+       end
+       return nothing  # pass through for other tools
+   end
+   ```
+
+> **Reference**: `2-solution_design.md:SD-005` (lifecycle hooks)
+
+---
+
+## 4. Common Error Codes
+
+| Error Condition | Response Code | Description |
+|-----------------|---------------|-------------|
+| Tool not found | `immediateOutcome("Tool <name> not found", true)` | `context.tools[toolCall.name]` returns `nothing` |
+| Missing required args | `immediateOutcome("Missing required arguments: <fields>", true)` | `validateRequiredArgs` returns error string |
+| Custom validation failed | `immediateOutcome("<tool.validateRequiredArgs result>", true)` | Tool-specific validation hook returns error |
+| beforeToolCall blocked | `immediateOutcome("Tool execution was blocked: <reason>", true)` | `beforeToolCall` returns `beforeToolCallResult(true, reason)` |
+| Operation aborted | `immediateOutcome("Operation aborted", true)` | `signal.aborted == true` after `beforeToolCall` |
+| Tool execution crashed | `immediateOutcome("<exception message>", true)` | `preptool.execute()` throws exception |
+| MCP error | `agentToolResult(["MCP error: <err_msg>"], {..., "error"=>err_msg}, ...)` | Server returns `response["error"]` JSON-RPC envelope |
+| MCP network error | `agentToolResult(["MCP call error: <err_msg>"], {..., "error"=>err_msg}, ...)` | Exception during `mcp("tools/call", ...)` |
+
+> **Reference**: `4-specification.md:SP-009, SP-010`, `src/agentCore.jl:989, 1013, 1019, 1033, 1097`
+
+---
+
+## 5. Debugging Checklist
+
+Per ASG Framework §Pre-Code Debugging Habit:
+
+| Stage | Debug Question |
+|-------|----------------|
+| **Implementation** | If a tool is not found, does `prepareToolCall` return `immediateOutcome` with "Tool not found"? |
+| **Testing** | If an LLM response has `tool_calls`, does `_extractToolCalls` return `(true, toolCallList, assistantMessageToolCall)`? |
+| **Deploying** | If the agent loop exits, does `eventSink` contain the error message? |
+
+---
+
+## 6. References
+
+- **API contract**: [`4-specification.md`](4-specification.md) — message types, MCP JSON-RPC format, response extraction formats
+- **Solution decisions**: [`2-solution_design.md`](2-solution_design.md) — component architecture, lifecycle hooks, format-agnostic LLM responses
+- **Walkthroughs**: [`3-walkthrough.md`](3-walkthrough.md) — end-to-end agent flow, error paths, hook sequences
+- **Validation tests**: [`6-validation.md`](6-validation.md) — test traceability matrix mapping requirements/specs to test files

src/agentCore.jl

@@ -264,18 +264,18 @@ function _agentLoop(agent::yiemAgent)
     while true
       while newUserMsg === nothing 
         if isready(agent.inputChannel)
-          agent.eventSink("_agentLoop 1 agent._state.messages length $(length(agent._state.messages))"; publish=false, log=true)
+          agent.eventSink("_agentLoop 1 agent._state.messages length $(length(agent._state.messages))"; publish=true)
           # agent process new user msg immediately after the current tool call finished.
           newUserMsg = take!(agent.inputChannel)   
-          # agent.eventSink("new user msg\n$newUserMsg"; publish=false, log=true)
+          # agent.eventSink("new user msg\n$newUserMsg"; publish=true)
         else
           # check followUp message after _processMessage() is done
           if typeof(processingTask) == Task && istaskdone(processingTask) == true
-            agent.eventSink("_agentLoop 2 agent._state.messages length $(length(agent._state.messages))"; publish=false, log=true)
+            agent.eventSink("_agentLoop 2 agent._state.messages length $(length(agent._state.messages))"; publish=true)
             # if agent runs is done but followUpChannel has messages,
             # put new message in inputChannel instead
             if isready(agent.followUpChannel)
-              agent.eventSink("_agentLoop 3 agent._state.messages length $(length(agent._state.messages))"; publish=false, log=true)
+              agent.eventSink("_agentLoop 3 agent._state.messages length $(length(agent._state.messages))"; publish=true)
               while isready(agent.followUpChannel)
                 followUpMsg = take!(agent.followUpChannel)
                 put!(agent.inputChannel, followUpMsg)
@@ -283,18 +283,18 @@ function _agentLoop(agent::yiemAgent)
               processingTask = nothing        # reset
               result = nothing                # reset
             else  # _processMessage() done and no followUp message.
-              agent.eventSink("_agentLoop 4 agent._state.messages length $(length(agent._state.messages))"; publish=false, log=true)
+              agent.eventSink("_agentLoop 4 agent._state.messages length $(length(agent._state.messages))"; publish=true)
               result = deepcopy(agent._state.messages[end])
-              agent.eventSink("_agentLoop 4-1 "; publish=false, log=true)
+              agent.eventSink("_agentLoop 4-1 "; publish=true)
               # filter out reasoningContent in-place
               filter!(c -> !(c isa reasoningContent), result.content)
-              agent.eventSink("_agentLoop 5 "; publish=false, log=true)
+              agent.eventSink("_agentLoop 5 "; publish=true)
               # format output
               respondToUI = _assistantMessageToOpenAI(result)
-              agent.eventSink("_agentLoop 6 "; publish=false, log=true)
+              agent.eventSink("_agentLoop 6 "; publish=true)
               put!(agent.outputChannel, respondToUI)
               if !isempty(result.content) && result.content[1] isa textContent
-                agent.eventSink(result.content[1].text; publish=false, log=true)
+                agent.eventSink(result.content[1].text; publish=true)
               end
               processingTask = nothing        # reset
               result = nothing                # reset
@@ -325,7 +325,7 @@ function _agentLoop(agent::yiemAgent)
       else
         # spawn new _processMessage() if it is not already running.
         if processingTask === nothing
-          agent.eventSink("_agentLoop 5 agent._state.messages length $(length(agent._state.messages))"; publish=false, log=true)
+          agent.eventSink("_agentLoop 5 agent._state.messages length $(length(agent._state.messages))"; publish=true)
           # discard all messages in followUpChannel
           while isready(agent.followUpChannel)
             _ = take!(agent.followUpChannel)
@@ -356,7 +356,7 @@ function _agentLoop(agent::yiemAgent)
         showerror(io, e, bt)
         println(io)
     end
-    eventSink(errMsg; publish=false, log=true)
+    eventSink(errMsg; publish=true)
 
     # On any error, send error response and exit the loop
     @error "Agent loop failed" error=e
@@ -404,7 +404,7 @@ function _processMessage(
   afterToolCall::Union{Function, Nothing},
   parallelToolExecute::Bool,
 )::Nothing
-  eventSink("_processMessage 1 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+  eventSink("_processMessage 1 _state.messages length $(length(agentMsgHistory))"; publish=true)
   # loop until llmCall() response didn't use tool calls
   final_response = nothing
 
@@ -425,43 +425,43 @@ function _processMessage(
     try
       # Drain inputChannel and convert OpenAI-format messages to userMessage type
       while isready(inputChannel)
-        eventSink("_processMessage 2 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+        eventSink("_processMessage 2 _state.messages length $(length(agentMsgHistory))"; publish=true)
         newUserMsg_openai = take!(inputChannel)
-        eventSink("_processMessage 3 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+        eventSink("_processMessage 3 _state.messages length $(length(agentMsgHistory))"; publish=true)
         if newUserMsg_openai === :shutdown
-          eventSink("_processMessage 4 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+          eventSink("_processMessage 4 _state.messages length $(length(agentMsgHistory))"; publish=true)
           # Re-emit shutdown signal for the loop to handle
           put!(inputChannel, :shutdown)
           break
         end
-        eventSink("_processMessage 5 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+        eventSink("_processMessage 5 _state.messages length $(length(agentMsgHistory))"; publish=true)
         newUserMsg = OpenAiToUserMessage(newUserMsg_openai)
         push!(agentMsgHistory, newUserMsg)
-        eventSink("_processMessage 6 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+        eventSink("_processMessage 6 _state.messages length $(length(agentMsgHistory))"; publish=true)
       end
-      eventSink("_processMessage 7 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+      eventSink("_processMessage 7 _state.messages length $(length(agentMsgHistory))"; publish=true)
       # call prepareContext()
       state = agentState(systemPrompt, nothing, tools, agentMsgHistory)
-      eventSink("_processMessage 8 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+      eventSink("_processMessage 8 _state.messages length $(length(agentMsgHistory))"; publish=true)
       preparedContext = prepareContext(state, eventSink)
-      eventSink("_processMessage 9 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+      eventSink("_processMessage 9 _state.messages length $(length(agentMsgHistory))"; publish=true)
       # Call formatMessagesForLLM() to format for LLM
       formattedMessages = formatMessagesForLLM(preparedContext, eventSink)
       
-      # eventSink("_processMessage 10 formattedMessages $formattedMessages"; publish=false, log=true)    
+      # eventSink("_processMessage 10 formattedMessages $formattedMessages"; publish=true)    
 
       """ response example
       response = JSON.Object{String, Any}("finish_reason" => "tool_calls", "index" => 0, "message" => JSON.Object{String, Any}("role" => "assistant", "content" => "", "reasoning_content" => "Here's a thinking process:\n\n1.  **Identify User Request**: The user is asking for the weather in Bangkok.\n2.  **Locate Relevant Tool**: I have a `getWeather` function available.\n3.  **Check Function Parameters**:\n   - `city` (required): City and country, e.g., 'San Francisco, CA' or 'Tokyo, Japan'\n   - `units` (optional, default \"celsius\"): Temperature scale (\"celsius\" or \"fahrenheit\")\n4.  **Prepare Parameters**:\n   - `city`: \"Bangkok, Thailand\" (adding country for clarity, though just \"Bangkok\" might work, following the example format is safer)\n   - `units`: Not specified, so I'll use the default (\"celsius\")\n5.  **Execute Tool Call**: Call `getWeather` with `city: \"Bangkok, Thailand\"`\n6.  **Anticipate Response**: The function will return current weather and forecast data for Bangkok. I'll then format it nicely for the user.\n   - *Self-Correction/Verification during thought*: The prompt says \"city: City and country, e.g., 'San Francisco, CA' or 'Tokyo, Japan'\". I'll use \"Bangkok, Thailand\". The `units` parameter is optional, so I'll omit it to use the default.\n   - Proceed with tool call.✅\n", "tool_calls" => Any[JSON.Object{String, Any}("type" => "function", "function" => JSON.Object{String, Any}("name" => "getWeather", "arguments" => "{\"city\":\"Bangkok, Thailand\"}"), "id" => "6fOilR5QPcdppbAAHluhkRyUDu3oWMAL")]))
       """
       response = llmCall(formattedMessages, eventSink=eventSink)
-      # eventSink("LLM response " * string(response); publish=false, log=true)
+      # eventSink("LLM response " * string(response); publish=true)
 
-      eventSink("_processMessage 11 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+      eventSink("_processMessage 11 _state.messages length $(length(agentMsgHistory))"; publish=true)
       # Extract tool calls from LLM response content blocks
       hasToolCalls, toolCallList, assistant_msg = _extractToolCalls(response)
-      eventSink("hasToolCalls: $hasToolCalls\ntoolCallList: $toolCallList _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
-      eventSink("assistant_msg " * string(assistant_msg); publish=false, log=true)
-      eventSink("_processMessage 11-1"; publish=false, log=true)
+      eventSink("hasToolCalls: $hasToolCalls\ntoolCallList: $toolCallList _state.messages length $(length(agentMsgHistory))"; publish=true)
+      eventSink("assistant_msg " * string(assistant_msg); publish=true)
+      eventSink("_processMessage 11-1"; publish=true)
 
       # Add assistant message (tool calls or text) to history for next LLM turn
       push!(agentMsgHistory, assistant_msg)
@@ -476,21 +476,21 @@ function _processMessage(
         )
 
         signal = abortSignal(false)
-        eventSink("_processMessage 12 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+        eventSink("_processMessage 12 _state.messages length $(length(agentMsgHistory))"; publish=true)
 
         # call executeToolCalls()
         toolResultBatch = executeToolCalls(preparedContext, assistant_msg, toolCallList, config, 
                                           signal, eventSink)      
-        eventSink("_processMessage 13 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+        eventSink("_processMessage 13 _state.messages length $(length(agentMsgHistory))"; publish=true)
 
         # save toolResults to messages
         for toolResult in toolResultBatch.messages
-          # eventSink("toolResult " * string(toolResult); publish=false, log=true)
+          # eventSink("toolResult " * string(toolResult); publish=true)
           push!(agentMsgHistory, toolResult)
         end
-        eventSink("_processMessage 14 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+        eventSink("_processMessage 14 _state.messages length $(length(agentMsgHistory))"; publish=true)
         if toolResultBatch.terminate
-          eventSink("_processMessage 15 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+          eventSink("_processMessage 15 _state.messages length $(length(agentMsgHistory))"; publish=true)
           # If toolResultBatch requested termination, build a final response
           final_content = [textContent("Tool execution completed.")]
           for toolResult in toolResultBatch.messages
@@ -504,7 +504,7 @@ function _processMessage(
               end
             end
           end
-          eventSink("_processMessage 16 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+          eventSink("_processMessage 16 _state.messages length $(length(agentMsgHistory))"; publish=true)
           final_response = assistantMessage(
             role="assistant",
             content=final_content,
@@ -523,7 +523,7 @@ function _processMessage(
           break
         end
       else
-        eventSink("_processMessage 17 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+        eventSink("_processMessage 17 _state.messages length $(length(agentMsgHistory))"; publish=true)
         # LLM did not use tool calls — 
         break
       end
@@ -534,12 +534,12 @@ function _processMessage(
             println(io)
         end
 
-        eventSink(err_msg; publish=false, log=true)
+        eventSink(err_msg; publish=true)
     end
   end
 
 
-  eventSink("_processMessage 18 _state.messages length $(length(agentMsgHistory))"; publish=false, log=true)
+  eventSink("_processMessage 18 _state.messages length $(length(agentMsgHistory))"; publish=true)
   return nothing
 end
 
@@ -979,47 +979,47 @@ function prepareToolCall(
   signal::abortSignal,
   eventSink
 )::Union{preparedToolCall,immediateOutcome}
-  eventSink("prepareToolCall 1"; publish=false, log=true)
+  eventSink("prepareToolCall 1"; publish=true)
 
   # pick a called tool from tool store
   tool = get(context.tools, toolCall.name, nothing) 
   
   if tool === nothing
-    eventSink("prepareToolCall 2"; publish=false, log=true)
+    eventSink("prepareToolCall 2"; publish=true)
     return immediateOutcome(createErrorToolResult("Tool $toolCall.name not found"), true)
   end
 
   try
-    eventSink("prepareToolCall 3"; publish=false, log=true)
+    eventSink("prepareToolCall 3"; publish=true)
     # 1. prepare arguments (tool-specific transform)
     prepared = prepareToolCallArguments(tool, toolCall)
-    eventSink("prepared " * string(prepared.arguments); publish=false, log=true)
-    eventSink("prepareToolCall 4"; publish=false, log=true)
+    eventSink("prepared " * string(prepared.arguments); publish=true)
+    eventSink("prepareToolCall 4"; publish=true)
     validatedArgs = validateToolArguments(tool, prepared)
-    eventSink("validatedArgs " * string(validatedArgs); publish=false, log=true)
-    eventSink("prepareToolCall 5"; publish=false, log=true)
+    eventSink("validatedArgs " * string(validatedArgs); publish=true)
+    eventSink("prepareToolCall 5"; publish=true)
     # 2. beforeToolCall hook — can block
     if config.beforeToolCall !== nothing
-      eventSink("prepareToolCall 6"; publish=false, log=true)
+      eventSink("prepareToolCall 6"; publish=true)
 
       before = config.beforeToolCall(
         beforeToolCallContext(assistantMsg, toolCall, validatedArgs, context), 
         signal,
         eventSink
         )
-      eventSink("prepareToolCall 7"; publish=false, log=true)
+      eventSink("prepareToolCall 7"; publish=true)
       if signal.aborted
-        eventSink("prepareToolCall 8"; publish=false, log=true)
+        eventSink("prepareToolCall 8"; publish=true)
         return immediateOutcome(createErrorToolResult("Operation aborted"), true)
       end
 
       if before !== nothing && before.block
-        eventSink("prepareToolCall 9"; publish=false, log=true)
+        eventSink("prepareToolCall 9"; publish=true)
         return immediateOutcome(
           createErrorToolResult(get(before, :reason, "Tool execution was blocked")), true)
       end
     end
-    eventSink("prepareToolCall 10"; publish=false, log=true)
+    eventSink("prepareToolCall 10"; publish=true)
     return preparedToolCall(tool, toolCall, validatedArgs)
   catch e
     bt = catch_backtrace()
@@ -1028,7 +1028,7 @@ function prepareToolCall(
         println(io)
     end
 
-    eventSink(errMsg; publish=false, log=true)
+    eventSink(errMsg; publish=true)
 
     return immediateOutcome(createErrorToolResult(sprint(showerror, e)), true)
   end
@@ -1080,16 +1080,11 @@ function executePreparedToolCall(
     signal::Union{Nothing,abortSignal},
     eventSink,
   )::executedOutcome
-  eventSink("executePreparedToolCall 1"; publish=false, log=true)
+  eventSink("executePreparedToolCall 1"; publish=true)
 
   try
-
     result = prep.tool.execute(prep.toolCall.id, prep.args, signal, eventSink)
-
-
-    eventSink("executePreparedToolCall 2"; publish=false, log=true)
-    eventSink("tool execute raw result: " * sprint(show, result); publish=false, log=true)
-    eventSink("executePreparedToolCall 3"; publish=false, log=true)
+    # eventSink("tool execute raw result: " * sprint(show, result); publish=true)
     return executedOutcome(result, false)
   catch e
     bt = catch_backtrace()
@@ -1097,7 +1092,7 @@ function executePreparedToolCall(
         showerror(io, e, bt)
         println(io)
     end
-    eventSink(errMsg; publish=false, log=true)
+    eventSink(errMsg; publish=true)
 
     return executedOutcome(createErrorToolResult(sprint(showerror, e)), true)
   end
@@ -1170,10 +1165,10 @@ function finalizeExecutedToolCall(
     signal::Union{Nothing,abortSignal},
     eventSink
 )::finalizedOutcome
-  eventSink("finalizeExecutedToolCall 1"; publish=false, log=true)
+  eventSink("finalizeExecutedToolCall 1"; publish=true)
   result = executed.result
   isError = executed.isError
-  eventSink("finalizeExecutedToolCall result before afterToolCall\n$result"; publish=false, log=true)
+  # eventSink("finalizeExecutedToolCall result before afterToolCall\n$result"; publish=true)
   if config.afterToolCall !== nothing
     try
       after = config.afterToolCall(
@@ -1181,7 +1176,7 @@ function finalizeExecutedToolCall(
         signal,
         eventSink
       )
-      eventSink("finalizeExecutedToolCall 3"; publish=false, log=true)
+      eventSink("finalizeExecutedToolCall 3"; publish=true)
       if after !== nothing
         result = after
         isError = false
@@ -1192,13 +1187,13 @@ function finalizeExecutedToolCall(
           showerror(io, e, bt)
           println(io)
       end
-      eventSink(errMsg; publish=false, log=true)
+      eventSink(errMsg; publish=true)
 
       result = createErrorToolResult(sprint(showerror, e))
       isError = true
     end
   end
-  eventSink("finalizeExecutedToolCall 4"; publish=false, log=true)
+  eventSink("finalizeExecutedToolCall 4"; publish=true)
   return finalizedOutcome(prep.toolCall, result, isError)
 end
 
@@ -1264,39 +1259,39 @@ function executeToolCallsSequential(
     signal::abortSignal,
     eventSink,
 )::agentToolCallBatch
-  eventSink("executeToolCallsSequential 1"; publish=false, log=true)
+  eventSink("executeToolCallsSequential 1"; publish=true)
   finalizedCalls = finalizedOutcome[]
   messages = toolResultMessage[]
 
   for tc in toolCalls
-    eventSink("start tool execute $(tc.id), $(tc.name), $(tc.arguments)"; publish=false, log=true)
+    eventSink("start tool execute $(tc.id), $(tc.name), $(tc.arguments)"; publish=true)
     prep = prepareToolCall(context, assistantMsg, tc, config, signal, eventSink)
-    eventSink("executeToolCallsSequential " * string(prep.args); publish=false, log=true)
+    eventSink("executeToolCallsSequential " * string(prep.args); publish=true)
 
     if prep isa immediateOutcome
-      eventSink("executeToolCallsSequential 2-1"; publish=false, log=true)
+      eventSink("executeToolCallsSequential 2-1"; publish=true)
       finalized = finalizedOutcome(tc, prep.result, prep.isError)
-      eventSink("executeToolCallsSequential 2-2"; publish=false, log=true)
+      eventSink("executeToolCallsSequential 2-2"; publish=true)
     else
-      eventSink("executeToolCallsSequential 3"; publish=false, log=true)
+      eventSink("executeToolCallsSequential 3"; publish=true)
       executed = executePreparedToolCall(prep, signal, eventSink)
 
-      eventSink("executeToolCallsSequential 3-1"; publish=false, log=true)
+      eventSink("executeToolCallsSequential 3-1"; publish=true)
       finalized = finalizeExecutedToolCall(context, assistantMsg, prep, executed, config,
                                           signal, eventSink)
-      eventSink("executeToolCallsSequential 3-2"; publish=false, log=true)
+      eventSink("executeToolCallsSequential 3-2"; publish=true)
     end
-    eventSink("executeToolCallsSequential 4"; publish=false, log=true)
+    eventSink("executeToolCallsSequential 4"; publish=true)
     # eventSink("finalized $(finalized.toolCall.id), $(finalized.toolCall.name),
-    #                       $(finalized.result), $(finalized.isError)"; publish=false, log=true)
+    #                       $(finalized.result), $(finalized.isError)"; publish=true)
     push!(messages, createToolResultMessage(finalized))
     push!(finalizedCalls, finalized)
-    eventSink("executeToolCallsSequential 5"; publish=false, log=true)
+    eventSink("executeToolCallsSequential 5"; publish=true)
     if signal !== nothing && signal.aborted
         break
     end
   end
-  eventSink("executeToolCallsSequential 6"; publish=false, log=true)
+  eventSink("executeToolCallsSequential 6"; publish=true)
   return agentToolCallBatch(messages, shouldTerminate(finalizedCalls))
 end
 
@@ -1369,7 +1364,7 @@ function executeToolCallsParallel(
   entries = Union{finalizedOutcome,Task}[]
 
   for tc in toolCalls
-    eventSink(toolExecStartEvent(tc.id, tc.name, tc.arguments); publish=false, log=true)
+    eventSink(toolExecStartEvent(tc.id, tc.name, tc.arguments); publish=true)
 
     prep = prepareToolCall(context, assistantMsg, tc, config, signal, eventSink)
 
@@ -1468,7 +1463,7 @@ function executeToolCalls(
     eventSink,
   )::agentToolCallBatch
 
-  eventSink("_executeToolCalls 1"; publish=false, log=true)
+  eventSink("_executeToolCalls 1"; publish=true)
   hasSequential = false
   for tc in toolCalls
       t = get(context.tools, tc.name, nothing)
@@ -1477,13 +1472,13 @@ function executeToolCalls(
           break
       end
   end
-  eventSink("_executeToolCalls 2"; publish=false, log=true)
+  eventSink("_executeToolCalls 2"; publish=true)
   if config.toolExecution == "sequential" || hasSequential
-    eventSink("_executeToolCalls 3"; publish=false, log=true)
+    eventSink("_executeToolCalls 3"; publish=true)
     return executeToolCallsSequential(context, assistantMsg, toolCalls, config, signal,
                                       eventSink)
   else
-    eventSink("_executeToolCalls 4"; publish=false, log=true)
+    eventSink("_executeToolCalls 4"; publish=true)
     return executeToolCallsParallel(context, assistantMsg, toolCalls, config, signal,
                                     eventSink)
   end