YiemAgent/docs/solution-design.md

Solution Design: AgentCore.jl - Julia Agent Framework

1. Problem Decomposition

This project addresses several interconnected problems in building AI agent systems:

Problem	Description	User Impact
P-001: Complex state management	AI agents need to maintain conversation history, tool states, and system prompts across multiple turns	Without proper state management, conversations lose context and become inconsistent
P-002: Tool execution orchestration	LLMs often request multiple tool calls that need to be executed and results returned	Complex coordination required between LLM calls and tool execution
P-003: Session persistence	Agent sessions need to survive restarts and support branching for experiments	Loss of conversation history requires re-conversation and poor UX
P-004: Event monitoring	Need to observe agent behavior for debugging and operational visibility	Black-box agents are difficult to debug and monitor in production
P-005: Context window management	LLMs have limited context windows, requiring history management	Long conversations get truncated, losing important context

2. Solution Approach

The solution implements a layered agent framework with clear separation of concerns:

Approach: Implement a low-level agent loop with stateless execution, wrapped in a high-level Agent struct that manages state, queuing, and event streaming. Sessions are persisted to JSONL storage with support for compaction and branching.

Key Principles:

• Separate concerns: low-level loop vs. high-level agent vs. session storage
• Event-driven architecture: all agent activity is observable via events
• Extensible tool system: tools are first-class objects with execution logic
• Immutable core: low-level loop operates on pure data structures
• Flexible queuing: support for steering and follow-up message queues

3. Alternatives Considered

Alternative	Pros	Cons	Decision
Single monolithic agent class	Simple to understand, no architectural complexity	Hard to test, difficult to extend, state management becomes complex	Rejected - would not scale for complex deployments
Actor-based concurrency	Built-in concurrency model, isolation	Heavy overhead, different semantics than required	Rejected - Julia's async primitives sufficient
Callback-based event system	Familiar pattern, lightweight	Difficult to manage subscriptions, error handling complex	Rejected - Julia's async channels better suited
Full actor model (e.g., GenStage)	Strong guarantees, backpressure	Overkill for this use case, learning curve	Rejected - simpler event streaming sufficient

4. High-Level Component Diagram

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#3b82f6'}}}%%
flowchart TB
    subgraph "User Layer"
        A[User Request]
        B[Event Subscriber]
    end

    subgraph "Agent Layer"
        C[Agent]
        D[AgentState]
        E[PendingMessageQueue]
    end

    subgraph "AgentLoop Layer"
        F[agentLoop]
        G[AgentContext]
        H[AgentLoopConfig]
    end

    subgraph "Tool Layer"
        I[Bash Tool]
        J[Read Tool]
        K[Write Tool]
        L[Edit Tool]
    end

    subgraph "Session Layer"
        M[Session Storage]
        N[JSONL Repo]
        O[InMemory Repo]
    end

    A --> C
    B -->|event stream| C
    C --> D
    C --> E
    C -->|start loop| F
    F --> G
    F --> H
    F -->|execute tool| I
    F -->|execute tool| J
    F -->|execute tool| K
    F -->|execute tool| L
    C -->|persist state| M
    M --> N
    M --> O

Component Descriptions:

• Agent (FR-001, FR-004): High-level interface that manages conversation state, event subscriptions, and message queues. Acts as a facade over the agent loop.

• AgentLoop (FR-002): Low-level execution engine that handles LLM calls, tool execution, and event emission. Operates on pure data structures.

• Session Storage (FR-003): Persists conversation history and metadata. Supports both JSONL file storage and in-memory storage for testing.

• Tools (FR-002): Executable units that perform actions like bash commands, file operations, and database queries. Each tool has execute logic and optional argument preparation.

• Event System (FR-004): Publish-subscribe mechanism for observing agent activity. Enables monitoring, debugging, and external integration.

5. Decision Rationale

Decision ID	Decision	Rationale	Alternatives Rejected
SD-001: Separate Agent and AgentLoop	Clear separation of concerns with Agent managing state and AgentLoop handling execution	Keeps low-level loop pure and testable	Combined class would mix concerns and reduce testability
SD-002: Event-driven architecture	Enables monitoring, debugging, and extensibility without modifying core logic	Callbacks would be harder to manage and compose	Direct method calls would require tight coupling
SD-003: JSONL-based persistence	Simple, human-readable format with good Julia ecosystem support	Binary formats would be harder to debug and inspect	Database dependency would complicate deployment
SD-004: Julia type system for type safety	Compile-time guarantees, better IDE support, clearer intent	Runtime checks would be less robust	Dynamic typing would increase bugs in production
SD-005: Two-level queuing (steering vs. follow-up)	Supports both immediate conversation correction and post-completion follow-ups	Single queue would not support both use cases	Complex state machine would be needed
SD-006: Branch-based session navigation	Enables experiment tracking, rollback, and parallel conversation paths	Linear history would not support A/B testing	Version control system would be overkill

6. Risk Assessment

Risk	Impact	Probability	Mitigation
R-001: Performance degradation with large sessions	High	Medium	Implement session compaction, provide metrics for monitoring
R-002: Tool execution failures breaking conversation	High	Medium	Graceful error handling, retry logic, clear error messages to LLM
R-003: Data loss from storage failures	High	Low	Support multiple storage backends, implement backup procedures
R-004: Event system overwhelming subscribers	Medium	Medium	Implement backpressure, provide filtering options, limit event queue size
R-005: Context window exhaustion	Medium	Medium	Automatic compaction, configurable retention policies, monitoring

7. Requirements Traceability

Solution Component	Requirement ID	Decision ID	Description
Agent struct	FR-001	SD-001	Manages conversation state with message history and tools
AgentLoop execution	FR-002	SD-002	Executes LLM calls and tool calls with proper state handling
Session persistence	FR-003	SD-003	JSONL storage with repo abstraction for flexibility
Event system	FR-004	SD-002	Publish-subscribe events for monitoring and debugging
Queuing system	FR-005	SD-005	Steering and follow-up queues for conversation management
Tool framework	FR-002	SD-002	Executable tools with error handling and result reporting
Compaction	FR-003	SD-003	Session history management to fit context windows

8. Implementation Guidance

Module Structure:

• src/types.jl: Core data types and interfaces
• src/agent_loop.jl: Low-level execution engine
• src/agent.jl: High-level Agent wrapper
• src/session/: Session persistence layer
• src/tools/: Built-in tool implementations
• src/messages.jl: Message transformation logic

Key Patterns:

• Use Julia's multiple dispatch for extensible tool system
• Implement async channels for event streaming
• Use immutable data structures where possible for safety
• Provide both synchronous and asynchronous APIs
• Design for testability with pure functions in low-level modules