Solution Design: AgentCore.jl - Julia Agent Framework

1. Problem Decomposition

This project addresses several interconnected problems in building AI agent systems:

Problem	Description	User Impact
P-001: Complex state management	AI agents need to maintain conversation history, tool states, and system prompts across multiple turns	Without proper state management, conversations lose context and become inconsistent
P-002: Tool execution orchestration	LLMs often request multiple tool calls that need to be executed and results returned	Complex coordination required between LLM calls and tool execution
P-003: Session persistence	Agent sessions need to survive restarts and support branching for experiments	Loss of conversation history requires re-conversation and poor UX
P-004: Event monitoring	Need to observe agent behavior for debugging and operational visibility	Black-box agents are difficult to debug and monitor in production
P-005: Context window management	LLMs have limited context windows, requiring history management	Long conversations get truncated, losing important context

2. Solution Approach

The solution implements a layered agent framework with clear separation of concerns:

Approach: Implement a low-level agent loop with stateless execution, wrapped in a high-level Agent struct that manages state, queuing, and event streaming. Sessions are persisted to JSONL storage with support for compaction and branching.

Key Principles:

Separate concerns: low-level loop vs. high-level agent vs. session storage
Event-driven architecture: all agent activity is observable via events
Extensible tool system: tools are first-class objects with execution logic
Immutable core: low-level loop operates on pure data structures
Flexible queuing: support for steering and follow-up message queues

3. Alternatives Considered

Alternative	Pros	Cons	Decision
Single monolithic agent class	Simple to understand, no architectural complexity	Hard to test, difficult to extend, state management becomes complex	Rejected - would not scale for complex deployments
Actor-based concurrency	Built-in concurrency model, isolation	Heavy overhead, different semantics than required	Rejected - Julia's async primitives sufficient
Callback-based event system	Familiar pattern, lightweight	Difficult to manage subscriptions, error handling complex	Rejected - Julia's async channels better suited
Full actor model (e.g., GenStage)	Strong guarantees, backpressure	Overkill for this use case, learning curve	Rejected - simpler event streaming sufficient

4. High-Level Component Diagram

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#3b82f6'}}}%%
flowchart TB
    subgraph "User Layer"
        A[User Request]
        B[Event Subscriber]
    end

    subgraph "Agent Layer"
        C[Agent]
        D[AgentState]
        E[PendingMessageQueue]
    end

    subgraph "AgentLoop Layer"
        F[agentLoop]
        G[AgentContext]
        H[AgentLoopConfig]
    end

    subgraph "Tool Layer"
        I[Bash Tool]
        J[Read Tool]
        K[Write Tool]
        L[Edit Tool]
    end

    subgraph "Session Layer"
        M[Session Storage]
        N[JSONL Repo]
        O[InMemory Repo]
    end

    A --> C
    B -->|event stream| C
    C --> D
    C --> E
    C -->|start loop| F
    F --> G
    F --> H
    F -->|execute tool| I
    F -->|execute tool| J
    F -->|execute tool| K
    F -->|execute tool| L
    C -->|persist state| M
    M --> N
    M --> O

Component Descriptions:

Agent (FR-001, FR-004): High-level interface that manages conversation state, event subscriptions, and message queues. Acts as a facade over the agent loop.
AgentLoop (FR-002): Low-level execution engine that handles LLM calls, tool execution, and event emission. Operates on pure data structures.
Session Storage (FR-003): Persists conversation history and metadata. Supports both JSONL file storage and in-memory storage for testing.
Tools (FR-002): Executable units that perform actions like bash commands, file operations, and database queries. Each tool has execute logic and optional argument preparation.
Event System (FR-004): Publish-subscribe mechanism for observing agent activity. Enables monitoring, debugging, and external integration.

5. Decision Rationale

Decision ID	Decision	Rationale	Alternatives Rejected
SD-001: Separate Agent and AgentLoop	Clear separation of concerns with Agent managing state and AgentLoop handling execution	Keeps low-level loop pure and testable	Combined class would mix concerns and reduce testability
SD-002: Event-driven architecture	Enables monitoring, debugging, and extensibility without modifying core logic	Callbacks would be harder to manage and compose	Direct method calls would require tight coupling
SD-003: JSONL-based persistence	Simple, human-readable format with good Julia ecosystem support	Binary formats would be harder to debug and inspect	Database dependency would complicate deployment
SD-004: Julia type system for type safety	Compile-time guarantees, better IDE support, clearer intent	Runtime checks would be less robust	Dynamic typing would increase bugs in production
SD-005: Two-level queuing (steering vs. follow-up)	Supports both immediate conversation correction and post-completion follow-ups	Single queue would not support both use cases	Complex state machine would be needed
SD-006: Branch-based session navigation	Enables experiment tracking, rollback, and parallel conversation paths	Linear history would not support A/B testing	Version control system would be overkill

6. Risk Assessment

Risk	Impact	Probability	Mitigation
R-001: Performance degradation with large sessions	High	Medium	Implement session compaction, provide metrics for monitoring
R-002: Tool execution failures breaking conversation	High	Medium	Graceful error handling, retry logic, clear error messages to LLM
R-003: Data loss from storage failures	High	Low	Support multiple storage backends, implement backup procedures
R-004: Event system overwhelming subscribers	Medium	Medium	Implement backpressure, provide filtering options, limit event queue size
R-005: Context window exhaustion	Medium	Medium	Automatic compaction, configurable retention policies, monitoring

7. Requirements Traceability

Solution Component	Requirement ID	Decision ID	Description
Agent struct	FR-001	SD-001	Manages conversation state with message history and tools
AgentLoop execution	FR-002	SD-002	Executes LLM calls and tool calls with proper state handling
Session persistence	FR-003	SD-003	JSONL storage with repo abstraction for flexibility
Event system	FR-004	SD-002	Publish-subscribe events for monitoring and debugging
Queuing system	FR-005	SD-005	Steering and follow-up queues for conversation management
Tool framework	FR-002	SD-002	Executable tools with error handling and result reporting
Compaction	FR-003	SD-003	Session history management to fit context windows

8. Implementation Guidance

Module Structure:

src/types.jl: Core data types and interfaces
src/agent_loop.jl: Low-level execution engine
src/agent.jl: High-level Agent wrapper
src/session/: Session persistence layer
src/tools/: Built-in tool implementations
src/messages.jl: Message transformation logic

Key Patterns:

Use Julia's multiple dispatch for extensible tool system
Implement async channels for event streaming
Use immutable data structures where possible for safety
Provide both synchronous and asynchronous APIs
Design for testability with pure functions in low-level modules

8.1 KiB Raw Blame History