149 lines
8.1 KiB
Markdown
149 lines
8.1 KiB
Markdown
# Solution Design: AgentCore.jl - Julia Agent Framework
|
|
|
|
## 1. Problem Decomposition
|
|
|
|
This project addresses several interconnected problems in building AI agent systems:
|
|
|
|
| Problem | Description | User Impact |
|
|
|---------|-------------|-------------|
|
|
| **P-001**: Complex state management | AI agents need to maintain conversation history, tool states, and system prompts across multiple turns | Without proper state management, conversations lose context and become inconsistent |
|
|
| **P-002**: Tool execution orchestration | LLMs often request multiple tool calls that need to be executed and results returned | Complex coordination required between LLM calls and tool execution |
|
|
| **P-003**: Session persistence | Agent sessions need to survive restarts and support branching for experiments | Loss of conversation history requires re-conversation and poor UX |
|
|
| **P-004**: Event monitoring | Need to observe agent behavior for debugging and operational visibility | Black-box agents are difficult to debug and monitor in production |
|
|
| **P-005**: Context window management | LLMs have limited context windows, requiring history management | Long conversations get truncated, losing important context |
|
|
|
|
## 2. Solution Approach
|
|
|
|
The solution implements a layered agent framework with clear separation of concerns:
|
|
|
|
**Approach**: Implement a low-level agent loop with stateless execution, wrapped in a high-level Agent struct that manages state, queuing, and event streaming. Sessions are persisted to JSONL storage with support for compaction and branching.
|
|
|
|
**Key Principles**:
|
|
- Separate concerns: low-level loop vs. high-level agent vs. session storage
|
|
- Event-driven architecture: all agent activity is observable via events
|
|
- Extensible tool system: tools are first-class objects with execution logic
|
|
- Immutable core: low-level loop operates on pure data structures
|
|
- Flexible queuing: support for steering and follow-up message queues
|
|
|
|
## 3. Alternatives Considered
|
|
|
|
| Alternative | Pros | Cons | Decision |
|
|
|-------------|------|------|----------|
|
|
| **Single monolithic agent class** | Simple to understand, no architectural complexity | Hard to test, difficult to extend, state management becomes complex | Rejected - would not scale for complex deployments |
|
|
| **Actor-based concurrency** | Built-in concurrency model, isolation | Heavy overhead, different semantics than required | Rejected - Julia's async primitives sufficient |
|
|
| **Callback-based event system** | Familiar pattern, lightweight | Difficult to manage subscriptions, error handling complex | Rejected - Julia's async channels better suited |
|
|
| **Full actor model (e.g., GenStage)** | Strong guarantees, backpressure | Overkill for this use case, learning curve | Rejected - simpler event streaming sufficient |
|
|
|
|
## 4. High-Level Component Diagram
|
|
|
|
```mermaid
|
|
%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#3b82f6'}}}%%
|
|
flowchart TB
|
|
subgraph "User Layer"
|
|
A[User Request]
|
|
B[Event Subscriber]
|
|
end
|
|
|
|
subgraph "Agent Layer"
|
|
C[Agent]
|
|
D[AgentState]
|
|
E[PendingMessageQueue]
|
|
end
|
|
|
|
subgraph "AgentLoop Layer"
|
|
F[agentLoop]
|
|
G[AgentContext]
|
|
H[AgentLoopConfig]
|
|
end
|
|
|
|
subgraph "Tool Layer"
|
|
I[Bash Tool]
|
|
J[Read Tool]
|
|
K[Write Tool]
|
|
L[Edit Tool]
|
|
end
|
|
|
|
subgraph "Session Layer"
|
|
M[Session Storage]
|
|
N[JSONL Repo]
|
|
O[InMemory Repo]
|
|
end
|
|
|
|
A --> C
|
|
B -->|event stream| C
|
|
C --> D
|
|
C --> E
|
|
C -->|start loop| F
|
|
F --> G
|
|
F --> H
|
|
F -->|execute tool| I
|
|
F -->|execute tool| J
|
|
F -->|execute tool| K
|
|
F -->|execute tool| L
|
|
C -->|persist state| M
|
|
M --> N
|
|
M --> O
|
|
```
|
|
|
|
**Component Descriptions**:
|
|
|
|
- **Agent** (FR-001, FR-004): High-level interface that manages conversation state, event subscriptions, and message queues. Acts as a facade over the agent loop.
|
|
|
|
- **AgentLoop** (FR-002): Low-level execution engine that handles LLM calls, tool execution, and event emission. Operates on pure data structures.
|
|
|
|
- **Session Storage** (FR-003): Persists conversation history and metadata. Supports both JSONL file storage and in-memory storage for testing.
|
|
|
|
- **Tools** (FR-002): Executable units that perform actions like bash commands, file operations, and database queries. Each tool has execute logic and optional argument preparation.
|
|
|
|
- **Event System** (FR-004): Publish-subscribe mechanism for observing agent activity. Enables monitoring, debugging, and external integration.
|
|
|
|
## 5. Decision Rationale
|
|
|
|
| Decision ID | Decision | Rationale | Alternatives Rejected |
|
|
|-------------|----------|-----------|----------------------|
|
|
| **SD-001**: Separate Agent and AgentLoop | Clear separation of concerns with Agent managing state and AgentLoop handling execution | Keeps low-level loop pure and testable | Combined class would mix concerns and reduce testability |
|
|
| **SD-002**: Event-driven architecture | Enables monitoring, debugging, and extensibility without modifying core logic | Callbacks would be harder to manage and compose | Direct method calls would require tight coupling |
|
|
| **SD-003**: JSONL-based persistence | Simple, human-readable format with good Julia ecosystem support | Binary formats would be harder to debug and inspect | Database dependency would complicate deployment |
|
|
| **SD-004**: Julia type system for type safety | Compile-time guarantees, better IDE support, clearer intent | Runtime checks would be less robust | Dynamic typing would increase bugs in production |
|
|
| **SD-005**: Two-level queuing (steering vs. follow-up) | Supports both immediate conversation correction and post-completion follow-ups | Single queue would not support both use cases | Complex state machine would be needed |
|
|
| **SD-006**: Branch-based session navigation | Enables experiment tracking, rollback, and parallel conversation paths | Linear history would not support A/B testing | Version control system would be overkill |
|
|
|
|
## 6. Risk Assessment
|
|
|
|
| Risk | Impact | Probability | Mitigation |
|
|
|------|--------|-------------|------------|
|
|
| **R-001**: Performance degradation with large sessions | High | Medium | Implement session compaction, provide metrics for monitoring |
|
|
| **R-002**: Tool execution failures breaking conversation | High | Medium | Graceful error handling, retry logic, clear error messages to LLM |
|
|
| **R-003**: Data loss from storage failures | High | Low | Support multiple storage backends, implement backup procedures |
|
|
| **R-004**: Event system overwhelming subscribers | Medium | Medium | Implement backpressure, provide filtering options, limit event queue size |
|
|
| **R-005**: Context window exhaustion | Medium | Medium | Automatic compaction, configurable retention policies, monitoring |
|
|
|
|
## 7. Requirements Traceability
|
|
|
|
| Solution Component | Requirement ID | Decision ID | Description |
|
|
|-------------------|----------------|-------------|-------------|
|
|
| Agent struct | FR-001 | SD-001 | Manages conversation state with message history and tools |
|
|
| AgentLoop execution | FR-002 | SD-002 | Executes LLM calls and tool calls with proper state handling |
|
|
| Session persistence | FR-003 | SD-003 | JSONL storage with repo abstraction for flexibility |
|
|
| Event system | FR-004 | SD-002 | Publish-subscribe events for monitoring and debugging |
|
|
| Queuing system | FR-005 | SD-005 | Steering and follow-up queues for conversation management |
|
|
| Tool framework | FR-002 | SD-002 | Executable tools with error handling and result reporting |
|
|
| Compaction | FR-003 | SD-003 | Session history management to fit context windows |
|
|
|
|
## 8. Implementation Guidance
|
|
|
|
**Module Structure**:
|
|
- `src/types.jl`: Core data types and interfaces
|
|
- `src/agent_loop.jl`: Low-level execution engine
|
|
- `src/agent.jl`: High-level Agent wrapper
|
|
- `src/session/`: Session persistence layer
|
|
- `src/tools/`: Built-in tool implementations
|
|
- `src/messages.jl`: Message transformation logic
|
|
|
|
**Key Patterns**:
|
|
- Use Julia's multiple dispatch for extensible tool system
|
|
- Implement async channels for event streaming
|
|
- Use immutable data structures where possible for safety
|
|
- Provide both synchronous and asynchronous APIs
|
|
- Design for testability with pure functions in low-level modules
|