575 lines
17 KiB
Markdown
575 lines
17 KiB
Markdown
# AgentCore.jl - Learning Guide
|
|
|
|
## How to Use This Documentation
|
|
|
|
### Top-Down Learning Approach
|
|
|
|
This documentation is organized in a **top-down** order, starting from high-level concepts and drilling down into implementation details. Follow this sequence:
|
|
|
|
1. **Architecture Overview** - Understand the big picture
|
|
2. **Agent Component** - Learn about state management and event streaming
|
|
3. **AgentLoop Component** - Understand the core LLM interaction loop
|
|
4. **Types & Messages** - Learn the data structures
|
|
5. **Session Management** - Understand conversation history
|
|
6. **Tools** - Learn about tool execution
|
|
|
|
### Learning Style
|
|
|
|
- **Visual learners**: Study the ASCII diagrams
|
|
- **Hands-on learners**: Code examples provided for each section
|
|
- **Conceptual learners**: Read summaries and overviews first
|
|
|
|
## Quick Start
|
|
|
|
### Minimal Example
|
|
|
|
```julia
|
|
using AgentCore
|
|
|
|
# Create agent
|
|
agent = Agent(Dict(
|
|
:systemPrompt => "You are a helpful assistant.",
|
|
:model => Model(...),
|
|
:tools => [bash_tool],
|
|
))
|
|
|
|
# Run conversation
|
|
prompt(agent, "Hello!")
|
|
|
|
# Wait for completion
|
|
waitForIdle(agent)
|
|
```
|
|
|
|
### Understanding the Flow
|
|
|
|
```
|
|
User Code
|
|
│
|
|
├─► Create Agent
|
|
│ ├─ Initialize state
|
|
│ ├─ Set up queues
|
|
│ └─ Register hooks
|
|
│
|
|
├─► prompt("Hello")
|
|
│ ├─ Validate input
|
|
│ └─ Start AgentLoop
|
|
│
|
|
├─► AgentLoop (runs in thread)
|
|
│ ├─ Stream LLM response
|
|
│ ├─ Execute tools
|
|
│ └─ Emit events
|
|
│
|
|
└─► Event handlers receive events
|
|
├─ MessageEndEvent
|
|
├─ ToolExecutionEndEvent
|
|
└─ AgentEndEvent
|
|
```
|
|
|
|
## Core Concepts
|
|
|
|
### Agent
|
|
|
|
**What it is**: High-level interface for LLM interactions
|
|
|
|
**What it does**:
|
|
- Manages conversation state
|
|
- Handles event streaming
|
|
- Queues steering/follow-up messages
|
|
- Provides hooks for customization
|
|
|
|
**Key methods**:
|
|
- `prompt()` - Start new conversation
|
|
- `continue!()` - Continue existing conversation
|
|
- `steer()` - Queue message for next turn
|
|
- `followUp()` - Queue message after stop
|
|
- `subscribe()` - Listen to events
|
|
- `waitForIdle()` - Wait for agent to finish processing
|
|
- `reset!()` - Clear transcript state and queued messages
|
|
- `clearAllQueues()` - Remove all queued steering and follow-up messages
|
|
- `hasQueuedMessages()` - Check if queues have pending messages
|
|
- `abort()` - Abort the current run
|
|
- `get_state()` - Get the current agent state
|
|
|
|
### AgentLoop
|
|
|
|
**What it is**: Core LLM interaction loop
|
|
|
|
**What it does**:
|
|
- Calls LLM API with streaming
|
|
- Executes tool calls (parallel or sequential)
|
|
- Emits lifecycle events
|
|
- Handles steering/follow-up messages
|
|
|
|
**Key functions**:
|
|
- `agentLoop()` - Start new conversation
|
|
- `agentLoopContinue()` - Continue conversation
|
|
- `runAgentLoop()` - Internal loop execution
|
|
- `streamAssistantResponse()` - LLM API call
|
|
- `executeToolCalls()` - Tool execution
|
|
|
|
### Session
|
|
|
|
**What it is**: Conversation history management
|
|
|
|
**What it does**:
|
|
- Persists messages to storage
|
|
- Supports branching
|
|
- Implements compaction
|
|
- Manages conversation tree
|
|
|
|
**Key methods**:
|
|
- `appendMessage()` - Add message
|
|
- `appendCompaction()` - Compress history with summary
|
|
- `moveTo()` - Navigate branches
|
|
- `buildContext()` - Build context for LLM
|
|
- `getBranch()` - Get branch entries
|
|
- `getSessionStats()` - Get session statistics
|
|
- `appendThinkingLevelChange()` - Record thinking level change
|
|
- `appendModelChange()` - Record model change
|
|
- `appendActiveToolsChange()` - Record active tools change
|
|
|
|
### Tools
|
|
|
|
**What it is**: Functions agents can call
|
|
|
|
**What they do**:
|
|
- Execute external operations
|
|
- Return results to agent
|
|
- Support streaming updates
|
|
- Implement hooks
|
|
|
|
**Built-in tools**:
|
|
- `bash` - Execute shell commands
|
|
- `read` - Read files
|
|
- `write` - Write files
|
|
- `edit` - Edit files
|
|
|
|
## Event System
|
|
|
|
### Event Types
|
|
|
|
```
|
|
AgentEvent
|
|
├─ AgentStartEvent / AgentEndEvent
|
|
├─ TurnStartEvent / TurnEndEvent
|
|
├─ MessageStartEvent / MessageEndEvent
|
|
├─ MessageUpdateEvent
|
|
├─ ToolExecutionStartEvent / ToolExecutionEndEvent
|
|
└─ ToolExecutionUpdateEvent
|
|
```
|
|
|
|
### Event Flow
|
|
|
|
```
|
|
AgentStartEvent
|
|
│
|
|
├─ TurnStartEvent
|
|
│ ├─ MessageStartEvent (user)
|
|
│ ├─ MessageEndEvent (user)
|
|
│ ├─ MessageStartEvent (assistant)
|
|
│ ├─ MessageUpdateEvent (streaming)
|
|
│ ├─ MessageEndEvent (assistant)
|
|
│ ├─ ToolExecutionStartEvent
|
|
│ ├─ ToolExecutionEndEvent
|
|
│ └─ TurnEndEvent
|
|
│
|
|
└─ AgentEndEvent
|
|
```
|
|
|
|
## Complete Data Flow with Type Transformations
|
|
|
|
This documentation shows how data is transformed through the agent lifecycle.
|
|
|
|
### Message Type Hierarchy
|
|
|
|
```
|
|
Message (for LLM API)
|
|
├── UserMessage (role: "user")
|
|
│ └── content::Vector{MessageContent}
|
|
│ ├── TextContent (text::String)
|
|
│ └── ImageContent (data::String, mime_type::String)
|
|
├── AssistantMessage (role: "assistant")
|
|
│ ├── content::Vector{MessageContent}
|
|
│ │ ├── TextContent
|
|
│ │ └── ToolCall (id, name, arguments::Dict{String, Any})
|
|
│ ├── usage::Usage
|
|
│ ├── stop_reason::String
|
|
│ └── timestamp::Timestamp
|
|
└── ToolResultMessage (role: "toolResult")
|
|
├── tool_call_id::String
|
|
├── tool_name::String
|
|
├── content::Vector{MessageContent}
|
|
├── details::Any
|
|
├── usage::Union{Usage, Nothing}
|
|
├── is_error::Bool
|
|
└── timestamp::Timestamp
|
|
|
|
AgentMessage (internal, abstract type)
|
|
├── UserMessage (same as above)
|
|
├── AssistantMessage (same as above)
|
|
├── ToolResultMessage (same as above, plus: role, added_tool_names)
|
|
├── BashExecutionMessage (custom)
|
|
│ ├── role, command, output, exit_code
|
|
│ ├── cancelled, truncated, full_output_path, timestamp
|
|
│ └── exclude_from_context
|
|
├── CompactionSummaryMessage (custom)
|
|
│ ├── role, summary, tokens_before, timestamp
|
|
│ └── converted to UserMessage for LLM
|
|
├── BranchSummaryMessage (custom)
|
|
│ ├── role, summary, from_id, timestamp
|
|
│ └── converted to UserMessage for LLM
|
|
└── CustomMessage (custom, extends AgentMessage)
|
|
├── message::AgentMessage
|
|
└── custom_type::String
|
|
```
|
|
|
|
### Complete Conversation Flow
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Step 1: User Input (Vector{AgentMessage}) │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
prompt(agent, "Hello!")
|
|
│
|
|
└─► normalizePromptInput()
|
|
Input: "Hello!"::String
|
|
Output: [UserMessage("user", [TextContent("Hello!")], timestamp)]
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Step 2: AgentLoop Processing │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
runAgentLoop()
|
|
│
|
|
├─► transform_context() (optional hook)
|
|
│ Input: [UserMessage(...)]::Vector{AgentMessage}
|
|
│ Output: [UserMessage(...)]::Vector{AgentMessage}
|
|
│
|
|
├─► convert_to_llm()
|
|
│ Input: [UserMessage(...)]::Vector{AgentMessage}
|
|
│ Output: [UserMessage(...)]::Vector{Message}
|
|
│
|
|
├─► stream_fn() - LLM API call
|
|
│ Input: model, Context(...), config
|
|
│ Output: AssistantMessage with ToolCall[]
|
|
│
|
|
├─► executeToolCalls()
|
|
│ Input: AssistantMessage (with ToolCall[])
|
|
│ Output: ToolResultMessage[]
|
|
│
|
|
└─► Emit events and append to context.messages
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Step 3: Final Conversation State │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
context.messages::Vector{AgentMessage}
|
|
├─ UserMessage("user", [TextContent("Hello!")], ...)
|
|
├─ AssistantMessage("assistant", [
|
|
│ TextContent("Hi there!"),
|
|
│ ToolCall("bash", {...})
|
|
│ ], ...)
|
|
└─ ToolResultMessage("toolResult", "bash", [TextContent("...")], ...)
|
|
|
|
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
│ Step 4: AgentEndEvent (final output) │
|
|
└─────────────────────────────────────────────────────────────────────────────┘
|
|
AgentEndEvent(messages::Vector{AgentMessage})
|
|
└─ Contains full conversation history
|
|
User Input (String / AgentMessage / Vector{AgentMessage})
|
|
│
|
|
├─► normalizePromptInput()
|
|
│ Input: input::Union{String, AgentMessage, Vector{AgentMessage}}
|
|
│ Output: Vector{AgentMessage}
|
|
│ • String → UserMessage("user", [TextContent(input)], timestamp)
|
|
│ • AgentMessage → [input]
|
|
│ • Vector{AgentMessage} → input (pass-through)
|
|
│
|
|
├─► prompt(agent, messages)
|
|
│ └─► runPromptMessages()
|
|
│
|
|
▼
|
|
AgentLoop Execution:
|
|
│
|
|
├─► transform_context() (optional hook)
|
|
│ Input: context.messages::Vector{AgentMessage}
|
|
│ Output: messages::Vector{AgentMessage} (transformed)
|
|
│
|
|
├─► convert_to_llm()
|
|
│ Input: messages::Vector{AgentMessage}
|
|
│ Output: llm_messages::Vector{Message}
|
|
│
|
|
│ AgentMessage → Message mapping:
|
|
│ • UserMessage → UserMessage (pass-through)
|
|
│ • AssistantMessage → AssistantMessage (pass-through)
|
|
│ • ToolResultMessage → ToolResultMessage (pass-through)
|
|
│ • BashExecutionMessage → UserMessage (text conversion)
|
|
│ • CompactionSummaryMessage → UserMessage (text wrapped)
|
|
│ • BranchSummaryMessage → UserMessage (text wrapped)
|
|
│
|
|
├─► LLM API Call (stream_fn)
|
|
│ Input: model, Context(system_prompt, llm_messages, tools), config
|
|
│ Output: Stream{AssistantMessageEvent}
|
|
│
|
|
├─► AssistantMessage (returned from LLM)
|
|
│ content::Vector{MessageContent}
|
|
│ └─ Contains: TextContent[] and/or ToolCall[]
|
|
│
|
|
├─► executeToolCalls() (if ToolCall[] in content)
|
|
│ │
|
|
│ ├─► prepareToolCall() for each ToolCall
|
|
│ │ Input: tool_call::ToolCall
|
|
│ │ Output: PreparedToolCall or ImmediateToolCallOutcome
|
|
│ │
|
|
│ ├─► executePreparedToolCall() (if prepared)
|
|
│ │ Input: PreparedToolCall
|
|
│ │ Output: ExecutedToolCallOutcome
|
|
│ │ tool.execute() returns AgentToolResultMutable
|
|
│ │
|
|
│ ├─► finalizeExecutedToolCall()
|
|
│ │ Input: ExecutedToolCallOutcome
|
|
│ │ Output: FinalizedToolCallOutcome
|
|
│ │
|
|
│ └─► createToolResultMessage()
|
|
│ Input: FinalizedToolCallOutcome
|
|
│ Output: ToolResultMessage
|
|
│ • role: "toolResult"
|
|
│ • tool_call_id, tool_name
|
|
│ • content::Vector{MessageContent}
|
|
│ • details, usage, added_tool_names
|
|
│ • is_error, timestamp
|
|
│
|
|
└─► Append to context.messages and new_messages
|
|
│
|
|
▼
|
|
Vector{AgentMessage} (final conversation history)
|
|
Contains: [UserMessage, AssistantMessage, ToolResultMessage, ...]
|
|
```
|
|
|
|
### Tool Execution Flow
|
|
|
|
```
|
|
ToolCall (in AssistantMessage.content)
|
|
│
|
|
├─ before_tool_call hook (optional)
|
|
│ Input: BeforeToolCallContext
|
|
│ Output: BeforeToolCallResult (block, reason) or nothing
|
|
│
|
|
├─ prepareToolCall()
|
|
│ Input: tool_call::ToolCall
|
|
│ Output: Union{PreparedToolCall, ImmediateToolCallOutcome}
|
|
│ • Validates tool exists
|
|
│ • Runs before_tool_call hook
|
|
│ • Runs prepare_arguments hook (optional)
|
|
│ • Runs validateToolArguments (optional)
|
|
│
|
|
├─ executePreparedToolCall() (if prepared)
|
|
│ Input: PreparedToolCall
|
|
│ Output: ExecutedToolCallOutcome
|
|
│ tool.execute() returns AgentToolResultMutable
|
|
│
|
|
├─ finalizeExecutedToolCall()
|
|
│ Input: ExecutedToolCallOutcome
|
|
│ Output: FinalizedToolCallOutcome
|
|
│ Runs after_tool_call hook (optional)
|
|
│
|
|
└─ createToolResultMessage()
|
|
Input: FinalizedToolCallOutcome
|
|
Output: ToolResultMessage
|
|
• role: "toolResult"
|
|
• tool_call_id, tool_name
|
|
• content::Vector{MessageContent}
|
|
• details, usage, added_tool_names
|
|
• is_error, timestamp
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
### 1. Use Hooks for Customization
|
|
|
|
```julia
|
|
# Before tool call
|
|
before_hook = (context, signal) -> begin
|
|
println("Executing: $(context.tool_call.name)")
|
|
return nothing
|
|
end
|
|
|
|
# After tool call
|
|
after_hook = (context, signal) -> begin
|
|
if context.is_error
|
|
println("Tool failed: $(context.tool_call.name)")
|
|
end
|
|
return nothing
|
|
end
|
|
```
|
|
|
|
### 2. Monitor Events
|
|
|
|
```julia
|
|
subscribe(agent) do event, signal
|
|
if event isa MessageEndEvent
|
|
println("Message: $(event.message)")
|
|
elseif event isa ToolExecutionEndEvent
|
|
println("Tool completed: $(event.tool_name)")
|
|
end
|
|
end
|
|
```
|
|
|
|
### 3. Use Steering for Redirection
|
|
|
|
```julia
|
|
# Agent is going wrong direction
|
|
steer(agent, UserMessage("Actually, let's do X instead"))
|
|
```
|
|
|
|
### 4. Use Follow-Up for Continuation
|
|
|
|
```julia
|
|
# Agent thinks it's done, but user wants more
|
|
followUp(agent, UserMessage("Wait, there's one more thing"))
|
|
```
|
|
|
|
## Common Patterns
|
|
|
|
### Pattern 1: Conversation with Memory
|
|
|
|
```julia
|
|
# Use Session to persist conversation
|
|
storage = JsonlSessionStorage(...)
|
|
session = Session(storage)
|
|
|
|
# Add messages to session
|
|
appendMessage(session, user_message)
|
|
appendMessage(session, assistant_message)
|
|
|
|
# Build context from session
|
|
context = buildContext(session)
|
|
```
|
|
|
|
### Pattern 2: Long Conversations
|
|
|
|
```julia
|
|
# Compact periodically to stay within context limits
|
|
if token_count > MAX_TOKENS * 0.8
|
|
compact_id = appendCompaction(
|
|
session,
|
|
summary,
|
|
first_kept_id,
|
|
token_count,
|
|
)
|
|
end
|
|
```
|
|
|
|
### Pattern 3: Branching Conversations
|
|
|
|
```julia
|
|
# User wants to explore alternative
|
|
session.moveTo(branch_point_id)
|
|
|
|
# Create new branch
|
|
moveTo(session, branch_point_id, summary=["summary" => "Exploring alternative approach"])
|
|
appendMessage(session, new_user_message)
|
|
```
|
|
|
|
### Pattern 4: Custom Tools
|
|
|
|
```julia
|
|
# Create custom tool
|
|
custom_tool = AgentTool(
|
|
"custom", # name
|
|
"Custom", # label
|
|
"Does custom thing", # description
|
|
parameters, # parameter schema
|
|
execute_function, # execute
|
|
nothing, # prepare_arguments (optional)
|
|
EXECUTION_PARALLEL, # execution_mode
|
|
)
|
|
|
|
# Add to agent
|
|
agent = Agent(Dict(:tools => [custom_tool]))
|
|
```
|
|
|
|
## Debugging
|
|
|
|
### Check Active Run
|
|
|
|
```julia
|
|
if !isnothing(agent.active_run)
|
|
println("Agent is busy")
|
|
else
|
|
println("Agent is idle")
|
|
end
|
|
```
|
|
|
|
### Clear Queues
|
|
|
|
```julia
|
|
clearAllQueues(agent)
|
|
```
|
|
|
|
### Reset State
|
|
|
|
```julia
|
|
reset!(agent)
|
|
```
|
|
|
|
## Performance Tips
|
|
|
|
1. **Use parallel execution** for independent tools
|
|
2. **Compact periodically** for long conversations
|
|
3. **Use thinking_level wisely** (higher = slower but better)
|
|
4. **Batch tool calls** when possible
|
|
5. **Cache LLM responses** when appropriate
|
|
|
|
## Troubleshooting
|
|
|
|
### Agent stuck in loop
|
|
|
|
```julia
|
|
# Check if agent is still processing
|
|
if hasQueuedMessages(agent)
|
|
# Clear queues
|
|
clearAllQueues(agent)
|
|
end
|
|
```
|
|
|
|
### Too many tokens
|
|
|
|
```julia
|
|
# Compact session
|
|
compact_id = appendCompaction(
|
|
session,
|
|
summary,
|
|
first_kept_id,
|
|
token_count,
|
|
)
|
|
```
|
|
|
|
### Tool execution failed
|
|
|
|
```julia
|
|
# Check tool result
|
|
if result.is_error
|
|
println("Tool failed: $(result.error)")
|
|
end
|
|
```
|
|
|
|
## Next Steps
|
|
|
|
1. Read **Architecture Overview** for deep understanding
|
|
2. Explore **Agent Component** for state management
|
|
3. Study **AgentLoop** for core logic
|
|
4. Learn **Types & Messages** for data structures
|
|
5. Master **Session Management** for persistence
|
|
6. Build **Tools** for custom functionality
|
|
|
|
## Resources
|
|
|
|
- Original TypeScript implementation: `@earendil-works/pi-agent-core`
|
|
- AgentCore.jl source code: `src/`
|
|
- Examples: `examples/`
|
|
|
|
## Community
|
|
|
|
For questions and discussions:
|
|
- GitHub Issues: `/issues`
|
|
- Documentation: `docs/`
|