YiemAgent/learning/README.md

AgentCore.jl - Learning Guide

How to Use This Documentation

Top-Down Learning Approach

This documentation is organized in a top-down order, starting from high-level concepts and drilling down into implementation details. Follow this sequence:

1. Architecture Overview - Understand the big picture
2. Agent Component - Learn about state management and event streaming
3. AgentLoop Component - Understand the core LLM interaction loop
4. Types & Messages - Learn the data structures
5. Session Management - Understand conversation history
6. Tools - Learn about tool execution

Learning Style

• Visual learners: Study the ASCII diagrams
• Hands-on learners: Code examples provided for each section
• Conceptual learners: Read summaries and overviews first

Quick Start

Minimal Example

using AgentCore

# Create agent
agent = Agent(Dict(
    :systemPrompt => "You are a helpful assistant.",
    :model => Model(...),
    :tools => [bash_tool],
))

# Run conversation
prompt(agent, "Hello!")

# Wait for completion
waitForIdle(agent)

Understanding the Flow

User Code
    │
    ├─► Create Agent
    │   ├─ Initialize state
    │   ├─ Set up queues
    │   └─ Register hooks
    │
    ├─► prompt("Hello")
    │   ├─ Validate input
    │   └─ Start AgentLoop
    │
    ├─► AgentLoop (runs in thread)
    │   ├─ Stream LLM response
    │   ├─ Execute tools
    │   └─ Emit events
    │
    └─► Event handlers receive events
        ├─ MessageEndEvent
        ├─ ToolExecutionEndEvent
        └─ AgentEndEvent

Core Concepts

Agent

What it is: High-level interface for LLM interactions

What it does:

• Manages conversation state
• Handles event streaming
• Queues steering/follow-up messages
• Provides hooks for customization

Key methods:

• prompt() - Start new conversation
• continue!() - Continue existing conversation
• steer() - Queue message for next turn
• followUp() - Queue message after stop
• subscribe() - Listen to events
• waitForIdle() - Wait for agent to finish processing
• reset!() - Clear transcript state and queued messages
• clearAllQueues() - Remove all queued steering and follow-up messages
• hasQueuedMessages() - Check if queues have pending messages
• abort() - Abort the current run
• get_state() - Get the current agent state

AgentLoop

What it is: Core LLM interaction loop

What it does:

• Calls LLM API with streaming
• Executes tool calls (parallel or sequential)
• Emits lifecycle events
• Handles steering/follow-up messages

Key functions:

• agentLoop() - Start new conversation
• agentLoopContinue() - Continue conversation
• runAgentLoop() - Internal loop execution
• streamAssistantResponse() - LLM API call
• executeToolCalls() - Tool execution

Session

What it is: Conversation history management

What it does:

• Persists messages to storage
• Supports branching
• Implements compaction
• Manages conversation tree

Key methods:

• appendMessage() - Add message
• appendCompaction() - Compress history with summary
• moveTo() - Navigate branches
• buildContext() - Build context for LLM
• getBranch() - Get branch entries
• getSessionStats() - Get session statistics
• appendThinkingLevelChange() - Record thinking level change
• appendModelChange() - Record model change
• appendActiveToolsChange() - Record active tools change

Tools

What it is: Functions agents can call

What they do:

• Execute external operations
• Return results to agent
• Support streaming updates
• Implement hooks

Built-in tools:

• bash - Execute shell commands
• read - Read files
• write - Write files
• edit - Edit files

Event System

Event Types

AgentEvent
├─ AgentStartEvent / AgentEndEvent
├─ TurnStartEvent / TurnEndEvent
├─ MessageStartEvent / MessageEndEvent
├─ MessageUpdateEvent
├─ ToolExecutionStartEvent / ToolExecutionEndEvent
└─ ToolExecutionUpdateEvent

Event Flow

AgentStartEvent
    │
    ├─ TurnStartEvent
    │   ├─ MessageStartEvent (user)
    │   ├─ MessageEndEvent (user)
    │   ├─ MessageStartEvent (assistant)
    │   ├─ MessageUpdateEvent (streaming)
    │   ├─ MessageEndEvent (assistant)
    │   ├─ ToolExecutionStartEvent
    │   ├─ ToolExecutionEndEvent
    │   └─ TurnEndEvent
    │
    └─ AgentEndEvent

Complete Data Flow with Type Transformations

This documentation shows how data is transformed through the agent lifecycle.

Message Type Hierarchy

Message (for LLM API)
├── UserMessage (role: "user")
│   └── content::Vector{MessageContent}
│       ├── TextContent (text::String)
│       └── ImageContent (data::String, mime_type::String)
├── AssistantMessage (role: "assistant")
│   ├── content::Vector{MessageContent}
│   │   ├── TextContent
│   │   └── ToolCall (id, name, arguments::Dict{String, Any})
│   ├── usage::Usage
│   ├── stop_reason::String
│   └── timestamp::Timestamp
└── ToolResultMessage (role: "toolResult")
    ├── tool_call_id::String
    ├── tool_name::String
    ├── content::Vector{MessageContent}
    ├── details::Any
    ├── usage::Union{Usage, Nothing}
    ├── is_error::Bool
    └── timestamp::Timestamp

AgentMessage (internal, abstract type)
├── UserMessage (same as above)
├── AssistantMessage (same as above)
├── ToolResultMessage (same as above, plus: role, added_tool_names)
├── BashExecutionMessage (custom)
│   ├── role, command, output, exit_code
│   ├── cancelled, truncated, full_output_path, timestamp
│   └── exclude_from_context
├── CompactionSummaryMessage (custom)
│   ├── role, summary, tokens_before, timestamp
│   └── converted to UserMessage for LLM
├── BranchSummaryMessage (custom)
│   ├── role, summary, from_id, timestamp
│   └── converted to UserMessage for LLM
└── CustomMessage (custom, extends AgentMessage)
    ├── message::AgentMessage
    └── custom_type::String

Complete Conversation Flow

┌─────────────────────────────────────────────────────────────────────────────┐
│ Step 1: User Input (Vector{AgentMessage})                                 │
└─────────────────────────────────────────────────────────────────────────────┘
prompt(agent, "Hello!")
    │
    └─► normalizePromptInput()
         Input:  "Hello!"::String
         Output: [UserMessage("user", [TextContent("Hello!")], timestamp)]

┌─────────────────────────────────────────────────────────────────────────────┐
│ Step 2: AgentLoop Processing                                                │
└─────────────────────────────────────────────────────────────────────────────┘
runAgentLoop()
    │
    ├─► transform_context() (optional hook)
    │    Input:  [UserMessage(...)]::Vector{AgentMessage}
    │    Output: [UserMessage(...)]::Vector{AgentMessage}
    │
    ├─► convert_to_llm()
    │    Input:  [UserMessage(...)]::Vector{AgentMessage}
    │    Output: [UserMessage(...)]::Vector{Message}
    │
    ├─► stream_fn() - LLM API call
    │    Input:  model, Context(...), config
    │    Output: AssistantMessage with ToolCall[]
    │
    ├─► executeToolCalls()
    │    Input:  AssistantMessage (with ToolCall[])
    │    Output: ToolResultMessage[]
    │
    └─► Emit events and append to context.messages

┌─────────────────────────────────────────────────────────────────────────────┐
│ Step 3: Final Conversation State                                          │
└─────────────────────────────────────────────────────────────────────────────┘
context.messages::Vector{AgentMessage}
├─ UserMessage("user", [TextContent("Hello!")], ...)
├─ AssistantMessage("assistant", [
│     TextContent("Hi there!"),
│     ToolCall("bash", {...})
│   ], ...)
└─ ToolResultMessage("toolResult", "bash", [TextContent("...")], ...)

┌─────────────────────────────────────────────────────────────────────────────┐
│ Step 4: AgentEndEvent (final output)                                      │
└─────────────────────────────────────────────────────────────────────────────┘
AgentEndEvent(messages::Vector{AgentMessage})
    └─ Contains full conversation history
User Input (String / AgentMessage / Vector{AgentMessage})
    │
    ├─► normalizePromptInput()
    │   Input:  input::Union{String, AgentMessage, Vector{AgentMessage}}
    │   Output: Vector{AgentMessage}
    │      • String → UserMessage("user", [TextContent(input)], timestamp)
    │      • AgentMessage → [input]
    │      • Vector{AgentMessage} → input (pass-through)
    │
    ├─► prompt(agent, messages)
    │   └─► runPromptMessages()
    │
    ▼
AgentLoop Execution:
    │
    ├─► transform_context() (optional hook)
    │   Input:  context.messages::Vector{AgentMessage}
    │   Output: messages::Vector{AgentMessage} (transformed)
    │
    ├─► convert_to_llm()
    │   Input:  messages::Vector{AgentMessage}
    │   Output: llm_messages::Vector{Message}
    │
    │   AgentMessage → Message mapping:
    │   • UserMessage → UserMessage (pass-through)
    │   • AssistantMessage → AssistantMessage (pass-through)
    │   • ToolResultMessage → ToolResultMessage (pass-through)
    │   • BashExecutionMessage → UserMessage (text conversion)
    │   • CompactionSummaryMessage → UserMessage (text wrapped)
    │   • BranchSummaryMessage → UserMessage (text wrapped)
    │
    ├─► LLM API Call (stream_fn)
    │   Input:  model, Context(system_prompt, llm_messages, tools), config
    │   Output: Stream{AssistantMessageEvent}
    │
    ├─► AssistantMessage (returned from LLM)
    │   content::Vector{MessageContent}
    │   └─ Contains: TextContent[] and/or ToolCall[]
    │
    ├─► executeToolCalls() (if ToolCall[] in content)
    │   │
    │   ├─► prepareToolCall() for each ToolCall
    │   │   Input:  tool_call::ToolCall
    │   │   Output: PreparedToolCall or ImmediateToolCallOutcome
    │   │
    │   ├─► executePreparedToolCall() (if prepared)
    │   │   Input:  PreparedToolCall
    │   │   Output: ExecutedToolCallOutcome
    │   │   tool.execute() returns AgentToolResultMutable
    │   │
    │   ├─► finalizeExecutedToolCall()
    │   │   Input:  ExecutedToolCallOutcome
    │   │   Output: FinalizedToolCallOutcome
    │   │
    │   └─► createToolResultMessage()
    │       Input:  FinalizedToolCallOutcome
    │       Output: ToolResultMessage
    │           • role: "toolResult"
    │           • tool_call_id, tool_name
    │           • content::Vector{MessageContent}
    │           • details, usage, added_tool_names
    │           • is_error, timestamp
    │
    └─► Append to context.messages and new_messages
        │
        ▼
    Vector{AgentMessage} (final conversation history)
        Contains: [UserMessage, AssistantMessage, ToolResultMessage, ...]

Tool Execution Flow

ToolCall (in AssistantMessage.content)
    │
    ├─ before_tool_call hook (optional)
    │   Input: BeforeToolCallContext
    │   Output: BeforeToolCallResult (block, reason) or nothing
    │
    ├─ prepareToolCall()
    │   Input:  tool_call::ToolCall
    │   Output: Union{PreparedToolCall, ImmediateToolCallOutcome}
    │       • Validates tool exists
    │       • Runs before_tool_call hook
    │       • Runs prepare_arguments hook (optional)
    │       • Runs validateToolArguments (optional)
    │
    ├─ executePreparedToolCall() (if prepared)
    │   Input:  PreparedToolCall
    │   Output: ExecutedToolCallOutcome
    │   tool.execute() returns AgentToolResultMutable
    │
    ├─ finalizeExecutedToolCall()
    │   Input:  ExecutedToolCallOutcome
    │   Output: FinalizedToolCallOutcome
    │   Runs after_tool_call hook (optional)
    │
    └─ createToolResultMessage()
        Input:  FinalizedToolCallOutcome
        Output: ToolResultMessage
            • role: "toolResult"
            • tool_call_id, tool_name
            • content::Vector{MessageContent}
            • details, usage, added_tool_names
            • is_error, timestamp

Best Practices

1. Use Hooks for Customization

# Before tool call
before_hook = (context, signal) -> begin
    println("Executing: $(context.tool_call.name)")
    return nothing
end

# After tool call
after_hook = (context, signal) -> begin
    if context.is_error
        println("Tool failed: $(context.tool_call.name)")
    end
    return nothing
end

2. Monitor Events

subscribe(agent) do event, signal
    if event isa MessageEndEvent
        println("Message: $(event.message)")
    elseif event isa ToolExecutionEndEvent
        println("Tool completed: $(event.tool_name)")
    end
end

3. Use Steering for Redirection

# Agent is going wrong direction
steer(agent, UserMessage("Actually, let's do X instead"))

4. Use Follow-Up for Continuation

# Agent thinks it's done, but user wants more
followUp(agent, UserMessage("Wait, there's one more thing"))

Common Patterns

Pattern 1: Conversation with Memory

# Use Session to persist conversation
storage = JsonlSessionStorage(...)
session = Session(storage)

# Add messages to session
appendMessage(session, user_message)
appendMessage(session, assistant_message)

# Build context from session
context = buildContext(session)

Pattern 2: Long Conversations

# Compact periodically to stay within context limits
if token_count > MAX_TOKENS * 0.8
    compact_id = appendCompaction(
        session,
        summary,
        first_kept_id,
        token_count,
    )
end

Pattern 3: Branching Conversations

# User wants to explore alternative
session.moveTo(branch_point_id)

# Create new branch
moveTo(session, branch_point_id, summary=["summary" => "Exploring alternative approach"])
appendMessage(session, new_user_message)

Pattern 4: Custom Tools

# Create custom tool
custom_tool = AgentTool(
    "custom",           # name
    "Custom",           # label
    "Does custom thing", # description
    parameters,         # parameter schema
    execute_function,   # execute
    nothing,            # prepare_arguments (optional)
    EXECUTION_PARALLEL, # execution_mode
)

# Add to agent
agent = Agent(Dict(:tools => [custom_tool]))

Debugging

Check Active Run

if !isnothing(agent.active_run)
    println("Agent is busy")
else
    println("Agent is idle")
end

Clear Queues

clearAllQueues(agent)

Reset State

reset!(agent)

Performance Tips

1. Use parallel execution for independent tools
2. Compact periodically for long conversations
3. Use thinking_level wisely (higher = slower but better)
4. Batch tool calls when possible
5. Cache LLM responses when appropriate

Troubleshooting

Agent stuck in loop

# Check if agent is still processing
if hasQueuedMessages(agent)
    # Clear queues
    clearAllQueues(agent)
end

Too many tokens

# Compact session
compact_id = appendCompaction(
    session,
    summary,
    first_kept_id,
    token_count,
)

Tool execution failed

# Check tool result
if result.is_error
    println("Tool failed: $(result.error)")
end

Next Steps

1. Read Architecture Overview for deep understanding
2. Explore Agent Component for state management
3. Study AgentLoop for core logic
4. Learn Types & Messages for data structures
5. Master Session Management for persistence
6. Build Tools for custom functionality

Resources

• Original TypeScript implementation: @earendil-works/pi-agent-core
• AgentCore.jl source code: src/
• Examples: examples/

Community

For questions and discussions:

• GitHub Issues: /issues
• Documentation: docs/