# AgentCore.jl - Agent Component Deep Dive ## Agent Structure ```julia mutable struct Agent _state::AgentState listeners::Set{Tuple{Function, Ref{Bool}}} steering_queue::PendingMessageQueue follow_up_queue::PendingMessageQueue convert_to_llm::Function transform_context::Union{Function, Nothing} stream_function::StreamFn get_api_key::Union{Function, Nothing} on_payload::Union{Function, Nothing} on_response::Union{Function, Nothing} before_tool_call::Union{Function, Nothing} after_tool_call::Union{Function, Nothing} prepare_next_turn::Union{Function, Nothing} prepare_next_turn_with_context::Union{Function, Nothing} active_run::Union{ActiveRun, Nothing} session_id::Union{String, Nothing} thinking_budgets::Union{Dict{String, Int64}, Nothing} transport::String max_retry_delay_ms::Union{Int64, Nothing} tool_execution::ToolExecutionMode end ``` ## Agent Lifecycle ### 1. Initialization ```julia # Create agent with options agent = Agent(Dict{Symbol, Any}( :systemPrompt => "You are a helpful assistant", :model => Model(...), :thinkingLevel => THINKING_MEDIUM, :tools => [bash_tool, read_tool], :steeringMode => QUEUE_ONE_AT_A_TIME, :followUpMode => QUEUE_ONE_AT_A_TIME, :toolExecution => EXECUTION_PARALLEL, )) # Subscribe to events unsubscribe = subscribe(agent) do event, signal if event isa MessageEndEvent println("Message: $(event.message)") elseif event isa ToolExecutionEndEvent println("Tool completed: $(event.tool_name)") end end ``` ### 2. Message Queues #### Steering Queue - Messages injected **after** the current assistant turn finishes - Used to correct or redirect the agent's behavior - Example: "Actually, let's do X instead" #### Follow-Up Queue - Messages run **only after** the agent would otherwise stop - Used to continue conversation when agent thinks it's done - Example: "Wait, there's one more thing" #### Queue Modes - `QUEUE_ALL` - Drain all messages at once - `QUEUE_ONE_AT_A_TIME` - Process one message at a time ```julia # Queue a steering message steer(agent, UserMessage(...)) # Queue a follow-up message followUp(agent, UserMessage(...)) # Check if queues have items hasQueuedMessages(agent) # Returns Bool # Clear queues clearSteeringQueue(agent) clearFollowUpQueue(agent) clearAllQueues(agent) ``` ### 3. Event System #### Agent Events ```julia abstract type AgentEvent end # Lifecycle events struct AgentStartEvent <: AgentEvent end struct AgentEndEvent <: AgentEvent messages::Vector{AgentMessage} end # Turn events struct TurnStartEvent <: AgentEvent end struct TurnEndEvent <: AgentEvent message::AgentMessage tool_results::Vector{ToolResultMessage} end # Message events struct MessageStartEvent <: AgentEvent message::AgentMessage end struct MessageUpdateEvent <: AgentEvent message::AgentMessage assistant_message_event::Any end struct MessageEndEvent <: AgentEvent message::AgentMessage end # Tool execution events struct ToolExecutionStartEvent <: AgentEvent tool_call_id::String tool_name::String args::Any end struct ToolExecutionUpdateEvent <: AgentEvent tool_call_id::String tool_name::String args::Any partial_result::Any end struct ToolExecutionEndEvent <: AgentEvent tool_call_id::String tool_name::String result::Any is_error::Bool end ``` #### Event Flow Diagram ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Event Timeline │ └─────────────────────────────────────────────────────────────────────────┘ AgentStartEvent │ ├─ TurnStartEvent │ │ │ ├─ MessageStartEvent (user prompt) │ ├─ MessageEndEvent (user prompt) │ │ │ ├─ [Loop starts] │ │ │ │ │ ├─ MessageStartEvent (assistant response) │ │ ├─ MessageUpdateEvent (text delta 1) │ │ ├─ MessageUpdateEvent (text delta 2) │ │ ├─ MessageUpdateEvent (tool call delta) │ │ ├─ MessageEndEvent (assistant complete) │ │ │ │ │ ├─ ToolExecutionStartEvent (tc1) │ │ ├─ ToolExecutionUpdateEvent (partial result) │ │ ├─ ToolExecutionEndEvent (tc1 done) │ │ │ │ │ ├─ ToolExecutionStartEvent (tc2) │ │ ├─ ToolExecutionEndEvent (tc2 done) │ │ │ │ │ └─ TurnEndEvent (assistant + tools) │ │ │ └─ [Next turn if needed] │ └─ AgentEndEvent (final messages) ``` ### 4. State Management ```julia mutable struct AgentState system_prompt::String model::Model thinking_level::ThinkingLevel tools::Vector{AgentTool} messages::Vector{AgentMessage} is_streaming::Bool streaming_message::Union{AgentMessage, Nothing} pending_tool_calls::Set{String} error_message::Union{String, Nothing} end ``` #### State Access ```julia # Get current state state = get_state(agent) # Reset state reset!(agent) # Clears messages, queues, and runtime state ``` ### 5. Main Methods #### prompt() ```julia # Start a new conversation prompt(agent, "Hello, how are you?") # With multiple messages prompt(agent, [ UserMessage(...), AssistantMessage(...), UserMessage(...) ]) # With images prompt(agent, "Analyze this image", [ImageContent(data, "image/png")]) ``` #### continue!() ```julia # Continue from current transcript # Last message must be user or tool-result continue!(agent) ``` #### steer() and followUp() ```julia # Steering: Redirect after next assistant turn steer(agent, UserMessage(...)) # Follow-up: Continue after agent would stop followUp(agent, UserMessage(...)) ``` ### 6. Hooks #### convert_to_llm ```julia # Transform messages before sending to LLM function myConvertToLlm(messages::Vector{AgentMessage}) return filter( m -> m.role in ["user", "assistant", "toolResult"], messages ) end agent = Agent(Dict(:convertToLlm => myConvertToLlm)) ``` #### transform_context ```julia # Transform context before LLM call function myTransformContext(messages, signal) # Can truncate, filter, or modify messages return messages end agent = Agent(Dict(:transformContext => myTransformContext)) ``` #### before_tool_call ```julia # Hook before tool execution function myBeforeToolCall(context, signal) println("About to execute: $(context.tool_call.name)") return nothing # Return block=true to prevent execution end agent = Agent(Dict(:beforeToolCall => myBeforeToolCall)) ``` #### after_tool_call ```julia # Hook after tool execution function myAfterToolCall(context, signal) # Can modify tool result return AfterToolCallResult( content = context.result.content, terminate = context.result.terminate ) end agent = Agent(Dict(:afterToolCall => myAfterToolCall)) ``` #### prepare_next_turn ```julia # Modify context/model/thinking level between turns function myPrepareNextTurn(context, signal) # context: PrepareNextTurnContext # Returns AgentLoopTurnUpdate or nothing return AgentLoopTurnUpdate( context = context.context, model = context.context.model, # Can change model thinking_level = THINKING_HIGH # Can change thinking level ) end agent = Agent(Dict(:prepareNextTurn => myPrepareNextTurn)) ``` ### 7. Active Run Management ```julia # Check if agent is busy if !isnothing(agent.active_run) # Agent is processing abort(agent) # Abort current run end # Wait for completion wait_for_idle(agent) # Returns Promise ``` ## Complete Example ```julia using AgentCore # 1. Create agent agent = Agent(Dict( :systemPrompt => "You are a helpful assistant.", :model => Model(...), :tools => [bash_tool, read_tool], )) # 2. Subscribe to events events_received = [] unsubscribe = subscribe(agent) do event, signal push!(events_received, event) if event isa MessageEndEvent println("Message: $(event.message)") end end # 3. Start conversation prompt(agent, "What's in the current directory?") # 4. Wait for completion wait_for_idle(agent) # 5. Check final state state = get_state(agent) println("Total messages: $(length(state.messages))") # 6. Continue with steering steer(agent, UserMessage(...)) wait_for_idle(agent) # 7. Clean up unsubscribe() # Stop listening reset!(agent) # Clear state ``` ## Key Concepts ### Message Queueing ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Message Queue Behavior │ └─────────────────────────────────────────────────────────────────────────┘ Scenario: User sends message, agent responds with tool calls ┌────────────────────────────────────────────────────────────┐ │ Time 0: User sends message │ │ ┌──────────────┐ │ │ │ prompt(msg) │ │ │ └──────────────┘ │ │ │ │ │ ▼ │ │ ┌─────────────┐ │ │ │ AgentLoop │ │ │ │ processes │ │ │ │ msg │ │ │ └─────────────┘ │ └────────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Time 1: Agent responds with tool calls │ │ ┌──────────────────────────────────────────────────────┐ │ │ │ AssistantMessage: │ │ │ │ content: [Text("I'll check..."), │ │ │ │ ToolCall("bash", {...}), │ │ │ │ ToolCall("read", {...})] │ │ │ └──────────────────────────────────────────────────────┘ │ └────────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Time 2: User queues steering message │ │ ┌──────────────────┐ │ │ │ steer(msg2) │ ──► steering_queue.push(msg2) │ │ └──────────────────┘ │ │ │ │ (msg2 not processed yet!) │ └────────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Time 3: Tool execution │ │ ┌──────────────────────────────────────────────────────┐ │ │ │ Execute bash tool... │ │ │ │ Execute read tool... │ │ │ │ Emit ToolResultMessage[] │ │ │ └──────────────────────────────────────────────────────┘ │ └────────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Time 4: Agent responds to tool results │ │ ┌──────────────────────────────────────────────────────┐ │ │ │ AssistantMessage (2nd turn): │ │ │ │ content: [Text("The results are...")] │ │ │ └──────────────────────────────────────────────────────┘ │ └────────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Time 5: Steering message processed │ │ ┌──────────────────────────────────────────────────────┐ │ │ │ steering_queue.drain() → [msg2] │ │ │ │ Emit msg2 as UserMessage │ │ │ └──────────────────────────────────────────────────────┘ │ └────────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────────────────────────────────────────────────┐ │ Time 6: Next turn (agent responds to steering) │ │ ┌──────────────────────────────────────────────────────┐ │ │ │ AssistantMessage (3rd turn): │ │ │ │ content: [Text("Okay, I'll do X instead...")] │ │ │ └──────────────────────────────────────────────────────┘ │ └────────────────────────────────────────────────────────────┘ ``` ### Queue Behavior Summary | Action | Queue | When Processed | |--------|-------|----------------| | `prompt()` | N/A | Immediate | | `steer()` | steering_queue | After assistant turn completes | | `followUp()` | follow_up_queue | After agent would normally stop | | `continue!()` | N/A | Immediately if last message is user/tool | ## Best Practices 1. **Use steering for redirects**: When user wants to change direction mid-conversation 2. **Use follow-up for continuation**: When agent thinks it's done but user wants more 3. **Subscribe to events**: Monitor agent behavior and debug issues 4. **Clear queues**: Use `clearAllQueues()` when resetting conversation 5. **Check active run**: Don't call `prompt()` while agent is busy