Files
YiemAgent/learning/07-AGENTHARNESS.md
T
2026-07-29 11:14:04 +07:00

755 lines
28 KiB
Markdown

# AgentCore.jl - AgentHarness Deep Dive
## AgentHarness Architecture
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ AgentHarness Layer │
└─────────────────────────────────────────────────────────────────────────────┐
┌─────────────────────────────────────────────────────────────────────────────┐
│ AgentHarness = Agent + Session + Resources │
│ │
│ ┌───────────────────────────────────────────────────────────────────────┐ │
│ │ AgentHarness │ │
│ │ - Manages Agent instances │ │
│ │ - Provides Session persistence │ │
│ │ - Manages resources (skills, prompt templates) │ │
│ │ - Handles extension hooks │ │
│ │ - Coordinates tool execution with context │ │
│ └───────────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────┼─────────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Agent │ │ SessionRepo │ │ Resources │ │
│ │ (state, │ │ (create, │ │ (skills, │ │
│ │ events) │ │ open, │ │ templates) │ │
│ └──────────────┘ │ list) │ └──────────────┘ │
│ └──────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ Session │ │
│ │ (history, │ │
│ │ branching) │ │
│ └──────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────┐
│ AgentHarnessEvent System │
└─────────────────────────────────────────────────────────────────────────────┘
AgentEvent (from Agent)
├─ AgentHarnessOwnEvent
│ ├─ BeforeAgentStartEvent
│ ├─ ContextEvent
│ ├─ BeforeProviderRequestEvent
│ ├─ BeforeProviderPayloadEvent
│ ├─ AfterProviderResponseEvent
│ ├─ ToolCallEvent
│ ├─ ToolResultEvent
│ ├─ SessionBeforeCompactEvent
│ ├─ SessionCompactEvent
│ ├─ SessionBeforeTreeEvent
│ ├─ SessionTreeEvent
│ ├─ ModelUpdateEvent
│ ├─ ThinkingLevelUpdateEvent
│ ├─ ToolsUpdateEvent
│ ├─ ResourcesUpdateEvent
│ └─ ... (other session events)
└─ AgentEvent (from AgentLoop)
├─ AgentStartEvent / AgentEndEvent
├─ TurnStartEvent / TurnEndEvent
├─ MessageStartEvent / MessageEndEvent
└─ ToolExecutionStartEvent / ToolExecutionEndEvent
```
## AgentHarness Components
### 1. AgentHarnessOptions
```julia
mutable struct AgentHarnessOptions{
TC, TSkill<:Skill, TPromptTemplate<:PromptTemplate, TTool<:AgentHarnessTool
}
session::Session
models::Any
tools::Union{Vector{TTool}, Nothing}
resources::Union{AgentHarnessResources{TSkill, TPromptTemplate}, Nothing}
system_prompt::Union{AgentHarnessSystemPrompt{TC, TSkill, TPromptTemplate, TTool}, Nothing}
stream_options::Union{AgentHarnessStreamOptions, Nothing}
retry::Union{Any, Nothing}
model::Model
thinking_level::Union{ThinkingLevel, Nothing}
active_tool_names::Union{Vector{String}, Nothing}
steering_mode::Union{QueueMode, Nothing}
follow_up_mode::Union{QueueMode, Nothing}
tool_context::Union{AgentHarnessToolContextSource{TC}, Nothing}
end
```
**Purpose**: Configure AgentHarness with all necessary options
**Key fields**:
- `session`: Session instance for persistence
- `models`: Available models
- `tools`: Agent tools
- `resources`: Skills and prompt templates
- `system_prompt`: System prompt (string or function)
- `stream_options`: LLM streaming options
- `model`: Default model
- `thinking_level`: Default thinking level
- `active_tool_names`: Active tools
- `tool_context`: Context source for tools
### 2. AgentHarnessResources
```julia
mutable struct AgentHarnessResources{TSkill<:Skill, TPromptTemplate<:PromptTemplate}
promptTemplates::Union{Vector{TPromptTemplate}, Nothing}
skills::Union{Vector{TSkill}, Nothing}
end
```
**Purpose**: Load and manage skills and prompt templates
### 3. Skill
```julia
mutable struct Skill
name::String
description::String
content::String
filePath::String
disableModelInvocation::Bool
end
```
**Purpose**: Define specialized instructions for specific tasks
**Format**:
```markdown
<!-- SKILL.md -->
{
"name": "File Operations",
"description": "Handle file system operations",
"disable-model-invocation": false
}
---
# File Operations Skill
This skill provides instructions for working with files...
```
### 4. PromptTemplate
```julia
mutable struct PromptTemplate
name::String
description::Union{String, Nothing}
content::String
end
```
**Purpose**: Reusable prompt snippets with arguments
**Format**:
```markdown
<!-- template.md -->
{
"description": "Generate commit message"
}
---
Generate a git commit message for:
$1
$ARGUMENTS
```
### 5. AgentHarnessStreamOptions
```julia
mutable struct AgentHarnessStreamOptions
transport::Union{String, Nothing}
timeout_ms::Union{Int64, Nothing}
max_retries::Union{Int64, Nothing}
max_retry_delay_ms::Union{Int64, Nothing}
headers::Union{Dict{String, String}, Nothing}
metadata::Union{Dict{String, Any}, Nothing}
cache_retention::Union{String, Nothing}
end
```
**Purpose**: Configure LLM API call options
## SessionRepo Interface
```julia
abstract type SessionRepo<
TMetadata<:SessionMetadata,
TCreateOptions,
TListOptions
> end
```
### Repo Methods
```julia
# Create new session
create(repo::SessionRepo, options::TCreateOptions)::Promise{Session}
# Open existing session
open(repo::SessionRepo, metadata::TMetadata)::Promise{Session}
# List sessions
list(repo::SessionRepo, options::TListOptions)::Promise{Vector{TMetadata}}
# Delete session
delete(repo::SessionRepo, metadata::TMetadata)::Promise{Nothing}
# Fork session (create branch)
fork(repo::SessionRepo, source::TMetadata, options::Dict{String, Any})::Promise{Session}
```
### JsonlSessionRepo
```julia
# JSONL-based session repository
# - Sessions stored as JSONL files
# - Supports create, open, list, delete, fork
# - Branch navigation via session tree
```
## Extension Hooks
### Hook Types
```julia
# Before agent starts
BeforeAgentStartEvent
├─ prompt: String
├─ images: Union{Vector{ImageContent}, Nothing}
├─ system_prompt: String
└─ resources: AgentHarnessResources
BeforeAgentStartResult
├─ messages: Union{Vector{AgentMessage}, Nothing}
└─ system_prompt: Union{String, Nothing}
# Context event
ContextEvent
└─ messages: Vector{AgentMessage}
ContextResult
└─ messages: Vector{AgentMessage}
# Before LLM request
BeforeProviderRequestEvent
├─ model: Model
├─ session_id: String
└─ stream_options: AgentHarnessStreamOptions
BeforeProviderRequestResult
└─ stream_options: Union{AgentHarnessStreamOptionsPatch, Nothing}
# Before LLM payload
BeforeProviderPayloadEvent
├─ model: Model
└─ payload: Any
BeforeProviderPayloadResult
└─ payload: Any
# After LLM response
AfterProviderResponseEvent
├─ status: Int64
└─ headers: Dict{String, String}
# Tool call
ToolCallEvent
├─ tool_call_id: String
├─ tool_name: String
└─ input: Dict{String, Any}
ToolCallResult
├─ block: Union{Bool, Nothing}
└─ reason: Union{String, Nothing}
# Tool result
ToolResultEvent
├─ tool_call_id: String
├─ tool_name: String
├─ input: Dict{String, Any}
├─ content: Vector{MessageContent}
├─ details: Any
├─ is_error: Bool
└─ usage: Union{Usage, Nothing}
ToolResultPatch
├─ content: Union{Vector{MessageContent}, Nothing}
├─ details: Union{Any, Nothing}
├─ is_error: Union{Bool, Nothing}
├─ usage: Union{Usage, Nothing}
└─ terminate: Union{Bool, Nothing}
# Session compaction
SessionBeforeCompactEvent
├─ preparation: Any
├─ branch_entries: Vector{SessionTreeEntry}
├─ custom_instructions: Union{String, Nothing}
└─ signal: Any
SessionBeforeCompactResult
├─ cancel: Union{Bool, Nothing}
└─ compaction: Union{CompactResult, Nothing}
SessionCompactEvent
├─ compaction_entry: CompactionEntry
└─ from_hook: Bool
# Session tree (branching)
SessionBeforeTreeEvent
├─ preparation: Any
└─ signal: Any
SessionBeforeTreeResult
├─ cancel: Union{Bool, Nothing}
├─ summary: Union{Dict{String, Any}, Nothing}
├─ custom_instructions: Union{String, Nothing}
├─ replace_instructions: Union{Bool, Nothing}
└─ label: Union{String, Nothing}
SessionTreeEvent
├─ new_leaf_id: Union{String, Nothing}
├─ old_leaf_id: Union{String, Nothing}
├─ summary_entry: Union{BranchSummaryEntry, Nothing}
└─ from_hook: Union{Bool, Nothing}
```
### Hook Usage Examples
#### BeforeAgentStartHook
```julia
function beforeAgentStart(event, signal)
# Modify system prompt based on context
new_system_prompt = "$(event.system_prompt)\n\nUser prefers concise responses."
# Prepend initial messages
initial_messages = [
UserMessage("user", [TextContent("Context: $(event.prompt)")], timestamp),
]
return BeforeAgentStartResult(
initial_messages,
new_system_prompt,
)
end
# Configure harness
harness = AgentHarness(Dict(
:beforeAgentStart => beforeAgentStart,
))
```
#### BeforeProviderPayloadHook
```julia
function beforeProviderPayload(event, signal)
# Modify LLM payload before sending
payload = event.payload
# Add custom metadata
payload.metadata = merge(payload.metadata, Dict(
"session_id" => event.session_id,
"timestamp" => Dates.now(),
))
return BeforeProviderPayloadResult(payload)
end
```
#### ToolCallHook
```julia
function toolCall(event, signal)
# Block dangerous tool calls
if event.tool_name == "bash" && contains(event.input["command"], "rm -rf /")
return ToolCallResult(true, "Blocking dangerous command")
end
# Log tool execution
println("Tool call: $(event.tool_name)")
return nothing # Allow execution
end
```
#### BeforeCompactHook
```julia
function beforeCompact(event, signal)
# Add custom instructions for compaction
custom_instructions = """
Focus on retaining user preferences and key decisions.
Omit verbose tool outputs that don't add value.
"""
return SessionBeforeCompactResult(
false, # Don't cancel
Dict(
"summary" => "Custom compaction with focus on user intent",
"custom_instructions" => custom_instructions,
),
)
end
```
## Tool Context
### AgentHarnessToolContextSource
```julia
mutable struct AgentHarnessToolContextSource{TContext}
context::Union{TContext, Function}
end
```
**Purpose**: Provide context to tools during execution
### Tool Execution Context
```julia
# Tools receive context from AgentHarness
tool.execute(
tool_call_id,
params,
signal,
on_update,
context, # From AgentHarnessToolContextSource
)
# Context can be:
# - Static value
# - Function that returns value
```
## Complete Example
```julia
using AgentCore
# 1. Create skills
skills, skill_diagnostics = loadSkills(
execution_env,
"/path/to/skills",
)
# 2. Create prompt templates
templates, template_diagnostics = loadPromptTemplates(
execution_env,
"/path/to/templates",
)
# 3. Create resources
resources = AgentHarnessResources(
templates,
skills,
)
# 4. Create session repo
repo = JsonlSessionRepo(
"/path/to/sessions",
)
# 5. Create session
session = create(repo, Dict(
"cwd" => "/path/to/project",
"metadata" => Dict("project" => "my-project"),
))
# 6. Configure tools
bash_tool = createBashTool()
read_tool = createReadTool()
tools = [bash_tool, read_tool]
# 7. Configure hooks
hooks = Dict(
:beforeAgentStart => beforeAgentStartHook,
:beforeProviderPayload => beforePayloadHook,
:toolCall => toolCallHook,
)
# 8. Create harness
harness = AgentHarness(Dict(
:session => session,
:models => models,
:tools => tools,
:resources => resources,
:system_prompt => "You are a helpful assistant.",
:model => Model(...),
:thinking_level => THINKING_MEDIUM,
:active_tool_names => ["bash", "read"],
:steering_mode => QUEUE_ONE_AT_A_TIME,
:follow_up_mode => QUEUE_ONE_AT_A_TIME,
:tool_context => AgentHarnessToolContextSource(context),
:stream_options => AgentHarnessStreamOptions(
transport = "auto",
timeout_ms = 30000,
max_retries = 3,
),
))
# 9. Subscribe to events
subscribe(harness) do event, signal
if event isa BeforeAgentStartEvent
println("Agent starting...")
elseif event isa MessageEndEvent
println("Message: $(event.message)")
end
end
# 10. Run conversation
harness.prompt("What files are in the current directory?")
# 11. Wait for completion
wait_for_idle(harness)
# 12. Manage branches
session.moveTo(some_entry_id) # Fork from entry
```
## Hook Execution Flow
```
User Code
├─► AgentHarness.prompt()
┌────────────────────────────────────────────────────────────────────────┐
│ BeforeAgentStartEvent │
│ ├─ User prompt │
│ ├─ System prompt │
│ └─ Resources │
│ │ │
│ └─► beforeAgentStart hook (optional) │
│ └─► BeforeAgentStartResult (optional modifications) │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ Agent.createLoopConfig() │
│ └─► Merge options with hooks │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ Agent.prompt() │
│ └─► Start AgentLoop │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ AgentLoop.agentLoop() │
│ │ │
│ ├─► transform_context hook (optional) │
│ └─► convert_to_llm() │
│ └─► Message[] for LLM API │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ BeforeProviderRequestEvent │
│ ├─ Model │
│ ├─ Session ID │
│ └─ Stream Options │
│ │ │
│ └─► beforeProviderRequest hook (optional) │
│ └─► BeforeProviderRequestResult (optional modifications) │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ StreamFn (LLM API call) │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ AfterProviderResponseEvent │
│ ├─ Status code │
│ └─ Response headers │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ BeforeProviderPayloadEvent │
│ ├─ Model │
│ └─ Payload (before sending) │
│ │ │
│ └─► beforeProviderPayload hook (optional) │
│ └─► BeforeProviderPayloadResult (optional modifications) │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ LLM API Request │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ Assistant Message (streaming) │
│ │ │
│ ├─► Text deltas │
│ └─► Tool calls │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ Tool Execution (for each tool call) │
│ │ │
│ ├─► before_tool_call hook (Agent) │
│ ├─► toolCall hook (Harness - optional) │
│ │ └─► ToolCallResult (can block execution) │
│ ├─► prepareToolCall() │
│ ├─► execute() │
│ │ └─► Tool execution with context │
│ ├─► after_tool_call hook (Agent) │
│ └─► toolResult hook (Harness - optional) │
│ └─► ToolResultPatch (can modify result) │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ AgentLoop continues with tool results │
│ │ │
│ ├─► Next LLM call with tool results │
│ └─► Or end of conversation │
└────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────┐
│ AgentEndEvent │
│ └─► Final messages in session │
└────────────────────────────────────────────────────────────────────────┘
```
## Session Management with Harness
```julia
# Create harness with session repo
repo = JsonlSessionRepo("/path/to/sessions")
# Create session
session = create(repo, Dict(
"cwd" => "/path/to/project",
"metadata" => Dict("name" => "my-session"),
))
# Or open existing session
metadata = JsonlSessionMetadata(...)
session = open(repo, metadata)
# List sessions
sessions = list(repo, Dict())
for meta in sessions
println("Session: $(meta.id)")
end
# Delete session
delete(repo, metadata)
# Fork session (branch)
forked_session = fork(repo, source_metadata, Dict(
"summary" => "Branch for feature X",
))
```
## Resources Management
```julia
# Load skills from directory
skills, diagnostics = loadSkills(
execution_env,
"/path/to/skills",
)
# Load prompt templates from directory
templates, diagnostics = loadPromptTemplates(
execution_env,
"/path/to/templates",
)
# Create resources
resources = AgentHarnessResources(
templates,
skills,
)
# Use in harness
harness = AgentHarness(Dict(
:resources => resources,
))
```
## Best Practices
1. **Use hooks for logging and validation**
- `beforeAgentStart` for initialization
- `beforeProviderPayload` for custom metadata
- `toolCall` for blocking dangerous operations
2. **Organize skills by domain**
- File operations
- Database queries
- HTTP requests
- Git operations
3. **Use templates for common patterns**
- Commit message generation
- Code review instructions
- Testing prompts
4. **Manage sessions carefully**
- Compact periodically
- Use branches for exploration
- Clean up old sessions
5. **Monitor resource usage**
- Track token counts
- Watch API costs
- Optimize tool execution
## Troubleshooting
### Hook not being called
```julia
# Check hook is registered
if isnothing(harness.beforeAgentStart)
println("Hook not registered")
end
```
### Session not persisting
```julia
# Check repo is configured
if isnothing(harness.repo)
println("No repo configured")
end
```
### Resources not loading
```julia
# Check diagnostics
for diag in skill_diagnostics
println("Skill warning: $(diag.message)")
end
```