YiemAgent/learning/07-AGENTHARNESS.md

# AgentCore.jl - AgentHarness Design Reference

## Status

> **Note**: The AgentHarness module (`src/agent_harness.jl`) is **not yet implemented**. This document
> describes the intended design based on types defined in `src/harness_types.jl`. The types, events,
> and interfaces below are defined but the harness that connects them is a planned feature.
>
> Several modules referenced in `src/AgentCore.jl` are also not yet implemented:
> `compaction/compaction.jl`, `compaction/utils.jl`, `compaction/branch_summarization.jl`,
> `utils/truncate.jl`, `utils/shell_output.jl`, `proxy.jl`.
>
> Type placeholders not yet defined: `AgentLoopConfig`, `Promise`, `AbortSignal`, `EventStream`,
> `Context`. The `SessionRepo` methods in `harness_types.jl` return `Promise()` stubs.

## AgentHarness Architecture (Planned)

```
AgentHarness = Agent + Session + Resources + Hooks

  AgentHarness (to be implemented in src/agent_harness.jl)
  ├── Manages Agent instances
  ├── Provides Session persistence via SessionRepo
  ├── Manages resources (skills, prompt templates)
  ├── Handles extension hooks (BeforeAgentStart, BeforeProviderPayload, etc.)
  └── Coordinates tool execution with AgentHarnessToolContextSource

  AgentHarnessOptions (src/harness_types.jl:1067)
  ├── session::Session
  ├── models::Any
  ├── tools::Union{Vector{TTool}, Nothing}
  ├── resources::Union{AgentHarnessResources, Nothing}
  ├── system_prompt::Union{AgentHarnessSystemPrompt, Nothing}
  ├── stream_options::Union{AgentHarnessStreamOptions, Nothing}
  ├── retry::Union{Any, Nothing}
  ├── model::Model
  ├── thinking_level::Union{ThinkingLevel, Nothing}
  ├── active_tool_names::Union{Vector{String}, Nothing}
  ├── steering_mode::Union{QueueMode, Nothing}
  ├── follow_up_mode::Union{QueueMode, Nothing}
  └── tool_context::Union{AgentHarnessToolContextSource, Nothing}
```

## Event Type Hierarchy (Actual)

The harness event types are defined as `mutable struct` in `harness_types.jl`.
They are NOT subtypes of `AgentHarnessEvent` or `AgentHarnessOwnEvent` - those
abstract types exist but nothing inherits from them.

```
AgentEvent (abstract, types.jl:196)
├── AgentStartEvent (types.jl:198)
├── AgentEndEvent (types.jl:199)
├── TurnStartEvent (types.jl:202)
├── TurnEndEvent (types.jl:203)
├── MessageStartEvent (types.jl:207)
├── MessageUpdateEvent (types.jl:210)
├── MessageEndEvent (types.jl:214)
├── ToolExecutionStartEvent (types.jl:217)
├── ToolExecutionUpdateEvent (types.jl:222)
└── ToolExecutionEndEvent (types.jl:228)

AgentHarnessOwnEvent (abstract, harness_types.jl:850)
  └── (nothing inherits from this)

AgentHarnessEvent (abstract, harness_types.jl:856)
  └── (nothing inherits from this)

Harness event structs (harness_types.jl) - mutable structs, not subtypes:
├── BeforeAgentStartEvent (line 653)
├── ContextEvent (line 665)
├── BeforeProviderRequestEvent (line 674)
├── BeforeProviderPayloadEvent (line 685)
├── AfterProviderResponseEvent (line 695)
├── ToolCallEvent (line 705)
├── ToolResultEvent (line 716)
├── SessionBeforeCompactEvent (line 731)
├── SessionCompactEvent (line 743)
├── SessionBeforeTreeEvent (line 753)
├── SessionTreeEvent (line 763)
├── RetryScheduledEvent (line 775)
├── RetryAttemptStartEvent (line 788)
├── RetryFinishedEvent (line 797)
├── ModelUpdateEvent (line 806)
├── ThinkingLevelUpdateEvent (line 817)
├── ToolsUpdateEvent (line 827)
└── ResourcesUpdateEvent (line 840)
```

## Types (from harness_types.jl)

### AgentHarnessOptions (line 1067)

```julia
mutable struct AgentHarnessOptions{TC<:Any, TSkill<:Skill, TPromptTemplate<:PromptTemplate, TTool<:AgentHarnessTool}
    session::Session
    models::Any
    tools::Union{Vector{TTool}, Nothing}
    resources::Union{AgentHarnessResources{TSkill, TPromptTemplate}, Nothing}
    system_prompt::Union{AgentHarnessSystemPrompt{TC, TSkill, TPromptTemplate, TTool}, Nothing}
    stream_options::Union{AgentHarnessStreamOptions, Nothing}
    retry::Union{Any, Nothing}
    model::Model
    thinking_level::Union{ThinkingLevel, Nothing}
    active_tool_names::Union{Vector{String}, Nothing}
    steering_mode::Union{QueueMode, Nothing}
    follow_up_mode::Union{QueueMode, Nothing}
    tool_context::Union{AgentHarnessToolContextSource{TC}, Nothing}
end
```

**Purpose**: Configure AgentHarness with all necessary options (defined but harness not implemented).

### AgentHarnessResources (line 82)

```julia
mutable struct AgentHarnessResources{TSkill<:Skill, TPromptTemplate<:PromptTemplate}
    promptTemplates::Union{Vector{TPromptTemplate}, Nothing}
    skills::Union{Vector{TSkill}, Nothing}
end
```

### Skill (line 68)

```julia
mutable struct Skill
    name::String
    description::String
    content::String
    filePath::String
    disableModelInvocation::Bool
end
```

**Loading**: `loadSkills(env, dir)` is defined in `skills.jl` but **parsing is stubbed** - currently returns `nothing, diagnostics`. The frontmatter parsing code (lines 266-300 of skills.jl) is commented out as TODO.

**Skill format**:
```markdown
<!-- SKILL.md -->
{
  "name": "File Operations",
  "description": "Handle file system operations",
  "disable-model-invocation": false
}
---

# File Operations Skill

This skill provides instructions for working with files...
```

### PromptTemplate (line 76)

```julia
mutable struct PromptTemplate
    name::String
    description::Union{String, Nothing}
    content::String
end
```

**Loading**: `loadPromptTemplates(env, paths)` is defined in `prompt_templates.jl` but **parsing is stubbed** - currently returns `nothing, diagnostics`. Frontmatter parsing is commented out as TODO (lines 188-215).

**Format**:
```markdown
<!-- template.md -->
{
  "description": "Generate commit message"
}
---

Generate a git commit message for:
$1
$ARGUMENTS
```

### AgentHarnessStreamOptions (line 109)

```julia
mutable struct AgentHarnessStreamOptions
    transport::Union{String, Nothing}
    timeout_ms::Union{Int64, Nothing}
    max_retries::Union{Int64, Nothing}
    max_retry_delay_ms::Union{Int64, Nothing}
    headers::Union{Dict{String, String}, Nothing}
    metadata::Union{Dict{String, Any}, Nothing}
    cache_retention::Union{String, Nothing}
end
```

### AgentHarnessStreamOptionsPatch (line 119)

```julia
mutable struct AgentHarnessStreamOptionsPatch
    transport::Union{String, Nothing}
    timeout_ms::Union{Int64, Nothing}
    max_retries::Union{Int64, Nothing}
    max_retry_delay_ms::Union{Int64, Nothing}
    cache_retention::Union{String, Nothing}
    headers::Union{Dict{String, String}, Nothing}
    metadata::Union{Dict{String, Any}, Nothing}
end
```

### AgentHarnessTool (line 91)

```julia
mutable struct AgentHarnessTool{TContext, TParameters, TDetails}
    name::String
    label::String
    description::String
    parameters::TParameters
    execute::Function
    prepareArguments::Union{Function, Nothing}
    executionMode::Union{ToolExecutionMode, Nothing}
end
```

### AgentHarnessToolContextSource (line 101)

```julia
mutable struct AgentHarnessToolContextSource{TContext}
    context::Union{TContext, Function}
end
```

### AgentHarnessSystemPrompt (line 1059)

```julia
mutable struct AgentHarnessSystemPrompt{TC<:Any, TSkill<:Skill, TPromptTemplate<:PromptTemplate, TTool<:AgentHarnessTool}
    value::Union{String, Function}
end
```

## SessionRepo Interface (stubs in harness_types.jl:564-588)

```julia
abstract type SessionRepo<
    TMetadata<:SessionMetadata,
    TCreateOptions,
    TListOptions
> end

function create(repo::SessionRepo, options::TCreateOptions)::Promise{Session}
    return Promise()  # STUB - Promise type not defined
end

function open(repo::SessionRepo, metadata::TMetadata)::Promise{Session}
    return Promise()  # STUB
end

function list(repo::SessionRepo, options::TListOptions)::Promise{Vector{TMetadata}}
    return Promise()  # STUB
end

function delete(repo::SessionRepo, metadata::TMetadata)::Promise{Nothing}
    return Promise()  # STUB
end

function fork(repo::SessionRepo, source::TMetadata, options::Dict{String, Any})::Promise{Session}
    return Promise()  # STUB
end
```

> **Note**: These methods are stubs in `harness_types.jl`. `Promise` is not defined anywhere.

### JsonlSessionRepo (src/session/jsonl_repo.jl)

```julia
mutable struct JsonlSessionRepo <: SessionRepo{
    JsonlSessionMetadata,
    JsonlSessionCreateOptions,
    JsonlSessionListOptions
}
    fs::Any
    sessions_root_input::String
    sessions_root::Union{String, Nothing}

    function JsonlSessionRepo(; sessions_root::String, fs::Any)
        new(fs, sessions_root, nothing)
    end
end
```

> **Note**: Constructor uses **keyword arguments** (`sessions_root=`, `fs=`), NOT positional.

### JsonlSessionStorage (src/session/jsonl_storage.jl)

- `file_path::String`
- `metadata::T` (SessionMetadata)
- `entries::Vector{SessionTreeEntry}`
- `by_id::Dict{String, SessionTreeEntry}`
- `labels_by_id::Dict{String, String}`
- `current_leaf_id::Union{String, Nothing}`

Methods: `getMetadata`, `getLeafId`, `setLeafId`, `createEntryId`, `appendEntry`, `getEntry`, `findEntries`, `getLabel`, `getSessionName`, `getSessionStats`, `getPathToRootOrCompaction`, `getEntries`.

## Agent (src/agent.jl)

```julia
mutable struct Agent
    _state::AgentState
    listeners::Set{Tuple{Function, Ref{Bool}}}
    steering_queue::PendingMessageQueue
    follow_up_queue::PendingMessageQueue
    convert_to_llm::Function
    transform_context::Union{Function, Nothing}
    stream_function::StreamFn
    get_api_key::Union{Function, Nothing}
    on_payload::Union{Function, Nothing}
    on_response::Union{Function, Nothing}
    before_tool_call::Union{Function, Nothing}
    after_tool_call::Union{Function, Nothing}
    prepare_next_turn::Union{Function, Nothing}
    prepare_next_turn_with_context::Union{Function, Nothing}
    active_run::Union{ActiveRun, Nothing}
    session_id::Union{String, Nothing}
    thinking_budgets::Union{Dict{String, Int64}, Nothing}
    transport::String
    max_retry_delay_ms::Union{Int64, Nothing}
    tool_execution::ToolExecutionMode
end
```

Key methods:
- `subscribe(agent, listener)` - subscribe to events, returns unsubscribe function
- `get_state(agent)` - get current AgentState
- `steer(agent, message)` - queue message for injection after current turn
- `followUp(agent, message)` - queue message to run after agent would stop
- `prompt(agent, input, images)` - start a new prompt (input can be String, AgentMessage, or Vector{AgentMessage})
- `continue!(agent)` - continue from current transcript
- `waitForIdle(agent)` - resolve when current run finishes
- `abort(agent)` - abort current run (partially implemented)
- `reset!(agent)` - clear all state
- `clearSteeringQueue(agent)` / `clearFollowUpQueue(agent)` / `clearAllQueues(agent)`
- `hasQueuedMessages(agent)` - check for pending messages
- `createContextSnapshot(agent)` - create AgentContext snapshot
- `createLoopConfig(agent, options)` - create AgentLoopConfig

> **Note**: `AgentLoopConfig` type is **not defined** in any visible file. It is referenced in `agent.jl:368` and `agent_loop.jl`.

## AgentLoop (src/agent_loop.jl)

Key functions:
- `agentLoop(prompts, context, config, signal, stream_fn)` - main loop, returns EventStream
- `agentLoopContinue(context, config, signal, stream_fn)` - continue from existing context
- `runAgentLoop(...)` - internal run, emits events via `emit::AgentEventSink`
- `runAgentLoopContinue(...)` - internal continue run
- `runLoop(...)` - shared main loop logic
- `streamAssistantResponse(...)` - stream LLM response with event emission
- `executeToolCalls(...)` - execute tool calls (sequential or parallel)
- `executeToolCallsSequential(...)` - sequential execution
- `executeToolCallsParallel(...)` - parallel execution via Threads.@spawn

The loop flow:
1. `AgentStartEvent` emitted
2. `TurnStartEvent` emitted (first turn only from agentLoop, not from runLoop)
3. Steering messages drained and emitted as `MessageStartEvent`/`MessageEndEvent`
4. `streamAssistantResponse` called - transforms context, converts to LLM messages, calls stream_fn
5. For each tool call in response: execute sequentially or in parallel
6. `TurnEndEvent` emitted with message and tool results
7. `prepare_next_turn` hook (if configured) called
8. If `should_stop_after_turn` returns true or no pending messages, `AgentEndEvent` emitted
9. Follow-up messages drained and loop repeats

### AgentLoopConfig fields (referenced, not defined)

Created in `agent.jl:368-401`:
```
model, reasoning (thinking_level), session_id, on_payload, on_response,
transport, thinking_budgets, max_retry_delay_ms, tool_execution,
before_tool_call, after_tool_call, prepare_next_turn, convert_to_llm,
transform_context, get_api_key, get_steering_messages, get_follow_up_messages
```

## Hook System (Planned - Harness Not Implemented)

The following hook types are defined as event/result structs in `harness_types.jl`
but **no harness implementation exists to trigger or handle them**. These are
intended to be used by the future AgentHarness module.

### BeforeAgentStartEvent (line 653)
```julia
mutable struct BeforeAgentStartEvent{TSkill, TPromptTemplate}
    type::String
    prompt::String
    images::Union{Vector{ImageContent}, Nothing}
    system_prompt::String
    resources::AgentHarnessResources{TSkill, TPromptTemplate}
end
```
**Result**: `BeforeAgentStartResult` (line 862) - `messages::Union{Vector{AgentMessage}, Nothing}`, `system_prompt::Union{String, Nothing}`

### ContextEvent (line 665)
```julia
mutable struct ContextEvent
    type::String
    messages::Vector{AgentMessage}
end
```
**Result**: `ContextResult` (line 871) - `messages::Vector{AgentMessage}`

### BeforeProviderRequestEvent (line 674)
```julia
mutable struct BeforeProviderRequestEvent
    type::String
    model::Model
    session_id::String
    stream_options::AgentHarnessStreamOptions
end
```
**Result**: `BeforeProviderRequestResult` (line 879) - `stream_options::Union{AgentHarnessStreamOptionsPatch, Nothing}`

### BeforeProviderPayloadEvent (line 685)
```julia
mutable struct BeforeProviderPayloadEvent
    type::String
    model::Model
    payload::Any
end
```
**Result**: `BeforeProviderPayloadResult` (line 887) - `payload::Any`

### AfterProviderResponseEvent (line 695)
```julia
mutable struct AfterProviderResponseEvent
    type::String
    status::Int64
    headers::Dict{String, String}
end
```

### ToolCallEvent (line 705)
```julia
mutable struct ToolCallEvent
    type::String
    tool_call_id::String
    tool_name::String
    input::Dict{String, Any}
end
```
**Result**: `ToolCallResult` (line 895) - `block::Union{Bool, Nothing}`, `reason::Union{String, Nothing}`

### ToolResultEvent (line 716)
```julia
mutable struct ToolResultEvent
    type::String
    tool_call_id::String
    tool_name::String
    input::Dict{String, Any}
    content::Vector{MessageContent}
    details::Any
    is_error::Bool
    usage::Union{Usage, Nothing}
end
```
**Result**: `ToolResultPatch` (line 904) - `content`, `details`, `is_error`, `usage`, `terminate` (all Union{...})

### SessionBeforeCompactEvent (line 731)
```julia
mutable struct SessionBeforeCompactEvent
    type::String
    preparation::Any
    branch_entries::Vector{SessionTreeEntry}
    custom_instructions::Union{String, Nothing}
    signal::Any
end
```
**Result**: `SessionBeforeCompactResult` (line 916) - `cancel::Union{Bool, Nothing}`, `compaction::Union{CompactResult, Nothing}`

### SessionBeforeTreeEvent (line 753)
```julia
mutable struct SessionBeforeTreeEvent
    type::String
    preparation::Any
    signal::Any
end
```
**Result**: `SessionBeforeTreeResult` (line 925) - `cancel`, `summary`, `custom_instructions`, `replace_instructions`, `label`

### SessionCompactEvent (line 743)
```julia
mutable struct SessionCompactEvent
    type::String
    compaction_entry::CompactionEntry
    from_hook::Bool
end
```

### SessionTreeEvent (line 763)
```julia
mutable struct SessionTreeEvent
    type::String
    new_leaf_id::Union{String, Nothing}
    old_leaf_id::Union{String, Nothing}
    summary_entry::Union{BranchSummaryEntry, Nothing}
    from_hook::Union{Bool, Nothing}
end
```

## Session (src/session/session.jl)

```julia
mutable struct Session{T<:SessionMetadata}
    storage::SessionStorage{T}
    context_build_options::SessionContextBuildOptions
end
```

Key methods:
- `getMetadata(session)` / `getStorage(session)` / `getLeafId(session)` / `getEntry(session, id)`
- `getEntries(session, options)` / `getBranch(session, from_id)`
- `buildContextEntries(session, options)` / `buildContext(session, options)`
- `getLabel(session, id)` / `getSessionStats(session)` / `getSessionName(session)`
- `appendMessage(session, message)` → entry_id
- `appendThinkingLevelChange(session, level)` → entry_id
- `appendModelChange(session, provider, model_id)` → entry_id
- `appendActiveToolsChange(session, active_tool_names)` → entry_id
- `appendCompaction(session, summary, first_kept_entry_id, tokens_before, ...)` → entry_id
- `appendCustomEntry(session, custom_type, data)` → entry_id
- `appendCustomMessageEntry(session, custom_type, content, display, details)` → entry_id
- `appendLabel(session, target_id, label)` → entry_id
- `appendSessionName(session, name)` → entry_id
- `moveTo(session, entry_id, summary)` → new_leaf_id or nothing (line 392)

## Session Tree Entries (types.jl and harness_types.jl)

```julia
abstract type SessionTreeEntry end

struct MessageEntry <: SessionTreeEntry
    base::SessionTreeEntryBase  # or direct fields in harness_types.jl
    message::AgentMessage
end

struct ThinkingLevelChangeEntry <: SessionTreeEntry
    base::SessionTreeEntryBase
    thinking_level::String
end

struct ModelChangeEntry <: SessionTreeEntry
    base::SessionTreeEntryBase
    provider::String
    model_id::String
end

struct ActiveToolsChangeEntry <: SessionTreeEntry
    base::SessionTreeEntryBase
    active_tool_names::Vector{String}
end

struct CompactionEntry{T} <: SessionTreeEntry
    base::SessionTreeEntryBase
    summary::String
    first_kept_entry_id::Union{String, Nothing}
    tokens_before::Int64
    retained_tail::Union{Vector{AgentMessage}, Nothing}
    details::Union{T, Nothing}
    usage::Union{Usage, Nothing}
    from_hook::Bool
end

struct BranchSummaryEntry{T} <: SessionTreeEntry
    base::SessionTreeEntryBase
    from_id::String
    summary::String
    details::Union{T, Nothing}
    usage::Union{Usage, Nothing}
    from_hook::Bool
end

struct CustomEntry{T} <: SessionTreeEntry
    base::SessionTreeEntryBase
    custom_type::String
    data::Union{T, Nothing}
end

struct CustomMessageEntry{T} <: SessionTreeEntry
    base::SessionTreeEntryBase
    custom_type::String
    content::String
    details::Union{T, Nothing}
    display::Bool
end

struct LabelEntry <: SessionTreeEntry
    base::SessionTreeEntryBase
    target_id::String
    label::Union{String, Nothing}
end

struct SessionInfoEntry <: SessionTreeEntry
    base::SessionTreeEntryBase
    name::Union{String, Nothing}
end

struct LeafEntry <: SessionTreeEntry
    base::SessionTreeEntryBase
    target_id::Union{String, Nothing}
end
```

## Resource Loading (stubs)

### loadSkills (skills.jl:61)

```julia
skills, diagnostics = loadSkills(env, "/path/to/skills")
```

> **Note**: Parsing is **stubbed** (line 302 returns `nothing, diagnostics`). The frontmatter parsing code is commented out (lines 266-300). `formatSkillInvocation(skill, additional_instructions)` is implemented.

### loadPromptTemplates (prompt_templates.jl:43)

```julia
templates, diagnostics = loadPromptTemplates(env, "/path/to/templates")
```

> **Note**: Parsing is **stubbed** (line 217 returns `nothing, diagnostics`). The frontmatter parsing code is commented out (lines 188-215). `formatPromptTemplateInvocation(template, args)` and `parseCommandArgs(args_string)` and `substituteArgs(content, args)` are implemented.

## Missing Types / Modules

The following types are referenced in the code but **not defined**:
- `AgentLoopConfig` - referenced in `agent.jl:368`, `agent_loop.jl`
- `Promise` - referenced in `harness_types.jl`
- `AbortSignal` - referenced in `agent_loop.jl`
- `EventStream` - referenced in `agent_loop.jl:158`
- `Context` - referenced in `agent_loop.jl:376`
- `AgentToolResultMutable` - referenced in `agent_loop.jl`
- `FinalizedToolCallOutcome`, `PreparedToolCall`, `ImmediateToolCallOutcome`, `ExecutedToolCallOutcome` - defined in `agent_loop.jl:639-661` (these exist)

The following modules are referenced in `AgentCore.jl` but **files don't exist**:
- `compaction/compaction.jl`
- `compaction/utils.jl`
- `compaction/branch_summarization.jl`
- `utils/truncate.jl`
- `utils/shell_output.jl`
- `proxy.jl`

## AgentCore Exports (from AgentCore.jl:61-147)

The module exports: AgentMessage, AgentTool, AgentContext, AgentEvent, ThinkingLevel, ToolExecutionMode, QueueMode, AgentState, Agent, AgentOptions, AgentLoopConfig, agentLoop, agentLoopContinue, runAgentLoop, runAgentLoopContinue, AgentHarness, AgentHarnessOptions, AgentHarnessEvent, AgentHarnessResources, AgentHarnessSystemPrompt, Session, SessionStorage, SessionRepo, JsonlSessionStorage, JsonlSessionRepo, InMemorySessionStorage, InMemorySessionRepo, createBashTool, createReadTool, createWriteTool, createEditTool, ExecutionEnv, compact, prepareCompaction, DEFAULT_COMPACTION_SETTINGS, generateSummary, generateBranchSummary, truncateHead, truncateTail, formatSize, DEFAULT_MAX_LINES, DEFAULT_MAX_BYTES, convertToLlm, bashExecutionToText, formatSkillsForSystemPrompt, loadSkills, formatSkillInvocation, loadPromptTemplates, formatPromptTemplateInvocation, parseCommandArgs, substituteArgs, streamProxy, ProxyStreamOptions, setDefaultStreamFn, getDefaultStreamFn, uuidv7, create_timestamp.

## Best Practices

1. **Agent hooks** (planned): Use `beforeAgentStart` for initialization, `beforeProviderPayload` for custom metadata, `toolCall` for blocking dangerous operations
2. **Skills**: Organize by domain (file operations, database queries, HTTP requests, git operations)
3. **Templates**: Use for common patterns (commit messages, code review, testing prompts)
4. **Sessions**: Compact periodically, use branches for exploration, clean up old sessions
5. **Monitoring**: Track token counts, watch API costs, optimize tool execution
6. **Tool execution**: Choose between `EXECUTION_SEQUENTIAL` and `EXECUTION_PARALLEL` based on tool dependencies