pi_harness/packages/agent/learning/06-AGENTHARNESS-REFERENCE.md

AgentHarness Reference

Overview

AgentHarness is the high-level API that wraps the core agent with session management, persistence, branching, and tool context binding.

---

Key Differences: Agent vs AgentHarness

Feature	Agent (Core)	AgentHarness
Session Persistence	No	Yes (JSONL/Memory)
Branching	No	Yes
Context Compaction	No	Yes
Tool Context	Manual	Automatic binding
Skills/Templates	Manual	Built-in
State Management	Manual	Automatic
Event Hooks	Basic	Rich system

---

AgentHarness Architecture

┌─────────────────────────────────────────────────────────────┐
│                    AgentHarness                             │
├─────────────────────────────────────────────────────────────┤
│  State                                                      │
│  ├─ Session (persistence)                                   │
│  ├─ Model                                                   │
│  ├─ ThinkingLevel                                           │
│  ├─ Tools (Map)                                             │
│  ├─ ActiveTools (string[])                                  │
│  └─ SystemPrompt (string or function)                       │
│                                                             │
│  Queues                                                     │
│  ├─ steerQueue (messages to interrupt agent)               │
│  ├─ followUpQueue (messages after agent stops)             │
│  └─ nextTurnQueue (messages for next turn)                 │
│                                                             │
│  Hooks                                                      │
│  ├─ before_agent_start                                      │
│  ├─ context                                                 │
│  ├─ tool_call                                               │
│  ├─ tool_result                                             │
│  ├─ session_before_compact                                  │
│  ├─ session_before_tree                                     │
│  ├─ before_provider_request                                 │
│  └─ before_provider_payload                                 │
│                                                             │
│  Methods                                                    │
│  ├─ prompt() - Run new conversation                        │
│  ├─ skill() - Execute skill                                │
│  ├─ promptFromTemplate() - Run template                    │
│  ├─ steer() - Interrupt agent                              │
│  ├─ followUp() - Queue message                             │
│  ├─ compact() - Compress context                           │
│  ├─ navigateTree() - Branch session                        │
│  └─ subscribe() - Add event listener                       │
└─────────────────────────────────────────────────────────────┘

---

Core Concepts

1. Session

The session holds conversation history as a tree:

interface Session<TMetadata extends SessionMetadata = SessionMetadata> {
  readonly id: string;
  readonly storage: SessionStorage<TMetadata>;
  
  getMetadata(): Promise<TMetadata>;
  getLeafId(): Promise<string>;
  getEntry(id: string): Promise<SessionTreeEntry | undefined>;
  getBranch(): Promise<SessionTreeEntry[]>;
  buildContext(options?: SessionContextBuildOptions): Promise<SessionContext>;
  
  appendMessage(message: AgentMessage): Promise<string>;
  appendModelChange(provider: string, modelId: string): Promise<string>;
  appendThinkingLevelChange(thinkingLevel: ThinkingLevel): Promise<string>;
  appendActiveToolsChange(activeToolNames: string[]): Promise<string>;
  appendCompaction(...): Promise<string>;
  appendBranchSummary(...): Promise<string>;
  
  fork(targetId: string): Promise<Session>;
}

2. Resources

Skills and prompt templates available to the agent:

interface AgentHarnessResources<TSkill = Skill, TPromptTemplate = PromptTemplate> {
  skills?: TSkill[];
  promptTemplates?: TPromptTemplate[];
}

interface Skill {
  name: string;
  description: string;
  content: string;
  filePath: string;
  disableModelInvocation?: boolean;
}

interface PromptTemplate {
  name: string;
  description?: string;
  content: string;
}

3. Tool Context

Context passed to all tool executions:

interface ToolContext {
  userId: string;
  environment: "dev" | "staging" | "prod";
  // ... custom properties
}

// Zero-arg function for dynamic context
type ToolContextProvider<TContext> = () => TContext | Promise<TContext>;

---

AgentHarness API

Constructor

constructor(options: AgentHarnessOptions<TContext, TSkill, TPromptTemplate, TTool>)

Options:

interface AgentHarnessOptions<TContext, TSkill, TPromptTemplate, TTool> {
  session: Session;  // Session storage
  models: Models;  // LLM provider
  resources?: AgentHarnessResources<TSkill, TPromptTemplate>;
  streamOptions?: AgentHarnessStreamOptions;
  retry?: RetryPolicy;
  
  // System prompt
  systemPrompt?: 
    | string  // Static string
    | AgentHarnessSystemPrompt<TContext, TSkill, TPromptTemplate, TTool>;  // Dynamic function
  
  // Tool context
  toolContext?: AgentHarnessToolContextSource<TContext>;
  
  // Tools
  tools?: TTool[];
  
  // Active tools
  activeToolNames?: string[];
  
  // Model and thinking
  model: Model<any>;
  thinkingLevel?: ThinkingLevel;
  
  // Queue modes
  steeringMode?: QueueMode;
  followUpMode?: QueueMode;
}

Example:

const harness = new AgentHarness({
  session: memorySession,
  models: models,
  resources: {
    skills: [weatherSkill, gitSkill],
    promptTemplates: [summaryTemplate]
  },
  systemPrompt: async ({ session, model, activeTools, resources }) => {
    const sessionMetadata = await session.getMetadata();
    const toolsList = activeTools.map(t => t.name).join(", ");
    
    return `You are an AI assistant with access to tools: ${toolsList}.
    
Current session: ${sessionMetadata.id}
Date: ${new Date().toISOString()}

Available skills:
${resources.skills?.map(s => `- ${s.name}: ${s.description}`).join("\n")}
`;
  },
  toolContext: { userId: "user123", environment: "prod" },
  tools: [weatherTool, gitTool, readFileTool],
  activeToolNames: ["weather", "git"],
  model: gpt4Model,
  thinkingLevel: "medium"
});

System Prompt

Static string:

systemPrompt: "You are a helpful assistant."

Dynamic function:

systemPrompt: async ({ 
  session, 
  model, 
  thinkingLevel, 
  activeTools, 
  resources 
}) => {
  const metadata = await session.getMetadata();
  
  return `System: ${metadata.id}
Model: ${model.id}
Date: ${new Date().toISOString()}

Active tools: ${activeTools.map(t => t.name).join(", ")}
`;
};

---

Main Methods

prompt()

Run a new prompt:

async prompt(text: string, options?: { images?: ImageContent[] }): Promise<AssistantMessage>

Flow:

1. Validate harness is idle
2. Create turn state (context, tools, system prompt)
3. Emit before_agent_start hook
4. Run agent loop with prompt
5. Return assistant message

Example:

const message = await harness.prompt("What's the weather in London?");
console.log(message.content);  // Assistant response

skill()

Execute a named skill:

async skill(name: string, additionalInstructions?: string): Promise<AssistantMessage>

Example:

const message = await harness.skill("git", "Also create a PR for the changes");
// Skill content injected into prompt

promptFromTemplate()

Execute a prompt template:

async promptFromTemplate(
  name: string, 
  args: string[] = []
): Promise<AssistantMessage>

Example:

// Template: "Fix the following error: {{0}}"
const message = await harness.promptFromTemplate("fix_error", ["TypeError: x is undefined"]);

steer()

Interrupt agent mid-execution:

async steer(text: string, options?: { images?: ImageContent[] }): Promise<void>

Example:

await harness.prompt("Write a long report...");
// While agent is working...
await harness.steer("Wait, change focus to climate change");
// Agent continues with new instructions

followUp()

Queue message for after agent stops:

async followUp(text: string, options?: { images?: ImageContent[] }): Promise<void>

Example:

await harness.prompt("Analyze this data...");
// Agent finishes...
await harness.followUp("Now create a summary");
// Agent continues with summary request

nextTurn()

Queue message for next turn (doesn't interrupt current turn):

async nextTurn(text: string, options?: { images?: ImageContent[] }): Promise<void>

Difference from steer():

• steer(): Interrupts immediately
• nextTurn(): Waits for current turn to finish

compact()

Compress conversation history:

async compact(customInstructions?: string): Promise<CompactResult>

Returns:

interface CompactResult {
  summary: string;
  firstKeptEntryId?: string;
  tokensBefore: number;
  usage?: Usage;
  retainedTail?: AgentMessage[];
  details?: unknown;
}

Example:

const result = await harness.compact();
console.log(`Compressed from ${result.tokensBefore} tokens to summary`);

navigateTree()

Navigate conversation tree (branching):

async navigateTree(
  targetId: string,
  options?: {
    summarize?: boolean;
    customInstructions?: string;
    replaceInstructions?: boolean;
    label?: string;
  }
): Promise<NavigateTreeResult>

Returns:

interface NavigateTreeResult {
  cancelled: boolean;
  editorText?: string;  // If target is user message
  summaryEntry?: BranchSummaryEntry;
}

Example:

// Navigate to earlier point in conversation
const result = await harness.navigateTree("entry_abc123", { summarize: true });

// Create branch from current point
const newHarness = createNewHarness();
await newHarness.navigateTree("entry_xyz789");

---

State Management

Model

getModel(): Model<any>;

async setModel(model: Model<any>): Promise<void>;

Example:

console.log(harness.getModel().id);  // "gpt-4"

await harness.setModel(gpt4oModel);

Thinking Level

getThinkingLevel(): ThinkingLevel;

async setThinkingLevel(level: ThinkingLevel): Promise<void>;

Levels: "off", "minimal", "low", "medium", "high", "xhigh", "max"

Example:

await harness.setThinkingLevel("high");  // More reasoning for complex tasks

Tools

getTools(): TTool[];
getActiveTools(): TTool[];

async setTools(tools: TTool[], activeToolNames?: string[]): Promise<void>;
async setActiveTools(toolNames: string[]): Promise<void>;

Example:

// Add new tool
await harness.setTools([...harness.getTools(), newTool]);

// Change active tools
await harness.setActiveTools(["read", "write"]);

Resources

getResources(): AgentHarnessResources<TSkill, TPromptTemplate>;

async setResources(resources: AgentHarnessResources<TSkill, TPromptTemplate>): Promise<void>;

Example:

await harness.setResources({
  skills: [...harness.getResources().skills, newSkill]
});

---

Queue Management

Steering Queue

getSteeringMode(): QueueMode;

async setSteeringMode(mode: QueueMode): Promise<void>;

Modes:

• "all": Drain all queued messages at once
• "one-at-a-time": Drain one message at a time

Follow-up Queue

getFollowUpMode(): QueueMode;

async setFollowUpMode(mode: QueueMode): Promise<void>;

Queue Helpers

// Clear all queued messages
harness.clearAllQueues();

// Check if queues have pending messages
harness.hasQueuedMessages();  // boolean

---

Event Handling

Subscribe to All Events

subscribe(
  listener: (event: AgentHarnessEvent<TSkill, TPromptTemplate>, signal?: AbortSignal) => Promise<void> | void
): () => void;

Event types:

type AgentHarnessEvent<TSkill, TPromptTemplate> =
  // Agent events (forwarded from core agent)
  | { type: "agent_start" }
  | { type: "agent_end"; messages: AgentMessage[] }
  | { type: "turn_start" }
  | { type: "turn_end"; message: AgentMessage; toolResults: ToolResultMessage[] }
  | { type: "message_start"; message: AgentMessage }
  | { type: "message_update"; message: AgentMessage; assistantMessageEvent: AssistantMessageEvent }
  | { type: "message_end"; message: AgentMessage }
  | { type: "tool_execution_start"; toolCallId: string; toolName: string; args: any }
  | { type: "tool_execution_update"; toolCallId: string; toolName: string; args: any; partialResult: any }
  | { type: "tool_execution_end"; toolCallId: string; toolName: string; result: any; isError: boolean }
  
  // Harness-specific events
  | { type: "before_agent_start"; ... }
  | { type: "context"; messages: AgentMessage[] }
  | { type: "tool_call"; ... }
  | { type: "tool_result"; ... }
  | { type: "session_before_compact"; ... }
  | { type: "session_before_tree"; ... }
  | { type: "before_provider_request"; ... }
  | { type: "before_provider_payload"; ... }
  | { type: "after_provider_response"; ... }
  | { type: "save_point"; ... }
  | { type: "settled"; ... }
  | { type: "model_update"; ... }
  | { type: "thinking_level_update"; ... }
  | { type: "tools_update"; ... }
  | { type: "resources_update"; ... }
  | { type: "session_compact"; ... }
  | { type: "session_tree"; ... }
  | { type: "queue_update"; ... }
  | { type: "retry_scheduled"; ... }
  | { type: "retry_attempt_start"; ... }
  | { type: "retry_finished"; ... }
  | { type: "abort"; clearedSteer: UserMessage[]; clearedFollowUp: UserMessage[] };

Example:

const unsubscribe = harness.subscribe(async (event, signal) => {
  if (event.type === "message_end") {
    console.log("Message:", event.message.role);
  }
  
  if (event.type === "agent_end") {
    console.log("Conversation complete");
  }
  
  if (event.type === "tool_execution_end") {
    console.log("Tool:", event.toolName, "completed");
  }
});

Subscribe to Specific Events

on<TType extends keyof AgentHarnessEventResultMap>(
  type: TType,
  handler: (event: Extract<AgentHarnessOwnEvent, { type: TType }>) => Promise<AgentHarnessEventResultMap[TType]> | AgentHarnessEventResultMap[TType]
): () => void;

Example:

// Handle tool calls
harness.on("tool_call", async ({ toolCallId, toolName, input }) => {
  console.log(`Tool ${toolName} called with:`, input);
  return undefined;  // Allow execution
});

// Handle tool results
harness.on("tool_result", async ({ toolName, content, isError }) => {
  console.log(`Tool ${toolName} result:`, isError ? "Error" : "Success");
  return undefined;  // Use default result
});

// Modify system prompt
harness.on("before_agent_start", async ({ systemPrompt }) => {
  return {
    systemPrompt: `${systemPrompt}\n\nRemember to be concise.`
  };
});

---

Session Persistence

Append Message

async appendMessage(message: AgentMessage): Promise<void>;

Example:

// Manually add message to session
await harness.appendMessage({
  role: "user",
  content: [{ type: "text", text: "Custom message" }],
  timestamp: Date.now()
});

Flush Pending Writes

async abort(): Promise<AbortResult>

Returns:

interface AbortResult {
  clearedSteer: UserMessage[];
  clearedFollowUp: UserMessage[];
}

Example:

const result = await harness.abort();
console.log(`Cleared ${result.clearedSteer.length} steering messages`);

Wait for Idle

async waitForIdle(): Promise<void>;

Example:

await harness.prompt("Do something...");
await harness.waitForIdle();  // Wait for completion
console.log("Done");

---

Error Handling

Error Codes

type AgentHarnessErrorCode =
  | "busy"  // Agent is already processing
  | "invalid_state"  // Invalid state for operation
  | "invalid_argument"  // Invalid arguments
  | "session"  // Session error
  | "hook"  // Hook error
  | "auth"  // Authentication error
  | "compaction"  // Compaction error
  | "branch_summary"  // Branch summary error
  | "unknown";  // Unknown error

Error Handling Pattern

try {
  await harness.prompt("Do something");
} catch (error) {
  if (error instanceof AgentHarnessError) {
    switch (error.code) {
      case "busy":
        console.log("Agent busy, try again later");
        break;
      case "compaction":
        console.log("Compaction failed:", error.message);
        break;
      case "hook":
        console.log("Hook error:", error.cause?.message);
        break;
      default:
        console.log("Error:", error.message);
    }
  }
}

---

Advanced Patterns

1. Dynamic System Prompt

systemPrompt: async ({ session, model, activeTools, resources }) => {
  const metadata = await session.getMetadata();
  
  // Customize based on session type
  if (metadata.type === "coding") {
    return `You are a coding assistant. Use tools: ${activeTools.map(t => t.name).join(", ")}`;
  } else if (metadata.type === "writing") {
    return `You are a writing assistant. Focus on clarity and style.`;
  }
  
  return "You are a helpful assistant.";
}

2. Conditional Tool Activation

// Enable tools based on user request
harness.on("before_agent_start", async ({ prompt }) => {
  if (prompt.includes("weather")) {
    return {
      messages: [{ role: "user", content: [{ type: "text", text: "Enable weather tool" }] }]
    };
  }
  return undefined;
});

3. Session Branching

async function exploreAlternative(harness: AgentHarness, prompt: string): Promise<AssistantMessage> {
  // Get current leaf
  const leafId = await harness.session.getLeafId();
  
  // Create branch
  const branchSession = await harness.session.fork(leafId);
  const branchHarness = new AgentHarness({
    ...harnessOptions,
    session: branchSession
  });
  
  // Run alternative
  return await branchHarness.prompt(prompt);
}

4. Custom Compaction

harness.on("session_before_compact", async ({ preparation }) => {
  // Skip compaction for short sessions
  if (preparation.tokensBefore < 1000) {
    return { cancel: true };
  }
  
  // Provide custom summary
  return {
    compaction: {
      summary: "User asked about X, Y, Z and assistant provided guidance.",
      tokensBefore: preparation.tokensBefore,
      firstKeptEntryId: preparation.firstKeptEntry.id,
      details: { manual: true }
    }
  };
});

5. Tool Execution Logging

harness.on("tool_call", async ({ toolName, input }) => {
  console.log(`[TOOL_CALL] ${toolName}:`, JSON.stringify(input, null, 2));
  return undefined;
});

harness.on("tool_result", async ({ toolName, content, isError }) => {
  console.log(`[TOOL_RESULT] ${toolName}:`, isError ? "❌" : "✅");
  return undefined;
});

---

Summary

AgentHarness provides:

• Session persistence and tree navigation
• Built-in tool context binding
• Rich hook system for customization
• Skills and prompt templates
• Context compaction and branching

Key methods:

• prompt() - Main interaction
• steer() / followUp() - Queue management
• compact() - Context management
• navigateTree() - Branching

Key patterns:

• Dynamic system prompts
• Conditional tool activation
• Session branching for experimentation
• Hook-based customization