pi_harness/packages/agent/learning/05-TOOL-EXECUTION.md

Tool Execution Guide

Overview

Tools are how the agent interacts with the external world. They can read files, execute commands, make API calls, or perform any action.

---

Tool Definition

Basic Structure

interface AgentTool<TParameters extends TSchema, TDetails> extends Tool<TParameters> {
  label: string;  // Human-readable name for UI
  prepareArguments?: (args: unknown) => Static<TParameters>;  // Optional arg transformation
  execute(
    toolCallId: string,
    params: Static<TParameters>,
    signal?: AbortSignal,
    onUpdate?: AgentToolUpdateCallback<TDetails>
  ): Promise<AgentToolResult<TDetails>>;
}

Tool Result

interface AgentToolResult<T> {
  content: (TextContent | ImageContent)[];  // Returned to model
  details: T;  // Arbitrary data for logs/UI
  usage?: Usage;  // Tool-specific usage (not for LLM context)
  addedToolNames?: string[];  // New tools introduced
  terminate?: boolean;  // Early termination hint
}

---

Tool Execution Flow

1. LLM sends tool call
   └─► AssistantMessage with toolCall content block

2. prepareToolCall()
   ├─► Find tool by name
   ├─► prepareArguments() [optional]
   ├─► validateToolArguments()
   └─► beforeToolCall() hook
       ├─► Return {block: true} → Error tool result
       └─► Continue

3. executePreparedToolCall()
   ├─► tool.execute() with onUpdate callback
   └─► onUpdate(partialResult) → Emit tool_execution_update

4. finalizeExecutedToolCall()
   └─► afterToolCall() hook
       └─► Override result fields

5. Emit events
   ├─► tool_execution_end
   ├─► message_start (toolResult)
   └─► message_end (toolResult)

---

Built-in Tools

1. Bash Tool

Purpose: Execute shell commands.

Parameters:

interface BashToolInput {
  command: string;
}

Returns: Command output as text.

Options:

• cwd: Working directory
• timeout: Command timeout in seconds
• maxStdoutLines: Truncate stdout after N lines
• maxStderrLines: Truncate stderr after N lines

Example:

const bashTool = createBashTool({
  cwd: "/home/user/project",
  timeout: 30,
  maxStdoutLines: 1000,
  maxStderrLines: 100
});

await bashTool.execute(
  "run_123",
  { command: "ls -la" },
  undefined,
  onUpdate
);

// Result:
// {
//   content: [{ type: "text", text: "drwxr-xr-x  ... " }],
//   details: {
//     command: "ls -la",
//     cwd: "/home/user/project",
//     exitCode: 0,
//     stdout: "...",
//     stderr: ""
//   }
// }

2. Read Tool

Purpose: Read files (text or binary).

Parameters:

interface ReadToolInput {
  path: string;
  startLine?: number;  // Optional line range
  endLine?: number;
}

Returns: File contents as text or images (for image files).

Options:

• maxSize: Maximum file size in bytes
• maxLines: Maximum lines for text files
• maxTotalSize: Maximum total bytes for multiple files
• imageProcessor: Custom image handler

Example:

const readTool = createReadTool({
  maxSize: 1024 * 1024,  // 1MB
  maxLines: 5000,
  imageProcessor: async (buffer) => ({
    type: "text",
    text: `Image of ${buffer.length} bytes`
  })
});

await readTool.execute(
  "read_456",
  { path: "src/app.ts", startLine: 1, endLine: 50 },
  undefined,
  onUpdate
);

// Result:
// {
//   content: [{ type: "text", text: "import React from 'react';\n..." }],
//   details: { path: "src/app.ts", linesRead: 50 }
// }

3. Write Tool

Purpose: Write files (create or overwrite).

Parameters:

interface WriteToolInput {
  path: string;
  content: string;
}

Returns: Success/failure message.

Example:

const writeTool = createWriteTool();

await writeTool.execute(
  "write_789",
  { path: "src/app.ts", content: "console.log('Hello');" },
  undefined,
  onUpdate
);

// Result:
// {
//   content: [{ type: "text", text: "✓ Wrote 25 bytes to src/app.ts" }],
//   details: { path: "src/app.ts", bytesWritten: 25 }
// }

4. Edit Tool

Purpose: Make precise edits to files using line numbers or search/replace.

Parameters:

interface EditToolInput {
  path: string;
  startLine: number;
  endLine: number;
  content: string;
}

Returns: Success/failure message with diff.

Example:

const editTool = createEditTool();

await editTool.execute(
  "edit_101",
  { path: "src/app.ts", startLine: 5, endLine: 10, content: "const x = 42;" },
  undefined,
  onUpdate
);

// Result:
// {
//   content: [{ type: "text", text: "✓ Edited lines 5-10 in src/app.ts" }],
//   details: {
//     path: "src/app.ts",
//     startLine: 5,
//     endLine: 10,
//     linesChanged: 6,
//     diff: "- const x = 1\n+ const x = 42"
//   }
// }

---

Creating Custom Tools

Basic Custom Tool

const weatherTool: AgentTool<TSchema, WeatherDetails> = {
  name: "get_weather",
  label: "Get Weather",
  description: "Get current weather for a city",
  parameters: Type.Object({
    city: Type.String({ description: "City name" })
  }),
  execute: async (toolCallId, params, signal, onUpdate) => {
    try {
      const response = await fetch(
        `https://api.weather.com/v1/weather?city=${params.city}`,
        { signal }
      );
      
      if (!response.ok) {
        throw new Error(`Weather API error: ${response.status}`);
      }
      
      const data = await response.json();
      
      return {
        content: [{ type: "text", text: `Temperature: ${data.temp}°C` }],
        details: {
          city: params.city,
          temp: data.temp,
          humidity: data.humidity,
          condition: data.condition
        },
        usage: {
          input: 0,
          output: 0,
          cacheRead: 0,
          cacheWrite: 0,
          totalTokens: 0,
          cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 }
        }
      };
    } catch (error) {
      if (error instanceof Error && error.name === "AbortError") {
        throw error;  // Re-throw abort
      }
      return {
        content: [{ type: "text", text: `Error: ${error.message}` }],
        details: { error: error.message },
        isError: true
      };
    }
  }
};

Tool with Streaming Updates

const backupTool: AgentTool<TSchema, BackupDetails> = {
  name: "backup_database",
  label: "Backup Database",
  description: "Create database backup with progress updates",
  parameters: Type.Object({
    database: Type.String(),
    destination: Type.String()
  }),
  execute: async (toolCallId, params, signal, onUpdate) => {
    const totalSize = await getDatabaseSize(params.database);
    let uploaded = 0;
    
    const stream = createBackupStream(params.database);
    
    for await (const chunk of stream) {
      uploaded += chunk.length;
      
      // Stream progress updates
      onUpdate({
        content: [{ 
          type: "text", 
          text: `Backup progress: ${(uploaded / totalSize * 100).toFixed(1)}%` 
        }],
        details: { uploaded, total: totalSize }
      });
      
      if (signal?.aborted) {
        throw new Error("Backup cancelled");
      }
    }
    
    await uploadToStorage(stream, params.destination);
    
    return {
      content: [{ type: "text", text: "Backup completed successfully" }],
      details: {
        database: params.database,
        destination: params.destination,
        size: uploaded,
        duration: Date.now() - startTime
      }
    };
  }
};

Tool with Custom Error Handling

const apiTool: AgentTool<TSchema, ApiDetails> = {
  name: "make_api_call",
  label: "Make API Call",
  description: "Make HTTP request to external API",
  parameters: Type.Object({
    url: Type.String({ format: "uri" }),
    method: Type.Optional(Type.String({ enum: ["GET", "POST", "PUT", "DELETE"] })),
    headers: Type.Optional(Type.Record(Type.String(), Type.String())),
    body: Type.Optional(Type.String())
  }),
  execute: async (toolCallId, params, signal, onUpdate) => {
    try {
      const response = await fetch(params.url, {
        method: params.method || "GET",
        headers: params.headers,
        body: params.body,
        signal
      });
      
      // Handle HTTP errors
      if (!response.ok) {
        const errorBody = await response.text();
        return {
          content: [{ 
            type: "text", 
            text: `HTTP ${response.status}: ${response.statusText}\n${errorBody}` 
          }],
          details: {
            url: params.url,
            method: params.method,
            statusCode: response.status,
            body: errorBody
          },
          isError: true
        };
      }
      
      const contentType = response.headers.get("content-type") || "";
      let responseText = await response.text();
      
      // Handle JSON responses
      if (contentType.includes("application/json")) {
        try {
          const jsonData = JSON.parse(responseText);
          responseText = JSON.stringify(jsonData, null, 2);
        } catch {
          // Not valid JSON, use as-is
        }
      }
      
      return {
        content: [{ type: "text", text: responseText }],
        details: {
          url: params.url,
          method: params.method,
          statusCode: response.status,
          headers: Object.fromEntries(response.headers.entries())
        }
      };
    } catch (error) {
      // Handle network errors
      return {
        content: [{ type: "text", text: `Network error: ${error.message}` }],
        details: {
          url: params.url,
          error: error.message
        },
        isError: true
      };
    }
  }
};

---

Tool Configuration

Tool Options

Tools can be configured with options:

const bashTool = createBashTool({
  cwd: "/home/user/project",
  timeout: 30,
  maxStdoutLines: 1000,
  maxStderrLines: 100
});

const readTool = createReadTool({
  maxSize: 1024 * 1024,  // 1MB
  maxLines: 5000,
  maxTotalSize: 10 * 1024 * 1024  // 10MB total
});

Tool Context

Tools can receive application context:

interface ToolContext {
  userId: string;
  environment: "dev" | "staging" | "prod";
  permissions: string[];
}

const tool: AgentHarnessTool<ToolContext> = {
  name: "deploy_service",
  label: "Deploy Service",
  description: "Deploy service to environment",
  parameters: Type.Object({
    service: Type.String(),
    environment: Type.String({ enum: ["dev", "staging", "prod"] })
  }),
  execute: async (toolCallId, params, signal, onUpdate, context) => {
    // Access context
    if (!context.permissions.includes("deploy")) {
      throw new Error("Permission denied");
    }
    
    if (context.environment === "prod" && !params.environment) {
      throw new Error("Must specify environment for prod deployment");
    }
    
    // ...
  }
};

const harness = new AgentHarness({
  tools: [tool],
  toolContext: {
    userId: "user123",
    environment: "prod",
    permissions: ["read", "write", "deploy"]
  }
});

---

Tool Execution Modes

Sequential Mode

Tools marked as sequential execute one at a time:

const sequentialTool: AgentTool<TSchema> = {
  name: "sequential_tool",
  label: "Sequential Tool",
  description: "Must run one at a time",
  parameters: Type.Object({}),
  executionMode: "sequential",  // Key point
  execute: async (toolCallId, params, signal, onUpdate) => {
    // This tool won't run concurrently with other sequential tools
    // Even if LLM sends multiple tool calls
  }
};

Parallel Mode (Default)

Tools execute concurrently by default:

const parallelTool: AgentTool<TSchema> = {
  name: "parallel_tool",
  label: "Parallel Tool",
  description: "Can run concurrently",
  parameters: Type.Object({}),
  // executionMode defaults to "parallel"
  execute: async (toolCallId, params, signal, onUpdate) => {
    // This tool can run alongside other parallel tools
  }
};

Agent-Level Execution Mode

const agent = new Agent({
  initialState: {...},
  streamFn: ...
  toolExecution: "sequential"  // All tools sequential by default
});

---

Error Handling

Tool Errors

Tools should throw on critical errors (abort, timeout) but return error results on recoverable errors:

execute: async (toolCallId, params, signal, onUpdate) => {
  try {
    // Check for abort first
    if (signal?.aborted) {
      throw new Error("Operation aborted");
    }
    
    // Do work...
    
    // Return error result for recoverable errors
    return {
      content: [{ type: "text", text: "Error: Invalid input" }],
      details: { error: "Invalid input" },
      isError: true
    };
  } catch (error) {
    // Re-throw abort errors
    if (error instanceof Error && error.name === "AbortError") {
      throw error;
    }
    
    // Return error result for other errors
    return {
      content: [{ type: "text", text: `Error: ${error.message}` }],
      details: { error: error.message },
      isError: true
    };
  }
}

Blockable Tools

Use beforeToolCall hook to block tool execution:

beforeToolCall: async ({ toolCall, args }, signal) => {
  if (toolCall.name === "bash") {
    // Check for dangerous commands
    const dangerousPatterns = ["rm -rf", "sudo", "dd if="];
    for (const pattern of dangerousPatterns) {
      if (args.command?.includes(pattern)) {
        return { block: true, reason: "Dangerous command blocked" };
      }
    }
  }
  
  return undefined;  // Allow execution
}

---

Best Practices

1. Respect Abort Signals

execute: async (toolCallId, params, signal, onUpdate) => {
  if (signal?.aborted) {
    throw new Error("Operation aborted");
  }
  
  // Long-running operation
  for await (const item of longProcess()) {
    if (signal?.aborted) {
      throw new Error("Operation aborted");
    }
    onUpdate({ content: [{ type: "text", text: "Processing..." }] });
  }
}

2. Return Meaningful Error Messages

// Bad
return { content: [{ type: "text", text: "Error" }], isError: true };

// Good
return {
  content: [{ type: "text", text: "Failed to read file: permission denied" }],
  details: { path: "/etc/passwd", error: "EACCES" },
  isError: true
};

3. Stream Progress for Long Operations

execute: async (toolCallId, params, signal, onUpdate) => {
  for (let i = 0; i < 100; i++) {
    // Do work...
    onUpdate({
      content: [{ type: "text", text: `Progress: ${i}%` }],
      details: { progress: i }
    });
  }
  
  return {
    content: [{ type: "text", text: "Complete" }],
    details: { progress: 100 }
  };
}

4. Use Proper Tool Result Types

interface BashDetails {
  command: string;
  cwd: string;
  exitCode: number;
  stdout: string;
  stderr: string;
}

return {
  content: [{ type: "text", text: "Command executed" }],
  details: { command, cwd, exitCode, stdout, stderr } as BashDetails
};

5. Handle Large Outputs

execute: async (toolCallId, params, signal, onUpdate) => {
  const stdoutLines: string[] = [];
  const stderrLines: string[] = [];
  
  for await (const chunk of process.stdout) {
    stdoutLines.push(chunk);
    if (stdoutLines.length > MAX_LINES) {
      break;  // Truncate
    }
  }
  
  return {
    content: [{ type: "text", text: truncate(stdoutLines.join("\n")) }],
    details: { stdout: stdoutLines.join("\n") }
  };
}

---

Summary

Tools are the bridge between the agent and the external world.

Key principles:

• Return isError: true for recoverable errors
• Throw on abort/timeout
• Stream progress for long operations
• Respect abort signals throughout
• Use detailed error messages

Built-in tools:

• bash: Execute shell commands
• read: Read files
• write: Write files
• edit: Make precise edits

Custom tools can do anything: API calls, database queries, file operations, etc.