ton/pi_harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge main into model-registry

Browse Source
This commit is contained in:
Mario Zechner
2026-06-22 14:00:18 +02:00
parent 019b022068 717a8f958a
commit abbd911693
220 changed files with 10488 additions and 4354 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/ai/CHANGELOG.md
+80
View File
@@ -23,6 +23,86 @@
- Fixed Claude Fable 5 thinking-off requests to omit Anthropic's unsupported `thinking.type: "disabled"` payload ([#5567](https://github.com/earendil-works/pi/pull/5567) by [@tmustier](https://github.com/tmustier)).
## [0.79.10] - 2026-06-22
### Fixed
- Fixed OpenAI-compatible streaming to preserve encrypted `reasoning_details` that arrive before matching tool call deltas ([#5114](https://github.com/earendil-works/pi/issues/5114)).
## [0.79.9] - 2026-06-20
### Added
- Added configurable `chat-template` thinking support for OpenAI-compatible providers that use `chat_template_kwargs`, such as DeepSeek models behind vLLM ([#5673](https://github.com/earendil-works/pi/issues/5673)).
### Fixed
- Fixed Fireworks GLM-5.2 metadata to use the OpenAI-compatible Chat Completions endpoint with `reasoning_effort` support ([#5923](https://github.com/earendil-works/pi/issues/5923)).
- Fixed OpenRouter GLM-5.2 metadata to expose `xhigh` reasoning and send OpenRouter's native `xhigh` effort ([#5770](https://github.com/earendil-works/pi/issues/5770)).
- Fixed GitHub Copilot OAuth model availability to use the authenticated account's model picker catalog ([#5897](https://github.com/earendil-works/pi/issues/5897)).
## [0.79.8] - 2026-06-19
### Added
- Added `@earendil-works/pi-ai/base` and direct provider registration exports for bundlers that want selective provider transports without root built-in registration ([#5348](https://github.com/earendil-works/pi/pull/5348) by [@FredKSchott](https://github.com/FredKSchott)).
- Added prompt caching for Mistral requests using the pi session ID as `prompt_cache_key`, including cached-token usage and cost accounting ([#5854](https://github.com/earendil-works/pi/issues/5854)).
- Added the OpenRouter Fusion alias as `openrouter/fusion` ([#5866](https://github.com/earendil-works/pi/pull/5866) by [@dannote](https://github.com/dannote)).
## [0.79.7] - 2026-06-18
### Added
- Added GLM-5.2 model to the OpenCode Go subscription model catalog ([#5860](https://github.com/earendil-works/pi/issues/5860)).
## [0.79.6] - 2026-06-16
### Fixed
- Fixed OpenCode Go DeepSeek V4 thinking-off requests to send the provider's `thinking: { type: "disabled" }` compatibility parameter.
## [0.79.5] - 2026-06-16
### Added
- Added provider-scoped `StreamOptions.env` overrides for provider configuration, including Cloudflare endpoint placeholders, Azure OpenAI, Google Vertex, Amazon Bedrock, cache retention, and proxy environment lookups ([#5728](https://github.com/earendil-works/pi/issues/5728)).
### Fixed
- Fixed OpenAI Responses streaming to tolerate null message content from OpenAI-compatible servers before tool calls ([#5819](https://github.com/earendil-works/pi/issues/5819)).
- Fixed OpenCode DeepSeek V4 thinking requests to avoid sending both `thinking` and `reasoning_effort` ([#5818](https://github.com/earendil-works/pi/issues/5818)).
- Fixed Z.AI GLM-5.2 thinking requests to send `reasoning_effort` with the provider's `high`/`max` effort mapping ([#5770](https://github.com/earendil-works/pi/issues/5770)).
- Fixed Google and `google-vertex` Gemini model metadata to map `latest` aliases to the current models, add Gemini 3.5 Flash for Vertex, correct Gemini 2.5 Flash Vertex cache pricing, and remove shut-down Vertex preview models ([#5761](https://github.com/earendil-works/pi/issues/5761)).
- Fixed Moonshot AI China model metadata to include Kimi K2.7 Code, and omitted unsupported thinking-off payloads for Kimi K2.7 Code models ([#5760](https://github.com/earendil-works/pi/issues/5760)).
## [0.79.4] - 2026-06-15
### Fixed
- Fixed Anthropic 1-hour prompt-cache write cost accounting to price 1-hour cache writes at 2x input instead of the 5-minute cache-write rate ([#5738](https://github.com/earendil-works/pi/pull/5738) by [@theBucky](https://github.com/theBucky)).
- Fixed GitHub Copilot Claude adaptive-thinking effort metadata to match manually checked Copilot model capabilities ([#4637](https://github.com/earendil-works/pi/issues/4637)).
- Fixed OpenCode/OpenCode Go completion models that reject `prompt_cache_retention` to omit long-retention cache fields when `cacheRetention` is `long` ([#5702](https://github.com/earendil-works/pi/issues/5702)).
## [0.79.3] - 2026-06-13
### Fixed
- Restored OpenAI GPT-5.4/GPT-5.5 and OpenAI Codex GPT-5.4/GPT-5.4 mini/GPT-5.5 context window metadata to the observed 272k-token Codex backend limit, avoiding a billing hazard from sending prompts above Codex's accepted limit (reported by [@trethore](https://github.com/trethore)).
## [0.79.2] - 2026-06-12
### Added
- Added AWS data retention documentation links to Amazon Bedrock unsupported data retention mode validation errors ([#5561](https://github.com/earendil-works/pi/pull/5561) by [@unexge](https://github.com/unexge)).
### Fixed
- Fixed OpenAI-compatible context overflow detection for parenthesized `maximum context length (N)` errors ([#5677](https://github.com/earendil-works/pi/issues/5677)).
- Fixed OpenAI GPT-5.4/GPT-5.5 and OpenAI Codex GPT-5.4/GPT-5.4 mini/GPT-5.5 context window metadata to match current OpenAI limits ([#5644](https://github.com/earendil-works/pi/issues/5644)).
- Increased the OpenAI Codex Responses SSE response-header timeout to 20 seconds to reduce false-positive stalls while retaining the bounded wait introduced for zero-event hangs ([#4945](https://github.com/earendil-works/pi/issues/4945)).
- Fixed Anthropic refusal stops to preserve provider `stop_details` explanations in error messages ([#5666](https://github.com/earendil-works/pi/pull/5666) by [@rwachtler](https://github.com/rwachtler)).
- Fixed Claude Fable 5 thinking-off requests to omit Anthropic's unsupported `thinking.type: "disabled"` payload ([#5567](https://github.com/earendil-works/pi/pull/5567) by [@tmustier](https://github.com/tmustier)).
## [0.79.1] - 2026-06-09
### Added
packages/ai/README.md
+21 -1
View File
@@ -1055,7 +1055,8 @@ interface OpenAICompletionsCompat {
requiresAssistantAfterToolResult?: boolean; // Whether tool results must be followed by an assistant message (default: false)
requiresThinkingAsText?: boolean; // Whether thinking blocks must be converted to text (default: false)
requiresReasoningContentOnAssistantMessages?: boolean; // Whether all replayed assistant messages must include empty reasoning_content when reasoning is enabled (default: auto-detected for DeepSeek)
thinkingFormat?: 'openai' | 'openrouter' | 'deepseek' | 'together' | 'zai' | 'qwen' | 'qwen-chat-template' | 'string-thinking' | 'ant-ling'; // Format for reasoning param: 'openai' uses reasoning_effort, 'openrouter' uses reasoning: { effort }, 'deepseek' uses thinking: { type } plus reasoning_effort when supported, 'together' uses reasoning: { enabled } plus reasoning_effort when supported, 'zai' uses enable_thinking, 'qwen' uses enable_thinking, 'qwen-chat-template' uses chat_template_kwargs.enable_thinking, 'string-thinking' uses top-level thinking, 'ant-ling' uses reasoning: { effort } only for mapped efforts (default: openai)
thinkingFormat?: 'openai' | 'openrouter' | 'deepseek' | 'together' | 'zai' | 'qwen' | 'chat-template' | 'qwen-chat-template' | 'string-thinking' | 'ant-ling'; // Format for reasoning param: 'openai' uses reasoning_effort, 'openrouter' uses reasoning: { effort }, 'deepseek' uses thinking: { type } plus reasoning_effort when supported, 'together' uses reasoning: { enabled } plus reasoning_effort when supported, 'zai' uses thinking: { type }, 'qwen' uses enable_thinking, 'chat-template' uses configurable chat_template_kwargs, 'qwen-chat-template' uses chat_template_kwargs.enable_thinking and preserve_thinking, 'string-thinking' uses top-level thinking, 'ant-ling' uses reasoning: { effort } only for mapped efforts (default: openai)
chatTemplateKwargs?: Record<string, string | number | boolean | null | { '$var': 'thinking.enabled' | 'thinking.effort'; omitWhenOff?: boolean }>; // chat_template_kwargs values; use $var for pi-controlled thinking values
cacheControlFormat?: 'anthropic'; // Anthropic-style cache_control on system prompt, last tool, and last user/assistant text content
openRouterRouting?: OpenRouterRouting; // OpenRouter routing preferences (default: {})
vercelGatewayRouting?: VercelGatewayRouting; // Vercel AI Gateway routing preferences (default: {})
@@ -1265,6 +1266,25 @@ Browser compatibility notes:
- OAuth login flows are Node-only. They are lazy-loaded behind bundler-opaque imports, so registering an OAuth-capable provider does not pull Node-only code into a browser bundle — only actually logging in would.
- Use a server-side proxy or backend service if you need Bedrock or OAuth-based auth from a web app.
### Provider-Scoped Environment Overrides
Pass `env` in stream options to scope provider configuration to a request. Values in `env` are used before process environment variables for provider auth and configuration such as Cloudflare account IDs, Azure OpenAI settings, Vertex project/location, Bedrock settings, `PI_CACHE_RETENTION`, and `HTTP_PROXY`/`HTTPS_PROXY`.
```typescript
const models = builtinModels();
const model = models.getModel('cloudflare-ai-gateway', 'workers-ai/@cf/moonshotai/kimi-k2.6')!;
const response = await models.complete(model, context, {
env: {
CLOUDFLARE_API_KEY: '...',
CLOUDFLARE_ACCOUNT_ID: 'account-id',
CLOUDFLARE_GATEWAY_ID: 'gateway-id'
}
});
```
Use this when one process needs different provider settings per request, or when ambient environment variables should not leak into a provider call.
## OAuth Providers
Several providers support OAuth authentication instead of static API keys:
packages/ai/package.json
+5 -4
View File
@@ -1,6 +1,6 @@
{
"name": "@earendil-works/pi-ai",
"version": "0.79.1",
"version": "0.79.10",
"description": "Unified LLM API with automatic model discovery and provider configuration",
"type": "module",
"main": "./dist/index.js",
@@ -86,9 +86,10 @@
"dependencies": {
"@anthropic-ai/sdk": "0.91.1",
"@aws-sdk/client-bedrock-runtime": "3.1048.0",
"@smithy/node-http-handler": "4.7.3",
"@google/genai": "1.52.0",
"@mistralai/mistralai": "2.2.1",
"@mistralai/mistralai": "2.2.6",
"@opentelemetry/api": "1.9.0",
"@smithy/node-http-handler": "4.7.3",
"http-proxy-agent": "7.0.2",
"https-proxy-agent": "7.0.6",
"openai": "6.26.0",
@@ -118,6 +119,6 @@
"devDependencies": {
"@types/node": "24.12.4",
"canvas": "3.2.3",
"vitest": "3.2.4"
"vitest": "4.1.9"
}
}
packages/ai/scripts/generate-models.ts
+179 -182
View File
@@ -68,6 +68,8 @@ const KIMI_STATIC_HEADERS = {
"User-Agent": "KimiCLI/1.5",
} as const;
const MOONSHOT_CN_MIRRORED_MODEL_IDS = new Set(["kimi-k2.7-code", "kimi-k2.7-code-highspeed"]);
const TOGETHER_BASE_URL = "https://api.together.ai/v1";
const TOGETHER_BASE_COMPAT: OpenAICompletionsCompat = {
supportsStore: false,
@@ -121,6 +123,7 @@ const TOGETHER_TOGGLE_REASONING_LEVEL_MAP = {
const AI_GATEWAY_MODELS_URL = "https://ai-gateway.vercel.sh/v1";
const AI_GATEWAY_BASE_URL = "https://ai-gateway.vercel.sh";
const VERTEX_BASE_URL = "https://{location}-aiplatform.googleapis.com";
const NVIDIA_BASE_URL = "https://integrate.api.nvidia.com/v1";
const NVIDIA_HEADERS = {
"NVCF-POLL-SECONDS": "3600",
@@ -154,6 +157,13 @@ const NVIDIA_NIM_UNSUPPORTED_MODELS = new Set([
"upstage/solar-10.7b-instruct",
]);
const ZAI_TOOL_STREAM_UNSUPPORTED_MODELS = new Set(["glm-4.5", "glm-4.5-air", "glm-4.5-flash", "glm-4.5v"]);
const ZAI_GLM52_THINKING_LEVEL_MAP = {
minimal: null,
low: "high",
medium: "high",
high: "high",
xhigh: "max",
} as const;
const EAGER_TOOL_INPUT_STREAMING_UNSUPPORTED_ANTHROPIC_MODELS = new Set([
"github-copilot:claude-haiku-4.5",
"github-copilot:claude-sonnet-4",
@@ -187,6 +197,23 @@ const OPENAI_RESPONSES_NONE_REASONING_MODELS = new Set([
"gpt-5.5",
]);
const OPENCODE_OPENAI_COMPLETIONS_LONG_CACHE_RETENTION_UNSUPPORTED_MODELS = new Set([
"opencode:deepseek-v4-flash",
"opencode:deepseek-v4-pro",
"opencode:kimi-k2.5",
"opencode:kimi-k2.6",
"opencode:minimax-m2.7",
"opencode-go:kimi-k2.6",
]);
// Checked manually against the authenticated GitHub Copilot /models endpoint on 2026-06-15.
// Keep this to narrow corrections over models.dev metadata instead of snapshotting Copilot's catalog.
const GITHUB_COPILOT_THINKING_LEVEL_OVERRIDES = {
"claude-opus-4.7": { minimal: "low" },
"claude-opus-4.8": { minimal: "low" },
"claude-sonnet-4.6": { minimal: "low", xhigh: "max" },
} satisfies Record<string, NonNullable<Model<Api>["thinkingLevelMap"]>>;
function mergeThinkingLevelMap(model: Model<any>, map: NonNullable<Model<any>["thinkingLevelMap"]>): void {
model.thinkingLevelMap = { ...model.thinkingLevelMap, ...map };
}
@@ -251,7 +278,8 @@ function isGemini3ProModel(modelId: string): boolean {
}
function isGemini3FlashModel(modelId: string): boolean {
return /gemini-3(?:\.\d+)?-flash/.test(modelId.toLowerCase());
const id = modelId.toLowerCase();
return /gemini-3(?:\.\d+)?-flash/.test(id) || id === "gemini-flash-latest" || id === "gemini-flash-lite-latest";
}
function isGemma4Model(modelId: string): boolean {
@@ -330,6 +358,15 @@ function applyThinkingLevelMetadata(model: Model<any>): void {
if (model.provider === "openai-codex" && supportsOpenAiXhigh(model.id)) {
mergeThinkingLevelMap(model, { minimal: "low" });
}
if (
(model.provider === "moonshotai" || model.provider === "moonshotai-cn") &&
(model.id === "kimi-k2.7-code" || model.id === "kimi-k2.7-code-highspeed")
) {
// Kimi K2.7 Code is always-thinking. Official docs say
// `thinking: { type: "disabled" }` is rejected, and callers can omit
// the thinking parameter to use the enabled default.
mergeThinkingLevelMap(model, { off: null });
}
if (model.provider === "openrouter" && model.id.startsWith("inception/mercury-2")) {
// Mercury 2 in instant mode (reasoning_effort: "none") disables tool calling.
// Mark "off" unsupported so the openai-completions provider omits the reasoning param
@@ -337,6 +374,12 @@ function applyThinkingLevelMetadata(model: Model<any>): void {
// Pi's low/medium/high pass through verbatim; OpenRouter normalizes to Mercury's vocabulary.
mergeThinkingLevelMap(model, { off: null });
}
if (model.provider === "openrouter" && model.id === "z-ai/glm-5.2") {
mergeThinkingLevelMap(model, { xhigh: "xhigh" });
}
if (model.provider === "fireworks" && model.id === "accounts/fireworks/models/glm-5p2") {
mergeThinkingLevelMap(model, { off: "none", minimal: null, low: "high", medium: "high", xhigh: "max" });
}
if (model.provider === "opencode-go" && model.id === "kimi-k2.6") {
// OpenCode Go exposes Kimi K2.6 thinking as on/off, not distinct effort tiers.
mergeThinkingLevelMap(model, { minimal: null, low: null, medium: null });
@@ -349,6 +392,12 @@ function applyThinkingLevelMetadata(model: Model<any>): void {
// Ring reasons by default. Only high/xhigh have documented explicit effort controls.
mergeThinkingLevelMap(model, ANT_LING_RING_THINKING_LEVEL_MAP);
}
if (model.provider === "github-copilot") {
const override = GITHUB_COPILOT_THINKING_LEVEL_OVERRIDES[model.id];
if (override) {
mergeThinkingLevelMap(model, override);
}
}
}
function getAnthropicMessagesCompat(provider: string, modelId: string): AnthropicMessagesCompat | undefined {
@@ -372,6 +421,10 @@ function normalizeNvidiaModelId(modelId: string): string {
return modelId.toLowerCase().replaceAll("_", ".");
}
function roundCost(value: number): number {
return Number(value.toFixed(6));
}
async function fetchNvidiaNimModelIds(): Promise<Map<string, string>> {
try {
console.log("Fetching models from NVIDIA NIM API...");
@@ -417,10 +470,10 @@ async function fetchOpenRouterModels(): Promise<Model<any>[]> {
}
// Convert pricing from $/token to $/million tokens
const inputCost = parseFloat(model.pricing?.prompt || "0") * 1_000_000;
const outputCost = parseFloat(model.pricing?.completion || "0") * 1_000_000;
const cacheReadCost = parseFloat(model.pricing?.input_cache_read || "0") * 1_000_000;
const cacheWriteCost = parseFloat(model.pricing?.input_cache_write || "0") * 1_000_000;
const inputCost = roundCost(parseFloat(model.pricing?.prompt || "0") * 1_000_000);
const outputCost = roundCost(parseFloat(model.pricing?.completion || "0") * 1_000_000);
const cacheReadCost = roundCost(parseFloat(model.pricing?.input_cache_read || "0") * 1_000_000);
const cacheWriteCost = roundCost(parseFloat(model.pricing?.input_cache_write || "0") * 1_000_000);
const normalizedModel: Model<any> = {
id: modelKey,
@@ -476,10 +529,10 @@ async function fetchAiGatewayModels(): Promise<Model<any>[]> {
input.push("image");
}
const inputCost = toNumber(model.pricing?.input) * 1_000_000;
const outputCost = toNumber(model.pricing?.output) * 1_000_000;
const cacheReadCost = toNumber(model.pricing?.input_cache_read) * 1_000_000;
const cacheWriteCost = toNumber(model.pricing?.input_cache_write) * 1_000_000;
const inputCost = roundCost(toNumber(model.pricing?.input) * 1_000_000);
const outputCost = roundCost(toNumber(model.pricing?.output) * 1_000_000);
const cacheReadCost = roundCost(toNumber(model.pricing?.input_cache_read) * 1_000_000);
const cacheWriteCost = roundCost(toNumber(model.pricing?.input_cache_write) * 1_000_000);
models.push({
id: model.id,
@@ -586,6 +639,13 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
for (const [modelId, model] of Object.entries(data.google.models)) {
const m = model as ModelsDevModel;
if (m.tool_call !== true) continue;
let source = m;
if (modelId === "gemini-flash-latest") {
source = (data.google.models["gemini-3.5-flash"] as ModelsDevModel | undefined) ?? m;
}
if (modelId === "gemini-flash-lite-latest") {
source = (data.google.models["gemini-3.1-flash-lite"] as ModelsDevModel | undefined) ?? m;
}
models.push({
id: modelId,
@@ -593,16 +653,57 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
api: "google-generative-ai",
provider: "google",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
reasoning: m.reasoning === true,
input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"],
reasoning: source.reasoning === true,
input: source.modalities?.input?.includes("image") ? ["text", "image"] : ["text"],
cost: {
input: m.cost?.input || 0,
output: m.cost?.output || 0,
cacheRead: m.cost?.cache_read || 0,
cacheWrite: m.cost?.cache_write || 0,
input: source.cost?.input || 0,
output: source.cost?.output || 0,
cacheRead: source.cost?.cache_read || 0,
cacheWrite: source.cost?.cache_write || 0,
},
contextWindow: m.limit?.context || 4096,
maxTokens: m.limit?.output || 4096,
contextWindow: source.limit?.context || 4096,
maxTokens: source.limit?.output || 4096,
});
}
}
// Process Google Vertex Gemini models. The google-vertex models.dev catalog also includes
// Claude, OpenAI, and other MaaS models that do not use the @google/genai Gemini streaming
// path implemented by our google-vertex provider.
if (data["google-vertex"]?.models) {
for (const [modelId, model] of Object.entries(data["google-vertex"].models)) {
const m = model as ModelsDevModel;
if (m.tool_call !== true) continue;
if (!modelId.startsWith("gemini-")) continue;
if (modelId === "gemini-3.1-flash-lite-preview") continue;
let source = m;
if (modelId === "gemini-flash-latest") {
source = (data["google-vertex"].models["gemini-3.5-flash"] as ModelsDevModel | undefined) ?? m;
}
if (modelId === "gemini-flash-lite-latest") {
source = (data["google-vertex"].models["gemini-3.1-flash-lite"] as ModelsDevModel | undefined) ?? m;
}
// models.dev reports Vertex cache_read/cache_write values for Gemini 2.5 Flash that
// do not match the official Gemini API standard pricing table. pi only accounts
// cachedContentTokenCount as cacheRead.
const cacheRead = modelId === "gemini-2.5-flash" ? 0.03 : source.cost?.cache_read || 0;
models.push({
id: modelId,
name: m.name || modelId,
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: source.reasoning === true,
input: source.modalities?.input?.includes("image") ? ["text", "image"] : ["text"],
cost: {
input: source.cost?.input || 0,
output: source.cost?.output || 0,
cacheRead,
cacheWrite: 0,
},
contextWindow: source.limit?.context || 4096,
maxTokens: source.limit?.output || 4096,
});
}
}
@@ -806,6 +907,8 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
if (m.tool_call !== true) continue;
const supportsImage = m.modalities?.input?.includes("image");
const isGlm52 = modelId === "glm-5.2";
models.push({
id: modelId,
name: m.name || modelId,
@@ -813,6 +916,7 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
provider,
baseUrl,
reasoning: m.reasoning === true,
...(isGlm52 ? { thinkingLevelMap: ZAI_GLM52_THINKING_LEVEL_MAP } : {}),
input: supportsImage ? ["text", "image"] : ["text"],
cost: {
input: m.cost?.input || 0,
@@ -823,6 +927,7 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
compat: {
supportsDeveloperRole: false,
thinkingFormat: "zai",
...(isGlm52 ? { supportsReasoningEffort: true } : {}),
...(!ZAI_TOOL_STREAM_UNSUPPORTED_MODELS.has(modelId) ? { zaiToolStream: true } : {}),
},
contextWindow: m.limit?.context || 4096,
@@ -849,7 +954,7 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
cost: {
input: m.cost?.input || 0,
output: m.cost?.output || 0,
cacheRead: m.cost?.cache_read || 0,
cacheRead: m.cost?.cache_read ?? (m.cost?.input ? roundCost(m.cost.input * 0.1) : 0),
cacheWrite: m.cost?.cache_write || 0,
},
contextWindow: m.limit?.context || 4096,
@@ -1066,6 +1171,13 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
if (api === "openai-completions") {
compat = { ...(compat ?? {}), maxTokensField: "max_tokens" };
if (
OPENCODE_OPENAI_COMPLETIONS_LONG_CACHE_RETENTION_UNSUPPORTED_MODELS.has(
`${variant.provider}:${modelId}`,
)
) {
compat = { ...compat, supportsLongCacheRetention: false };
}
}
models.push({
@@ -1228,12 +1340,27 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
supportsStrictMode: false,
thinkingFormat: "deepseek",
};
const getMoonshotProviderModels = (key: "moonshotai" | "moonshotai-cn"): Record<string, ModelsDevModel> => {
const providerModels = data[key]?.models as Record<string, ModelsDevModel> | undefined;
return providerModels ? { ...providerModels } : {};
};
const moonshotModels = {
moonshotai: getMoonshotProviderModels("moonshotai"),
"moonshotai-cn": getMoonshotProviderModels("moonshotai-cn"),
};
// models.dev can lag the CN catalog while the global Moonshot catalog already
// has the model. Mirror selected current model IDs into moonshotai-cn until
// upstream CN metadata catches up.
for (const modelId of MOONSHOT_CN_MIRRORED_MODEL_IDS) {
const model = moonshotModels.moonshotai[modelId];
if (model && !moonshotModels["moonshotai-cn"][modelId]) {
moonshotModels["moonshotai-cn"][modelId] = model;
}
}
for (const { key, provider, baseUrl } of moonshotVariants) {
if (!data[key]?.models) continue;
for (const [modelId, model] of Object.entries(data[key].models)) {
const m = model as ModelsDevModel;
for (const [modelId, m] of Object.entries(moonshotModels[key])) {
if (m.tool_call !== true) continue;
models.push({
@@ -1390,7 +1517,11 @@ async function generateModels() {
candidate.cost.output = 1.9;
candidate.cost.cacheRead = 0.119;
}
if (candidate.provider === "fireworks" && candidate.id === "accounts/fireworks/models/glm-5p2") {
candidate.api = "openai-completions";
candidate.baseUrl = "https://api.fireworks.ai/inference/v1";
candidate.compat = { supportsStore: false, supportsDeveloperRole: false };
}
}
@@ -1731,9 +1862,10 @@ async function generateModels() {
for (const candidate of allModels) {
if (candidate.api === "openai-completions" && candidate.id.includes("deepseek-v4")) {
const preservesNativeReasoningEffort = candidate.provider === "openrouter" || candidate.provider === "opencode";
candidate.compat = {
...candidate.compat,
...(candidate.provider === "openrouter"
...(preservesNativeReasoningEffort
? {
requiresReasoningContentOnAssistantMessages:
deepseekCompat.requiresReasoningContentOnAssistantMessages,
@@ -1908,166 +2040,31 @@ async function generateModels() {
});
}
const VERTEX_BASE_URL = "https://{location}-aiplatform.googleapis.com";
const vertexModels: Model<"google-vertex">[] = [
{
id: "gemini-3-pro-preview",
name: "Gemini 3 Pro Preview (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
// Add "fusion" alias for openrouter/fusion. OpenRouter exposes Fusion as a
// router alias/plugin entry point; its model metadata does not advertise
// tools, but the alias resolves to a concrete model that can invoke caller
// tools and has the openrouter:fusion server tool auto-injected.
if (!allModels.some(m => m.provider === "openrouter" && m.id === "openrouter/fusion")) {
allModels.push({
id: "openrouter/fusion",
name: "OpenRouter: Fusion",
api: "openai-completions",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
reasoning: true,
input: ["text", "image"],
cost: { input: 2, output: 12, cacheRead: 0.2, cacheWrite: 0 },
input: ["text"],
cost: {
// we dont know about the costs because Fusion routes to multiple models
// and then charges you for the underlying used models
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 64000,
},
{
id: "gemini-3.1-pro-preview",
name: "Gemini 3.1 Pro Preview (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 2, output: 12, cacheRead: 0.2, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 65536,
},
{
id: "gemini-3.1-pro-preview-customtools",
name: "Gemini 3.1 Pro Preview Custom Tools (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 2, output: 12, cacheRead: 0.2, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 65536,
},
{
id: "gemini-3-flash-preview",
name: "Gemini 3 Flash Preview (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 0.5, output: 3, cacheRead: 0.05, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 65536,
},
{
id: "gemini-2.0-flash",
name: "Gemini 2.0 Flash (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: false,
input: ["text", "image"],
cost: { input: 0.15, output: 0.6, cacheRead: 0.0375, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 8192,
},
{
id: "gemini-2.0-flash-lite",
name: "Gemini 2.0 Flash Lite (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 0.075, output: 0.3, cacheRead: 0.01875, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 65536,
},
{
id: "gemini-2.5-pro",
name: "Gemini 2.5 Pro (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 1.25, output: 10, cacheRead: 0.125, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 65536,
},
{
id: "gemini-2.5-flash",
name: "Gemini 2.5 Flash (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 0.3, output: 2.5, cacheRead: 0.03, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 65536,
},
{
id: "gemini-2.5-flash-lite-preview-09-2025",
name: "Gemini 2.5 Flash Lite Preview 09-25 (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 0.1, output: 0.4, cacheRead: 0.01, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 65536,
},
{
id: "gemini-2.5-flash-lite",
name: "Gemini 2.5 Flash Lite (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 0.1, output: 0.4, cacheRead: 0.01, cacheWrite: 0 },
contextWindow: 1048576,
maxTokens: 65536,
},
{
id: "gemini-1.5-pro",
name: "Gemini 1.5 Pro (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: false,
input: ["text", "image"],
cost: { input: 1.25, output: 5, cacheRead: 0.3125, cacheWrite: 0 },
contextWindow: 1000000,
maxTokens: 8192,
},
{
id: "gemini-1.5-flash",
name: "Gemini 1.5 Flash (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: false,
input: ["text", "image"],
cost: { input: 0.075, output: 0.3, cacheRead: 0.01875, cacheWrite: 0 },
contextWindow: 1000000,
maxTokens: 8192,
},
{
id: "gemini-1.5-flash-8b",
name: "Gemini 1.5 Flash-8B (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: VERTEX_BASE_URL,
reasoning: false,
input: ["text", "image"],
cost: { input: 0.0375, output: 0.15, cacheRead: 0.01, cacheWrite: 0 },
contextWindow: 1000000,
maxTokens: 8192,
},
];
allModels.push(...vertexModels);
maxTokens: 30000,
});
}
// Azure Foundry deploys these with larger context windows than OpenAI's own API,
// which caps gpt-5.4/gpt-5.5 at 272k. See models-sold-directly-by-azure docs.
packages/ai/src/api/anthropic-messages.ts
+33 -16
View File
@@ -5,6 +5,7 @@ import type {
MessageCreateParamsStreaming,
MessageParam,
RawMessageStreamEvent,
RefusalStopDetails,
} from "@anthropic-ai/sdk/resources/messages.js";
import { calculateCost } from "../models.ts";
import type {
@@ -16,6 +17,7 @@ import type {
ImageContent,
Message,
Model,
ProviderEnv,
SimpleStreamOptions,
StopReason,
StreamFunction,
@@ -29,6 +31,7 @@ import type {
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { parseJsonWithRepair, parseStreamingJson } from "../utils/json-parse.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { resolveCloudflareBaseUrl } from "./cloudflare.ts";
@@ -40,11 +43,11 @@ import { transformMessages } from "./transform-messages.ts";
* Resolve cache retention preference.
* Defaults to "short" and uses PI_CACHE_RETENTION for backward compatibility.
*/
function resolveCacheRetention(cacheRetention?: CacheRetention): CacheRetention {
function resolveCacheRetention(cacheRetention?: CacheRetention, env?: ProviderEnv): CacheRetention {
if (cacheRetention) {
return cacheRetention;
}
if (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long") {
if (getProviderEnvValue("PI_CACHE_RETENTION", env) === "long") {
return "long";
}
return "short";
@@ -53,8 +56,9 @@ function resolveCacheRetention(cacheRetention?: CacheRetention): CacheRetention
function getCacheControl(
model: Model<"anthropic-messages">,
cacheRetention?: CacheRetention,
env?: ProviderEnv,
): { retention: CacheRetention; cacheControl?: CacheControlEphemeral } {
const retention = resolveCacheRetention(cacheRetention);
const retention = resolveCacheRetention(cacheRetention, env);
if (retention === "none") {
return { retention };
}
@@ -493,7 +497,7 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
});
}
const cacheRetention = options?.cacheRetention ?? resolveCacheRetention();
const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
const created = createClient(
@@ -504,6 +508,7 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
options?.headers,
copilotDynamicHeaders,
cacheSessionId,
options?.env,
);
client = created.client;
isOAuth = created.isOAuthToken;
@@ -534,6 +539,7 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
output.usage.output = event.message.usage.output_tokens || 0;
output.usage.cacheRead = event.message.usage.cache_read_input_tokens || 0;
output.usage.cacheWrite = event.message.usage.cache_creation_input_tokens || 0;
output.usage.cacheWrite1h = event.message.usage.cache_creation?.ephemeral_1h_input_tokens || 0;
// Anthropic doesn't provide total_tokens, compute from components
output.usage.totalTokens =
output.usage.input + output.usage.output + output.usage.cacheRead + output.usage.cacheWrite;
@@ -660,7 +666,11 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
}
} else if (event.type === "message_delta") {
if (event.delta.stop_reason) {
output.stopReason = mapStopReason(event.delta.stop_reason);
const stopReasonResult = mapStopReason(event.delta.stop_reason, event.delta.stop_details);
output.stopReason = stopReasonResult.stopReason;
if (stopReasonResult.errorMessage) {
output.errorMessage = stopReasonResult.errorMessage;
}
}
// Only update usage fields if present (not null).
// Preserves input_tokens from message_start when proxies omit it in message_delta.
@@ -688,7 +698,7 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
}
if (output.stopReason === "aborted" || output.stopReason === "error") {
throw new Error("An unknown error occurred");
throw new Error(output.errorMessage || "An unknown error occurred");
}
stream.push({ type: "done", reason: output.stopReason, message: output });
@@ -788,6 +798,7 @@ function createClient(
optionsHeaders?: Record<string, string>,
dynamicHeaders?: Record<string, string>,
sessionId?: string,
env?: ProviderEnv,
): { client: Anthropic; isOAuthToken: boolean } {
// Adaptive thinking models have interleaved thinking built in, so skip the beta header.
const needsInterleavedBeta = interleavedThinking && model.compat?.forceAdaptiveThinking !== true;
@@ -803,7 +814,7 @@ function createClient(
const client = new Anthropic({
apiKey: null,
authToken: null,
baseURL: resolveCloudflareBaseUrl(model),
baseURL: resolveCloudflareBaseUrl(model, env),
dangerouslyAllowBrowser: true,
defaultHeaders: mergeHeaders(
{
@@ -896,7 +907,7 @@ function buildParams(
isOAuthToken: boolean,
options?: AnthropicOptions,
): MessageCreateParamsStreaming {
const { cacheControl } = getCacheControl(model, options?.cacheRetention);
const { cacheControl } = getCacheControl(model, options?.cacheRetention, options?.env);
const compat = getAnthropicCompat(model);
const params: MessageCreateParamsStreaming = {
model: model.id,
@@ -1202,22 +1213,28 @@ function convertTools(
});
}
function mapStopReason(reason: Anthropic.Messages.StopReason | string): StopReason {
function mapStopReason(
reason: Anthropic.Messages.StopReason | string,
stopDetails?: RefusalStopDetails | null,
): { stopReason: StopReason; errorMessage?: string } {
switch (reason) {
case "end_turn":
return "stop";
return { stopReason: "stop" };
case "max_tokens":
return "length";
return { stopReason: "length" };
case "tool_use":
return "toolUse";
return { stopReason: "toolUse" };
case "refusal":
return "error";
return {
stopReason: "error",
errorMessage: stopDetails?.explanation || `The model refused to complete the request`,
};
case "pause_turn": // Stop is good enough -> resubmit
return "stop";
return { stopReason: "stop" };
case "stop_sequence":
return "stop"; // We don't supply stop sequences, so this should never happen
return { stopReason: "stop" }; // We don't supply stop sequences, so this should never happen
case "sensitive": // Content flagged by safety filters (not yet in SDK types)
return "error";
return { stopReason: "error" };
default:
// Handle unknown stop reasons gracefully (API may add new values)
throw new Error(`Unhandled stop reason: ${reason}`);
packages/ai/src/api/azure-openai-responses.ts
+11 -4
View File
@@ -12,6 +12,7 @@ import type {
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.ts";
import { buildBaseOptions } from "./simple-options.ts";
@@ -36,7 +37,9 @@ function resolveDeploymentName(model: Model<"azure-openai-responses">, options?:
if (options?.azureDeploymentName) {
return options.azureDeploymentName;
}
const mappedDeployment = parseDeploymentNameMap(process.env.AZURE_OPENAI_DEPLOYMENT_NAME_MAP).get(model.id);
const mappedDeployment = parseDeploymentNameMap(
getProviderEnvValue("AZURE_OPENAI_DEPLOYMENT_NAME_MAP", options?.env),
).get(model.id);
return mappedDeployment || model.id;
}
@@ -198,10 +201,14 @@ function resolveAzureConfig(
model: Model<"azure-openai-responses">,
options?: AzureOpenAIResponsesOptions,
): { baseUrl: string; apiVersion: string } {
const apiVersion = options?.azureApiVersion || process.env.AZURE_OPENAI_API_VERSION || DEFAULT_AZURE_API_VERSION;
const apiVersion =
options?.azureApiVersion ||
getProviderEnvValue("AZURE_OPENAI_API_VERSION", options?.env) ||
DEFAULT_AZURE_API_VERSION;
const baseUrl = options?.azureBaseUrl?.trim() || process.env.AZURE_OPENAI_BASE_URL?.trim() || undefined;
const resourceName = options?.azureResourceName || process.env.AZURE_OPENAI_RESOURCE_NAME;
const baseUrl =
options?.azureBaseUrl?.trim() || getProviderEnvValue("AZURE_OPENAI_BASE_URL", options?.env)?.trim() || undefined;
const resourceName = options?.azureResourceName || getProviderEnvValue("AZURE_OPENAI_RESOURCE_NAME", options?.env);
let resolvedBaseUrl = baseUrl;
packages/ai/src/api/bedrock-converse-stream.ts
+58 -34
View File
@@ -1,3 +1,4 @@
import type { Agent as HttpsAgent } from "node:https";
import {
BedrockRuntimeClient,
type BedrockRuntimeClientConfig,
@@ -23,6 +24,8 @@ import {
} from "@aws-sdk/client-bedrock-runtime";
import { NodeHttpHandler } from "@smithy/node-http-handler";
import type { BuildMiddleware, DocumentType, MetadataBearer } from "@smithy/types";
import { HttpProxyAgent } from "http-proxy-agent";
import { HttpsProxyAgent } from "https-proxy-agent";
import { calculateCost } from "../models.ts";
import type {
Api,
@@ -31,6 +34,7 @@ import type {
Context,
ImageContent,
Model,
ProviderEnv,
SimpleStreamOptions,
StopReason,
StreamFunction,
@@ -45,7 +49,8 @@ import type {
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { parseStreamingJson } from "../utils/json-parse.ts";
import { createHttpProxyAgentsForTarget } from "../utils/node-http-proxy.ts";
import { resolveHttpProxyUrlForTarget } from "../utils/node-http-proxy.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { adjustMaxTokensForThinking, buildBaseOptions, clampReasoning } from "./simple-options.ts";
import { transformMessages } from "./transform-messages.ts";
@@ -119,18 +124,18 @@ export const stream: StreamFunction<"bedrock-converse-stream", BedrockOptions> =
const blocks = output.content as Block[];
const config: BedrockRuntimeClientConfig = {
profile: options.profile,
profile: options.profile || getProviderEnvValue("AWS_PROFILE", options.env),
};
const configuredRegion = getConfiguredBedrockRegion(options);
const hasConfiguredProfile = hasConfiguredBedrockProfile();
const hasAmbientConfiguredProfile = Boolean(getProviderEnvValue("AWS_PROFILE"));
const endpointRegion = getStandardBedrockEndpointRegion(model.baseUrl);
const useExplicitEndpoint = shouldUseExplicitBedrockEndpoint(
model.baseUrl,
configuredRegion,
hasConfiguredProfile,
hasAmbientConfiguredProfile,
);
// Only pin standard AWS Bedrock runtime endpoints when no region/profile is configured.
// Only pin standard AWS Bedrock runtime endpoints when no region or ambient AWS_PROFILE is configured.
// This preserves custom endpoints (VPC/proxy) from #3402 without forcing built-in
// catalog defaults such as us-east-1 to override AWS_REGION/AWS_PROFILE.
if (useExplicitEndpoint) {
@@ -138,8 +143,10 @@ export const stream: StreamFunction<"bedrock-converse-stream", BedrockOptions> =
}
// Resolve bearer token for Bedrock API key auth.
const bearerToken = options.bearerToken || process.env.AWS_BEARER_TOKEN_BEDROCK || undefined;
const useBearerToken = bearerToken !== undefined && process.env.AWS_BEDROCK_SKIP_AUTH !== "1";
const skipAuth = getProviderEnvValue("AWS_BEDROCK_SKIP_AUTH", options.env) === "1";
const bearerToken =
options.bearerToken || getProviderEnvValue("AWS_BEARER_TOKEN_BEDROCK", options.env) || undefined;
const useBearerToken = bearerToken !== undefined && !skipAuth;
// in Node.js/Bun environment only
if (typeof process !== "undefined" && (process.versions?.node || process.versions?.bun)) {
@@ -153,25 +160,33 @@ export const stream: StreamFunction<"bedrock-converse-stream", BedrockOptions> =
config.region = configuredRegion;
} else if (endpointRegion && useExplicitEndpoint) {
config.region = endpointRegion;
} else if (!hasConfiguredProfile) {
} else if (!hasAmbientConfiguredProfile) {
config.region = "us-east-1";
}
// Support proxies that don't need authentication
if (process.env.AWS_BEDROCK_SKIP_AUTH === "1") {
if (skipAuth) {
config.credentials = {
accessKeyId: "dummy-access-key",
secretAccessKey: "dummy-secret-key",
};
}
const proxyAgents = createHttpProxyAgentsForTarget(model.baseUrl);
if (proxyAgents) {
const credentials = getConfiguredBedrockCredentials(options.env);
if (!skipAuth && credentials) {
config.credentials = credentials;
}
const proxyUrl = resolveHttpProxyUrlForTarget(model.baseUrl, options.env);
if (proxyUrl) {
// Bedrock runtime uses NodeHttp2Handler by default since v3.798.0, which is based
// on `http2` module and has no support for http agent.
// Use NodeHttpHandler to support HTTP(S) proxy agents.
config.requestHandler = new NodeHttpHandler(proxyAgents);
} else if (process.env.AWS_BEDROCK_FORCE_HTTP1 === "1") {
config.requestHandler = new NodeHttpHandler({
httpAgent: new HttpProxyAgent(proxyUrl),
httpsAgent: new HttpsProxyAgent(proxyUrl) as unknown as HttpsAgent,
});
} else if (getProviderEnvValue("AWS_BEDROCK_FORCE_HTTP1", options.env) === "1") {
// Some custom endpoints require HTTP/1.1 instead of HTTP/2
config.requestHandler = new NodeHttpHandler();
}
@@ -192,12 +207,12 @@ export const stream: StreamFunction<"bedrock-converse-stream", BedrockOptions> =
if (options.headers && Object.keys(options.headers).length > 0) {
addCustomHeadersMiddleware(client, options.headers);
}
const cacheRetention = resolveCacheRetention(options.cacheRetention);
const cacheRetention = resolveCacheRetention(options.cacheRetention, options.env);
const inferenceMaxTokens = options.maxTokens ?? (isAnthropicClaudeModel(model) ? model.maxTokens : undefined);
let commandInput = {
modelId: model.id,
messages: convertMessages(context, model, cacheRetention),
system: buildSystemPrompt(context.systemPrompt, model, cacheRetention),
messages: convertMessages(context, model, cacheRetention, options.env),
system: buildSystemPrompt(context.systemPrompt, model, cacheRetention, options.env),
inferenceConfig: {
...(inferenceMaxTokens !== undefined && { maxTokens: inferenceMaxTokens }),
...(options.temperature !== undefined && { temperature: options.temperature }),
@@ -578,11 +593,11 @@ function mapThinkingLevelToEffort(
* Resolve cache retention preference.
* Defaults to "short" and uses PI_CACHE_RETENTION for backward compatibility.
*/
function resolveCacheRetention(cacheRetention?: CacheRetention): CacheRetention {
function resolveCacheRetention(cacheRetention?: CacheRetention, env?: ProviderEnv): CacheRetention {
if (cacheRetention) {
return cacheRetention;
}
if (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long") {
if (getProviderEnvValue("PI_CACHE_RETENTION", env) === "long") {
return "long";
}
return "short";
@@ -617,14 +632,14 @@ function isAnthropicClaudeModel(model: Model<"bedrock-converse-stream">): boolea
* As a last resort, set AWS_BEDROCK_FORCE_CACHE=1 to enable cache points.
* Amazon Nova models have automatic caching and don't need explicit cache points.
*/
function supportsPromptCaching(model: Model<"bedrock-converse-stream">): boolean {
function supportsPromptCaching(model: Model<"bedrock-converse-stream">, env?: ProviderEnv): boolean {
const candidates = getModelMatchCandidates(model.id, model.name);
const hasClaudeRef = candidates.some((s) => s.includes("claude"));
if (!hasClaudeRef) {
// Application inference profiles don't contain the model name in the ARN.
// Allow users to force cache points via environment variable.
if (typeof process !== "undefined" && process.env.AWS_BEDROCK_FORCE_CACHE === "1") return true;
if (getProviderEnvValue("AWS_BEDROCK_FORCE_CACHE", env) === "1") return true;
return false;
}
// Claude 4.x models (opus-4, sonnet-4, haiku-4)
@@ -652,13 +667,14 @@ function buildSystemPrompt(
systemPrompt: string | undefined,
model: Model<"bedrock-converse-stream">,
cacheRetention: CacheRetention,
env?: ProviderEnv,
): SystemContentBlock[] | undefined {
if (!systemPrompt) return undefined;
const blocks: SystemContentBlock[] = [{ text: sanitizeSurrogates(systemPrompt) }];
// Add cache point for supported Claude models when caching is enabled
if (cacheRetention !== "none" && supportsPromptCaching(model)) {
if (cacheRetention !== "none" && supportsPromptCaching(model, env)) {
blocks.push({
cachePoint: { type: CachePointType.DEFAULT, ...(cacheRetention === "long" ? { ttl: CacheTTL.ONE_HOUR } : {}) },
});
@@ -699,6 +715,7 @@ function convertMessages(
context: Context,
model: Model<"bedrock-converse-stream">,
cacheRetention: CacheRetention,
env?: ProviderEnv,
): Message[] {
const result: Message[] = [];
const transformedMessages = transformMessages(context.messages, model, normalizeToolCallId);
@@ -844,7 +861,7 @@ function convertMessages(
}
// Add cache point to the last user message for supported Claude models when caching is enabled
if (cacheRetention !== "none" && supportsPromptCaching(model) && result.length > 0) {
if (cacheRetention !== "none" && supportsPromptCaching(model, env) && result.length > 0) {
const lastMessage = result[result.length - 1];
if (lastMessage.role === ConversationRole.USER && lastMessage.content) {
(lastMessage.content as ContentBlock[]).push({
@@ -906,19 +923,26 @@ function mapStopReason(reason: string | undefined): StopReason {
}
function getConfiguredBedrockRegion(options: BedrockOptions): string | undefined {
if (typeof process === "undefined") {
return options.region;
}
return options.region || process.env.AWS_REGION || process.env.AWS_DEFAULT_REGION || undefined;
return (
options.region ||
getProviderEnvValue("AWS_REGION", options.env) ||
getProviderEnvValue("AWS_DEFAULT_REGION", options.env) ||
undefined
);
}
function hasConfiguredBedrockProfile(): boolean {
if (typeof process === "undefined") {
return false;
function getConfiguredBedrockCredentials(env?: ProviderEnv): BedrockRuntimeClientConfig["credentials"] | undefined {
const accessKeyId = getProviderEnvValue("AWS_ACCESS_KEY_ID", env);
const secretAccessKey = getProviderEnvValue("AWS_SECRET_ACCESS_KEY", env);
if (!accessKeyId || !secretAccessKey) {
return undefined;
}
return Boolean(process.env.AWS_PROFILE);
const sessionToken = getProviderEnvValue("AWS_SESSION_TOKEN", env);
return {
accessKeyId,
secretAccessKey,
...(sessionToken ? { sessionToken } : {}),
};
}
function getStandardBedrockEndpointRegion(baseUrl: string | undefined): string | undefined {
@@ -938,14 +962,14 @@ function getStandardBedrockEndpointRegion(baseUrl: string | undefined): string |
function shouldUseExplicitBedrockEndpoint(
baseUrl: string,
configuredRegion: string | undefined,
hasConfiguredProfile: boolean,
hasAmbientConfiguredProfile: boolean,
): boolean {
const endpointRegion = getStandardBedrockEndpointRegion(baseUrl);
if (!endpointRegion) {
return true;
}
return !configuredRegion && !hasConfiguredProfile;
return !configuredRegion && !hasAmbientConfiguredProfile;
}
function isGovCloudBedrockTarget(model: Model<"bedrock-converse-stream">, options: BedrockOptions): boolean {
packages/ai/src/api/cloudflare.ts
+5 -4
View File
@@ -1,4 +1,5 @@
import type { Api, Model } from "../types.ts";
import type { Api, Model, ProviderEnv } from "../types.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
/** Workers AI direct endpoint. */
export const CLOUDFLARE_WORKERS_AI_BASE_URL =
@@ -20,12 +21,12 @@ export function isCloudflareProvider(provider: string): boolean {
return provider === "cloudflare-workers-ai" || provider === "cloudflare-ai-gateway";
}
/** Substitute `{VAR}` placeholders in a Cloudflare baseUrl from process.env. */
export function resolveCloudflareBaseUrl(model: Model<Api>): string {
/** Substitute `{VAR}` placeholders in a Cloudflare baseUrl from provider env or process.env. */
export function resolveCloudflareBaseUrl(model: Model<Api>, env?: ProviderEnv): string {
const url = model.baseUrl;
if (!url.includes("{")) return url;
const baseUrl = url.replace(/\{([A-Z_][A-Z0-9_]*)\}/g, (_match, name: string) => {
const value = process.env[name];
const value = getProviderEnvValue(name, env);
if (!value) {
throw new Error(`${name} is required for provider ${model.provider} but is not set.`);
}
packages/ai/src/api/google-generative-ai.ts
+2 -1
View File
@@ -406,7 +406,8 @@ function isGemini3ProModel(model: Model<"google-generative-ai">): boolean {
}
function isGemini3FlashModel(model: Model<"google-generative-ai">): boolean {
return /gemini-3(?:\.\d+)?-flash/.test(model.id.toLowerCase());
const id = model.id.toLowerCase();
return /gemini-3(?:\.\d+)?-flash/.test(id) || id === "gemini-flash-latest" || id === "gemini-flash-lite-latest";
}
function getDisabledThinkingConfig(model: Model<"google-generative-ai">): ThinkingConfig {
packages/ai/src/api/google-vertex.ts
+18 -4
View File
@@ -14,6 +14,7 @@ import type {
Context,
Model,
ThinkingLevel as PiThinkingLevel,
ProviderEnv,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
@@ -23,6 +24,7 @@ import type {
ToolCall,
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import type { GoogleThinkingLevel } from "./google-shared.ts";
import {
@@ -91,7 +93,7 @@ export const stream: StreamFunction<"google-vertex", GoogleVertexOptions> = (
// Create the client using either a Vertex API key, if provided, or ADC with project and location
const client = apiKey
? createClientWithApiKey(model, apiKey, options?.headers)
: createClient(model, resolveProject(options), resolveLocation(options), options?.headers);
: createClient(model, resolveProject(options), resolveLocation(options), options?.headers, options?.env);
let params = buildParams(model, context, options);
const nextParams = await options?.onPayload?.(params, model);
if (nextParams !== undefined) {
@@ -333,12 +335,15 @@ function createClient(
project: string,
location: string,
optionsHeaders?: Record<string, string>,
env?: ProviderEnv,
): GoogleGenAI {
const googleAuthOptions = buildGoogleAuthOptions(env);
return new GoogleGenAI({
vertexai: true,
project,
location,
apiVersion: API_VERSION,
...(googleAuthOptions ? { googleAuthOptions } : {}),
httpOptions: buildHttpOptions(model, optionsHeaders),
});
}
@@ -394,6 +399,11 @@ function baseUrlIncludesApiVersion(baseUrl: string): boolean {
}
}
function buildGoogleAuthOptions(env?: ProviderEnv): { keyFilename: string } | undefined {
const keyFilename = getProviderEnvValue("GOOGLE_APPLICATION_CREDENTIALS", env);
return keyFilename ? { keyFilename } : undefined;
}
function resolveApiKey(options?: GoogleVertexOptions): string | undefined {
const apiKey = options?.apiKey?.trim();
if (!apiKey || apiKey === GCP_VERTEX_CREDENTIALS_MARKER || isPlaceholderApiKey(apiKey)) {
@@ -407,7 +417,10 @@ function isPlaceholderApiKey(apiKey: string): boolean {
}
function resolveProject(options?: GoogleVertexOptions): string {
const project = options?.project || process.env.GOOGLE_CLOUD_PROJECT || process.env.GCLOUD_PROJECT;
const project =
options?.project ||
getProviderEnvValue("GOOGLE_CLOUD_PROJECT", options?.env) ||
getProviderEnvValue("GCLOUD_PROJECT", options?.env);
if (!project) {
throw new Error(
"Vertex AI requires a project ID. Set GOOGLE_CLOUD_PROJECT/GCLOUD_PROJECT or pass project in options.",
@@ -417,7 +430,7 @@ function resolveProject(options?: GoogleVertexOptions): string {
}
function resolveLocation(options?: GoogleVertexOptions): string {
const location = options?.location || process.env.GOOGLE_CLOUD_LOCATION;
const location = options?.location || getProviderEnvValue("GOOGLE_CLOUD_LOCATION", options?.env);
if (!location) {
throw new Error("Vertex AI requires a location. Set GOOGLE_CLOUD_LOCATION or pass location in options.");
}
@@ -490,7 +503,8 @@ function isGemini3ProModel(model: Model<"google-generative-ai">): boolean {
}
function isGemini3FlashModel(model: Model<"google-generative-ai">): boolean {
return /gemini-3(?:\.\d+)?-flash/.test(model.id.toLowerCase());
const id = model.id.toLowerCase();
return /gemini-3(?:\.\d+)?-flash/.test(id) || id === "gemini-flash-latest" || id === "gemini-flash-lite-latest";
}
function getDisabledThinkingConfig(model: Model<"google-vertex">): ThinkingConfig {
packages/ai/src/api/mistral-conversations.ts
+35 -4
View File
@@ -226,7 +226,7 @@ function buildRequestOptions(model: Model<"mistral-conversations">, options?: Mi
// Mistral infrastructure uses `x-affinity` for KV-cache reuse (prefix caching).
// Respect explicit caller-provided header values.
if (options?.sessionId && !headers["x-affinity"]) {
if (shouldUsePromptCaching(options) && !headers["x-affinity"]) {
headers["x-affinity"] = options.sessionId;
}
@@ -255,6 +255,7 @@ function buildChatPayload(
if (options?.toolChoice) payload.toolChoice = mapToolChoice(options.toolChoice);
if (options?.promptMode) payload.promptMode = options.promptMode;
if (options?.reasoningEffort) payload.reasoningEffort = options.reasoningEffort;
if (shouldUsePromptCaching(options)) payload.promptCacheKey = options.sessionId;
if (context.systemPrompt) {
payload.messages.unshift({
@@ -266,6 +267,31 @@ function buildChatPayload(
return payload;
}
function shouldUsePromptCaching(options?: MistralOptions): options is MistralOptions & { sessionId: string } {
return options?.cacheRetention !== "none" && !!options?.sessionId;
}
function getMistralCachedPromptTokens(usage: unknown, promptTokens: number): number {
const rawUsage = usage as {
promptTokensDetails?: { cachedTokens?: unknown } | null;
prompt_tokens_details?: { cached_tokens?: unknown } | null;
promptTokenDetails?: { cachedTokens?: unknown } | null;
prompt_token_details?: { cached_tokens?: unknown } | null;
numCachedTokens?: unknown;
num_cached_tokens?: unknown;
};
const rawCachedTokens =
rawUsage.promptTokensDetails?.cachedTokens ??
rawUsage.prompt_tokens_details?.cached_tokens ??
rawUsage.promptTokenDetails?.cachedTokens ??
rawUsage.prompt_token_details?.cached_tokens ??
rawUsage.numCachedTokens ??
rawUsage.num_cached_tokens ??
0;
const cachedTokens = typeof rawCachedTokens === "number" && Number.isFinite(rawCachedTokens) ? rawCachedTokens : 0;
return Math.min(promptTokens, Math.max(0, cachedTokens));
}
async function consumeChatStream(
model: Model<"mistral-conversations">,
output: AssistantMessage,
@@ -305,11 +331,16 @@ async function consumeChatStream(
output.responseId ||= chunk.id;
if (chunk.usage) {
output.usage.input = chunk.usage.promptTokens || 0;
const promptTokens = chunk.usage.promptTokens || 0;
const cachedPromptTokens = getMistralCachedPromptTokens(chunk.usage, promptTokens);
output.usage.input = Math.max(0, promptTokens - cachedPromptTokens);
output.usage.output = chunk.usage.completionTokens || 0;
output.usage.cacheRead = 0;
output.usage.cacheRead = cachedPromptTokens;
output.usage.cacheWrite = 0;
output.usage.totalTokens = chunk.usage.totalTokens || output.usage.input + output.usage.output;
output.usage.totalTokens =
chunk.usage.totalTokens ||
output.usage.input + output.usage.output + output.usage.cacheRead + output.usage.cacheWrite;
calculateCost(model, output.usage);
}
packages/ai/src/api/openai-codex-responses.ts
+25 -18
View File
@@ -27,6 +27,7 @@ import type {
AssistantMessage,
Context,
Model,
ProviderEnv,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
@@ -40,6 +41,7 @@ import {
} from "../utils/diagnostics.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { resolveHttpProxyUrlForTarget } from "../utils/node-http-proxy.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.ts";
import { buildBaseOptions } from "./simple-options.ts";
@@ -53,7 +55,9 @@ const JWT_CLAIM_PATH = "https://api.openai.com/auth" as const;
const DEFAULT_MAX_RETRIES = 0;
const BASE_DELAY_MS = 1000;
const DEFAULT_MAX_RETRY_DELAY_MS = 60_000;
const DEFAULT_SSE_HEADER_TIMEOUT_MS = 10_000;
// Keep a bounded pre-header timeout so zero-event Codex SSE stalls fail instead of
// leaving callers stuck on "Working..." indefinitely. See #4945.
const DEFAULT_SSE_HEADER_TIMEOUT_MS = 20_000;
const DEFAULT_WEBSOCKET_CONNECT_TIMEOUT_MS = 15_000;
const CODEX_TOOL_CALL_PROVIDERS = new Set(["openai", "openai-codex", "opencode"]);
const WEBSOCKET_MESSAGE_TOO_BIG_CLOSE_CODE = 1009;
@@ -812,19 +816,13 @@ type WebSocketConstructor = new (
) => WebSocketLike;
let _cachedWebsocket: WebSocketConstructor | null = null;
async function getWebSocketConstructor(): Promise<WebSocketConstructor | null> {
if (_cachedWebsocket) return _cachedWebsocket;
async function getWebSocketConstructor(env?: ProviderEnv): Promise<WebSocketConstructor | null> {
if (!env && _cachedWebsocket) return _cachedWebsocket;
// bun doesn't respect http proxy envs, ref: https://github.com/oven-sh/bun/issues/15489
// TODO: remove this when bun supports proxy envs in websocket.
if (
process?.versions?.bun &&
(process.env.HTTP_PROXY || process.env.HTTPS_PROXY || process.env.http_proxy || process.env.https_proxy)
) {
const m = await dynamicImport("proxy-from-env");
const getProxyForUrl = (m as { getProxyForUrl: (url: string | object | URL) => string }).getProxyForUrl;
_cachedWebsocket = class extends WebSocket {
if (typeof process !== "undefined" && process.versions?.bun) {
const WebSocketWithProxy = class extends WebSocket {
constructor(url: string | URL, options?: string | string[] | Record<string, unknown>) {
let _opts: Record<string, unknown> = {};
if (Array.isArray(options) || typeof options === "string") {
@@ -833,11 +831,17 @@ async function getWebSocketConstructor(): Promise<WebSocketConstructor | null> {
_opts = { ...options };
}
const proxy = getProxyForUrl(url.toString().replace(/^wss:/, "https:").replace(/^ws:/, "http:"));
super(url, { ..._opts, ...(proxy ? { proxy } : {}) } as any);
const proxyUrl = resolveHttpProxyUrlForTarget(
url.toString().replace(/^wss:/, "https:").replace(/^ws:/, "http:"),
env,
);
super(url, { ..._opts, ...(proxyUrl ? { proxy: proxyUrl.toString() } : {}) } as any);
}
};
return _cachedWebsocket;
if (!env) {
_cachedWebsocket = WebSocketWithProxy;
}
return WebSocketWithProxy;
}
const ctor = (globalThis as { WebSocket?: unknown }).WebSocket;
@@ -892,8 +896,9 @@ async function connectWebSocket(
headers: Headers,
signal?: AbortSignal,
connectTimeoutMs = DEFAULT_WEBSOCKET_CONNECT_TIMEOUT_MS,
env?: ProviderEnv,
): Promise<WebSocketLike> {
const WebSocketCtor = await getWebSocketConstructor();
const WebSocketCtor = await getWebSocketConstructor(env);
if (!WebSocketCtor) {
throw new Error("WebSocket transport is not available in this runtime");
}
@@ -970,6 +975,7 @@ async function acquireWebSocket(
sessionId: string | undefined,
signal?: AbortSignal,
connectTimeoutMs?: number,
env?: ProviderEnv,
): Promise<{
socket: WebSocketLike;
entry?: CachedWebSocketConnection;
@@ -977,7 +983,7 @@ async function acquireWebSocket(
release: (options?: { keep?: boolean }) => void;
}> {
if (!sessionId) {
const socket = await connectWebSocket(url, headers, signal, connectTimeoutMs);
const socket = await connectWebSocket(url, headers, signal, connectTimeoutMs, env);
return {
socket,
reused: false,
@@ -1009,7 +1015,7 @@ async function acquireWebSocket(
};
}
if (cached.busy) {
const socket = await connectWebSocket(url, headers, signal, connectTimeoutMs);
const socket = await connectWebSocket(url, headers, signal, connectTimeoutMs, env);
return {
socket,
reused: false,
@@ -1024,7 +1030,7 @@ async function acquireWebSocket(
}
}
const socket = await connectWebSocket(url, headers, signal, connectTimeoutMs);
const socket = await connectWebSocket(url, headers, signal, connectTimeoutMs, env);
const entry: CachedWebSocketConnection = { socket, busy: true };
websocketSessionCache.set(sessionId, entry);
return {
@@ -1310,6 +1316,7 @@ async function processWebSocketStream(
options?.sessionId,
options?.signal,
websocketConnectTimeoutMs,
options?.env,
);
let keepConnection = true;
const useCachedContext = options?.transport === "websocket-cached" || options?.transport === "auto";
packages/ai/src/api/openai-completions.ts
+113 -15
View File
@@ -14,11 +14,13 @@ import { calculateCost, clampThinkingLevel } from "../models.ts";
import type {
AssistantMessage,
CacheRetention,
ChatTemplateKwargValue,
Context,
ImageContent,
Message,
Model,
OpenAICompletionsCompat,
ProviderEnv,
SimpleStreamOptions,
StopReason,
StreamFunction,
@@ -32,6 +34,7 @@ import type {
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { parseStreamingJson } from "../utils/json-parse.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
@@ -74,6 +77,20 @@ function isImageContentBlock(block: { type: string }): block is ImageContent {
return block.type === "image";
}
function isEncryptedReasoningDetail(detail: unknown): detail is OpenAIEncryptedReasoningDetail {
if (typeof detail !== "object" || detail === null) {
return false;
}
const candidate = detail as Record<string, unknown>;
return (
candidate.type === "reasoning.encrypted" &&
typeof candidate.id === "string" &&
candidate.id.length > 0 &&
typeof candidate.data === "string" &&
candidate.data.length > 0
);
}
export interface OpenAICompletionsOptions extends StreamOptions {
toolChoice?: "auto" | "none" | "required" | { type: "function"; function: { name: string } };
reasoningEffort?: "minimal" | "low" | "medium" | "high" | "xhigh";
@@ -88,8 +105,16 @@ type ResolvedOpenAICompletionsCompat = Omit<Required<OpenAICompletionsCompat>, "
cacheControlFormat?: OpenAICompletionsCompat["cacheControlFormat"];
};
type ResolvedChatTemplateKwargValue = string | number | boolean | null;
type ChatCompletionInstructionMessageParam = ChatCompletionDeveloperMessageParam | ChatCompletionSystemMessageParam;
type OpenAIEncryptedReasoningDetail = {
type: "reasoning.encrypted";
id: string;
data: string;
};
type ChatCompletionTextPartWithCacheControl = ChatCompletionContentPartText & {
cache_control?: OpenAICompatCacheControl;
};
@@ -98,11 +123,11 @@ type ChatCompletionToolWithCacheControl = OpenAI.Chat.Completions.ChatCompletion
cache_control?: OpenAICompatCacheControl;
};
function resolveCacheRetention(cacheRetention?: CacheRetention): CacheRetention {
function resolveCacheRetention(cacheRetention?: CacheRetention, env?: ProviderEnv): CacheRetention {
if (cacheRetention) {
return cacheRetention;
}
if (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long") {
if (getProviderEnvValue("PI_CACHE_RETENTION", env) === "long") {
return "long";
}
return "short";
@@ -140,9 +165,9 @@ export const stream: StreamFunction<"openai-completions", OpenAICompletionsOptio
throw new Error(`No API key for provider: ${model.provider}`);
}
const compat = getCompat(model);
const cacheRetention = resolveCacheRetention(options?.cacheRetention);
const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, compat);
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, compat, options?.env);
let params = buildParams(model, context, options, compat, cacheRetention);
const nextParams = await options?.onPayload?.(params, model);
if (nextParams !== undefined) {
@@ -171,6 +196,7 @@ export const stream: StreamFunction<"openai-completions", OpenAICompletionsOptio
let hasFinishReason = false;
const toolCallBlocksByIndex = new Map<number, StreamingToolCallBlock>();
const toolCallBlocksById = new Map<string, StreamingToolCallBlock>();
const pendingReasoningDetailsByToolCallId = new Map<string, string>();
const blocks = output.content as StreamingBlock[];
const getContentIndex = (block: StreamingBlock) => blocks.indexOf(block);
const finishBlock = (block: StreamingBlock) => {
@@ -226,6 +252,16 @@ export const stream: StreamFunction<"openai-completions", OpenAICompletionsOptio
}
return thinkingBlock;
};
const applyPendingReasoningDetail = (block: StreamingToolCallBlock) => {
if (!block.id) {
return;
}
const pendingReasoningDetail = pendingReasoningDetailsByToolCallId.get(block.id);
if (pendingReasoningDetail) {
block.thoughtSignature = pendingReasoningDetail;
pendingReasoningDetailsByToolCallId.delete(block.id);
}
};
const ensureToolCallBlock = (toolCall: StreamingToolCallDelta) => {
const streamIndex = typeof toolCall.index === "number" ? toolCall.index : undefined;
let block = streamIndex !== undefined ? toolCallBlocksByIndex.get(streamIndex) : undefined;
@@ -261,6 +297,7 @@ export const stream: StreamFunction<"openai-completions", OpenAICompletionsOptio
if (toolCall.id) {
toolCallBlocksById.set(toolCall.id, block);
}
applyPendingReasoningDetail(block);
return block;
};
@@ -370,15 +407,16 @@ export const stream: StreamFunction<"openai-completions", OpenAICompletionsOptio
}
}
const reasoningDetails = (choice.delta as any).reasoning_details;
if (reasoningDetails && Array.isArray(reasoningDetails)) {
const reasoningDetails = (choice.delta as { reasoning_details?: unknown }).reasoning_details;
if (Array.isArray(reasoningDetails)) {
for (const detail of reasoningDetails) {
if (detail.type === "reasoning.encrypted" && detail.id && detail.data) {
const matchingToolCall = output.content.find(
(b) => b.type === "toolCall" && b.id === detail.id,
) as ToolCall | undefined;
if (isEncryptedReasoningDetail(detail)) {
const serializedDetail = JSON.stringify(detail);
const matchingToolCall = toolCallBlocksById.get(detail.id);
if (matchingToolCall) {
matchingToolCall.thoughtSignature = JSON.stringify(detail);
matchingToolCall.thoughtSignature = serializedDetail;
} else {
pendingReasoningDetailsByToolCallId.set(detail.id, serializedDetail);
}
}
}
@@ -454,6 +492,7 @@ function createClient(
optionsHeaders?: Record<string, string>,
sessionId?: string,
compat: ResolvedOpenAICompletionsCompat = getCompat(model),
env?: ProviderEnv,
) {
const headers = { ...model.headers };
if (model.provider === "github-copilot") {
@@ -487,7 +526,7 @@ function createClient(
return new OpenAI({
apiKey,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model) : model.baseUrl,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model, env) : model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders,
});
@@ -498,7 +537,7 @@ function buildParams(
context: Context,
options?: OpenAICompletionsOptions,
compat: ResolvedOpenAICompletionsCompat = getCompat(model),
cacheRetention: CacheRetention = resolveCacheRetention(options?.cacheRetention),
cacheRetention: CacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env),
) {
const messages = convertMessages(model, context, compat);
const cacheControl = getCompatCacheControl(compat, cacheRetention);
@@ -554,8 +593,18 @@ function buildParams(
}
if (compat.thinkingFormat === "zai" && model.reasoning) {
const zaiParams = params as typeof params & { thinking?: { type: "enabled" | "disabled" } };
const zaiParams = params as Omit<typeof params, "reasoning_effort"> & {
thinking?: { type: "enabled" | "disabled" };
reasoning_effort?: string;
};
zaiParams.thinking = { type: options?.reasoningEffort ? "enabled" : "disabled" };
if (options?.reasoningEffort && compat.supportsReasoningEffort) {
const mappedEffort = model.thinkingLevelMap?.[options.reasoningEffort];
const effort = mappedEffort === undefined ? options.reasoningEffort : mappedEffort;
if (typeof effort === "string") {
zaiParams.reasoning_effort = effort;
}
}
} else if (compat.thinkingFormat === "qwen" && model.reasoning) {
(params as any).enable_thinking = !!options?.reasoningEffort;
} else if (compat.thinkingFormat === "qwen-chat-template" && model.reasoning) {
@@ -563,8 +612,17 @@ function buildParams(
enable_thinking: !!options?.reasoningEffort,
preserve_thinking: true,
};
} else if (compat.thinkingFormat === "chat-template" && model.reasoning) {
const chatTemplateKwargs = buildChatTemplateKwargs(model, options, compat);
if (chatTemplateKwargs) {
(params as any).chat_template_kwargs = chatTemplateKwargs;
}
} else if (compat.thinkingFormat === "deepseek" && model.reasoning) {
(params as any).thinking = { type: options?.reasoningEffort ? "enabled" : "disabled" };
if (options?.reasoningEffort) {
(params as any).thinking = { type: "enabled" };
} else if (model.thinkingLevelMap?.off !== null) {
(params as any).thinking = { type: "disabled" };
}
if (options?.reasoningEffort && compat.supportsReasoningEffort) {
(params as any).reasoning_effort =
model.thinkingLevelMap?.[options.reasoningEffort] ?? options.reasoningEffort;
@@ -629,6 +687,44 @@ function buildParams(
return params;
}
function buildChatTemplateKwargs(
model: Model<"openai-completions">,
options: OpenAICompletionsOptions | undefined,
compat: ResolvedOpenAICompletionsCompat,
): Record<string, ResolvedChatTemplateKwargValue> | undefined {
const kwargs: Record<string, ResolvedChatTemplateKwargValue> = {};
for (const [key, value] of Object.entries(compat.chatTemplateKwargs)) {
const resolved = resolveChatTemplateKwargValue(model, options, value);
if (resolved !== undefined) {
kwargs[key] = resolved;
}
}
return Object.keys(kwargs).length > 0 ? kwargs : undefined;
}
function resolveChatTemplateKwargValue(
model: Model<"openai-completions">,
options: OpenAICompletionsOptions | undefined,
value: ChatTemplateKwargValue,
): ResolvedChatTemplateKwargValue | undefined {
if (typeof value !== "object" || value === null) {
return value;
}
const reasoningEffort = options?.reasoningEffort;
if (!reasoningEffort && value.omitWhenOff) {
return undefined;
}
if (value.$var === "thinking.enabled") {
return !!reasoningEffort;
}
const mappedValue = reasoningEffort ? model.thinkingLevelMap?.[reasoningEffort] : model.thinkingLevelMap?.off;
return mappedValue === undefined ? reasoningEffort : typeof mappedValue === "string" ? mappedValue : undefined;
}
function getCompatCacheControl(
compat: ResolvedOpenAICompletionsCompat,
cacheRetention: CacheRetention,
@@ -1141,6 +1237,7 @@ function detectCompat(model: Model<"openai-completions">): ResolvedOpenAIComplet
: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia,
cacheControlFormat,
@@ -1179,6 +1276,7 @@ function getCompat(model: Model<"openai-completions">): ResolvedOpenAICompletion
thinkingFormat: model.compat.thinkingFormat ?? detected.thinkingFormat,
openRouterRouting: model.compat.openRouterRouting ?? {},
vercelGatewayRouting: model.compat.vercelGatewayRouting ?? detected.vercelGatewayRouting,
chatTemplateKwargs: model.compat.chatTemplateKwargs ?? detected.chatTemplateKwargs,
zaiToolStream: model.compat.zaiToolStream ?? detected.zaiToolStream,
supportsStrictMode: model.compat.supportsStrictMode ?? detected.supportsStrictMode,
cacheControlFormat: model.compat.cacheControlFormat ?? detected.cacheControlFormat,
packages/ai/src/api/openai-responses-shared.ts
+2 -1
View File
@@ -456,7 +456,8 @@ export async function processResponsesStream<TApi extends Api>(
});
currentBlock = null;
} else if (item.type === "message" && currentBlock?.type === "text") {
currentBlock.text = item.content.map((c) => (c.type === "output_text" ? c.text : c.refusal)).join("");
currentBlock.text =
item.content?.map((c) => (c.type === "output_text" ? c.text : c.refusal)).join("") || "";
currentBlock.textSignature = encodeTextSignatureV1(item.id, item.phase ?? undefined);
stream.push({
type: "text_end",
packages/ai/src/api/openai-responses.ts
+9 -6
View File
@@ -8,6 +8,7 @@ import type {
Context,
Model,
OpenAIResponsesCompat,
ProviderEnv,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
@@ -15,6 +16,7 @@ import type {
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
@@ -27,11 +29,11 @@ const OPENAI_TOOL_CALL_PROVIDERS = new Set(["openai", "openai-codex", "opencode"
* Resolve cache retention preference.
* Defaults to "short" and uses PI_CACHE_RETENTION for backward compatibility.
*/
function resolveCacheRetention(cacheRetention?: CacheRetention): CacheRetention {
function resolveCacheRetention(cacheRetention?: CacheRetention, env?: ProviderEnv): CacheRetention {
if (cacheRetention) {
return cacheRetention;
}
if (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long") {
if (getProviderEnvValue("PI_CACHE_RETENTION", env) === "long") {
return "long";
}
return "short";
@@ -111,9 +113,9 @@ export const stream: StreamFunction<"openai-responses", OpenAIResponsesOptions>
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const cacheRetention = resolveCacheRetention(options?.cacheRetention);
const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId);
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, options?.env);
let params = buildParams(model, context, options);
const nextParams = await options?.onPayload?.(params, model);
if (nextParams !== undefined) {
@@ -185,6 +187,7 @@ function createClient(
apiKey: string,
optionsHeaders?: Record<string, string>,
sessionId?: string,
env?: ProviderEnv,
) {
const compat = getCompat(model);
const headers = { ...model.headers };
@@ -220,7 +223,7 @@ function createClient(
return new OpenAI({
apiKey,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model) : model.baseUrl,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model, env) : model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders,
});
@@ -229,7 +232,7 @@ function createClient(
function buildParams(model: Model<"openai-responses">, context: Context, options?: OpenAIResponsesOptions) {
const messages = convertResponsesMessages(model, context, OPENAI_TOOL_CALL_PROVIDERS);
const cacheRetention = resolveCacheRetention(options?.cacheRetention);
const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
const compat = getCompat(model);
const params: ResponseCreateParamsStreaming = {
model: model.id,
packages/ai/src/api/simple-options.ts
+1
View File
@@ -17,6 +17,7 @@ export function buildBaseOptions(_model: Model<Api>, options?: SimpleStreamOptio
maxRetries: options?.maxRetries,
maxRetryDelayMs: options?.maxRetryDelayMs,
metadata: options?.metadata,
env: options?.env,
};
}
packages/ai/src/env-api-keys.ts
+27 -63
View File
@@ -23,44 +23,17 @@ if (typeof process !== "undefined" && (process.versions?.node || process.version
});
}
import type { KnownProvider } from "./types.ts";
let _procEnvCache: Map<string, string> | null = null;
/**
* Fallback for https://github.com/oven-sh/bun/issues/27802
* Bun compiled binaries have an empty `process.env` inside sandbox
* environments on Linux. We can recover the env from `/proc/self/environ`.
*/
function getProcEnv(key: string): string | undefined {
if (!process.versions?.bun) return undefined;
if (typeof process === "undefined") return undefined;
// If process.env already has entries, the bug is not triggered.
if (Object.keys(process.env).length > 0) return undefined;
if (_procEnvCache === null) {
_procEnvCache = new Map();
try {
const { readFileSync } = require("node:fs") as typeof import("node:fs");
const data = readFileSync("/proc/self/environ", "utf-8");
for (const entry of data.split("\0")) {
const idx = entry.indexOf("=");
if (idx > 0) {
_procEnvCache.set(entry.slice(0, idx), entry.slice(idx + 1));
}
}
} catch {
// /proc/self/environ may not be readable.
}
}
return _procEnvCache.get(key);
}
import type { KnownProvider, ProviderEnv } from "./types.ts";
import { getProviderEnvValue } from "./utils/provider-env.ts";
let cachedVertexAdcCredentialsExists: boolean | null = null;
function hasVertexAdcCredentials(): boolean {
function hasVertexAdcCredentials(env?: ProviderEnv): boolean {
const explicitCredentialsPath = env?.GOOGLE_APPLICATION_CREDENTIALS;
if (explicitCredentialsPath) {
return _existsSync ? _existsSync(explicitCredentialsPath) : false;
}
if (cachedVertexAdcCredentialsExists === null) {
// If node modules haven't loaded yet (async import race at startup),
// return false WITHOUT caching so the next call retries once they're ready.
@@ -75,7 +48,7 @@ function hasVertexAdcCredentials(): boolean {
}
// Check GOOGLE_APPLICATION_CREDENTIALS env var first (standard way)
const gacPath = process.env.GOOGLE_APPLICATION_CREDENTIALS || getProcEnv("GOOGLE_APPLICATION_CREDENTIALS");
const gacPath = getProviderEnvValue("GOOGLE_APPLICATION_CREDENTIALS", env);
if (gacPath) {
cachedVertexAdcCredentialsExists = _existsSync(gacPath);
} else {
@@ -143,13 +116,13 @@ function getApiKeyEnvVars(provider: string): readonly string[] | undefined {
* credential sources such as AWS profiles, AWS IAM credentials, and Google
* Application Default Credentials.
*/
export function findEnvKeys(provider: KnownProvider): string[] | undefined;
export function findEnvKeys(provider: string): string[] | undefined;
export function findEnvKeys(provider: string): string[] | undefined {
export function findEnvKeys(provider: KnownProvider, env?: ProviderEnv): string[] | undefined;
export function findEnvKeys(provider: string, env?: ProviderEnv): string[] | undefined;
export function findEnvKeys(provider: string, env?: ProviderEnv): string[] | undefined {
const envVars = getApiKeyEnvVars(provider);
if (!envVars) return undefined;
const found = envVars.filter((envVar) => !!process.env[envVar] || !!getProcEnv(envVar));
const found = envVars.filter((envVar) => !!getProviderEnvValue(envVar, env));
return found.length > 0 ? found : undefined;
}
@@ -158,25 +131,22 @@ export function findEnvKeys(provider: string): string[] | undefined {
*
* Will not return API keys for providers that require OAuth tokens.
*/
export function getEnvApiKey(provider: KnownProvider): string | undefined;
export function getEnvApiKey(provider: string): string | undefined;
export function getEnvApiKey(provider: string): string | undefined {
const envKeys = findEnvKeys(provider);
export function getEnvApiKey(provider: KnownProvider, env?: ProviderEnv): string | undefined;
export function getEnvApiKey(provider: string, env?: ProviderEnv): string | undefined;
export function getEnvApiKey(provider: string, env?: ProviderEnv): string | undefined {
const envKeys = findEnvKeys(provider, env);
if (envKeys?.[0]) {
return process.env[envKeys[0]] || getProcEnv(envKeys[0]);
return getProviderEnvValue(envKeys[0], env);
}
// Vertex AI supports either an explicit API key or Application Default Credentials.
// Auth is configured via `gcloud auth application-default login`.
if (provider === "google-vertex") {
const hasCredentials = hasVertexAdcCredentials();
const hasCredentials = hasVertexAdcCredentials(env);
const hasProject = !!(
process.env.GOOGLE_CLOUD_PROJECT ||
process.env.GCLOUD_PROJECT ||
getProcEnv("GOOGLE_CLOUD_PROJECT") ||
getProcEnv("GCLOUD_PROJECT")
getProviderEnvValue("GOOGLE_CLOUD_PROJECT", env) || getProviderEnvValue("GCLOUD_PROJECT", env)
);
const hasLocation = !!(process.env.GOOGLE_CLOUD_LOCATION || getProcEnv("GOOGLE_CLOUD_LOCATION"));
const hasLocation = !!getProviderEnvValue("GOOGLE_CLOUD_LOCATION", env);
if (hasCredentials && hasProject && hasLocation) {
return "<authenticated>";
@@ -192,18 +162,12 @@ export function getEnvApiKey(provider: string): string | undefined {
// 5. AWS_CONTAINER_CREDENTIALS_FULL_URI - ECS task roles (full URI)
// 6. AWS_WEB_IDENTITY_TOKEN_FILE - IRSA (IAM Roles for Service Accounts)
if (
process.env.AWS_PROFILE ||
(process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY) ||
process.env.AWS_BEARER_TOKEN_BEDROCK ||
process.env.AWS_CONTAINER_CREDENTIALS_RELATIVE_URI ||
process.env.AWS_CONTAINER_CREDENTIALS_FULL_URI ||
process.env.AWS_WEB_IDENTITY_TOKEN_FILE ||
getProcEnv("AWS_PROFILE") ||
(getProcEnv("AWS_ACCESS_KEY_ID") && getProcEnv("AWS_SECRET_ACCESS_KEY")) ||
getProcEnv("AWS_BEARER_TOKEN_BEDROCK") ||
getProcEnv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI") ||
getProcEnv("AWS_CONTAINER_CREDENTIALS_FULL_URI") ||
getProcEnv("AWS_WEB_IDENTITY_TOKEN_FILE")
getProviderEnvValue("AWS_PROFILE", env) ||
(getProviderEnvValue("AWS_ACCESS_KEY_ID", env) && getProviderEnvValue("AWS_SECRET_ACCESS_KEY", env)) ||
getProviderEnvValue("AWS_BEARER_TOKEN_BEDROCK", env) ||
getProviderEnvValue("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI", env) ||
getProviderEnvValue("AWS_CONTAINER_CREDENTIALS_FULL_URI", env) ||
getProviderEnvValue("AWS_WEB_IDENTITY_TOKEN_FILE", env)
) {
return "<authenticated>";
}
packages/ai/src/image-models.generated.ts
+30
View File
@@ -95,6 +95,21 @@ export const IMAGE_MODELS = {
cacheWrite: 0.08333333333333334,
},
} satisfies ImagesModel<"openrouter-images">,
"google/gemini-3-pro-image": {
id: "google/gemini-3-pro-image",
name: "Google: Nano Banana Pro (Gemini 3 Pro Image)",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["image", "text"],
output: ["image", "text"],
cost: {
input: 2,
output: 12,
cacheRead: 0.19999999999999998,
cacheWrite: 0.375,
},
} satisfies ImagesModel<"openrouter-images">,
"google/gemini-3-pro-image-preview": {
id: "google/gemini-3-pro-image-preview",
name: "Google: Nano Banana Pro (Gemini 3 Pro Image Preview)",
@@ -110,6 +125,21 @@ export const IMAGE_MODELS = {
cacheWrite: 0.375,
},
} satisfies ImagesModel<"openrouter-images">,
"google/gemini-3.1-flash-image": {
id: "google/gemini-3.1-flash-image",
name: "Google: Nano Banana 2 (Gemini 3.1 Flash Image)",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["image", "text"],
output: ["image", "text"],
cost: {
input: 0.5,
output: 3,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"google/gemini-3.1-flash-image-preview": {
id: "google/gemini-3.1-flash-image-preview",
name: "Google: Nano Banana 2 (Gemini 3.1 Flash Image Preview)",
packages/ai/src/models.ts
+4 -1
View File
@@ -372,10 +372,13 @@ export function hasApi<TApi extends Api>(model: Model<Api>, api: TApi): model is
}
export function calculateCost<TApi extends Api>(model: Model<TApi>, usage: Usage): Usage["cost"] {
// Anthropic charges 2x base input for 1h cache writes.
const longWrite = usage.cacheWrite1h ?? 0;
const shortWrite = usage.cacheWrite - longWrite;
usage.cost.input = (model.cost.input / 1000000) * usage.input;
usage.cost.output = (model.cost.output / 1000000) * usage.output;
usage.cost.cacheRead = (model.cost.cacheRead / 1000000) * usage.cacheRead;
usage.cost.cacheWrite = (model.cost.cacheWrite / 1000000) * usage.cacheWrite;
usage.cost.cacheWrite = (model.cost.cacheWrite * shortWrite + model.cost.input * 2 * longWrite) / 1000000;
usage.cost.total = usage.cost.input + usage.cost.output + usage.cost.cacheRead + usage.cost.cacheWrite;
return usage.cost;
}
packages/ai/src/providers/cerebras.models.ts
+5 -22
View File
@@ -13,30 +13,13 @@ export const CEREBRAS_MODELS = {
reasoning: true,
input: ["text"],
cost: {
input: 0.25,
output: 0.69,
input: 0.35,
output: 0.75,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"llama3.1-8b": {
id: "llama3.1-8b",
name: "Llama 3.1 8B",
api: "openai-completions",
provider: "cerebras",
baseUrl: "https://api.cerebras.ai/v1",
reasoning: false,
input: ["text"],
cost: {
input: 0.1,
output: 0.1,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 32000,
maxTokens: 8000,
maxTokens: 40960,
} satisfies Model<"openai-completions">,
"zai-glm-4.7": {
id: "zai-glm-4.7",
@@ -44,7 +27,7 @@ export const CEREBRAS_MODELS = {
api: "openai-completions",
provider: "cerebras",
baseUrl: "https://api.cerebras.ai/v1",
reasoning: false,
reasoning: true,
input: ["text"],
cost: {
input: 2.25,
@@ -53,6 +36,6 @@ export const CEREBRAS_MODELS = {
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 40000,
maxTokens: 40960,
} satisfies Model<"openai-completions">,
} as const;
packages/ai/src/providers/cloudflare-workers-ai.models.ts
+36
View File
@@ -112,6 +112,24 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
contextWindow: 262144,
maxTokens: 256000,
} satisfies Model<"openai-completions">,
"@cf/moonshotai/kimi-k2.7-code": {
id: "@cf/moonshotai/kimi-k2.7-code",
name: "Kimi K2.7 Code",
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"@cf/nvidia/nemotron-3-120b-a12b": {
id: "@cf/nvidia/nemotron-3-120b-a12b",
name: "Nemotron 3 Super 120B",
@@ -202,4 +220,22 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
contextWindow: 131072,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"@cf/zai-org/glm-5.2": {
id: "@cf/zai-org/glm-5.2",
name: "Glm 5.2",
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
input: 1.4,
output: 4.4,
cacheRead: 0.26,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
} as const;
packages/ai/src/providers/fireworks.models.ts
+71 -34
View File
@@ -16,7 +16,7 @@ export const FIREWORKS_MODELS = {
cost: {
input: 0.14,
output: 0.28,
cacheRead: 0.03,
cacheRead: 0.028,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -58,6 +58,25 @@ export const FIREWORKS_MODELS = {
contextWindow: 202800,
maxTokens: 131072,
} satisfies Model<"anthropic-messages">,
"accounts/fireworks/models/glm-5p2": {
id: "accounts/fireworks/models/glm-5p2",
name: "GLM 5.2",
api: "openai-completions",
provider: "fireworks",
baseUrl: "https://api.fireworks.ai/inference/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false},
reasoning: true,
thinkingLevelMap: {"off":"none","minimal":null,"low":"high","medium":"high","xhigh":"max"},
input: ["text"],
cost: {
input: 1.4,
output: 4.4,
cacheRead: 0.26,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"accounts/fireworks/models/gpt-oss-120b": {
id: "accounts/fireworks/models/gpt-oss-120b",
name: "GPT OSS 120B",
@@ -94,24 +113,6 @@ export const FIREWORKS_MODELS = {
contextWindow: 131072,
maxTokens: 32768,
} satisfies Model<"anthropic-messages">,
"accounts/fireworks/models/kimi-k2p5": {
id: "accounts/fireworks/models/kimi-k2p5",
name: "Kimi K2.5",
api: "anthropic-messages",
provider: "fireworks",
baseUrl: "https://api.fireworks.ai/inference",
compat: {"sendSessionAffinityHeaders":true,"supportsEagerToolInputStreaming":false,"supportsCacheControlOnTools":false,"supportsLongCacheRetention":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.6,
output: 3,
cacheRead: 0.1,
cacheWrite: 0,
},
contextWindow: 256000,
maxTokens: 256000,
} satisfies Model<"anthropic-messages">,
"accounts/fireworks/models/kimi-k2p6": {
id: "accounts/fireworks/models/kimi-k2p6",
name: "Kimi K2.6",
@@ -130,23 +131,23 @@ export const FIREWORKS_MODELS = {
contextWindow: 262000,
maxTokens: 262000,
} satisfies Model<"anthropic-messages">,
"accounts/fireworks/models/minimax-m2p5": {
id: "accounts/fireworks/models/minimax-m2p5",
name: "MiniMax-M2.5",
"accounts/fireworks/models/kimi-k2p7-code": {
id: "accounts/fireworks/models/kimi-k2p7-code",
name: "Kimi K2.7 Code",
api: "anthropic-messages",
provider: "fireworks",
baseUrl: "https://api.fireworks.ai/inference",
compat: {"sendSessionAffinityHeaders":true,"supportsEagerToolInputStreaming":false,"supportsCacheControlOnTools":false,"supportsLongCacheRetention":false},
reasoning: true,
input: ["text"],
input: ["text", "image"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0.03,
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
contextWindow: 196608,
maxTokens: 196608,
contextWindow: 262000,
maxTokens: 262000,
} satisfies Model<"anthropic-messages">,
"accounts/fireworks/models/minimax-m2p7": {
id: "accounts/fireworks/models/minimax-m2p7",
@@ -166,9 +167,27 @@ export const FIREWORKS_MODELS = {
contextWindow: 196608,
maxTokens: 196608,
} satisfies Model<"anthropic-messages">,
"accounts/fireworks/models/qwen3p6-plus": {
id: "accounts/fireworks/models/qwen3p6-plus",
name: "Qwen 3.6 Plus",
"accounts/fireworks/models/minimax-m3": {
id: "accounts/fireworks/models/minimax-m3",
name: "MiniMax-M3",
api: "anthropic-messages",
provider: "fireworks",
baseUrl: "https://api.fireworks.ai/inference",
compat: {"sendSessionAffinityHeaders":true,"supportsEagerToolInputStreaming":false,"supportsCacheControlOnTools":false,"supportsLongCacheRetention":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0.06,
cacheWrite: 0,
},
contextWindow: 512000,
maxTokens: 512000,
} satisfies Model<"anthropic-messages">,
"accounts/fireworks/models/qwen3p7-plus": {
id: "accounts/fireworks/models/qwen3p7-plus",
name: "Qwen 3.7 Plus",
api: "anthropic-messages",
provider: "fireworks",
baseUrl: "https://api.fireworks.ai/inference",
@@ -176,9 +195,9 @@ export const FIREWORKS_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.5,
output: 3,
cacheRead: 0.1,
input: 0.4,
output: 1.6,
cacheRead: 0.08,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -238,4 +257,22 @@ export const FIREWORKS_MODELS = {
contextWindow: 262000,
maxTokens: 262000,
} satisfies Model<"anthropic-messages">,
"accounts/fireworks/routers/kimi-k2p7-code-fast": {
id: "accounts/fireworks/routers/kimi-k2p7-code-fast",
name: "Kimi K2.7 Code Fast",
api: "anthropic-messages",
provider: "fireworks",
baseUrl: "https://api.fireworks.ai/inference",
compat: {"sendSessionAffinityHeaders":true,"supportsEagerToolInputStreaming":false,"supportsCacheControlOnTools":false,"supportsLongCacheRetention":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 1.9,
output: 8,
cacheRead: 0.38,
cacheWrite: 0,
},
contextWindow: 262000,
maxTokens: 262000,
} satisfies Model<"anthropic-messages">,
} as const;
packages/ai/src/providers/fireworks.ts
+6 -2
View File
@@ -1,15 +1,19 @@
import { anthropicMessagesApi } from "../api/anthropic-messages.lazy.ts";
import { openAICompletionsApi } from "../api/openai-completions.lazy.ts";
import { envApiKeyAuth } from "../auth/helpers.ts";
import { createProvider, type Provider } from "../models.ts";
import { FIREWORKS_MODELS } from "./fireworks.models.ts";
export function fireworksProvider(): Provider<"anthropic-messages"> {
export function fireworksProvider(): Provider<"anthropic-messages" | "openai-completions"> {
return createProvider({
id: "fireworks",
name: "Fireworks",
baseUrl: "https://api.fireworks.ai/inference",
auth: { apiKey: envApiKeyAuth("Fireworks API key", ["FIREWORKS_API_KEY"]) },
models: Object.values(FIREWORKS_MODELS),
api: anthropicMessagesApi(),
api: {
"anthropic-messages": anthropicMessagesApi(),
"openai-completions": openAICompletionsApi(),
},
});
}
packages/ai/src/providers/github-copilot.models.ts
+22 -21
View File
@@ -4,6 +4,25 @@
import type { Model } from "../types.ts";
export const GITHUB_COPILOT_MODELS = {
"claude-fable-5": {
id: "claude-fable-5",
name: "Claude Fable 5",
api: "openai-completions",
provider: "github-copilot",
baseUrl: "https://api.individual.githubcopilot.com",
headers: {"User-Agent":"GitHubCopilotChat/0.35.0","Editor-Version":"vscode/1.107.0","Editor-Plugin-Version":"copilot-chat/0.35.0","Copilot-Integration-Id":"vscode-chat"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 10,
output: 50,
cacheRead: 1,
cacheWrite: 12.5,
},
contextWindow: 1000000,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
"claude-haiku-4.5": {
id: "claude-haiku-4.5",
name: "Claude Haiku 4.5 (latest)",
@@ -70,7 +89,7 @@ export const GITHUB_COPILOT_MODELS = {
headers: {"User-Agent":"GitHubCopilotChat/0.35.0","Editor-Version":"vscode/1.107.0","Editor-Plugin-Version":"copilot-chat/0.35.0","Copilot-Integration-Id":"vscode-chat"},
compat: {"forceAdaptiveThinking":true,"supportsTemperature":false},
reasoning: true,
thinkingLevelMap: {"xhigh":"xhigh"},
thinkingLevelMap: {"xhigh":"xhigh","minimal":"low"},
input: ["text", "image"],
cost: {
input: 5,
@@ -90,7 +109,7 @@ export const GITHUB_COPILOT_MODELS = {
headers: {"User-Agent":"GitHubCopilotChat/0.35.0","Editor-Version":"vscode/1.107.0","Editor-Plugin-Version":"copilot-chat/0.35.0","Copilot-Integration-Id":"vscode-chat"},
compat: {"forceAdaptiveThinking":true,"supportsTemperature":false},
reasoning: true,
thinkingLevelMap: {"xhigh":"xhigh"},
thinkingLevelMap: {"xhigh":"xhigh","minimal":"low"},
input: ["text", "image"],
cost: {
input: 5,
@@ -148,6 +167,7 @@ export const GITHUB_COPILOT_MODELS = {
headers: {"User-Agent":"GitHubCopilotChat/0.35.0","Editor-Version":"vscode/1.107.0","Editor-Plugin-Version":"copilot-chat/0.35.0","Copilot-Integration-Id":"vscode-chat"},
compat: {"forceAdaptiveThinking":true},
reasoning: true,
thinkingLevelMap: {"minimal":"low","xhigh":"max"},
input: ["text", "image"],
cost: {
input: 3,
@@ -405,23 +425,4 @@ export const GITHUB_COPILOT_MODELS = {
contextWindow: 400000,
maxTokens: 128000,
} satisfies Model<"openai-responses">,
"raptor-mini": {
id: "raptor-mini",
name: "Raptor mini",
api: "openai-completions",
provider: "github-copilot",
baseUrl: "https://api.individual.githubcopilot.com",
headers: {"User-Agent":"GitHubCopilotChat/0.35.0","Editor-Version":"vscode/1.107.0","Editor-Plugin-Version":"copilot-chat/0.35.0","Copilot-Integration-Id":"vscode-chat"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.25,
output: 2,
cacheRead: 0.025,
cacheWrite: 0,
},
contextWindow: 400000,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
} as const;
packages/ai/src/providers/google-vertex.models.ts
+69 -117
View File
@@ -4,94 +4,9 @@
import type { Model } from "../types.ts";
export const GOOGLE_VERTEX_MODELS = {
"gemini-1.5-flash": {
id: "gemini-1.5-flash",
name: "Gemini 1.5 Flash (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.075,
output: 0.3,
cacheRead: 0.01875,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 8192,
} satisfies Model<"google-vertex">,
"gemini-1.5-flash-8b": {
id: "gemini-1.5-flash-8b",
name: "Gemini 1.5 Flash-8B (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.0375,
output: 0.15,
cacheRead: 0.01,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 8192,
} satisfies Model<"google-vertex">,
"gemini-1.5-pro": {
id: "gemini-1.5-pro",
name: "Gemini 1.5 Pro (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: false,
input: ["text", "image"],
cost: {
input: 1.25,
output: 5,
cacheRead: 0.3125,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 8192,
} satisfies Model<"google-vertex">,
"gemini-2.0-flash": {
id: "gemini-2.0-flash",
name: "Gemini 2.0 Flash (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.15,
output: 0.6,
cacheRead: 0.0375,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 8192,
} satisfies Model<"google-vertex">,
"gemini-2.0-flash-lite": {
id: "gemini-2.0-flash-lite",
name: "Gemini 2.0 Flash Lite (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.075,
output: 0.3,
cacheRead: 0.01875,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 65536,
} satisfies Model<"google-vertex">,
"gemini-2.5-flash": {
id: "gemini-2.5-flash",
name: "Gemini 2.5 Flash (Vertex)",
name: "Gemini 2.5 Flash",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
@@ -108,24 +23,7 @@ export const GOOGLE_VERTEX_MODELS = {
} satisfies Model<"google-vertex">,
"gemini-2.5-flash-lite": {
id: "gemini-2.5-flash-lite",
name: "Gemini 2.5 Flash Lite (Vertex)",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.1,
output: 0.4,
cacheRead: 0.01,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 65536,
} satisfies Model<"google-vertex">,
"gemini-2.5-flash-lite-preview-09-2025": {
id: "gemini-2.5-flash-lite-preview-09-2025",
name: "Gemini 2.5 Flash Lite Preview 09-25 (Vertex)",
name: "Gemini 2.5 Flash-Lite",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
@@ -142,7 +40,7 @@ export const GOOGLE_VERTEX_MODELS = {
} satisfies Model<"google-vertex">,
"gemini-2.5-pro": {
id: "gemini-2.5-pro",
name: "Gemini 2.5 Pro (Vertex)",
name: "Gemini 2.5 Pro",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
@@ -159,7 +57,7 @@ export const GOOGLE_VERTEX_MODELS = {
} satisfies Model<"google-vertex">,
"gemini-3-flash-preview": {
id: "gemini-3-flash-preview",
name: "Gemini 3 Flash Preview (Vertex)",
name: "Gemini 3 Flash Preview",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
@@ -175,27 +73,27 @@ export const GOOGLE_VERTEX_MODELS = {
contextWindow: 1048576,
maxTokens: 65536,
} satisfies Model<"google-vertex">,
"gemini-3-pro-preview": {
id: "gemini-3-pro-preview",
name: "Gemini 3 Pro Preview (Vertex)",
"gemini-3.1-flash-lite": {
id: "gemini-3.1-flash-lite",
name: "Gemini 3.1 Flash Lite",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":"LOW","medium":null,"high":"HIGH"},
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 2,
output: 12,
cacheRead: 0.2,
input: 0.25,
output: 1.5,
cacheRead: 0.025,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 64000,
contextWindow: 1048576,
maxTokens: 65536,
} satisfies Model<"google-vertex">,
"gemini-3.1-pro-preview": {
id: "gemini-3.1-pro-preview",
name: "Gemini 3.1 Pro Preview (Vertex)",
name: "Gemini 3.1 Pro Preview",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
@@ -213,7 +111,7 @@ export const GOOGLE_VERTEX_MODELS = {
} satisfies Model<"google-vertex">,
"gemini-3.1-pro-preview-customtools": {
id: "gemini-3.1-pro-preview-customtools",
name: "Gemini 3.1 Pro Preview Custom Tools (Vertex)",
name: "Gemini 3.1 Pro Preview Custom Tools",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
@@ -229,4 +127,58 @@ export const GOOGLE_VERTEX_MODELS = {
contextWindow: 1048576,
maxTokens: 65536,
} satisfies Model<"google-vertex">,
"gemini-3.5-flash": {
id: "gemini-3.5-flash",
name: "Gemini 3.5 Flash",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 1.5,
output: 9,
cacheRead: 0.15,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 65536,
} satisfies Model<"google-vertex">,
"gemini-flash-latest": {
id: "gemini-flash-latest",
name: "Gemini Flash Latest",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 1.5,
output: 9,
cacheRead: 0.15,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 65536,
} satisfies Model<"google-vertex">,
"gemini-flash-lite-latest": {
id: "gemini-flash-lite-latest",
name: "Gemini Flash-Lite Latest",
api: "google-vertex",
provider: "google-vertex",
baseUrl: "https://{location}-aiplatform.googleapis.com",
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 0.25,
output: 1.5,
cacheRead: 0.025,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 65536,
} satisfies Model<"google-vertex">,
} as const;
packages/ai/src/providers/google.models.ts
+7 -5
View File
@@ -222,11 +222,12 @@ export const GOOGLE_MODELS = {
provider: "google",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 0.3,
output: 2.5,
cacheRead: 0.075,
input: 1.5,
output: 9,
cacheRead: 0.15,
cacheWrite: 0,
},
contextWindow: 1048576,
@@ -239,10 +240,11 @@ export const GOOGLE_MODELS = {
provider: "google",
baseUrl: "https://generativelanguage.googleapis.com/v1beta",
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 0.1,
output: 0.4,
input: 0.25,
output: 1.5,
cacheRead: 0.025,
cacheWrite: 0,
},
packages/ai/src/providers/kimi-coding.models.ts
+18
View File
@@ -4,6 +4,24 @@
import type { Model } from "../types.ts";
export const KIMI_CODING_MODELS = {
"k2p7": {
id: "k2p7",
name: "Kimi K2.7 Code",
api: "anthropic-messages",
provider: "kimi-coding",
baseUrl: "https://api.kimi.com/coding",
headers: {"User-Agent":"KimiCLI/1.5"},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 32768,
} satisfies Model<"anthropic-messages">,
"kimi-for-coding": {
id: "kimi-for-coding",
name: "Kimi For Coding",
packages/ai/src/providers/mistral.models.ts
+28 -28
View File
@@ -15,7 +15,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.3,
output: 0.9,
cacheRead: 0,
cacheRead: 0.03,
cacheWrite: 0,
},
contextWindow: 256000,
@@ -32,7 +32,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.4,
output: 2,
cacheRead: 0,
cacheRead: 0.04,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -49,7 +49,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.4,
output: 2,
cacheRead: 0,
cacheRead: 0.04,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -66,7 +66,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.4,
output: 2,
cacheRead: 0,
cacheRead: 0.04,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -83,7 +83,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.4,
output: 2,
cacheRead: 0,
cacheRead: 0.04,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -100,7 +100,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.1,
output: 0.3,
cacheRead: 0,
cacheRead: 0.01,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -117,7 +117,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.1,
output: 0.3,
cacheRead: 0,
cacheRead: 0.01,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -151,7 +151,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 2,
output: 5,
cacheRead: 0,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -168,7 +168,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.5,
output: 1.5,
cacheRead: 0,
cacheRead: 0.05,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -185,7 +185,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.04,
output: 0.04,
cacheRead: 0,
cacheRead: 0.004,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -202,7 +202,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.1,
output: 0.1,
cacheRead: 0,
cacheRead: 0.01,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -219,7 +219,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 2,
output: 6,
cacheRead: 0,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 131072,
@@ -236,7 +236,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.5,
output: 1.5,
cacheRead: 0,
cacheRead: 0.05,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -253,7 +253,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.5,
output: 1.5,
cacheRead: 0,
cacheRead: 0.05,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -270,7 +270,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.4,
output: 2,
cacheRead: 0,
cacheRead: 0.04,
cacheWrite: 0,
},
contextWindow: 131072,
@@ -287,7 +287,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.4,
output: 2,
cacheRead: 0,
cacheRead: 0.04,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -304,7 +304,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 1.5,
output: 7.5,
cacheRead: 0,
cacheRead: 0.15,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -338,7 +338,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.4,
output: 2,
cacheRead: 0,
cacheRead: 0.04,
cacheWrite: 0,
},
contextWindow: 262144,
@@ -355,7 +355,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.15,
output: 0.15,
cacheRead: 0,
cacheRead: 0.015,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -372,7 +372,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.1,
output: 0.3,
cacheRead: 0,
cacheRead: 0.01,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -389,7 +389,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.15,
output: 0.6,
cacheRead: 0,
cacheRead: 0.015,
cacheWrite: 0,
},
contextWindow: 256000,
@@ -406,7 +406,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.15,
output: 0.6,
cacheRead: 0,
cacheRead: 0.015,
cacheWrite: 0,
},
contextWindow: 256000,
@@ -423,7 +423,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.25,
output: 0.25,
cacheRead: 0,
cacheRead: 0.025,
cacheWrite: 0,
},
contextWindow: 8000,
@@ -440,7 +440,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.15,
output: 0.15,
cacheRead: 0,
cacheRead: 0.015,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -457,7 +457,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 2,
output: 6,
cacheRead: 0,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 64000,
@@ -474,7 +474,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.7,
output: 0.7,
cacheRead: 0,
cacheRead: 0.07,
cacheWrite: 0,
},
contextWindow: 32000,
@@ -491,7 +491,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 0.15,
output: 0.15,
cacheRead: 0,
cacheRead: 0.015,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -508,7 +508,7 @@ export const MISTRAL_MODELS = {
cost: {
input: 2,
output: 6,
cacheRead: 0,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 128000,
packages/ai/src/providers/moonshotai-cn.models.ts
+38
View File
@@ -130,4 +130,42 @@ export const MOONSHOTAI_CN_MODELS = {
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"kimi-k2.7-code": {
id: "kimi-k2.7-code",
name: "Kimi K2.7 Code",
api: "openai-completions",
provider: "moonshotai-cn",
baseUrl: "https://api.moonshot.cn/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"thinkingFormat":"deepseek"},
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"kimi-k2.7-code-highspeed": {
id: "kimi-k2.7-code-highspeed",
name: "Kimi K2.7 Code HighSpeed",
api: "openai-completions",
provider: "moonshotai-cn",
baseUrl: "https://api.moonshot.cn/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"thinkingFormat":"deepseek"},
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 1.9,
output: 8,
cacheRead: 0.38,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
} as const;
packages/ai/src/providers/moonshotai.models.ts
+38
View File
@@ -130,4 +130,42 @@ export const MOONSHOTAI_MODELS = {
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"kimi-k2.7-code": {
id: "kimi-k2.7-code",
name: "Kimi K2.7 Code",
api: "openai-completions",
provider: "moonshotai",
baseUrl: "https://api.moonshot.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"thinkingFormat":"deepseek"},
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"kimi-k2.7-code-highspeed": {
id: "kimi-k2.7-code-highspeed",
name: "Kimi K2.7 Code HighSpeed",
api: "openai-completions",
provider: "moonshotai",
baseUrl: "https://api.moonshot.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"thinkingFormat":"deepseek"},
reasoning: true,
thinkingLevelMap: {"off":null},
input: ["text", "image"],
cost: {
input: 1.9,
output: 8,
cacheRead: 0.38,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
} as const;
packages/ai/src/providers/nvidia.models.ts
-19
View File
@@ -289,25 +289,6 @@ export const NVIDIA_MODELS = {
contextWindow: 131072,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"qwen/qwen3-coder-480b-a35b-instruct": {
id: "qwen/qwen3-coder-480b-a35b-instruct",
name: "Qwen3 Coder 480B A35B Instruct",
api: "openai-completions",
provider: "nvidia",
baseUrl: "https://integrate.api.nvidia.com/v1",
headers: {"NVCF-POLL-SECONDS":"3600"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 66536,
} satisfies Model<"openai-completions">,
"qwen/qwen3.5-122b-a10b": {
id: "qwen/qwen3.5-122b-a10b",
name: "Qwen3.5 122B-A10B",
packages/ai/src/providers/opencode-go.models.ts
+32 -49
View File
@@ -42,24 +42,6 @@ export const OPENCODE_GO_MODELS = {
contextWindow: 1000000,
maxTokens: 384000,
} satisfies Model<"openai-completions">,
"glm-5": {
id: "glm-5",
name: "GLM-5",
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
input: 1,
output: 3.2,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 202752,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"glm-5.1": {
id: "glm-5.1",
name: "GLM-5.1",
@@ -78,23 +60,23 @@ export const OPENCODE_GO_MODELS = {
contextWindow: 202752,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"kimi-k2.5": {
id: "kimi-k2.5",
name: "Kimi K2.5",
"glm-5.2": {
id: "glm-5.2",
name: "GLM-5.2",
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text", "image"],
input: ["text"],
cost: {
input: 0.6,
output: 3,
cacheRead: 0.1,
input: 1.4,
output: 4.4,
cacheRead: 0.26,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
contextWindow: 1000000,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"kimi-k2.6": {
id: "kimi-k2.6",
@@ -102,7 +84,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens"},
compat: {"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
@@ -115,6 +97,24 @@ export const OPENCODE_GO_MODELS = {
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"kimi-k2.7-code": {
id: "kimi-k2.7-code",
name: "Kimi K2.7 Code",
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"mimo-v2.5": {
id: "mimo-v2.5",
name: "MiMo V2.5",
@@ -151,23 +151,6 @@ export const OPENCODE_GO_MODELS = {
contextWindow: 1048576,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
"minimax-m2.5": {
id: "minimax-m2.5",
name: "MiniMax M2.5",
api: "anthropic-messages",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go",
reasoning: true,
input: ["text"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0.03,
cacheWrite: 0,
},
contextWindow: 204800,
maxTokens: 65536,
} satisfies Model<"anthropic-messages">,
"minimax-m2.7": {
id: "minimax-m2.7",
name: "MiniMax M2.7",
@@ -188,16 +171,16 @@ export const OPENCODE_GO_MODELS = {
} satisfies Model<"openai-completions">,
"minimax-m3": {
id: "minimax-m3",
name: "MiniMax M3",
name: "MiniMax M3 (3x usage)",
api: "anthropic-messages",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go",
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0.06,
input: 0.1,
output: 0.4,
cacheRead: 0.02,
cacheWrite: 0,
},
contextWindow: 512000,
packages/ai/src/providers/opencode.models.ts
+6 -25
View File
@@ -22,25 +22,6 @@ export const OPENCODE_MODELS = {
contextWindow: 200000,
maxTokens: 32000,
} satisfies Model<"openai-completions">,
"claude-fable-5": {
id: "claude-fable-5",
name: "Claude Fable 5",
api: "anthropic-messages",
provider: "opencode",
baseUrl: "https://opencode.ai/zen",
compat: {"forceAdaptiveThinking":true},
reasoning: true,
thinkingLevelMap: {"off":null,"xhigh":"xhigh"},
input: ["text", "image"],
cost: {
input: 10,
output: 50,
cacheRead: 1,
cacheWrite: 12.5,
},
contextWindow: 1000000,
maxTokens: 128000,
} satisfies Model<"anthropic-messages">,
"claude-haiku-4-5": {
id: "claude-haiku-4-5",
name: "Claude Haiku 4.5",
@@ -207,7 +188,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -226,7 +207,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -245,7 +226,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -661,7 +642,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -679,7 +660,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens"},
compat: {"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -733,7 +714,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true,
input: ["text"],
cost: {
packages/ai/src/providers/openrouter.models.ts
+306 -203
View File
File diff suppressed because it is too large Load Diff
packages/ai/src/providers/together.models.ts
+107 -109
View File
@@ -4,25 +4,6 @@
import type { Model } from "../types.ts";
export const TOGETHER_MODELS = {
"MiniMaxAI/MiniMax-M2.5": {
id: "MiniMaxAI/MiniMax-M2.5",
name: "MiniMax-M2.5",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0.06,
cacheWrite: 0,
},
contextWindow: 204800,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"MiniMaxAI/MiniMax-M2.7": {
id: "MiniMaxAI/MiniMax-M2.7",
name: "MiniMax-M2.7",
@@ -42,28 +23,28 @@ export const TOGETHER_MODELS = {
contextWindow: 202752,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-235B-A22B-Instruct-2507-tput": {
id: "Qwen/Qwen3-235B-A22B-Instruct-2507-tput",
name: "Qwen3 235B A22B Instruct 2507 FP8",
"MiniMaxAI/MiniMax-M3": {
id: "MiniMaxAI/MiniMax-M3",
name: "MiniMax-M3",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
input: ["text", "image"],
cost: {
input: 0.2,
output: 0.6,
cacheRead: 0,
input: 0.3,
output: 1.2,
cacheRead: 0.06,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
contextWindow: 524288,
maxTokens: 250000,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8": {
id: "Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8",
name: "Qwen3 Coder 480B A35B Instruct",
"Qwen/Qwen2.5-7B-Instruct-Turbo": {
id: "Qwen/Qwen2.5-7B-Instruct-Turbo",
name: "Qwen 2.5 7B Instruct Turbo",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
@@ -71,27 +52,26 @@ export const TOGETHER_MODELS = {
reasoning: false,
input: ["text"],
cost: {
input: 2,
output: 2,
input: 0.3,
output: 0.3,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
contextWindow: 32768,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-Coder-Next-FP8": {
id: "Qwen/Qwen3-Coder-Next-FP8",
name: "Qwen3 Coder Next FP8",
"Qwen/Qwen3-235B-A22B-Instruct-2507-tput": {
id: "Qwen/Qwen3-235B-A22B-Instruct-2507-tput",
name: "Qwen3 235B A22B Instruct 2507 FP8",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
input: 0.5,
output: 1.2,
input: 0.2,
output: 0.6,
cacheRead: 0,
cacheWrite: 0,
},
@@ -117,6 +97,25 @@ export const TOGETHER_MODELS = {
contextWindow: 262144,
maxTokens: 130000,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-9B": {
id: "Qwen/Qwen3.5-9B",
name: "Qwen3.5 9B",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
cost: {
input: 0.17,
output: 0.25,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.6-Plus": {
id: "Qwen/Qwen3.6-Plus",
name: "Qwen3.6 Plus",
@@ -142,57 +141,18 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
input: 2.5,
output: 7.5,
input: 1.25,
output: 3.75,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 500000,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-V3": {
id: "deepseek-ai/DeepSeek-V3",
name: "DeepSeek-V3",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
cost: {
input: 1.25,
output: 1.25,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-V3-1": {
id: "deepseek-ai/DeepSeek-V3-1",
name: "DeepSeek V3.1",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
cost: {
input: 0.6,
output: 1.7,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-V4-Pro": {
id: "deepseek-ai/DeepSeek-V4-Pro",
name: "DeepSeek V4 Pro",
@@ -204,8 +164,8 @@ export const TOGETHER_MODELS = {
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null},
input: ["text"],
cost: {
input: 2.1,
output: 4.4,
input: 1.74,
output: 3.48,
cacheRead: 0.2,
cacheWrite: 0,
},
@@ -241,8 +201,8 @@ export const TOGETHER_MODELS = {
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
cost: {
input: 0.2,
output: 0.5,
input: 0.39,
output: 0.97,
cacheRead: 0,
cacheWrite: 0,
},
@@ -267,25 +227,6 @@ export const TOGETHER_MODELS = {
contextWindow: 131072,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"moonshotai/Kimi-K2.5": {
id: "moonshotai/Kimi-K2.5",
name: "Kimi K2.5",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
cost: {
input: 0.5,
output: 2.8,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"moonshotai/Kimi-K2.6": {
id: "moonshotai/Kimi-K2.6",
name: "Kimi K2.6",
@@ -305,6 +246,25 @@ export const TOGETHER_MODELS = {
contextWindow: 262144,
maxTokens: 131000,
} satisfies Model<"openai-completions">,
"moonshotai/Kimi-K2.7-Code": {
id: "moonshotai/Kimi-K2.7-Code",
name: "Kimi K2.7 Code",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"nvidia/nemotron-3-ultra-550b-a55b": {
id: "nvidia/nemotron-3-ultra-550b-a55b",
name: "Nemotron 3 Ultra 550B A55B",
@@ -343,6 +303,44 @@ export const TOGETHER_MODELS = {
contextWindow: 131072,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"openai/gpt-oss-20b": {
id: "openai/gpt-oss-20b",
name: "GPT OSS 20B",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"openai"},
reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null},
input: ["text"],
cost: {
input: 0.05,
output: 0.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"zai-org/GLM-5": {
id: "zai-org/GLM-5",
name: "GLM-5",
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
cost: {
input: 1,
output: 3.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 202752,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"zai-org/GLM-5.1": {
id: "zai-org/GLM-5.1",
name: "GLM-5.1",
packages/ai/src/providers/vercel-ai-gateway.models.ts
+145 -130
View File
@@ -98,7 +98,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.39999999999999997,
input: 0.4,
output: 4,
cacheRead: 0,
cacheWrite: 0,
@@ -168,7 +168,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1,
output: 5,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -268,7 +268,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.39999999999999997,
input: 0.4,
output: 4,
cacheRead: 0,
cacheWrite: 0,
@@ -285,8 +285,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.09999999999999999,
output: 0.39999999999999997,
input: 0.1,
output: 0.4,
cacheRead: 0.001,
cacheWrite: 0.125,
},
@@ -302,7 +302,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.39999999999999997,
input: 0.4,
output: 2.4,
cacheRead: 0.04,
cacheWrite: 0.5,
@@ -320,7 +320,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text", "image"],
cost: {
input: 0.6,
output: 3.5999999999999996,
output: 3.6,
cacheRead: 0,
cacheWrite: 0,
},
@@ -338,7 +338,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 0.5,
output: 3,
cacheRead: 0.09999999999999999,
cacheRead: 0.1,
cacheWrite: 0.625,
},
contextWindow: 1000000,
@@ -370,8 +370,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.39999999999999997,
output: 1.5999999999999999,
input: 0.4,
output: 1.6,
cacheRead: 0.08,
cacheWrite: 0.5,
},
@@ -404,7 +404,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.7999999999999999,
input: 0.8,
output: 4,
cacheRead: 0.08,
cacheWrite: 1,
@@ -412,25 +412,6 @@ export const VERCEL_AI_GATEWAY_MODELS = {
contextWindow: 200000,
maxTokens: 8192,
} satisfies Model<"anthropic-messages">,
"anthropic/claude-fable-5": {
id: "anthropic/claude-fable-5",
name: "Claude Fable 5",
api: "anthropic-messages",
provider: "vercel-ai-gateway",
baseUrl: "https://ai-gateway.vercel.sh",
compat: {"forceAdaptiveThinking":true},
reasoning: true,
thinkingLevelMap: {"off":null,"xhigh":"xhigh"},
input: ["text", "image"],
cost: {
input: 10,
output: 50,
cacheRead: 1,
cacheWrite: 12.5,
},
contextWindow: 1000000,
maxTokens: 128000,
} satisfies Model<"anthropic-messages">,
"anthropic/claude-haiku-4.5": {
id: "anthropic/claude-haiku-4.5",
name: "Claude Haiku 4.5",
@@ -442,7 +423,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1,
output: 5,
cacheRead: 0.09999999999999999,
cacheRead: 0.1,
cacheWrite: 1.25,
},
contextWindow: 200000,
@@ -635,7 +616,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text"],
cost: {
input: 0.25,
output: 0.8999999999999999,
output: 0.9,
cacheRead: 0,
cacheWrite: 0,
},
@@ -653,7 +634,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 0.25,
output: 2,
cacheRead: 0.049999999999999996,
cacheRead: 0.05,
cacheWrite: 0,
},
contextWindow: 256000,
@@ -838,8 +819,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.09999999999999999,
output: 0.39999999999999997,
input: 0.1,
output: 0.4,
cacheRead: 0.01,
cacheWrite: 0,
},
@@ -874,7 +855,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 0.5,
output: 3,
cacheRead: 0.049999999999999996,
cacheRead: 0.05,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -891,7 +872,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 2,
output: 12,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -942,7 +923,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 2,
output: 12,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -992,7 +973,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text", "image"],
cost: {
input: 0.14,
output: 0.39999999999999997,
output: 0.4,
cacheRead: 0,
cacheWrite: 0,
},
@@ -1010,7 +991,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 0.25,
output: 0.75,
cacheRead: 0.024999999999999998,
cacheRead: 0.025,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -1162,7 +1143,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text", "image"],
cost: {
input: 0.24,
output: 0.9700000000000001,
output: 0.97,
cacheRead: 0,
cacheWrite: 0,
},
@@ -1178,7 +1159,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.16999999999999998,
input: 0.17,
output: 0.66,
cacheRead: 0,
cacheWrite: 0,
@@ -1332,7 +1313,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text"],
cost: {
input: 0.3,
output: 0.8999999999999999,
output: 0.9,
cacheRead: 0,
cacheWrite: 0,
},
@@ -1348,7 +1329,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text"],
cost: {
input: 0.39999999999999997,
input: 0.4,
output: 2,
cacheRead: 0,
cacheWrite: 0,
@@ -1365,7 +1346,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text"],
cost: {
input: 0.09999999999999999,
input: 0.1,
output: 0.3,
cacheRead: 0,
cacheWrite: 0,
@@ -1382,7 +1363,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text"],
cost: {
input: 0.09999999999999999,
input: 0.1,
output: 0.3,
cacheRead: 0,
cacheWrite: 0,
@@ -1399,8 +1380,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text"],
cost: {
input: 0.09999999999999999,
output: 0.09999999999999999,
input: 0.1,
output: 0.1,
cacheRead: 0,
cacheWrite: 0,
},
@@ -1433,7 +1414,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.39999999999999997,
input: 0.4,
output: 2,
cacheRead: 0,
cacheWrite: 0,
@@ -1467,13 +1448,13 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text"],
cost: {
input: 0.02,
output: 0.04,
input: 0.15,
output: 0.15,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 131072,
contextWindow: 128000,
maxTokens: 128000,
} satisfies Model<"anthropic-messages">,
"mistral/mistral-small": {
id: "mistral/mistral-small",
@@ -1484,7 +1465,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.09999999999999999,
input: 0.1,
output: 0.3,
cacheRead: 0,
cacheWrite: 0,
@@ -1535,7 +1516,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text"],
cost: {
input: 0.5700000000000001,
input: 0.57,
output: 2.3,
cacheRead: 0,
cacheWrite: 0,
@@ -1560,40 +1541,6 @@ export const VERCEL_AI_GATEWAY_MODELS = {
contextWindow: 262114,
maxTokens: 262114,
} satisfies Model<"anthropic-messages">,
"moonshotai/kimi-k2-thinking-turbo": {
id: "moonshotai/kimi-k2-thinking-turbo",
name: "Kimi K2 Thinking Turbo",
api: "anthropic-messages",
provider: "vercel-ai-gateway",
baseUrl: "https://ai-gateway.vercel.sh",
reasoning: true,
input: ["text"],
cost: {
input: 1.15,
output: 8,
cacheRead: 0.15,
cacheWrite: 0,
},
contextWindow: 262114,
maxTokens: 262114,
} satisfies Model<"anthropic-messages">,
"moonshotai/kimi-k2-turbo": {
id: "moonshotai/kimi-k2-turbo",
name: "Kimi K2 Turbo",
api: "anthropic-messages",
provider: "vercel-ai-gateway",
baseUrl: "https://ai-gateway.vercel.sh",
reasoning: false,
input: ["text"],
cost: {
input: 1.15,
output: 8,
cacheRead: 0.15,
cacheWrite: 0,
},
contextWindow: 256000,
maxTokens: 16384,
} satisfies Model<"anthropic-messages">,
"moonshotai/kimi-k2.5": {
id: "moonshotai/kimi-k2.5",
name: "Kimi K2.5",
@@ -1605,7 +1552,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 0.6,
output: 3,
cacheRead: 0.09999999999999999,
cacheRead: 0.1,
cacheWrite: 0,
},
contextWindow: 262114,
@@ -1628,6 +1575,40 @@ export const VERCEL_AI_GATEWAY_MODELS = {
contextWindow: 262000,
maxTokens: 262000,
} satisfies Model<"anthropic-messages">,
"moonshotai/kimi-k2.7-code": {
id: "moonshotai/kimi-k2.7-code",
name: "Kimi K2.7 Code",
api: "anthropic-messages",
provider: "vercel-ai-gateway",
baseUrl: "https://ai-gateway.vercel.sh",
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0.19,
cacheWrite: 0,
},
contextWindow: 256000,
maxTokens: 32768,
} satisfies Model<"anthropic-messages">,
"moonshotai/kimi-k2.7-code-highspeed": {
id: "moonshotai/kimi-k2.7-code-highspeed",
name: "Kimi K2.7 Code High Speed",
api: "anthropic-messages",
provider: "vercel-ai-gateway",
baseUrl: "https://ai-gateway.vercel.sh",
reasoning: true,
input: ["text", "image"],
cost: {
input: 1.9,
output: 8,
cacheRead: 0.38,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 32768,
} satisfies Model<"anthropic-messages">,
"nvidia/nemotron-3-super-120b-a12b": {
id: "nvidia/nemotron-3-super-120b-a12b",
name: "NVIDIA Nemotron 3 Super 120B A12B",
@@ -1671,7 +1652,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.19999999999999998,
input: 0.2,
output: 0.6,
cacheRead: 0,
cacheWrite: 0,
@@ -1689,7 +1670,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text"],
cost: {
input: 0.06,
output: 0.22999999999999998,
output: 0.23,
cacheRead: 0,
cacheWrite: 0,
},
@@ -1739,9 +1720,9 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.39999999999999997,
output: 1.5999999999999999,
cacheRead: 0.09999999999999999,
input: 0.4,
output: 1.6,
cacheRead: 0.1,
cacheWrite: 0,
},
contextWindow: 1047576,
@@ -1756,9 +1737,9 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.09999999999999999,
output: 0.39999999999999997,
cacheRead: 0.024999999999999998,
input: 0.1,
output: 0.4,
cacheRead: 0.025,
cacheWrite: 0,
},
contextWindow: 1047576,
@@ -1860,7 +1841,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 0.25,
output: 2,
cacheRead: 0.024999999999999998,
cacheRead: 0.025,
cacheWrite: 0,
},
contextWindow: 400000,
@@ -1875,8 +1856,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.049999999999999996,
output: 0.39999999999999997,
input: 0.05,
output: 0.4,
cacheRead: 0.005,
cacheWrite: 0,
},
@@ -1945,7 +1926,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 0.25,
output: 2,
cacheRead: 0.024999999999999998,
cacheRead: 0.025,
cacheWrite: 0,
},
contextWindow: 400000,
@@ -2139,7 +2120,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
thinkingLevelMap: {"xhigh":"xhigh"},
input: ["text", "image"],
cost: {
input: 0.19999999999999998,
input: 0.2,
output: 1.25,
cacheRead: 0.02,
cacheWrite: 0,
@@ -2227,8 +2208,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text"],
cost: {
input: 0.049999999999999996,
output: 0.19999999999999998,
input: 0.05,
output: 0.2,
cacheRead: 0,
cacheWrite: 0,
},
@@ -2388,6 +2369,23 @@ export const VERCEL_AI_GATEWAY_MODELS = {
contextWindow: 200000,
maxTokens: 8000,
} satisfies Model<"anthropic-messages">,
"sakana/fugu-ultra": {
id: "sakana/fugu-ultra",
name: "Fugu Ultra",
api: "anthropic-messages",
provider: "vercel-ai-gateway",
baseUrl: "https://ai-gateway.vercel.sh",
reasoning: true,
input: ["text", "image"],
cost: {
input: 5,
output: 30,
cacheRead: 0.5,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 1000000,
} satisfies Model<"anthropic-messages">,
"stepfun/step-3.5-flash": {
id: "stepfun/step-3.5-flash",
name: "StepFun 3.5 Flash",
@@ -2399,8 +2397,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 0.09,
output: 0.3,
cacheRead: 0,
cacheWrite: 0.02,
cacheRead: 0.02,
cacheWrite: 0,
},
contextWindow: 262114,
maxTokens: 262114,
@@ -2414,7 +2412,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.19999999999999998,
input: 0.2,
output: 1.15,
cacheRead: 0.04,
cacheWrite: 0,
@@ -2431,9 +2429,9 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.19999999999999998,
input: 0.2,
output: 0.5,
cacheRead: 0.049999999999999996,
cacheRead: 0.05,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -2448,9 +2446,9 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.19999999999999998,
input: 0.2,
output: 0.5,
cacheRead: 0.049999999999999996,
cacheRead: 0.05,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -2467,7 +2465,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1.25,
output: 2.5,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 2000000,
@@ -2484,7 +2482,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1.25,
output: 2.5,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 2000000,
@@ -2501,7 +2499,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1.25,
output: 2.5,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 2000000,
@@ -2518,7 +2516,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1.25,
output: 2.5,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 2000000,
@@ -2535,7 +2533,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1.25,
output: 2.5,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 2000000,
@@ -2552,7 +2550,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1.25,
output: 2.5,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 2000000,
@@ -2569,7 +2567,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1.25,
output: 2.5,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -2586,7 +2584,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1,
output: 2,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 256000,
@@ -2601,7 +2599,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text"],
cost: {
input: 0.09999999999999999,
input: 0.1,
output: 0.3,
cacheRead: 0.01,
cacheWrite: 0,
@@ -2620,7 +2618,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
cost: {
input: 1,
output: 3,
cacheRead: 0.19999999999999998,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 1000000,
@@ -2686,7 +2684,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
reasoning: true,
input: ["text"],
cost: {
input: 0.19999999999999998,
input: 0.2,
output: 1.1,
cacheRead: 0.03,
cacheWrite: 0,
@@ -2704,7 +2702,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text", "image"],
cost: {
input: 0.6,
output: 1.7999999999999998,
output: 1.8,
cacheRead: 0.11,
cacheWrite: 0,
},
@@ -2738,8 +2736,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text", "image"],
cost: {
input: 0.3,
output: 0.8999999999999999,
cacheRead: 0.049999999999999996,
output: 0.9,
cacheRead: 0.05,
cacheWrite: 0,
},
contextWindow: 128000,
@@ -2789,7 +2787,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text"],
cost: {
input: 0.07,
output: 0.39999999999999997,
output: 0.4,
cacheRead: 0,
cacheWrite: 0,
},
@@ -2806,7 +2804,7 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text"],
cost: {
input: 0.06,
output: 0.39999999999999997,
output: 0.4,
cacheRead: 0.01,
cacheWrite: 0,
},
@@ -2823,8 +2821,8 @@ export const VERCEL_AI_GATEWAY_MODELS = {
input: ["text"],
cost: {
input: 1,
output: 3.1999999999999997,
cacheRead: 0.19999999999999998,
output: 3.2,
cacheRead: 0.2,
cacheWrite: 0,
},
contextWindow: 202800,
@@ -2864,6 +2862,23 @@ export const VERCEL_AI_GATEWAY_MODELS = {
contextWindow: 202800,
maxTokens: 64000,
} satisfies Model<"anthropic-messages">,
"zai/glm-5.2": {
id: "zai/glm-5.2",
name: "GLM 5.2",
api: "anthropic-messages",
provider: "vercel-ai-gateway",
baseUrl: "https://ai-gateway.vercel.sh",
reasoning: true,
input: ["text"],
cost: {
input: 1.5,
output: 4.5,
cacheRead: 0.3,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 128000,
} satisfies Model<"anthropic-messages">,
"zai/glm-5v-turbo": {
id: "zai/glm-5v-turbo",
name: "GLM 5V Turbo",
packages/ai/src/providers/zai-coding-cn.models.ts
+19
View File
@@ -76,6 +76,25 @@ export const ZAI_CODING_CN_MODELS = {
contextWindow: 200000,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"glm-5.2": {
id: "glm-5.2",
name: "GLM-5.2",
api: "openai-completions",
provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","supportsReasoningEffort":true,"zaiToolStream":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":"high","medium":"high","high":"high","xhigh":"max"},
input: ["text"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"glm-5v-turbo": {
id: "glm-5v-turbo",
name: "GLM-5V-Turbo",
packages/ai/src/providers/zai.models.ts
+19
View File
@@ -76,6 +76,25 @@ export const ZAI_MODELS = {
contextWindow: 200000,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"glm-5.2": {
id: "glm-5.2",
name: "GLM-5.2",
api: "openai-completions",
provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","supportsReasoningEffort":true,"zaiToolStream":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":"high","medium":"high","high":"high","xhigh":"max"},
input: ["text"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"glm-5v-turbo": {
id: "glm-5v-turbo",
name: "GLM-5V-Turbo",
packages/ai/src/types.ts
+24 -1
View File
@@ -74,6 +74,15 @@ export type ImagesProviderId = KnownImagesProvider | string;
export type ThinkingLevel = "minimal" | "low" | "medium" | "high" | "xhigh";
export type ModelThinkingLevel = "off" | ThinkingLevel;
export type ThinkingLevelMap = Partial<Record<ModelThinkingLevel, string | null>>;
export type ChatTemplateKwargValue =
| string
| number
| boolean
| null
| {
$var: "thinking.enabled" | "thinking.effort";
omitWhenOff?: boolean;
};
/** Token budgets for each thinking level (token-based providers only) */
export interface ThinkingBudgets {
@@ -88,6 +97,9 @@ export type CacheRetention = "none" | "short" | "long";
export type Transport = "sse" | "websocket" | "websocket-cached" | "auto";
/** Provider-scoped environment overrides. Values take precedence over process.env. */
export type ProviderEnv = Record<string, string>;
export interface ProviderResponse {
status: number;
headers: Record<string, string>;
@@ -162,6 +174,12 @@ export interface StreamOptions {
* For example, Anthropic uses `user_id` for abuse tracking and rate limiting.
*/
metadata?: Record<string, unknown>;
/**
* Provider-scoped environment values. These take precedence over process.env for
* provider configuration such as regional settings, endpoint placeholders, and
* proxy variables.
*/
env?: ProviderEnv;
}
export type ProviderStreamOptions = StreamOptions & Record<string, unknown>;
@@ -328,6 +346,8 @@ export interface Usage {
output: number;
cacheRead: number;
cacheWrite: number;
/** Subset of `cacheWrite` written with 1h retention. Only Anthropic reports this split. */
cacheWrite1h?: number;
totalTokens: number;
cost: {
input: number;
@@ -453,7 +473,7 @@ export interface OpenAICompletionsCompat {
requiresThinkingAsText?: boolean;
/** Whether all replayed assistant messages must include an empty reasoning_content field when reasoning is enabled. Default: auto-detected from URL. */
requiresReasoningContentOnAssistantMessages?: boolean;
/** Format for reasoning/thinking parameter. "openai" uses reasoning_effort, "openrouter" uses reasoning: { effort }, "deepseek" uses thinking: { type } plus reasoning_effort when supported, "together" uses reasoning: { enabled } plus reasoning_effort when supported, "zai" uses thinking: { type }, "qwen" uses top-level enable_thinking: boolean, "qwen-chat-template" uses chat_template_kwargs.enable_thinking, "string-thinking" uses top-level thinking: string, and "ant-ling" uses reasoning: { effort } only when the mapped effort is non-null. Default: "openai". */
/** Format for reasoning/thinking parameter. "openai" uses reasoning_effort, "openrouter" uses reasoning: { effort }, "deepseek" uses thinking: { type } plus reasoning_effort when supported, "together" uses reasoning: { enabled } plus reasoning_effort when supported, "zai" uses thinking: { type }, "qwen" uses top-level enable_thinking: boolean, "qwen-chat-template" uses chat_template_kwargs.enable_thinking and preserve_thinking, "chat-template" uses configurable chat_template_kwargs, "string-thinking" uses top-level thinking: string, and "ant-ling" uses reasoning: { effort } only when the mapped effort is non-null. Default: "openai". */
thinkingFormat?:
| "openai"
| "openrouter"
@@ -461,9 +481,12 @@ export interface OpenAICompletionsCompat {
| "together"
| "zai"
| "qwen"
| "chat-template"
| "qwen-chat-template"
| "string-thinking"
| "ant-ling";
/** Kwargs to send as `chat_template_kwargs` when `thinkingFormat` is `chat-template`. Use `{ "$var": "thinking.enabled" }` or `{ "$var": "thinking.effort" }` for pi-controlled thinking values. */
chatTemplateKwargs?: Record<string, ChatTemplateKwargValue>;
/** OpenRouter-compatible routing preferences sent as the `provider` request field. */
openRouterRouting?: OpenRouterRouting;
/** Vercel AI Gateway routing preferences. Only used when baseUrl points to Vercel AI Gateway. */
packages/ai/src/utils/node-http-proxy.ts
+22 -33
View File
@@ -1,7 +1,5 @@
import type { Agent as HttpAgent } from "node:http";
import type { Agent as HttpsAgent } from "node:https";
import { HttpProxyAgent } from "http-proxy-agent";
import { HttpsProxyAgent } from "https-proxy-agent";
import type { ProviderEnv } from "../types.ts";
import { getProviderEnvValue } from "./provider-env.ts";
const DEFAULT_PROXY_PORTS: Record<string, number> = {
ftp: 21,
@@ -12,16 +10,16 @@ const DEFAULT_PROXY_PORTS: Record<string, number> = {
wss: 443,
};
export interface NodeHttpProxyAgents {
httpAgent: HttpAgent;
httpsAgent: HttpsAgent;
}
export const UNSUPPORTED_PROXY_PROTOCOL_MESSAGE =
"Unsupported proxy protocol. SOCKS and PAC proxy URLs are not supported; use an HTTP or HTTPS proxy URL.";
function getProxyEnv(key: string): string {
return process.env[key.toLowerCase()] || process.env[key.toUpperCase()] || "";
function getProxyEnv(key: string, env?: ProviderEnv): string {
const lowercaseKey = key.toLowerCase();
const uppercaseKey = key.toUpperCase();
return (
env?.[lowercaseKey] ||
env?.[uppercaseKey] ||
getProviderEnvValue(lowercaseKey) ||
getProviderEnvValue(uppercaseKey) ||
""
);
}
function parseProxyTargetUrl(targetUrl: string | URL): URL | undefined {
@@ -36,8 +34,8 @@ function parseProxyTargetUrl(targetUrl: string | URL): URL | undefined {
}
}
function shouldProxyHostname(hostname: string, port: number): boolean {
const noProxy = getProxyEnv("no_proxy").toLowerCase();
function shouldProxyHostname(hostname: string, port: number, env?: ProviderEnv): boolean {
const noProxy = getProxyEnv("no_proxy", env).toLowerCase();
if (!noProxy) {
return true;
}
@@ -68,7 +66,7 @@ function shouldProxyHostname(hostname: string, port: number): boolean {
});
}
function getProxyForUrl(targetUrl: string | URL): string {
function getProxyForUrl(targetUrl: string | URL, env?: ProviderEnv): string {
const parsedUrl = parseProxyTargetUrl(targetUrl);
if (!parsedUrl?.protocol || !parsedUrl.host) {
return "";
@@ -77,19 +75,22 @@ function getProxyForUrl(targetUrl: string | URL): string {
const protocol = parsedUrl.protocol.split(":", 1)[0]!;
const hostname = parsedUrl.host.replace(/:\d*$/, "");
const port = Number.parseInt(parsedUrl.port, 10) || DEFAULT_PROXY_PORTS[protocol] || 0;
if (!shouldProxyHostname(hostname, port)) {
if (!shouldProxyHostname(hostname, port, env)) {
return "";
}
let proxy = getProxyEnv(`${protocol}_proxy`) || getProxyEnv("all_proxy");
let proxy = getProxyEnv(`${protocol}_proxy`, env) || getProxyEnv("all_proxy", env);
if (proxy && !proxy.includes("://")) {
proxy = `${protocol}://${proxy}`;
}
return proxy;
}
export function resolveHttpProxyUrlForTarget(targetUrl: string | URL): URL | undefined {
const proxy = getProxyForUrl(targetUrl);
export const UNSUPPORTED_PROXY_PROTOCOL_MESSAGE =
"Unsupported proxy protocol. SOCKS and PAC proxy URLs are not supported; use an HTTP or HTTPS proxy URL.";
export function resolveHttpProxyUrlForTarget(targetUrl: string | URL, env?: ProviderEnv): URL | undefined {
const proxy = getProxyForUrl(targetUrl, env);
if (!proxy) {
return undefined;
}
@@ -109,15 +110,3 @@ export function resolveHttpProxyUrlForTarget(targetUrl: string | URL): URL | und
return proxyUrl;
}
export function createHttpProxyAgentsForTarget(targetUrl: string | URL): NodeHttpProxyAgents | undefined {
const proxyUrl = resolveHttpProxyUrlForTarget(targetUrl);
if (!proxyUrl) {
return undefined;
}
return {
httpAgent: new HttpProxyAgent(proxyUrl),
httpsAgent: new HttpsProxyAgent(proxyUrl) as unknown as HttpsAgent,
};
}
packages/ai/src/utils/oauth/anthropic.ts
+2 -1
View File
@@ -7,6 +7,7 @@
import type { Server } from "node:http";
import type { OAuthAuth } from "../../auth/types.ts";
import { getProviderEnvValue } from "../provider-env.ts";
import { oauthErrorHtml, oauthSuccessHtml } from "./oauth-page.ts";
import { generatePKCE } from "./pkce.ts";
import type { OAuthCredentials, OAuthLoginCallbacks, OAuthPrompt, OAuthProviderInterface } from "./types.ts";
@@ -29,7 +30,7 @@ const decode = (s: string) => atob(s);
const CLIENT_ID = decode("OWQxYzI1MGEtZTYxYi00NGQ5LTg4ZWQtNTk0NGQxOTYyZjVl");
const AUTHORIZE_URL = "https://claude.ai/oauth/authorize";
const TOKEN_URL = "https://platform.claude.com/v1/oauth/token";
const CALLBACK_HOST = process.env.PI_OAUTH_CALLBACK_HOST || "127.0.0.1";
const CALLBACK_HOST = getProviderEnvValue("PI_OAUTH_CALLBACK_HOST") || "127.0.0.1";
const CALLBACK_PORT = 53692;
const CALLBACK_PATH = "/callback";
const REDIRECT_URI = `http://localhost:${CALLBACK_PORT}${CALLBACK_PATH}`;
packages/ai/src/utils/oauth/github-copilot.ts
+76 -7
View File
@@ -10,6 +10,7 @@ import type { OAuthCredentials, OAuthDeviceCodeInfo, OAuthLoginCallbacks, OAuthP
type CopilotCredentials = OAuthCredentials & {
enterpriseUrl?: string;
availableModelIds: string[];
};
const decode = (s: string) => atob(s);
@@ -21,6 +22,7 @@ const COPILOT_HEADERS = {
"Editor-Plugin-Version": "copilot-chat/0.35.0",
"Copilot-Integration-Id": "vscode-chat",
} as const;
const COPILOT_API_VERSION = "2026-06-01";
type DeviceCodeResponse = {
device_code: string;
@@ -89,6 +91,48 @@ export function getGitHubCopilotBaseUrl(token?: string, enterpriseDomain?: strin
return "https://api.individual.githubcopilot.com";
}
function asRecord(value: unknown): Record<string, unknown> | undefined {
return value && typeof value === "object" ? (value as Record<string, unknown>) : undefined;
}
function isSelectableCopilotModel(item: Record<string, unknown>): boolean {
const policy = asRecord(item.policy);
const capabilities = asRecord(item.capabilities);
const supports = asRecord(capabilities?.supports);
return item.model_picker_enabled === true && policy?.state !== "disabled" && supports?.tool_calls !== false;
}
function parseAvailableCopilotModelIds(raw: unknown): string[] {
const data = asRecord(raw)?.data;
if (!Array.isArray(data)) {
throw new Error("Invalid Copilot models response");
}
const ids: string[] = [];
for (const rawItem of data) {
const item = asRecord(rawItem);
const id = item?.id;
if (typeof id === "string" && item && isSelectableCopilotModel(item)) {
ids.push(id);
}
}
return ids;
}
async function fetchAvailableGitHubCopilotModelIds(copilotToken: string, enterpriseDomain?: string): Promise<string[]> {
const baseUrl = getGitHubCopilotBaseUrl(copilotToken, enterpriseDomain);
const raw = await fetchJson(`${baseUrl}/models`, {
headers: {
Accept: "application/json",
Authorization: `Bearer ${copilotToken}`,
...COPILOT_HEADERS,
"X-GitHub-Api-Version": COPILOT_API_VERSION,
},
signal: AbortSignal.timeout(5000),
});
return parseAvailableCopilotModelIds(raw);
}
async function fetchJson(url: string, init: RequestInit): Promise<unknown> {
const response = await fetch(url, init);
if (!response.ok) {
@@ -202,10 +246,7 @@ async function pollForGitHubAccessToken(
});
}
/**
* Refresh GitHub Copilot token
*/
export async function refreshGitHubCopilotToken(
async function refreshGitHubCopilotAccessToken(
refreshToken: string,
enterpriseDomain?: string,
): Promise<OAuthCredentials> {
@@ -239,6 +280,20 @@ export async function refreshGitHubCopilotToken(
};
}
/**
* Refresh GitHub Copilot token
*/
export async function refreshGitHubCopilotToken(
refreshToken: string,
enterpriseDomain?: string,
): Promise<OAuthCredentials> {
const credentials = await refreshGitHubCopilotAccessToken(refreshToken, enterpriseDomain);
return {
...credentials,
availableModelIds: await fetchAvailableGitHubCopilotModelIds(credentials.access, enterpriseDomain),
};
}
/**
* Enable a model for the user's GitHub Copilot account.
* This is required for some models (like Claude, Grok) before they can be used.
@@ -323,12 +378,18 @@ export async function loginGitHubCopilot(options: {
});
const githubAccessToken = await pollForGitHubAccessToken(domain, device, options.signal);
const credentials = await refreshGitHubCopilotToken(githubAccessToken, enterpriseDomain ?? undefined);
const credentials = await refreshGitHubCopilotAccessToken(githubAccessToken, enterpriseDomain ?? undefined);
// Enable all models after successful login
options.onProgress?.("Enabling models...");
await enableAllGitHubCopilotModels(credentials.access, enterpriseDomain ?? undefined);
return credentials;
// Fetch availability after policy enable so newly enabled models are included,
// while unavailable models are still filtered out.
return {
...credentials,
availableModelIds: await fetchAvailableGitHubCopilotModelIds(credentials.access, enterpriseDomain ?? undefined),
};
}
function copilotEnterpriseDomain(credential: OAuthCredential): string | undefined {
@@ -393,6 +454,14 @@ export const githubCopilotOAuthProvider: OAuthProviderInterface = {
const creds = credentials as CopilotCredentials;
const domain = creds.enterpriseUrl ? (normalizeDomain(creds.enterpriseUrl) ?? undefined) : undefined;
const baseUrl = getGitHubCopilotBaseUrl(creds.access, domain);
return models.map((m) => (m.provider === "github-copilot" ? { ...m, baseUrl } : m));
// Older stored Pi auth entries do not have account-specific model IDs yet;
// keep their existing generated-catalog behavior until the next refresh/login.
const availableModelIds = "availableModelIds" in creds ? new Set(creds.availableModelIds) : undefined;
return models.flatMap((m) => {
if (m.provider !== "github-copilot") return [m];
if (availableModelIds && !availableModelIds.has(m.id)) return [];
return [{ ...m, baseUrl }];
});
},
};
packages/ai/src/utils/oauth/openai-codex.ts
+2 -1
View File
@@ -18,6 +18,7 @@ if (typeof process !== "undefined" && (process.versions?.node || process.version
}
import type { OAuthAuth } from "../../auth/types.ts";
import { getProviderEnvValue } from "../provider-env.ts";
import { pollOAuthDeviceCodeFlow } from "./device-code.ts";
import { oauthErrorHtml, oauthSuccessHtml } from "./oauth-page.ts";
import { generatePKCE } from "./pkce.ts";
@@ -48,7 +49,7 @@ type OAuthToken = { access: string; refresh: string; expires: number };
type TokenOperation = "exchange" | "refresh";
function getCallbackHost(): string {
return typeof process !== "undefined" ? process.env.PI_OAUTH_CALLBACK_HOST || "127.0.0.1" : "127.0.0.1";
return getProviderEnvValue("PI_OAUTH_CALLBACK_HOST") || "127.0.0.1";
}
type DeviceAuthInfo = {
packages/ai/src/utils/overflow.ts
+3 -2
View File
@@ -12,6 +12,7 @@ import type { AssistantMessage } from "../types.ts";
* - Anthropic: "413 {\"error\":{\"type\":\"request_too_large\",\"message\":\"Request exceeds the maximum size\"}}"
* - OpenAI: "Your input exceeds the context window of this model"
* - OpenAI/LiteLLM: "Requested token count exceeds the model's maximum context length of 131072 tokens"
* - OpenAI-compatible: "Input length (265330) exceeds model's maximum context length (262144)."
* - Google: "The input token count (1196265) exceeds the maximum number of tokens allowed (1048575)"
* - xAI: "This model's maximum prompt length is 131072 but the request contains 537812 tokens"
* - Groq: "Please reduce the length of the messages or completion"
@@ -36,7 +37,7 @@ const OVERFLOW_PATTERNS = [
/request_too_large/i, // Anthropic request byte-size overflow (HTTP 413)
/input is too long for requested model/i, // Amazon Bedrock
/exceeds the context window/i, // OpenAI (Completions & Responses API)
/exceeds (?:the )?(?:model'?s )?maximum context length of [\d,]+ tokens?/i, // OpenAI-compatible proxies (LiteLLM)
/exceeds (?:the )?(?:model'?s )?maximum context length(?: of [\d,]+ tokens?|\s*\([\d,]+\))/i, // OpenAI-compatible proxies (LiteLLM)
/input token count.*exceeds the maximum/i, // Google (Gemini)
/maximum prompt length is \d+/i, // xAI (Grok)
/reduce the length of the messages/i, // Groq
@@ -85,7 +86,7 @@ const NON_OVERFLOW_PATTERNS = [
*
* **Reliable detection (returns error with detectable message):**
* - Anthropic: "prompt is too long: X tokens > Y maximum" or "request_too_large"
* - OpenAI (Completions & Responses): "exceeds the context window" or "exceeds the model's maximum context length of X tokens"
* - OpenAI (Completions & Responses): "exceeds the context window", "exceeds the model's maximum context length of X tokens", or "exceeds model's maximum context length (X)"
* - Google Gemini: "input token count exceeds the maximum"
* - xAI (Grok): "maximum prompt length is X but request contains Y"
* - Groq: "reduce the length of the messages"
packages/ai/src/utils/provider-env.ts
+52
View File
@@ -0,0 +1,52 @@
import type { ProviderEnv } from "../types.ts";
let procEnvCache: Map<string, string> | null = null;
/**
* Fallback for https://github.com/oven-sh/bun/issues/27802.
* Bun compiled binaries can expose an empty process.env inside Linux sandboxes
* even though /proc/self/environ contains the environment.
*
* This intentionally duplicates restoreSandboxEnv() in
* packages/coding-agent/src/bun/restore-sandbox-env.ts. The ai package can be
* used directly, without going through that entrypoint, so provider env lookup
* must not depend on process.env having been patched.
*/
function getBunSandboxEnvValue(name: string): string | undefined {
if (typeof process === "undefined" || !process.versions?.bun || Object.keys(process.env).length > 0) {
return undefined;
}
if (procEnvCache === null) {
procEnvCache = new Map();
try {
const { readFileSync } = require("node:fs") as {
readFileSync(path: string, encoding: BufferEncoding): string;
};
const data = readFileSync("/proc/self/environ", "utf-8");
for (const entry of data.split("\0")) {
const idx = entry.indexOf("=");
if (idx > 0) {
procEnvCache.set(entry.slice(0, idx), entry.slice(idx + 1));
}
}
} catch {
// /proc/self/environ may not exist or may not be readable.
}
}
return procEnvCache.get(name);
}
/**
* Resolve a provider env value from scoped overrides, normal process.env, then
* the duplicated Bun sandbox fallback for direct pi-ai consumers.
*/
export function getProviderEnvValue(name: string, env?: ProviderEnv): string | undefined {
return (
env?.[name] ||
(typeof process !== "undefined" ? process.env[name] : undefined) ||
getBunSandboxEnvValue(name) ||
undefined
);
}
packages/ai/test/anthropic-adaptive-thinking-models.test.ts
+1 -2
View File
@@ -5,9 +5,8 @@ import type { Api, Model } from "../src/types.ts";
const EXPECTED_CURRENT_ADAPTIVE_THINKING_MODELS = [
"anthropic/claude-fable-5",
"anthropic/claude-opus-4-8",
"opencode/claude-fable-5",
"cloudflare-ai-gateway/claude-fable-5",
"opencode/claude-opus-4-8",
"vercel-ai-gateway/anthropic/claude-fable-5",
"vercel-ai-gateway/anthropic/claude-opus-4.8",
];
packages/ai/test/anthropic-cache-write-1h-cost.test.ts
+86
View File
@@ -0,0 +1,86 @@
import type Anthropic from "@anthropic-ai/sdk";
import { describe, expect, it } from "vitest";
import { stream as streamAnthropic } from "../src/api/anthropic-messages.ts";
import { getModel } from "../src/compat.ts";
import type { Context } from "../src/types.ts";
function createSseResponse(events: Array<{ event: string; data: string }>): Response {
const body = events.map(({ event, data }) => `event: ${event}\ndata: ${data}\n`).join("\n");
return new Response(body, { status: 200, headers: { "content-type": "text/event-stream" } });
}
function createFakeAnthropicClient(response: Response): Anthropic {
return {
messages: { create: () => ({ asResponse: async () => response }) },
} as unknown as Anthropic;
}
function eventsWithCacheCreation(
cacheCreation: Record<string, number> | undefined,
): Array<{ event: string; data: string }> {
const startUsage: Record<string, unknown> = {
input_tokens: 100,
output_tokens: 0,
cache_read_input_tokens: 0,
cache_creation_input_tokens: 1_000_000,
};
if (cacheCreation) startUsage.cache_creation = cacheCreation;
return [
{
event: "message_start",
data: JSON.stringify({ type: "message_start", message: { id: "msg_test", usage: startUsage } }),
},
{
event: "content_block_start",
data: JSON.stringify({ type: "content_block_start", index: 0, content_block: { type: "text", text: "" } }),
},
{
event: "content_block_delta",
data: JSON.stringify({ type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "Hi" } }),
},
{ event: "content_block_stop", data: JSON.stringify({ type: "content_block_stop", index: 0 }) },
{
event: "message_delta",
data: JSON.stringify({
type: "message_delta",
delta: { stop_reason: "end_turn" },
usage: {
input_tokens: 100,
output_tokens: 5,
cache_read_input_tokens: 0,
cache_creation_input_tokens: 1_000_000,
},
}),
},
{ event: "message_stop", data: JSON.stringify({ type: "message_stop" }) },
];
}
// claude-opus-4-8: input 5, cacheWrite (5m) 6.25 per Mtok. 1h write = 2x input = 10.
const context: Context = { messages: [{ role: "user", content: "hi", timestamp: Date.now() }] };
describe("Anthropic 1h cache write cost", () => {
it("prices the 1h portion at 2x input and the rest at the 5m rate", async () => {
const model = getModel("anthropic", "claude-opus-4-8");
const response = createSseResponse(
eventsWithCacheCreation({ ephemeral_5m_input_tokens: 600_000, ephemeral_1h_input_tokens: 400_000 }),
);
const result = await streamAnthropic(model, context, { client: createFakeAnthropicClient(response) }).result();
expect(result.usage.cacheWrite).toBe(1_000_000);
expect(result.usage.cacheWrite1h).toBe(400_000);
// 600k * 6.25/Mtok + 400k * 10/Mtok = 3.75 + 4.0 = 7.75
expect(result.usage.cost.cacheWrite).toBeCloseTo(7.75, 10);
});
it("falls back to the 5m rate when no breakdown is reported", async () => {
const model = getModel("anthropic", "claude-opus-4-8");
const response = createSseResponse(eventsWithCacheCreation(undefined));
const result = await streamAnthropic(model, context, { client: createFakeAnthropicClient(response) }).result();
expect(result.usage.cacheWrite).toBe(1_000_000);
expect(result.usage.cacheWrite1h ?? 0).toBe(0);
// 1M * 6.25/Mtok = 6.25
expect(result.usage.cost.cacheWrite).toBeCloseTo(6.25, 10);
});
});
packages/ai/test/anthropic-sse-parsing.test.ts
+58
View File
@@ -166,6 +166,64 @@ describe("Anthropic raw SSE parsing", () => {
});
});
it("preserves refusal stop details from message_delta", async () => {
const model = getModel("anthropic", "claude-fable-5");
const context: Context = {
messages: [{ role: "user", content: "blocked request", timestamp: Date.now() }],
};
const explanation =
"This request triggered restrictions on violative cyber content and was blocked under Anthropic's Usage Policy. To learn more, provide feedback, or request an exemption based on how you use Claude, visit our help center: https://support.claude.com/en/articles/14604842-real-time-cyber-safeguards-on-claude.";
const response = createSseResponse([
{
event: "message_start",
data: JSON.stringify({
type: "message_start",
message: {
id: "msg_01XFUDYJgAACzvnptvVoYEL",
usage: {
input_tokens: 412,
output_tokens: 0,
cache_read_input_tokens: 0,
cache_creation_input_tokens: 0,
},
},
}),
},
{
event: "message_delta",
data: JSON.stringify({
type: "message_delta",
delta: {
stop_reason: "refusal",
stop_details: {
type: "refusal",
category: "cyber",
explanation,
},
},
usage: {
input_tokens: 412,
output_tokens: 0,
cache_read_input_tokens: 0,
cache_creation_input_tokens: 0,
},
}),
},
{
event: "message_stop",
data: JSON.stringify({ type: "message_stop" }),
},
]);
const stream = streamAnthropic(model, context, {
client: createFakeAnthropicClient(response),
});
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toBe(explanation);
});
it("ignores unknown SSE events after message_stop", async () => {
const model = getModel("anthropic", "claude-haiku-4-5");
const context: Context = {
packages/ai/test/bedrock-endpoint-resolution.test.ts
+30 -3
View File
@@ -44,7 +44,7 @@ vi.mock("@aws-sdk/client-bedrock-runtime", () => {
};
});
import { stream as streamBedrock } from "../src/api/bedrock-converse-stream.ts";
import { type BedrockOptions, stream as streamBedrock } from "../src/api/bedrock-converse-stream.ts";
import { getModel } from "../src/compat.ts";
import type { Context, Model } from "../src/types.ts";
@@ -83,8 +83,12 @@ afterEach(() => {
}
});
async function captureClientConfig(model: Model<"bedrock-converse-stream">): Promise<Record<string, unknown>> {
await streamBedrock(model, context, { cacheRetention: "none" }).result();
async function captureClientConfig(
model: Model<"bedrock-converse-stream">,
options: BedrockOptions = {},
): Promise<Record<string, unknown>> {
bedrockMock.constructorCalls.length = 0;
await streamBedrock(model, context, { cacheRetention: "none", ...options }).result();
expect(bedrockMock.constructorCalls).toHaveLength(1);
return bedrockMock.constructorCalls[0];
}
@@ -115,6 +119,29 @@ describe("bedrock endpoint resolution", () => {
expect(config.region).toBe("eu-central-1");
});
it("handles missing regions for explicit, scoped, and ambient profiles", async () => {
const model = getModel("amazon-bedrock", "eu.anthropic.claude-sonnet-4-5-20250929-v1:0");
let config = await captureClientConfig(model, { profile: "bedrock-profile" });
expect(config.profile).toBe("bedrock-profile");
expect(config.endpoint).toBe("https://bedrock-runtime.eu-central-1.amazonaws.com");
expect(config.region).toBe("eu-central-1");
config = await captureClientConfig(model, { env: { AWS_PROFILE: "scoped-bedrock-profile" } });
expect(config.profile).toBe("scoped-bedrock-profile");
expect(config.endpoint).toBe("https://bedrock-runtime.eu-central-1.amazonaws.com");
expect(config.region).toBe("eu-central-1");
process.env.AWS_PROFILE = "ambient-bedrock-profile";
config = await captureClientConfig(model);
expect(config.profile).toBe("ambient-bedrock-profile");
expect(config.endpoint).toBeUndefined();
expect(config.region).toBeUndefined();
});
it("still passes custom Bedrock endpoints through to the SDK client", async () => {
process.env.AWS_REGION = "us-west-2";
const baseModel = getModel("amazon-bedrock", "us.anthropic.claude-opus-4-8");
packages/ai/test/cache-retention.test.ts
+40
View File
@@ -3,6 +3,7 @@ import { stream as streamAnthropic } from "../src/api/anthropic-messages.ts";
import { stream as streamOpenAICompletions } from "../src/api/openai-completions.ts";
import { stream as streamOpenAIResponses } from "../src/api/openai-responses.ts";
import { getModel, stream } from "../src/compat.ts";
import { MODELS } from "../src/models.generated.ts";
import type { Context, Model } from "../src/types.ts";
class PayloadCaptured extends Error {
@@ -12,6 +13,11 @@ class PayloadCaptured extends Error {
}
}
interface OpenAICompletionsCachePayload {
prompt_cache_key?: string;
prompt_cache_retention?: string;
}
function stopAfterPayload<TPayload>(capture: (payload: TPayload) => void): (payload: unknown) => never {
return (payload: unknown): never => {
capture(payload as TPayload);
@@ -454,5 +460,39 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
expect(capturedPayload.prompt_cache_key).toBeUndefined();
expect(capturedPayload.prompt_cache_retention).toBeUndefined();
});
it.each([
MODELS.opencode["deepseek-v4-flash"],
MODELS.opencode["deepseek-v4-pro"],
MODELS.opencode["kimi-k2.5"],
MODELS.opencode["kimi-k2.6"],
MODELS.opencode["minimax-m2.7"],
MODELS["opencode-go"]["kimi-k2.6"],
] as const)("should omit long cache retention for $provider/$id", async (metadata) => {
const model = metadata as Model<"openai-completions">;
let capturedPayload: OpenAICompletionsCachePayload | undefined;
try {
const s = streamOpenAICompletions(model, context, {
apiKey: "fake-key",
cacheRetention: "long",
sessionId: "session-opencode-long-cache-unsupported",
onPayload: stopAfterPayload<OpenAICompletionsCachePayload>((payload) => {
capturedPayload = payload;
}),
});
for await (const event of s) {
if (event.type === "error") break;
}
} catch {
// Expected to fail
}
expect(model.compat?.supportsLongCacheRetention).toBe(false);
expect(capturedPayload).toBeDefined();
expect(capturedPayload?.prompt_cache_key).toBeUndefined();
expect(capturedPayload?.prompt_cache_retention).toBeUndefined();
});
});
});
packages/ai/test/github-copilot-anthropic.test.ts
+11
View File
@@ -1,6 +1,7 @@
import { describe, expect, it, vi } from "vitest";
import { stream as streamAnthropic } from "../src/api/anthropic-messages.ts";
import { getModel } from "../src/compat.ts";
import { getSupportedThinkingLevels } from "../src/models.ts";
import type { Context } from "../src/types.ts";
const mockState = vi.hoisted(() => ({
@@ -54,6 +55,16 @@ describe("Copilot Claude via Anthropic Messages", () => {
messages: [{ role: "user", content: "Hello", timestamp: Date.now() }],
};
it("applies Copilot-specific adaptive thinking effort overrides", () => {
const opus47 = getModel("github-copilot", "claude-opus-4.7");
expect(opus47.thinkingLevelMap).toMatchObject({ minimal: "low", xhigh: "xhigh" });
expect(getSupportedThinkingLevels(opus47)).toContain("xhigh");
const sonnet46 = getModel("github-copilot", "claude-sonnet-4.6");
expect(sonnet46.thinkingLevelMap).toMatchObject({ minimal: "low", xhigh: "max" });
expect(getSupportedThinkingLevels(sonnet46)).toContain("xhigh");
});
it("uses Bearer auth, Copilot headers, and valid Anthropic Messages payload", async () => {
const model = getModel("github-copilot", "claude-sonnet-4.6");
expect(model.api).toBe("anthropic-messages");
packages/ai/test/github-copilot-oauth.test.ts
+69 -1
View File
@@ -1,5 +1,10 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { loginGitHubCopilot } from "../src/utils/oauth/github-copilot.ts";
import { getModels } from "../src/compat.ts";
import {
githubCopilotOAuthProvider,
loginGitHubCopilot,
refreshGitHubCopilotToken,
} from "../src/utils/oauth/github-copilot.ts";
function jsonResponse(body: unknown, status: number = 200): Response {
return new Response(JSON.stringify(body), {
@@ -29,6 +34,57 @@ describe("GitHub Copilot OAuth device flow", () => {
vi.useRealTimers();
});
it("filters models to the authenticated account picker catalog", async () => {
const fetchMock = vi.fn(async (input: unknown, init?: RequestInit): Promise<Response> => {
const url = getUrl(input);
if (url.includes("/copilot_internal/v2/token")) {
return jsonResponse({
token: "tid=test;exp=9999999999;proxy-ep=proxy.individual.githubcopilot.com;",
expires_at: 9999999999,
});
}
if (url === "https://api.individual.githubcopilot.com/models") {
expect(init?.headers).toMatchObject({
Authorization: "Bearer tid=test;exp=9999999999;proxy-ep=proxy.individual.githubcopilot.com;",
});
return jsonResponse({
data: [
{
id: "gpt-4.1",
model_picker_enabled: true,
capabilities: { supports: { tool_calls: true } },
},
{
id: "claude-opus-4.7",
model_picker_enabled: true,
policy: { state: "disabled" },
capabilities: { supports: { tool_calls: true } },
},
{
id: "gpt-5.4-nano",
model_picker_enabled: false,
capabilities: { supports: { tool_calls: true } },
},
],
});
}
throw new Error(`Unexpected fetch URL: ${url}`);
});
vi.stubGlobal("fetch", fetchMock);
const credentials = await refreshGitHubCopilotToken("ghu_refresh_token");
expect(credentials.availableModelIds).toEqual(["gpt-4.1"]);
const modifiedModels = githubCopilotOAuthProvider.modifyModels?.(getModels("github-copilot"), credentials) ?? [];
expect(modifiedModels.filter((model) => model.provider === "github-copilot").map((model) => model.id)).toEqual([
"gpt-4.1",
]);
});
it("reports device-code details through onDeviceCode", async () => {
vi.useFakeTimers();
vi.setSystemTime(new Date("2026-03-09T00:00:00Z"));
@@ -57,6 +113,10 @@ describe("GitHub Copilot OAuth device flow", () => {
});
}
if (url.endsWith("/models")) {
return jsonResponse({ data: [] });
}
if (url.includes("/models/") && url.endsWith("/policy")) {
return new Response("", { status: 200 });
}
@@ -146,6 +206,10 @@ describe("GitHub Copilot OAuth device flow", () => {
});
}
if (url.endsWith("/models")) {
return jsonResponse({ data: [] });
}
if (url.includes("/models/") && url.endsWith("/policy")) {
return new Response("", { status: 200 });
}
@@ -231,6 +295,10 @@ describe("GitHub Copilot OAuth device flow", () => {
});
}
if (url.endsWith("/models")) {
return jsonResponse({ data: [] });
}
if (url.includes("/models/") && url.endsWith("/policy")) {
return new Response("", { status: 200 });
}
packages/ai/test/mistral-reasoning-mode.test.ts
+18
View File
@@ -5,6 +5,7 @@ import type { Context, Model, SimpleStreamOptions } from "../src/types.ts";
interface MistralPayload {
promptMode?: "reasoning";
reasoningEffort?: "none" | "high";
promptCacheKey?: string;
}
function makeContext(): Context {
@@ -76,4 +77,21 @@ describe("Mistral reasoning mode selection", () => {
expect(payload.reasoningEffort).toBeUndefined();
expect(payload.promptMode).toBeUndefined();
});
it("uses the session id as prompt cache key", async () => {
const payload = await capturePayload(getModel("mistral", "mistral-large-latest"), {
sessionId: "session-123",
});
expect(payload.promptCacheKey).toBe("session-123");
});
it("omits prompt cache key when cache retention is disabled", async () => {
const payload = await capturePayload(getModel("mistral", "mistral-large-latest"), {
sessionId: "session-123",
cacheRetention: "none",
});
expect(payload.promptCacheKey).toBeUndefined();
});
});
packages/ai/test/node-http-proxy.test.ts
+11
View File
@@ -54,6 +54,17 @@ describe("node HTTP proxy resolution", () => {
);
});
it("prefers scoped proxy env aliases before process env aliases", () => {
resetProxyEnv();
process.env.https_proxy = "http://process-proxy.example:8080";
expect(
resolveHttpProxyUrlForTarget("https://bedrock-runtime.us-east-1.amazonaws.com", {
HTTPS_PROXY: "http://scoped-proxy.example:8080",
})?.toString(),
).toBe("http://scoped-proxy.example:8080/");
});
it("rejects SOCKS and PAC proxy URLs explicitly", () => {
resetProxyEnv();
process.env.HTTPS_PROXY = "socks5://proxy.example:1080";
packages/ai/test/openai-codex-stream.test.ts
+10 -2
View File
@@ -361,13 +361,21 @@ describe("openai-codex streaming", () => {
apiKey: token,
transport: "sse",
}).result();
let settled = false;
const observedResultPromise = resultPromise.then((result) => {
settled = true;
return result;
});
await vi.advanceTimersByTimeAsync(0);
expect(fetchMock).toHaveBeenCalledTimes(1);
await vi.advanceTimersByTimeAsync(10_000);
const result = await resultPromise;
expect(settled).toBe(false);
await vi.advanceTimersByTimeAsync(10_000);
const result = await observedResultPromise;
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toBe("Codex SSE response headers timed out after 10000ms");
expect(result.errorMessage).toBe("Codex SSE response headers timed out after 20000ms");
});
it("aborts SSE body reads after response headers arrive", async () => {
packages/ai/test/openai-completions-empty-tools.test.ts
+25
View File
@@ -162,6 +162,31 @@ describe("openai-completions empty tools handling", () => {
expect(clientOptions.defaultHeaders?.["cf-aig-authorization"]).toBe("Bearer test");
});
it("uses provider env before process.env for Cloudflare AI Gateway base URL", async () => {
process.env.CLOUDFLARE_ACCOUNT_ID = "process-account";
process.env.CLOUDFLARE_GATEWAY_ID = "process-gateway";
const model = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!;
await streamSimple(
model,
{
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{
apiKey: "test",
env: {
CLOUDFLARE_ACCOUNT_ID: "provider-account",
CLOUDFLARE_GATEWAY_ID: "provider-gateway",
},
},
).result();
const clientOptions = mockState.lastClientOptions as { baseURL?: string };
expect(clientOptions.baseURL).toBe(
"https://gateway.ai.cloudflare.com/v1/provider-account/provider-gateway/compat",
);
});
it("preserves inline upstream Authorization for Cloudflare AI Gateway BYOK requests", async () => {
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
packages/ai/test/openai-completions-reasoning-details.test.ts
+118
View File
@@ -0,0 +1,118 @@
import { Type } from "typebox";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { stream as streamOpenAICompletions } from "../src/api/openai-completions.ts";
import type { AssistantMessage, Model, Tool } from "../src/types.ts";
const mockState = vi.hoisted(() => ({
chunkSets: [] as unknown[][],
payloads: [] as unknown[],
}));
vi.mock("openai", () => {
class FakeOpenAI {
chat = {
completions: {
create: (payload: unknown) => {
mockState.payloads.push(payload);
const chunks = mockState.chunkSets.shift() ?? [];
const stream = {
async *[Symbol.asyncIterator]() {
for (const chunk of chunks) {
yield chunk;
}
},
};
const result = Promise.resolve(stream) as Promise<typeof stream> & {
withResponse: () => Promise<{ data: typeof stream; response: { status: number; headers: Headers } }>;
};
result.withResponse = async () => ({
data: stream,
response: { status: 200, headers: new Headers() },
});
return result;
},
},
};
}
return { default: FakeOpenAI };
});
const reasoningDetail = { type: "reasoning.encrypted", id: "call_1", data: "encrypted-signature" };
const readTool: Tool = {
name: "read",
description: "Read a file",
parameters: Type.Object({ path: Type.String() }),
};
function model(): Model<"openai-completions"> {
return {
id: "google/gemini-test",
name: "Gemini Test",
api: "openai-completions",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 100_000,
maxTokens: 4096,
};
}
function chunk(delta: Record<string, unknown>, finishReason: string | null = null): unknown {
return {
id: "chatcmpl-test",
model: "google/gemini-test",
choices: [{ index: 0, delta, finish_reason: finishReason }],
};
}
function toolCallChunk(): unknown {
return chunk({
tool_calls: [
{
index: 0,
id: "call_1",
type: "function",
function: { name: "read", arguments: '{"path":"README.md"}' },
},
],
});
}
async function runOpenAICompletionsStream(messages: AssistantMessage[] = []): Promise<AssistantMessage> {
return await streamOpenAICompletions(model(), { messages, tools: [readTool] }, { apiKey: "test" }).result();
}
function getAssistantPayload(payload: unknown): { reasoning_details?: unknown } | undefined {
const messages = (payload as { messages?: Array<{ role?: string; reasoning_details?: unknown }> }).messages ?? [];
return messages.find((message) => message.role === "assistant");
}
describe("openai-completions reasoning_details streaming", () => {
beforeEach(() => {
mockState.chunkSets = [];
mockState.payloads = [];
});
it("preserves reasoning_details that arrive before their matching tool call", async () => {
mockState.chunkSets = [
[chunk({ reasoning_details: [reasoningDetail] }), toolCallChunk(), chunk({}, "tool_calls")],
[chunk({ content: "ok" }), chunk({}, "stop")],
];
const assistantMessage = await runOpenAICompletionsStream();
const toolCall = assistantMessage.content.find((block) => block.type === "toolCall");
expect(toolCall).toMatchObject({
type: "toolCall",
id: "call_1",
name: "read",
arguments: { path: "README.md" },
thoughtSignature: JSON.stringify(reasoningDetail),
});
await runOpenAICompletionsStream([assistantMessage]);
expect(getAssistantPayload(mockState.payloads[1])?.reasoning_details).toEqual([reasoningDetail]);
});
});
packages/ai/test/openai-completions-thinking-as-text.test.ts
+1
View File
@@ -34,6 +34,7 @@ const compat = {
thinkingFormat: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: true,
cacheControlFormat: undefined,
packages/ai/test/openai-completions-tool-choice.test.ts
+241 -1
View File
@@ -2,7 +2,7 @@ import { Type } from "typebox";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { convertMessages } from "../src/api/openai-completions.ts";
import { getModel, stream, streamSimple } from "../src/compat.ts";
import type { AssistantMessage, Model, Tool, ToolResultMessage } from "../src/types.ts";
import type { AssistantMessage, Model, SimpleStreamOptions, Tool, ToolResultMessage } from "../src/types.ts";
const mockState = vi.hoisted(() => ({
lastParams: undefined as unknown,
@@ -63,6 +63,46 @@ vi.mock("openai", () => {
return { default: FakeOpenAI };
});
const localOpenAICompletionsModel = {
api: "openai-completions",
provider: "local-vllm",
baseUrl: "http://localhost:8000/v1",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128000,
maxTokens: 8192,
} satisfies Omit<Model<"openai-completions">, "id" | "name" | "compat">;
type CapturedParams = {
chat_template_kwargs?: Record<string, unknown>;
thinking?: unknown;
reasoning_effort?: string;
};
async function captureSimpleParams(
model: Model<"openai-completions">,
reasoning?: SimpleStreamOptions["reasoning"],
): Promise<CapturedParams> {
let payload: unknown;
await streamSimple(
model,
{
messages: [{ role: "user", content: "Hi", timestamp: Date.now() }],
},
{
apiKey: "test",
reasoning,
onPayload: (params: unknown) => {
payload = params;
},
},
).result();
return (payload ?? mockState.lastParams) as CapturedParams;
}
describe("openai-completions tool_choice", () => {
beforeEach(() => {
mockState.lastParams = undefined;
@@ -256,6 +296,86 @@ describe("openai-completions tool_choice", () => {
expect(getModel("zai", "glm-4.5-air")?.compat?.zaiToolStream).toBeUndefined();
});
it("stores z.ai GLM-5.2 effort metadata", () => {
for (const provider of ["zai", "zai-coding-cn"] as const) {
const model = getModel(provider, "glm-5.2")!;
expect(model.compat?.supportsReasoningEffort).toBe(true);
expect(model.thinkingLevelMap).toEqual({
minimal: null,
low: "high",
medium: "high",
high: "high",
xhigh: "max",
});
}
});
it("maps z.ai GLM-5.2 thinking levels to reasoning_effort", async () => {
const model = getModel("zai", "glm-5.2")!;
const cases = [
{ reasoning: "low", effort: "high" },
{ reasoning: "medium", effort: "high" },
{ reasoning: "high", effort: "high" },
{ reasoning: "xhigh", effort: "max" },
] as const;
for (const testCase of cases) {
let payload: unknown;
await streamSimple(
model,
{
messages: [
{
role: "user",
content: "Hi",
timestamp: Date.now(),
},
],
},
{
apiKey: "test",
reasoning: testCase.reasoning,
onPayload: (params: unknown) => {
payload = params;
},
},
).result();
const params = (payload ?? mockState.lastParams) as { thinking?: unknown; reasoning_effort?: string };
expect(params.thinking).toEqual({ type: "enabled" });
expect(params.reasoning_effort).toBe(testCase.effort);
}
});
it("omits z.ai GLM-5.2 reasoning_effort when thinking is off", async () => {
const model = getModel("zai", "glm-5.2")!;
let payload: unknown;
await streamSimple(
model,
{
messages: [
{
role: "user",
content: "Hi",
timestamp: Date.now(),
},
],
},
{
apiKey: "test",
onPayload: (params: unknown) => {
payload = params;
},
},
).result();
const params = (payload ?? mockState.lastParams) as { thinking?: unknown; reasoning_effort?: string };
expect(params.thinking).toEqual({ type: "disabled" });
expect(params.reasoning_effort).toBeUndefined();
});
it("omits tool_stream for unsupported z.ai models", async () => {
const model = getModel("zai", "glm-4.5-air")!;
const tools: Tool[] = [
@@ -1063,6 +1183,7 @@ describe("openai-completions tool_choice", () => {
thinkingFormat: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: true,
sendSessionAffinityHeaders: false,
@@ -1119,6 +1240,54 @@ describe("openai-completions tool_choice", () => {
expect(params.reasoning_effort).toBeUndefined();
});
it("omits disabled thinking for Moonshot Kimi K2.7 Code models", async () => {
const cases = [getModel("moonshotai", "kimi-k2.7-code"), getModel("moonshotai-cn", "kimi-k2.7-code")];
for (const model of cases) {
expect(model).toBeDefined();
let payload: unknown;
await streamSimple(
model!,
{
messages: [{ role: "user", content: "Hi", timestamp: Date.now() }],
},
{
apiKey: "test",
onPayload: (params: unknown) => {
payload = params;
},
},
).result();
const params = (payload ?? mockState.lastParams) as { thinking?: unknown; reasoning_effort?: string };
expect(params.thinking).toBeUndefined();
expect(params.reasoning_effort).toBeUndefined();
}
});
it("keeps disabled thinking for Moonshot Kimi K2.6 when thinking is off", async () => {
const model = getModel("moonshotai-cn", "kimi-k2.6")!;
let payload: unknown;
await streamSimple(
model,
{
messages: [{ role: "user", content: "Hi", timestamp: Date.now() }],
},
{
apiKey: "test",
onPayload: (params: unknown) => {
payload = params;
},
},
).result();
const params = (payload ?? mockState.lastParams) as { thinking?: unknown; reasoning_effort?: string };
expect(params.thinking).toEqual({ type: "disabled" });
expect(params.reasoning_effort).toBeUndefined();
});
it("sends max_tokens for OpenCode completions models", async () => {
const cases = [getModel("opencode-go", "kimi-k2.6")!, getModel("opencode", "grok-build-0.1")!] as const;
@@ -1322,6 +1491,77 @@ describe("openai-completions tool_choice", () => {
expect(params.reasoning_effort).toBeUndefined();
});
it("uses configurable chat template boolean thinking kwargs", async () => {
const model = {
...localOpenAICompletionsModel,
id: "deepseek-ai/DeepSeek-V3.1",
name: "DeepSeek V3.1 via vLLM",
compat: {
thinkingFormat: "chat-template",
supportsReasoningEffort: false,
chatTemplateKwargs: { thinking: { $var: "thinking.enabled" } },
},
} satisfies Model<"openai-completions">;
for (const testCase of [
{ reasoning: "high" as const, expected: true },
{ reasoning: undefined, expected: false },
]) {
const params = await captureSimpleParams(model, testCase.reasoning);
expect(params.chat_template_kwargs).toEqual({ thinking: testCase.expected });
expect(params.thinking).toBeUndefined();
expect(params.reasoning_effort).toBeUndefined();
}
});
it("uses qwen chat template thinking kwargs", async () => {
const model = {
...localOpenAICompletionsModel,
id: "Qwen/Qwen3-Coder",
name: "Qwen3 Coder via vLLM",
compat: {
thinkingFormat: "qwen-chat-template",
supportsReasoningEffort: false,
},
} satisfies Model<"openai-completions">;
for (const testCase of [
{ reasoning: "high" as const, expected: true },
{ reasoning: undefined, expected: false },
]) {
const params = await captureSimpleParams(model, testCase.reasoning);
expect(params.chat_template_kwargs).toEqual({
enable_thinking: testCase.expected,
preserve_thinking: true,
});
expect(params.reasoning_effort).toBeUndefined();
}
});
it("uses configurable chat template effort kwargs with static kwargs", async () => {
const model = {
...localOpenAICompletionsModel,
id: "unsloth/gpt-oss-120b-GGUF",
name: "GPT OSS via vLLM",
thinkingLevelMap: { xhigh: "max" },
compat: {
thinkingFormat: "chat-template",
supportsReasoningEffort: false,
chatTemplateKwargs: {
preserve_thinking: true,
reasoning_effort: { $var: "thinking.effort", omitWhenOff: true },
},
},
} satisfies Model<"openai-completions">;
const params = await captureSimpleParams(model, "xhigh");
expect(params.chat_template_kwargs).toEqual({ preserve_thinking: true, reasoning_effort: "max" });
expect(params.reasoning_effort).toBeUndefined();
});
it("uses Ant Ling compatibility metadata", async () => {
const model = getModel("ant-ling", "Ring-2.6-1T")!;
let payload: unknown;
packages/ai/test/openai-completions-tool-result-images.test.ts
+1
View File
@@ -32,6 +32,7 @@ const compat: Required<OpenAICompletionsCompat> = {
thinkingFormat: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: true,
cacheControlFormat: "anthropic",
packages/ai/test/overflow.test.ts
+7
View File
@@ -49,6 +49,13 @@ describe("isContextOverflow", () => {
expect(isContextOverflow(message, 131072)).toBe(true);
});
it("detects OpenAI-compatible parenthesized maximum context length errors", () => {
const message = createErrorMessage(
"Error: 400 Input length (265330) exceeds model's maximum context length (262144).",
);
expect(isContextOverflow(message, 262144)).toBe(true);
});
it("detects OpenRouter Poolside maximum allowed input length errors", () => {
const message = createErrorMessage(
"Provider returned error: Input length 131393 exceeds the maximum allowed input length of 131040 tokens.",
packages/ai/test/supports-xhigh.test.ts
+9
View File
@@ -69,6 +69,15 @@ describe("getSupportedThinkingLevels", () => {
expect(getSupportedThinkingLevels(model!)).toEqual(["off", "high"]);
});
it("excludes thinking off for Moonshot Kimi K2.7 Code models", () => {
const cases = [getModel("moonshotai", "kimi-k2.7-code"), getModel("moonshotai-cn", "kimi-k2.7-code")];
for (const model of cases) {
expect(model).toBeDefined();
expect(getSupportedThinkingLevels(model!)).toEqual(["minimal", "low", "medium", "high"]);
}
});
it("includes only high for OpenCode Grok Build", () => {
const model = getModel("opencode", "grok-build-0.1");
expect(model).toBeDefined();