ton/pi_harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ai): complete models runtime migration

Browse Source
This commit is contained in:
Mario Zechner
2026-06-23 15:29:17 +02:00
parent 470a4736a3
commit 129eb460cd
47 changed files with 1502 additions and 576 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/agent/docs/models.md
+8 -7
View File
@@ -702,10 +702,10 @@ Built-in provider factories use `createProvider()` internally. models.json custo
`@earendil-works/pi-ai/compat` preserves the old global API surface until the coding-agent migration deletes it. New code never imports it.
Old semantics being preserved: global `stream()` dispatched purely on `model.api` via the api-registry, with env API key injection. The compat module reproduces this exactly — it does not route through a `Models` collection, so compat consumers get zero behavioral drift (a `Models`-routed variant was considered and dropped: a model with a known provider id but a different api would dispatch wrong, and auth semantics would shift mid-migration). The harness `Models` instance (Phase 6/7) is where new-path streaming happens.
Old semantics being preserved: global `stream()` can still dispatch by `model.api` through the legacy api-registry for custom providers, mutated models, and tests/extensions that override a built-in API implementation.
- `stream/complete/streamSimple/completeSimple(model, ctx, opts)`: api-dispatch via the api-registry plus `getEnvApiKey` injection, verbatim old behavior.
- The builtin api registration side effect moves from the root barrel into compat. It skips api ids that already have a registration, since compat may load after a test or extension registered an override. `registerApiProvider()/unregisterApiProviders()` keep feeding the registry; `resetApiProviders()` clears and re-registers builtins.
- `stream/complete/streamSimple/completeSimple(model, ctx, opts)`: real built-in provider/model/api matches route through a singleton `builtinModels()` collection, so provider auth/env/baseUrl behavior is shared with the new runtime. Unknown providers, mutated models, or overridden API registrations fall back to api-registry dispatch plus `getEnvApiKey` injection.
- The builtin api registration side effect moves from the root barrel into compat. It skips api ids that already have a registration, since compat may load after a test or extension has already registered an override. `registerApiProvider()/unregisterApiProviders()` keep feeding the compat-local registry; `resetApiProviders()` clears and re-registers builtins.
- Sync `getModel/getModels/getProviders` are deprecated aliases of `getBuiltinModel/getBuiltinModels/getBuiltinProviders` from `providers/all` (they were always pure generated-catalog reads — verified: nothing ever mutated the old `modelRegistry`).
- Re-exports the per-API lazy stream wrappers (incl. `setBedrockProviderModule`), `env-api-keys.ts`, and the image-generation registry/catalogs; none of these stay on the root barrel.
- `export * from "./index.ts"`: compat is a strict superset of the core entrypoint, so consumers switch a file's import path wholesale without symbol surgery.
@@ -873,15 +873,16 @@ Decisions:
- Runtime `--api-key` overrides are a store overlay (an override reads as an ephemeral stored api-key credential, masking stored OAuth — matches today's priority). Every registered provider is guaranteed an `apiKey` auth slot so overrides apply to OAuth-only providers too.
- `ModelRegistry.getAll`/`find`/`getAvailable` become async, delegating to the collection (no materialized snapshot, no sync lies; dynamic providers like llama.cpp work). The extension-facing `modelRegistry` surface changes accordingly (breaking, changelogged); extensions also get the collection itself as the forward API.
- models.json keeps FULL feature parity, implemented as provider decoration: builtin factories wrapped so `getModels()` applies provider `baseUrl`/`compat` overlays, `modelOverrides`, and custom-model merges (async-safe); provider `apiKey`/`headers`/`authHeader` configs become that provider's `ApiKeyAuth` (config first, factory auth fallback); parse errors keep `getError()` semantics.
- Extension `ProviderConfig` parity: provider-keyed `streamSimple`, old-style `oauth` adapted to `OAuthAuth` (`modifyModels` -> `getModels` wrap + `toAuth`), full model replacement per provider. The api-registry `registerApiProvider` dual-write stays for compat consumers (extensions calling global `complete()`); it dies with compat.
- Extension `ProviderConfig` parity: provider-keyed `streamSimple`, old-style `oauth` adapted to `OAuthAuth` (`modifyModels` -> `getModels` wrap + `toAuth`), full model replacement per provider. Legacy `registerApiProvider` writes stay compat-local for consumers that call global `complete()`; they die with compat.
- Copilot: stored-credential baseUrl applied in the wrapped `getModels()` (extension-visible models stay correct) plus per-request `toAuth().baseUrl`.
- Cloudflare: provider-auth substitution (key + `CLOUDFLARE_ACCOUNT_ID`/`CLOUDFLARE_GATEWAY_ID` from credential metadata/env -> `ModelAuth.baseUrl`); impl-side `resolveCloudflareBaseUrl` stays until compat dies (it is idempotent on substituted URLs), keeping extension `complete()` calls working.
- Cloudflare: provider-auth substitution (key + `CLOUDFLARE_ACCOUNT_ID`/`CLOUDFLARE_GATEWAY_ID` from credential metadata/env -> `ModelAuth.baseUrl`). Built-in compat calls route through `Models`, so they use the same provider auth path.
Ordering:
- [x] pi-ai rework first: `Provider.getModels()` sync + optional `refreshModels()`; `Models.getModels`/`getModel` sync, `Models.refresh(provider?)` async; `createProvider` takes `models` array + optional `refreshModels` fetcher (in-flight dedupe). Reverses Phase 1's async-listing decision — see "Provider model listing" for rationale (sync-or-async unions breed latent sync assumptions; async-only breaks sync consumer surfaces like extension `find`/`getAll`).
- [ ] `FileCredentialStore` (ports the auth.json lock backend, reads legacy `type: "api_key"` tags) + `--api-key` overlay + `$ENV`/`!command` resolution; tests.
- [ ] Cloudflare provider auth in pi-ai factories; copilot `getModels` baseUrl wrap.
- [x] Cloudflare provider auth in pi-ai factories.
- [ ] Copilot `getModels` baseUrl wrap.
- [ ] Extension-OAuth adapter (old `OAuthProviderInterface` config -> `OAuthAuth`).
- [ ] ModelRegistry rebuild: owns `MutableModels`, async reads, models.json decoration, provider-keyed extension streams, facade auth ops; AuthStorage deleted.
- [ ] Consumer rewiring: agent-session, sdk (`credentials?: CredentialStore` option replaces `authStorage`; sdk.md updated), model-resolver, interactive login/status UI on `prompt()/notify()`, cli `--api-key`.
@@ -891,7 +892,7 @@ Ordering:
- [ ] AgentSession -> AgentHarness; the registry facade dies in favor of harness `Models`.
- [ ] Move ALL internal `/compat` imports to the new API: every package's src, all tests, and the example extensions (examples then demonstrate the new API). Nothing inside the repo may import `/compat` at that point.
- [ ] Delete `/compat`, `api-registry.ts`, `env-api-keys.ts`, the extension-loader root-to-compat alias, the old `pi-ai/oauth` registry and `OAuthProviderInterface` (incl. `usesCallbackServer`), and the impl-side cloudflare substitution. This is the extension-author breaking release; changelog carries the migration guide.
- [ ] Delete `/compat`, `env-api-keys.ts`, the extension-loader root-to-compat alias, the old `pi-ai/oauth` registry and `OAuthProviderInterface` (incl. `usesCallbackServer`), and the compat-local legacy API registry. This is the extension-author breaking release; changelog carries the migration guide.
### Deferred / follow-ups
packages/agent/test/e2e.test.ts
+1 -1
View File
@@ -9,7 +9,7 @@ import {
registerFauxProvider,
type ToolResultMessage,
type UserMessage,
} from "@earendil-works/pi-ai";
} from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest";
import { Agent, type AgentEvent } from "../src/index.ts";
import { calculateTool } from "./utils/calculate.ts";
packages/ai/CHANGELOG.md
+28
View File
@@ -8,6 +8,33 @@
- Renamed the `Provider` type to `ProviderId`. `Provider` now names the runtime provider interface (id, name, auth, model listing, stream behavior).
- API implementation modules moved from `src/providers/` to `@earendil-works/pi-ai/api/*`, renamed by API id (`anthropic` -> `api/anthropic-messages`, `google` -> `api/google-generative-ai`, `mistral` -> `api/mistral-conversations`, `amazon-bedrock` -> `api/bedrock-converse-stream`), each exporting exactly `stream` and `streamSimple`. The old per-impl export names (`streamAnthropic`, `streamSimpleAnthropic`, ...) are gone; the legacy package subpaths (`./anthropic`, `./google`, ...) keep working and point at the new modules.
Migration guide:
- To keep the old global API temporarily, change imports from `@earendil-works/pi-ai` to `@earendil-works/pi-ai/compat`. The compat entrypoint preserves `stream`/`complete`, generated catalog reads, API registry helpers, env API-key helpers, lazy API wrappers, and image globals, but it will be removed in a future release.
- To migrate to the new runtime, create a `Models` collection and call methods on it:
```ts
import { builtinModels } from "@earendil-works/pi-ai/providers/all";
const models = builtinModels();
const model = models.getModel("anthropic", "claude-haiku-4-5");
if (!model) throw new Error("model not found");
const message = await models.complete(model, {
messages: [{ role: "user", content: "Hello", timestamp: Date.now() }],
});
```
- For an isolated provider set, register provider factories explicitly:
```ts
import { createModels } from "@earendil-works/pi-ai";
import { anthropicProvider } from "@earendil-works/pi-ai/providers/anthropic";
const models = createModels();
models.setProvider(anthropicProvider());
```
### Added
- New `Models` runtime: `createModels()` builds an isolated provider collection with sync model reads (`getModels`/`getModel` return the last-known lists), an explicit async `refresh(provider?)` for dynamic providers, auth resolution (`getAuth`), and `stream`/`complete`/`streamSimple`/`completeSimple` that resolve auth through the owning provider. `createProvider()` builds providers from parts (single API implementation or a map dispatched on `model.api`; static `models` array plus an optional `refreshModels` fetcher with in-flight dedupe); `hasApi()` narrows dynamically listed models.
@@ -22,6 +49,7 @@
### Fixed
- Fixed Amazon Bedrock endpoint resolution to honor scoped `AWS_PROFILE` values.
- Fixed Cloudflare providers to require account/gateway configuration and route built-in `/compat` requests through provider auth.
- Fixed OpenAI Codex Responses WebSocket sessions to reconnect once when OpenAI's connection limit is reached before output starts ([#5973](https://github.com/earendil-works/pi/issues/5973)).
- Fixed OpenCode Go GLM-5.2 metadata to expose `xhigh` reasoning and send `reasoning_effort: "max"` ([#5967](https://github.com/earendil-works/pi/issues/5967)).
- Fixed Claude Fable 5 thinking-off requests to omit Anthropic's unsupported `thinking.type: "disabled"` payload ([#5567](https://github.com/earendil-works/pi/pull/5567) by [@tmustier](https://github.com/tmustier)).
packages/ai/scripts/generate-models.ts
+143 -3
View File
@@ -277,10 +277,148 @@ function isAnthropicTemperatureUnsupportedModel(modelId: string): boolean {
return id.includes("opus-4-7") || id.includes("opus-4.7") || id.includes("opus-4-8") || id.includes("opus-4.8");
}
const OPENAI_COMPLETIONS_DEFAULT_COMPAT = {
supportsStore: true,
supportsDeveloperRole: true,
supportsReasoningEffort: true,
supportsUsageInStreaming: true,
maxTokensField: "max_completion_tokens",
requiresToolResultName: false,
requiresAssistantAfterToolResult: false,
requiresThinkingAsText: false,
requiresReasoningContentOnAssistantMessages: false,
thinkingFormat: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: true,
sendSessionAffinityHeaders: false,
supportsLongCacheRetention: true,
} satisfies Required<Omit<OpenAICompletionsCompat, "cacheControlFormat">> & {
cacheControlFormat?: OpenAICompletionsCompat["cacheControlFormat"];
};
type OpenAICompletionsResolvedCompat = typeof OPENAI_COMPLETIONS_DEFAULT_COMPAT & {
cacheControlFormat?: OpenAICompletionsCompat["cacheControlFormat"];
};
function mergeAnthropicMessagesCompat(model: Model<Api>, compat: AnthropicMessagesCompat): void {
model.compat = { ...(model.compat as AnthropicMessagesCompat | undefined), ...compat };
}
function detectOpenAICompletionsCompat(model: Model<"openai-completions">): OpenAICompletionsResolvedCompat {
const provider = model.provider;
const baseUrl = model.baseUrl;
const isZai =
provider === "zai" ||
provider === "zai-coding-cn" ||
baseUrl.includes("api.z.ai") ||
baseUrl.includes("open.bigmodel.cn");
const isTogether =
provider === "together" || baseUrl.includes("api.together.ai") || baseUrl.includes("api.together.xyz");
const isMoonshot = provider === "moonshotai" || provider === "moonshotai-cn" || baseUrl.includes("api.moonshot.");
const isOpenRouter = provider === "openrouter" || baseUrl.includes("openrouter.ai");
const isCloudflareWorkersAI = provider === "cloudflare-workers-ai" || baseUrl.includes("api.cloudflare.com");
const isCloudflareAiGateway = provider === "cloudflare-ai-gateway" || baseUrl.includes("gateway.ai.cloudflare.com");
const isNvidia = provider === "nvidia" || baseUrl.includes("integrate.api.nvidia.com");
const isAntLing = provider === "ant-ling" || baseUrl.includes("api.ant-ling.com");
const isNonStandard =
isNvidia ||
provider === "cerebras" ||
baseUrl.includes("cerebras.ai") ||
provider === "xai" ||
baseUrl.includes("api.x.ai") ||
isTogether ||
baseUrl.includes("chutes.ai") ||
baseUrl.includes("deepseek.com") ||
isZai ||
isMoonshot ||
provider === "opencode" ||
baseUrl.includes("opencode.ai") ||
isCloudflareWorkersAI ||
isCloudflareAiGateway ||
isAntLing;
const useMaxTokens =
baseUrl.includes("chutes.ai") || isMoonshot || isCloudflareAiGateway || isTogether || isNvidia || isAntLing;
const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
const isDeepSeek = provider === "deepseek" || baseUrl.includes("deepseek.com");
const isOpenRouterDeveloperRoleModel =
isOpenRouter && (model.id.startsWith("anthropic/") || model.id.startsWith("openai/"));
const cacheControlFormat = provider === "openrouter" && model.id.startsWith("anthropic/") ? "anthropic" : undefined;
return {
supportsStore: !isNonStandard,
supportsDeveloperRole: isOpenRouterDeveloperRoleModel || (!isNonStandard && !isOpenRouter),
supportsReasoningEffort:
!isGrok && !isZai && !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia && !isAntLing,
supportsUsageInStreaming: true,
maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens",
requiresToolResultName: false,
requiresAssistantAfterToolResult: false,
requiresThinkingAsText: false,
requiresReasoningContentOnAssistantMessages: isDeepSeek,
thinkingFormat: isDeepSeek
? "deepseek"
: isZai
? "zai"
: isTogether
? "together"
: isAntLing
? "ant-ling"
: isOpenRouter
? "openrouter"
: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia,
...(cacheControlFormat ? { cacheControlFormat } : {}),
sendSessionAffinityHeaders: false,
supportsLongCacheRetention: !(
isTogether ||
isCloudflareWorkersAI ||
isCloudflareAiGateway ||
isNvidia ||
isAntLing
),
};
}
function isPlainEmptyObject(value: unknown): boolean {
return typeof value === "object" && value !== null && !Array.isArray(value) && Object.keys(value).length === 0;
}
function openAICompletionsCompatDelta(compat: OpenAICompletionsResolvedCompat): OpenAICompletionsCompat {
const delta: OpenAICompletionsCompat = {};
for (const [key, value] of Object.entries(compat)) {
const defaultValue = OPENAI_COMPLETIONS_DEFAULT_COMPAT[key as keyof typeof OPENAI_COMPLETIONS_DEFAULT_COMPAT];
if (isPlainEmptyObject(value) && isPlainEmptyObject(defaultValue)) continue;
if (value !== defaultValue) {
(delta as Record<string, unknown>)[key] = value;
}
}
return delta;
}
function mergeOpenAICompletionsCompat(model: Model<Api>, compat: OpenAICompletionsCompat): void {
model.compat = { ...(model.compat as OpenAICompletionsCompat | undefined), ...compat };
}
function applyOpenAICompletionsCompatMetadata(model: Model<Api>): void {
if (model.api !== "openai-completions") return;
const detected = openAICompletionsCompatDelta(detectOpenAICompletionsCompat(model as Model<"openai-completions">));
model.compat = { ...detected, ...(model.compat as OpenAICompletionsCompat | undefined) };
if (Object.keys(model.compat).length === 0) {
delete model.compat;
}
}
function isGemini3ProModel(modelId: string): boolean {
return /gemini-3(?:\.\d+)?-pro/.test(modelId.toLowerCase());
}
@@ -854,9 +992,10 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
continue;
}
// workers-ai/* through the gateway forwards x-session-affinity to
// the underlying Workers AI runtime for prefix-cache routing.
const compat = upstream === "workers-ai" ? { sendSessionAffinityHeaders: true } : undefined;
// Gateway passthroughs forward session affinity headers to upstreams that
// use them for cache/routing affinity.
const compat =
upstream === "anthropic" || upstream === "workers-ai" ? { sendSessionAffinityHeaders: true } : undefined;
models.push({
id,
@@ -2096,6 +2235,7 @@ async function generateModels() {
for (const model of allModels) {
applyThinkingLevelMetadata(model);
applyOpenAICompletionsCompatMetadata(model);
}
// Group by provider and deduplicate by model ID
packages/ai/src/api-registry.ts
-98
View File
@@ -1,98 +0,0 @@
import type {
Api,
AssistantMessageEventStream,
Context,
Model,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "./types.ts";
export type ApiStreamFunction = (
model: Model<Api>,
context: Context,
options?: StreamOptions,
) => AssistantMessageEventStream;
export type ApiStreamSimpleFunction = (
model: Model<Api>,
context: Context,
options?: SimpleStreamOptions,
) => AssistantMessageEventStream;
export interface ApiProvider<TApi extends Api = Api, TOptions extends StreamOptions = StreamOptions> {
api: TApi;
stream: StreamFunction<TApi, TOptions>;
streamSimple: StreamFunction<TApi, SimpleStreamOptions>;
}
interface ApiProviderInternal {
api: Api;
stream: ApiStreamFunction;
streamSimple: ApiStreamSimpleFunction;
}
type RegisteredApiProvider = {
provider: ApiProviderInternal;
sourceId?: string;
};
const apiProviderRegistry = new Map<string, RegisteredApiProvider>();
function wrapStream<TApi extends Api, TOptions extends StreamOptions>(
api: TApi,
stream: StreamFunction<TApi, TOptions>,
): ApiStreamFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return stream(model as Model<TApi>, context, options as TOptions);
};
}
function wrapStreamSimple<TApi extends Api>(
api: TApi,
streamSimple: StreamFunction<TApi, SimpleStreamOptions>,
): ApiStreamSimpleFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return streamSimple(model as Model<TApi>, context, options);
};
}
export function registerApiProvider<TApi extends Api, TOptions extends StreamOptions>(
provider: ApiProvider<TApi, TOptions>,
sourceId?: string,
): void {
apiProviderRegistry.set(provider.api, {
provider: {
api: provider.api,
stream: wrapStream(provider.api, provider.stream),
streamSimple: wrapStreamSimple(provider.api, provider.streamSimple),
},
sourceId,
});
}
export function getApiProvider(api: Api): ApiProviderInternal | undefined {
return apiProviderRegistry.get(api)?.provider;
}
export function getApiProviders(): ApiProviderInternal[] {
return Array.from(apiProviderRegistry.values(), (entry) => entry.provider);
}
export function unregisterApiProviders(sourceId: string): void {
for (const [api, entry] of apiProviderRegistry.entries()) {
if (entry.sourceId === sourceId) {
apiProviderRegistry.delete(api);
}
}
}
export function clearApiProviders(): void {
apiProviderRegistry.clear();
}
packages/ai/src/api/anthropic-messages.ts
+45 -58
View File
@@ -18,6 +18,7 @@ import type {
Message,
Model,
ProviderEnv,
ProviderHeaders,
SimpleStreamOptions,
StopReason,
StreamFunction,
@@ -34,7 +35,6 @@ import { parseJsonWithRepair, parseStreamingJson } from "../utils/json-parse.ts"
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { adjustMaxTokensForThinking, buildBaseOptions } from "./simple-options.ts";
import { transformMessages } from "./transform-messages.ts";
@@ -170,16 +170,11 @@ const INTERLEAVED_THINKING_BETA = "interleaved-thinking-2025-05-14";
function getAnthropicCompat(
model: Model<"anthropic-messages">,
): Required<Omit<AnthropicMessagesCompat, "forceAdaptiveThinking">> {
// Auto-detect session affinity and cache control support from provider
const isFireworks = model.provider === "fireworks";
const isCloudflareAiGatewayAnthropic =
model.provider === "cloudflare-ai-gateway" && model.baseUrl.includes("anthropic");
return {
supportsEagerToolInputStreaming: model.compat?.supportsEagerToolInputStreaming ?? !isFireworks,
supportsLongCacheRetention: model.compat?.supportsLongCacheRetention ?? !isFireworks,
sendSessionAffinityHeaders:
model.compat?.sendSessionAffinityHeaders ?? !!(isFireworks || isCloudflareAiGatewayAnthropic),
supportsCacheControlOnTools: model.compat?.supportsCacheControlOnTools ?? !isFireworks,
supportsEagerToolInputStreaming: model.compat?.supportsEagerToolInputStreaming ?? true,
supportsLongCacheRetention: model.compat?.supportsLongCacheRetention ?? true,
sendSessionAffinityHeaders: model.compat?.sendSessionAffinityHeaders ?? false,
supportsCacheControlOnTools: model.compat?.supportsCacheControlOnTools ?? true,
supportsTemperature: model.compat?.supportsTemperature ?? true,
allowEmptySignature: model.compat?.allowEmptySignature ?? false,
};
@@ -247,8 +242,8 @@ export interface AnthropicOptions extends StreamOptions {
client?: Anthropic;
}
function mergeHeaders(...headerSources: (Record<string, string | null> | undefined)[]): Record<string, string | null> {
const merged: Record<string, string | null> = {};
function mergeHeaders(...headerSources: (ProviderHeaders | undefined)[]): ProviderHeaders {
const merged: ProviderHeaders = {};
for (const headers of headerSources) {
if (headers) {
Object.assign(merged, headers);
@@ -257,6 +252,27 @@ function mergeHeaders(...headerSources: (Record<string, string | null> | undefin
return merged;
}
function hasHeader(headers: ProviderHeaders | undefined, name: string): boolean {
if (!headers) return false;
const expected = name.toLowerCase();
for (const [key, value] of Object.entries(headers)) {
if (key.toLowerCase() === expected && value !== null && value.trim().length > 0) return true;
}
return false;
}
function assertRequestAuth(provider: string, apiKey: string | undefined, headers: ProviderHeaders | undefined): void {
if (apiKey) return;
if (
hasHeader(headers, "authorization") ||
hasHeader(headers, "x-api-key") ||
hasHeader(headers, "cf-aig-authorization")
) {
return;
}
throw new Error(`No API key for provider: ${provider}`);
}
interface ServerSentEvent {
event: string | null;
data: string;
@@ -484,9 +500,7 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
isOAuth = false;
} else {
const apiKey = options?.apiKey;
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
assertRequestAuth(model.provider, apiKey, options?.headers);
let copilotDynamicHeaders: Record<string, string> | undefined;
if (model.provider === "github-copilot") {
@@ -508,7 +522,6 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
options?.headers,
copilotDynamicHeaders,
cacheSessionId,
options?.env,
);
client = created.client;
isOAuth = created.isOAuthToken;
@@ -748,12 +761,9 @@ export const streamSimple: StreamFunction<"anthropic-messages", SimpleStreamOpti
context: Context,
options?: SimpleStreamOptions,
): AssistantMessageEventStream => {
const apiKey = options?.apiKey;
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
assertRequestAuth(model.provider, options?.apiKey, options?.headers);
const base = buildBaseOptions(model, options, apiKey);
const base = buildBaseOptions(model, options, options?.apiKey);
if (!options?.reasoning) {
return stream(model, context, { ...base, thinkingEnabled: false } satisfies AnthropicOptions);
}
@@ -792,13 +802,12 @@ function isOAuthToken(apiKey: string): boolean {
function createClient(
model: Model<"anthropic-messages">,
apiKey: string,
apiKey: string | undefined,
interleavedThinking: boolean,
useFineGrainedToolStreamingBeta: boolean,
optionsHeaders?: Record<string, string>,
optionsHeaders?: ProviderHeaders,
dynamicHeaders?: Record<string, string>,
sessionId?: string,
env?: ProviderEnv,
): { client: Anthropic; isOAuthToken: boolean } {
// Adaptive thinking models have interleaved thinking built in, so skip the beta header.
const needsInterleavedBeta = interleavedThinking && model.compat?.forceAdaptiveThinking !== true;
@@ -810,34 +819,11 @@ function createClient(
betaFeatures.push(INTERLEAVED_THINKING_BETA);
}
if (model.provider === "cloudflare-ai-gateway") {
const client = new Anthropic({
apiKey: null,
authToken: null,
baseURL: resolveCloudflareBaseUrl(model, env),
dangerouslyAllowBrowser: true,
defaultHeaders: mergeHeaders(
{
accept: "application/json",
"anthropic-dangerous-direct-browser-access": "true",
"cf-aig-authorization": `Bearer ${apiKey}`,
"x-api-key": null,
Authorization: null,
...(betaFeatures.length > 0 ? { "anthropic-beta": betaFeatures.join(",") } : {}),
},
model.headers,
optionsHeaders,
),
});
return { client, isOAuthToken: false };
}
// Copilot: Bearer auth, selective betas.
if (model.provider === "github-copilot") {
const client = new Anthropic({
apiKey: null,
authToken: apiKey,
authToken: apiKey ?? null,
baseURL: model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders: mergeHeaders(
@@ -856,7 +842,7 @@ function createClient(
}
// OAuth: Bearer auth, Claude Code identity headers
if (isOAuthToken(apiKey)) {
if (apiKey && isOAuthToken(apiKey)) {
const client = new Anthropic({
apiKey: null,
authToken: apiKey,
@@ -878,15 +864,10 @@ function createClient(
return { client, isOAuthToken: true };
}
// API key auth
const sessionAffinityHeaders: Record<string, string | null> =
// API key or header-owned auth.
const sessionAffinityHeaders: ProviderHeaders =
sessionId && getAnthropicCompat(model).sendSessionAffinityHeaders ? { "x-session-affinity": sessionId } : {};
const client = new Anthropic({
apiKey,
authToken: null,
baseURL: model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders: mergeHeaders(
const defaultHeaders = mergeHeaders(
{
accept: "application/json",
"anthropic-dangerous-direct-browser-access": "true",
@@ -895,7 +876,13 @@ function createClient(
sessionAffinityHeaders,
model.headers,
optionsHeaders,
),
);
const client = new Anthropic({
apiKey: apiKey ?? null,
authToken: null,
baseURL: model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders,
});
return { client, isOAuthToken: false };
packages/ai/src/api/bedrock-converse-stream.ts
+4 -2
View File
@@ -48,6 +48,7 @@ import type {
ToolResultMessage,
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { providerHeadersToRecord } from "../utils/headers.ts";
import { parseStreamingJson } from "../utils/json-parse.ts";
import { resolveHttpProxyUrlForTarget } from "../utils/node-http-proxy.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
@@ -204,8 +205,9 @@ export const stream: StreamFunction<"bedrock-converse-stream", BedrockOptions> =
try {
const client = new BedrockRuntimeClient(config);
if (options.headers && Object.keys(options.headers).length > 0) {
addCustomHeadersMiddleware(client, options.headers);
const customHeaders = providerHeadersToRecord(options.headers);
if (customHeaders) {
addCustomHeadersMiddleware(client, customHeaders);
}
const cacheRetention = resolveCacheRetention(options.cacheRetention, options.env);
const inferenceMaxTokens = options.maxTokens ?? (isAnthropicClaudeModel(model) ? model.maxTokens : undefined);
packages/ai/src/api/cloudflare.ts
-21
View File
@@ -1,6 +1,3 @@
import type { Api, Model, ProviderEnv } from "../types.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
/** Workers AI direct endpoint. */
export const CLOUDFLARE_WORKERS_AI_BASE_URL =
"https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1";
@@ -16,21 +13,3 @@ export const CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL =
/** AI Gateway → Anthropic passthrough. */
export const CLOUDFLARE_AI_GATEWAY_ANTHROPIC_BASE_URL =
"https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic";
export function isCloudflareProvider(provider: string): boolean {
return provider === "cloudflare-workers-ai" || provider === "cloudflare-ai-gateway";
}
/** Substitute `{VAR}` placeholders in a Cloudflare baseUrl from provider env or process.env. */
export function resolveCloudflareBaseUrl(model: Model<Api>, env?: ProviderEnv): string {
const url = model.baseUrl;
if (!url.includes("{")) return url;
const baseUrl = url.replace(/\{([A-Z_][A-Z0-9_]*)\}/g, (_match, name: string) => {
const value = getProviderEnvValue(name, env);
if (!value) {
throw new Error(`${name} is required for provider ${model.provider} but is not set.`);
}
return value;
});
return baseUrl;
}
packages/ai/src/api/google-generative-ai.ts
+6 -3
View File
@@ -10,6 +10,7 @@ import type {
AssistantMessage,
Context,
Model,
ProviderHeaders,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
@@ -20,6 +21,7 @@ import type {
ToolCall,
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { providerHeadersToRecord } from "../utils/headers.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import type { GoogleThinkingLevel } from "./google-shared.ts";
import {
@@ -318,15 +320,16 @@ export const streamSimple: StreamFunction<"google-generative-ai", SimpleStreamOp
function createClient(
model: Model<"google-generative-ai">,
apiKey?: string,
optionsHeaders?: Record<string, string>,
optionsHeaders?: ProviderHeaders,
): GoogleGenAI {
const httpOptions: { baseUrl?: string; apiVersion?: string; headers?: Record<string, string> } = {};
if (model.baseUrl) {
httpOptions.baseUrl = model.baseUrl;
httpOptions.apiVersion = ""; // baseUrl already includes version path, don't append
}
if (model.headers || optionsHeaders) {
httpOptions.headers = { ...model.headers, ...optionsHeaders };
const headers = providerHeadersToRecord({ ...model.headers, ...optionsHeaders });
if (headers) {
httpOptions.headers = headers;
}
return new GoogleGenAI({
packages/ai/src/api/google-vertex.ts
+8 -8
View File
@@ -15,6 +15,7 @@ import type {
Model,
ThinkingLevel as PiThinkingLevel,
ProviderEnv,
ProviderHeaders,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
@@ -24,6 +25,7 @@ import type {
ToolCall,
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { providerHeadersToRecord } from "../utils/headers.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import type { GoogleThinkingLevel } from "./google-shared.ts";
@@ -334,7 +336,7 @@ function createClient(
model: Model<"google-vertex">,
project: string,
location: string,
optionsHeaders?: Record<string, string>,
optionsHeaders?: ProviderHeaders,
env?: ProviderEnv,
): GoogleGenAI {
const googleAuthOptions = buildGoogleAuthOptions(env);
@@ -351,7 +353,7 @@ function createClient(
function createClientWithApiKey(
model: Model<"google-vertex">,
apiKey: string,
optionsHeaders?: Record<string, string>,
optionsHeaders?: ProviderHeaders,
): GoogleGenAI {
return new GoogleGenAI({
vertexai: true,
@@ -361,10 +363,7 @@ function createClientWithApiKey(
});
}
function buildHttpOptions(
model: Model<"google-vertex">,
optionsHeaders?: Record<string, string>,
): HttpOptions | undefined {
function buildHttpOptions(model: Model<"google-vertex">, optionsHeaders?: ProviderHeaders): HttpOptions | undefined {
const httpOptions: HttpOptions = {};
const baseUrl = resolveCustomBaseUrl(model.baseUrl);
if (baseUrl) {
@@ -375,8 +374,9 @@ function buildHttpOptions(
}
}
if (model.headers || optionsHeaders) {
httpOptions.headers = { ...model.headers, ...optionsHeaders };
const headers = providerHeadersToRecord({ ...model.headers, ...optionsHeaders });
if (headers) {
httpOptions.headers = headers;
}
return Object.keys(httpOptions).length > 0 ? httpOptions : undefined;
packages/ai/src/api/openai-codex-responses.ts
+8 -3
View File
@@ -28,6 +28,7 @@ import type {
Context,
Model,
ProviderEnv,
ProviderHeaders,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
@@ -1467,14 +1468,18 @@ function createCodexRequestId(): string {
function buildBaseCodexHeaders(
initHeaders: Record<string, string> | undefined,
additionalHeaders: Record<string, string> | undefined,
additionalHeaders: ProviderHeaders | undefined,
accountId: string,
token: string,
): Headers {
const headers = new Headers(initHeaders);
for (const [key, value] of Object.entries(additionalHeaders || {})) {
if (value === null) {
headers.delete(key);
} else {
headers.set(key, value);
}
}
headers.set("Authorization", `Bearer ${token}`);
headers.set("chatgpt-account-id", accountId);
headers.set("originator", "pi");
@@ -1485,7 +1490,7 @@ function buildBaseCodexHeaders(
function buildSSEHeaders(
initHeaders: Record<string, string> | undefined,
additionalHeaders: Record<string, string> | undefined,
additionalHeaders: ProviderHeaders | undefined,
accountId: string,
token: string,
sessionId?: string,
@@ -1505,7 +1510,7 @@ function buildSSEHeaders(
function buildWebSocketHeaders(
initHeaders: Record<string, string> | undefined,
additionalHeaders: Record<string, string> | undefined,
additionalHeaders: ProviderHeaders | undefined,
accountId: string,
token: string,
requestId: string,
packages/ai/src/api/openai-completions.ts
+56 -125
View File
@@ -21,6 +21,7 @@ import type {
Model,
OpenAICompletionsCompat,
ProviderEnv,
ProviderHeaders,
SimpleStreamOptions,
StopReason,
StreamFunction,
@@ -36,7 +37,6 @@ import { headersToRecord } from "../utils/headers.ts";
import { parseStreamingJson } from "../utils/json-parse.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { buildBaseOptions } from "./simple-options.ts";
@@ -47,6 +47,21 @@ import { transformMessages } from "./transform-messages.ts";
* This is needed because Anthropic (via proxy) requires the tools param
* to be present when messages include tool_calls or tool role messages.
*/
function hasHeader(headers: ProviderHeaders | undefined, name: string): boolean {
if (!headers) return false;
const expected = name.toLowerCase();
for (const [key, value] of Object.entries(headers)) {
if (key.toLowerCase() === expected && value !== null && value.trim().length > 0) return true;
}
return false;
}
function getClientApiKey(provider: string, apiKey: string | undefined, headers: ProviderHeaders | undefined): string {
if (apiKey) return apiKey;
if (hasHeader(headers, "authorization") || hasHeader(headers, "cf-aig-authorization")) return "unused";
throw new Error(`No API key for provider: ${provider}`);
}
function hasToolHistory(messages: Message[]): boolean {
for (const msg of messages) {
if (msg.role === "toolResult") {
@@ -160,14 +175,11 @@ export const stream: StreamFunction<"openai-completions", OpenAICompletionsOptio
};
try {
const apiKey = options?.apiKey;
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const apiKey = getClientApiKey(model.provider, options?.apiKey, options?.headers);
const compat = getCompat(model);
const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, compat, options?.env);
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, compat);
let params = buildParams(model, context, options, compat, cacheRetention);
const nextParams = await options?.onPayload?.(params, model);
if (nextParams !== undefined) {
@@ -468,12 +480,9 @@ export const streamSimple: StreamFunction<"openai-completions", SimpleStreamOpti
context: Context,
options?: SimpleStreamOptions,
): AssistantMessageEventStream => {
const apiKey = options?.apiKey;
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
getClientApiKey(model.provider, options?.apiKey, options?.headers);
const base = buildBaseOptions(model, options, apiKey);
const base = buildBaseOptions(model, options, options?.apiKey);
const clampedReasoning = options?.reasoning ? clampThinkingLevel(model, options.reasoning) : undefined;
const reasoningEffort = clampedReasoning === "off" ? undefined : clampedReasoning;
const toolChoice = (options as OpenAICompletionsOptions | undefined)?.toolChoice;
@@ -489,12 +498,11 @@ function createClient(
model: Model<"openai-completions">,
context: Context,
apiKey: string,
optionsHeaders?: Record<string, string>,
optionsHeaders?: ProviderHeaders,
sessionId?: string,
compat: ResolvedOpenAICompletionsCompat = getCompat(model),
env?: ProviderEnv,
) {
const headers = { ...model.headers };
const headers: ProviderHeaders = { ...model.headers };
if (model.provider === "github-copilot") {
const hasImages = hasCopilotVisionInput(context.messages);
const copilotHeaders = buildCopilotDynamicHeaders({
@@ -515,20 +523,11 @@ function createClient(
Object.assign(headers, optionsHeaders);
}
const defaultHeaders =
model.provider === "cloudflare-ai-gateway"
? {
...headers,
Authorization: headers.Authorization ?? null,
"cf-aig-authorization": `Bearer ${apiKey}`,
}
: headers;
return new OpenAI({
apiKey,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model, env) : model.baseUrl,
baseURL: model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders,
defaultHeaders: headers,
});
}
@@ -674,7 +673,7 @@ function buildParams(
}
// Vercel AI Gateway provider routing preferences
if (model.baseUrl.includes("ai-gateway.vercel.sh") && model.compat?.vercelGatewayRouting) {
if (model.compat?.vercelGatewayRouting) {
const routing = model.compat.vercelGatewayRouting;
if (routing.only || routing.order) {
const gatewayOptions: Record<string, string[]> = {};
@@ -1164,123 +1163,55 @@ function mapStopReason(reason: ChatCompletionChunk.Choice["finish_reason"] | str
}
}
/**
* Detect compatibility settings from provider and baseUrl for known providers.
* Provider takes precedence over URL-based detection since it's explicitly configured.
* Returns a fully resolved OpenAICompletionsCompat object with all fields set.
*/
function detectCompat(model: Model<"openai-completions">): ResolvedOpenAICompletionsCompat {
const provider = model.provider;
const baseUrl = model.baseUrl;
const isZai =
provider === "zai" ||
provider === "zai-coding-cn" ||
baseUrl.includes("api.z.ai") ||
baseUrl.includes("open.bigmodel.cn");
const isTogether =
provider === "together" || baseUrl.includes("api.together.ai") || baseUrl.includes("api.together.xyz");
const isMoonshot = provider === "moonshotai" || provider === "moonshotai-cn" || baseUrl.includes("api.moonshot.");
const isOpenRouter = provider === "openrouter" || baseUrl.includes("openrouter.ai");
const isCloudflareWorkersAI = provider === "cloudflare-workers-ai" || baseUrl.includes("api.cloudflare.com");
const isCloudflareAiGateway = provider === "cloudflare-ai-gateway" || baseUrl.includes("gateway.ai.cloudflare.com");
const isNvidia = provider === "nvidia" || baseUrl.includes("integrate.api.nvidia.com");
const isAntLing = provider === "ant-ling" || baseUrl.includes("api.ant-ling.com");
const isNonStandard =
isNvidia ||
provider === "cerebras" ||
baseUrl.includes("cerebras.ai") ||
provider === "xai" ||
baseUrl.includes("api.x.ai") ||
isTogether ||
baseUrl.includes("chutes.ai") ||
baseUrl.includes("deepseek.com") ||
isZai ||
isMoonshot ||
provider === "opencode" ||
baseUrl.includes("opencode.ai") ||
isCloudflareWorkersAI ||
isCloudflareAiGateway ||
isAntLing;
const useMaxTokens =
baseUrl.includes("chutes.ai") || isMoonshot || isCloudflareAiGateway || isTogether || isNvidia || isAntLing;
const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
const isDeepSeek = provider === "deepseek" || baseUrl.includes("deepseek.com");
const isOpenRouterDeveloperRoleModel =
isOpenRouter && (model.id.startsWith("anthropic/") || model.id.startsWith("openai/"));
const cacheControlFormat = provider === "openrouter" && model.id.startsWith("anthropic/") ? "anthropic" : undefined;
return {
supportsStore: !isNonStandard,
supportsDeveloperRole: isOpenRouterDeveloperRoleModel || (!isNonStandard && !isOpenRouter),
supportsReasoningEffort:
!isGrok && !isZai && !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia && !isAntLing,
const DEFAULT_COMPAT: ResolvedOpenAICompletionsCompat = {
supportsStore: true,
supportsDeveloperRole: true,
supportsReasoningEffort: true,
supportsUsageInStreaming: true,
maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens",
maxTokensField: "max_completion_tokens",
requiresToolResultName: false,
requiresAssistantAfterToolResult: false,
requiresThinkingAsText: false,
requiresReasoningContentOnAssistantMessages: isDeepSeek,
thinkingFormat: isDeepSeek
? "deepseek"
: isZai
? "zai"
: isTogether
? "together"
: isAntLing
? "ant-ling"
: isOpenRouter
? "openrouter"
: "openai",
requiresReasoningContentOnAssistantMessages: false,
thinkingFormat: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia,
cacheControlFormat,
supportsStrictMode: true,
cacheControlFormat: undefined,
sendSessionAffinityHeaders: false,
supportsLongCacheRetention: !(
isTogether ||
isCloudflareWorkersAI ||
isCloudflareAiGateway ||
isNvidia ||
isAntLing
),
};
}
supportsLongCacheRetention: true,
};
/**
* Get resolved compatibility settings for a model.
* Uses explicit model.compat if provided, otherwise auto-detects from provider/URL.
* Uses explicit generated/custom model.compat over OpenAI-standard defaults.
*/
function getCompat(model: Model<"openai-completions">): ResolvedOpenAICompletionsCompat {
const detected = detectCompat(model);
if (!model.compat) return detected;
if (!model.compat) return DEFAULT_COMPAT;
return {
supportsStore: model.compat.supportsStore ?? detected.supportsStore,
supportsDeveloperRole: model.compat.supportsDeveloperRole ?? detected.supportsDeveloperRole,
supportsReasoningEffort: model.compat.supportsReasoningEffort ?? detected.supportsReasoningEffort,
supportsUsageInStreaming: model.compat.supportsUsageInStreaming ?? detected.supportsUsageInStreaming,
maxTokensField: model.compat.maxTokensField ?? detected.maxTokensField,
requiresToolResultName: model.compat.requiresToolResultName ?? detected.requiresToolResultName,
supportsStore: model.compat.supportsStore ?? DEFAULT_COMPAT.supportsStore,
supportsDeveloperRole: model.compat.supportsDeveloperRole ?? DEFAULT_COMPAT.supportsDeveloperRole,
supportsReasoningEffort: model.compat.supportsReasoningEffort ?? DEFAULT_COMPAT.supportsReasoningEffort,
supportsUsageInStreaming: model.compat.supportsUsageInStreaming ?? DEFAULT_COMPAT.supportsUsageInStreaming,
maxTokensField: model.compat.maxTokensField ?? DEFAULT_COMPAT.maxTokensField,
requiresToolResultName: model.compat.requiresToolResultName ?? DEFAULT_COMPAT.requiresToolResultName,
requiresAssistantAfterToolResult:
model.compat.requiresAssistantAfterToolResult ?? detected.requiresAssistantAfterToolResult,
requiresThinkingAsText: model.compat.requiresThinkingAsText ?? detected.requiresThinkingAsText,
model.compat.requiresAssistantAfterToolResult ?? DEFAULT_COMPAT.requiresAssistantAfterToolResult,
requiresThinkingAsText: model.compat.requiresThinkingAsText ?? DEFAULT_COMPAT.requiresThinkingAsText,
requiresReasoningContentOnAssistantMessages:
model.compat.requiresReasoningContentOnAssistantMessages ??
detected.requiresReasoningContentOnAssistantMessages,
thinkingFormat: model.compat.thinkingFormat ?? detected.thinkingFormat,
openRouterRouting: model.compat.openRouterRouting ?? {},
vercelGatewayRouting: model.compat.vercelGatewayRouting ?? detected.vercelGatewayRouting,
chatTemplateKwargs: model.compat.chatTemplateKwargs ?? detected.chatTemplateKwargs,
zaiToolStream: model.compat.zaiToolStream ?? detected.zaiToolStream,
supportsStrictMode: model.compat.supportsStrictMode ?? detected.supportsStrictMode,
cacheControlFormat: model.compat.cacheControlFormat ?? detected.cacheControlFormat,
sendSessionAffinityHeaders: model.compat.sendSessionAffinityHeaders ?? detected.sendSessionAffinityHeaders,
supportsLongCacheRetention: model.compat.supportsLongCacheRetention ?? detected.supportsLongCacheRetention,
DEFAULT_COMPAT.requiresReasoningContentOnAssistantMessages,
thinkingFormat: model.compat.thinkingFormat ?? DEFAULT_COMPAT.thinkingFormat,
openRouterRouting: model.compat.openRouterRouting ?? DEFAULT_COMPAT.openRouterRouting,
vercelGatewayRouting: model.compat.vercelGatewayRouting ?? DEFAULT_COMPAT.vercelGatewayRouting,
chatTemplateKwargs: model.compat.chatTemplateKwargs ?? DEFAULT_COMPAT.chatTemplateKwargs,
zaiToolStream: model.compat.zaiToolStream ?? DEFAULT_COMPAT.zaiToolStream,
supportsStrictMode: model.compat.supportsStrictMode ?? DEFAULT_COMPAT.supportsStrictMode,
cacheControlFormat: model.compat.cacheControlFormat ?? DEFAULT_COMPAT.cacheControlFormat,
sendSessionAffinityHeaders: model.compat.sendSessionAffinityHeaders ?? DEFAULT_COMPAT.sendSessionAffinityHeaders,
supportsLongCacheRetention: model.compat.supportsLongCacheRetention ?? DEFAULT_COMPAT.supportsLongCacheRetention,
};
}
packages/ai/src/api/openai-responses.ts
+24 -25
View File
@@ -9,6 +9,7 @@ import type {
Model,
OpenAIResponsesCompat,
ProviderEnv,
ProviderHeaders,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
@@ -17,7 +18,6 @@ import type {
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.ts";
@@ -25,6 +25,21 @@ import { buildBaseOptions } from "./simple-options.ts";
const OPENAI_TOOL_CALL_PROVIDERS = new Set(["openai", "openai-codex", "opencode"]);
function hasHeader(headers: ProviderHeaders | undefined, name: string): boolean {
if (!headers) return false;
const expected = name.toLowerCase();
for (const [key, value] of Object.entries(headers)) {
if (key.toLowerCase() === expected && value !== null && value.trim().length > 0) return true;
}
return false;
}
function getClientApiKey(provider: string, apiKey: string | undefined, headers: ProviderHeaders | undefined): string {
if (apiKey) return apiKey;
if (hasHeader(headers, "authorization") || hasHeader(headers, "cf-aig-authorization")) return "unused";
throw new Error(`No API key for provider: ${provider}`);
}
/**
* Resolve cache retention preference.
* Defaults to "short" and uses PI_CACHE_RETENTION for backward compatibility.
@@ -109,13 +124,10 @@ export const stream: StreamFunction<"openai-responses", OpenAIResponsesOptions>
try {
// Create OpenAI client
const apiKey = options?.apiKey;
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const apiKey = getClientApiKey(model.provider, options?.apiKey, options?.headers);
const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, options?.env);
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId);
let params = buildParams(model, context, options);
const nextParams = await options?.onPayload?.(params, model);
if (nextParams !== undefined) {
@@ -166,12 +178,9 @@ export const streamSimple: StreamFunction<"openai-responses", SimpleStreamOption
context: Context,
options?: SimpleStreamOptions,
): AssistantMessageEventStream => {
const apiKey = options?.apiKey;
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
getClientApiKey(model.provider, options?.apiKey, options?.headers);
const base = buildBaseOptions(model, options, apiKey);
const base = buildBaseOptions(model, options, options?.apiKey);
const clampedReasoning = options?.reasoning ? clampThinkingLevel(model, options.reasoning) : undefined;
const reasoningEffort = clampedReasoning === "off" ? undefined : clampedReasoning;
@@ -185,12 +194,11 @@ function createClient(
model: Model<"openai-responses">,
context: Context,
apiKey: string,
optionsHeaders?: Record<string, string>,
optionsHeaders?: ProviderHeaders,
sessionId?: string,
env?: ProviderEnv,
) {
const compat = getCompat(model);
const headers = { ...model.headers };
const headers: ProviderHeaders = { ...model.headers };
if (model.provider === "github-copilot") {
const hasImages = hasCopilotVisionInput(context.messages);
const copilotHeaders = buildCopilotDynamicHeaders({
@@ -212,20 +220,11 @@ function createClient(
Object.assign(headers, optionsHeaders);
}
const defaultHeaders =
model.provider === "cloudflare-ai-gateway"
? {
...headers,
Authorization: headers.Authorization ?? null,
"cf-aig-authorization": `Bearer ${apiKey}`,
}
: headers;
return new OpenAI({
apiKey,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model, env) : model.baseUrl,
baseURL: model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders,
defaultHeaders: headers,
});
}
packages/ai/src/api/openrouter-images.ts
+4 -6
View File
@@ -13,9 +13,10 @@ import type {
ImagesFunction,
ImagesModel,
ImagesOptions,
ProviderHeaders,
TextContent,
} from "../types.ts";
import { headersToRecord } from "../utils/headers.ts";
import { headersToRecord, providerHeadersToRecord } from "../utils/headers.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
interface OpenRouterGeneratedImage {
@@ -106,16 +107,13 @@ export const generateImages: ImagesFunction<"openrouter-images", ImagesOptions>
function createClient(
model: ImagesModel<"openrouter-images">,
apiKey: string,
optionsHeaders?: Record<string, string>,
optionsHeaders?: ProviderHeaders,
): OpenAI {
return new OpenAI({
apiKey,
baseURL: model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders: {
...model.headers,
...optionsHeaders,
},
defaultHeaders: providerHeadersToRecord({ ...model.headers, ...optionsHeaders }),
});
}
packages/ai/src/auth/types.ts
+2 -2
View File
@@ -1,4 +1,4 @@
import type { Api, ImagesApi, ImagesModel, Model, ProviderEnv } from "../types.ts";
import type { Api, ImagesApi, ImagesModel, Model, ProviderEnv, ProviderHeaders } from "../types.ts";
import type { OAuthCredentials } from "../utils/oauth/types.ts";
/**
@@ -7,7 +7,7 @@ import type { OAuthCredentials } from "../utils/oauth/types.ts";
*/
export interface ModelAuth {
apiKey?: string;
headers?: Record<string, string>;
headers?: ProviderHeaders;
baseUrl?: string;
}
packages/ai/src/compat.ts
+130 -4
View File
@@ -19,7 +19,6 @@ export * from "./api/mistral-conversations.lazy.ts";
export * from "./api/openai-codex-responses.lazy.ts";
export * from "./api/openai-completions.lazy.ts";
export * from "./api/openai-responses.lazy.ts";
export * from "./api-registry.ts";
export * from "./env-api-keys.ts";
export * from "./image-models.ts";
export * from "./images.ts";
@@ -36,11 +35,12 @@ import { mistralConversationsApi } from "./api/mistral-conversations.lazy.ts";
import { openAICodexResponsesApi } from "./api/openai-codex-responses.lazy.ts";
import { openAICompletionsApi } from "./api/openai-completions.lazy.ts";
import { openAIResponsesApi } from "./api/openai-responses.lazy.ts";
import { clearApiProviders, getApiProvider, registerApiProvider } from "./api-registry.ts";
import { getEnvApiKey } from "./env-api-keys.ts";
import { getBuiltinModel, getBuiltinModels, getBuiltinProviders } from "./providers/all.ts";
import { builtinModels, getBuiltinModel, getBuiltinModels, getBuiltinProviders } from "./providers/all.ts";
import { createFauxCore, type FauxProviderRegistration, type RegisterFauxProviderOptions } from "./providers/faux.ts";
import type {
Api,
ApiStreamOptions,
AssistantMessage,
AssistantMessageEventStream,
Context,
@@ -48,6 +48,7 @@ import type {
ProviderStreamOptions,
ProviderStreams,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "./types.ts";
@@ -60,6 +61,113 @@ export const getModels = getBuiltinModels;
/** @deprecated Static catalog read. Use `getBuiltinProviders` from "@earendil-works/pi-ai/providers/all" or `Models.getProviders()`. */
export const getProviders = getBuiltinProviders;
export type ApiStreamFunction = (
model: Model<Api>,
context: Context,
options?: StreamOptions,
) => AssistantMessageEventStream;
export type ApiStreamSimpleFunction = (
model: Model<Api>,
context: Context,
options?: SimpleStreamOptions,
) => AssistantMessageEventStream;
export interface ApiProvider<TApi extends Api = Api, TOptions extends StreamOptions = StreamOptions> {
api: TApi;
stream: StreamFunction<TApi, TOptions>;
streamSimple: StreamFunction<TApi, SimpleStreamOptions>;
}
interface ApiProviderInternal {
api: Api;
stream: ApiStreamFunction;
streamSimple: ApiStreamSimpleFunction;
}
type RegisteredApiProvider = {
provider: ApiProviderInternal;
sourceId?: string;
};
const apiProviderRegistry = new Map<string, RegisteredApiProvider>();
function wrapStream<TApi extends Api, TOptions extends StreamOptions>(
api: TApi,
stream: StreamFunction<TApi, TOptions>,
): ApiStreamFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return stream(model as Model<TApi>, context, options as TOptions);
};
}
function wrapStreamSimple<TApi extends Api>(
api: TApi,
streamSimple: StreamFunction<TApi, SimpleStreamOptions>,
): ApiStreamSimpleFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return streamSimple(model as Model<TApi>, context, options);
};
}
export function registerApiProvider<TApi extends Api, TOptions extends StreamOptions>(
provider: ApiProvider<TApi, TOptions>,
sourceId?: string,
): void {
apiProviderRegistry.set(provider.api, {
provider: {
api: provider.api,
stream: wrapStream(provider.api, provider.stream),
streamSimple: wrapStreamSimple(provider.api, provider.streamSimple),
},
sourceId,
});
}
export function getApiProvider(api: Api): ApiProviderInternal | undefined {
return apiProviderRegistry.get(api)?.provider;
}
export function getApiProviders(): ApiProviderInternal[] {
return Array.from(apiProviderRegistry.values(), (entry) => entry.provider);
}
export function unregisterApiProviders(sourceId: string): void {
for (const [api, entry] of apiProviderRegistry.entries()) {
if (entry.sourceId === sourceId) {
apiProviderRegistry.delete(api);
}
}
}
function clearApiProviders(): void {
apiProviderRegistry.clear();
}
export function registerFauxProvider(options: RegisterFauxProviderOptions = {}): FauxProviderRegistration {
const core = createFauxCore(options);
const sourceId = `faux-provider-${Math.random().toString(36).slice(2, 10)}`;
registerApiProvider({ api: core.api, stream: core.stream, streamSimple: core.streamSimple }, sourceId);
return {
api: core.api,
models: core.models,
getModel: core.getModel,
state: core.state,
setResponses: core.setResponses,
appendResponses: core.appendResponses,
getPendingResponseCount: core.getPendingResponseCount,
unregister() {
unregisterApiProviders(sourceId);
},
};
}
const BUILTIN_APIS: [Api, ProviderStreams][] = [
["anthropic-messages", anthropicMessagesApi()],
["openai-completions", openAICompletionsApi()],
@@ -72,6 +180,8 @@ const BUILTIN_APIS: [Api, ProviderStreams][] = [
["bedrock-converse-stream", bedrockConverseStreamApi()],
];
const builtinApiProviderInstances = new Map<Api, ReturnType<typeof getApiProvider>>();
/**
* Registers the builtin API implementations into the api-registry without
* clobbering existing entries: compat may load after a test or extension has
@@ -79,18 +189,23 @@ const BUILTIN_APIS: [Api, ProviderStreams][] = [
*/
export function registerBuiltInApiProviders(): void {
for (const [api, streams] of BUILTIN_APIS) {
if (getApiProvider(api)) continue;
if (!getApiProvider(api)) {
registerApiProvider({ api, stream: streams.stream, streamSimple: streams.streamSimple });
}
builtinApiProviderInstances.set(api, getApiProvider(api));
}
}
export function resetApiProviders(): void {
clearApiProviders();
builtinApiProviderInstances.clear();
registerBuiltInApiProviders();
}
registerBuiltInApiProviders();
const compatModels = builtinModels();
function hasExplicitApiKey(apiKey: string | undefined): apiKey is string {
return typeof apiKey === "string" && apiKey.trim().length > 0;
}
@@ -105,6 +220,11 @@ function withEnvApiKey<TOptions extends StreamOptions>(
return { ...options, apiKey } as TOptions;
}
function shouldUseBuiltinModels(model: Model<Api>): boolean {
const builtin = compatModels.getModel(model.provider, model.id);
return builtin?.api === model.api && getApiProvider(model.api) === builtinApiProviderInstances.get(model.api);
}
function resolveApiProvider(api: Api) {
const provider = getApiProvider(api);
if (!provider) {
@@ -118,6 +238,9 @@ export function stream<TApi extends Api>(
context: Context,
options?: ProviderStreamOptions,
): AssistantMessageEventStream {
if (shouldUseBuiltinModels(model)) {
return compatModels.stream(model, context, options as ApiStreamOptions<TApi> | undefined);
}
const provider = resolveApiProvider(model.api);
return provider.stream(model, context, withEnvApiKey(model, options) as StreamOptions);
}
@@ -136,6 +259,9 @@ export function streamSimple<TApi extends Api>(
context: Context,
options?: SimpleStreamOptions,
): AssistantMessageEventStream {
if (shouldUseBuiltinModels(model)) {
return compatModels.streamSimple(model, context, options);
}
const provider = resolveApiProvider(model.api);
return provider.streamSimple(model, context, withEnvApiKey(model, options));
}
packages/ai/src/models.ts
+3 -2
View File
@@ -11,6 +11,7 @@ import type {
Context,
Model,
ModelThinkingLevel,
ProviderHeaders,
ProviderStreams,
SimpleStreamOptions,
StreamOptions,
@@ -33,7 +34,7 @@ export interface Provider<TApi extends Api = Api> {
readonly name: string;
readonly baseUrl?: string;
readonly headers?: Record<string, string>;
readonly headers?: ProviderHeaders;
/**
* Required: at least one of `apiKey`/`oauth`. Every provider has auth
@@ -287,7 +288,7 @@ export interface CreateProviderOptions<TApi extends Api = Api> {
/** Display name. Default: `id`. */
name?: string;
baseUrl?: string;
headers?: Record<string, string>;
headers?: ProviderHeaders;
/** Required — every provider has auth semantics, even ambient/keyless ones. */
auth: ProviderAuth;
/** Initial model list (empty for purely dynamic providers). */
packages/ai/src/providers/ant-ling.models.ts
+3 -3
View File
@@ -10,7 +10,7 @@ export const ANT_LING_MODELS = {
api: "openai-completions",
provider: "ant-ling",
baseUrl: "https://api.ant-ling.com/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"ant-ling","supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
@@ -28,7 +28,7 @@ export const ANT_LING_MODELS = {
api: "openai-completions",
provider: "ant-ling",
baseUrl: "https://api.ant-ling.com/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"ant-ling","supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
@@ -46,7 +46,7 @@ export const ANT_LING_MODELS = {
api: "openai-completions",
provider: "ant-ling",
baseUrl: "https://api.ant-ling.com/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"thinkingFormat":"ant-ling"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"ant-ling","supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh"},
input: ["text"],
packages/ai/src/providers/cerebras.models.ts
+2
View File
@@ -10,6 +10,7 @@ export const CEREBRAS_MODELS = {
api: "openai-completions",
provider: "cerebras",
baseUrl: "https://api.cerebras.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
@@ -27,6 +28,7 @@ export const CEREBRAS_MODELS = {
api: "openai-completions",
provider: "cerebras",
baseUrl: "https://api.cerebras.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
packages/ai/src/providers/cloudflare-ai-gateway.models.ts
+21 -9
View File
@@ -10,6 +10,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text", "image"],
cost: {
@@ -27,6 +28,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text", "image"],
cost: {
@@ -44,6 +46,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text", "image"],
cost: {
@@ -61,6 +64,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text", "image"],
cost: {
@@ -78,6 +82,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text", "image"],
cost: {
@@ -95,6 +100,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text", "image"],
cost: {
@@ -112,7 +118,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true},
compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true},
reasoning: true,
thinkingLevelMap: {"off":null,"xhigh":"xhigh"},
input: ["text", "image"],
@@ -131,6 +137,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -148,6 +155,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -165,6 +173,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -182,6 +191,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -199,7 +209,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true},
compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true},
reasoning: true,
thinkingLevelMap: {"xhigh":"max"},
input: ["text", "image"],
@@ -218,7 +228,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true,"supportsTemperature":false},
compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true,"supportsTemperature":false},
reasoning: true,
thinkingLevelMap: {"xhigh":"xhigh"},
input: ["text", "image"],
@@ -237,7 +247,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true,"supportsTemperature":false},
compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true,"supportsTemperature":false},
reasoning: true,
thinkingLevelMap: {"xhigh":"xhigh"},
input: ["text", "image"],
@@ -256,6 +266,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -273,6 +284,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -290,7 +302,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true},
compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -587,7 +599,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -605,7 +617,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -623,7 +635,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
@@ -641,7 +653,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
packages/ai/src/providers/cloudflare-ai-gateway.ts
+2 -2
View File
@@ -1,9 +1,9 @@
import { anthropicMessagesApi } from "../api/anthropic-messages.lazy.ts";
import { openAICompletionsApi } from "../api/openai-completions.lazy.ts";
import { openAIResponsesApi } from "../api/openai-responses.lazy.ts";
import { envApiKeyAuth } from "../auth/helpers.ts";
import { createProvider, type Provider } from "../models.ts";
import { CLOUDFLARE_AI_GATEWAY_MODELS } from "./cloudflare-ai-gateway.models.ts";
import { cloudflareAIGatewayAuth } from "./cloudflare-auth.ts";
export function cloudflareAIGatewayProvider(): Provider<
"anthropic-messages" | "openai-completions" | "openai-responses"
@@ -11,7 +11,7 @@ export function cloudflareAIGatewayProvider(): Provider<
return createProvider({
id: "cloudflare-ai-gateway",
name: "Cloudflare AI Gateway",
auth: { apiKey: envApiKeyAuth("Cloudflare API key", ["CLOUDFLARE_API_KEY"]) },
auth: { apiKey: cloudflareAIGatewayAuth() },
models: Object.values(CLOUDFLARE_AI_GATEWAY_MODELS),
api: {
"anthropic-messages": anthropicMessagesApi(),
packages/ai/src/providers/cloudflare-auth.ts
+108
View File
@@ -0,0 +1,108 @@
import type { ApiKeyAuth, ApiKeyCredential, AuthContext } from "../auth/types.ts";
import type { Api, ImagesApi, ImagesModel, Model, ProviderEnv } from "../types.ts";
const CLOUDFLARE_API_KEY = "CLOUDFLARE_API_KEY";
const CLOUDFLARE_ACCOUNT_ID = "CLOUDFLARE_ACCOUNT_ID";
const CLOUDFLARE_GATEWAY_ID = "CLOUDFLARE_GATEWAY_ID";
type CloudflareAuthKind = "workers-ai" | "ai-gateway";
async function resolveValue(input: {
name: string;
ctx: AuthContext;
credential: ApiKeyCredential | undefined;
}): Promise<string | undefined> {
if (input.credential) {
if (input.name === CLOUDFLARE_API_KEY) return input.credential.key;
return input.credential.metadata?.[input.name];
}
return input.ctx.env(input.name);
}
function resolveCloudflareBaseUrl(
model: Model<Api> | ImagesModel<ImagesApi>,
accountId: string,
gatewayId: string | undefined,
): string {
return model.baseUrl
.replaceAll(`{${CLOUDFLARE_ACCOUNT_ID}}`, accountId)
.replaceAll(`{${CLOUDFLARE_GATEWAY_ID}}`, gatewayId ?? "");
}
async function resolveCloudflareEnv(input: {
kind: CloudflareAuthKind;
model: Model<Api> | ImagesModel<ImagesApi>;
ctx: AuthContext;
credential: ApiKeyCredential | undefined;
}): Promise<{ apiKey: string; env: ProviderEnv; baseUrl: string; source: string } | undefined> {
const apiKey = await resolveValue({ name: CLOUDFLARE_API_KEY, ctx: input.ctx, credential: input.credential });
const accountId = await resolveValue({ name: CLOUDFLARE_ACCOUNT_ID, ctx: input.ctx, credential: input.credential });
const gatewayId =
input.kind === "ai-gateway"
? await resolveValue({ name: CLOUDFLARE_GATEWAY_ID, ctx: input.ctx, credential: input.credential })
: undefined;
if (!apiKey || !accountId || (input.kind === "ai-gateway" && !gatewayId)) return undefined;
return {
apiKey,
env: {
CLOUDFLARE_ACCOUNT_ID: accountId,
...(gatewayId ? { CLOUDFLARE_GATEWAY_ID: gatewayId } : {}),
},
baseUrl: resolveCloudflareBaseUrl(input.model, accountId, gatewayId),
source: input.credential ? "stored credential" : CLOUDFLARE_API_KEY,
};
}
export function cloudflareWorkersAIAuth(): ApiKeyAuth {
return {
name: "Cloudflare API key",
login: async (callbacks) => {
const key = await callbacks.prompt({ type: "secret", message: "Enter Cloudflare API key" });
const accountId = await callbacks.prompt({ type: "text", message: "Enter Cloudflare account ID" });
return { type: "api-key", key, metadata: { CLOUDFLARE_ACCOUNT_ID: accountId } };
},
resolve: async ({ model, ctx, credential }) => {
const resolved = await resolveCloudflareEnv({ kind: "workers-ai", model, ctx, credential });
if (!resolved) return undefined;
return {
auth: { apiKey: resolved.apiKey, baseUrl: resolved.baseUrl },
env: resolved.env,
source: resolved.source,
};
},
};
}
export function cloudflareAIGatewayAuth(): ApiKeyAuth {
return {
name: "Cloudflare API key",
login: async (callbacks) => {
const key = await callbacks.prompt({ type: "secret", message: "Enter Cloudflare API key" });
const accountId = await callbacks.prompt({ type: "text", message: "Enter Cloudflare account ID" });
const gatewayId = await callbacks.prompt({ type: "text", message: "Enter Cloudflare AI Gateway ID" });
return {
type: "api-key",
key,
metadata: { CLOUDFLARE_ACCOUNT_ID: accountId, CLOUDFLARE_GATEWAY_ID: gatewayId },
};
},
resolve: async ({ model, ctx, credential }) => {
const resolved = await resolveCloudflareEnv({ kind: "ai-gateway", model, ctx, credential });
if (!resolved) return undefined;
return {
auth: {
headers: {
"cf-aig-authorization": `Bearer ${resolved.apiKey}`,
Authorization: null,
"x-api-key": null,
},
baseUrl: resolved.baseUrl,
},
env: resolved.env,
source: resolved.source,
};
},
};
}
packages/ai/src/providers/cloudflare-workers-ai.models.ts
+13 -13
View File
@@ -10,7 +10,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -28,7 +28,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text"],
cost: {
@@ -46,7 +46,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text"],
cost: {
@@ -64,7 +64,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text", "image"],
cost: {
@@ -82,7 +82,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: false,
input: ["text"],
cost: {
@@ -100,7 +100,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -118,7 +118,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -136,7 +136,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
@@ -154,7 +154,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
@@ -172,7 +172,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
@@ -190,7 +190,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
@@ -208,7 +208,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
@@ -226,7 +226,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions",
provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
packages/ai/src/providers/cloudflare-workers-ai.ts
+2 -2
View File
@@ -1,13 +1,13 @@
import { openAICompletionsApi } from "../api/openai-completions.lazy.ts";
import { envApiKeyAuth } from "../auth/helpers.ts";
import { createProvider, type Provider } from "../models.ts";
import { cloudflareWorkersAIAuth } from "./cloudflare-auth.ts";
import { CLOUDFLARE_WORKERS_AI_MODELS } from "./cloudflare-workers-ai.models.ts";
export function cloudflareWorkersAIProvider(): Provider<"openai-completions"> {
return createProvider({
id: "cloudflare-workers-ai",
name: "Cloudflare Workers AI",
auth: { apiKey: envApiKeyAuth("Cloudflare API key", ["CLOUDFLARE_API_KEY"]) },
auth: { apiKey: cloudflareWorkersAIAuth() },
models: Object.values(CLOUDFLARE_WORKERS_AI_MODELS),
api: openAICompletionsApi(),
});
packages/ai/src/providers/deepseek.models.ts
+2 -2
View File
@@ -10,7 +10,7 @@ export const DEEPSEEK_MODELS = {
api: "openai-completions",
provider: "deepseek",
baseUrl: "https://api.deepseek.com",
compat: {"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -29,7 +29,7 @@ export const DEEPSEEK_MODELS = {
api: "openai-completions",
provider: "deepseek",
baseUrl: "https://api.deepseek.com",
compat: {"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
packages/ai/src/providers/faux.ts
+1 -21
View File
@@ -1,4 +1,3 @@
import { registerApiProvider, unregisterApiProviders } from "../api-registry.ts";
import { createProvider, type Provider } from "../models.ts";
import type {
AssistantMessage,
@@ -401,7 +400,7 @@ async function streamWithDeltas(
stream.end(message);
}
function createFauxCore(options: RegisterFauxProviderOptions) {
export function createFauxCore(options: RegisterFauxProviderOptions) {
const api = options.api ?? randomId(DEFAULT_API);
const provider = options.provider ?? DEFAULT_PROVIDER;
const minTokenSize = Math.max(
@@ -508,25 +507,6 @@ function createFauxCore(options: RegisterFauxProviderOptions) {
};
}
/** Registers the faux api into the legacy global api-registry. */
export function registerFauxProvider(options: RegisterFauxProviderOptions = {}): FauxProviderRegistration {
const core = createFauxCore(options);
const sourceId = randomId("faux-provider");
registerApiProvider({ api: core.api, stream: core.stream, streamSimple: core.streamSimple }, sourceId);
return {
api: core.api,
models: core.models,
getModel: core.getModel,
state: core.state,
setResponses: core.setResponses,
appendResponses: core.appendResponses,
getPendingResponseCount: core.getPendingResponseCount,
unregister() {
unregisterApiProviders(sourceId);
},
};
}
/**
* Faux provider for tests built on explicit `Models` collections:
*
packages/ai/src/providers/huggingface.models.ts
+396
View File
@@ -4,6 +4,24 @@
import type { Model } from "../types.ts";
export const HUGGINGFACE_MODELS = {
"MiniMaxAI/MiniMax-M2": {
id: "MiniMaxAI/MiniMax-M2",
name: "MiniMax-M2",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 204800,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
"MiniMaxAI/MiniMax-M2.1": {
id: "MiniMaxAI/MiniMax-M2.1",
name: "MiniMax-M2.1",
@@ -58,6 +76,42 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 204800,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"MiniMaxAI/MiniMax-M3": {
id: "MiniMaxAI/MiniMax-M3",
name: "MiniMax-M3",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 524288,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-235B-A22B": {
id: "Qwen/Qwen3-235B-A22B",
name: "Qwen3 235B-A22B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.2,
output: 0.8,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 40960,
maxTokens: 16384,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-235B-A22B-Thinking-2507": {
id: "Qwen/Qwen3-235B-A22B-Thinking-2507",
name: "Qwen3-235B-A22B-Thinking-2507",
@@ -76,6 +130,42 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-32B": {
id: "Qwen/Qwen3-32B",
name: "Qwen3 32B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.29,
output: 0.59,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 16384,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-Coder-30B-A3B-Instruct": {
id: "Qwen/Qwen3-Coder-30B-A3B-Instruct",
name: "Qwen3-Coder 30B-A3B Instruct",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: false,
input: ["text"],
cost: {
input: 0.07,
output: 0.26,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-Coder-480B-A35B-Instruct": {
id: "Qwen/Qwen3-Coder-480B-A35B-Instruct",
name: "Qwen3-Coder-480B-A35B-Instruct",
@@ -148,6 +238,60 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-122B-A10B": {
id: "Qwen/Qwen3.5-122B-A10B",
name: "Qwen3.5 122B-A10B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.4,
output: 3.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-27B": {
id: "Qwen/Qwen3.5-27B",
name: "Qwen3.5 27B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.3,
output: 2.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-35B-A3B": {
id: "Qwen/Qwen3.5-35B-A3B",
name: "Qwen3.5 35B-A3B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.25,
output: 2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-397B-A17B": {
id: "Qwen/Qwen3.5-397B-A17B",
name: "Qwen3.5-397B-A17B",
@@ -166,6 +310,42 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-9B": {
id: "Qwen/Qwen3.5-9B",
name: "Qwen3.5 9B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.17,
output: 0.25,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.6-35B-A3B": {
id: "Qwen/Qwen3.6-35B-A3B",
name: "Qwen3.6 35B-A3B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.15,
output: 0.95,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"XiaomiMiMo/MiMo-V2-Flash": {
id: "XiaomiMiMo/MiMo-V2-Flash",
name: "MiMo-V2-Flash",
@@ -184,6 +364,24 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144,
maxTokens: 4096,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-R1": {
id: "deepseek-ai/DeepSeek-R1",
name: "DeepSeek-R1",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.7,
output: 2.5,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 64000,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-R1-0528": {
id: "deepseek-ai/DeepSeek-R1-0528",
name: "DeepSeek-R1-0528",
@@ -220,6 +418,24 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 163840,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-V4-Flash": {
id: "deepseek-ai/DeepSeek-V4-Flash",
name: "DeepSeek V4 Flash",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.14,
output: 0.28,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 384000,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-V4-Pro": {
id: "deepseek-ai/DeepSeek-V4-Pro",
name: "DeepSeek V4 Pro",
@@ -238,6 +454,60 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 1048576,
maxTokens: 393216,
} satisfies Model<"openai-completions">,
"google/gemma-4-26B-A4B-it": {
id: "google/gemma-4-26B-A4B-it",
name: "Gemma 4 26B A4B IT",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.13,
output: 0.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"google/gemma-4-31B-it": {
id: "google/gemma-4-31B-it",
name: "Gemma 4 31B IT",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.14,
output: 0.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"meta-llama/Llama-3.3-70B-Instruct": {
id: "meta-llama/Llama-3.3-70B-Instruct",
name: "Llama-3.3-70B-Instruct",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: false,
input: ["text"],
cost: {
input: 0.59,
output: 0.79,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 4096,
} satisfies Model<"openai-completions">,
"moonshotai/Kimi-K2-Instruct": {
id: "moonshotai/Kimi-K2-Instruct",
name: "Kimi-K2-Instruct",
@@ -328,6 +598,114 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"moonshotai/Kimi-K2.7-Code": {
id: "moonshotai/Kimi-K2.7-Code",
name: "Kimi K2.7 Code",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"stepfun-ai/Step-3.5-Flash": {
id: "stepfun-ai/Step-3.5-Flash",
name: "Step 3.5 Flash",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.1,
output: 0.3,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 256000,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.5": {
id: "zai-org/GLM-4.5",
name: "GLM-4.5",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.6,
output: 2.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 98304,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.5-Air": {
id: "zai-org/GLM-4.5-Air",
name: "GLM-4.5-Air",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.13,
output: 0.85,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 98304,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.5V": {
id: "zai-org/GLM-4.5V",
name: "GLM-4.5V",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.6,
output: 1.8,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 65536,
maxTokens: 16384,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.6": {
id: "zai-org/GLM-4.6",
name: "GLM-4.6",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.55,
output: 2.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 204800,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.7": {
id: "zai-org/GLM-4.7",
name: "GLM-4.7",
@@ -400,4 +778,22 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 202752,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"zai-org/GLM-5.2": {
id: "zai-org/GLM-5.2",
name: "GLM-5.2",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 1.4,
output: 4.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
} as const;
packages/ai/src/providers/opencode-go.models.ts
+10 -10
View File
@@ -10,7 +10,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -29,7 +29,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -48,7 +48,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
@@ -66,7 +66,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -85,7 +85,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
@@ -104,7 +104,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -122,7 +122,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -140,7 +140,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
@@ -158,7 +158,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
@@ -193,7 +193,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions",
provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"thinkingFormat":"qwen","maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"thinkingFormat":"qwen","maxTokensField":"max_tokens"},
reasoning: true,
input: ["text", "image"],
cost: {
packages/ai/src/providers/opencode.models.ts
+32 -14
View File
@@ -10,7 +10,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
@@ -188,7 +188,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -207,7 +207,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -226,7 +226,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"],
@@ -299,7 +299,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
@@ -317,7 +317,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
@@ -329,6 +329,24 @@ export const OPENCODE_MODELS = {
contextWindow: 204800,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"glm-5.2": {
id: "glm-5.2",
name: "GLM-5.2",
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
input: 1.4,
output: 4.4,
cacheRead: 0.26,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"gpt-5": {
id: "gpt-5",
name: "GPT-5",
@@ -623,7 +641,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"supportsReasoningEffort":false,"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens"},
reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
@@ -642,7 +660,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -660,7 +678,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -678,7 +696,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -696,7 +714,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
@@ -714,7 +732,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true,
input: ["text"],
cost: {
@@ -732,7 +750,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
@@ -750,7 +768,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
packages/ai/src/providers/openrouter.models.ts
+282 -44
View File
File diff suppressed because it is too large Load Diff
packages/ai/src/providers/together.models.ts
+19 -19
View File
@@ -10,7 +10,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null},
input: ["text"],
@@ -29,7 +29,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
@@ -48,7 +48,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
@@ -66,7 +66,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
@@ -84,7 +84,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
@@ -103,7 +103,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
@@ -122,7 +122,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
@@ -141,7 +141,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
@@ -159,7 +159,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null},
input: ["text"],
@@ -178,7 +178,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
@@ -196,7 +196,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
@@ -215,7 +215,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false,
input: ["text"],
cost: {
@@ -233,7 +233,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"],
@@ -252,7 +252,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
@@ -271,7 +271,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
@@ -290,7 +290,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"openai"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","thinkingFormat":"openai","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null},
input: ["text"],
@@ -309,7 +309,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"openai"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","thinkingFormat":"openai","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null},
input: ["text"],
@@ -328,7 +328,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
@@ -347,7 +347,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions",
provider: "together",
baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"],
packages/ai/src/providers/xai.models.ts
+7
View File
@@ -10,6 +10,7 @@ export const XAI_MODELS = {
api: "openai-completions",
provider: "xai",
baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: false,
input: ["text"],
cost: {
@@ -27,6 +28,7 @@ export const XAI_MODELS = {
api: "openai-completions",
provider: "xai",
baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: false,
input: ["text"],
cost: {
@@ -44,6 +46,7 @@ export const XAI_MODELS = {
api: "openai-completions",
provider: "xai",
baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: false,
input: ["text", "image"],
cost: {
@@ -61,6 +64,7 @@ export const XAI_MODELS = {
api: "openai-completions",
provider: "xai",
baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -78,6 +82,7 @@ export const XAI_MODELS = {
api: "openai-completions",
provider: "xai",
baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -95,6 +100,7 @@ export const XAI_MODELS = {
api: "openai-completions",
provider: "xai",
baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: true,
input: ["text", "image"],
cost: {
@@ -112,6 +118,7 @@ export const XAI_MODELS = {
api: "openai-completions",
provider: "xai",
baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: false,
input: ["text"],
cost: {
packages/ai/src/providers/zai-coding-cn.models.ts
+6 -6
View File
@@ -10,7 +10,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions",
provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai"},
reasoning: true,
input: ["text"],
cost: {
@@ -28,7 +28,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions",
provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
input: ["text"],
cost: {
@@ -46,7 +46,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions",
provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
input: ["text"],
cost: {
@@ -64,7 +64,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions",
provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
input: ["text"],
cost: {
@@ -82,7 +82,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions",
provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","supportsReasoningEffort":true,"zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":"high","medium":"high","high":"high","xhigh":"max"},
input: ["text"],
@@ -101,7 +101,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions",
provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
input: ["text", "image"],
cost: {
packages/ai/src/providers/zai.models.ts
+6 -6
View File
@@ -10,7 +10,7 @@ export const ZAI_MODELS = {
api: "openai-completions",
provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai"},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai"},
reasoning: true,
input: ["text"],
cost: {
@@ -28,7 +28,7 @@ export const ZAI_MODELS = {
api: "openai-completions",
provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
input: ["text"],
cost: {
@@ -46,7 +46,7 @@ export const ZAI_MODELS = {
api: "openai-completions",
provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
input: ["text"],
cost: {
@@ -64,7 +64,7 @@ export const ZAI_MODELS = {
api: "openai-completions",
provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
input: ["text"],
cost: {
@@ -82,7 +82,7 @@ export const ZAI_MODELS = {
api: "openai-completions",
provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","supportsReasoningEffort":true,"zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
thinkingLevelMap: {"minimal":null,"low":"high","medium":"high","high":"high","xhigh":"max"},
input: ["text"],
@@ -101,7 +101,7 @@ export const ZAI_MODELS = {
api: "openai-completions",
provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true},
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true,
input: ["text", "image"],
cost: {
packages/ai/src/types.ts
+5 -2
View File
@@ -99,6 +99,7 @@ export type Transport = "sse" | "websocket" | "websocket-cached" | "auto";
/** Provider-scoped environment overrides. Values take precedence over process.env. */
export type ProviderEnv = Record<string, string>;
export type ProviderHeaders = Record<string, string | null>;
export interface ProviderResponse {
status: number;
@@ -142,8 +143,9 @@ export interface StreamOptions {
* On AWS Bedrock these are injected via a Smithy `build`-step middleware so
* they are covered by SigV4 signing; reserved headers (`x-amz-*`,
* `authorization`, `host`) are silently ignored to preserve SigV4 / bearer auth.
* A null value suppresses a provider/API default header with the same name.
*/
headers?: Record<string, string>;
headers?: ProviderHeaders;
/**
* HTTP request timeout in milliseconds for providers/SDKs that support it.
* For example, OpenAI and Anthropic SDK clients default to 10 minutes.
@@ -256,8 +258,9 @@ export interface ImagesOptions {
/**
* Optional custom HTTP headers to include in API requests.
* Merged with provider defaults; can override default headers.
* A null value suppresses a provider/API default header with the same name.
*/
headers?: Record<string, string>;
headers?: ProviderHeaders;
/**
* HTTP request timeout in milliseconds for providers/SDKs that support it.
*/
packages/ai/src/utils/headers.ts
+11
View File
@@ -1,3 +1,5 @@
import type { ProviderHeaders } from "../types.ts";
export function headersToRecord(headers: Headers): Record<string, string> {
const result: Record<string, string> = {};
for (const [key, value] of headers.entries()) {
@@ -5,3 +7,12 @@ export function headersToRecord(headers: Headers): Record<string, string> {
}
return result;
}
export function providerHeadersToRecord(headers: ProviderHeaders | undefined): Record<string, string> | undefined {
if (!headers) return undefined;
const result: Record<string, string> = {};
for (const [key, value] of Object.entries(headers)) {
if (value !== null) result[key] = value;
}
return Object.keys(result).length > 0 ? result : undefined;
}
packages/ai/test/compat-env.test.ts
+5 -5
View File
@@ -9,7 +9,7 @@ const model: Model<"openai-responses"> = {
id: "test-model",
name: "Test Model",
api: "openai-responses",
provider: "openai",
provider: "custom-openai",
baseUrl: "https://example.test/v1",
reasoning: false,
input: ["text"],
@@ -38,12 +38,12 @@ function message(): AssistantMessage {
};
}
describe("compat env API key injection", () => {
describe("compat legacy API fallback", () => {
afterEach(() => {
resetApiProviders();
});
it("uses request-scoped env when injecting provider API keys", async () => {
it("dispatches unknown providers through the legacy API registry", async () => {
let capturedApiKey: string | undefined;
registerApiProvider({
api: "openai-responses",
@@ -67,8 +67,8 @@ describe("compat env API key injection", () => {
},
});
await complete(model, context, { env: { OPENAI_API_KEY: "scoped-key" } });
await complete(model, context, { apiKey: "request-key" });
expect(capturedApiKey).toBe("scoped-key");
expect(capturedApiKey).toBe("request-key");
});
});
packages/ai/test/openai-completions-empty-tools.test.ts
+14 -22
View File
@@ -127,6 +127,7 @@ describe("openai-completions empty tools handling", () => {
});
it("uses conservative OpenAI-compatible fields for Cloudflare AI Gateway /compat models", async () => {
process.env.CLOUDFLARE_API_KEY = "cf-token";
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
const model = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!;
@@ -137,7 +138,7 @@ describe("openai-completions empty tools handling", () => {
systemPrompt: "You are helpful.",
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{ apiKey: "test", maxTokens: 1234, reasoning: "high" },
{ maxTokens: 1234, reasoning: "high" },
).result();
const params = mockState.lastParams as {
@@ -159,35 +160,25 @@ describe("openai-completions empty tools handling", () => {
};
expect(clientOptions.baseURL).toBe("https://gateway.ai.cloudflare.com/v1/account-id/gateway-id/compat");
expect(clientOptions.defaultHeaders?.Authorization).toBeNull();
expect(clientOptions.defaultHeaders?.["cf-aig-authorization"]).toBe("Bearer test");
expect(clientOptions.defaultHeaders?.["cf-aig-authorization"]).toBe("Bearer cf-token");
});
it("uses provider env before process.env for Cloudflare AI Gateway base URL", async () => {
process.env.CLOUDFLARE_ACCOUNT_ID = "process-account";
process.env.CLOUDFLARE_GATEWAY_ID = "process-gateway";
it("resolves Cloudflare AI Gateway base URL through provider auth", async () => {
process.env.CLOUDFLARE_API_KEY = "cf-token";
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
const model = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!;
await streamSimple(
model,
{
await streamSimple(model, {
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{
apiKey: "test",
env: {
CLOUDFLARE_ACCOUNT_ID: "provider-account",
CLOUDFLARE_GATEWAY_ID: "provider-gateway",
},
},
).result();
}).result();
const clientOptions = mockState.lastClientOptions as { baseURL?: string };
expect(clientOptions.baseURL).toBe(
"https://gateway.ai.cloudflare.com/v1/provider-account/provider-gateway/compat",
);
expect(clientOptions.baseURL).toBe("https://gateway.ai.cloudflare.com/v1/account-id/gateway-id/compat");
});
it("preserves inline upstream Authorization for Cloudflare AI Gateway BYOK requests", async () => {
process.env.CLOUDFLARE_API_KEY = "cf-token";
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
const model = getModel("cloudflare-ai-gateway", "gpt-5.1")!;
@@ -197,7 +188,7 @@ describe("openai-completions empty tools handling", () => {
{
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{ apiKey: "cf-token", headers: { Authorization: "Bearer upstream-token" } },
{ headers: { Authorization: "Bearer upstream-token" } },
).result();
const clientOptions = mockState.lastClientOptions as { defaultHeaders?: Record<string, unknown> };
@@ -206,6 +197,7 @@ describe("openai-completions empty tools handling", () => {
});
it("sends session affinity headers for Workers AI through Cloudflare AI Gateway", async () => {
process.env.CLOUDFLARE_API_KEY = "cf-token";
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
const workersModel = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!;
@@ -215,7 +207,7 @@ describe("openai-completions empty tools handling", () => {
{
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{ apiKey: "test", sessionId: "session-1" },
{ sessionId: "session-1" },
).result();
const clientOptions = mockState.lastClientOptions as { defaultHeaders?: Record<string, string> };
packages/ai/test/providers.test.ts
+51
View File
@@ -5,6 +5,8 @@ import { createModels, createProvider } from "../src/models.ts";
import { builtinModels, builtinProviders } from "../src/providers/all.ts";
import { amazonBedrockProvider } from "../src/providers/amazon-bedrock.ts";
import { anthropicProvider } from "../src/providers/anthropic.ts";
import { cloudflareAIGatewayProvider } from "../src/providers/cloudflare-ai-gateway.ts";
import { cloudflareWorkersAIProvider } from "../src/providers/cloudflare-workers-ai.ts";
import { fauxAssistantMessage, fauxProvider } from "../src/providers/faux.ts";
import { googleVertexProvider } from "../src/providers/google-vertex.ts";
import type { Api, Context, Model, ProviderStreams } from "../src/types.ts";
@@ -66,6 +68,55 @@ describe("builtin providers", () => {
expect(await unconfigured.getAuth(model)).toBeUndefined();
});
it("requires Cloudflare Workers AI account config and returns scoped env", async () => {
const missingAccount = createModels({ authContext: fakeAuthContext({ CLOUDFLARE_API_KEY: "cf-key" }) });
missingAccount.setProvider(cloudflareWorkersAIProvider());
const model = missingAccount.getModels("cloudflare-workers-ai")[0];
expect(await missingAccount.getAuth(model)).toBeUndefined();
const configured = createModels({
authContext: fakeAuthContext({ CLOUDFLARE_API_KEY: "cf-key", CLOUDFLARE_ACCOUNT_ID: "account-id" }),
});
configured.setProvider(cloudflareWorkersAIProvider());
const result = await configured.getAuth(model);
expect(result?.auth).toEqual({
apiKey: "cf-key",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/account-id/ai/v1",
});
expect(result?.env).toEqual({ CLOUDFLARE_ACCOUNT_ID: "account-id" });
});
it("requires Cloudflare AI Gateway account and gateway config and returns scoped env headers", async () => {
const missingGateway = createModels({
authContext: fakeAuthContext({ CLOUDFLARE_API_KEY: "cf-key", CLOUDFLARE_ACCOUNT_ID: "account-id" }),
});
missingGateway.setProvider(cloudflareAIGatewayProvider());
const model = missingGateway.getModels("cloudflare-ai-gateway")[0];
expect(await missingGateway.getAuth(model)).toBeUndefined();
const configured = createModels({
authContext: fakeAuthContext({
CLOUDFLARE_API_KEY: "cf-key",
CLOUDFLARE_ACCOUNT_ID: "account-id",
CLOUDFLARE_GATEWAY_ID: "gateway-id",
}),
});
configured.setProvider(cloudflareAIGatewayProvider());
const result = await configured.getAuth(model);
expect(result?.auth).toEqual({
headers: {
"cf-aig-authorization": "Bearer cf-key",
Authorization: null,
"x-api-key": null,
},
baseUrl: "https://gateway.ai.cloudflare.com/v1/account-id/gateway-id/anthropic",
});
expect(result?.env).toEqual({
CLOUDFLARE_ACCOUNT_ID: "account-id",
CLOUDFLARE_GATEWAY_ID: "gateway-id",
});
});
it("resolves vertex via ADC file plus project and location", async () => {
const adc = "~/.config/gcloud/application_default_credentials.json";
const configured = createModels({
packages/coding-agent/src/core/provider-attribution.ts
+4 -4
View File
@@ -1,4 +1,4 @@
import type { Api, Model } from "@earendil-works/pi-ai";
import type { Api, Model, ProviderHeaders } from "@earendil-works/pi-ai";
import type { SettingsManager } from "./settings-manager.ts";
import { isInstallTelemetryEnabled } from "./telemetry.ts";
@@ -92,9 +92,9 @@ export function mergeProviderAttributionHeaders(
model: Model<Api>,
settingsManager: SettingsManager,
sessionId: string | undefined,
...headerSources: Array<Record<string, string> | undefined>
): Record<string, string> | undefined {
const merged = {
...headerSources: Array<ProviderHeaders | undefined>
): ProviderHeaders | undefined {
const merged: ProviderHeaders = {
...getSessionHeaders(model, sessionId),
...getDefaultAttributionHeaders(model, settingsManager),
};
packages/coding-agent/test/agent-session-runtime-events.test.ts
+1 -1
View File
@@ -1,7 +1,7 @@
import { existsSync, mkdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest";
import {
type CreateAgentSessionRuntimeFactory,
packages/coding-agent/test/sdk-openrouter-attribution.test.ts
+2 -1
View File
@@ -6,6 +6,7 @@ import {
type AssistantMessage,
createAssistantMessageEventStream,
type Model,
type ProviderHeaders,
type SimpleStreamOptions,
} from "@earendil-works/pi-ai";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
@@ -88,7 +89,7 @@ describe("createAgentSession provider attribution headers", () => {
requestHeaders?: Record<string, string>;
sessionId?: string;
} = {},
): Promise<Record<string, string> | undefined> {
): Promise<ProviderHeaders | undefined> {
const settingsManager = SettingsManager.create(cwd, agentDir);
if (options.telemetryEnabled === false) {
settingsManager.setEnableInstallTelemetry(false);
packages/coding-agent/test/suite/agent-session-runtime.test.ts
+1 -1
View File
@@ -1,7 +1,7 @@
import { existsSync, mkdirSync, realpathSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, parse } from "node:path";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest";
import {
type CreateAgentSessionRuntimeFactory,
packages/coding-agent/test/suite/harness.ts
+7 -2
View File
@@ -7,8 +7,13 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import type { AgentMessage, AgentTool } from "@earendil-works/pi-agent-core";
import { Agent } from "@earendil-works/pi-agent-core";
import type { FauxModelDefinition, FauxProviderRegistration, FauxResponseStep, Model } from "@earendil-works/pi-ai";
import { registerFauxProvider } from "@earendil-works/pi-ai";
import type {
FauxModelDefinition,
FauxProviderRegistration,
FauxResponseStep,
Model,
} from "@earendil-works/pi-ai/compat";
import { registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { AgentSession, type AgentSessionEvent } from "../../src/core/agent-session.ts";
import { AuthStorage } from "../../src/core/auth-storage.ts";
import type { ExtensionRunner } from "../../src/core/extensions/index.ts";
packages/coding-agent/test/suite/regressions/2753-reload-stale-resource-settings.test.ts
+1 -1
View File
@@ -1,7 +1,7 @@
import { existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { registerFauxProvider } from "@earendil-works/pi-ai";
import { registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest";
import {
type CreateAgentSessionRuntimeFactory,
packages/coding-agent/test/suite/regressions/2860-replaced-session-context.test.ts
+1 -1
View File
@@ -1,7 +1,7 @@
import { existsSync, mkdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest";
import type { AgentSession } from "../../../src/core/agent-session.ts";
import {
packages/coding-agent/test/suite/regressions/5596-missing-theme-export.test.ts
+1 -1
View File
@@ -2,7 +2,7 @@ import { existsSync, mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { Agent } from "@earendil-works/pi-agent-core";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest";
import { AgentSession } from "../../../src/core/agent-session.ts";
import { AuthStorage } from "../../../src/core/auth-storage.ts";