feat(ai): complete models runtime migration

This commit is contained in:
Mario Zechner
2026-06-23 15:29:17 +02:00
parent 470a4736a3
commit 129eb460cd
47 changed files with 1502 additions and 576 deletions
+8 -7
View File
@@ -702,10 +702,10 @@ Built-in provider factories use `createProvider()` internally. models.json custo
`@earendil-works/pi-ai/compat` preserves the old global API surface until the coding-agent migration deletes it. New code never imports it. `@earendil-works/pi-ai/compat` preserves the old global API surface until the coding-agent migration deletes it. New code never imports it.
Old semantics being preserved: global `stream()` dispatched purely on `model.api` via the api-registry, with env API key injection. The compat module reproduces this exactly — it does not route through a `Models` collection, so compat consumers get zero behavioral drift (a `Models`-routed variant was considered and dropped: a model with a known provider id but a different api would dispatch wrong, and auth semantics would shift mid-migration). The harness `Models` instance (Phase 6/7) is where new-path streaming happens. Old semantics being preserved: global `stream()` can still dispatch by `model.api` through the legacy api-registry for custom providers, mutated models, and tests/extensions that override a built-in API implementation.
- `stream/complete/streamSimple/completeSimple(model, ctx, opts)`: api-dispatch via the api-registry plus `getEnvApiKey` injection, verbatim old behavior. - `stream/complete/streamSimple/completeSimple(model, ctx, opts)`: real built-in provider/model/api matches route through a singleton `builtinModels()` collection, so provider auth/env/baseUrl behavior is shared with the new runtime. Unknown providers, mutated models, or overridden API registrations fall back to api-registry dispatch plus `getEnvApiKey` injection.
- The builtin api registration side effect moves from the root barrel into compat. It skips api ids that already have a registration, since compat may load after a test or extension registered an override. `registerApiProvider()/unregisterApiProviders()` keep feeding the registry; `resetApiProviders()` clears and re-registers builtins. - The builtin api registration side effect moves from the root barrel into compat. It skips api ids that already have a registration, since compat may load after a test or extension has already registered an override. `registerApiProvider()/unregisterApiProviders()` keep feeding the compat-local registry; `resetApiProviders()` clears and re-registers builtins.
- Sync `getModel/getModels/getProviders` are deprecated aliases of `getBuiltinModel/getBuiltinModels/getBuiltinProviders` from `providers/all` (they were always pure generated-catalog reads — verified: nothing ever mutated the old `modelRegistry`). - Sync `getModel/getModels/getProviders` are deprecated aliases of `getBuiltinModel/getBuiltinModels/getBuiltinProviders` from `providers/all` (they were always pure generated-catalog reads — verified: nothing ever mutated the old `modelRegistry`).
- Re-exports the per-API lazy stream wrappers (incl. `setBedrockProviderModule`), `env-api-keys.ts`, and the image-generation registry/catalogs; none of these stay on the root barrel. - Re-exports the per-API lazy stream wrappers (incl. `setBedrockProviderModule`), `env-api-keys.ts`, and the image-generation registry/catalogs; none of these stay on the root barrel.
- `export * from "./index.ts"`: compat is a strict superset of the core entrypoint, so consumers switch a file's import path wholesale without symbol surgery. - `export * from "./index.ts"`: compat is a strict superset of the core entrypoint, so consumers switch a file's import path wholesale without symbol surgery.
@@ -873,15 +873,16 @@ Decisions:
- Runtime `--api-key` overrides are a store overlay (an override reads as an ephemeral stored api-key credential, masking stored OAuth — matches today's priority). Every registered provider is guaranteed an `apiKey` auth slot so overrides apply to OAuth-only providers too. - Runtime `--api-key` overrides are a store overlay (an override reads as an ephemeral stored api-key credential, masking stored OAuth — matches today's priority). Every registered provider is guaranteed an `apiKey` auth slot so overrides apply to OAuth-only providers too.
- `ModelRegistry.getAll`/`find`/`getAvailable` become async, delegating to the collection (no materialized snapshot, no sync lies; dynamic providers like llama.cpp work). The extension-facing `modelRegistry` surface changes accordingly (breaking, changelogged); extensions also get the collection itself as the forward API. - `ModelRegistry.getAll`/`find`/`getAvailable` become async, delegating to the collection (no materialized snapshot, no sync lies; dynamic providers like llama.cpp work). The extension-facing `modelRegistry` surface changes accordingly (breaking, changelogged); extensions also get the collection itself as the forward API.
- models.json keeps FULL feature parity, implemented as provider decoration: builtin factories wrapped so `getModels()` applies provider `baseUrl`/`compat` overlays, `modelOverrides`, and custom-model merges (async-safe); provider `apiKey`/`headers`/`authHeader` configs become that provider's `ApiKeyAuth` (config first, factory auth fallback); parse errors keep `getError()` semantics. - models.json keeps FULL feature parity, implemented as provider decoration: builtin factories wrapped so `getModels()` applies provider `baseUrl`/`compat` overlays, `modelOverrides`, and custom-model merges (async-safe); provider `apiKey`/`headers`/`authHeader` configs become that provider's `ApiKeyAuth` (config first, factory auth fallback); parse errors keep `getError()` semantics.
- Extension `ProviderConfig` parity: provider-keyed `streamSimple`, old-style `oauth` adapted to `OAuthAuth` (`modifyModels` -> `getModels` wrap + `toAuth`), full model replacement per provider. The api-registry `registerApiProvider` dual-write stays for compat consumers (extensions calling global `complete()`); it dies with compat. - Extension `ProviderConfig` parity: provider-keyed `streamSimple`, old-style `oauth` adapted to `OAuthAuth` (`modifyModels` -> `getModels` wrap + `toAuth`), full model replacement per provider. Legacy `registerApiProvider` writes stay compat-local for consumers that call global `complete()`; they die with compat.
- Copilot: stored-credential baseUrl applied in the wrapped `getModels()` (extension-visible models stay correct) plus per-request `toAuth().baseUrl`. - Copilot: stored-credential baseUrl applied in the wrapped `getModels()` (extension-visible models stay correct) plus per-request `toAuth().baseUrl`.
- Cloudflare: provider-auth substitution (key + `CLOUDFLARE_ACCOUNT_ID`/`CLOUDFLARE_GATEWAY_ID` from credential metadata/env -> `ModelAuth.baseUrl`); impl-side `resolveCloudflareBaseUrl` stays until compat dies (it is idempotent on substituted URLs), keeping extension `complete()` calls working. - Cloudflare: provider-auth substitution (key + `CLOUDFLARE_ACCOUNT_ID`/`CLOUDFLARE_GATEWAY_ID` from credential metadata/env -> `ModelAuth.baseUrl`). Built-in compat calls route through `Models`, so they use the same provider auth path.
Ordering: Ordering:
- [x] pi-ai rework first: `Provider.getModels()` sync + optional `refreshModels()`; `Models.getModels`/`getModel` sync, `Models.refresh(provider?)` async; `createProvider` takes `models` array + optional `refreshModels` fetcher (in-flight dedupe). Reverses Phase 1's async-listing decision — see "Provider model listing" for rationale (sync-or-async unions breed latent sync assumptions; async-only breaks sync consumer surfaces like extension `find`/`getAll`). - [x] pi-ai rework first: `Provider.getModels()` sync + optional `refreshModels()`; `Models.getModels`/`getModel` sync, `Models.refresh(provider?)` async; `createProvider` takes `models` array + optional `refreshModels` fetcher (in-flight dedupe). Reverses Phase 1's async-listing decision — see "Provider model listing" for rationale (sync-or-async unions breed latent sync assumptions; async-only breaks sync consumer surfaces like extension `find`/`getAll`).
- [ ] `FileCredentialStore` (ports the auth.json lock backend, reads legacy `type: "api_key"` tags) + `--api-key` overlay + `$ENV`/`!command` resolution; tests. - [ ] `FileCredentialStore` (ports the auth.json lock backend, reads legacy `type: "api_key"` tags) + `--api-key` overlay + `$ENV`/`!command` resolution; tests.
- [ ] Cloudflare provider auth in pi-ai factories; copilot `getModels` baseUrl wrap. - [x] Cloudflare provider auth in pi-ai factories.
- [ ] Copilot `getModels` baseUrl wrap.
- [ ] Extension-OAuth adapter (old `OAuthProviderInterface` config -> `OAuthAuth`). - [ ] Extension-OAuth adapter (old `OAuthProviderInterface` config -> `OAuthAuth`).
- [ ] ModelRegistry rebuild: owns `MutableModels`, async reads, models.json decoration, provider-keyed extension streams, facade auth ops; AuthStorage deleted. - [ ] ModelRegistry rebuild: owns `MutableModels`, async reads, models.json decoration, provider-keyed extension streams, facade auth ops; AuthStorage deleted.
- [ ] Consumer rewiring: agent-session, sdk (`credentials?: CredentialStore` option replaces `authStorage`; sdk.md updated), model-resolver, interactive login/status UI on `prompt()/notify()`, cli `--api-key`. - [ ] Consumer rewiring: agent-session, sdk (`credentials?: CredentialStore` option replaces `authStorage`; sdk.md updated), model-resolver, interactive login/status UI on `prompt()/notify()`, cli `--api-key`.
@@ -891,7 +892,7 @@ Ordering:
- [ ] AgentSession -> AgentHarness; the registry facade dies in favor of harness `Models`. - [ ] AgentSession -> AgentHarness; the registry facade dies in favor of harness `Models`.
- [ ] Move ALL internal `/compat` imports to the new API: every package's src, all tests, and the example extensions (examples then demonstrate the new API). Nothing inside the repo may import `/compat` at that point. - [ ] Move ALL internal `/compat` imports to the new API: every package's src, all tests, and the example extensions (examples then demonstrate the new API). Nothing inside the repo may import `/compat` at that point.
- [ ] Delete `/compat`, `api-registry.ts`, `env-api-keys.ts`, the extension-loader root-to-compat alias, the old `pi-ai/oauth` registry and `OAuthProviderInterface` (incl. `usesCallbackServer`), and the impl-side cloudflare substitution. This is the extension-author breaking release; changelog carries the migration guide. - [ ] Delete `/compat`, `env-api-keys.ts`, the extension-loader root-to-compat alias, the old `pi-ai/oauth` registry and `OAuthProviderInterface` (incl. `usesCallbackServer`), and the compat-local legacy API registry. This is the extension-author breaking release; changelog carries the migration guide.
### Deferred / follow-ups ### Deferred / follow-ups
+1 -1
View File
@@ -9,7 +9,7 @@ import {
registerFauxProvider, registerFauxProvider,
type ToolResultMessage, type ToolResultMessage,
type UserMessage, type UserMessage,
} from "@earendil-works/pi-ai"; } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest"; import { afterEach, describe, expect, it } from "vitest";
import { Agent, type AgentEvent } from "../src/index.ts"; import { Agent, type AgentEvent } from "../src/index.ts";
import { calculateTool } from "./utils/calculate.ts"; import { calculateTool } from "./utils/calculate.ts";
+28
View File
@@ -8,6 +8,33 @@
- Renamed the `Provider` type to `ProviderId`. `Provider` now names the runtime provider interface (id, name, auth, model listing, stream behavior). - Renamed the `Provider` type to `ProviderId`. `Provider` now names the runtime provider interface (id, name, auth, model listing, stream behavior).
- API implementation modules moved from `src/providers/` to `@earendil-works/pi-ai/api/*`, renamed by API id (`anthropic` -> `api/anthropic-messages`, `google` -> `api/google-generative-ai`, `mistral` -> `api/mistral-conversations`, `amazon-bedrock` -> `api/bedrock-converse-stream`), each exporting exactly `stream` and `streamSimple`. The old per-impl export names (`streamAnthropic`, `streamSimpleAnthropic`, ...) are gone; the legacy package subpaths (`./anthropic`, `./google`, ...) keep working and point at the new modules. - API implementation modules moved from `src/providers/` to `@earendil-works/pi-ai/api/*`, renamed by API id (`anthropic` -> `api/anthropic-messages`, `google` -> `api/google-generative-ai`, `mistral` -> `api/mistral-conversations`, `amazon-bedrock` -> `api/bedrock-converse-stream`), each exporting exactly `stream` and `streamSimple`. The old per-impl export names (`streamAnthropic`, `streamSimpleAnthropic`, ...) are gone; the legacy package subpaths (`./anthropic`, `./google`, ...) keep working and point at the new modules.
Migration guide:
- To keep the old global API temporarily, change imports from `@earendil-works/pi-ai` to `@earendil-works/pi-ai/compat`. The compat entrypoint preserves `stream`/`complete`, generated catalog reads, API registry helpers, env API-key helpers, lazy API wrappers, and image globals, but it will be removed in a future release.
- To migrate to the new runtime, create a `Models` collection and call methods on it:
```ts
import { builtinModels } from "@earendil-works/pi-ai/providers/all";
const models = builtinModels();
const model = models.getModel("anthropic", "claude-haiku-4-5");
if (!model) throw new Error("model not found");
const message = await models.complete(model, {
messages: [{ role: "user", content: "Hello", timestamp: Date.now() }],
});
```
- For an isolated provider set, register provider factories explicitly:
```ts
import { createModels } from "@earendil-works/pi-ai";
import { anthropicProvider } from "@earendil-works/pi-ai/providers/anthropic";
const models = createModels();
models.setProvider(anthropicProvider());
```
### Added ### Added
- New `Models` runtime: `createModels()` builds an isolated provider collection with sync model reads (`getModels`/`getModel` return the last-known lists), an explicit async `refresh(provider?)` for dynamic providers, auth resolution (`getAuth`), and `stream`/`complete`/`streamSimple`/`completeSimple` that resolve auth through the owning provider. `createProvider()` builds providers from parts (single API implementation or a map dispatched on `model.api`; static `models` array plus an optional `refreshModels` fetcher with in-flight dedupe); `hasApi()` narrows dynamically listed models. - New `Models` runtime: `createModels()` builds an isolated provider collection with sync model reads (`getModels`/`getModel` return the last-known lists), an explicit async `refresh(provider?)` for dynamic providers, auth resolution (`getAuth`), and `stream`/`complete`/`streamSimple`/`completeSimple` that resolve auth through the owning provider. `createProvider()` builds providers from parts (single API implementation or a map dispatched on `model.api`; static `models` array plus an optional `refreshModels` fetcher with in-flight dedupe); `hasApi()` narrows dynamically listed models.
@@ -22,6 +49,7 @@
### Fixed ### Fixed
- Fixed Amazon Bedrock endpoint resolution to honor scoped `AWS_PROFILE` values. - Fixed Amazon Bedrock endpoint resolution to honor scoped `AWS_PROFILE` values.
- Fixed Cloudflare providers to require account/gateway configuration and route built-in `/compat` requests through provider auth.
- Fixed OpenAI Codex Responses WebSocket sessions to reconnect once when OpenAI's connection limit is reached before output starts ([#5973](https://github.com/earendil-works/pi/issues/5973)). - Fixed OpenAI Codex Responses WebSocket sessions to reconnect once when OpenAI's connection limit is reached before output starts ([#5973](https://github.com/earendil-works/pi/issues/5973)).
- Fixed OpenCode Go GLM-5.2 metadata to expose `xhigh` reasoning and send `reasoning_effort: "max"` ([#5967](https://github.com/earendil-works/pi/issues/5967)). - Fixed OpenCode Go GLM-5.2 metadata to expose `xhigh` reasoning and send `reasoning_effort: "max"` ([#5967](https://github.com/earendil-works/pi/issues/5967)).
- Fixed Claude Fable 5 thinking-off requests to omit Anthropic's unsupported `thinking.type: "disabled"` payload ([#5567](https://github.com/earendil-works/pi/pull/5567) by [@tmustier](https://github.com/tmustier)). - Fixed Claude Fable 5 thinking-off requests to omit Anthropic's unsupported `thinking.type: "disabled"` payload ([#5567](https://github.com/earendil-works/pi/pull/5567) by [@tmustier](https://github.com/tmustier)).
+143 -3
View File
@@ -277,10 +277,148 @@ function isAnthropicTemperatureUnsupportedModel(modelId: string): boolean {
return id.includes("opus-4-7") || id.includes("opus-4.7") || id.includes("opus-4-8") || id.includes("opus-4.8"); return id.includes("opus-4-7") || id.includes("opus-4.7") || id.includes("opus-4-8") || id.includes("opus-4.8");
} }
const OPENAI_COMPLETIONS_DEFAULT_COMPAT = {
supportsStore: true,
supportsDeveloperRole: true,
supportsReasoningEffort: true,
supportsUsageInStreaming: true,
maxTokensField: "max_completion_tokens",
requiresToolResultName: false,
requiresAssistantAfterToolResult: false,
requiresThinkingAsText: false,
requiresReasoningContentOnAssistantMessages: false,
thinkingFormat: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: true,
sendSessionAffinityHeaders: false,
supportsLongCacheRetention: true,
} satisfies Required<Omit<OpenAICompletionsCompat, "cacheControlFormat">> & {
cacheControlFormat?: OpenAICompletionsCompat["cacheControlFormat"];
};
type OpenAICompletionsResolvedCompat = typeof OPENAI_COMPLETIONS_DEFAULT_COMPAT & {
cacheControlFormat?: OpenAICompletionsCompat["cacheControlFormat"];
};
function mergeAnthropicMessagesCompat(model: Model<Api>, compat: AnthropicMessagesCompat): void { function mergeAnthropicMessagesCompat(model: Model<Api>, compat: AnthropicMessagesCompat): void {
model.compat = { ...(model.compat as AnthropicMessagesCompat | undefined), ...compat }; model.compat = { ...(model.compat as AnthropicMessagesCompat | undefined), ...compat };
} }
function detectOpenAICompletionsCompat(model: Model<"openai-completions">): OpenAICompletionsResolvedCompat {
const provider = model.provider;
const baseUrl = model.baseUrl;
const isZai =
provider === "zai" ||
provider === "zai-coding-cn" ||
baseUrl.includes("api.z.ai") ||
baseUrl.includes("open.bigmodel.cn");
const isTogether =
provider === "together" || baseUrl.includes("api.together.ai") || baseUrl.includes("api.together.xyz");
const isMoonshot = provider === "moonshotai" || provider === "moonshotai-cn" || baseUrl.includes("api.moonshot.");
const isOpenRouter = provider === "openrouter" || baseUrl.includes("openrouter.ai");
const isCloudflareWorkersAI = provider === "cloudflare-workers-ai" || baseUrl.includes("api.cloudflare.com");
const isCloudflareAiGateway = provider === "cloudflare-ai-gateway" || baseUrl.includes("gateway.ai.cloudflare.com");
const isNvidia = provider === "nvidia" || baseUrl.includes("integrate.api.nvidia.com");
const isAntLing = provider === "ant-ling" || baseUrl.includes("api.ant-ling.com");
const isNonStandard =
isNvidia ||
provider === "cerebras" ||
baseUrl.includes("cerebras.ai") ||
provider === "xai" ||
baseUrl.includes("api.x.ai") ||
isTogether ||
baseUrl.includes("chutes.ai") ||
baseUrl.includes("deepseek.com") ||
isZai ||
isMoonshot ||
provider === "opencode" ||
baseUrl.includes("opencode.ai") ||
isCloudflareWorkersAI ||
isCloudflareAiGateway ||
isAntLing;
const useMaxTokens =
baseUrl.includes("chutes.ai") || isMoonshot || isCloudflareAiGateway || isTogether || isNvidia || isAntLing;
const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
const isDeepSeek = provider === "deepseek" || baseUrl.includes("deepseek.com");
const isOpenRouterDeveloperRoleModel =
isOpenRouter && (model.id.startsWith("anthropic/") || model.id.startsWith("openai/"));
const cacheControlFormat = provider === "openrouter" && model.id.startsWith("anthropic/") ? "anthropic" : undefined;
return {
supportsStore: !isNonStandard,
supportsDeveloperRole: isOpenRouterDeveloperRoleModel || (!isNonStandard && !isOpenRouter),
supportsReasoningEffort:
!isGrok && !isZai && !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia && !isAntLing,
supportsUsageInStreaming: true,
maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens",
requiresToolResultName: false,
requiresAssistantAfterToolResult: false,
requiresThinkingAsText: false,
requiresReasoningContentOnAssistantMessages: isDeepSeek,
thinkingFormat: isDeepSeek
? "deepseek"
: isZai
? "zai"
: isTogether
? "together"
: isAntLing
? "ant-ling"
: isOpenRouter
? "openrouter"
: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia,
...(cacheControlFormat ? { cacheControlFormat } : {}),
sendSessionAffinityHeaders: false,
supportsLongCacheRetention: !(
isTogether ||
isCloudflareWorkersAI ||
isCloudflareAiGateway ||
isNvidia ||
isAntLing
),
};
}
function isPlainEmptyObject(value: unknown): boolean {
return typeof value === "object" && value !== null && !Array.isArray(value) && Object.keys(value).length === 0;
}
function openAICompletionsCompatDelta(compat: OpenAICompletionsResolvedCompat): OpenAICompletionsCompat {
const delta: OpenAICompletionsCompat = {};
for (const [key, value] of Object.entries(compat)) {
const defaultValue = OPENAI_COMPLETIONS_DEFAULT_COMPAT[key as keyof typeof OPENAI_COMPLETIONS_DEFAULT_COMPAT];
if (isPlainEmptyObject(value) && isPlainEmptyObject(defaultValue)) continue;
if (value !== defaultValue) {
(delta as Record<string, unknown>)[key] = value;
}
}
return delta;
}
function mergeOpenAICompletionsCompat(model: Model<Api>, compat: OpenAICompletionsCompat): void {
model.compat = { ...(model.compat as OpenAICompletionsCompat | undefined), ...compat };
}
function applyOpenAICompletionsCompatMetadata(model: Model<Api>): void {
if (model.api !== "openai-completions") return;
const detected = openAICompletionsCompatDelta(detectOpenAICompletionsCompat(model as Model<"openai-completions">));
model.compat = { ...detected, ...(model.compat as OpenAICompletionsCompat | undefined) };
if (Object.keys(model.compat).length === 0) {
delete model.compat;
}
}
function isGemini3ProModel(modelId: string): boolean { function isGemini3ProModel(modelId: string): boolean {
return /gemini-3(?:\.\d+)?-pro/.test(modelId.toLowerCase()); return /gemini-3(?:\.\d+)?-pro/.test(modelId.toLowerCase());
} }
@@ -854,9 +992,10 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
continue; continue;
} }
// workers-ai/* through the gateway forwards x-session-affinity to // Gateway passthroughs forward session affinity headers to upstreams that
// the underlying Workers AI runtime for prefix-cache routing. // use them for cache/routing affinity.
const compat = upstream === "workers-ai" ? { sendSessionAffinityHeaders: true } : undefined; const compat =
upstream === "anthropic" || upstream === "workers-ai" ? { sendSessionAffinityHeaders: true } : undefined;
models.push({ models.push({
id, id,
@@ -2096,6 +2235,7 @@ async function generateModels() {
for (const model of allModels) { for (const model of allModels) {
applyThinkingLevelMetadata(model); applyThinkingLevelMetadata(model);
applyOpenAICompletionsCompatMetadata(model);
} }
// Group by provider and deduplicate by model ID // Group by provider and deduplicate by model ID
-98
View File
@@ -1,98 +0,0 @@
import type {
Api,
AssistantMessageEventStream,
Context,
Model,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "./types.ts";
export type ApiStreamFunction = (
model: Model<Api>,
context: Context,
options?: StreamOptions,
) => AssistantMessageEventStream;
export type ApiStreamSimpleFunction = (
model: Model<Api>,
context: Context,
options?: SimpleStreamOptions,
) => AssistantMessageEventStream;
export interface ApiProvider<TApi extends Api = Api, TOptions extends StreamOptions = StreamOptions> {
api: TApi;
stream: StreamFunction<TApi, TOptions>;
streamSimple: StreamFunction<TApi, SimpleStreamOptions>;
}
interface ApiProviderInternal {
api: Api;
stream: ApiStreamFunction;
streamSimple: ApiStreamSimpleFunction;
}
type RegisteredApiProvider = {
provider: ApiProviderInternal;
sourceId?: string;
};
const apiProviderRegistry = new Map<string, RegisteredApiProvider>();
function wrapStream<TApi extends Api, TOptions extends StreamOptions>(
api: TApi,
stream: StreamFunction<TApi, TOptions>,
): ApiStreamFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return stream(model as Model<TApi>, context, options as TOptions);
};
}
function wrapStreamSimple<TApi extends Api>(
api: TApi,
streamSimple: StreamFunction<TApi, SimpleStreamOptions>,
): ApiStreamSimpleFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return streamSimple(model as Model<TApi>, context, options);
};
}
export function registerApiProvider<TApi extends Api, TOptions extends StreamOptions>(
provider: ApiProvider<TApi, TOptions>,
sourceId?: string,
): void {
apiProviderRegistry.set(provider.api, {
provider: {
api: provider.api,
stream: wrapStream(provider.api, provider.stream),
streamSimple: wrapStreamSimple(provider.api, provider.streamSimple),
},
sourceId,
});
}
export function getApiProvider(api: Api): ApiProviderInternal | undefined {
return apiProviderRegistry.get(api)?.provider;
}
export function getApiProviders(): ApiProviderInternal[] {
return Array.from(apiProviderRegistry.values(), (entry) => entry.provider);
}
export function unregisterApiProviders(sourceId: string): void {
for (const [api, entry] of apiProviderRegistry.entries()) {
if (entry.sourceId === sourceId) {
apiProviderRegistry.delete(api);
}
}
}
export function clearApiProviders(): void {
apiProviderRegistry.clear();
}
+49 -62
View File
@@ -18,6 +18,7 @@ import type {
Message, Message,
Model, Model,
ProviderEnv, ProviderEnv,
ProviderHeaders,
SimpleStreamOptions, SimpleStreamOptions,
StopReason, StopReason,
StreamFunction, StreamFunction,
@@ -34,7 +35,6 @@ import { parseJsonWithRepair, parseStreamingJson } from "../utils/json-parse.ts"
import { getProviderEnvValue } from "../utils/provider-env.ts"; import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts"; import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts"; import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { adjustMaxTokensForThinking, buildBaseOptions } from "./simple-options.ts"; import { adjustMaxTokensForThinking, buildBaseOptions } from "./simple-options.ts";
import { transformMessages } from "./transform-messages.ts"; import { transformMessages } from "./transform-messages.ts";
@@ -170,16 +170,11 @@ const INTERLEAVED_THINKING_BETA = "interleaved-thinking-2025-05-14";
function getAnthropicCompat( function getAnthropicCompat(
model: Model<"anthropic-messages">, model: Model<"anthropic-messages">,
): Required<Omit<AnthropicMessagesCompat, "forceAdaptiveThinking">> { ): Required<Omit<AnthropicMessagesCompat, "forceAdaptiveThinking">> {
// Auto-detect session affinity and cache control support from provider
const isFireworks = model.provider === "fireworks";
const isCloudflareAiGatewayAnthropic =
model.provider === "cloudflare-ai-gateway" && model.baseUrl.includes("anthropic");
return { return {
supportsEagerToolInputStreaming: model.compat?.supportsEagerToolInputStreaming ?? !isFireworks, supportsEagerToolInputStreaming: model.compat?.supportsEagerToolInputStreaming ?? true,
supportsLongCacheRetention: model.compat?.supportsLongCacheRetention ?? !isFireworks, supportsLongCacheRetention: model.compat?.supportsLongCacheRetention ?? true,
sendSessionAffinityHeaders: sendSessionAffinityHeaders: model.compat?.sendSessionAffinityHeaders ?? false,
model.compat?.sendSessionAffinityHeaders ?? !!(isFireworks || isCloudflareAiGatewayAnthropic), supportsCacheControlOnTools: model.compat?.supportsCacheControlOnTools ?? true,
supportsCacheControlOnTools: model.compat?.supportsCacheControlOnTools ?? !isFireworks,
supportsTemperature: model.compat?.supportsTemperature ?? true, supportsTemperature: model.compat?.supportsTemperature ?? true,
allowEmptySignature: model.compat?.allowEmptySignature ?? false, allowEmptySignature: model.compat?.allowEmptySignature ?? false,
}; };
@@ -247,8 +242,8 @@ export interface AnthropicOptions extends StreamOptions {
client?: Anthropic; client?: Anthropic;
} }
function mergeHeaders(...headerSources: (Record<string, string | null> | undefined)[]): Record<string, string | null> { function mergeHeaders(...headerSources: (ProviderHeaders | undefined)[]): ProviderHeaders {
const merged: Record<string, string | null> = {}; const merged: ProviderHeaders = {};
for (const headers of headerSources) { for (const headers of headerSources) {
if (headers) { if (headers) {
Object.assign(merged, headers); Object.assign(merged, headers);
@@ -257,6 +252,27 @@ function mergeHeaders(...headerSources: (Record<string, string | null> | undefin
return merged; return merged;
} }
function hasHeader(headers: ProviderHeaders | undefined, name: string): boolean {
if (!headers) return false;
const expected = name.toLowerCase();
for (const [key, value] of Object.entries(headers)) {
if (key.toLowerCase() === expected && value !== null && value.trim().length > 0) return true;
}
return false;
}
function assertRequestAuth(provider: string, apiKey: string | undefined, headers: ProviderHeaders | undefined): void {
if (apiKey) return;
if (
hasHeader(headers, "authorization") ||
hasHeader(headers, "x-api-key") ||
hasHeader(headers, "cf-aig-authorization")
) {
return;
}
throw new Error(`No API key for provider: ${provider}`);
}
interface ServerSentEvent { interface ServerSentEvent {
event: string | null; event: string | null;
data: string; data: string;
@@ -484,9 +500,7 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
isOAuth = false; isOAuth = false;
} else { } else {
const apiKey = options?.apiKey; const apiKey = options?.apiKey;
if (!apiKey) { assertRequestAuth(model.provider, apiKey, options?.headers);
throw new Error(`No API key for provider: ${model.provider}`);
}
let copilotDynamicHeaders: Record<string, string> | undefined; let copilotDynamicHeaders: Record<string, string> | undefined;
if (model.provider === "github-copilot") { if (model.provider === "github-copilot") {
@@ -508,7 +522,6 @@ export const stream: StreamFunction<"anthropic-messages", AnthropicOptions> = (
options?.headers, options?.headers,
copilotDynamicHeaders, copilotDynamicHeaders,
cacheSessionId, cacheSessionId,
options?.env,
); );
client = created.client; client = created.client;
isOAuth = created.isOAuthToken; isOAuth = created.isOAuthToken;
@@ -748,12 +761,9 @@ export const streamSimple: StreamFunction<"anthropic-messages", SimpleStreamOpti
context: Context, context: Context,
options?: SimpleStreamOptions, options?: SimpleStreamOptions,
): AssistantMessageEventStream => { ): AssistantMessageEventStream => {
const apiKey = options?.apiKey; assertRequestAuth(model.provider, options?.apiKey, options?.headers);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const base = buildBaseOptions(model, options, apiKey); const base = buildBaseOptions(model, options, options?.apiKey);
if (!options?.reasoning) { if (!options?.reasoning) {
return stream(model, context, { ...base, thinkingEnabled: false } satisfies AnthropicOptions); return stream(model, context, { ...base, thinkingEnabled: false } satisfies AnthropicOptions);
} }
@@ -792,13 +802,12 @@ function isOAuthToken(apiKey: string): boolean {
function createClient( function createClient(
model: Model<"anthropic-messages">, model: Model<"anthropic-messages">,
apiKey: string, apiKey: string | undefined,
interleavedThinking: boolean, interleavedThinking: boolean,
useFineGrainedToolStreamingBeta: boolean, useFineGrainedToolStreamingBeta: boolean,
optionsHeaders?: Record<string, string>, optionsHeaders?: ProviderHeaders,
dynamicHeaders?: Record<string, string>, dynamicHeaders?: Record<string, string>,
sessionId?: string, sessionId?: string,
env?: ProviderEnv,
): { client: Anthropic; isOAuthToken: boolean } { ): { client: Anthropic; isOAuthToken: boolean } {
// Adaptive thinking models have interleaved thinking built in, so skip the beta header. // Adaptive thinking models have interleaved thinking built in, so skip the beta header.
const needsInterleavedBeta = interleavedThinking && model.compat?.forceAdaptiveThinking !== true; const needsInterleavedBeta = interleavedThinking && model.compat?.forceAdaptiveThinking !== true;
@@ -810,34 +819,11 @@ function createClient(
betaFeatures.push(INTERLEAVED_THINKING_BETA); betaFeatures.push(INTERLEAVED_THINKING_BETA);
} }
if (model.provider === "cloudflare-ai-gateway") {
const client = new Anthropic({
apiKey: null,
authToken: null,
baseURL: resolveCloudflareBaseUrl(model, env),
dangerouslyAllowBrowser: true,
defaultHeaders: mergeHeaders(
{
accept: "application/json",
"anthropic-dangerous-direct-browser-access": "true",
"cf-aig-authorization": `Bearer ${apiKey}`,
"x-api-key": null,
Authorization: null,
...(betaFeatures.length > 0 ? { "anthropic-beta": betaFeatures.join(",") } : {}),
},
model.headers,
optionsHeaders,
),
});
return { client, isOAuthToken: false };
}
// Copilot: Bearer auth, selective betas. // Copilot: Bearer auth, selective betas.
if (model.provider === "github-copilot") { if (model.provider === "github-copilot") {
const client = new Anthropic({ const client = new Anthropic({
apiKey: null, apiKey: null,
authToken: apiKey, authToken: apiKey ?? null,
baseURL: model.baseUrl, baseURL: model.baseUrl,
dangerouslyAllowBrowser: true, dangerouslyAllowBrowser: true,
defaultHeaders: mergeHeaders( defaultHeaders: mergeHeaders(
@@ -856,7 +842,7 @@ function createClient(
} }
// OAuth: Bearer auth, Claude Code identity headers // OAuth: Bearer auth, Claude Code identity headers
if (isOAuthToken(apiKey)) { if (apiKey && isOAuthToken(apiKey)) {
const client = new Anthropic({ const client = new Anthropic({
apiKey: null, apiKey: null,
authToken: apiKey, authToken: apiKey,
@@ -878,24 +864,25 @@ function createClient(
return { client, isOAuthToken: true }; return { client, isOAuthToken: true };
} }
// API key auth // API key or header-owned auth.
const sessionAffinityHeaders: Record<string, string | null> = const sessionAffinityHeaders: ProviderHeaders =
sessionId && getAnthropicCompat(model).sendSessionAffinityHeaders ? { "x-session-affinity": sessionId } : {}; sessionId && getAnthropicCompat(model).sendSessionAffinityHeaders ? { "x-session-affinity": sessionId } : {};
const defaultHeaders = mergeHeaders(
{
accept: "application/json",
"anthropic-dangerous-direct-browser-access": "true",
...(betaFeatures.length > 0 ? { "anthropic-beta": betaFeatures.join(",") } : {}),
},
sessionAffinityHeaders,
model.headers,
optionsHeaders,
);
const client = new Anthropic({ const client = new Anthropic({
apiKey, apiKey: apiKey ?? null,
authToken: null, authToken: null,
baseURL: model.baseUrl, baseURL: model.baseUrl,
dangerouslyAllowBrowser: true, dangerouslyAllowBrowser: true,
defaultHeaders: mergeHeaders( defaultHeaders,
{
accept: "application/json",
"anthropic-dangerous-direct-browser-access": "true",
...(betaFeatures.length > 0 ? { "anthropic-beta": betaFeatures.join(",") } : {}),
},
sessionAffinityHeaders,
model.headers,
optionsHeaders,
),
}); });
return { client, isOAuthToken: false }; return { client, isOAuthToken: false };
@@ -48,6 +48,7 @@ import type {
ToolResultMessage, ToolResultMessage,
} from "../types.ts"; } from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts"; import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { providerHeadersToRecord } from "../utils/headers.ts";
import { parseStreamingJson } from "../utils/json-parse.ts"; import { parseStreamingJson } from "../utils/json-parse.ts";
import { resolveHttpProxyUrlForTarget } from "../utils/node-http-proxy.ts"; import { resolveHttpProxyUrlForTarget } from "../utils/node-http-proxy.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts"; import { getProviderEnvValue } from "../utils/provider-env.ts";
@@ -204,8 +205,9 @@ export const stream: StreamFunction<"bedrock-converse-stream", BedrockOptions> =
try { try {
const client = new BedrockRuntimeClient(config); const client = new BedrockRuntimeClient(config);
if (options.headers && Object.keys(options.headers).length > 0) { const customHeaders = providerHeadersToRecord(options.headers);
addCustomHeadersMiddleware(client, options.headers); if (customHeaders) {
addCustomHeadersMiddleware(client, customHeaders);
} }
const cacheRetention = resolveCacheRetention(options.cacheRetention, options.env); const cacheRetention = resolveCacheRetention(options.cacheRetention, options.env);
const inferenceMaxTokens = options.maxTokens ?? (isAnthropicClaudeModel(model) ? model.maxTokens : undefined); const inferenceMaxTokens = options.maxTokens ?? (isAnthropicClaudeModel(model) ? model.maxTokens : undefined);
-21
View File
@@ -1,6 +1,3 @@
import type { Api, Model, ProviderEnv } from "../types.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts";
/** Workers AI direct endpoint. */ /** Workers AI direct endpoint. */
export const CLOUDFLARE_WORKERS_AI_BASE_URL = export const CLOUDFLARE_WORKERS_AI_BASE_URL =
"https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1"; "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1";
@@ -16,21 +13,3 @@ export const CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL =
/** AI Gateway → Anthropic passthrough. */ /** AI Gateway → Anthropic passthrough. */
export const CLOUDFLARE_AI_GATEWAY_ANTHROPIC_BASE_URL = export const CLOUDFLARE_AI_GATEWAY_ANTHROPIC_BASE_URL =
"https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic"; "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic";
export function isCloudflareProvider(provider: string): boolean {
return provider === "cloudflare-workers-ai" || provider === "cloudflare-ai-gateway";
}
/** Substitute `{VAR}` placeholders in a Cloudflare baseUrl from provider env or process.env. */
export function resolveCloudflareBaseUrl(model: Model<Api>, env?: ProviderEnv): string {
const url = model.baseUrl;
if (!url.includes("{")) return url;
const baseUrl = url.replace(/\{([A-Z_][A-Z0-9_]*)\}/g, (_match, name: string) => {
const value = getProviderEnvValue(name, env);
if (!value) {
throw new Error(`${name} is required for provider ${model.provider} but is not set.`);
}
return value;
});
return baseUrl;
}
+6 -3
View File
@@ -10,6 +10,7 @@ import type {
AssistantMessage, AssistantMessage,
Context, Context,
Model, Model,
ProviderHeaders,
SimpleStreamOptions, SimpleStreamOptions,
StreamFunction, StreamFunction,
StreamOptions, StreamOptions,
@@ -20,6 +21,7 @@ import type {
ToolCall, ToolCall,
} from "../types.ts"; } from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts"; import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { providerHeadersToRecord } from "../utils/headers.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts"; import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import type { GoogleThinkingLevel } from "./google-shared.ts"; import type { GoogleThinkingLevel } from "./google-shared.ts";
import { import {
@@ -318,15 +320,16 @@ export const streamSimple: StreamFunction<"google-generative-ai", SimpleStreamOp
function createClient( function createClient(
model: Model<"google-generative-ai">, model: Model<"google-generative-ai">,
apiKey?: string, apiKey?: string,
optionsHeaders?: Record<string, string>, optionsHeaders?: ProviderHeaders,
): GoogleGenAI { ): GoogleGenAI {
const httpOptions: { baseUrl?: string; apiVersion?: string; headers?: Record<string, string> } = {}; const httpOptions: { baseUrl?: string; apiVersion?: string; headers?: Record<string, string> } = {};
if (model.baseUrl) { if (model.baseUrl) {
httpOptions.baseUrl = model.baseUrl; httpOptions.baseUrl = model.baseUrl;
httpOptions.apiVersion = ""; // baseUrl already includes version path, don't append httpOptions.apiVersion = ""; // baseUrl already includes version path, don't append
} }
if (model.headers || optionsHeaders) { const headers = providerHeadersToRecord({ ...model.headers, ...optionsHeaders });
httpOptions.headers = { ...model.headers, ...optionsHeaders }; if (headers) {
httpOptions.headers = headers;
} }
return new GoogleGenAI({ return new GoogleGenAI({
+8 -8
View File
@@ -15,6 +15,7 @@ import type {
Model, Model,
ThinkingLevel as PiThinkingLevel, ThinkingLevel as PiThinkingLevel,
ProviderEnv, ProviderEnv,
ProviderHeaders,
SimpleStreamOptions, SimpleStreamOptions,
StreamFunction, StreamFunction,
StreamOptions, StreamOptions,
@@ -24,6 +25,7 @@ import type {
ToolCall, ToolCall,
} from "../types.ts"; } from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts"; import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { providerHeadersToRecord } from "../utils/headers.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts"; import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts"; import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import type { GoogleThinkingLevel } from "./google-shared.ts"; import type { GoogleThinkingLevel } from "./google-shared.ts";
@@ -334,7 +336,7 @@ function createClient(
model: Model<"google-vertex">, model: Model<"google-vertex">,
project: string, project: string,
location: string, location: string,
optionsHeaders?: Record<string, string>, optionsHeaders?: ProviderHeaders,
env?: ProviderEnv, env?: ProviderEnv,
): GoogleGenAI { ): GoogleGenAI {
const googleAuthOptions = buildGoogleAuthOptions(env); const googleAuthOptions = buildGoogleAuthOptions(env);
@@ -351,7 +353,7 @@ function createClient(
function createClientWithApiKey( function createClientWithApiKey(
model: Model<"google-vertex">, model: Model<"google-vertex">,
apiKey: string, apiKey: string,
optionsHeaders?: Record<string, string>, optionsHeaders?: ProviderHeaders,
): GoogleGenAI { ): GoogleGenAI {
return new GoogleGenAI({ return new GoogleGenAI({
vertexai: true, vertexai: true,
@@ -361,10 +363,7 @@ function createClientWithApiKey(
}); });
} }
function buildHttpOptions( function buildHttpOptions(model: Model<"google-vertex">, optionsHeaders?: ProviderHeaders): HttpOptions | undefined {
model: Model<"google-vertex">,
optionsHeaders?: Record<string, string>,
): HttpOptions | undefined {
const httpOptions: HttpOptions = {}; const httpOptions: HttpOptions = {};
const baseUrl = resolveCustomBaseUrl(model.baseUrl); const baseUrl = resolveCustomBaseUrl(model.baseUrl);
if (baseUrl) { if (baseUrl) {
@@ -375,8 +374,9 @@ function buildHttpOptions(
} }
} }
if (model.headers || optionsHeaders) { const headers = providerHeadersToRecord({ ...model.headers, ...optionsHeaders });
httpOptions.headers = { ...model.headers, ...optionsHeaders }; if (headers) {
httpOptions.headers = headers;
} }
return Object.keys(httpOptions).length > 0 ? httpOptions : undefined; return Object.keys(httpOptions).length > 0 ? httpOptions : undefined;
@@ -28,6 +28,7 @@ import type {
Context, Context,
Model, Model,
ProviderEnv, ProviderEnv,
ProviderHeaders,
SimpleStreamOptions, SimpleStreamOptions,
StreamFunction, StreamFunction,
StreamOptions, StreamOptions,
@@ -1467,13 +1468,17 @@ function createCodexRequestId(): string {
function buildBaseCodexHeaders( function buildBaseCodexHeaders(
initHeaders: Record<string, string> | undefined, initHeaders: Record<string, string> | undefined,
additionalHeaders: Record<string, string> | undefined, additionalHeaders: ProviderHeaders | undefined,
accountId: string, accountId: string,
token: string, token: string,
): Headers { ): Headers {
const headers = new Headers(initHeaders); const headers = new Headers(initHeaders);
for (const [key, value] of Object.entries(additionalHeaders || {})) { for (const [key, value] of Object.entries(additionalHeaders || {})) {
headers.set(key, value); if (value === null) {
headers.delete(key);
} else {
headers.set(key, value);
}
} }
headers.set("Authorization", `Bearer ${token}`); headers.set("Authorization", `Bearer ${token}`);
headers.set("chatgpt-account-id", accountId); headers.set("chatgpt-account-id", accountId);
@@ -1485,7 +1490,7 @@ function buildBaseCodexHeaders(
function buildSSEHeaders( function buildSSEHeaders(
initHeaders: Record<string, string> | undefined, initHeaders: Record<string, string> | undefined,
additionalHeaders: Record<string, string> | undefined, additionalHeaders: ProviderHeaders | undefined,
accountId: string, accountId: string,
token: string, token: string,
sessionId?: string, sessionId?: string,
@@ -1505,7 +1510,7 @@ function buildSSEHeaders(
function buildWebSocketHeaders( function buildWebSocketHeaders(
initHeaders: Record<string, string> | undefined, initHeaders: Record<string, string> | undefined,
additionalHeaders: Record<string, string> | undefined, additionalHeaders: ProviderHeaders | undefined,
accountId: string, accountId: string,
token: string, token: string,
requestId: string, requestId: string,
+65 -134
View File
@@ -21,6 +21,7 @@ import type {
Model, Model,
OpenAICompletionsCompat, OpenAICompletionsCompat,
ProviderEnv, ProviderEnv,
ProviderHeaders,
SimpleStreamOptions, SimpleStreamOptions,
StopReason, StopReason,
StreamFunction, StreamFunction,
@@ -36,7 +37,6 @@ import { headersToRecord } from "../utils/headers.ts";
import { parseStreamingJson } from "../utils/json-parse.ts"; import { parseStreamingJson } from "../utils/json-parse.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts"; import { getProviderEnvValue } from "../utils/provider-env.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts"; import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts"; import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts"; import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { buildBaseOptions } from "./simple-options.ts"; import { buildBaseOptions } from "./simple-options.ts";
@@ -47,6 +47,21 @@ import { transformMessages } from "./transform-messages.ts";
* This is needed because Anthropic (via proxy) requires the tools param * This is needed because Anthropic (via proxy) requires the tools param
* to be present when messages include tool_calls or tool role messages. * to be present when messages include tool_calls or tool role messages.
*/ */
function hasHeader(headers: ProviderHeaders | undefined, name: string): boolean {
if (!headers) return false;
const expected = name.toLowerCase();
for (const [key, value] of Object.entries(headers)) {
if (key.toLowerCase() === expected && value !== null && value.trim().length > 0) return true;
}
return false;
}
function getClientApiKey(provider: string, apiKey: string | undefined, headers: ProviderHeaders | undefined): string {
if (apiKey) return apiKey;
if (hasHeader(headers, "authorization") || hasHeader(headers, "cf-aig-authorization")) return "unused";
throw new Error(`No API key for provider: ${provider}`);
}
function hasToolHistory(messages: Message[]): boolean { function hasToolHistory(messages: Message[]): boolean {
for (const msg of messages) { for (const msg of messages) {
if (msg.role === "toolResult") { if (msg.role === "toolResult") {
@@ -160,14 +175,11 @@ export const stream: StreamFunction<"openai-completions", OpenAICompletionsOptio
}; };
try { try {
const apiKey = options?.apiKey; const apiKey = getClientApiKey(model.provider, options?.apiKey, options?.headers);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const compat = getCompat(model); const compat = getCompat(model);
const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env); const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId; const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, compat, options?.env); const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, compat);
let params = buildParams(model, context, options, compat, cacheRetention); let params = buildParams(model, context, options, compat, cacheRetention);
const nextParams = await options?.onPayload?.(params, model); const nextParams = await options?.onPayload?.(params, model);
if (nextParams !== undefined) { if (nextParams !== undefined) {
@@ -468,12 +480,9 @@ export const streamSimple: StreamFunction<"openai-completions", SimpleStreamOpti
context: Context, context: Context,
options?: SimpleStreamOptions, options?: SimpleStreamOptions,
): AssistantMessageEventStream => { ): AssistantMessageEventStream => {
const apiKey = options?.apiKey; getClientApiKey(model.provider, options?.apiKey, options?.headers);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const base = buildBaseOptions(model, options, apiKey); const base = buildBaseOptions(model, options, options?.apiKey);
const clampedReasoning = options?.reasoning ? clampThinkingLevel(model, options.reasoning) : undefined; const clampedReasoning = options?.reasoning ? clampThinkingLevel(model, options.reasoning) : undefined;
const reasoningEffort = clampedReasoning === "off" ? undefined : clampedReasoning; const reasoningEffort = clampedReasoning === "off" ? undefined : clampedReasoning;
const toolChoice = (options as OpenAICompletionsOptions | undefined)?.toolChoice; const toolChoice = (options as OpenAICompletionsOptions | undefined)?.toolChoice;
@@ -489,12 +498,11 @@ function createClient(
model: Model<"openai-completions">, model: Model<"openai-completions">,
context: Context, context: Context,
apiKey: string, apiKey: string,
optionsHeaders?: Record<string, string>, optionsHeaders?: ProviderHeaders,
sessionId?: string, sessionId?: string,
compat: ResolvedOpenAICompletionsCompat = getCompat(model), compat: ResolvedOpenAICompletionsCompat = getCompat(model),
env?: ProviderEnv,
) { ) {
const headers = { ...model.headers }; const headers: ProviderHeaders = { ...model.headers };
if (model.provider === "github-copilot") { if (model.provider === "github-copilot") {
const hasImages = hasCopilotVisionInput(context.messages); const hasImages = hasCopilotVisionInput(context.messages);
const copilotHeaders = buildCopilotDynamicHeaders({ const copilotHeaders = buildCopilotDynamicHeaders({
@@ -515,20 +523,11 @@ function createClient(
Object.assign(headers, optionsHeaders); Object.assign(headers, optionsHeaders);
} }
const defaultHeaders =
model.provider === "cloudflare-ai-gateway"
? {
...headers,
Authorization: headers.Authorization ?? null,
"cf-aig-authorization": `Bearer ${apiKey}`,
}
: headers;
return new OpenAI({ return new OpenAI({
apiKey, apiKey,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model, env) : model.baseUrl, baseURL: model.baseUrl,
dangerouslyAllowBrowser: true, dangerouslyAllowBrowser: true,
defaultHeaders, defaultHeaders: headers,
}); });
} }
@@ -674,7 +673,7 @@ function buildParams(
} }
// Vercel AI Gateway provider routing preferences // Vercel AI Gateway provider routing preferences
if (model.baseUrl.includes("ai-gateway.vercel.sh") && model.compat?.vercelGatewayRouting) { if (model.compat?.vercelGatewayRouting) {
const routing = model.compat.vercelGatewayRouting; const routing = model.compat.vercelGatewayRouting;
if (routing.only || routing.order) { if (routing.only || routing.order) {
const gatewayOptions: Record<string, string[]> = {}; const gatewayOptions: Record<string, string[]> = {};
@@ -1164,123 +1163,55 @@ function mapStopReason(reason: ChatCompletionChunk.Choice["finish_reason"] | str
} }
} }
/** const DEFAULT_COMPAT: ResolvedOpenAICompletionsCompat = {
* Detect compatibility settings from provider and baseUrl for known providers. supportsStore: true,
* Provider takes precedence over URL-based detection since it's explicitly configured. supportsDeveloperRole: true,
* Returns a fully resolved OpenAICompletionsCompat object with all fields set. supportsReasoningEffort: true,
*/ supportsUsageInStreaming: true,
function detectCompat(model: Model<"openai-completions">): ResolvedOpenAICompletionsCompat { maxTokensField: "max_completion_tokens",
const provider = model.provider; requiresToolResultName: false,
const baseUrl = model.baseUrl; requiresAssistantAfterToolResult: false,
requiresThinkingAsText: false,
const isZai = requiresReasoningContentOnAssistantMessages: false,
provider === "zai" || thinkingFormat: "openai",
provider === "zai-coding-cn" || openRouterRouting: {},
baseUrl.includes("api.z.ai") || vercelGatewayRouting: {},
baseUrl.includes("open.bigmodel.cn"); chatTemplateKwargs: {},
const isTogether = zaiToolStream: false,
provider === "together" || baseUrl.includes("api.together.ai") || baseUrl.includes("api.together.xyz"); supportsStrictMode: true,
const isMoonshot = provider === "moonshotai" || provider === "moonshotai-cn" || baseUrl.includes("api.moonshot."); cacheControlFormat: undefined,
const isOpenRouter = provider === "openrouter" || baseUrl.includes("openrouter.ai"); sendSessionAffinityHeaders: false,
const isCloudflareWorkersAI = provider === "cloudflare-workers-ai" || baseUrl.includes("api.cloudflare.com"); supportsLongCacheRetention: true,
const isCloudflareAiGateway = provider === "cloudflare-ai-gateway" || baseUrl.includes("gateway.ai.cloudflare.com"); };
const isNvidia = provider === "nvidia" || baseUrl.includes("integrate.api.nvidia.com");
const isAntLing = provider === "ant-ling" || baseUrl.includes("api.ant-ling.com");
const isNonStandard =
isNvidia ||
provider === "cerebras" ||
baseUrl.includes("cerebras.ai") ||
provider === "xai" ||
baseUrl.includes("api.x.ai") ||
isTogether ||
baseUrl.includes("chutes.ai") ||
baseUrl.includes("deepseek.com") ||
isZai ||
isMoonshot ||
provider === "opencode" ||
baseUrl.includes("opencode.ai") ||
isCloudflareWorkersAI ||
isCloudflareAiGateway ||
isAntLing;
const useMaxTokens =
baseUrl.includes("chutes.ai") || isMoonshot || isCloudflareAiGateway || isTogether || isNvidia || isAntLing;
const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
const isDeepSeek = provider === "deepseek" || baseUrl.includes("deepseek.com");
const isOpenRouterDeveloperRoleModel =
isOpenRouter && (model.id.startsWith("anthropic/") || model.id.startsWith("openai/"));
const cacheControlFormat = provider === "openrouter" && model.id.startsWith("anthropic/") ? "anthropic" : undefined;
return {
supportsStore: !isNonStandard,
supportsDeveloperRole: isOpenRouterDeveloperRoleModel || (!isNonStandard && !isOpenRouter),
supportsReasoningEffort:
!isGrok && !isZai && !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia && !isAntLing,
supportsUsageInStreaming: true,
maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens",
requiresToolResultName: false,
requiresAssistantAfterToolResult: false,
requiresThinkingAsText: false,
requiresReasoningContentOnAssistantMessages: isDeepSeek,
thinkingFormat: isDeepSeek
? "deepseek"
: isZai
? "zai"
: isTogether
? "together"
: isAntLing
? "ant-ling"
: isOpenRouter
? "openrouter"
: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
chatTemplateKwargs: {},
zaiToolStream: false,
supportsStrictMode: !isMoonshot && !isTogether && !isCloudflareAiGateway && !isNvidia,
cacheControlFormat,
sendSessionAffinityHeaders: false,
supportsLongCacheRetention: !(
isTogether ||
isCloudflareWorkersAI ||
isCloudflareAiGateway ||
isNvidia ||
isAntLing
),
};
}
/** /**
* Get resolved compatibility settings for a model. * Get resolved compatibility settings for a model.
* Uses explicit model.compat if provided, otherwise auto-detects from provider/URL. * Uses explicit generated/custom model.compat over OpenAI-standard defaults.
*/ */
function getCompat(model: Model<"openai-completions">): ResolvedOpenAICompletionsCompat { function getCompat(model: Model<"openai-completions">): ResolvedOpenAICompletionsCompat {
const detected = detectCompat(model); if (!model.compat) return DEFAULT_COMPAT;
if (!model.compat) return detected;
return { return {
supportsStore: model.compat.supportsStore ?? detected.supportsStore, supportsStore: model.compat.supportsStore ?? DEFAULT_COMPAT.supportsStore,
supportsDeveloperRole: model.compat.supportsDeveloperRole ?? detected.supportsDeveloperRole, supportsDeveloperRole: model.compat.supportsDeveloperRole ?? DEFAULT_COMPAT.supportsDeveloperRole,
supportsReasoningEffort: model.compat.supportsReasoningEffort ?? detected.supportsReasoningEffort, supportsReasoningEffort: model.compat.supportsReasoningEffort ?? DEFAULT_COMPAT.supportsReasoningEffort,
supportsUsageInStreaming: model.compat.supportsUsageInStreaming ?? detected.supportsUsageInStreaming, supportsUsageInStreaming: model.compat.supportsUsageInStreaming ?? DEFAULT_COMPAT.supportsUsageInStreaming,
maxTokensField: model.compat.maxTokensField ?? detected.maxTokensField, maxTokensField: model.compat.maxTokensField ?? DEFAULT_COMPAT.maxTokensField,
requiresToolResultName: model.compat.requiresToolResultName ?? detected.requiresToolResultName, requiresToolResultName: model.compat.requiresToolResultName ?? DEFAULT_COMPAT.requiresToolResultName,
requiresAssistantAfterToolResult: requiresAssistantAfterToolResult:
model.compat.requiresAssistantAfterToolResult ?? detected.requiresAssistantAfterToolResult, model.compat.requiresAssistantAfterToolResult ?? DEFAULT_COMPAT.requiresAssistantAfterToolResult,
requiresThinkingAsText: model.compat.requiresThinkingAsText ?? detected.requiresThinkingAsText, requiresThinkingAsText: model.compat.requiresThinkingAsText ?? DEFAULT_COMPAT.requiresThinkingAsText,
requiresReasoningContentOnAssistantMessages: requiresReasoningContentOnAssistantMessages:
model.compat.requiresReasoningContentOnAssistantMessages ?? model.compat.requiresReasoningContentOnAssistantMessages ??
detected.requiresReasoningContentOnAssistantMessages, DEFAULT_COMPAT.requiresReasoningContentOnAssistantMessages,
thinkingFormat: model.compat.thinkingFormat ?? detected.thinkingFormat, thinkingFormat: model.compat.thinkingFormat ?? DEFAULT_COMPAT.thinkingFormat,
openRouterRouting: model.compat.openRouterRouting ?? {}, openRouterRouting: model.compat.openRouterRouting ?? DEFAULT_COMPAT.openRouterRouting,
vercelGatewayRouting: model.compat.vercelGatewayRouting ?? detected.vercelGatewayRouting, vercelGatewayRouting: model.compat.vercelGatewayRouting ?? DEFAULT_COMPAT.vercelGatewayRouting,
chatTemplateKwargs: model.compat.chatTemplateKwargs ?? detected.chatTemplateKwargs, chatTemplateKwargs: model.compat.chatTemplateKwargs ?? DEFAULT_COMPAT.chatTemplateKwargs,
zaiToolStream: model.compat.zaiToolStream ?? detected.zaiToolStream, zaiToolStream: model.compat.zaiToolStream ?? DEFAULT_COMPAT.zaiToolStream,
supportsStrictMode: model.compat.supportsStrictMode ?? detected.supportsStrictMode, supportsStrictMode: model.compat.supportsStrictMode ?? DEFAULT_COMPAT.supportsStrictMode,
cacheControlFormat: model.compat.cacheControlFormat ?? detected.cacheControlFormat, cacheControlFormat: model.compat.cacheControlFormat ?? DEFAULT_COMPAT.cacheControlFormat,
sendSessionAffinityHeaders: model.compat.sendSessionAffinityHeaders ?? detected.sendSessionAffinityHeaders, sendSessionAffinityHeaders: model.compat.sendSessionAffinityHeaders ?? DEFAULT_COMPAT.sendSessionAffinityHeaders,
supportsLongCacheRetention: model.compat.supportsLongCacheRetention ?? detected.supportsLongCacheRetention, supportsLongCacheRetention: model.compat.supportsLongCacheRetention ?? DEFAULT_COMPAT.supportsLongCacheRetention,
}; };
} }
+24 -25
View File
@@ -9,6 +9,7 @@ import type {
Model, Model,
OpenAIResponsesCompat, OpenAIResponsesCompat,
ProviderEnv, ProviderEnv,
ProviderHeaders,
SimpleStreamOptions, SimpleStreamOptions,
StreamFunction, StreamFunction,
StreamOptions, StreamOptions,
@@ -17,7 +18,6 @@ import type {
import { AssistantMessageEventStream } from "../utils/event-stream.ts"; import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts"; import { headersToRecord } from "../utils/headers.ts";
import { getProviderEnvValue } from "../utils/provider-env.ts"; import { getProviderEnvValue } from "../utils/provider-env.ts";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts"; import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts"; import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.ts"; import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.ts";
@@ -25,6 +25,21 @@ import { buildBaseOptions } from "./simple-options.ts";
const OPENAI_TOOL_CALL_PROVIDERS = new Set(["openai", "openai-codex", "opencode"]); const OPENAI_TOOL_CALL_PROVIDERS = new Set(["openai", "openai-codex", "opencode"]);
function hasHeader(headers: ProviderHeaders | undefined, name: string): boolean {
if (!headers) return false;
const expected = name.toLowerCase();
for (const [key, value] of Object.entries(headers)) {
if (key.toLowerCase() === expected && value !== null && value.trim().length > 0) return true;
}
return false;
}
function getClientApiKey(provider: string, apiKey: string | undefined, headers: ProviderHeaders | undefined): string {
if (apiKey) return apiKey;
if (hasHeader(headers, "authorization") || hasHeader(headers, "cf-aig-authorization")) return "unused";
throw new Error(`No API key for provider: ${provider}`);
}
/** /**
* Resolve cache retention preference. * Resolve cache retention preference.
* Defaults to "short" and uses PI_CACHE_RETENTION for backward compatibility. * Defaults to "short" and uses PI_CACHE_RETENTION for backward compatibility.
@@ -109,13 +124,10 @@ export const stream: StreamFunction<"openai-responses", OpenAIResponsesOptions>
try { try {
// Create OpenAI client // Create OpenAI client
const apiKey = options?.apiKey; const apiKey = getClientApiKey(model.provider, options?.apiKey, options?.headers);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env); const cacheRetention = resolveCacheRetention(options?.cacheRetention, options?.env);
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId; const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
const client = createClient(model, context, apiKey, options?.headers, cacheSessionId, options?.env); const client = createClient(model, context, apiKey, options?.headers, cacheSessionId);
let params = buildParams(model, context, options); let params = buildParams(model, context, options);
const nextParams = await options?.onPayload?.(params, model); const nextParams = await options?.onPayload?.(params, model);
if (nextParams !== undefined) { if (nextParams !== undefined) {
@@ -166,12 +178,9 @@ export const streamSimple: StreamFunction<"openai-responses", SimpleStreamOption
context: Context, context: Context,
options?: SimpleStreamOptions, options?: SimpleStreamOptions,
): AssistantMessageEventStream => { ): AssistantMessageEventStream => {
const apiKey = options?.apiKey; getClientApiKey(model.provider, options?.apiKey, options?.headers);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const base = buildBaseOptions(model, options, apiKey); const base = buildBaseOptions(model, options, options?.apiKey);
const clampedReasoning = options?.reasoning ? clampThinkingLevel(model, options.reasoning) : undefined; const clampedReasoning = options?.reasoning ? clampThinkingLevel(model, options.reasoning) : undefined;
const reasoningEffort = clampedReasoning === "off" ? undefined : clampedReasoning; const reasoningEffort = clampedReasoning === "off" ? undefined : clampedReasoning;
@@ -185,12 +194,11 @@ function createClient(
model: Model<"openai-responses">, model: Model<"openai-responses">,
context: Context, context: Context,
apiKey: string, apiKey: string,
optionsHeaders?: Record<string, string>, optionsHeaders?: ProviderHeaders,
sessionId?: string, sessionId?: string,
env?: ProviderEnv,
) { ) {
const compat = getCompat(model); const compat = getCompat(model);
const headers = { ...model.headers }; const headers: ProviderHeaders = { ...model.headers };
if (model.provider === "github-copilot") { if (model.provider === "github-copilot") {
const hasImages = hasCopilotVisionInput(context.messages); const hasImages = hasCopilotVisionInput(context.messages);
const copilotHeaders = buildCopilotDynamicHeaders({ const copilotHeaders = buildCopilotDynamicHeaders({
@@ -212,20 +220,11 @@ function createClient(
Object.assign(headers, optionsHeaders); Object.assign(headers, optionsHeaders);
} }
const defaultHeaders =
model.provider === "cloudflare-ai-gateway"
? {
...headers,
Authorization: headers.Authorization ?? null,
"cf-aig-authorization": `Bearer ${apiKey}`,
}
: headers;
return new OpenAI({ return new OpenAI({
apiKey, apiKey,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model, env) : model.baseUrl, baseURL: model.baseUrl,
dangerouslyAllowBrowser: true, dangerouslyAllowBrowser: true,
defaultHeaders, defaultHeaders: headers,
}); });
} }
+4 -6
View File
@@ -13,9 +13,10 @@ import type {
ImagesFunction, ImagesFunction,
ImagesModel, ImagesModel,
ImagesOptions, ImagesOptions,
ProviderHeaders,
TextContent, TextContent,
} from "../types.ts"; } from "../types.ts";
import { headersToRecord } from "../utils/headers.ts"; import { headersToRecord, providerHeadersToRecord } from "../utils/headers.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts"; import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
interface OpenRouterGeneratedImage { interface OpenRouterGeneratedImage {
@@ -106,16 +107,13 @@ export const generateImages: ImagesFunction<"openrouter-images", ImagesOptions>
function createClient( function createClient(
model: ImagesModel<"openrouter-images">, model: ImagesModel<"openrouter-images">,
apiKey: string, apiKey: string,
optionsHeaders?: Record<string, string>, optionsHeaders?: ProviderHeaders,
): OpenAI { ): OpenAI {
return new OpenAI({ return new OpenAI({
apiKey, apiKey,
baseURL: model.baseUrl, baseURL: model.baseUrl,
dangerouslyAllowBrowser: true, dangerouslyAllowBrowser: true,
defaultHeaders: { defaultHeaders: providerHeadersToRecord({ ...model.headers, ...optionsHeaders }),
...model.headers,
...optionsHeaders,
},
}); });
} }
+2 -2
View File
@@ -1,4 +1,4 @@
import type { Api, ImagesApi, ImagesModel, Model, ProviderEnv } from "../types.ts"; import type { Api, ImagesApi, ImagesModel, Model, ProviderEnv, ProviderHeaders } from "../types.ts";
import type { OAuthCredentials } from "../utils/oauth/types.ts"; import type { OAuthCredentials } from "../utils/oauth/types.ts";
/** /**
@@ -7,7 +7,7 @@ import type { OAuthCredentials } from "../utils/oauth/types.ts";
*/ */
export interface ModelAuth { export interface ModelAuth {
apiKey?: string; apiKey?: string;
headers?: Record<string, string>; headers?: ProviderHeaders;
baseUrl?: string; baseUrl?: string;
} }
+131 -5
View File
@@ -19,7 +19,6 @@ export * from "./api/mistral-conversations.lazy.ts";
export * from "./api/openai-codex-responses.lazy.ts"; export * from "./api/openai-codex-responses.lazy.ts";
export * from "./api/openai-completions.lazy.ts"; export * from "./api/openai-completions.lazy.ts";
export * from "./api/openai-responses.lazy.ts"; export * from "./api/openai-responses.lazy.ts";
export * from "./api-registry.ts";
export * from "./env-api-keys.ts"; export * from "./env-api-keys.ts";
export * from "./image-models.ts"; export * from "./image-models.ts";
export * from "./images.ts"; export * from "./images.ts";
@@ -36,11 +35,12 @@ import { mistralConversationsApi } from "./api/mistral-conversations.lazy.ts";
import { openAICodexResponsesApi } from "./api/openai-codex-responses.lazy.ts"; import { openAICodexResponsesApi } from "./api/openai-codex-responses.lazy.ts";
import { openAICompletionsApi } from "./api/openai-completions.lazy.ts"; import { openAICompletionsApi } from "./api/openai-completions.lazy.ts";
import { openAIResponsesApi } from "./api/openai-responses.lazy.ts"; import { openAIResponsesApi } from "./api/openai-responses.lazy.ts";
import { clearApiProviders, getApiProvider, registerApiProvider } from "./api-registry.ts";
import { getEnvApiKey } from "./env-api-keys.ts"; import { getEnvApiKey } from "./env-api-keys.ts";
import { getBuiltinModel, getBuiltinModels, getBuiltinProviders } from "./providers/all.ts"; import { builtinModels, getBuiltinModel, getBuiltinModels, getBuiltinProviders } from "./providers/all.ts";
import { createFauxCore, type FauxProviderRegistration, type RegisterFauxProviderOptions } from "./providers/faux.ts";
import type { import type {
Api, Api,
ApiStreamOptions,
AssistantMessage, AssistantMessage,
AssistantMessageEventStream, AssistantMessageEventStream,
Context, Context,
@@ -48,6 +48,7 @@ import type {
ProviderStreamOptions, ProviderStreamOptions,
ProviderStreams, ProviderStreams,
SimpleStreamOptions, SimpleStreamOptions,
StreamFunction,
StreamOptions, StreamOptions,
} from "./types.ts"; } from "./types.ts";
@@ -60,6 +61,113 @@ export const getModels = getBuiltinModels;
/** @deprecated Static catalog read. Use `getBuiltinProviders` from "@earendil-works/pi-ai/providers/all" or `Models.getProviders()`. */ /** @deprecated Static catalog read. Use `getBuiltinProviders` from "@earendil-works/pi-ai/providers/all" or `Models.getProviders()`. */
export const getProviders = getBuiltinProviders; export const getProviders = getBuiltinProviders;
export type ApiStreamFunction = (
model: Model<Api>,
context: Context,
options?: StreamOptions,
) => AssistantMessageEventStream;
export type ApiStreamSimpleFunction = (
model: Model<Api>,
context: Context,
options?: SimpleStreamOptions,
) => AssistantMessageEventStream;
export interface ApiProvider<TApi extends Api = Api, TOptions extends StreamOptions = StreamOptions> {
api: TApi;
stream: StreamFunction<TApi, TOptions>;
streamSimple: StreamFunction<TApi, SimpleStreamOptions>;
}
interface ApiProviderInternal {
api: Api;
stream: ApiStreamFunction;
streamSimple: ApiStreamSimpleFunction;
}
type RegisteredApiProvider = {
provider: ApiProviderInternal;
sourceId?: string;
};
const apiProviderRegistry = new Map<string, RegisteredApiProvider>();
function wrapStream<TApi extends Api, TOptions extends StreamOptions>(
api: TApi,
stream: StreamFunction<TApi, TOptions>,
): ApiStreamFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return stream(model as Model<TApi>, context, options as TOptions);
};
}
function wrapStreamSimple<TApi extends Api>(
api: TApi,
streamSimple: StreamFunction<TApi, SimpleStreamOptions>,
): ApiStreamSimpleFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return streamSimple(model as Model<TApi>, context, options);
};
}
export function registerApiProvider<TApi extends Api, TOptions extends StreamOptions>(
provider: ApiProvider<TApi, TOptions>,
sourceId?: string,
): void {
apiProviderRegistry.set(provider.api, {
provider: {
api: provider.api,
stream: wrapStream(provider.api, provider.stream),
streamSimple: wrapStreamSimple(provider.api, provider.streamSimple),
},
sourceId,
});
}
export function getApiProvider(api: Api): ApiProviderInternal | undefined {
return apiProviderRegistry.get(api)?.provider;
}
export function getApiProviders(): ApiProviderInternal[] {
return Array.from(apiProviderRegistry.values(), (entry) => entry.provider);
}
export function unregisterApiProviders(sourceId: string): void {
for (const [api, entry] of apiProviderRegistry.entries()) {
if (entry.sourceId === sourceId) {
apiProviderRegistry.delete(api);
}
}
}
function clearApiProviders(): void {
apiProviderRegistry.clear();
}
export function registerFauxProvider(options: RegisterFauxProviderOptions = {}): FauxProviderRegistration {
const core = createFauxCore(options);
const sourceId = `faux-provider-${Math.random().toString(36).slice(2, 10)}`;
registerApiProvider({ api: core.api, stream: core.stream, streamSimple: core.streamSimple }, sourceId);
return {
api: core.api,
models: core.models,
getModel: core.getModel,
state: core.state,
setResponses: core.setResponses,
appendResponses: core.appendResponses,
getPendingResponseCount: core.getPendingResponseCount,
unregister() {
unregisterApiProviders(sourceId);
},
};
}
const BUILTIN_APIS: [Api, ProviderStreams][] = [ const BUILTIN_APIS: [Api, ProviderStreams][] = [
["anthropic-messages", anthropicMessagesApi()], ["anthropic-messages", anthropicMessagesApi()],
["openai-completions", openAICompletionsApi()], ["openai-completions", openAICompletionsApi()],
@@ -72,6 +180,8 @@ const BUILTIN_APIS: [Api, ProviderStreams][] = [
["bedrock-converse-stream", bedrockConverseStreamApi()], ["bedrock-converse-stream", bedrockConverseStreamApi()],
]; ];
const builtinApiProviderInstances = new Map<Api, ReturnType<typeof getApiProvider>>();
/** /**
* Registers the builtin API implementations into the api-registry without * Registers the builtin API implementations into the api-registry without
* clobbering existing entries: compat may load after a test or extension has * clobbering existing entries: compat may load after a test or extension has
@@ -79,18 +189,23 @@ const BUILTIN_APIS: [Api, ProviderStreams][] = [
*/ */
export function registerBuiltInApiProviders(): void { export function registerBuiltInApiProviders(): void {
for (const [api, streams] of BUILTIN_APIS) { for (const [api, streams] of BUILTIN_APIS) {
if (getApiProvider(api)) continue; if (!getApiProvider(api)) {
registerApiProvider({ api, stream: streams.stream, streamSimple: streams.streamSimple }); registerApiProvider({ api, stream: streams.stream, streamSimple: streams.streamSimple });
}
builtinApiProviderInstances.set(api, getApiProvider(api));
} }
} }
export function resetApiProviders(): void { export function resetApiProviders(): void {
clearApiProviders(); clearApiProviders();
builtinApiProviderInstances.clear();
registerBuiltInApiProviders(); registerBuiltInApiProviders();
} }
registerBuiltInApiProviders(); registerBuiltInApiProviders();
const compatModels = builtinModels();
function hasExplicitApiKey(apiKey: string | undefined): apiKey is string { function hasExplicitApiKey(apiKey: string | undefined): apiKey is string {
return typeof apiKey === "string" && apiKey.trim().length > 0; return typeof apiKey === "string" && apiKey.trim().length > 0;
} }
@@ -105,6 +220,11 @@ function withEnvApiKey<TOptions extends StreamOptions>(
return { ...options, apiKey } as TOptions; return { ...options, apiKey } as TOptions;
} }
function shouldUseBuiltinModels(model: Model<Api>): boolean {
const builtin = compatModels.getModel(model.provider, model.id);
return builtin?.api === model.api && getApiProvider(model.api) === builtinApiProviderInstances.get(model.api);
}
function resolveApiProvider(api: Api) { function resolveApiProvider(api: Api) {
const provider = getApiProvider(api); const provider = getApiProvider(api);
if (!provider) { if (!provider) {
@@ -118,6 +238,9 @@ export function stream<TApi extends Api>(
context: Context, context: Context,
options?: ProviderStreamOptions, options?: ProviderStreamOptions,
): AssistantMessageEventStream { ): AssistantMessageEventStream {
if (shouldUseBuiltinModels(model)) {
return compatModels.stream(model, context, options as ApiStreamOptions<TApi> | undefined);
}
const provider = resolveApiProvider(model.api); const provider = resolveApiProvider(model.api);
return provider.stream(model, context, withEnvApiKey(model, options) as StreamOptions); return provider.stream(model, context, withEnvApiKey(model, options) as StreamOptions);
} }
@@ -136,6 +259,9 @@ export function streamSimple<TApi extends Api>(
context: Context, context: Context,
options?: SimpleStreamOptions, options?: SimpleStreamOptions,
): AssistantMessageEventStream { ): AssistantMessageEventStream {
if (shouldUseBuiltinModels(model)) {
return compatModels.streamSimple(model, context, options);
}
const provider = resolveApiProvider(model.api); const provider = resolveApiProvider(model.api);
return provider.streamSimple(model, context, withEnvApiKey(model, options)); return provider.streamSimple(model, context, withEnvApiKey(model, options));
} }
+3 -2
View File
@@ -11,6 +11,7 @@ import type {
Context, Context,
Model, Model,
ModelThinkingLevel, ModelThinkingLevel,
ProviderHeaders,
ProviderStreams, ProviderStreams,
SimpleStreamOptions, SimpleStreamOptions,
StreamOptions, StreamOptions,
@@ -33,7 +34,7 @@ export interface Provider<TApi extends Api = Api> {
readonly name: string; readonly name: string;
readonly baseUrl?: string; readonly baseUrl?: string;
readonly headers?: Record<string, string>; readonly headers?: ProviderHeaders;
/** /**
* Required: at least one of `apiKey`/`oauth`. Every provider has auth * Required: at least one of `apiKey`/`oauth`. Every provider has auth
@@ -287,7 +288,7 @@ export interface CreateProviderOptions<TApi extends Api = Api> {
/** Display name. Default: `id`. */ /** Display name. Default: `id`. */
name?: string; name?: string;
baseUrl?: string; baseUrl?: string;
headers?: Record<string, string>; headers?: ProviderHeaders;
/** Required — every provider has auth semantics, even ambient/keyless ones. */ /** Required — every provider has auth semantics, even ambient/keyless ones. */
auth: ProviderAuth; auth: ProviderAuth;
/** Initial model list (empty for purely dynamic providers). */ /** Initial model list (empty for purely dynamic providers). */
+3 -3
View File
@@ -10,7 +10,7 @@ export const ANT_LING_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "ant-ling", provider: "ant-ling",
baseUrl: "https://api.ant-ling.com/v1", baseUrl: "https://api.ant-ling.com/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"ant-ling","supportsLongCacheRetention":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -28,7 +28,7 @@ export const ANT_LING_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "ant-ling", provider: "ant-ling",
baseUrl: "https://api.ant-ling.com/v1", baseUrl: "https://api.ant-ling.com/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"ant-ling","supportsLongCacheRetention":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -46,7 +46,7 @@ export const ANT_LING_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "ant-ling", provider: "ant-ling",
baseUrl: "https://api.ant-ling.com/v1", baseUrl: "https://api.ant-ling.com/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"thinkingFormat":"ant-ling"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"ant-ling","supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh"}, thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh"},
input: ["text"], input: ["text"],
@@ -10,6 +10,7 @@ export const CEREBRAS_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cerebras", provider: "cerebras",
baseUrl: "https://api.cerebras.ai/v1", baseUrl: "https://api.cerebras.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -27,6 +28,7 @@ export const CEREBRAS_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cerebras", provider: "cerebras",
baseUrl: "https://api.cerebras.ai/v1", baseUrl: "https://api.cerebras.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -10,6 +10,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -27,6 +28,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -44,6 +46,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -61,6 +64,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -78,6 +82,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -95,6 +100,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -112,7 +118,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true}, compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true},
reasoning: true, reasoning: true,
thinkingLevelMap: {"off":null,"xhigh":"xhigh"}, thinkingLevelMap: {"off":null,"xhigh":"xhigh"},
input: ["text", "image"], input: ["text", "image"],
@@ -131,6 +137,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -148,6 +155,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -165,6 +173,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -182,6 +191,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -199,7 +209,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true}, compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true},
reasoning: true, reasoning: true,
thinkingLevelMap: {"xhigh":"max"}, thinkingLevelMap: {"xhigh":"max"},
input: ["text", "image"], input: ["text", "image"],
@@ -218,7 +228,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true,"supportsTemperature":false}, compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true,"supportsTemperature":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"xhigh":"xhigh"}, thinkingLevelMap: {"xhigh":"xhigh"},
input: ["text", "image"], input: ["text", "image"],
@@ -237,7 +247,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true,"supportsTemperature":false}, compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true,"supportsTemperature":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"xhigh":"xhigh"}, thinkingLevelMap: {"xhigh":"xhigh"},
input: ["text", "image"], input: ["text", "image"],
@@ -256,6 +266,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -273,6 +284,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -290,7 +302,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "anthropic-messages", api: "anthropic-messages",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/anthropic",
compat: {"forceAdaptiveThinking":true}, compat: {"sendSessionAffinityHeaders":true,"forceAdaptiveThinking":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -587,7 +599,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -605,7 +617,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -623,7 +635,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -641,7 +653,7 @@ export const CLOUDFLARE_AI_GATEWAY_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-ai-gateway", provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat", baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -1,9 +1,9 @@
import { anthropicMessagesApi } from "../api/anthropic-messages.lazy.ts"; import { anthropicMessagesApi } from "../api/anthropic-messages.lazy.ts";
import { openAICompletionsApi } from "../api/openai-completions.lazy.ts"; import { openAICompletionsApi } from "../api/openai-completions.lazy.ts";
import { openAIResponsesApi } from "../api/openai-responses.lazy.ts"; import { openAIResponsesApi } from "../api/openai-responses.lazy.ts";
import { envApiKeyAuth } from "../auth/helpers.ts";
import { createProvider, type Provider } from "../models.ts"; import { createProvider, type Provider } from "../models.ts";
import { CLOUDFLARE_AI_GATEWAY_MODELS } from "./cloudflare-ai-gateway.models.ts"; import { CLOUDFLARE_AI_GATEWAY_MODELS } from "./cloudflare-ai-gateway.models.ts";
import { cloudflareAIGatewayAuth } from "./cloudflare-auth.ts";
export function cloudflareAIGatewayProvider(): Provider< export function cloudflareAIGatewayProvider(): Provider<
"anthropic-messages" | "openai-completions" | "openai-responses" "anthropic-messages" | "openai-completions" | "openai-responses"
@@ -11,7 +11,7 @@ export function cloudflareAIGatewayProvider(): Provider<
return createProvider({ return createProvider({
id: "cloudflare-ai-gateway", id: "cloudflare-ai-gateway",
name: "Cloudflare AI Gateway", name: "Cloudflare AI Gateway",
auth: { apiKey: envApiKeyAuth("Cloudflare API key", ["CLOUDFLARE_API_KEY"]) }, auth: { apiKey: cloudflareAIGatewayAuth() },
models: Object.values(CLOUDFLARE_AI_GATEWAY_MODELS), models: Object.values(CLOUDFLARE_AI_GATEWAY_MODELS),
api: { api: {
"anthropic-messages": anthropicMessagesApi(), "anthropic-messages": anthropicMessagesApi(),
@@ -0,0 +1,108 @@
import type { ApiKeyAuth, ApiKeyCredential, AuthContext } from "../auth/types.ts";
import type { Api, ImagesApi, ImagesModel, Model, ProviderEnv } from "../types.ts";
const CLOUDFLARE_API_KEY = "CLOUDFLARE_API_KEY";
const CLOUDFLARE_ACCOUNT_ID = "CLOUDFLARE_ACCOUNT_ID";
const CLOUDFLARE_GATEWAY_ID = "CLOUDFLARE_GATEWAY_ID";
type CloudflareAuthKind = "workers-ai" | "ai-gateway";
async function resolveValue(input: {
name: string;
ctx: AuthContext;
credential: ApiKeyCredential | undefined;
}): Promise<string | undefined> {
if (input.credential) {
if (input.name === CLOUDFLARE_API_KEY) return input.credential.key;
return input.credential.metadata?.[input.name];
}
return input.ctx.env(input.name);
}
function resolveCloudflareBaseUrl(
model: Model<Api> | ImagesModel<ImagesApi>,
accountId: string,
gatewayId: string | undefined,
): string {
return model.baseUrl
.replaceAll(`{${CLOUDFLARE_ACCOUNT_ID}}`, accountId)
.replaceAll(`{${CLOUDFLARE_GATEWAY_ID}}`, gatewayId ?? "");
}
async function resolveCloudflareEnv(input: {
kind: CloudflareAuthKind;
model: Model<Api> | ImagesModel<ImagesApi>;
ctx: AuthContext;
credential: ApiKeyCredential | undefined;
}): Promise<{ apiKey: string; env: ProviderEnv; baseUrl: string; source: string } | undefined> {
const apiKey = await resolveValue({ name: CLOUDFLARE_API_KEY, ctx: input.ctx, credential: input.credential });
const accountId = await resolveValue({ name: CLOUDFLARE_ACCOUNT_ID, ctx: input.ctx, credential: input.credential });
const gatewayId =
input.kind === "ai-gateway"
? await resolveValue({ name: CLOUDFLARE_GATEWAY_ID, ctx: input.ctx, credential: input.credential })
: undefined;
if (!apiKey || !accountId || (input.kind === "ai-gateway" && !gatewayId)) return undefined;
return {
apiKey,
env: {
CLOUDFLARE_ACCOUNT_ID: accountId,
...(gatewayId ? { CLOUDFLARE_GATEWAY_ID: gatewayId } : {}),
},
baseUrl: resolveCloudflareBaseUrl(input.model, accountId, gatewayId),
source: input.credential ? "stored credential" : CLOUDFLARE_API_KEY,
};
}
export function cloudflareWorkersAIAuth(): ApiKeyAuth {
return {
name: "Cloudflare API key",
login: async (callbacks) => {
const key = await callbacks.prompt({ type: "secret", message: "Enter Cloudflare API key" });
const accountId = await callbacks.prompt({ type: "text", message: "Enter Cloudflare account ID" });
return { type: "api-key", key, metadata: { CLOUDFLARE_ACCOUNT_ID: accountId } };
},
resolve: async ({ model, ctx, credential }) => {
const resolved = await resolveCloudflareEnv({ kind: "workers-ai", model, ctx, credential });
if (!resolved) return undefined;
return {
auth: { apiKey: resolved.apiKey, baseUrl: resolved.baseUrl },
env: resolved.env,
source: resolved.source,
};
},
};
}
export function cloudflareAIGatewayAuth(): ApiKeyAuth {
return {
name: "Cloudflare API key",
login: async (callbacks) => {
const key = await callbacks.prompt({ type: "secret", message: "Enter Cloudflare API key" });
const accountId = await callbacks.prompt({ type: "text", message: "Enter Cloudflare account ID" });
const gatewayId = await callbacks.prompt({ type: "text", message: "Enter Cloudflare AI Gateway ID" });
return {
type: "api-key",
key,
metadata: { CLOUDFLARE_ACCOUNT_ID: accountId, CLOUDFLARE_GATEWAY_ID: gatewayId },
};
},
resolve: async ({ model, ctx, credential }) => {
const resolved = await resolveCloudflareEnv({ kind: "ai-gateway", model, ctx, credential });
if (!resolved) return undefined;
return {
auth: {
headers: {
"cf-aig-authorization": `Bearer ${resolved.apiKey}`,
Authorization: null,
"x-api-key": null,
},
baseUrl: resolved.baseUrl,
},
env: resolved.env,
source: resolved.source,
};
},
};
}
@@ -10,7 +10,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -28,7 +28,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -46,7 +46,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -64,7 +64,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -82,7 +82,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -100,7 +100,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -118,7 +118,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -136,7 +136,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -154,7 +154,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -172,7 +172,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -190,7 +190,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -208,7 +208,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -226,7 +226,7 @@ export const CLOUDFLARE_WORKERS_AI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "cloudflare-workers-ai", provider: "cloudflare-workers-ai",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1", baseUrl: "https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1",
compat: {"sendSessionAffinityHeaders":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsLongCacheRetention":false,"sendSessionAffinityHeaders":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -1,13 +1,13 @@
import { openAICompletionsApi } from "../api/openai-completions.lazy.ts"; import { openAICompletionsApi } from "../api/openai-completions.lazy.ts";
import { envApiKeyAuth } from "../auth/helpers.ts";
import { createProvider, type Provider } from "../models.ts"; import { createProvider, type Provider } from "../models.ts";
import { cloudflareWorkersAIAuth } from "./cloudflare-auth.ts";
import { CLOUDFLARE_WORKERS_AI_MODELS } from "./cloudflare-workers-ai.models.ts"; import { CLOUDFLARE_WORKERS_AI_MODELS } from "./cloudflare-workers-ai.models.ts";
export function cloudflareWorkersAIProvider(): Provider<"openai-completions"> { export function cloudflareWorkersAIProvider(): Provider<"openai-completions"> {
return createProvider({ return createProvider({
id: "cloudflare-workers-ai", id: "cloudflare-workers-ai",
name: "Cloudflare Workers AI", name: "Cloudflare Workers AI",
auth: { apiKey: envApiKeyAuth("Cloudflare API key", ["CLOUDFLARE_API_KEY"]) }, auth: { apiKey: cloudflareWorkersAIAuth() },
models: Object.values(CLOUDFLARE_WORKERS_AI_MODELS), models: Object.values(CLOUDFLARE_WORKERS_AI_MODELS),
api: openAICompletionsApi(), api: openAICompletionsApi(),
}); });
+2 -2
View File
@@ -10,7 +10,7 @@ export const DEEPSEEK_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "deepseek", provider: "deepseek",
baseUrl: "https://api.deepseek.com", baseUrl: "https://api.deepseek.com",
compat: {"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -29,7 +29,7 @@ export const DEEPSEEK_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "deepseek", provider: "deepseek",
baseUrl: "https://api.deepseek.com", baseUrl: "https://api.deepseek.com",
compat: {"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
+1 -21
View File
@@ -1,4 +1,3 @@
import { registerApiProvider, unregisterApiProviders } from "../api-registry.ts";
import { createProvider, type Provider } from "../models.ts"; import { createProvider, type Provider } from "../models.ts";
import type { import type {
AssistantMessage, AssistantMessage,
@@ -401,7 +400,7 @@ async function streamWithDeltas(
stream.end(message); stream.end(message);
} }
function createFauxCore(options: RegisterFauxProviderOptions) { export function createFauxCore(options: RegisterFauxProviderOptions) {
const api = options.api ?? randomId(DEFAULT_API); const api = options.api ?? randomId(DEFAULT_API);
const provider = options.provider ?? DEFAULT_PROVIDER; const provider = options.provider ?? DEFAULT_PROVIDER;
const minTokenSize = Math.max( const minTokenSize = Math.max(
@@ -508,25 +507,6 @@ function createFauxCore(options: RegisterFauxProviderOptions) {
}; };
} }
/** Registers the faux api into the legacy global api-registry. */
export function registerFauxProvider(options: RegisterFauxProviderOptions = {}): FauxProviderRegistration {
const core = createFauxCore(options);
const sourceId = randomId("faux-provider");
registerApiProvider({ api: core.api, stream: core.stream, streamSimple: core.streamSimple }, sourceId);
return {
api: core.api,
models: core.models,
getModel: core.getModel,
state: core.state,
setResponses: core.setResponses,
appendResponses: core.appendResponses,
getPendingResponseCount: core.getPendingResponseCount,
unregister() {
unregisterApiProviders(sourceId);
},
};
}
/** /**
* Faux provider for tests built on explicit `Models` collections: * Faux provider for tests built on explicit `Models` collections:
* *
@@ -4,6 +4,24 @@
import type { Model } from "../types.ts"; import type { Model } from "../types.ts";
export const HUGGINGFACE_MODELS = { export const HUGGINGFACE_MODELS = {
"MiniMaxAI/MiniMax-M2": {
id: "MiniMaxAI/MiniMax-M2",
name: "MiniMax-M2",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 204800,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
"MiniMaxAI/MiniMax-M2.1": { "MiniMaxAI/MiniMax-M2.1": {
id: "MiniMaxAI/MiniMax-M2.1", id: "MiniMaxAI/MiniMax-M2.1",
name: "MiniMax-M2.1", name: "MiniMax-M2.1",
@@ -58,6 +76,42 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 204800, contextWindow: 204800,
maxTokens: 131072, maxTokens: 131072,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"MiniMaxAI/MiniMax-M3": {
id: "MiniMaxAI/MiniMax-M3",
name: "MiniMax-M3",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.3,
output: 1.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 524288,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-235B-A22B": {
id: "Qwen/Qwen3-235B-A22B",
name: "Qwen3 235B-A22B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.2,
output: 0.8,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 40960,
maxTokens: 16384,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-235B-A22B-Thinking-2507": { "Qwen/Qwen3-235B-A22B-Thinking-2507": {
id: "Qwen/Qwen3-235B-A22B-Thinking-2507", id: "Qwen/Qwen3-235B-A22B-Thinking-2507",
name: "Qwen3-235B-A22B-Thinking-2507", name: "Qwen3-235B-A22B-Thinking-2507",
@@ -76,6 +130,42 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144, contextWindow: 262144,
maxTokens: 131072, maxTokens: 131072,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"Qwen/Qwen3-32B": {
id: "Qwen/Qwen3-32B",
name: "Qwen3 32B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.29,
output: 0.59,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 16384,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-Coder-30B-A3B-Instruct": {
id: "Qwen/Qwen3-Coder-30B-A3B-Instruct",
name: "Qwen3-Coder 30B-A3B Instruct",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: false,
input: ["text"],
cost: {
input: 0.07,
output: 0.26,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3-Coder-480B-A35B-Instruct": { "Qwen/Qwen3-Coder-480B-A35B-Instruct": {
id: "Qwen/Qwen3-Coder-480B-A35B-Instruct", id: "Qwen/Qwen3-Coder-480B-A35B-Instruct",
name: "Qwen3-Coder-480B-A35B-Instruct", name: "Qwen3-Coder-480B-A35B-Instruct",
@@ -148,6 +238,60 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144, contextWindow: 262144,
maxTokens: 131072, maxTokens: 131072,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-122B-A10B": {
id: "Qwen/Qwen3.5-122B-A10B",
name: "Qwen3.5 122B-A10B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.4,
output: 3.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-27B": {
id: "Qwen/Qwen3.5-27B",
name: "Qwen3.5 27B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.3,
output: 2.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-35B-A3B": {
id: "Qwen/Qwen3.5-35B-A3B",
name: "Qwen3.5 35B-A3B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.25,
output: 2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-397B-A17B": { "Qwen/Qwen3.5-397B-A17B": {
id: "Qwen/Qwen3.5-397B-A17B", id: "Qwen/Qwen3.5-397B-A17B",
name: "Qwen3.5-397B-A17B", name: "Qwen3.5-397B-A17B",
@@ -166,6 +310,42 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144, contextWindow: 262144,
maxTokens: 32768, maxTokens: 32768,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"Qwen/Qwen3.5-9B": {
id: "Qwen/Qwen3.5-9B",
name: "Qwen3.5 9B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.17,
output: 0.25,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"Qwen/Qwen3.6-35B-A3B": {
id: "Qwen/Qwen3.6-35B-A3B",
name: "Qwen3.6 35B-A3B",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.15,
output: 0.95,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 65536,
} satisfies Model<"openai-completions">,
"XiaomiMiMo/MiMo-V2-Flash": { "XiaomiMiMo/MiMo-V2-Flash": {
id: "XiaomiMiMo/MiMo-V2-Flash", id: "XiaomiMiMo/MiMo-V2-Flash",
name: "MiMo-V2-Flash", name: "MiMo-V2-Flash",
@@ -184,6 +364,24 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144, contextWindow: 262144,
maxTokens: 4096, maxTokens: 4096,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-R1": {
id: "deepseek-ai/DeepSeek-R1",
name: "DeepSeek-R1",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.7,
output: 2.5,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 64000,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-R1-0528": { "deepseek-ai/DeepSeek-R1-0528": {
id: "deepseek-ai/DeepSeek-R1-0528", id: "deepseek-ai/DeepSeek-R1-0528",
name: "DeepSeek-R1-0528", name: "DeepSeek-R1-0528",
@@ -220,6 +418,24 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 163840, contextWindow: 163840,
maxTokens: 65536, maxTokens: 65536,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-V4-Flash": {
id: "deepseek-ai/DeepSeek-V4-Flash",
name: "DeepSeek V4 Flash",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.14,
output: 0.28,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 1048576,
maxTokens: 384000,
} satisfies Model<"openai-completions">,
"deepseek-ai/DeepSeek-V4-Pro": { "deepseek-ai/DeepSeek-V4-Pro": {
id: "deepseek-ai/DeepSeek-V4-Pro", id: "deepseek-ai/DeepSeek-V4-Pro",
name: "DeepSeek V4 Pro", name: "DeepSeek V4 Pro",
@@ -238,6 +454,60 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 1048576, contextWindow: 1048576,
maxTokens: 393216, maxTokens: 393216,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"google/gemma-4-26B-A4B-it": {
id: "google/gemma-4-26B-A4B-it",
name: "Gemma 4 26B A4B IT",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.13,
output: 0.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"google/gemma-4-31B-it": {
id: "google/gemma-4-31B-it",
name: "Gemma 4 31B IT",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.14,
output: 0.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 32768,
} satisfies Model<"openai-completions">,
"meta-llama/Llama-3.3-70B-Instruct": {
id: "meta-llama/Llama-3.3-70B-Instruct",
name: "Llama-3.3-70B-Instruct",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: false,
input: ["text"],
cost: {
input: 0.59,
output: 0.79,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 4096,
} satisfies Model<"openai-completions">,
"moonshotai/Kimi-K2-Instruct": { "moonshotai/Kimi-K2-Instruct": {
id: "moonshotai/Kimi-K2-Instruct", id: "moonshotai/Kimi-K2-Instruct",
name: "Kimi-K2-Instruct", name: "Kimi-K2-Instruct",
@@ -328,6 +598,114 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 262144, contextWindow: 262144,
maxTokens: 262144, maxTokens: 262144,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"moonshotai/Kimi-K2.7-Code": {
id: "moonshotai/Kimi-K2.7-Code",
name: "Kimi K2.7 Code",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 262144,
} satisfies Model<"openai-completions">,
"stepfun-ai/Step-3.5-Flash": {
id: "stepfun-ai/Step-3.5-Flash",
name: "Step 3.5 Flash",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.1,
output: 0.3,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 256000,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.5": {
id: "zai-org/GLM-4.5",
name: "GLM-4.5",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.6,
output: 2.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 98304,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.5-Air": {
id: "zai-org/GLM-4.5-Air",
name: "GLM-4.5-Air",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.13,
output: 0.85,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 98304,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.5V": {
id: "zai-org/GLM-4.5V",
name: "GLM-4.5V",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.6,
output: 1.8,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 65536,
maxTokens: 16384,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.6": {
id: "zai-org/GLM-4.6",
name: "GLM-4.6",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 0.55,
output: 2.2,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 204800,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"zai-org/GLM-4.7": { "zai-org/GLM-4.7": {
id: "zai-org/GLM-4.7", id: "zai-org/GLM-4.7",
name: "GLM-4.7", name: "GLM-4.7",
@@ -400,4 +778,22 @@ export const HUGGINGFACE_MODELS = {
contextWindow: 202752, contextWindow: 202752,
maxTokens: 131072, maxTokens: 131072,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"zai-org/GLM-5.2": {
id: "zai-org/GLM-5.2",
name: "GLM-5.2",
api: "openai-completions",
provider: "huggingface",
baseUrl: "https://router.huggingface.co/v1",
compat: {"supportsDeveloperRole":false},
reasoning: true,
input: ["text"],
cost: {
input: 1.4,
output: 4.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 262144,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
} as const; } as const;
+10 -10
View File
@@ -10,7 +10,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -29,7 +29,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -48,7 +48,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -66,7 +66,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"}, thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -85,7 +85,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"], input: ["text", "image"],
@@ -104,7 +104,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -122,7 +122,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -140,7 +140,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -158,7 +158,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -193,7 +193,7 @@ export const OPENCODE_GO_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode-go", provider: "opencode-go",
baseUrl: "https://opencode.ai/zen/go/v1", baseUrl: "https://opencode.ai/zen/go/v1",
compat: {"thinkingFormat":"qwen","maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"thinkingFormat":"qwen","maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
+32 -14
View File
@@ -10,7 +10,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -188,7 +188,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -207,7 +207,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","requiresReasoningContentOnAssistantMessages":true},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -226,7 +226,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -299,7 +299,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -317,7 +317,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -329,6 +329,24 @@ export const OPENCODE_MODELS = {
contextWindow: 204800, contextWindow: 204800,
maxTokens: 131072, maxTokens: 131072,
} satisfies Model<"openai-completions">, } satisfies Model<"openai-completions">,
"glm-5.2": {
id: "glm-5.2",
name: "GLM-5.2",
api: "openai-completions",
provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true,
input: ["text"],
cost: {
input: 1.4,
output: 4.4,
cacheRead: 0.26,
cacheWrite: 0,
},
contextWindow: 1000000,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"gpt-5": { "gpt-5": {
id: "gpt-5", id: "gpt-5",
name: "GPT-5", name: "GPT-5",
@@ -623,7 +641,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"supportsReasoningEffort":false,"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null},
input: ["text", "image"], input: ["text", "image"],
@@ -642,7 +660,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -660,7 +678,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"thinkingFormat":"deepseek","supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -678,7 +696,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -696,7 +714,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -714,7 +732,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens","supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens","supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -732,7 +750,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -750,7 +768,7 @@ export const OPENCODE_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "opencode", provider: "opencode",
baseUrl: "https://opencode.ai/zen/v1", baseUrl: "https://opencode.ai/zen/v1",
compat: {"maxTokensField":"max_tokens"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"maxTokensField":"max_tokens"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
File diff suppressed because it is too large Load Diff
+19 -19
View File
@@ -10,7 +10,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"off":null,"minimal":null,"low":null,"medium":null},
input: ["text"], input: ["text"],
@@ -29,7 +29,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"], input: ["text", "image"],
@@ -48,7 +48,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -66,7 +66,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -84,7 +84,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"], input: ["text", "image"],
@@ -103,7 +103,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"], input: ["text", "image"],
@@ -122,7 +122,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"], input: ["text"],
@@ -141,7 +141,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -159,7 +159,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null},
input: ["text"], input: ["text"],
@@ -178,7 +178,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -196,7 +196,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"], input: ["text", "image"],
@@ -215,7 +215,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -233,7 +233,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text", "image"], input: ["text", "image"],
@@ -252,7 +252,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"], input: ["text"],
@@ -271,7 +271,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"], input: ["text"],
@@ -290,7 +290,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"openai"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","thinkingFormat":"openai","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null}, thinkingLevelMap: {"off":null,"minimal":null},
input: ["text"], input: ["text"],
@@ -309,7 +309,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"openai"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"maxTokensField":"max_tokens","thinkingFormat":"openai","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"off":null,"minimal":null}, thinkingLevelMap: {"off":null,"minimal":null},
input: ["text"], input: ["text"],
@@ -328,7 +328,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"], input: ["text"],
@@ -347,7 +347,7 @@ export const TOGETHER_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "together", provider: "together",
baseUrl: "https://api.together.ai/v1", baseUrl: "https://api.together.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","supportsStrictMode":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"maxTokensField":"max_tokens","thinkingFormat":"together","supportsStrictMode":false,"supportsLongCacheRetention":false},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":null,"medium":null}, thinkingLevelMap: {"minimal":null,"low":null,"medium":null},
input: ["text"], input: ["text"],
+7
View File
@@ -10,6 +10,7 @@ export const XAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "xai", provider: "xai",
baseUrl: "https://api.x.ai/v1", baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -27,6 +28,7 @@ export const XAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "xai", provider: "xai",
baseUrl: "https://api.x.ai/v1", baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -44,6 +46,7 @@ export const XAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "xai", provider: "xai",
baseUrl: "https://api.x.ai/v1", baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: false, reasoning: false,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -61,6 +64,7 @@ export const XAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "xai", provider: "xai",
baseUrl: "https://api.x.ai/v1", baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -78,6 +82,7 @@ export const XAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "xai", provider: "xai",
baseUrl: "https://api.x.ai/v1", baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -95,6 +100,7 @@ export const XAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "xai", provider: "xai",
baseUrl: "https://api.x.ai/v1", baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
@@ -112,6 +118,7 @@ export const XAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "xai", provider: "xai",
baseUrl: "https://api.x.ai/v1", baseUrl: "https://api.x.ai/v1",
compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false},
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -10,7 +10,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai-coding-cn", provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4", baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -28,7 +28,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai-coding-cn", provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4", baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -46,7 +46,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai-coding-cn", provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4", baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -64,7 +64,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai-coding-cn", provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4", baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -82,7 +82,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai-coding-cn", provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4", baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","supportsReasoningEffort":true,"zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":"high","medium":"high","high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":"high","medium":"high","high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -101,7 +101,7 @@ export const ZAI_CODING_CN_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai-coding-cn", provider: "zai-coding-cn",
baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4", baseUrl: "https://open.bigmodel.cn/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
+6 -6
View File
@@ -10,7 +10,7 @@ export const ZAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai", provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4", baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai"}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai"},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -28,7 +28,7 @@ export const ZAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai", provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4", baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -46,7 +46,7 @@ export const ZAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai", provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4", baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -64,7 +64,7 @@ export const ZAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai", provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4", baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
input: ["text"], input: ["text"],
cost: { cost: {
@@ -82,7 +82,7 @@ export const ZAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai", provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4", baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","supportsReasoningEffort":true,"zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":true,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
thinkingLevelMap: {"minimal":null,"low":"high","medium":"high","high":"high","xhigh":"max"}, thinkingLevelMap: {"minimal":null,"low":"high","medium":"high","high":"high","xhigh":"max"},
input: ["text"], input: ["text"],
@@ -101,7 +101,7 @@ export const ZAI_MODELS = {
api: "openai-completions", api: "openai-completions",
provider: "zai", provider: "zai",
baseUrl: "https://api.z.ai/api/coding/paas/v4", baseUrl: "https://api.z.ai/api/coding/paas/v4",
compat: {"supportsDeveloperRole":false,"thinkingFormat":"zai","zaiToolStream":true}, compat: {"supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"thinkingFormat":"zai","zaiToolStream":true},
reasoning: true, reasoning: true,
input: ["text", "image"], input: ["text", "image"],
cost: { cost: {
+5 -2
View File
@@ -99,6 +99,7 @@ export type Transport = "sse" | "websocket" | "websocket-cached" | "auto";
/** Provider-scoped environment overrides. Values take precedence over process.env. */ /** Provider-scoped environment overrides. Values take precedence over process.env. */
export type ProviderEnv = Record<string, string>; export type ProviderEnv = Record<string, string>;
export type ProviderHeaders = Record<string, string | null>;
export interface ProviderResponse { export interface ProviderResponse {
status: number; status: number;
@@ -142,8 +143,9 @@ export interface StreamOptions {
* On AWS Bedrock these are injected via a Smithy `build`-step middleware so * On AWS Bedrock these are injected via a Smithy `build`-step middleware so
* they are covered by SigV4 signing; reserved headers (`x-amz-*`, * they are covered by SigV4 signing; reserved headers (`x-amz-*`,
* `authorization`, `host`) are silently ignored to preserve SigV4 / bearer auth. * `authorization`, `host`) are silently ignored to preserve SigV4 / bearer auth.
* A null value suppresses a provider/API default header with the same name.
*/ */
headers?: Record<string, string>; headers?: ProviderHeaders;
/** /**
* HTTP request timeout in milliseconds for providers/SDKs that support it. * HTTP request timeout in milliseconds for providers/SDKs that support it.
* For example, OpenAI and Anthropic SDK clients default to 10 minutes. * For example, OpenAI and Anthropic SDK clients default to 10 minutes.
@@ -256,8 +258,9 @@ export interface ImagesOptions {
/** /**
* Optional custom HTTP headers to include in API requests. * Optional custom HTTP headers to include in API requests.
* Merged with provider defaults; can override default headers. * Merged with provider defaults; can override default headers.
* A null value suppresses a provider/API default header with the same name.
*/ */
headers?: Record<string, string>; headers?: ProviderHeaders;
/** /**
* HTTP request timeout in milliseconds for providers/SDKs that support it. * HTTP request timeout in milliseconds for providers/SDKs that support it.
*/ */
+11
View File
@@ -1,3 +1,5 @@
import type { ProviderHeaders } from "../types.ts";
export function headersToRecord(headers: Headers): Record<string, string> { export function headersToRecord(headers: Headers): Record<string, string> {
const result: Record<string, string> = {}; const result: Record<string, string> = {};
for (const [key, value] of headers.entries()) { for (const [key, value] of headers.entries()) {
@@ -5,3 +7,12 @@ export function headersToRecord(headers: Headers): Record<string, string> {
} }
return result; return result;
} }
export function providerHeadersToRecord(headers: ProviderHeaders | undefined): Record<string, string> | undefined {
if (!headers) return undefined;
const result: Record<string, string> = {};
for (const [key, value] of Object.entries(headers)) {
if (value !== null) result[key] = value;
}
return Object.keys(result).length > 0 ? result : undefined;
}
+5 -5
View File
@@ -9,7 +9,7 @@ const model: Model<"openai-responses"> = {
id: "test-model", id: "test-model",
name: "Test Model", name: "Test Model",
api: "openai-responses", api: "openai-responses",
provider: "openai", provider: "custom-openai",
baseUrl: "https://example.test/v1", baseUrl: "https://example.test/v1",
reasoning: false, reasoning: false,
input: ["text"], input: ["text"],
@@ -38,12 +38,12 @@ function message(): AssistantMessage {
}; };
} }
describe("compat env API key injection", () => { describe("compat legacy API fallback", () => {
afterEach(() => { afterEach(() => {
resetApiProviders(); resetApiProviders();
}); });
it("uses request-scoped env when injecting provider API keys", async () => { it("dispatches unknown providers through the legacy API registry", async () => {
let capturedApiKey: string | undefined; let capturedApiKey: string | undefined;
registerApiProvider({ registerApiProvider({
api: "openai-responses", api: "openai-responses",
@@ -67,8 +67,8 @@ describe("compat env API key injection", () => {
}, },
}); });
await complete(model, context, { env: { OPENAI_API_KEY: "scoped-key" } }); await complete(model, context, { apiKey: "request-key" });
expect(capturedApiKey).toBe("scoped-key"); expect(capturedApiKey).toBe("request-key");
}); });
}); });
@@ -127,6 +127,7 @@ describe("openai-completions empty tools handling", () => {
}); });
it("uses conservative OpenAI-compatible fields for Cloudflare AI Gateway /compat models", async () => { it("uses conservative OpenAI-compatible fields for Cloudflare AI Gateway /compat models", async () => {
process.env.CLOUDFLARE_API_KEY = "cf-token";
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id"; process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id"; process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
const model = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!; const model = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!;
@@ -137,7 +138,7 @@ describe("openai-completions empty tools handling", () => {
systemPrompt: "You are helpful.", systemPrompt: "You are helpful.",
messages: [{ role: "user", content: "hi", timestamp: Date.now() }], messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
}, },
{ apiKey: "test", maxTokens: 1234, reasoning: "high" }, { maxTokens: 1234, reasoning: "high" },
).result(); ).result();
const params = mockState.lastParams as { const params = mockState.lastParams as {
@@ -159,35 +160,25 @@ describe("openai-completions empty tools handling", () => {
}; };
expect(clientOptions.baseURL).toBe("https://gateway.ai.cloudflare.com/v1/account-id/gateway-id/compat"); expect(clientOptions.baseURL).toBe("https://gateway.ai.cloudflare.com/v1/account-id/gateway-id/compat");
expect(clientOptions.defaultHeaders?.Authorization).toBeNull(); expect(clientOptions.defaultHeaders?.Authorization).toBeNull();
expect(clientOptions.defaultHeaders?.["cf-aig-authorization"]).toBe("Bearer test"); expect(clientOptions.defaultHeaders?.["cf-aig-authorization"]).toBe("Bearer cf-token");
}); });
it("uses provider env before process.env for Cloudflare AI Gateway base URL", async () => { it("resolves Cloudflare AI Gateway base URL through provider auth", async () => {
process.env.CLOUDFLARE_ACCOUNT_ID = "process-account"; process.env.CLOUDFLARE_API_KEY = "cf-token";
process.env.CLOUDFLARE_GATEWAY_ID = "process-gateway"; process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
const model = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!; const model = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!;
await streamSimple( await streamSimple(model, {
model, messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
{ }).result();
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{
apiKey: "test",
env: {
CLOUDFLARE_ACCOUNT_ID: "provider-account",
CLOUDFLARE_GATEWAY_ID: "provider-gateway",
},
},
).result();
const clientOptions = mockState.lastClientOptions as { baseURL?: string }; const clientOptions = mockState.lastClientOptions as { baseURL?: string };
expect(clientOptions.baseURL).toBe( expect(clientOptions.baseURL).toBe("https://gateway.ai.cloudflare.com/v1/account-id/gateway-id/compat");
"https://gateway.ai.cloudflare.com/v1/provider-account/provider-gateway/compat",
);
}); });
it("preserves inline upstream Authorization for Cloudflare AI Gateway BYOK requests", async () => { it("preserves inline upstream Authorization for Cloudflare AI Gateway BYOK requests", async () => {
process.env.CLOUDFLARE_API_KEY = "cf-token";
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id"; process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id"; process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
const model = getModel("cloudflare-ai-gateway", "gpt-5.1")!; const model = getModel("cloudflare-ai-gateway", "gpt-5.1")!;
@@ -197,7 +188,7 @@ describe("openai-completions empty tools handling", () => {
{ {
messages: [{ role: "user", content: "hi", timestamp: Date.now() }], messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
}, },
{ apiKey: "cf-token", headers: { Authorization: "Bearer upstream-token" } }, { headers: { Authorization: "Bearer upstream-token" } },
).result(); ).result();
const clientOptions = mockState.lastClientOptions as { defaultHeaders?: Record<string, unknown> }; const clientOptions = mockState.lastClientOptions as { defaultHeaders?: Record<string, unknown> };
@@ -206,6 +197,7 @@ describe("openai-completions empty tools handling", () => {
}); });
it("sends session affinity headers for Workers AI through Cloudflare AI Gateway", async () => { it("sends session affinity headers for Workers AI through Cloudflare AI Gateway", async () => {
process.env.CLOUDFLARE_API_KEY = "cf-token";
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id"; process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id"; process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
const workersModel = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!; const workersModel = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6")!;
@@ -215,7 +207,7 @@ describe("openai-completions empty tools handling", () => {
{ {
messages: [{ role: "user", content: "hi", timestamp: Date.now() }], messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
}, },
{ apiKey: "test", sessionId: "session-1" }, { sessionId: "session-1" },
).result(); ).result();
const clientOptions = mockState.lastClientOptions as { defaultHeaders?: Record<string, string> }; const clientOptions = mockState.lastClientOptions as { defaultHeaders?: Record<string, string> };
+51
View File
@@ -5,6 +5,8 @@ import { createModels, createProvider } from "../src/models.ts";
import { builtinModels, builtinProviders } from "../src/providers/all.ts"; import { builtinModels, builtinProviders } from "../src/providers/all.ts";
import { amazonBedrockProvider } from "../src/providers/amazon-bedrock.ts"; import { amazonBedrockProvider } from "../src/providers/amazon-bedrock.ts";
import { anthropicProvider } from "../src/providers/anthropic.ts"; import { anthropicProvider } from "../src/providers/anthropic.ts";
import { cloudflareAIGatewayProvider } from "../src/providers/cloudflare-ai-gateway.ts";
import { cloudflareWorkersAIProvider } from "../src/providers/cloudflare-workers-ai.ts";
import { fauxAssistantMessage, fauxProvider } from "../src/providers/faux.ts"; import { fauxAssistantMessage, fauxProvider } from "../src/providers/faux.ts";
import { googleVertexProvider } from "../src/providers/google-vertex.ts"; import { googleVertexProvider } from "../src/providers/google-vertex.ts";
import type { Api, Context, Model, ProviderStreams } from "../src/types.ts"; import type { Api, Context, Model, ProviderStreams } from "../src/types.ts";
@@ -66,6 +68,55 @@ describe("builtin providers", () => {
expect(await unconfigured.getAuth(model)).toBeUndefined(); expect(await unconfigured.getAuth(model)).toBeUndefined();
}); });
it("requires Cloudflare Workers AI account config and returns scoped env", async () => {
const missingAccount = createModels({ authContext: fakeAuthContext({ CLOUDFLARE_API_KEY: "cf-key" }) });
missingAccount.setProvider(cloudflareWorkersAIProvider());
const model = missingAccount.getModels("cloudflare-workers-ai")[0];
expect(await missingAccount.getAuth(model)).toBeUndefined();
const configured = createModels({
authContext: fakeAuthContext({ CLOUDFLARE_API_KEY: "cf-key", CLOUDFLARE_ACCOUNT_ID: "account-id" }),
});
configured.setProvider(cloudflareWorkersAIProvider());
const result = await configured.getAuth(model);
expect(result?.auth).toEqual({
apiKey: "cf-key",
baseUrl: "https://api.cloudflare.com/client/v4/accounts/account-id/ai/v1",
});
expect(result?.env).toEqual({ CLOUDFLARE_ACCOUNT_ID: "account-id" });
});
it("requires Cloudflare AI Gateway account and gateway config and returns scoped env headers", async () => {
const missingGateway = createModels({
authContext: fakeAuthContext({ CLOUDFLARE_API_KEY: "cf-key", CLOUDFLARE_ACCOUNT_ID: "account-id" }),
});
missingGateway.setProvider(cloudflareAIGatewayProvider());
const model = missingGateway.getModels("cloudflare-ai-gateway")[0];
expect(await missingGateway.getAuth(model)).toBeUndefined();
const configured = createModels({
authContext: fakeAuthContext({
CLOUDFLARE_API_KEY: "cf-key",
CLOUDFLARE_ACCOUNT_ID: "account-id",
CLOUDFLARE_GATEWAY_ID: "gateway-id",
}),
});
configured.setProvider(cloudflareAIGatewayProvider());
const result = await configured.getAuth(model);
expect(result?.auth).toEqual({
headers: {
"cf-aig-authorization": "Bearer cf-key",
Authorization: null,
"x-api-key": null,
},
baseUrl: "https://gateway.ai.cloudflare.com/v1/account-id/gateway-id/anthropic",
});
expect(result?.env).toEqual({
CLOUDFLARE_ACCOUNT_ID: "account-id",
CLOUDFLARE_GATEWAY_ID: "gateway-id",
});
});
it("resolves vertex via ADC file plus project and location", async () => { it("resolves vertex via ADC file plus project and location", async () => {
const adc = "~/.config/gcloud/application_default_credentials.json"; const adc = "~/.config/gcloud/application_default_credentials.json";
const configured = createModels({ const configured = createModels({
@@ -1,4 +1,4 @@
import type { Api, Model } from "@earendil-works/pi-ai"; import type { Api, Model, ProviderHeaders } from "@earendil-works/pi-ai";
import type { SettingsManager } from "./settings-manager.ts"; import type { SettingsManager } from "./settings-manager.ts";
import { isInstallTelemetryEnabled } from "./telemetry.ts"; import { isInstallTelemetryEnabled } from "./telemetry.ts";
@@ -92,9 +92,9 @@ export function mergeProviderAttributionHeaders(
model: Model<Api>, model: Model<Api>,
settingsManager: SettingsManager, settingsManager: SettingsManager,
sessionId: string | undefined, sessionId: string | undefined,
...headerSources: Array<Record<string, string> | undefined> ...headerSources: Array<ProviderHeaders | undefined>
): Record<string, string> | undefined { ): ProviderHeaders | undefined {
const merged = { const merged: ProviderHeaders = {
...getSessionHeaders(model, sessionId), ...getSessionHeaders(model, sessionId),
...getDefaultAttributionHeaders(model, settingsManager), ...getDefaultAttributionHeaders(model, settingsManager),
}; };
@@ -1,7 +1,7 @@
import { existsSync, mkdirSync, rmSync } from "node:fs"; import { existsSync, mkdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai"; import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest"; import { afterEach, describe, expect, it } from "vitest";
import { import {
type CreateAgentSessionRuntimeFactory, type CreateAgentSessionRuntimeFactory,
@@ -6,6 +6,7 @@ import {
type AssistantMessage, type AssistantMessage,
createAssistantMessageEventStream, createAssistantMessageEventStream,
type Model, type Model,
type ProviderHeaders,
type SimpleStreamOptions, type SimpleStreamOptions,
} from "@earendil-works/pi-ai"; } from "@earendil-works/pi-ai";
import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { afterEach, beforeEach, describe, expect, it } from "vitest";
@@ -88,7 +89,7 @@ describe("createAgentSession provider attribution headers", () => {
requestHeaders?: Record<string, string>; requestHeaders?: Record<string, string>;
sessionId?: string; sessionId?: string;
} = {}, } = {},
): Promise<Record<string, string> | undefined> { ): Promise<ProviderHeaders | undefined> {
const settingsManager = SettingsManager.create(cwd, agentDir); const settingsManager = SettingsManager.create(cwd, agentDir);
if (options.telemetryEnabled === false) { if (options.telemetryEnabled === false) {
settingsManager.setEnableInstallTelemetry(false); settingsManager.setEnableInstallTelemetry(false);
@@ -1,7 +1,7 @@
import { existsSync, mkdirSync, realpathSync, rmSync } from "node:fs"; import { existsSync, mkdirSync, realpathSync, rmSync } from "node:fs";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join, parse } from "node:path"; import { join, parse } from "node:path";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai"; import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest"; import { afterEach, describe, expect, it } from "vitest";
import { import {
type CreateAgentSessionRuntimeFactory, type CreateAgentSessionRuntimeFactory,
+7 -2
View File
@@ -7,8 +7,13 @@ import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import type { AgentMessage, AgentTool } from "@earendil-works/pi-agent-core"; import type { AgentMessage, AgentTool } from "@earendil-works/pi-agent-core";
import { Agent } from "@earendil-works/pi-agent-core"; import { Agent } from "@earendil-works/pi-agent-core";
import type { FauxModelDefinition, FauxProviderRegistration, FauxResponseStep, Model } from "@earendil-works/pi-ai"; import type {
import { registerFauxProvider } from "@earendil-works/pi-ai"; FauxModelDefinition,
FauxProviderRegistration,
FauxResponseStep,
Model,
} from "@earendil-works/pi-ai/compat";
import { registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { AgentSession, type AgentSessionEvent } from "../../src/core/agent-session.ts"; import { AgentSession, type AgentSessionEvent } from "../../src/core/agent-session.ts";
import { AuthStorage } from "../../src/core/auth-storage.ts"; import { AuthStorage } from "../../src/core/auth-storage.ts";
import type { ExtensionRunner } from "../../src/core/extensions/index.ts"; import type { ExtensionRunner } from "../../src/core/extensions/index.ts";
@@ -1,7 +1,7 @@
import { existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import { registerFauxProvider } from "@earendil-works/pi-ai"; import { registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest"; import { afterEach, describe, expect, it } from "vitest";
import { import {
type CreateAgentSessionRuntimeFactory, type CreateAgentSessionRuntimeFactory,
@@ -1,7 +1,7 @@
import { existsSync, mkdirSync, rmSync } from "node:fs"; import { existsSync, mkdirSync, rmSync } from "node:fs";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai"; import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest"; import { afterEach, describe, expect, it } from "vitest";
import type { AgentSession } from "../../../src/core/agent-session.ts"; import type { AgentSession } from "../../../src/core/agent-session.ts";
import { import {
@@ -2,7 +2,7 @@ import { existsSync, mkdtempSync, rmSync } from "node:fs";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import { Agent } from "@earendil-works/pi-agent-core"; import { Agent } from "@earendil-works/pi-agent-core";
import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai"; import { fauxAssistantMessage, registerFauxProvider } from "@earendil-works/pi-ai/compat";
import { afterEach, describe, expect, it } from "vitest"; import { afterEach, describe, expect, it } from "vitest";
import { AgentSession } from "../../../src/core/agent-session.ts"; import { AgentSession } from "../../../src/core/agent-session.ts";
import { AuthStorage } from "../../../src/core/auth-storage.ts"; import { AuthStorage } from "../../../src/core/auth-storage.ts";