diff --git a/docs/src/content/docs/en/configuration/profiles.md b/docs/src/content/docs/en/configuration/profiles.md index 9554bb1a..b5305683 100644 --- a/docs/src/content/docs/en/configuration/profiles.md +++ b/docs/src/content/docs/en/configuration/profiles.md @@ -93,7 +93,7 @@ The OpenCode CLI must be available as `opencode` in the CCR Desktop process envi | Environment variables | Injected into Codex CLI or ChatGPT. Claude Code-specific model discovery variables are not passed to Codex. | | Bot | Applies only to the ChatGPT app entry. | -After saving, use the terminal button on the config card to copy the Codex CLI command, for example `ccr-app "Codex - Work"`. Use the play button to open ChatGPT. Following the CodexL launch model, CCR starts the Electron executable inside the ChatGPT app bundle directly, gives it an isolated user-data directory, and points `CODEX_CLI_PATH` at the CCR middleware. The middleware forwards app-server traffic to ChatGPT's bundled Codex CLI and only adapts the account display. When the default Codex home has a valid, unexpired ChatGPT login, CCR bridges that token in memory without copying it into the isolated profile. Without a valid token, CCR reports a local non-OpenAI compatibility identity so current ChatGPT builds do not enter a repeated authentication and attestation loop. To make the native app-server select its official API marketplace, CCR creates the exact `ccr-local-profile` bootstrap only during process startup and removes it after the first native response; it is also cleaned after startup or abnormal exit and is never retained as login state. Every other authentication file is preserved. Older `Codex.app` installations remain supported. +After saving, use the terminal button on the config card to copy the Codex CLI command, for example `ccr-app "Codex - Work"`. Use the play button to open ChatGPT. Following the CodexL launch model, CCR starts the Electron executable inside the ChatGPT app bundle directly, gives it an isolated user-data directory, and points `CODEX_CLI_PATH` at the CCR middleware. The middleware forwards app-server traffic to ChatGPT's bundled Codex CLI and only adapts the account display. Each profile reads ChatGPT login state only from its own Codex home. A profile without credentials reports a local non-OpenAI compatibility identity so current ChatGPT builds do not enter a repeated authentication and attestation loop. Shared login is opt-in: set `CCR_CODEX_CHATGPT_AUTH_FILE` (or the legacy `CODEXL_CODEX_CHATGPT_AUTH_FILE`) to a valid `auth.json` when a profile should bridge that token in memory without copying it. To make the native app-server select its official API marketplace, CCR creates the exact `ccr-local-profile` bootstrap only during process startup and removes it after the first native response; it is also cleaned after startup or abnormal exit and is never retained as login state. Every other authentication file is preserved. Older `Codex.app` installations remain supported. Model and public plugin listings are not synthesized by the middleware. The native Codex app-server reads the generated `model_catalog_json` and handles `model/list` plus public `plugin/list` requests unchanged. This lets Codex refresh the official public [`openai/plugins`](https://github.com/openai/plugins) Git marketplace over the network. In a virtual workspace, only account-private marketplace requests are answered with an explicit empty result because the native service requires real ChatGPT authentication for those sections; they are never replaced with local plugins. Any downloaded Git checkout is owned only by Codex as its normal last-known-good data, not used by CCR as a replacement catalog. diff --git a/docs/src/content/docs/zh/configuration/profile.md b/docs/src/content/docs/zh/configuration/profile.md index f0a2bf22..4e1708ae 100644 --- a/docs/src/content/docs/zh/configuration/profile.md +++ b/docs/src/content/docs/zh/configuration/profile.md @@ -93,7 +93,7 @@ CCR Desktop 进程环境中必须能够执行 `opencode`。如果 CLI 安装在 | 环境变量 | 注入 Codex CLI 或 ChatGPT。Claude Code 专用的模型发现变量不会传给 Codex。 | | Bot | 只在 ChatGPT App 入口生效。 | -保存后,Codex CLI 使用配置卡片里的终端图标复制命令,例如 `ccr-app "Codex - Work"`。ChatGPT 使用播放图标打开。CCR 按照 CodexL 的启动方式,直接运行 ChatGPT App bundle 内的 Electron 可执行文件,为它设置隔离的用户数据目录,并把 `CODEX_CLI_PATH` 指向 CCR 中间层。中间层把 app-server 流量转发给 ChatGPT 内置的 Codex CLI,只适配账号展示:默认 Codex Home 已有有效且未过期的 ChatGPT 登录时,CCR 只在内存中桥接 token,不会把它复制进隔离配置;没有有效 token 时则返回本地非 OpenAI 兼容身份,避免新版 ChatGPT 进入循环鉴权和 attestation。为让原生 app-server 选择官方 API marketplace,CCR 只在进程启动阶段创建精确的 `ccr-local-profile` 引导标记,收到第一条原生响应后立即删除;正常启动后或异常退出时也会清理,不会把它保留成登录状态。其他认证文件全部保留。旧版 `Codex.app` 仍然兼容。 +保存后,Codex CLI 使用配置卡片里的终端图标复制命令,例如 `ccr-app "Codex - Work"`。ChatGPT 使用播放图标打开。CCR 按照 CodexL 的启动方式,直接运行 ChatGPT App bundle 内的 Electron 可执行文件,为它设置隔离的用户数据目录,并把 `CODEX_CLI_PATH` 指向 CCR 中间层。中间层把 app-server 流量转发给 ChatGPT 内置的 Codex CLI,只适配账号展示:每个配置只从自己的 Codex Home 读取 ChatGPT 登录状态;没有凭据时返回本地非 OpenAI 兼容身份,避免新版 ChatGPT 进入循环鉴权和 attestation。共享登录改为显式启用:只有设置 `CCR_CODEX_CHATGPT_AUTH_FILE`(或兼容变量 `CODEXL_CODEX_CHATGPT_AUTH_FILE`)并指向有效 `auth.json` 时,配置才会在内存中桥接该 token,且不会复制认证文件。为让原生 app-server 选择官方 API marketplace,CCR 只在进程启动阶段创建精确的 `ccr-local-profile` 引导标记,收到第一条原生响应后立即删除;正常启动后或异常退出时也会清理,不会把它保留成登录状态。其他认证文件全部保留。旧版 `Codex.app` 仍然兼容。 模型和公共插件列表不再由中间层合成。原生 Codex app-server 读取生成的 `model_catalog_json`,并原样处理 `model/list` 与公共 `plugin/list` 请求,因此 Codex 可以自行联网刷新官方公开 [`openai/plugins`](https://github.com/openai/plugins) Git marketplace。虚拟 workspace 中,只有必须使用真实 ChatGPT 鉴权的账号私有 marketplace 请求会得到明确空结果,绝不会用本地插件替代。下载后的 Git checkout 只作为 Codex 自己的常规 last-known-good 数据,CCR 不会拿它替代远端目录。 diff --git a/packages/core/src/agents/codex/app-launch.ts b/packages/core/src/agents/codex/app-launch.ts index ed1f59ec..fbccc265 100644 --- a/packages/core/src/agents/codex/app-launch.ts +++ b/packages/core/src/agents/codex/app-launch.ts @@ -337,10 +337,10 @@ function codexSharedChatGptAuthEnv(): Record { process.env.CCR_CODEX_CHATGPT_AUTH_FILE, process.env.CODEXL_CODEX_CHATGPT_AUTH_FILE ].map((value) => value?.trim()).find((value) => value && isFile(resolveUserPath(value))); - const authFile = configured ? resolveUserPath(configured) : path.join(os.homedir(), ".codex", "auth.json"); - if (!isFile(authFile)) { + if (!configured) { return {}; } + const authFile = resolveUserPath(configured); return { CCR_CODEX_CHATGPT_AUTH_FILE: authFile, CODEXL_CODEX_CHATGPT_AUTH_FILE: authFile diff --git a/packages/core/src/agents/codex/model-catalog.ts b/packages/core/src/agents/codex/model-catalog.ts index 56ab41f3..ecf42cae 100644 --- a/packages/core/src/agents/codex/model-catalog.ts +++ b/packages/core/src/agents/codex/model-catalog.ts @@ -9,6 +9,7 @@ import { import { codexDefaultBaseUrl, readCodexLocalModelCatalog } from "@ccr/core/agents/local-providers/codex"; import { localAgentProviderApiKey } from "@ccr/core/agents/local-providers/shared"; import { normalizeProviderBaseUrl } from "@ccr/core/providers/url"; +import { resolveUsageModelAttribution } from "@ccr/core/usage/model-attribution"; const fusionModelProviderName = "Fusion"; const codexDefaultContextWindow = 128_000; @@ -174,12 +175,24 @@ function codexModelCapabilityProfile( config?: Partial> ): CodexCapabilityProfile { const selector = parseModelSelector(model); - const provider = selector?.provider ? findConfiguredProvider(config, selector.provider) : findConfiguredProviderForModel(config, model); - const providerModel = selector?.model ?? model; + const attributionConfig = config + ? { + Providers: config.Providers ?? [], + virtualModelProfiles: config.virtualModelProfiles ?? [] + } + : undefined; + const attribution = resolveUsageModelAttribution(attributionConfig, model); + const provider = attribution.provider + ? findConfiguredProvider(config, attribution.provider) + : selector?.provider + ? findConfiguredProvider(config, selector.provider) + : findConfiguredProviderForModel(config, attribution.model ?? model); + const providerModel = attribution.model ?? selector?.model ?? model; const providerModelMetadata = provider ? providerModelMetadataFor(provider, providerModel) ?? localCodexModelMetadataFor(provider, providerModel) : undefined; - const catalogEntry = findModelCatalogEntry(model); + const physicalModelSelector = provider ? `${provider.name}/${providerModel}` : providerModel; + const catalogEntry = findModelCatalogEntry(physicalModelSelector); const capabilities = catalogEntry?.capabilities ?? {}; const providerProtocol = provider ? codexProviderProtocol(provider) : undefined; const supportsFusionVision = codexVirtualModelSupportsFusionVision(model, config); diff --git a/packages/core/src/agents/local-providers/grok.ts b/packages/core/src/agents/local-providers/grok.ts index 85f33420..7842de65 100644 --- a/packages/core/src/agents/local-providers/grok.ts +++ b/packages/core/src/agents/local-providers/grok.ts @@ -444,7 +444,7 @@ function importGrokProviderWithAuth( baseUrl: GROK_API_MEDIA_BASE_URL, endpoint: `${GROK_API_MEDIA_BASE_URL}/videos/generations`, source: "preset" as const, - type: "openai_video_generations" as const + type: "xai_video_generations" as const } ], models: uniqueStrings([ @@ -516,7 +516,11 @@ export function normalizeGrokProviderMediaCapabilities(provider: GatewayProvider if (!isLocalGrokProvider(provider)) { return provider; } - const capabilities = [...(provider.capabilities ?? [])]; + const capabilities = (provider.capabilities ?? []).map((capability) => + capability.type === "openai_video_generations" && isXaiApiBaseUrl(capability.baseUrl) + ? { ...capability, type: "xai_video_generations" as const } + : capability + ); const ensureCapability = (capability: GatewayProviderCapability): void => { if (!capabilities.some((item) => item.type === capability.type)) { capabilities.push(capability); @@ -534,7 +538,7 @@ export function normalizeGrokProviderMediaCapabilities(provider: GatewayProvider baseUrl: GROK_API_MEDIA_BASE_URL, endpoint: `${GROK_API_MEDIA_BASE_URL}/videos/generations`, source: "preset", - type: "openai_video_generations" + type: "xai_video_generations" }); return { ...provider, @@ -547,6 +551,14 @@ export function normalizeGrokProviderMediaCapabilities(provider: GatewayProvider }; } +function isXaiApiBaseUrl(value: string): boolean { + try { + return /(?:^|\.)api\.x\.ai$/i.test(new URL(value).hostname); + } catch { + return false; + } +} + function isLocalGrokProvider(provider: GatewayProviderConfig): boolean { if (providerApiKey(provider) !== localAgentProviderApiKey) { return false; diff --git a/packages/core/src/config/config.ts b/packages/core/src/config/config.ts index fa308acb..a81131f5 100644 --- a/packages/core/src/config/config.ts +++ b/packages/core/src/config/config.ts @@ -1459,6 +1459,9 @@ function parseProviderCapabilityProtocol(value: string | undefined): GatewayProv if (normalized === "openai_video_generations" || normalized === "openai_videos") { return "openai_video_generations"; } + if (normalized === "xai_video_generations" || normalized === "xai_videos") { + return "xai_video_generations"; + } if (normalized === "anthropic" || normalized === "anthropic_messages") { return "anthropic_messages"; } diff --git a/packages/core/src/contracts/app.ts b/packages/core/src/contracts/app.ts index b2a2b953..0d288ccf 100644 --- a/packages/core/src/contracts/app.ts +++ b/packages/core/src/contracts/app.ts @@ -153,7 +153,8 @@ export type GatewayProviderProtocol = export type GatewayMediaProtocol = | "openai_image_generations" - | "openai_video_generations"; + | "openai_video_generations" + | "xai_video_generations"; export type GatewayProviderCapabilityProtocol = GatewayProviderProtocol | GatewayMediaProtocol; diff --git a/packages/core/src/media/contracts.ts b/packages/core/src/media/contracts.ts index d07abf2c..91a5b037 100644 --- a/packages/core/src/media/contracts.ts +++ b/packages/core/src/media/contracts.ts @@ -51,10 +51,10 @@ export type ImageEditRequest = ImageGenerateRequest & { export type VideoGenerateRequest = { aspectRatio?: string; - duration: 6 | 10; + duration: number; images: string[]; prompt: string; - resolution: "480p" | "720p"; + resolution: string; }; export type MediaRequest = ImageEditRequest | ImageGenerateRequest | VideoGenerateRequest; diff --git a/packages/core/src/media/executors.ts b/packages/core/src/media/executors.ts index 74bcac55..690c7a5f 100644 --- a/packages/core/src/media/executors.ts +++ b/packages/core/src/media/executors.ts @@ -5,6 +5,7 @@ import { isIP } from "node:net"; import os from "node:os"; import path from "node:path"; import { setTimeout as delay } from "node:timers/promises"; +import type { GatewayMediaProtocol } from "@ccr/core/contracts/app"; import type { ImageEditRequest, ImageGenerateRequest, MediaExecutionContext, MediaExecutionResult, VideoGenerateRequest } from "@ccr/core/media/contracts"; import { detectMediaType } from "@ccr/core/media/storage"; import { fetchWithSystemProxy } from "@ccr/core/proxy/system-proxy-fetch"; @@ -14,6 +15,7 @@ const maxArtifactRedirects = 5; export type GatewayMediaTarget = { model: string; + protocol: GatewayMediaProtocol; providerBaseUrl: string; providerName: string; providerSelector: string; @@ -327,7 +329,7 @@ async function readApiResponse(response: Response): Promise= 500); } if (!isRecord(payload)) throw mediaError("invalid_api_response", "Media gateway returned a non-object response.", false); @@ -371,6 +373,32 @@ function sanitizeRemoteError(value: string): string { return value.replace(/Bearer\s+\S+/gi, "Bearer [redacted]").replace(/[A-Za-z0-9_-]{40,}/g, "[redacted]").trim().slice(0, 2000); } +function coreGatewayErrorMessage(payload: Record): string | undefined { + const error = isRecord(payload.error) ? payload.error : undefined; + const fallback = readString(error ?? payload, "message") ?? readString(payload, "message"); + const attempts = Array.isArray(error?.attempts) ? error.attempts.filter(isRecord) : []; + const failures = attempts + .map(formatGatewayAttempt) + .filter((value): value is string => Boolean(value)); + if (!failures.length) return fallback; + return [...new Set(failures)].slice(0, 3).join("; "); +} + +function formatGatewayAttempt(attempt: Record): string | undefined { + const message = readString(attempt, "message"); + const details = isRecord(attempt.details) ? attempt.details : undefined; + const detail = details ? readString(details, "message", "error", "raw", "code") : undefined; + const core = message && detail && !message.includes(detail) + ? `${message}: ${detail}` + : message ?? detail; + if (!core) return undefined; + const provider = readString(attempt, "provider_name", "provider"); + const stage = readString(attempt, "stage"); + const status = typeof attempt.status === "number" && Number.isFinite(attempt.status) ? `HTTP ${attempt.status}` : undefined; + const context = [provider, stage, status].filter(Boolean).join(", "); + return context ? `${core} (${context})` : core; +} + function stripUndefined(value: Record): Record { return Object.fromEntries(Object.entries(value).filter(([, item]) => item !== undefined)); } diff --git a/packages/core/src/media/models.ts b/packages/core/src/media/models.ts index 4f01d053..7f275844 100644 --- a/packages/core/src/media/models.ts +++ b/packages/core/src/media/models.ts @@ -3,12 +3,23 @@ import { GROK_API_DEFAULT_VIDEO_MODEL, GROK_CLI_MEDIA_MODEL_SELECTOR } from "@ccr/core/contracts/app"; -import type { GatewayProviderConfig } from "@ccr/core/contracts/app"; +import type { GatewayMediaProtocol, GatewayProviderConfig } from "@ccr/core/contracts/app"; const localAgentProviderApiKey = "ccr-local-agent-login"; export type GrokMediaKind = "image" | "video"; +export type VideoGenerationConstraints = { + aspectRatios: string[]; + defaultDuration?: number; + defaultResolution?: string; + durationMaximum?: number; + durationMinimum?: number; + durations?: number[]; + requiredParameters: Array<"aspect_ratio" | "duration" | "resolution">; + resolutions: string[]; +}; + export type GrokMediaModelOption = { label: string; value: string; @@ -47,12 +58,34 @@ export function grokMediaModelsForProvider(provider: GatewayProviderConfig, kind } export function providerSupportsMediaKind(provider: GatewayProviderConfig, kind: GrokMediaKind): boolean { - const capability = kind === "image" ? "openai_image_generations" : "openai_video_generations"; + const capabilities = kind === "image" + ? ["openai_image_generations"] + : ["xai_video_generations", "openai_video_generations"]; return isImportedGrokAgentProvider(provider) || - (provider.capabilities ?? []).some((item) => item.type === capability) || + (provider.capabilities ?? []).some((item) => capabilities.includes(item.type)) || (provider.models ?? []).some((model) => grokMediaModelKind(model) === kind); } +export function videoGenerationConstraints(protocol: GatewayMediaProtocol): VideoGenerationConstraints { + if (protocol === "openai_video_generations") { + return { + aspectRatios: ["16:9", "9:16"], + durations: [4, 8, 12], + requiredParameters: ["duration", "aspect_ratio", "resolution"], + resolutions: ["720p"] + }; + } + return { + aspectRatios: ["1:1", "16:9", "9:16", "4:3", "3:4", "3:2", "2:3"], + defaultDuration: 6, + defaultResolution: "480p", + durationMaximum: 15, + durationMinimum: 1, + requiredParameters: [], + resolutions: ["480p", "720p"] + }; +} + export function createGrokMediaModelOptions( providers: GatewayProviderConfig[], kind: GrokMediaKind diff --git a/packages/core/src/media/service.ts b/packages/core/src/media/service.ts index 48119fe8..e6465362 100644 --- a/packages/core/src/media/service.ts +++ b/packages/core/src/media/service.ts @@ -7,7 +7,7 @@ import type { AppConfig, GatewayMediaProtocol, MediaToolsConfig } from "@ccr/cor import type { ImageEditRequest, ImageGenerateRequest, MediaArtifact, MediaExecutionContext, MediaExecutionResult, MediaJob, MediaJobError, MediaOperation, MediaRequest, PublicMediaArtifact, PublicMediaJob, VideoGenerateRequest } from "@ccr/core/media/contracts"; import { GatewayMediaExecutor } from "@ccr/core/media/executors"; import type { GatewayMediaTarget, GatewayMediaTransport } from "@ccr/core/media/executors"; -import { grokMediaModelKind, isImportedGrokAgentProvider, migrateLegacyGrokMediaModelSelector, providerSupportsMediaKind } from "@ccr/core/media/models"; +import { grokMediaModelKind, isImportedGrokAgentProvider, migrateLegacyGrokMediaModelSelector, providerSupportsMediaKind, videoGenerationConstraints } from "@ccr/core/media/models"; import { detectMediaType, MediaArtifactStore, MediaJobStore } from "@ccr/core/media/storage"; import { mediaToolBindingsForConfig } from "@ccr/core/media/tools"; import type { MediaToolBinding } from "@ccr/core/media/tools"; @@ -129,12 +129,32 @@ export class MediaService { } videoStart(args: Record, modelSelector: string): PublicMediaJob { - const durationValue = numberValue(args.duration) ?? 6; - if (durationValue !== 6 && durationValue !== 10) throw new Error("duration must be 6 or 10 seconds."); - const resolution = optionalString(args.resolution) ?? "480p"; - if (resolution !== "480p" && resolution !== "720p") throw new Error("resolution must be 480p or 720p."); + const target = resolveProviderMediaTarget(this.requireConfig(), modelSelector, "video-generate"); + const constraints = videoGenerationConstraints(target.protocol); + const durationValue = numberValue(args.duration) ?? constraints.defaultDuration; + if (durationValue === undefined) throw new Error(`duration is required for ${target.protocol}.`); + if (!Number.isInteger(durationValue)) throw new Error("duration must be an integer number of seconds."); + if (constraints.durations && !constraints.durations.includes(durationValue)) { + throw new Error(`duration must be one of ${constraints.durations.join(", ")} seconds for ${target.protocol}.`); + } + if (constraints.durationMinimum !== undefined && constraints.durationMaximum !== undefined && + (durationValue < constraints.durationMinimum || durationValue > constraints.durationMaximum)) { + throw new Error(`duration must be between ${constraints.durationMinimum} and ${constraints.durationMaximum} seconds for ${target.protocol}.`); + } + const resolution = optionalString(args.resolution) ?? constraints.defaultResolution; + if (!resolution) throw new Error(`resolution is required for ${target.protocol}.`); + if (!constraints.resolutions.includes(resolution)) { + throw new Error(`resolution must be one of ${constraints.resolutions.join(", ")} for ${target.protocol}.`); + } + const aspectRatio = optionalString(args.aspect_ratio); + if (constraints.requiredParameters.includes("aspect_ratio") && !aspectRatio) { + throw new Error(`aspect_ratio is required for ${target.protocol}.`); + } + if (aspectRatio && !constraints.aspectRatios.includes(aspectRatio)) { + throw new Error(`aspect_ratio must be one of ${constraints.aspectRatios.join(", ")} for ${target.protocol}.`); + } const request: VideoGenerateRequest = { - aspectRatio: optionalString(args.aspect_ratio), + aspectRatio, duration: durationValue, images: args.images === undefined && args.image === undefined ? [] : this.validateImages(args.images ?? args.image, 1, 7), prompt: requiredPrompt(args.prompt), @@ -162,15 +182,22 @@ export class MediaService { capabilities(): Record { const config = this.requireConfig(); const runtime = config.mediaTools; + const bindings = this.toolBindings(); return { backend: "gateway-media-api", - bindings: this.toolBindings(), + bindings, constraints: { imageEditMaxInputs: 3, inputFileMaxBytes: maxInputBytes, videoReferenceMaxInputs: 7, - videoDurations: [6, 10], - videoResolutions: ["480p", "720p"] + videoTools: bindings + .filter((binding) => binding.operation === "video-generate" && binding.protocol) + .map((binding) => ({ + modelSelector: binding.modelSelector, + name: binding.name, + protocol: binding.protocol, + ...videoGenerationConstraints(binding.protocol!) + })) }, enabled: runtime.enabled, operations: ["image-generate", "image-edit", "video-generate", "image-to-video", "reference-to-video"] @@ -178,7 +205,11 @@ export class MediaService { } toolBindings(): MediaToolBinding[] { - return mediaToolBindingsForConfig(this.requireConfig()); + return mediaToolBindingsForConfig(this.requireConfig()).map((binding) => + binding.operation === "video-generate" + ? { ...binding, protocol: resolveProviderMediaTarget(this.requireConfig(), binding.modelSelector, binding.operation).protocol } + : binding + ); } bindingForTool(name: string): MediaToolBinding | undefined { @@ -536,9 +567,14 @@ export function resolveProviderMediaTarget(config: AppConfig, selector: string, throw new Error(`Provider ${provider.name} does not declare ${kind ?? "media"} generation support.`); } const protocol: GatewayMediaProtocol = kind === "video" - ? "openai_video_generations" + ? provider.capabilities?.some((item) => item.type === "xai_video_generations") || isImportedGrokAgentProvider(provider) + ? "xai_video_generations" + : "openai_video_generations" : "openai_image_generations"; - const capability = provider.capabilities?.find((item) => item.type === protocol); + const capability = provider.capabilities?.find((item) => item.type === protocol) ?? + (protocol === "xai_video_generations" && isImportedGrokAgentProvider(provider) + ? provider.capabilities?.find((item) => item.type === "openai_video_generations") + : undefined); const providerBaseUrl = capability?.baseUrl ?? provider.baseurl ?? provider.baseUrl ?? provider.api_base_url; if (!providerBaseUrl?.trim()) { throw new Error(`Provider ${provider.name} does not configure a media API base URL.`); @@ -549,6 +585,7 @@ export function resolveProviderMediaTarget(config: AppConfig, selector: string, const credential = sortProviderCredentialsForConfig(activeProviderCredentials(provider))[0]; return { model: resolved.model, + protocol, providerBaseUrl: providerBaseUrl.trim(), providerName: provider.name, providerSelector: credential diff --git a/packages/core/src/media/tools.ts b/packages/core/src/media/tools.ts index a741cf4f..027b8c0a 100644 --- a/packages/core/src/media/tools.ts +++ b/packages/core/src/media/tools.ts @@ -6,14 +6,15 @@ import { GROK_MEDIA_JOB_GET_TOOL_NAME, GROK_MEDIA_VIDEO_START_TOOL_NAME } from "@ccr/core/contracts/app"; -import type { AppConfig } from "@ccr/core/contracts/app"; +import type { AppConfig, GatewayMediaProtocol } from "@ccr/core/contracts/app"; import type { MediaOperation } from "@ccr/core/media/contracts"; -import { defaultGrokMediaModelSelector, migrateLegacyGrokMediaModelSelector } from "@ccr/core/media/models"; +import { defaultGrokMediaModelSelector, migrateLegacyGrokMediaModelSelector, videoGenerationConstraints } from "@ccr/core/media/models"; export type MediaToolBinding = { modelSelector: string; name: string; operation: MediaOperation | "capabilities" | "job-cancel" | "job-get"; + protocol?: GatewayMediaProtocol; }; export type MediaMcpToolDefinition = { @@ -87,18 +88,30 @@ export function mediaMcpToolDefinition(binding: MediaToolBinding): MediaMcpToolD }, ["images", "prompt"]), name: binding.name }; - if (binding.operation === "video-generate") return { - description: `Start a media provider video job with ${binding.modelSelector}. Returns immediately with a job id. Supply zero images for text-to-video, one for image-to-video, or two to seven reference images.`, - inputSchema: objectSchema({ - aspect_ratio: { description: "Optional output aspect ratio.", type: "string" }, - duration: { description: "Video duration in seconds.", enum: [6, 10], type: "number" }, - idempotency_key: { description: "Stable caller-generated key that prevents duplicate paid submissions.", type: "string" }, - images: { description: "Up to seven absolute local image paths.", items: { type: "string" }, maxItems: 7, type: "array" }, - prompt: { description: "Video generation prompt.", maxLength: 20000, type: "string" }, - resolution: { description: "Requested output resolution.", enum: ["480p", "720p"], type: "string" } - }, ["prompt"]), - name: binding.name - }; + if (binding.operation === "video-generate") { + const protocol = binding.protocol ?? "xai_video_generations"; + const constraints = videoGenerationConstraints(protocol); + const durationSchema = constraints.durations + ? { description: `Video duration in seconds for ${protocol}.`, enum: constraints.durations, type: "integer" } + : { + description: `Video duration in seconds for ${protocol}.`, + maximum: constraints.durationMaximum, + minimum: constraints.durationMinimum, + type: "integer" + }; + return { + description: `Start a ${protocol} video job with ${binding.modelSelector}. Returns immediately with a job id. Supply zero images for text-to-video, one for image-to-video, or two to seven reference images.`, + inputSchema: objectSchema({ + aspect_ratio: { description: `Output aspect ratio for ${protocol}.`, enum: constraints.aspectRatios, type: "string" }, + duration: durationSchema, + idempotency_key: { description: "Stable caller-generated key that prevents duplicate paid submissions.", type: "string" }, + images: { description: "Up to seven absolute local image paths.", items: { type: "string" }, maxItems: 7, type: "array" }, + prompt: { description: "Video generation prompt.", maxLength: 20000, type: "string" }, + resolution: { description: `Requested output resolution for ${protocol}.`, enum: constraints.resolutions, type: "string" } + }, ["prompt", ...constraints.requiredParameters]), + name: binding.name + }; + } if (binding.operation === "job-get") return { description: "Get the current state and artifact of a media job.", inputSchema: objectSchema({ job_id: { description: "Job id returned by a start or image call.", type: "string" } }, ["job_id"]), diff --git a/packages/core/src/providers/probe.ts b/packages/core/src/providers/probe.ts index 2ade2ef5..aeb75c08 100644 --- a/packages/core/src/providers/probe.ts +++ b/packages/core/src/providers/probe.ts @@ -70,7 +70,8 @@ const protocolOrder: GatewayProviderCapabilityProtocol[] = [ "gemini_generate_content", "gemini_interactions", "openai_image_generations", - "openai_video_generations" + "openai_video_generations", + "xai_video_generations" ]; const modelSourceOrder: ModelSource[] = ["openai", "anthropic", "gemini"]; @@ -718,7 +719,7 @@ function mediaProbeBody(protocol: GatewayProviderCapabilityProtocol): Record ({ baseUrl, endpoint: `${baseUrl}/videos/generations` @@ -1087,7 +1088,9 @@ function protocolModelSource(protocol: GatewayProviderCapabilityProtocol): Model } function isChatProtocol(protocol: GatewayProviderCapabilityProtocol): protocol is GatewayProviderProtocol { - return protocol !== "openai_image_generations" && protocol !== "openai_video_generations"; + return protocol !== "openai_image_generations" && + protocol !== "openai_video_generations" && + protocol !== "xai_video_generations"; } function isMediaProtocol(protocol: GatewayProviderCapabilityProtocol): boolean { diff --git a/packages/core/src/providers/runtime-topology.ts b/packages/core/src/providers/runtime-topology.ts index 55624b99..d95ef32a 100644 --- a/packages/core/src/providers/runtime-topology.ts +++ b/packages/core/src/providers/runtime-topology.ts @@ -431,6 +431,9 @@ export function normalizeProviderCapabilityProtocol(value: unknown): GatewayProv if (normalized === "openai_video_generations" || normalized === "openai_videos") { return "openai_video_generations"; } + if (normalized === "xai_video_generations" || normalized === "xai_videos") { + return "xai_video_generations"; + } return undefined; } diff --git a/packages/core/src/routing/model-registry.ts b/packages/core/src/routing/model-registry.ts index 6e7801ff..ec042b5a 100644 --- a/packages/core/src/routing/model-registry.ts +++ b/packages/core/src/routing/model-registry.ts @@ -230,7 +230,8 @@ const providerInternalProtocols = new Set([ "openai_chat_completions", "openai_image_generations", "openai_responses", - "openai_video_generations" + "openai_video_generations", + "xai_video_generations" ]); function sanitizeProviderHeaderId(value: string | undefined): string | undefined { diff --git a/packages/core/test/integration/mcp/grok-media-service.test.mjs b/packages/core/test/integration/mcp/grok-media-service.test.mjs index 7cb20f07..142645dc 100644 --- a/packages/core/test/integration/mcp/grok-media-service.test.mjs +++ b/packages/core/test/integration/mcp/grok-media-service.test.mjs @@ -8,6 +8,7 @@ import { normalizeGrokProviderMediaCapabilities } from "@ccr/core/agents/local-p import { createDefaultAppConfig } from "@ccr/core/config/default-config.ts"; import { GatewayMediaExecutor } from "@ccr/core/media/executors.ts"; import { MediaService, mediaServiceForTest, resolveProviderMediaTarget } from "@ccr/core/media/service.ts"; +import { mediaMcpToolDefinition } from "@ccr/core/media/tools.ts"; import { MEDIA_ARTIFACT_PATH_PREFIX, handleMediaArtifactRequest, handleMediaToolsMcpRequest } from "@ccr/core/mcp/grok-media-mcp.ts"; const png = Buffer.concat([ @@ -33,7 +34,7 @@ test("media tools bind profile-specific runtime names to gateway media models", assert.deepEqual(service.toolBindings(), [ { modelSelector: "Media Provider/grok-imagine-image-quality", name: "image_generate_test", operation: "image-generate" }, { modelSelector: "Media Provider/grok-imagine-image-quality", name: "image_edit_test", operation: "image-edit" }, - { modelSelector: "Media Provider/grok-imagine-video", name: "video_generate_test", operation: "video-generate" }, + { modelSelector: "Media Provider/grok-imagine-video", name: "video_generate_test", operation: "video-generate", protocol: "xai_video_generations" }, { modelSelector: "Media Provider/grok-imagine-video", name: "media_job_get_test", operation: "job-get" }, { modelSelector: "Media Provider/grok-imagine-video", name: "media_job_cancel_test", operation: "job-cancel" } ]); @@ -49,6 +50,7 @@ test("media tools bind profile-specific runtime names to gateway media models", test("media artifact downloads reject private-network URLs from public providers", async () => { const executor = new GatewayMediaExecutor({ model: "image-model", + protocol: "openai_image_generations", providerBaseUrl: "https://8.8.8.8/v1", providerName: "Public Media Provider", providerSelector: "public-media::openai_image_generations" @@ -61,6 +63,7 @@ test("media artifact downloads reject private-network URLs from public providers const localExecutor = new GatewayMediaExecutor({ model: "image-model", + protocol: "openai_image_generations", providerBaseUrl: "http://127.0.0.1:3000/v1", providerName: "Local Media Provider", providerSelector: "local-media::openai_image_generations" @@ -71,6 +74,48 @@ test("media artifact downloads reject private-network URLs from public providers ); }); +test("media gateway errors expose the concrete failed provider attempt", async (t) => { + const server = createServer(async (_request, response) => { + response.writeHead(400, { "content-type": "application/json" }); + response.end(JSON.stringify({ + error: { + attempts: [{ + message: "xAI video duration \"6\" cannot be represented by OpenAI, which supports 4, 8, or 12 seconds.", + provider: "openai", + provider_name: "grok-cli-api::openai_video_generations", + stage: "upstream_request_build", + status: 400 + }], + message: "All target providers failed." + } + })); + }); + if (!await listenOrSkip(t, server)) return; + t.after(() => server.close()); + + const executor = new GatewayMediaExecutor({ + model: "grok-imagine-video", + protocol: "xai_video_generations", + providerBaseUrl: "https://api.x.ai/v1", + providerName: "Grok CLI API", + providerSelector: "grok-cli-api::xai_video_generations" + }, { baseUrl: baseUrl(server) }); + await assert.rejects( + executor.videoGenerate({ duration: 6, images: [], prompt: "Animate", resolution: "480p" }, { + job: { id: "media-error-test" }, + onRemoteRequestId() {}, + signal: new AbortController().signal + }), + (error) => { + assert.equal(error.code, "gateway_http_400"); + assert.match(error.message, /xAI video duration "6" cannot be represented by OpenAI/); + assert.match(error.message, /upstream_request_build/); + assert.doesNotMatch(error.message, /All target providers failed/); + return true; + } + ); +}); + test("implicit media input roots reject the filesystem root and home directory", () => { const home = path.resolve(os.homedir()); assert.equal(mediaServiceForTest.isSafeImplicitWorkingDirectory(path.parse(home).root, home), false); @@ -78,6 +123,20 @@ test("implicit media input roots reject the filesystem root and home directory", assert.equal(mediaServiceForTest.isSafeImplicitWorkingDirectory(path.join(home, "workspace", "project"), home), true); }); +test("OpenAI video bindings expose the exact cross-protocol parameter subset", () => { + const tool = mediaMcpToolDefinition({ + modelSelector: "OpenAI/sora", + name: "video_generate_openai", + operation: "video-generate", + protocol: "openai_video_generations" + }); + + assert.deepEqual(tool.inputSchema.properties.duration.enum, [4, 8, 12]); + assert.deepEqual(tool.inputSchema.properties.aspect_ratio.enum, ["16:9", "9:16"]); + assert.deepEqual(tool.inputSchema.properties.resolution.enum, ["720p"]); + assert.deepEqual(tool.inputSchema.required, ["prompt", "duration", "aspect_ratio", "resolution"]); +}); + test("an imported Grok Agent supplies OAuth-backed Grok API media models without an API key", async (t) => { const root = mkdtempSync(path.join(os.tmpdir(), "ccr-grok-agent-media-")); const config = createDefaultAppConfig({ generatedConfigFile: "/tmp/ccr-gateway.config.json" }); @@ -114,7 +173,7 @@ test("an imported Grok Agent supplies OAuth-backed Grok API media models without assert.deepEqual(service.toolBindings(), [ { modelSelector: "Imported Grok/grok-imagine-image-quality", name: "image_generate_imported", operation: "image-generate" }, - { modelSelector: "Imported Grok/grok-imagine-video", name: "video_generate_imported", operation: "video-generate" } + { modelSelector: "Imported Grok/grok-imagine-video", name: "video_generate_imported", operation: "video-generate", protocol: "xai_video_generations" } ]); const target = resolveProviderMediaTarget(config, "Imported Grok/grok-imagine-image-quality", "image-generate"); assert.equal(target.model, "grok-imagine-image-quality"); @@ -189,6 +248,13 @@ test("provider image jobs use the internal media gateway, persist artifacts, and "media_job_get_test", "media_job_cancel_test" ]); + const videoTool = listPayload.result.tools.find((tool) => tool.name === "video_generate_test"); + assert.equal(videoTool.inputSchema.properties.duration.type, "integer"); + assert.equal(videoTool.inputSchema.properties.duration.minimum, 1); + assert.equal(videoTool.inputSchema.properties.duration.maximum, 15); + assert.deepEqual(videoTool.inputSchema.properties.aspect_ratio.enum, ["1:1", "16:9", "9:16", "4:3", "3:4", "3:2", "2:3"]); + assert.deepEqual(videoTool.inputSchema.properties.resolution.enum, ["480p", "720p"]); + assert.deepEqual(videoTool.inputSchema.required, ["prompt"]); const callResponse = await fetch(`${baseUrl(server)}/mcp`, { body: JSON.stringify({ @@ -312,7 +378,7 @@ function mediaConfig(baseUrlValue) { capabilities: [ { baseUrl: `${baseUrlValue}/v1`, source: "detected", type: "openai_chat_completions" }, { baseUrl: `${baseUrlValue}/v1`, source: "detected", type: "openai_image_generations" }, - { baseUrl: `${baseUrlValue}/v1`, source: "detected", type: "openai_video_generations" } + { baseUrl: `${baseUrlValue}/v1`, source: "detected", type: "xai_video_generations" } ], models: ["grok-imagine-image-quality", "grok-imagine-video"], name: "Media Provider" diff --git a/packages/core/test/unit/agents/codex-app-model-catalog.test.mjs b/packages/core/test/unit/agents/codex-app-model-catalog.test.mjs index 2d65c1ff..021dcc18 100644 --- a/packages/core/test/unit/agents/codex-app-model-catalog.test.mjs +++ b/packages/core/test/unit/agents/codex-app-model-catalog.test.mjs @@ -11,16 +11,18 @@ import { writeCodexCompatibleAppModelCatalog } from "@ccr/core/agents/codex/app-launch.ts"; -test("ChatGPT app launch bridges an existing shared Codex login without copying it", () => { +test("ChatGPT app launch shares Codex login only when an auth file is explicitly configured", () => { const root = mkdtempSync(path.join(os.tmpdir(), "ccr-chatgpt-shared-auth-")); const authFile = path.join(root, "auth.json"); const previousCcr = process.env.CCR_CODEX_CHATGPT_AUTH_FILE; const previousCodexl = process.env.CODEXL_CODEX_CHATGPT_AUTH_FILE; try { writeFileSync(authFile, JSON.stringify({ auth_mode: "chatgpt", tokens: { access_token: "token" } })); - process.env.CCR_CODEX_CHATGPT_AUTH_FILE = authFile; + delete process.env.CCR_CODEX_CHATGPT_AUTH_FILE; delete process.env.CODEXL_CODEX_CHATGPT_AUTH_FILE; + assert.deepEqual(codexSharedChatGptAuthEnvForTest(), {}); + process.env.CCR_CODEX_CHATGPT_AUTH_FILE = authFile; assert.deepEqual(codexSharedChatGptAuthEnvForTest(), { CCR_CODEX_CHATGPT_AUTH_FILE: authFile, CODEXL_CODEX_CHATGPT_AUTH_FILE: authFile diff --git a/packages/core/test/unit/agents/codex-model-catalog.test.mjs b/packages/core/test/unit/agents/codex-model-catalog.test.mjs index d659850d..56c8fdbc 100644 --- a/packages/core/test/unit/agents/codex-model-catalog.test.mjs +++ b/packages/core/test/unit/agents/codex-model-catalog.test.mjs @@ -516,3 +516,93 @@ test("codex catalog marks prefixed Fusion virtual models with legacy web search assert.equal(model.web_search_tool_type, "text"); assert.equal(model.apply_patch_tool_type, "freeform"); }); + +test("codex catalog inherits reasoning levels from each Fusion alias base model", () => { + const cases = [ + ["gpt", "gpt-5.6-sol", ["low", "medium", "high", "xhigh", "max", "ultra"], "medium"], + ["anthropic", "claude-sonnet-4.6", ["low", "medium", "high", "max"], "high"], + ["google", "gemini-3.5-flash", ["minimal", "low", "medium", "high"], "medium"], + ["kimi", "kimi-for-coding", [], null] + ]; + + for (const [providerName, baseModel, expectedEfforts, expectedDefault] of cases) { + const alias = `fusion-${providerName}`; + const model = catalogModelFor({ + Providers: [ + { name: providerName, type: "openai_responses", models: [baseModel] } + ], + virtualModelProfiles: [ + { + baseModel: { fixedModel: `${providerName}/${baseModel}`, mode: "fixed" }, + displayName: alias, + enabled: true, + execution: { + clientToolsPolicy: "allow", + maxToolCalls: 4, + maxTurns: 4, + mode: "tool_loop", + streamMode: "optimistic" + }, + id: alias, + key: alias, + match: { exactAliases: [alias], prefixes: [], suffixes: [] }, + materialization: { enabled: true, includeInGatewayModels: true }, + tools: [] + } + ] + }, `Fusion/${alias}`); + + assert.deepEqual(model.supported_reasoning_levels.map((level) => level.effort), expectedEfforts); + assert.equal(model.default_reasoning_level, expectedDefault); + } +}); + +test("codex catalog inherits explicit provider reasoning metadata through Fusion aliases", () => { + const model = catalogModelFor({ + Providers: [ + { + modelMetadata: { + "custom-base": { + contextWindow: 320000, + defaultReasoningLevel: "high", + supportedReasoningLevels: [ + { description: "Quick", effort: "low" }, + { description: "Thorough", effort: "high" } + ], + supportsReasoningSummaries: true + } + }, + models: ["custom-base"], + name: "Custom Provider", + type: "openai_responses" + } + ], + virtualModelProfiles: [ + { + baseModel: { fixedModel: "Custom Provider/custom-base", mode: "fixed" }, + displayName: "Custom Fusion", + enabled: true, + execution: { + clientToolsPolicy: "allow", + maxToolCalls: 4, + maxTurns: 4, + mode: "tool_loop", + streamMode: "optimistic" + }, + id: "custom-fusion", + key: "custom-fusion", + match: { exactAliases: ["custom-fusion"], prefixes: [], suffixes: [] }, + materialization: { enabled: true, includeInGatewayModels: true }, + tools: [] + } + ] + }, "Fusion/custom-fusion"); + + assert.deepEqual(model.supported_reasoning_levels, [ + { description: "Quick", effort: "low" }, + { description: "Thorough", effort: "high" } + ]); + assert.equal(model.default_reasoning_level, "high"); + assert.equal(model.supports_reasoning_summaries, true); + assert.equal(model.context_window, 320000); +}); diff --git a/packages/core/test/unit/agents/local-agent-provider-grok.test.mjs b/packages/core/test/unit/agents/local-agent-provider-grok.test.mjs index 81a36290..58c2353d 100644 --- a/packages/core/test/unit/agents/local-agent-provider-grok.test.mjs +++ b/packages/core/test/unit/agents/local-agent-provider-grok.test.mjs @@ -42,7 +42,7 @@ test("Grok local provider imports bearer token and model override plugin", async assert.equal(result.provider.apiKey, "ccr-local-agent-login"); assert.deepEqual( result.provider.capabilities.map((capability) => capability.type), - ["openai_responses", "openai_image_generations", "openai_video_generations"] + ["openai_responses", "openai_image_generations", "xai_video_generations"] ); assert.ok(result.provider.models.includes("grok-imagine-image-quality")); assert.ok(result.provider.models.includes("grok-imagine-video")); @@ -269,12 +269,31 @@ test("persisted Grok Agent providers are upgraded with gateway media capabilitie assert.deepEqual(provider.capabilities?.map((capability) => capability.type), [ "openai_responses", "openai_image_generations", - "openai_video_generations" + "xai_video_generations" ]); assert.ok(provider.models.includes("grok-imagine-image-quality")); assert.ok(provider.models.includes("grok-imagine-video")); }); +test("persisted Grok Agent providers migrate legacy xAI video capabilities", () => { + const provider = normalizeGrokProviderMediaCapabilities({ + api_base_url: grokDefaultBaseUrl, + api_key: localAgentProviderApiKey, + capabilities: [{ + baseUrl: "https://api.x.ai/v1", + endpoint: "https://api.x.ai/v1/videos/generations", + source: "preset", + type: "openai_video_generations" + }], + models: ["grok-imagine-video"], + name: "Grok CLI API", + protocol: "openai_responses" + }); + + assert.equal(provider.capabilities?.some((capability) => capability.type === "openai_video_generations"), false); + assert.equal(provider.capabilities?.some((capability) => capability.type === "xai_video_generations"), true); +}); + async function withGrokHome(run) { const previousGrokHome = process.env.GROK_HOME; const previousGrokAuthFile = process.env.GROK_AUTH_FILE; diff --git a/packages/core/test/unit/gateway/gateway-media-config.test.mjs b/packages/core/test/unit/gateway/gateway-media-config.test.mjs index 8a1a4c1e..14aeb065 100644 --- a/packages/core/test/unit/gateway/gateway-media-config.test.mjs +++ b/packages/core/test/unit/gateway/gateway-media-config.test.mjs @@ -20,7 +20,8 @@ test("core gateway compiles media capabilities and provider plugin aliases for c capabilities: [ { baseUrl: "https://chat.example/v1", source: "detected", type: "openai_chat_completions" }, { baseUrl: "https://media.example/v1", source: "detected", type: "openai_image_generations" }, - { baseUrl: "https://media.example/v1", source: "detected", type: "openai_video_generations" } + { baseUrl: "https://media.example/v1", source: "detected", type: "openai_video_generations" }, + { baseUrl: "https://api.x.ai/v1", source: "detected", type: "xai_video_generations" } ], credentials: [{ apiKey: "provider-key", enabled: true, id: "primary", priority: 1 }], id: "media-provider", @@ -50,7 +51,8 @@ test("core gateway compiles media capabilities and provider plugin aliases for c assert.deepEqual(providers.map((provider) => [provider.name, provider.type, provider.baseurl]), [ ["media-provider::openai_chat_completions::cred:primary", "openai_chat_completions", "https://chat.example/v1"], ["media-provider::openai_image_generations::cred:primary", "openai_image_generations", "https://media.example/v1"], - ["media-provider::openai_video_generations::cred:primary", "openai_video_generations", "https://media.example/v1"] + ["media-provider::openai_video_generations::cred:primary", "openai_video_generations", "https://media.example/v1"], + ["media-provider::xai_video_generations::cred:primary", "xai_video_generations", "https://api.x.ai/v1"] ]); assert.deepEqual( providerPlugins.map((plugin) => plugin.providerName).filter(Boolean).sort(), @@ -58,7 +60,8 @@ test("core gateway compiles media capabilities and provider plugin aliases for c "Media Provider", "media-provider::openai_chat_completions::cred:primary", "media-provider::openai_image_generations::cred:primary", - "media-provider::openai_video_generations::cred:primary" + "media-provider::openai_video_generations::cred:primary", + "media-provider::xai_video_generations::cred:primary" ].sort() ); }); diff --git a/packages/ui/src/pages/home/shared/providers.ts b/packages/ui/src/pages/home/shared/providers.ts index 546e08b3..a0a9f0d5 100644 --- a/packages/ui/src/pages/home/shared/providers.ts +++ b/packages/ui/src/pages/home/shared/providers.ts @@ -2054,7 +2054,9 @@ export function providerCapabilitiesForProtocols( .filter((item): item is GatewayProviderCapability => Boolean(item)); const detectedMediaCapabilities = detectedCapabilities.filter((capability) => - capability.type === "openai_image_generations" || capability.type === "openai_video_generations" + capability.type === "openai_image_generations" || + capability.type === "openai_video_generations" || + capability.type === "xai_video_generations" ); return mergeProviderCapabilities(selectedCapabilities, detectedMediaCapabilities); }