qwen-code/packages/sdk-python/tests/unit/test_validation.py
Dinesh Palanisamy 85ae860cce
feat(core): add OpenAI Responses API content generator (#8169)
* feat(core): add OpenAI Responses API content generator

Adds an HTTP/SSE-only content generator for OpenAI's Responses API
(/v1/responses), selected via AuthType.USE_OPENAI_RESPONSES. Reasoning
continuity round-trips through part.thoughtSignature, matching the
existing Anthropic thinking-signature mechanism, so compaction and
history consolidation work unchanged.

Closes #889

* fix(cli): add openai-responses to AUTH_PREFLIGHT_ENV_KEYS

AUTH_PREFLIGHT_ENV_KEYS didn't have an entry for the new
AuthType.USE_OPENAI_RESPONSES ('openai-responses'), and it isn't in
AUTH_PREFLIGHT_WAIVED_AUTH_TYPES either, so the drift-detection test
(AUTH_PREFLIGHT_AUDITED_AUTH_TYPES covers every AuthType) failed with
expect(uncovered).toEqual([]).

OPENAI_API_KEY is the correct env var, matching the existing mapping
for this auth type in packages/cli/src/config/auth.ts:23
([AuthType.USE_OPENAI_RESPONSES]: 'OPENAI_API_KEY') and
packages/core/src/models/constants.ts's AUTH_ENV_MAPPINGS.

Verification: authPreflight.test.ts 3/3 passing (was failing before
this fix), tsc --noEmit clean, eslint clean.

* fix(openai-responses): address review findings on the streaming pipeline and converter

Addresses the automated /review findings (doudouOUC, qwen-code-ci-bot) left
on #8169 after the AUTH_PREFLIGHT_ENV_KEYS fix.

Critical fixes in responses-pipeline.ts's streamRequest:
- baseUrl fallback used `??`, which does not catch an empty string --
  USE_OPENAI_RESPONSES has no case in ModelRegistry.getDefaultBaseUrl, so an
  unconfigured baseUrl resolves to '' and every request hit the relative
  path /v1/responses. Switched to `||`.
- streamRequest called the global `fetch` instead of the `fetch`
  buildRuntimeFetchOptions pins alongside its dispatcher. Node's built-in
  undici can be a different major version than the bundled one, and handing
  it a foreign dispatcher throws `invalid onError method` -- this broke
  every streaming request whenever a dispatcher was installed (which is the
  common case, proxy or not).
- Network errors from fetch() and non-ok HTTP responses were thrown
  unredacted, leaking proxy credentials into error messages/logs.
- No Content-Type validation: an HTTP 200 non-SSE body (a gateway/proxy
  block page, or a baseUrl pointed at a non-API endpoint) silently yielded
  zero events instead of a surfaced, retryable failure.
- The SSE parser only flushed a pending event on a blank line and never
  after the read loop ended, so a stream that closes without a trailing
  blank line (a proxy stripping trailing whitespace, or an interrupted
  connection) silently dropped its last event -- including the terminal
  response.completed event carrying usageMetadata/finishReason.

Critical fixes in index.ts's embedContent:
- Missing redactProxyError on the embedding error path (present on the
  sibling openaiContentGenerator and on anthropicContentGenerator).
- The embed client passed baseUrl to the OpenAI SDK unnormalized; this
  generator's own /v1-less baseUrl convention (the streaming pipeline
  strips/re-appends /v1 itself) 404s every embedding call against the SDK,
  which does not auto-append /v1 to a custom baseURL. Normalized the same
  way the streaming pipeline does.

Suggestions fixed in responses-converter.ts:
- `buf?.args ?? fc.arguments`: initFunctionCall seeds buf.args to '', so a
  buffer that received output_item.added but no
  function_call_arguments.delta events never fell through to the done
  item's own complete arguments. Switched to `||`.
- A user turn with text + image parts was split into two separate
  'message' items; the Responses API treats them as two distinct turns, so
  the model could fail to associate a question with its image. Parts now
  accumulate per-turn and flush as one message with a multi-part content
  array, in original relative order around any function_call/
  function_call_output/reasoning items.
- Non-image inlineData and all fileData references were silently dropped
  with no diagnostic. They now become a text placeholder noting the
  unsupported attachment (no input_file part type is wired up for this
  generator yet) instead of vanishing.
- Tool results were sent as the raw { output } / { error } JSON envelope
  instead of the unwrapped string the sibling Chat Completions converter
  sends, double-encoding and quote-escaping any tool output that is itself
  JSON.
- Removed ResponsesStreamState.reset(), an unused method contradicting the
  documented one-instance-per-stream contract.

Other:
- Extracted the embedContent text-extraction logic (byte-identical between
  this generator and the sibling openaiContentGenerator) into a shared
  extract-text-from-contents.ts helper.
- Added test coverage: multi-part message merging, attachment placeholders,
  tool-result envelope unwrapping, the buf.args empty-buffer fallback,
  sanitizePromptCacheKey's truncation/hash path, the data-only SSE framing
  branch (the shape the Responses API actually emits, previously untested),
  and the USE_OPENAI_RESPONSES factory route in createContentGenerator.
- responses-pipeline.test.ts previously stubbed the global fetch, which the
  dispatcher/fetch fix above now bypasses (the pipeline calls the pinned
  fetch instead); switched to mocking buildRuntimeFetchOptions directly so
  the mock intercepts the fetch the pipeline actually uses.

* fix(openai-responses): close remaining review gaps from PR #8169

Re-audited every open review thread on PR #8169 against the current
code (commit 09ff5d996). Most had already been fixed in that commit;
this addresses the six that were still open, all Suggestions (no
Criticals remained).

Two real behavior fixes:
- responses-pipeline.ts: a stream that closes with a final `data: `
  line carrying no trailing newline at all was silently dropped.
  `buffer.split('\n')` never routes an unterminated line through the
  main per-line loop, and the existing post-loop flush only handled a
  missing trailing *blank line* (an already-parsed event awaiting its
  terminator), not an entirely unterminated line still sitting in
  `buffer`. Added a targeted post-loop check for this case, with a
  regression test confirmed failing before the fix (0 chunks yielded)
  and passing after.
- index.ts: `getOpenAIClient()` never passed `customHeaders` to the
  OpenAI SDK client, while the streaming pipeline
  (responses-pipeline.ts) already applies them to every request --
  headers configured for the streaming path (e.g. a proxy auth header)
  silently never reached `embedContent`. Added `defaultHeaders`,
  mirroring the pipeline. Regression test confirmed failing before the
  fix and passing after.

Four test-coverage gaps closed:
- contentGenerator.test.ts: the USE_OPENAI_RESPONSES routing test
  only asserted `totalTokens > 0`, which a mocked Chat generator
  would also satisfy. Added a dedicated mock for
  createOpenAIResponsesContentGenerator with a distinguishable
  sentinel and asserted the Chat generator's mock was never invoked.
- index.test.ts: `embedContent`'s `model.includes('embed') ? model :
  'text-embedding-ada-002'` branch was never asserted on either side;
  added the missing `model` assertion to an existing case plus a new
  case for the `includes('embed')` branch.
- index.test.ts: `generateContent`/`generateContentStream`'s
  `request.config?.abortSignal ?? undefined` forwarding was only ever
  exercised with `undefined`. Added cases passing a real
  `AbortController` signal through both methods.
- responses-converter.test.ts: `normalizeResponsesParameters`'s wiring
  into `convertGeminiToolsToResponsesTools` was untested end-to-end --
  every existing tool-conversion case already had `properties`, making
  normalization a no-op. Added a zero-arg-tool case that actually
  exercises it.

Verified: 100 tests passing across the four touched suites (39 + 21 +
18 + 22), each new/changed assertion confirmed to fail on the
pre-fix code where applicable. Typecheck and lint clean.

* fix(core): address review round on the Responses wire (7 criticals + coverage)

Addresses the review round on #8169. Every finding was re-verified against
the code before acting; all were confirmed. Each behavior fix has a
regression test that was checked red-before / green-after in place.

Retry, timeout and error classification:
- generateContentStream returned a lazy async generator, so its promise
  resolved before any network I/O and connection-time 5xx/429 could never
  reach retryWithBackoff. Split the pipeline into an eager connect phase
  (fetch + non-2xx handling + content-type guard) and a lazy body-iteration
  phase behind connectStream(), and wired index.ts to it. executeStream is
  retained and now delegates, so existing callers are unaffected.
- The stream read loop had no timeout layer at all and the undici dispatcher
  uses headersTimeout:0/bodyTimeout:0, so a 200-then-silent upstream hung
  the turn forever with no user present to cancel. Added an idle watchdog
  honoring streamIdleTimeoutMs / QWEN_STREAM_IDLE_TIMEOUT_MS, reusing the
  Chat pipeline's shared constants and its retryable ETIMEDOUT semantics.
  (Note: the reviewer's cited helper `withStreamInactivityTimeout` does not
  exist; the real sibling is `withStreamGuards`, whose structure is mirrored
  here rather than imported, since importing it pulls in the whole OpenAI
  SDK graph.)
- Mid-stream `error` / `response.failed` frames arrive after a 200 OK and
  were thrown as plain Errors with no `.status` and no `.code`, so they
  classified as `unknown` and missed the rate-limit retry, transport replay,
  model fallback and persistent-mode wait. Map the known codes onto a status
  before throwing.
- getOpenAIClient passed raw timeout/maxRetries to the SDK, bypassing
  resolveRequestTimeout (so `timeout: 0`, a documented "disable" knob, meant
  immediate abort) and DEFAULT_MAX_RETRIES.

Tool-call argument handling:
- A corrupt non-empty delta buffer shadowed the done item's authoritative
  `arguments`, dispatching tool calls with `{}`. Fall back to the done
  item's string on parse failure.
- Malformed-but-repairable arguments bypassed the safeJsonParse/jsonrepair
  fallback every sibling wire applies to the same input class.
- Arguments parsing to an array/null/primitive flowed through the
  `as Record` cast into functionCall.args; collapse non-objects to `{}` as
  streamingToolCallParser does.

Fidelity and observability:
- max_output_tokens: buildRequest never read request.config.maxOutputTokens,
  dropping the per-send window clamp geminiChat computes (#5950).
- functionResponse.parts media was silently discarded; tool-returned images
  now reach the wire as a follow-up input_image message.
- The converter never called setGenAiUsageProvenance while always
  materializing cachedContentTokenCount, so every turn reported cache
  statistics as provider-reported.

Reachability and diagnostics:
- AuthType.USE_OPENAI_RESPONSES was missing from the --auth-type yargs
  choices, so `qwen --auth-type=openai-responses` was rejected outright.
- The new PROTOCOL_ITEMS entry was unreachable UI. Added the auth type to
  customProvider.protocolOptions; verified the type is wired end to end
  (contentGenerator creation, AUTH_PREFLIGHT_ENV_KEYS, validateAuthMethod,
  DEFAULT_BASE_URLS) so the wizard option resolves to a working generator.
- systemInfo.ts and the RUM base_url gate were not extended for the new auth
  type, degrading /bug and /about output and losing endpoint attribution.
- De-duplicated the part->text lambda in extract-text-from-contents.

Coverage: filled the 29 flagged untested-behavior sites (multi-line data:
frames, UTF-8 split across chunk boundaries, extra_body precedence with an
explicit 0, custom header forwarding, abort-signal identity and mid-stream
abort, dispatcher forwarding, proxy-credential redaction, the non-SSE
content-type guard, malformed frames, empty-merge, and the auth/telemetry
entries).

Verification: 111/111 in openaiResponsesContentGenerator, 54/54 in the other
touched core suites, 402/404 in the touched CLI suites (the 2 failures are
pre-existing and reproduce on a pristine checkout -- config.test.ts reads
the ambient OTEL_EXPORTER_OTLP_ENDPOINT). tsc clean, eslint clean, prettier
clean.

* refactor(core): actually de-duplicate the part->text lambda in extractTextFromContents

Correction to dcd8a67d9, whose commit message claimed this was done. It was
not -- the subagent that owned the file returned a WONTFIX for it and the
claim went into the message unverified. The lambda was still duplicated
verbatim between the array-input and single-content branches of the very
helper whose JSDoc promises "a change to how Part text is shaped only needs
to be made once".

Extracted `textOfParts` and routed both branches through it. Behavior is
unchanged for every input shape.

Added the collocated test file the helper never had, including a guard that
converts the same Content bare and wrapped in a single-element array and
asserts the two agree -- the drift this extraction exists to prevent, and
which the pre-extraction code had already let happen once. Verified by
re-injecting a divergent copy into the array branch: that test alone goes
red, the other five stay green.

117/117 across the Responses directory plus the new file; tsc, eslint and
prettier clean.

* fix(core): address round-2 review on the Responses wire

The first automatic review to complete on this PR (previous three attempts
timed out). 30 inline findings plus five Criticals that were recorded in the
review ledger but never posted inline. Every finding was verified against
the code before acting.

Pipeline:
- max_output_tokens precedence reopened the user's ceiling. Last round used
  `request.config?.maxOutputTokens ?? samplingParams.max_tokens`, which
  returns the larger per-send value for config 1000 + per-send 8000. Now
  routes through tokenLimits' `reconcileMaxTokens` "smaller wins" invariant,
  which both sibling wires call and which exists precisely so a new provider
  can't silently reopen that ceiling. The reviewer was right about our own
  previous fix.
- `tool_choice: 'auto'` and `parallel_tool_calls: true` were sent
  unconditionally. Toolless requests are reachable here (compaction via
  baseLlmClient.generateText, text-mode side queries) and strict endpoints
  400 on tool_choice with no tools, which can wedge compaction in a retry
  loop. Gated on tools being present, matching the sibling.
- The connect phase had no timeout at all, and `ContentGeneratorConfig.timeout`
  was never read by this pipeline. An endpoint that completes TCP/TLS but
  never sends headers blocked the turn forever: the pinned undici Agent runs
  headersTimeout:0/bodyTimeout:0, the idle watchdog only wraps reader.read()
  inside iterateBody, and retryWithBackoff has no per-attempt timeout.
  Added StreamConnectTimeoutError via the same resolveRequestTimeout helper
  the embed client uses.
- Only the idle watchdog existed, so an upstream drip-feeding one chunk per
  window re-armed it forever. Added StreamLifetimeExceededError using the
  sibling's shared DEFAULT_STREAM_MAX_LIFETIME_MS /
  QWEN_STREAM_MAX_LIFETIME_MS_ENV constants, and folded the idle/lifetime
  resolvers into one helper that now warns on invalid values instead of
  silently discarding them.

Converter:
- A delta buffer that parsed cleanly to a non-object (array/null/primitive)
  bypassed the done-item authoritative-arguments fallback, which only ran in
  the catch branch, so a tool call dispatched with `{}` while valid
  `arguments` sat in the same event. Residual of last round's fix, which
  covered only the parse-throwing case.
- Refusals were silently dropped: no `response.refusal.delta` / `.done`
  handling, so a refusal produced zero content chunks, threw
  NO_RESPONSE_TEXT, and retried four times re-sending the full prompt before
  surfacing a misleading error. Added both events and the refusal content
  part; deltas emit as text, mirroring output_text.delta.
- Tool-parameter precedence was inverted relative to both siblings
  (`parameters ?? parametersJsonSchema`); swapped to match.

Generator, CLI and SDKs:
- LazyContentGenerator.useSummarizedThinking() omitted the new auth type.
- The custom-provider test mixed barrel imports (resolving to dist/) with
  source imports, so the per-file vitest flow saw an undefined
  AuthType.USE_OPENAI_RESPONSES against a stale dist. Both sides now come
  from source.
- Both bundled SDKs enumerate every CLI auth type but omitted this one, so
  SDK users could not select it: added to the TypeScript zod enum and union
  and to the Python Literal and validation set, with a parametrized test
  pinning the whole set against the CLI's --auth-type choices.

Coverage: filled the flagged gaps across all three files — per-output_index
function-call isolation, cached_tokens:0 absent-vs-zero, the reasoning
shape-guard, functionResponse text-append, string systemInstruction,
multi-Tool conversion, non-mutation recursion, proxy hand-off, apiKeyEnvKey
fallback, Content-Type, error-body excerpt, .status stamping, ETIMEDOUT
code, per-chunk timer reset, the no-body guard, empty-string baseUrl, the
countTokens fallback arithmetic, maxRetries pass-through, the no-baseUrl
embed branch, trailing-slash normalization, and abort-signal null handling.

Every behavior change is mutation-verified: fix reverted, test observed red,
fix restored.

Not changed, flagged for maintainers:
- `prompt_cache_key` is keyed on `userPromptId` (`${sessionId}####${counter}`),
  which changes every turn and so defeats cross-turn prefix-cache affinity.
  Keying on the stable session prefix is a behavior change, so it is raised
  rather than made unilaterally.
- ACP `buildAuthMethods()` advertises only USE_OPENAI — it does not advertise
  Anthropic, Gemini or Qwen OAuth either, so this is pre-existing scope
  rather than a gap introduced here.

One ledger Critical (the pinned `fetch` being discarded in favour of Node's
global) is INVALID as written: it describes pre-round-1 code. `fetchFn =
runtimeOptions?.fetch ?? fetch` landed in 09ff5d996, and the pipeline tests
deliver their mock through `runtimeOptions.fetch` without stubbing the
global, so a regression to global fetch would fail the suite.

Verification: 174 core tests across the six touched core suites, 419 CLI
tests across seven, 119 Python SDK unit tests; tsc clean for core, cli and
sdk-typescript; eslint, prettier and ruff clean.

* fix(core): key the Responses prefix cache on the session, not the turn

`prompt_cache_key` was set to `sanitizePromptCacheKey(userPromptId)`, and
userPromptId is `${sessionId}########${counter}` -- it changes on every send.
Each turn therefore got its own cache namespace and no request in a session
could ever hit the prefix cache the previous one wrote. Nothing fails when
that happens; the entire cost is silent latency and spend, which is why it
survived until review flagged it.

Keyed on `qwen-code:${sessionId}` instead, matching the Chat wire's
`applyOfficialOpenAIPromptCaching` (openaiContentGenerator/prefix-caching.ts).

Note on provenance, since it changes how this reads: that sibling behavior
landed on main in 6897ef744 (#8418) on 2026-08-07, which is AFTER this branch
last merged main (770146526) and is not in this branch's history. So this was
not a case of ignoring an existing helper -- main converged on session-scoped
keying independently while this PR was in review, and the two wires would
otherwise have disagreed on merge.

The `qwen-code:` prefix is written as a literal here because
prefix-caching.ts on this branch's base predates the constant. Once main is
merged in, this should import the shared CACHE_KEY_PREFIX rather than keep a
second copy of the string.

`Config.getSessionId()` is typed `string` and never returns undefined, so the
guard covers the reachable empty-string case only; the mock and the fallback
test were written against that rather than a `undefined` state the real API
cannot produce.

Test pins the actual property -- two different userPromptIds in one session
produce the same key. Reverting to per-turn keying fails it. 135 tests pass
across the directory; tsc, eslint and prettier clean.

* fix(core): route the setup wizard's enable_thinking through reasoning.effort on the Responses wire

wenshao's local verification report on #8169 found that picking
`openai-responses` in the setup wizard and enabling "Enable thinking"
produced a session with zero reasoning, because the wizard writes the
DashScope-specific `extra_body.enable_thinking` toggle for every
protocol. The Responses wire has no DashScope-detection layer (unlike
the OpenAI Chat pipeline), so it forwarded `enable_thinking` onto the
wire verbatim as an undefined field instead of requesting reasoning at
all.

- provider-config.ts: buildAdvancedGenerationConfig now takes the
  resolved protocol and writes `reasoning: { effort: 'medium' }`
  instead of `extra_body.enable_thinking` when the protocol is
  `openai-responses`.
- responses-pipeline.ts: defense in depth for configs that still carry
  a legacy `extra_body.enable_thinking` (hand-edited settings, ACP,
  etc.) — buildReasoning() treats it as `effort: 'medium'` when no
  explicit `reasoning` is set, and the extra_body merge always strips
  the key so it can never reach the wire literally.
- docs: add `openai-responses` to the supported-auth-types and SDK
  tables, a full config example, a row in the reasoning per-provider
  table, and correct settings.md's `extra_body` note (it previously
  said the field was `openai`/`qwen-oauth` only).

Addresses PR #8169 review findings 3.1 and 3.2.

* fix(core): reject unsupported Responses image MIME types

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(responses): recover rejected reasoning replay once

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(responses): reject malformed replay errors

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(responses): constrain proxy error recovery

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* fix(auth): expose OpenAI Responses consistently

* fix(responses): bound error response handling

* fix(responses): honor request-scoped controls

* fix(auth): align Responses setup preview

* fix(responses): preserve request thinking opt-out

* fix(responses): require SSE content type

* test(cli): include Responses auth in top-level help expectation

* fix(responses): address protocol compatibility review

* fix(responses): redact diagnostics and normalize custom headers

---------

Co-authored-by: Palanisamy, Dinesh <Dinesh.Palanisamy@netapp.com>
Co-authored-by: Shaojin Wen <shaojin.wensj@alibaba-inc.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: Dragon <52599892+DragonnZhang@users.noreply.github.com>
Co-authored-by: probe <probe@local>
2026-09-09 22:36:23 +00:00

482 lines
16 KiB
Python

from __future__ import annotations
import json
import re
from pathlib import Path
from typing import Any, cast, get_args
import pytest
from qwen_code_sdk.errors import ValidationError
from qwen_code_sdk.types import PermissionMode, QueryOptions, TimeoutOptions
from qwen_code_sdk.validation import validate_query_options
VALID_UUID = "123e4567-e89b-12d3-a456-426614174000"
def test_rejects_resume_with_continue_session() -> None:
with pytest.raises(ValidationError, match="resume together with continue_session"):
validate_query_options(
QueryOptions(
resume=VALID_UUID,
continue_session=True,
)
)
def test_rejects_session_id_with_resume() -> None:
with pytest.raises(ValidationError, match="Cannot use session_id with resume"):
validate_query_options(
QueryOptions(
session_id=VALID_UUID,
resume="223e4567-e89b-12d3-a456-426614174000",
)
)
def test_rejects_invalid_session_id() -> None:
with pytest.raises(ValidationError, match="Invalid session_id"):
validate_query_options(QueryOptions(session_id="not-a-uuid"))
def test_rejects_invalid_resume() -> None:
with pytest.raises(ValidationError, match="Invalid resume"):
validate_query_options(QueryOptions(resume="not-a-uuid"))
@pytest.mark.parametrize(
"value",
[
"{12345678-1234-4234-8234-123456781234}",
"urn:uuid:12345678-1234-4234-8234-123456781234",
"12345678123442348234123456781234",
],
)
def test_rejects_non_canonical_session_id(value: str) -> None:
# uuid.UUID() accepts these spellings, but the value is passed to the CLI
# verbatim as --session-id, so only the canonical 8-4-4-4-12 form works.
with pytest.raises(ValidationError, match="canonical"):
validate_query_options(QueryOptions(session_id=value))
@pytest.mark.parametrize(
"value",
[
"{12345678-1234-4234-8234-123456781234}",
"urn:uuid:12345678-1234-4234-8234-123456781234",
"12345678123442348234123456781234",
],
)
def test_rejects_non_canonical_resume(value: str) -> None:
with pytest.raises(ValidationError, match="canonical"):
validate_query_options(QueryOptions(resume=value))
def test_accepts_canonical_session_id_in_either_case() -> None:
# Case is not part of canonical form; an uppercase UUID is valid input.
validate_query_options(QueryOptions(session_id=VALID_UUID))
validate_query_options(QueryOptions(session_id=VALID_UUID.upper()))
@pytest.mark.parametrize(
"mode",
get_args(PermissionMode),
)
def test_accepts_valid_permission_modes(mode: str) -> None:
validate_query_options(QueryOptions.from_mapping({"permission_mode": mode}))
def test_permission_modes_match_core_contract() -> None:
contract_path = (
Path(__file__).parents[3] / "core" / "src" / "config" / "approval-modes.json"
)
expected = set(json.loads(contract_path.read_text(encoding="utf-8")))
assert set(get_args(PermissionMode)) == expected
def test_rejects_invalid_permission_mode() -> None:
with pytest.raises(
ValidationError,
match=re.escape(
"Invalid permission_mode: 'unsafe-mode'. "
"Expected one of: default, plan, auto-edit, auto, yolo."
),
):
validate_query_options(
QueryOptions.from_mapping({"permission_mode": "unsafe-mode"})
)
def test_rejects_invalid_auth_type() -> None:
with pytest.raises(ValidationError, match="Invalid auth_type"):
validate_query_options(QueryOptions.from_mapping({"auth_type": "custom"}))
@pytest.mark.parametrize(
"auth_type",
["openai", "openai-responses", "anthropic", "qwen-oauth", "gemini", "vertex-ai"],
)
def test_accepts_every_cli_auth_type(auth_type: str) -> None:
# Pins the SDK's auth-type set against the CLI's --auth-type choices. An
# auth type the CLI accepts but the SDK omits is rejected here with no
# other signal, so enumerate the whole set rather than spot-checking one.
validate_query_options(QueryOptions.from_mapping({"auth_type": auth_type}))
def test_from_mapping_rejects_non_callable_can_use_tool() -> None:
with pytest.raises(TypeError, match="can_use_tool must be callable"):
QueryOptions.from_mapping({"can_use_tool": "bad"})
def test_from_mapping_rejects_non_callable_stderr() -> None:
with pytest.raises(TypeError, match="stderr must be callable"):
QueryOptions.from_mapping({"stderr": "bad"})
def test_validation_rejects_non_callable_can_use_tool() -> None:
with pytest.raises(ValidationError, match="can_use_tool must be callable"):
validate_query_options(QueryOptions(can_use_tool=cast(Any, "bad")))
def test_validation_rejects_non_callable_stderr() -> None:
with pytest.raises(ValidationError, match="stderr must be callable"):
validate_query_options(QueryOptions(stderr=cast(Any, "bad")))
def test_from_mapping_rejects_sync_can_use_tool() -> None:
def can_use_tool( # type: ignore[no-untyped-def]
tool_name, tool_input, context
):
return {"behavior": "deny", "message": "bad"}
with pytest.raises(TypeError, match="can_use_tool must be an async callable"):
QueryOptions.from_mapping({"can_use_tool": can_use_tool})
def test_validation_rejects_sync_can_use_tool() -> None:
def can_use_tool( # type: ignore[no-untyped-def]
tool_name, tool_input, context
):
return {"behavior": "deny", "message": "bad"}
with pytest.raises(ValidationError, match="can_use_tool must be an async callable"):
validate_query_options(QueryOptions(can_use_tool=cast(Any, can_use_tool)))
def test_from_mapping_rejects_can_use_tool_with_wrong_arity() -> None:
async def can_use_tool(
tool_name: str,
tool_input: dict[str, Any],
) -> dict[str, str]:
return {"behavior": "deny"}
with pytest.raises(
TypeError, match="can_use_tool must accept exactly 3 positional arguments"
):
QueryOptions.from_mapping({"can_use_tool": can_use_tool})
def test_validation_rejects_can_use_tool_with_wrong_arity() -> None:
async def can_use_tool(
tool_name: str,
tool_input: dict[str, Any],
) -> dict[str, str]:
return {"behavior": "deny"}
with pytest.raises(
ValidationError,
match="can_use_tool must accept exactly 3 positional arguments",
):
validate_query_options(QueryOptions(can_use_tool=cast(Any, can_use_tool)))
def test_from_mapping_rejects_stderr_with_wrong_arity() -> None:
def stderr() -> None:
return None
with pytest.raises(
TypeError, match="stderr must accept exactly 1 positional argument"
):
QueryOptions.from_mapping({"stderr": stderr})
def test_validation_rejects_stderr_with_wrong_arity() -> None:
def stderr() -> None:
return None
with pytest.raises(
ValidationError, match="stderr must accept exactly 1 positional argument"
):
validate_query_options(QueryOptions(stderr=cast(Any, stderr)))
def test_rejects_invalid_max_session_turns() -> None:
with pytest.raises(ValidationError, match="max_session_turns"):
validate_query_options(QueryOptions(max_session_turns=-2))
def test_rejects_fractional_max_session_turns() -> None:
with pytest.raises(ValidationError, match="max_session_turns"):
validate_query_options(QueryOptions(max_session_turns=cast(Any, 0.5)))
@pytest.mark.parametrize("value", [True, False])
def test_rejects_bool_max_session_turns(value: bool) -> None:
with pytest.raises(ValidationError, match="max_session_turns"):
validate_query_options(QueryOptions(max_session_turns=cast(Any, value)))
def test_rejects_empty_qwen_executable_path() -> None:
with pytest.raises(
ValidationError, match="path_to_qwen_executable cannot be empty"
):
validate_query_options(QueryOptions(path_to_qwen_executable=" "))
def test_timeout_rejects_non_numeric_value() -> None:
with pytest.raises(TypeError, match=r"timeout\.can_use_tool must be a positive"):
TimeoutOptions.from_mapping({"can_use_tool": "fast"})
def test_timeout_rejects_negative_value() -> None:
pattern = r"timeout\.control_request must be a positive"
with pytest.raises(ValueError, match=pattern):
TimeoutOptions.from_mapping({"control_request": -1})
def test_timeout_rejects_boolean_value() -> None:
with pytest.raises(TypeError, match=r"timeout\.stream_close must be a positive"):
TimeoutOptions.from_mapping({"stream_close": True})
def test_rejects_invalid_max_tool_calls() -> None:
with pytest.raises(ValidationError, match="max_tool_calls"):
validate_query_options(QueryOptions(max_tool_calls=-2))
def test_rejects_invalid_max_subagent_depth() -> None:
with pytest.raises(ValidationError, match="max_subagent_depth"):
validate_query_options(QueryOptions(max_subagent_depth=0))
@pytest.mark.parametrize("value", [True, False, 0.5])
def test_rejects_non_integer_max_tool_calls(value: object) -> None:
# The error message promises "-1 or a non-negative integer", and the value
# is stringified straight onto the CLI, so `True` would become
# `--max-tool-calls True`.
with pytest.raises(ValidationError, match="max_tool_calls"):
validate_query_options(QueryOptions(max_tool_calls=cast(Any, value)))
@pytest.mark.parametrize("value", [True, False, 2.5])
def test_rejects_non_integer_max_subagent_depth(value: object) -> None:
with pytest.raises(ValidationError, match="max_subagent_depth"):
validate_query_options(QueryOptions(max_subagent_depth=cast(Any, value)))
def test_accepts_valid_integer_limits() -> None:
# The in-range integers these options are documented to take must survive.
validate_query_options(
QueryOptions(max_tool_calls=-1, max_session_turns=-1, max_subagent_depth=1)
)
validate_query_options(
QueryOptions(max_tool_calls=0, max_session_turns=0, max_subagent_depth=100)
)
def test_rejects_agents_missing_required_fields() -> None:
with pytest.raises(ValidationError, match="missing required field"):
validate_query_options(QueryOptions(agents=[{"name": "test"}]))
def test_rejects_extra_args_with_reserved_flags() -> None:
with pytest.raises(ValidationError, match="reserved flag"):
validate_query_options(QueryOptions(extra_args=["--input-format"]))
@pytest.mark.parametrize(
"flag",
[
"--model",
"-m",
"--auth-type",
"--approval-mode",
"--insecure",
"--yolo",
"-y",
"--allowed-tools",
"--exclude-tools",
"--resume",
"-r",
"--continue",
"-c",
"--session-id",
"--proxy",
"--channel",
"--output-format",
"-o",
"--openai-base-url",
"--openai-api-key",
"--mcp-config",
"--prompt",
"-p",
"--prompt-interactive",
"-i",
"--add-dir",
"--input-file",
"--extensions",
"-e",
"--sandbox",
"-s",
"--no-sandbox",
"--no-insecure",
"--no-safe-mode",
"--sandbox-image",
"--fork-session",
"--max-tool-calls",
"--max-subagent-depth",
"--max-session-turns",
"--system-prompt",
"--append-system-prompt",
"--include-directories",
"--allowed-mcp-server-names",
"--disabled-slash-commands",
"--include-partial-messages",
"--chat-recording",
"--openai-logging",
"--openai-logging-dir",
"--json-schema",
"--json-fd",
"--json-file",
],
)
def test_rejects_extra_args_with_security_sensitive_flags(flag: str) -> None:
with pytest.raises(ValidationError, match="reserved flag"):
validate_query_options(QueryOptions(extra_args=[flag]))
@pytest.mark.parametrize(
"flag",
[
"--model=qwen-max",
"--auth-type=openai",
"--approval-mode=yolo",
"--insecure=true",
"--yolo=true",
"--proxy=http://localhost:8080",
],
)
def test_rejects_extra_args_with_flag_value_syntax(flag: str) -> None:
with pytest.raises(ValidationError, match="reserved flag"):
validate_query_options(QueryOptions(extra_args=[flag]))
def test_accepts_extra_args_with_non_reserved_flags() -> None:
validate_query_options(QueryOptions(extra_args=["--some-unknown-flag", "--value"]))
def test_rejects_fallback_model_exceeding_max() -> None:
with pytest.raises(ValidationError, match="fallback_model supports a maximum of 3"):
validate_query_options(QueryOptions(fallback_model=["a", "b", "c", "d"]))
def test_rejects_empty_proxy() -> None:
with pytest.raises(ValidationError, match="proxy cannot be empty"):
validate_query_options(QueryOptions(proxy=" "))
def test_rejects_fork_session_without_resume() -> None:
with pytest.raises(ValidationError, match="fork_session requires resume"):
validate_query_options(QueryOptions(fork_session=True))
def test_accepts_fork_session_with_resume() -> None:
validate_query_options(
QueryOptions(
fork_session=True,
resume="123e4567-e89b-12d3-a456-426614174000",
)
)
def test_rejects_invalid_effort() -> None:
with pytest.raises(ValidationError, match="Invalid effort"):
validate_query_options(QueryOptions(effort="invalid")) # type: ignore[arg-type]
def test_accepts_valid_effort() -> None:
for effort in ("low", "medium", "high", "xhigh", "max"):
validate_query_options(QueryOptions(effort=effort)) # type: ignore[arg-type]
@pytest.mark.parametrize(
"field_name",
[
"include_directories",
"extensions",
"allowed_mcp_server_names",
"disabled_slash_commands",
"fallback_model",
],
)
def test_rejects_comma_in_list_fields(field_name: str) -> None:
with pytest.raises(ValidationError, match="cannot contain commas"):
validate_query_options(QueryOptions(**{field_name: ["valid", "invalid,comma"]}))
def test_from_mapping_parses_all_new_fields() -> None:
opts = QueryOptions.from_mapping(
{
"fork_session": True,
"resume": VALID_UUID,
"max_tool_calls": 50,
"max_subagent_depth": 3,
"include_directories": ["/dir1", "/dir2"],
"extra_args": ["--verbose"],
"extensions": ["ext1"],
"allowed_mcp_server_names": ["server1"],
"fallback_model": ["model-a", "model-b"],
"proxy": "http://proxy:8080",
"sandbox": True,
"safe_mode": True,
"insecure": True,
"worktree": True,
"disabled_slash_commands": ["/cmd1"],
"agents": [{"name": "a", "description": "b", "systemPrompt": "c"}],
}
)
assert opts.fork_session is True
assert opts.resume == VALID_UUID
assert opts.max_tool_calls == 50
assert opts.max_subagent_depth == 3
assert opts.include_directories == ["/dir1", "/dir2"]
assert opts.extra_args == ["--verbose"]
assert opts.extensions == ["ext1"]
assert opts.allowed_mcp_server_names == ["server1"]
assert opts.fallback_model == ["model-a", "model-b"]
assert opts.proxy == "http://proxy:8080"
assert opts.sandbox is True
assert opts.safe_mode is True
assert opts.insecure is True
assert opts.worktree is True
assert opts.disabled_slash_commands == ["/cmd1"]
assert opts.agents == [{"name": "a", "description": "b", "systemPrompt": "c"}]
def test_from_mapping_defaults_new_fields_to_none() -> None:
opts = QueryOptions.from_mapping({})
assert opts.fork_session is False
assert opts.max_tool_calls is None
assert opts.max_subagent_depth is None
assert opts.include_directories is None
assert opts.extra_args is None
assert opts.extensions is None
assert opts.allowed_mcp_server_names is None
assert opts.fallback_model is None
assert opts.proxy is None
assert opts.sandbox is False
assert opts.safe_mode is False
assert opts.insecure is False
assert opts.worktree is False
assert opts.disabled_slash_commands is None
assert opts.agents is None