qwen-code/integration-tests
Shaojin Wen cfb594252e
fix(managed-agent): numeric parity and family-list hoisting for publication families, with end-to-end admission and wedge witnesses (#13815)
* fix(managed-agent): admit background Shell and Monitor capture publication families

The capture-publication reserve refused every payload but foreground Shell:
the contract hard-refused is_background and any toolName but
run_shell_command (#13533 A1), the store's checkpoint gate re-spelled the
same shell-only assumption one layer below, and the hosted turn's
checkpoint fork recorded the canonical-input digest only for shell. The
three gates now share one family list (PUBLISHABLE_TOOL_NAMES) covering
foreground/background Shell and Monitor, numeric bounds accept
integral-valued numerics in parity with the TypeScript mirror, and the
turn fork covers monitoring. Families admit only via the shared fixture
vectors replayed byte-identically in both languages (22 vectors incl.
boundary and exponent-spelling cases); mutation witness: removing the
background arm reddens both suites. End-to-end, admitted background and
Monitor reserves answer grant OPEN through the full Spring/broker/daemon
chain (probe-gated witness lanes).

* test(managed-agent): background-publication witness ITs with shared probe wiring

Adds the end-to-end witnesses built while reconstructing #13533's findings:
HostedBackgroundPublicationIT pins admitted background/Monitor reserves
answering grant OPEN through the real Spring controller/stores/broker/daemon
chain (strict detached-arm pins; environment-declined not_started named as
the unlanded B1 dependency), and HostedRecoveryBlockedWedgeIT keeps the A2
wedge probes as regression coverage fed by a driver-relayed reserve 400.
Both boot a shared PublicationProbeWiring extracted under
com.alibaba.qwen.managedprobes (outside the app's component scan), with the
refusal-diagnostic ProbeAdvice registered only in the publication context.
All lanes are gated behind -Dqwen.wedge.probe=true, leaving the CI
hosted-harness-mysql lane unaffected without it.

* docs(design): managed tool publication background arm (EN/zh-CN)

Design record for the #13533 A1 slice: why the single contract gains a
background/Monitor arm rather than background captures routing through a
second reserve (the digest-exact binding is family-independent, so option B
duplicates the grant lifecycle for no invariant), the admitted payload
families with integral-valued numeric bounds, the three closed shell-only
layers the witness pinned (contract, store checkpoint gate, turn fork), the
scope boundaries (no domain enablement; B1/B2/A2 named separately; the
environment-declined 409 cause recorded as an A3-family follow-up), and the
acceptance criteria with their mutation witnesses.

* docs(design): drop the superseded background-arm drafts

Upstream #13789 landed its own shape of the A1 fix (per-family arms with
directory and its own verification), so the drafts written around this
branch's earlier approach (shared constant, strict markers, no directory)
would record decisions that contradict what main now runs. The remaining
delta's rationale moves to the PR thread.

* test(integration): enumerate the wedge driver in the relay-caller witness

The sdk-java lane pins the exact set of drivers calling the shared relay
helper; adding a new legitimate caller (the recovery-blocked wedge driver)
in the same slice must update the enumeration or every later run of
'Check Hosted proxy header relay' fails 5-vs-6. The witness also keeps its
purpose: a driver hand-rolling its own relay stays absent from the list
and is caught — the new entry only records that this one uses the shared
helper, which is exactly the posture the pin exists to keep.

* test(managed-agent): rename probe witnesses off the Hosted*IT roster fingerprint

The hosted fault-gates lane requires every Hosted*IT source class to run
at least one test; the two probe-gated witnesses added with the A1 slice
always record zero executions without -Dqwen.wedge.probe=true (1 run,
1 skipped, which counts as zero against the roster), so the rerun of the
lane fails at 'Check that every Hosted integration test class ran' with
'::error: ... ran no test in this job'. Make the probe intent structural
by renaming them to one-shot ...Gate classes (the O4Gates' shape): the
failsafe include glob '**/Hosted*IT.java' no longer schedules them, and
both roster family predicates (hosted ^Hosted.*IT$ and non-hosted & non-IT)
miss them. Probe lanes keep invoking by simple name; behavior unchanged.
Verified: test-compile + checkstyle green, no residual old-name references
outside archive snapshots.

* feat(cli): add a test-only H3 admission switch for hosted probe witnesses

QWEN_MANAGED_SESSION_TEST_DOMAINS admits the listed child_run/monitor_run
domains at the two turn-admission checks only, so the probe witnesses can
run against an unmodified bundle while the store-side commit gate keeps
refusing what the frozen registry disables; production never sets it.
Also reverts the publication inputDigest fork to upstream's form: since
the merge, the store-side checkpoint gate admits the monitor family
through PUBLISHABLE_TOOL_NAMES, which made the extra monitoring disjunct
a provable no-op.

* test(managed-agent): harden the hosted probe witnesses and collect them in a profile

- Move ProbeAdvice out of the scanned package into
  com.alibaba.qwen.managedprobes so only a witness that lists it as an
  explicit application source registers it; the wedge witness omits it and
  keeps observing the production envelope as-is.
- Add the hosted-harness-gates failsafe profile collecting
  **/Hosted*Gate.java with the probe flag on, keeping the Hosted*IT
  roster, its CI lane and both roster checks untouched.
- Give both gates the driver-process lifecycle finally, budgets that
  cover the worst arm (driver waitFor 1500s / wedge watch 900s under
  @Timeout 1800s and a 2400s fork ceiling), an accurate deadline-expiry
  assertion message, quoted wrapper paths that reject quotes and
  newlines, per-gate evidence subtrees, and a session sweep that binds
  every probe session id with capped-sample naming.
- Point the witness javadoc at #13533 and PR #13815 instead of the
  removed design doc, and name this PR's actual delta for the monitor
  arm (PUBLISHABLE_TOOL_NAMES extraction and the boundedIntField
  numeric widening) rather than a chain lifted elsewhere.
- Driver: the header now says the reserve refusal is relay-injected and
  never reaches the server; the daemon gets the admission switch; each
  family replays its captured OPEN reserve with the same publicationId
  and only its digests forged, fired before the blocked settlement, and
  the 400 refusal reason must name the digest.

* test(core): pin fractional numeric spellings in the publication payload corpus

Adds shell-timeout-fractional, monitor-idle-timeout-fractional and
monitor-max-events-fractional (each carrying a 1.5 field, all invalid)
so the integral-only clause of boundedIntField is pinned in both
languages by the shared, digest-stamped fixture vectors.

* test(cli): register the test-only admission switch in the process.env guard

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* test(managed-agent): align the witnesses with the H3-enabled registry

Upstream #13889 (#13856) enabled everything the witnesses' simulated
switch was papering over (shell kind and monitor_run domain are now
enabled for submission), so the switch remnants are removed: the driver's
daemon env entry, the process.env guard allowance, and both gates'
javadoc mentions. The gates now assert straight on the plain registry
against the unmodified bundle — exactly the reachability proof the review
demanded for the record-commit path.

The Arm-3 oracle is repointed at the layer that actually answers: a
forged-digest replay of the captured OPEN reserve is refused by
ToolPublicationStore.lockOriginal() with "Broker execution identity
conflicts" before any payload validation, so the assertion checks
status 400 plus the identity-conflict text instead of a digest mention
that can never occur.

* test(managed-agent): pin the arm-3 oracle to the placement/replay refusal layer

Since the upstream merge, ToolPublicationStore.applyLocked() runs
lockPublicationPlacement before lockOriginal, so a forged-digest replay
of the captured OPEN reserve is refused at the placement check —
"Reservation replay conflicts", comparing the replayed binding digest
against the persisted row — before any identity or payload validation.
The wire oracle is repointed to that layer (status 400 plus the
reservation-replay-conflict message); the driver comment, the gate
javadoc and the reviewer plan wording follow the layer that actually
answers.

* test(managed-agent): make the probe-turn verdict terminal instead of wall-clock

driveProbeTurn's completed path latched from an initial 'completed' with
no terminal-state check: after the two threshold exits, the fall-through
waited for !hasActivePrompt but never verified the turn ended as a
completion — a store request that hung got the daemon to fail the turn,
and the driver journaled HOSTED_WEDGE_ABSENT green beside the wedged
probe's evidence. The threshold path now returns 'wedged' only when the
turn is still active past its budget; otherwise it breaks the polling
loop and the transcript gate decides. The completed path now requires
exactly one turn_complete for its promptId on both the threshold exit
and the post-deadline fall-through, mirroring the recovery path's own
assertion. A roster-witness pin in hosted-relay-headers.test.ts keeps
that gate pinned inside the driver's completed path; removing it must
turn the relay test red.

* test(managed-agent): widen the wedge wait, wipe evidence per run and correct gate attribution

- The post-clear driver wait rises from 300 s to 600 s: the driver's own
  post-clear budget runs to roughly 510 s (two sequential 240 s windows
  plus a 30 s transcript fetch), so a legitimate A2 recovery inside it
  would see the intended AssertionError('A2 REGRESSION: ...') route
  through an opaque driver timeout, with Files.copy(driver-final.log)
  skipped by the finally that also destroys the driver.
- Both admission gates wipe their own evidence subtree right after
  Files.createDirectories: runs share the fixed
  target/wedge-evidence directory, so an earlier wedged run's artifacts
  otherwise survive into a later green run's bundle, and node-reports/
  accumulates its SIGUSR2 dumps across runs forever.
- HostedBackgroundPublicationGate's javadoc no longer credits this PR
  with the H3 family admission: #13789 landed that at the merge base,
  and requireCheckpoint already allowed the monitor family there; this
  witness's own delta is PUBLISHABLE_TOOL_NAMES, boundedIntField and
  the shared payloadVectors corpus. The monitor arm's closing claim
  re-points at the store-side checkpoint gate reading the same constant.

* ci(sdk-java): collect the hosted admission gates with a hosted-gates roster family

The hosted-harness-gates profile from this PR's round-2 merges matched
no automated lane: surefire defaults skip *Gate.java, no workflow
referenced the profile, and check-failsafe-reports had no roster mode
for it — every lane stayed green without ever executing the witnesses.
Wire it the way -Po4-mysql-gates is wired:

- A new 'Verify Hosted admission gates' step in the Hosted process fault
  gates job, after that lane's report upload so its clean cannot discard
  the hosted lane's already-uploaded failsafe reports and fg7-cases
  evidence, with a 50 minute ceiling above the profile's 2400 s shared
  fork budget plus the wrapped compile/surefire/checkstyle work, and a
  matching 'Check that every Hosted admission gate class ran' roster
  step through a new hosted-gates family (/^Hosted.*Gate$/).
- check-failsafe-reports.js registers the family beside the o4 modes
  with its usage string updated; Hosted*Gate classes stay out of both
  existing 'hosted' / 'non-hosted' families.
- scripts/tests/check-failsafe-reports.test.js gains the new family's
  happy/outside-family/ran-no-test pins, and the upper bound in
  scripts/tests/hosted-process-ci.test.js moves from (145, 158, 160) to
  (195, 208, 210) for the added step.

* test(integration): bind the completed-path sentinel to driveProbeTurn's transcript gate

doudouOUC's verified mutation on dbId 4255682183 showed the R3-18 sentinel
stayed green after removing the completed-path transcript gate: the slice
started at the first 'const transcript = await daemon.json(', so with the
gate deleted indexOf slid forward into wedgeEvidence, whose predicates are
byte-identical. Bound the inspected source to driveProbeTurn, ending before
wedgeEvidence, so removing the gate now turns the sentinel red. Verified
both directions at this head: 4/4 pass unmutated; deleting the gate fails
the first toContain on '/transcript?cursor=0&limit=256'; wedgeEvidence's
fallback predicate supply is unreachable after slicing.

* test(sdk-java): refuse evidence wipe roots outside the module target

yiliang114's P3 (dbId 4256822620) on the wedge gate's recursive wipe: the
resolved evidence root comes straight from -Dqwen.wedge.evidence and was
recursively cleared with no containment check, so a local run pointed at a
shared directory would delete unrelated content there. Both Hosted*Gate
classes now assert the normalized evidence path stays under the module
target/ before any recursive wipe, in the same assertion style the file
already uses for its wrapper paths.

Verified both directions on this head: with the property pointed outside
target/, HostedRecoveryWedgeGate and HostedBackgroundPublicationGate fail in
under a second at the new assertion and seeded sentinel files outside
survive; with the default target/wedge-evidence path both gates proceed past
the guard and fail only later at the intended MySQL stage (Druid/Flyway
communications link failure against a dead port), leaving
target/wedge-evidence/{wedge,publication} created.

---------

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: 易良 <1204183885@qq.com>
2026-10-13 13:12:53 +00:00
..
baselines test(managed-agent): record Hosted latency baseline (#12945) 2026-09-29 15:18:48 +00:00
cli feat(managed-agent): recover Hosted G3 parked turns (#13777) 2026-10-12 13:05:52 +00:00
concurrent-runner chore: enforce Playwright version parity and close the deferred #11101 follow-ups (#11371) 2026-09-12 11:40:46 +00:00
fixtures feat: chat transcript mr2a html export (#10076) 2026-09-04 14:36:35 +00:00
helpers fix(managed-agent): numeric parity and family-list hoisting for publication families, with end-to-end admission and wedge witnesses (#13815) 2026-10-13 13:12:53 +00:00
hook-integration fix(integration-tests): make the project typecheckable and fix what that found (#8693) 2026-08-08 16:32:31 +00:00
interactive fix(core): surface PreToolUse ask content on MCP tool confirmations (#13687) (#13697) 2026-10-09 02:28:32 +00:00
sdk-typescript feat(core): defer agent and goal declarations by default (#13033) 2026-10-06 04:13:24 +00:00
terminal-bench fix(integration-tests): make the project typecheckable and fix what that found (#8693) 2026-08-08 16:32:31 +00:00
terminal-capture feat(web-shell): present structured shell execution results (#12311) 2026-09-21 11:32:42 +00:00
channel-plugin.test.ts fix(integration-tests): make the project typecheckable and fix what that found (#8693) 2026-08-08 16:32:31 +00:00
chat-transcript-contract.test.ts feat: chat transcript mr2a html export (#10076) 2026-09-04 14:36:35 +00:00
chat-transcript-document.test.ts perf(export): split the transcript renderer's embedded CSS into a versioned asset (#11485) 2026-09-10 07:15:32 +00:00
e2e-memory-defaults.ts test(integration): disable managed auto-memory by default in E2E harnesses (#13009) (#13013) 2026-10-01 14:34:35 +00:00
export-artifact-contract.test.ts feat(web-shell): expose slash-command exports as artifacts (#12050) 2026-09-20 02:37:25 +00:00
fake-dashscope-server.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
fake-openai-server.test.ts fix(cli): close the remaining OpenTUI parity gaps against ink (#12309) 2026-09-22 07:18:27 +00:00
fake-openai-server.ts fix(cli): close the remaining OpenTUI parity gaps against ink (#12309) 2026-09-22 07:18:27 +00:00
globalSetup.test.ts feat(core): resolve model limits and modalities from a models.dev catalog (#11959) 2026-10-02 14:39:11 +00:00
globalSetup.ts feat(core): resolve model limits and modalities from a models.dev catalog (#11959) 2026-10-02 14:39:11 +00:00
qwen-live-harness.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
qwen-live-m1-call.test.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
qwen-live-m2-inject.test.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
qwen-live-m2-permission.test.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
qwen-live-m2-steering.test.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
qwen-live-m4-acp-call.test.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
qwen-live-m4-acp-multibackend.test.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
qwen-live-m4-acp-permission.test.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
qwen-live-m4-acp-steering.test.ts feat(live): add visual input, proactive assistance and memory (#11369) 2026-09-11 03:13:13 +00:00
ready-prompt-budget.ts test(integration): let workflow-completion reply waits inherit the helper default (#13552) (#13555) 2026-10-08 09:09:39 +00:00
reasoning-overrides.mjs feat(core): support external reasoning profiles and defaults (#11856) 2026-09-15 11:44:17 +00:00
renderer-matrix.test.ts feat(opentui): bundle assets, CI matrix and parity tooling (Batch 7) (#10770) 2026-09-02 08:36:20 +00:00
renderer-matrix.ts feat(opentui): bundle assets, CI matrix and parity tooling (Batch 7) (#10770) 2026-09-02 08:36:20 +00:00
scratch-dir.test.ts test(e2e): isolate protocol checks and ACP test state (#11286) 2026-09-08 05:05:36 +00:00
scratch-dir.ts test(managed-agent): Make the Hosted no-tool gate assertions falsifiable (#12819) 2026-09-27 05:42:46 +00:00
skill-hooks-invocation-parity.test.ts fix(skills): re-apply a Skill's side effects when a session is resumed (#11280) 2026-09-14 01:02:42 +00:00
skill-hooks-resume.test.ts fix(skills): re-apply a Skill's side effects when a session is resumed (#11280) 2026-09-14 01:02:42 +00:00
test-helper.test.ts test(integration): disable managed auto-memory by default in E2E harnesses (#13009) (#13013) 2026-10-01 14:34:35 +00:00
test-helper.ts feat(core): resolve model limits and modalities from a models.dev catalog (#11959) 2026-10-02 14:39:11 +00:00
test-mcp-server.ts # 🚀 Sync Gemini CLI v0.2.1 - Major Feature Update (#483) 2025-09-01 14:48:55 +08:00
tsconfig.json feat(channels): add IMAP and SMTP email channel (#12939) 2026-10-03 02:54:25 +00:00
vitest.config.ts ci: serialize integration tests on shared ECS runners (#10567) 2026-08-31 01:20:35 +00:00
vitest.firstoutput.config.ts fix(test): Restore first-output benchmark measurement validity and correct its artifact schema (#7820) 2026-07-28 06:51:03 +00:00
vitest.hosted.config.ts test(managed-agent): add Hosted no-tool process gates (#12733) 2026-09-26 08:33:27 +00:00
vitest.loadtest.config.ts feat(daemon): merge daemon-mode feature batch into main (#4490) 2026-06-12 00:34:49 +08:00
vitest.terminal-bench.config.ts Fix E2E caused by Terminal Bench test (#529) 2025-09-08 10:51:14 +08:00