The two cancellation-fence it.each blocks (and two sibling call sites) in
hosted-mcp-session.test.ts poll vi.waitFor with vitest's hard-coded 1s
default window around conditions reached through the shared main-CI host's
scheduling and fsynced durable-write stalls. On the post-merge run —
coverage enabled, four forks on a contended ECS host — that window can be
exceeded even though nothing is wrong; the stalled poll rethrows its last
assertion ('expected false to be true') after 1s. Which of the six phases
exceeds the window on a given run is a dice roll, matching the mixed
pass/fail pattern in the issue.
Reproduced the failure signature by injecting a 1.2s stall ahead of each
block's 'entered' flag: all six phases failed at the first waitFor with
'expected false to be true'. With the explicit 10s window the same probe
passes; without the probe the full file stays green (49/49). The logic
under test is deterministic, so widening the poll window is the
root-cause fix — the same fix#13323 applied to the sibling Hook fence
suite. Assertions unchanged.
* docs(managed-agent): draft the FG12b and FG12e fault-gate designs
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* docs(managed-agent): correct the FG12b and FG12e designs after audit
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* docs(managed-agent): address the FG12b and FG12e design review
* test(managed-agent): add the FG12b MCP store fault gates
The eight-case ledger and a test-only faulting record store wrapper are in place and compile clean. The class is not yet green on JDK 21 + MariaDB: store-reject-settled, -releasing, -drained, -released and -reload-chain still fail. The shared fork budget re-pin (hostedForkHealthySeconds, CI fork literal, pom and workflow comments) is not done; only the roster entry is updated.
* docs(managed-agent): close the remaining FG12b/e review findings
* test(managed-agent): re-pin the FG12b fork budget
The FG12b store-failure class is green 8/8 on JDK 21 + MariaDB (49.0 s measured). Re-pin hostedForkHealthySeconds to 4858 s, the CI fork to 5468 s, the Verify step to 102 minutes, and the job step sum to 182, with the workflow, pom and meta-test comments updated in the same diff.
* test(managed-agent): harden the FG12b store fault gates
Round 1 audit fixes: the settled cell asserts the dispatch_started record and zero response rows after the failed commit, reopens the stopped authority with the original operation/runtime pins, and reaches that rollback assertion under a skip-rejection mutation (red: expected dispatch_started, was settled). The released cell pins the exact owner-release path and requires two confirmed 200/released releases, with exactly one resend on the next detach. Owner/storage retention now compares the exact runtime-session/generation identity, reload-chain asserts the settled rev 2 and no duplicate chain, and the static gate is cleared in teardown. Local evidence is MariaDB 10.11, not the lane's MySQL 8.4.6. Budget: 120 s allowance for the measured 50.3 s class -> healthy 4878 s, fork 5488 s, Verify 102 minutes.
* docs(managed-agent): design the FG12f stream gap gates
* test(managed-agent): hold the MCP fixture call until the test releases it
* style(managed-agent): format the FG12b design tables
* docs(managed-agent): correct the FG12b budget provenance and release cell
* test(managed-agent): drop the unused FG12c helpers from the FG12b class
* style(managed-agent): realign the FG12b design tables
* test(managed-agent): add the FG12f MCP stream-gap gates
One MCP tool call is parked inside the stdio fixture while a client reads the Session stream, the client is cut mid-call, and the resumed client must continue the same ledger exactly once. The public route resumes with Last-Event-ID and the WebShell route with afterSequence, so a cursor that is ignored replays and reddens the union on a duplicate instead of timing out. The third case replaces the configuration while the call is held and pins that the running Session keeps its revision, its server and one contiguous stream.
* docs(managed-agent): re-pin the FG12f budget to the merged base
The parent branch merged main's event-stream liveness gates, which added 440 s to the shared fork's healthy sum before this class. The measurement is unchanged (104.6 s, 210 s budgeted) but the totals it lands on are now 5803 s healthy, 6403 s fork and a 117-minute Verify step.
* test(managed-agent): drive the real MCP configuration route in the FG12f replacement case
The third case now posts the Harness-private configuration route while the call is held, so the committed configuration generation advances to revision 2 and the released call keeps reading it; the immutable Agent-definition append it used before never reached the replace path. All three cases also pin the fixture's physical effect: one mcp.effect record and one durable proof row, so a resume that re-invokes the tool reddens even when the published completion is unchanged.
* test(managed-agent): pin the FG12f replacement case to the public definition route
The Harness-private configuration route requires the client id the Broker mints for the Harness session, which a Spring-created Session never exposes, so the case answered 404. The case now commits a new MCP server revision through the public definition route while the call is parked and pins that the running Session keeps revision 1, the replacement is stored once, one server start exists, and the fixture's physical append happens exactly once.
* docs(managed-agent): narrow the FG12f replacement case to what it pins
The case commits a new MCP server revision while the call is held; it does not replace the running Session's MCP connection. The Harness-private configuration route cannot carry that here because it requires the client id the Broker mints, which a Spring-created Session never exposes, so it answers hosted_session_not_found. The design now records runtime connection replacement as a gap instead of claiming it.
* test(managed-agent): re-pin the FG12f budget to the merged-base measurement
The three cases measure 121.1 s on the merged base rather than the 104.6 s they measured before it, so the class is budgeted at 2x (242 s): the healthy sum becomes 5985 s, the fork 6585 s and the Verify step stays at 120 minutes.
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: 易良 <1204183885@qq.com>
* test(e2e): widen the channel-worker sandbox budgets (#14584)
The channel-less-boot scenario starts a nested Docker-sandbox ACP session
that loads the linked channel extension (#10318), so its cost is bounded by
the daemon's initialize budget rather than by the fixture's host-side work.
Run 39121969242 spent 139.9s across three attempts on a contended
sandbox:docker shard - one attempt hit the 60s ceiling - while the same test
passes in 16.3s on a healthy docker shard and the same job's sandboxed CLI
starts took 22-26s each.
Raise the fixture's initialize budget from 60s to 120s and the scenario's
ceiling from 60s to 240s. Every assertion and input is unchanged; only the
ceilings move.
* fix(channel): let the worker's daemon requests outlast a slow sandbox start
The channel worker's DaemonClient used the SDK's 30s default request budget,
which also caps POST /session - the call that waits for the ACP child's
initialize inside the workspace sandbox. That client deadline, not the
daemon's initialize budget, is what turned a slow start into
"Could not create task": #14584's failing attempts returned about 40s after
the test began, worker connect plus the 30s cutoff.
Give the worker a request budget that covers a 120s sandbox-scale initialize
budget with headroom. prompt() is already exempt from fetchTimeoutMs, and the
supervisor's own startup/delivery budgets stay at 30s, so only the daemon
calls that can legitimately be slow get the wider window.
Refs #14584.
* test(e2e): correct the channel-worker budget comment (#14584)
The fixture no longer matches qwen-serve-routes.test.ts - the sibling fixtures
stay at 60s - and the 240s ceiling absorbs two full 120s starts, not four: the
scenario creates two tasks before the daemon restart and reloads both after.
Say both plainly.
* revert: drop the client-wide request budget in favor of #14588
The client-wide fetchTimeoutMs also capped session resume at 180s, so a daemon
advertising a restore budget above 170s regressed, and it bypassed #14588's
derived session-operation deadline for the very caller that hit the bug. #14588
owns the client-side deadline; this PR keeps only the fixture ceilings, which
the third failed attempt needed on its own.
Refs #14584, #14588.
* test(e2e): give the channel-worker ceiling room for the scenario's fixed cost
The 240s ceiling was exactly 2 x 120s, so two slow starts at the full budget
left nothing for the daemon spawn, worker boot, channel connect and restart
(~4.5s measured on a quiet host, more in the sandbox). Raise it to 300s and say
what the extra 60s buys.
---------
Co-authored-by: 易良 <1204183885@qq.com>
* docs(managed-agent): draft the FG12b and FG12e fault-gate designs
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* docs(managed-agent): correct the FG12b and FG12e designs after audit
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* docs(managed-agent): address the FG12b and FG12e design review
* test(managed-agent): add the FG12b MCP store fault gates
The eight-case ledger and a test-only faulting record store wrapper are in place and compile clean. The class is not yet green on JDK 21 + MariaDB: store-reject-settled, -releasing, -drained, -released and -reload-chain still fail. The shared fork budget re-pin (hostedForkHealthySeconds, CI fork literal, pom and workflow comments) is not done; only the roster entry is updated.
* docs(managed-agent): close the remaining FG12b/e review findings
* test(managed-agent): re-pin the FG12b fork budget
The FG12b store-failure class is green 8/8 on JDK 21 + MariaDB (49.0 s measured). Re-pin hostedForkHealthySeconds to 4858 s, the CI fork to 5468 s, the Verify step to 102 minutes, and the job step sum to 182, with the workflow, pom and meta-test comments updated in the same diff.
* test(managed-agent): harden the FG12b store fault gates
Round 1 audit fixes: the settled cell asserts the dispatch_started record and zero response rows after the failed commit, reopens the stopped authority with the original operation/runtime pins, and reaches that rollback assertion under a skip-rejection mutation (red: expected dispatch_started, was settled). The released cell pins the exact owner-release path and requires two confirmed 200/released releases, with exactly one resend on the next detach. Owner/storage retention now compares the exact runtime-session/generation identity, reload-chain asserts the settled rev 2 and no duplicate chain, and the static gate is cleared in teardown. Local evidence is MariaDB 10.11, not the lane's MySQL 8.4.6. Budget: 120 s allowance for the measured 50.3 s class -> healthy 4878 s, fork 5488 s, Verify 102 minutes.
* style(managed-agent): format the FG12b design tables
* docs(managed-agent): correct the FG12b budget provenance and release cell
* test(managed-agent): drop the unused FG12c helpers from the FG12b class
* style(managed-agent): realign the FG12b design tables
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: 易良 <1204183885@qq.com>
* fix(managed-agent): free Workspace storage after a refused release
A worker that refuses, times out, or hangs on release skipped the final
ownership.release, leaving ma_storage_owner.holder_key stranded. Every later
acquire then failed 409 workspace_busy, and nothing else cleared the row.
The worker now bounds its memoized release and answers before the Broker's
HTTP timeout, mapping a refusal to managed_runtime_owner_hold_pending and a
deadline to managed_runtime_release_timeout. A stale attempt cannot retire a
Session that was re-acquired after the timeout.
The Broker passes those codes through, and the drain cascade escalates one
refusal to the durable stop it already owns: stop the worker, persist the
receipt, finish the Session locally, then clear ownership through the new
transport hook. That hook clears only an exact holder whose binding is
DRAINING with a matching drain receipt, so a mere transport failure still
retains ownership as the acquire side requires.
* fix(managed-agent): keep a worker release refusal retryable
providerFailure() already recognized managed_runtime_owner_hold_pending but
returned it non-retryable, so releaseSavedSession rewrote it to
workspace_close_identity_unverified and the drain escalation never matched.
Mark the ownership codes retryable and recognize release_timeout on the same
path; the provider transport test pins both codes.
* fix(managed-agent): gate the retired-holder scan on a persisted receipt
On the plain close path the receipt from stopAcknowledgedRuntime reaches the
row only in the final CAS, so releaseRetired could not prove anything and the
scan only re-read every retired Session. Scan only when renewal.current already
carries the receipt; that path has cleared every holder by acknowledgement,
while the absence and escalation paths persist the receipt before the scan.
* test(managed-agent): pin the release-retired proof predicate
Cover both sides of the safety boundary: a DRAINING binding with a matching
receipt clears the holder, and the same binding without a receipt while the
holder is set returns false without calling ownership.release.
* test(managed-agent): drop the hook-cancel reply through a broker proxy
Front the embedded broker with the existing forwarding proxy and arm a
DROP on the hook-cancel control reply. The owner already aborted, so the
gate must resolve the same cancellation through a later status poll, drain
the unit exactly once, and never issue a second physical Hook call.
* test(managed-agent): budget the FG13d dropped-reply cancel cell
The class gains a fourth fixture, so the shared Hosted fork grows by 150 s:
healthy 4908, fork 5518, Verify 105 minutes and job 200, with the three
pinned comments kept in step.
* docs(managed-agent): record the FG13d dropped-reply cancel case and witness
* feat(managed-agent): bound half-open Hosted event streams behind a keepalive capability (#14135)
The Harness advertises its events-route keepalive as event-keepalive/1,
and the Java client arms the idle watchdog only against a Harness that
advertises it, so a rolling upgrade never aborts the quiet streams of an
older Harness. The abort is now the typed HostedHarnessStreamIdleException:
the coordinator retries it through the existing re-attach path and never
settles or cancels the Turn on silence alone. The watchdog ticks every
tenth of the bound, so an abort lands 45 to 49.5 s after the last byte.
HostedEventStreamLivenessIT runs the probes on the packaged chain behind
a TCP relay that holds the events connection half-open or stalls it, and
the hosted-harness-mysql fork budget grows to 4793 s. A design note
records the numbers, the capability, the outcome and the other Hosted
streams.
Closes#14135
* test(managed-agent): frame relayed requests and see virtual-thread readers in the liveness gate
The event-stream relay classified a connection, rewrote its Host header
and matched keepalives within single reads, so a request or a keepalive
split across TCP segments escaped the stall and half-open injection or
the cadence count. It now buffers each request head, forwards the body
by its Content-Length, matches keepalives across reads, and records and
resets a request it cannot frame. HostedEventStreamRelayTest pins each
split.
The half-open probe checked reader release with Thread.getAllStackTraces(),
which leaves out the virtual threads Turn readers run on. It now reads a
JDK thread dump, and first requires that dump to show the parked reader.
* test(managed-agent): count each keepalive one relay read carries (#14135)
The relay recorded one stamp per read that held a keepalive, so two keepalives in one read counted once, and the split-keepalive case could report a size of 1 whenever the pieces happened to land together. It now counts every match in the read window; the carried tail is a byte shorter than a keepalive, so none counts twice. A new case sends two keepalives in one write.
* test(sdk-java): park the published-window probe mid-interval (#14135)
The watchdog's ticks run from the stream's creation, and the probe parked right after it. A 2000 ms bound is a whole number of half-budget 1000 ms ticks, so that park sat on a tick of the mutant too, and wherever the scheduler ran a few ms late (macOS) the mutant aborted at ~2000 ms and passed. Parking 300 ms in puts the park mid-interval for both ticks: the shipped 200 ms tick aborts at 2100 ms and a 1000 ms tick at 2700 ms, so the limit drops to 2400 ms.
Keep per-Session root and head waits from holding the publication guard through the database lock-wait timeout, while preserving ordinary writer behavior and shorter configured statement budgets.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
When `rg` is not installed, the file-search crawler printed an alarming
block of raw failure diagnostics for a single failed spawn, and printed
it twice:
[crawler] command spawn failed: rg --files --no-require-git --hidden ENOENT Error: spawn rg ENOENT
[crawler] falling back to fdir (ripgrep unavailable)
[crawler] command failed: rg --files --no-require-git --hidden exit=-2
Two defects, fixed separately:
- The ripgrep probe is an optional-capability probe whose failure is
already narrated by the "falling back to fdir (ripgrep unavailable)"
warn, so its raw spawn diagnostics are pure noise. Pass
`silentOnFailure: true`, matching the three git probes that already do.
- `runCommand` consulted its `settled` flag only inside `finalize()`,
which guards `resolve()`. The logging sites sit outside that guard, so
when a spawn fails the `error` handler logs "command spawn failed" and
the `close` handler then re-reports the very same failure as "command
failed ... exit=-2" (libuv surfacing UV_ENOENT as the close code on
Linux). Add a `!settled` leg to the close handler's condition.
Deliberately out of scope: routing the probe through a resolved ripgrep
binary, and memoizing "rg is unavailable". The `git ls-files --cached`
call is also left non-silent on purpose - its degradation is already
narrated, and keeping it loud leaves the double-report observable.
Refs #14537
Patrol-Run: qwen-issue-patrol/jmvdlnotoa2
Co-authored-by: yiliang114 <yiliang114@users.noreply.github.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
A reconcile_run repair checked only session.active, which the wake pump
does not hold while it reads an input, projects the transcript or
classifies a crash. A repair that finished inside one of those awaits
left the pass acting on a consumed input, and the Session ended
recovery-blocked until a reload (#14538). A message stop's settle of a
crashed message turn had the same race.
HostedMonitorWakeScheduler.exclusive() runs work with no pass in flight
and none able to start. A pass in flight hands the pump straight to the
first waiter when it returns, and the pump stays held until that work
returns. A pass still running after the wait keeps the pump and the
work never runs; later callers do not wait for the hold in flight when
that wait ran out. Closing the pump lets its waiters go without their
work.
Both routes settle inside it, waiting at most HOSTED_WAKE_PASS_WAIT_MS
(10 s) for a pass, and only when they have something to settle. Each
re-reads its guard after the wait, since a prompt or a close may have
claimed the Session meanwhile:
- reconcile_run: a load pass that settled the crash leaves it nothing
to repair, and it answers with the run that pass failed, which lets
the occurrence resolve. A pass that ended blocked is repaired by the
route as before.
- A message stop leaves a crash that the pass left blocked to the
pump's blocked cycle, and anything a prompt now holds to the relay's
next stop.
The pump also re-arms when a pass ends blocked and its settle-verify
read finds nothing. A stopped Session's message input is filtered out
of that read, so such a pass armed no retry, and the block stayed until
an unrelated kick or a reload.
Fixes#14538
When the store refuses a tool publication reservation at a limit, the
Hosted Harness answers the refused batch as not started (#14452). For a
429 it told every call "Retry after the running tools finish." That is
wrong for two shapes of batch:
- A batch with more output-capturing calls than its Session could ever
hold at once can never reserve. The model was told to wait and could
repeat the same batch every turn.
- Calls that need no reservation at all (a read, an edit, an agent)
got the same hint, although no limit applied to them.
The store now puts `admits` into the refusal's details (and its WARN
line): how many reservations of this size the Session could hold at
once under every limit. The tenant and Session byte limits are reduced
by the Session's own used bytes, which stay charged until its output is
collected after it retires. Nothing else is subtracted: a pending
finish can still give its terminal back, an open capture gives back
what it holds beyond its used bytes, and other Sessions retire. So
`admits` is an upper bound, and a 429 always admits at least one.
ManagedSessionStoreHttpError carries the envelope's other scalar fields
as `details`, including when the retry loop rewraps a transient or
contended failure.
The Harness compares the batch's reservations with `admits`:
- More than it: the capturing calls are told the Session can run at
most that many of them at a time and to send at most that many in one
batch, and the refused reservation is not retried.
- Otherwise, a batch of several capturing calls is also told it may
send fewer of them, since fewer may fit while other tools run. A
single capturing call keeps the old answer.
- Calls that reserve nothing are told the batch was refused for other
calls, and that they can go in a batch without them. This applies
under a 409 as well.
A 409 keeps its answer to the capturing calls, and so does a 429
without a usable count, apart from the "send fewer" hint above.
Refs #14518 (items 1 and 2).
A third method of HostedAutomationFaultGatesIT, with four cases:
- store-transport: the store commits a manual run's revision 1 and its
answer to the Harness is lost; the store client's retry meets the
commit the store holds, and the run fires once with no restart;
- revision-1 / revision-2: the journal refuses one revision of a manual
run's fire; after the store client's three attempts the caller gets a
retryable 503 automation_operation_unknown, the Session writes nothing
more until it is reloaded, and after a cold restart the same key fires
the same run once;
- projection: the schedule mirror's projection of an update is refused
inside the commit's transaction, so the revision rolls back with it;
after a restart the retry commits revision 2 once.
Only this method fronts the Session Store with a relay and runs a 15 s
writer lease, so a restarted Harness takes the Session over quickly.
Four mutation witnesses are red, one of them a bundle patch of the
Harness's write latch. Found while writing it: right after a restart the
public routes pass the cold Harness's writer conflict through as
409 managed_session_writer_conflict, which the gate retries through.
Also folds in #14551's two teardown nits: each trigger drop gets its own
try, and the refusal table is dropped too.
The method measures 62.1-72.4 s and is budgeted at ~145 s: the healthy
sum moves to 5033 s and the fork to 5643 s, inside the Verify step's 105
minutes.
A recurring task that ages out is fired raw one last time and then
deleted. That final fire rebuilt its job from the on-disk task record,
so it carried the previous fire's lastFiredAt -- or none at all for a
task that never fired -- instead of a stamp for this fire. Consumers
read lastFiredAt as "Triggered at" and fall back to their own clock
when it is missing, which a buffered delivery makes arbitrarily late.
Stamp the final jobs with the detection minute the way the catch-up
branch already does, and pin it with a test that fails on the previous
behaviour.
Part of #11738
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Keep the exclusive root UPSERT as the single initialization path before creating the Session event stream.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* fix(managed-agent): verify ambiguous Linux workspace birth times
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): pin the #14461 recovery accessors in the session lock-order audit (#14505)
#14461 added hasLiveTurn and markTurnRecoveryBlocked to
ManagedAgentStore without updating the pinned turn-row accessor set in
SessionRowLockOrderTest, so the exact-set assertion fails on any tree
containing both — including this PR's merge commit, whose fast tier is
red because of it. hasLiveTurn is a read-only helper and
markTurnRecoveryBlocked is fenced by the live dispatch lease rather than
the session-row lock, and the audit finds no lock-ordering violation in
either, so both belong in the pinned accessor set.
* docs(managed-agent): note Maven report output breaks the root ESLint sweep (#14505)
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com>
Co-authored-by: qwen-code-autofix <qwen-code-autofix@localhost>
* fix(managed-agent): preserve rotated keys during operator recovery
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): scope-key the Hosted lane's ma_resource helper reads
The FG7c adapter-stop and FG6c crash gates timed out in the hosted
process-fault lane when their test helpers read ma_resource by raw
tenant/session/resource IDs: the table's indexes are resource_key and
(session_scope_key, resource_id), so those reads full-scan a table the
whole shared fork accumulates into. Lead the same reads with the
session scope key, keeping the raw IDs as residual predicates — the
shape production reads and HostedGateTestSupport already use.
Timeouts and assertions are unchanged.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com>
The hook published canUp/canDown for history navigation buttons that were
never built, and recomputed them on every push, navigation step and reset.
Each recompute allocated a fresh object, so React re-rendered the host
composer six times per navigation cycle to update state with no reader.
Remove the state, its sync helper, the six call sites and the stale comment
describing the missing buttons, and add a render-count witness so the wasted
re-renders cannot return silently.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* fix(channels/base): strip zero-ink code points from display projections
PROMPT_UNSAFE_INVISIBLES enumerated the C1 block, Cf and the variation
selectors, so a projection built only from Default_Ignorable fillers
(U+3164/U+115F/U+1160/U+FFA0), U+2800 or a lone surrogate survived
untouched: the line rendered as empty while still carrying
attacker-chosen bytes.
Derive the set from Unicode properties the way git-remotes.ts already
does, and list U+2800 plus the lone-surrogate range explicitly since
neither is Default_Ignorable. The u flag matches by code point, so an
intact surrogate pair never lands in that term.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* fix(channels/base): keep advertised attachment paths byte-resolvable (#14510)
The widened PROMPT_UNSAFE_INVISIBLES class also backs sanitizePromptPath,
which renders the `saved to:` path sent to the model, while adapters store
attachments under the original basename (DingTalk: basename(fileName)). A
filename carrying a zero-ink char (U+3164 HANGUL FILLER, U+2800 BRAILLE
PATTERN BLANK, a lone surrogate) was advertised with a space in its place,
so the read-file tool got ENOENT for a file that exists on disk.
Keep a path-scoped strip set at the pre-widening class (line-breakers, bidi,
C1, Cf, variation selectors) for sanitizePromptPath; zero-ink chars cannot
break or reorder the rendered line, so stripping them buys nothing and costs
resolvability. Adds a regression asserting byte-identity for both chars.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: qwen-code-ci-bot <qwen-code-ci-bot@users.noreply.github.com>
* test(managed-agent): fix the FG12d post-merge evidence and ledger P3s
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): resolve the FG12d case letter before boot can fail
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
A workspace-scoped delta must not reach a session admitted at a different root, and the change must still be acknowledged. The user-scope positive case landed in #14525; this is its discriminating pair, so dropping the workspace filter turns it red.
Refs #12737
* test(managed-agent): add the FG11b Spring-side automation store-fault gates (#14446)
A second method of HostedAutomationFaultGatesIT, on the FG11a boot, with
six cases sharing one minute boundary:
- occurrence-insert: a refused occurrence record fires nothing, and the
slot fires once after the refusal drops;
- occurrence-settle: a refused settle keeps the occurrence firing and
re-driven against the same run, one run chain and one model Turn;
- watermark: a refused watermark advance re-derives the slot every tick,
and its recorded decision stands;
- receipt-claim / receipt-settle: a refused request receipt relays
nothing, or re-drives the same operation id once it drops;
- revision-race: a fire parked while an update commits revision 2 meets
automation_revision_stale and is re-pinned and fired once, never
skipped.
Refusals are tenant-fenced MySQL triggers (the FG9b/FG10c pattern); the
relay gains a hold that parks an operation before the Harness. Six
mutation witnesses are red; one single-layer watermark mutation is
equivalent and recorded. The Harness-side store faults are slice 2.
The method measures 30.8-64.5 s locally and is budgeted at ~130 s: the
healthy sum moves to 4738 s and the fork to 5348 s, inside the Verify
step's 100 minutes.
* test(managed-agent): pin FG11b's recovered occurrence to the refused slot (#14446)
Review round 1: occurrence-insert took the first recorded scheduled
occurrence after the refusal dropped, without checking it was the slot the
trigger refused. A regression that skipped past the refused slot, letting
the watermark advance, would pass on the next minute's fire.
The refusal row now records what it refused (the occurrence key, or the
watermark slot). occurrence-insert and occurrence-settle check that every
refusal names one slot and that the slot they recover is that one.
A/B with a mutation that swallows the refused decision so the watermark
advances past it: the previous assertions stayed green (163.3 s, the next
minute fired in its place); the new ones go red at "every refusal is the
one slot's record".
---------
Co-authored-by: 易良 <1204183885@qq.com>
The sealed-Managed arm of the lock-record validator is the only thing
that stands between a corrupt commit proof and a certified takeover
being handed that corrupt proof, yet nothing pinned it: mutating any one
of its four checks left the whole deterministic suite green, and a real
daemon archived the corrupted seal.
Corrupting only the commit proof of an otherwise valid sealed record now
has a witness, so acquire fails closed on a malformed lock instead of
treating the record as a seal. A fractional, textual, negative or absent
sequence and a short, uppercase, non-hex or array-wrapped hash are each
rejected; a zero sequence stays a genuine seal, pinning the bound as
>= 0 rather than > 0.
Each of the four checks is now load-bearing: dropping the safe-integer
guard, the non-negative bound, the string guard or the 64-hex pattern
turns at least one new case red.
Part of #13443
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
The main CI watcher and the PR scan share one infrastructure-signature
list. A per-host SDK Java test lock starving past its flock budget carried
no signature, so main run 39104863059 filed #14550 for a lane in which no
test ever ran.
The annotation names a lock budget, not an assertion: several runner
instances share one HOME on the ECS pool, so a fast-tier job can wait
behind sibling Maven holds that outlast its own acquisition budget. Main
runs now rerun that class once instead of filing, and a red check on a PR
is reran once; the existing attempt-2 guard still files when the
starvation is not transient.
The new pattern also covers the E2E and release docker lock messages,
which share the same starvation semantics.
* test(managed-agent): cover workspace-change request validation
A workspace change with a non-integer revision or an unknown kind is rejected before it applies anything, and neither answer can be mistaken for the acknowledgement the Bridge expects. Pin both.
Refs #12737
* test(managed-agent): discriminate the workspace-change validation error
Assert the validation message instead of any throw, and use a non-string kind, so removing a check cannot pass the case through an unrelated TypeError.
Refs #14525
* test(managed-agent): pin that a user-scope permission change reaches a worktree session
A user-scope permission delta must reach every live session in the child, including one admitted at a worktree root. The workspace-scope filter must not silently drop it while the change is acknowledged. This is the audience half of the permission path; the workspace-filter negative case follows.
Refs #14525
* test(managed-agent): keep the unknown-string kind rejection covered
Switching the second case to a non-string kind pinned the type check but stopped exercising the switch default arm. Add a third case with an unknown string kind and assert its own message, so the default rejection cannot be removed while the suite stays green.
Refs #14525
---------
Co-authored-by: 易良 <1204183885@qq.com>
* test(managed-agent): add the FG12d MCP release fault gate (#14447)
* test(managed-agent): pin the FG12d release and settle witnesses
Require the held release dispatch to answer running before the kill, so
a synchronously settled release cannot satisfy the lookup witness; filter
the status witness to a settled 200 for the original release id; assert
the released server exits once instead of claiming it survives; accept
the lifecycle 409 conflicts while retrying settle_resources; and correct
the docs' saveEffects/status/timing claims.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): tighten the FG12d settle retry and status target
Retry only hosted_turn_active on the lifecycle route; a
hosted_lifecycle_operation_conflict cannot self-heal and should fail
with its body. Join the settled-status witness to the original
runtimeSessionId, and correct the docs' latest measured seconds.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: 易良 <1204183885@qq.com>
* test(managed-agent): pin the FG13d cancel against a stopped worker
SIGSTOP the durable worker while the command holds, then cancel through
the Harness route. The owner cannot answer the lookup, so the record must
stay recovery_blocked/outcome_unknown and keep the unit populated instead
of inventing an end. Adds 150 s to the shared fork budget and records the
matching mutation witness in the design.
* test(managed-agent): hold the stopped-worker fixture indefinitely
SIGSTOP freezes the worker, not the fixture's own hold timer: a 60 s hold
would end during the two 30 s owner lookups, kill the child and write the
allow, so a correct implementation could fail on scheduling. Hold until
teardown instead, matching the worker-kill cells.
---------
Co-authored-by: 易良 <1204183885@qq.com>
The reconcile_run route can run a crashed wake turn's aftermath itself:
when the Session's pump is not running one, the route claims the turn
slot, settles the crash, lifts the recovery block and answers
`repaired: true`. No test reached that branch. The existing "repairs
through the reconcile route while blocked" test is settled by the
pump's own aftermath, and its route only reports. Dropping the branch,
dropping `repaired = true`, dropping the route's liftRecoveryBlock,
dropping its claim of the turn slot, or dropping the `finally` that
hands the slot back all kept every test green.
The new test lets the pump's first aftermath pass lose its Broker
release, which leaves the Session recovery-blocked with the crashed
input pending, and parks every later kick so the pump cannot retry. It
waits until a retry kick has been parked, so the pump's pass is over.
A reconcile whose release is refused too must answer `repaired: false`
and leave the block and the input in place. Once the Broker answers,
the next reconcile must consume the input while holding the turn slot,
answer `repaired: true`, release once, consume exactly once, lift the
block and hand the slot back. A further reconcile only reports the
settled run, and the Session closes with 204.
Co-authored-by: 易良 <1204183885@qq.com>
* test(managed-agent): add the FG11a automation lost-reply fault gates (#14446)
The first packaged lane that turns automation on: in-JVM Spring with the
scanner running, a real Harness behind a relay that drops one committed
answer, real MySQL and a model stub.
Eight cases share one boot and one minute boundary:
- create-reply, update-reply (a newer update lands meanwhile),
noop-update-reply and retire-reply: the retry answers the committed
revision and never commits again or over a newer one;
- manual-run-reply and scheduled-fire-reply: the re-drive meets the
same arun_ run, with one run chain and one model Turn;
- reconcile-reply: a lost reconcile answer leaves the slot skipped
under its own refusal, never unknown;
- public-reply: a lost public 202 replays the receipt without a second
Harness operation.
Five mutation witnesses are red; one equivalent mutation is recorded in
the design. The fixture runs one Turn per Session first: the automation
verbs only load a Session on the Harness, so one created without input
is refused 404 managed_session_not_found.
The class measures 46.3-85.0 s locally; it is budgeted at ~170 s, so the
shared fork's healthy sum moves to 4359 s and its timeout to 4969 s,
inside the Verify step's 95 minutes.
* test(managed-agent): keep FG11a's clocked cases off the scanner until armed (#14446)
Review round 1:
- The scanner ran from boot, so a minute boundary between a definition's
create and its armed fault, or its held manual run, let a slot fire
first: attempts 0 for scheduled-fire-reply, 409 automation_run_skipped
for reconcile-reply. SchedulerHold now parks the scanner's only thread
while the three clocked cases set up; manual runs fire on the request
thread and are unaffected.
- assertOneRun collected every fire and run command of the schedule, so a
legitimate second slot on a slow runner read as a duplicate. The
exactly-once checks are now scoped to the occurrence under test.
- The design docs give the Verify step's 95 minutes, describe both fixes,
and say a first red skips the later cases, which are re-run per case.
* test(managed-agent): add the FG13d Hook cancel and timeout fault gates
Extend the packaged Hook rig with a descendant that ignores SIGTERM, and
pin two arms against the real stack: a cancel must escalate to cgroup.kill
after the grace and settle cancelled only once the unit is empty; a 2 s
timeout must kill the unit, keep the output unapplied, and block the tool.
The class adds 200 s to the shared fork budget and is registered in the
pinned Hosted class list.
* test(managed-agent): drop the unused FG13c helpers from the FG13d class
The new class was copied from the owner-restart rig, so it carried the
Harness-kill, worker-kill and status helpers those cells used. This slice
cancels through the Harness route and waits on JDBC, so those helpers are
dead; removing them keeps the class to what it exercises.
* docs(managed-agent): cite the whole Hook unit terminate() range in the FG13d design
* test(managed-agent): check the drained unit at the first terminal observation
The cells waited for the terminal record and only then waited out the
descendant, so a regression that settled early while the drain continued
still passed. Inspect the first terminal observation instead: the
descendant must be gone and the unit empty at that instant, and document
the early-terminal mutation witness.
* test(managed-agent): add the FG12c MCP restart fault gates (#14447)
Kill the packaged Harness, the durable Runtime worker and the stdio MCP
server, and prove that the surviving generation never mints a second
owner, never resends a control and never fabricates a settled result.
The fixture parks a held resource read until a release file appears and
records stdin close, so the worker-kill cells assert the child's fate.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* fix(managed-agent): close FG12c review gaps and re-pin the fork budget
Wait for dispatch_started and a held configure before killing the
Harness; drive a real recovery on the same Session after an idle worker
kill and assert the identity is unchanged; validate the operation status
response and its published result instead of discarding the lookup. The
five cases now measure 49 s, so the healthy sum moves to 3938 s and the
CI fork to 4548 s.
* fix(managed-agent): wait for the refused recovery acquire in FG12c
A 202 only admits the prompt; the MCP refresh and acquire run later in
the turn's declarations. The idle-worker cell now waits for the recovery
acquire to be refused with 409 by the dead generation before it reads
the configuration identity, so a fast assertion cannot observe only the
pre-crash state. A direct 409/503 refusal must name a recovery error.
* test(managed-agent): harden the FG12c witnesses against false-green paths
Sequence the proxy wire so baseline markers cannot race an in-flight
relay; pin the idle-worker recovery acquire to the dead generation's
409 body; count invokes and server starts in the worker/stdio cases; and
pair the configure status answer with the persisted record state.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): close the FG12c round-2 high findings
Wait for the successful status entry in the recorded wire instead of the
relay-entry control, so a slow responder cannot produce a false red; and
require a direct 409/503 refusal to be backed by the dead generation's
own refusal. Wait for the fixture to exit before asserting it is gone.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): accept both dead-generation acquire refusals
The killed worker can refuse the recovery acquire with 409 'Runtime
generation is no longer live' or 503 'Managed Runtime process is not
alive.'; accept both, anchored to the :acquire request. Require the
configure status answer to be terminal before pairing it, and await the
durable worker registration before killing it.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): make the FG12c status witness terminal-aware
Select the first terminal status answer instead of locking the first
200 (which can be running), stop requiring an acquire that a
pre-admission refusal never issues, and accept the third legitimate
dead-generation refusal message.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* docs(managed-agent): sync the FG12c budget with the merged FG13c ledger
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): only retry the dead writer lease on reload
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): add the FG13c Hook owner-restart fault gates (#14449)
* test(managed-agent): tighten the FG13c hook fault gates after audit
* test(managed-agent): address the FG13c review findings
* test(managed-agent): settle the FG13c harness-kill hook after a Turn cancel
* test(managed-agent): address the FG13c review census
* test(managed-agent): reconcile the FG13c hook before the terminal wait
* test(managed-agent): tighten the FG13c fresh-prompt reconcile after audit
* test(managed-agent): drop the stale SIGTERM claim from the FG13c fixture
* test(managed-agent): settle the FG13c Harness-kill Hook via the Harness status route
The public prompt cannot trigger reconciliation: the control plane's
active-Turn gate answers 409 in insertTurnCommand before any Harness
dispatch, and the cancelled Turn never leaves CANCELLING. Re-attach the
Session to the replacement Harness with the passive recovery load and
read the operation through the authenticated hooks/operations route,
which asks the record's original runtimeSessionId and settles it from
the receipt the worker still holds.
* test(managed-agent): pin the FG13c kill window and the settled owner
The Harness-kill cell could pass without proving anything if the Hook
had already allowed before the kill: the old Harness would have settled
the record and the status read would see a terminal one. Hold longer and
refuse to kill while an allow is already on disk. Assert after
settlement that the binding still names the same worker and generation,
which is what the design claims.
* test(managed-agent): reuse the replacement attachment instead of forcing a load
A Session the coordinator already attached is resident with a Hook-pending
Turn, so a second passive load of it is refused with takeover_unsupported;
the forced call sat outside the wait and exited the cell before the status
route could run. Probe the status route first, reuse whatever attachment
serves it, and ask for a passive re-attach only on 404, at most once a
second.
* test(managed-agent): keep a refused re-attach inside the wait
untilAsserted retries an AssertionError, not a RuntimeException, so a
takeover_unsupported refusal would still have exited the cell. Swallow it
at the call site and report the message on the assertion instead.
* test(managed-agent): send the Harness contract headers on the Hook status probe
Every /session route runs the hosted-harness contract middleware, so the
status probe needs X-Qwen-Harness-Protocol-Version and the boot id of the
process it is talking to. The identity now carries both from the attachment
ref, and a stale generation counts as an attachment that cannot serve the
route, like a missing Session.
---------
Co-authored-by: 易良 <1204183885@qq.com>
Names recorded on channel A survived a replace to channel B until the next hasRuntimeMcpServers() call. An add on B then adopted them, so removing B's own server still reported the runtime as holding one. Clear the set when the live child changes before recording a new name.
Refs #14488
The guard on the shared hosted-harness-mysql failsafe fork read only
seven of its nineteen classes. A ceiling raised in any of the other
twelve, or a new Hosted*IT class, stayed green until the fork was killed
on main. Two classes declared no ceiling at all.
- Discover every Hosted*IT class the fork's include picks up, in any
package, and pin the list, so a new class lands on a red that asks for
its runtime in hostedForkHealthySeconds.
- Hold each class to the per-ceiling rule the FG7-FG10 letters already
used; the team lead and member command classes keep the summed rule.
- Require a ceiling on every case, and give the two untimed classes one
(HostedCommittedEventLineReplayIT 180 s, HostedWorkspaceRecoveryWorkerIT
120 s).
- Cite the lane measurement beside the allowance: the nineteen classes
ran 3085 s on run 39087108383, under the 3874 s allowance.
- Reuse the class discovery in the Spring context check.
The one-argument form of the load request's child Workspace setter
passed false for the snapshot capability, so restating child
Workspaces on a copy that served snapshots silently dropped them.
Every other copy of the request keeps the flag. The one-argument form
now keeps the flag the copy carries, so only turning child Workspaces
off drops it, as the two-argument form already does.
No control-plane caller uses the one-argument form today; this removes
the trap for the next one.
* fix(managed-agent): keep runtime MCP state and permission sync honest
Clear the recorded runtime-MCP names when the live child that held them is
replaced, so a retired channel does not keep every later session on Legacy.
Diff each settings scope's own rule set when a permission change is
propagated: the merged view carries workspace rules into the User pass, which
then applies them to every session. Make the sync strict for a propagated
change and fail the acknowledgement when a live session rejects a rule, so
the Bridge keeps the channel quarantined instead of lifting the fence.
Refs #14483
(cherry picked from commit 3439ca9da9)
* test(acp-bridge): pin that an added runtime MCP server is observed
The paired selector reads hasRuntimeMcpServers() before it lets a session
select Managed. Pin that a successful workspace-level runtime add is visible
to the bridge.
Refs #14488
* fix(managed-agent): diff the effective permission set for workspace changes
The live permission manager keeps the scopes' rules merged, without provenance. Diffing each scope separately can remove a rule another scope still provides: clearing a workspace deny while the user scope also denies it would make read_file runnable again, and the acknowledgement would hide it. Diff the effective set instead, and keep the workspace filter on the apply.
Refs #14488
* fix(managed-agent): carry the permission scope through a workspace change
A single SettingScope.Workspace pass filtered user-scope deltas out of sessions admitted at another root, such as a worktree session, while still acknowledging the change. Carry the written scope in the change payload and use it to pick the audience: User reaches every live session, Workspace keeps the workspace filter. The content still comes from the effective merged set, so a rule another scope provides is never removed.
Refs #14488
* test(acp-bridge): include the permission scope in the propagated payload
The workspace-change payload now carries the written scope, so the exact assertion in the paired-engine test must include it. This also fixes the failing Test (ubuntu) lane on the same head.
Refs #14488
* test(cli): include the permission scope in the propagated payload
The workspace-change payload now carries the written scope. Update the CLI paired-engine assertion to include it, alongside the acp-bridge one.
Refs #14488
* test(managed-agent): parse only complete rows in the wedge driver wire
* test(managed-agent): pin the wire reader to complete rows
The blocked-wedge driver's wire reader had no executable coverage: the
driver cannot be imported (it parses argv and boots a daemon at load),
and the Java gates that spawn it are skipped on PR events. Extract the
parse into a side-effect-free helper so a unit test can exercise it, and
pin the mutants that matter with two complete rows plus an unterminated
tail.
---------
Co-authored-by: 易良 <1204183885@qq.com>
* fix(managed-agent): keep a crash-blocked workflow run unknown and honestly notified, gated by FG10d (#14165)
FG10d, the last FG10 letter, SIGKILLs the packaged Harness under an
attached workflow run, with its agent() request in flight and with its
work done but the child Turn's settlement swallowed at the store, then
starts a cold one. The run must never be re-executed or settled as if
observed: the Turn parks marked hosted_workflow_recovery_blocked, the
cold load refuses, and the record settles recovery_blocked over
outcome_unknown with one blocked notification to the lead.
The gate found two defects in that honest-settlement path:
- A retained close debt wrote the walk done and its CHILD_RESULT row
delivered whatever the arm proved, so the unknown the recovery-blocked
arm and the give-up chain intend read as a delivered result wherever
the host cannot close the child at once. The debt now parks under the
walk's own classification (retainCloseDebt), as classify does.
- settleRecoveryBlocked rendered the parent's notification from the
record it replaces, so under <status>blocked</status> the result line
read "run running (unknown)". It now renders the blocked revision.
The store proxy gains a SWALLOW disposition and the class a killHarness
helper. The Hosted fork's healthy allowance composes 3743 + 131 = 3874 s
and the fork moves 4353 -> 4484 s; the Verify step and job ceiling hold.
The relay design gains FG10d's section; the a+d design points at it.
* fix(managed-agent): release a parent whose child walk retired unknown, and sweep FG10d's review nits (#14165)
Session resource collection held a retired parent while a CHILD_SESSION_START
walk was not done/orphaned, but the relay's `unknown` is just as terminal and
is never rewritten, so the hold had no release; the close-debt fix carried it
to hosts without close capability. The collector now treats `unknown` as
terminal; an owed close rides its own CHILD_SESSION_CLOSE row and reads no
parent resource.
Also: pin the give-up arm's no-close classification, refuse a close row in
retainCloseDebt itself, correct the give-up javadoc and test comment that
still described the old either/or, and sweep the stale four-letter and
FG10d-selector text in the IT header, the CI comment and both designs.
The session-row lock-order canary asserts the turn-row accessor
population exactly. #14461 added the lifecycle gate's live-Turn count and
the recovery-blocked verdict, both of which reach ma_turn, without
updating that pin, so the main Managed Agent gate went red.
Add the two members with the reason the order does not bite on them:
the count runs under its caller's session lock, and the verdict updates
the Turn row alone.
#14435 made placement resolution go through the three-argument
RuntimeProvisioner.createRequest so a bound Session's Harness id travels on
its own. Mockito never runs interface default methods, so this IT's
two-argument stub stopped answering the call the Broker makes: the
provisioner returned null, findOrCreate(null) threw "request is required",
and the Broker mapped that internal failure to a 400
runtime_broker_invalid_request at tool-sessions:acquire.
Stub the overload the Broker actually calls so the acceptance chain receives
the request it constructed.
