mirror of
https://github.com/QwenLM/qwen-code.git
synced 2026-10-21 14:21:48 +00:00
feat(managed-agent): member board tools ride the session_command channel end to end (H4e-b3b) (#13928)
Some checks are pending
SDK Java / Runtime Broker and Managed Agent MariaDB / Java 21 (push) Waiting to run
Qwen Code CI / Classify PR (push) Waiting to run
Qwen Code CI / Test (ubuntu-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / Lint & Static (ubuntu-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / web-shell E2E Smoke (ubuntu-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / Test (macos-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / Test (windows-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / Integration Tests (no-AK, No Sandbox) (push) Blocked by required conditions
Qwen Code CI / Integration Tests (CLI, No Sandbox) (push) Blocked by required conditions
Qwen Code CI / Desktop Shell (ubuntu-22.04) (push) Blocked by required conditions
Qwen Code CI / Desktop Shell (windows-2022) (push) Blocked by required conditions
E2E Tests / Build for E2E (push) Waiting to run
E2E Tests / E2E Test (Linux) - sandbox:docker - shard 1/1 (push) Blocked by required conditions
E2E Tests / E2E Test (Linux) - sandbox:none - shard 1/1 (push) Blocked by required conditions
E2E Tests / E2E Test - macOS - shard 1/2 (push) Blocked by required conditions
E2E Tests / E2E Test - macOS - shard 2/2 (push) Blocked by required conditions
E2E Tests / E2E Interactive - OpenTUI renderer (bun) (push) Blocked by required conditions
E2E Tests / channel-plugin E2E (nightly) (push) Blocked by required conditions
E2E Tests / cron-interactive E2E (nightly) (push) Blocked by required conditions
E2E Tests / web-shell Browser Regression (push) Waiting to run
SDK Java / ubuntu-latest / Java 11 (push) Waiting to run
SDK Java / ubuntu-latest / Java 17 (push) Waiting to run
SDK Java / macos-latest / Java 21 (push) Waiting to run
SDK Java / ubuntu-latest / Java 21 (push) Waiting to run
SDK Java / windows-latest / Java 21 (push) Waiting to run
SDK Java / Flyway migration version uniqueness (push) Waiting to run
SDK Java / Hosted process fault gates / MySQL 8.4 / Java 21 (push) Waiting to run
SDK Java / Real daemon E2E / Java 11 (push) Waiting to run
Some checks are pending
SDK Java / Runtime Broker and Managed Agent MariaDB / Java 21 (push) Waiting to run
Qwen Code CI / Classify PR (push) Waiting to run
Qwen Code CI / Test (ubuntu-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / Lint & Static (ubuntu-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / web-shell E2E Smoke (ubuntu-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / Test (macos-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / Test (windows-latest, Node 22.x) (push) Blocked by required conditions
Qwen Code CI / Integration Tests (no-AK, No Sandbox) (push) Blocked by required conditions
Qwen Code CI / Integration Tests (CLI, No Sandbox) (push) Blocked by required conditions
Qwen Code CI / Desktop Shell (ubuntu-22.04) (push) Blocked by required conditions
Qwen Code CI / Desktop Shell (windows-2022) (push) Blocked by required conditions
E2E Tests / Build for E2E (push) Waiting to run
E2E Tests / E2E Test (Linux) - sandbox:docker - shard 1/1 (push) Blocked by required conditions
E2E Tests / E2E Test (Linux) - sandbox:none - shard 1/1 (push) Blocked by required conditions
E2E Tests / E2E Test - macOS - shard 1/2 (push) Blocked by required conditions
E2E Tests / E2E Test - macOS - shard 2/2 (push) Blocked by required conditions
E2E Tests / E2E Interactive - OpenTUI renderer (bun) (push) Blocked by required conditions
E2E Tests / channel-plugin E2E (nightly) (push) Blocked by required conditions
E2E Tests / cron-interactive E2E (nightly) (push) Blocked by required conditions
E2E Tests / web-shell Browser Regression (push) Waiting to run
SDK Java / ubuntu-latest / Java 11 (push) Waiting to run
SDK Java / ubuntu-latest / Java 17 (push) Waiting to run
SDK Java / macos-latest / Java 21 (push) Waiting to run
SDK Java / ubuntu-latest / Java 21 (push) Waiting to run
SDK Java / windows-latest / Java 21 (push) Waiting to run
SDK Java / Flyway migration version uniqueness (push) Waiting to run
SDK Java / Hosted process fault gates / MySQL 8.4 / Java 21 (push) Waiting to run
SDK Java / Real daemon E2E / Java 11 (push) Waiting to run
* feat(managed-agent): member board tools ride the session_command channel end to end (H4e-b3b)
Carry decision 13's second wave from the team-member-commands design: members of an open team write and read their leader's task board through the session_command channel. A member's board call mints an outbound entry on her own journal before her Turn parks on an await_command checkpoint (never carrying an input, digest-bound args/reply pairs, per-run budget, batch-locked against Workspace or Runtime siblings); the message relay's new command pass drives it in sender order to the leader — planned → handover → execute → answered, never through consume — and the parked call folds the leader's reply back and marks the entry consumed. The leader executes off the model Turn path: replay-stable receipts for board_writes, an open-team member on its running chain head for fresh commands, the member-facing refusal texts, and a board_read that changes nothing and never receipts.
Both languages carry the contract, the lineage and resource rules, and the store side of the line. The lead's execute answers a not-yet-attached sender's run with a 409 session_command_blocked the relay retries without spending attempts (it can outrun the control plane's attach commit), while a genuine lineage or evidence mismatch stays a terminal session_command_refused. The wait holds a 2-minute bound from the outbox's commit; its timeout tells planned asks 'not sent', handed asks 'reached the leader, reply missing, write possibly landed', and locks the rest of the batch under one text, and the harness descent into await_command now resumes or settles every named wait from the journal alone (decision 14). A member Session declares the three tools exactly behind her lineage, the enabled domain, and her member preamble — never before all three are true — and join keeps refusing 'parent' as a name.
On the aisle where recovery and workspace restore walk the journal, session_command joins the recoverable-domain whitelist so a member holding a commanded write never wedges her later loads, and qwencode classifies await_command like await_agent so a Turn parked on it takes over cleanly. G3 physical: two members write and read back the board, their leader reads it, a member's write races a leader's write without a phantom create, and a member Harness restart mid-wait lands her report anyway — each over a real MariaDB stack with scripted models (HostedTeamMemberCommandMySqlIT, 2/2).
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): admit the scenario lead over the internal TeamLead lane (#13674)
The public Shell admission a8990042 introduced is behind its own
workspace-shell opt-in and approval-mode gate, so a Session written to
hosted-workspace-shell/1 by test fixtures now refuses every wake POST as
workspace_unavailable even when the lane is enabled. The MySQL Scenario
ITs ride the same internal TeamLead profile the baseline migrated to:
insert the lead over the trusted internal lane and let it pin the team
lead profile itself; no public Shell flag is involved.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* test(managed-agent): align the team MySQL ITs with b3b and the broker opt-out
Members of an open team now declare the three board verbs (task_create,
task_list, task_update) through the session_command channel, so the
team-lead fixture's lead-only exclusion narrows to team_create and
team_delete. The scenario IT's child-result settles may end close_debt
whatever the OS — delivered, with only the workspace close owed — so
both settle gates accept it as terminal.
The broker's durable local-process mode only exists on Linux; the
team-lead fixture now opts it out on non-Linux hosts instead of
shipping the flag unconditionally, keeping Linux CI's boot surface
identical. Without workspace execution a workspace Session's close
answers 409 workspace_unavailable: closeAndSettle tolerates it (no
operation binds to wait on) and the cascade case aborts through
Assumptions rather than failing where its premise cannot run.
Verified locally on macOS: both ITs, 5 tests run, 0 failures, 0
errors, 1 skipped (the cascade case abort), BUILD SUCCESS.
* test(cli): give the budget witness its own timeout budget
The merged-b3a run slows each member-side command commit enough that
255 hand-minted outbound records plus the assertion ride past Vitest's
15 s default on a shared runner, while the rule's own check is single-
digit seconds on an idle box. The budget's witness now carries an
explicit 30 s allowance rather than riding the suite-wide default.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* fix(managed-agent): judge command order by the outbound's open window
A cancelled outbound's leftover relaying row never gets another scan —
HANDOVER_SQL selects only planned/accepting/unknown records — so the
ledger-only open predicate held every later command of that sender
behind it forever, deferring each one every five seconds without
spending an attempt. The row's open window is the record's
planned/accepting span; a terminal record never blocks again, whatever
row state its last step left.
Refs: QwenLM/qwen-code#13928 review 4244450992.
* fix(managed-agent): classify a sender-terminal command on sight
HANDOVER_SQL still selects a command record whose accepting span the
member's own timeout flipped to cancelled or unknown, and workCommand
only distinguished planned: the terminal entry fell through to the
leader's execute, re-running a command its sender already saw settled —
the second execution decision 2 forbids, live in the member's board the
next time it re-reads or retries a write. A non-planned, non-accepting
record is classified on sight and returned, whatever the ledger row
holds; the classification depends on the record's own committed state,
not on the row having been classified first.
Refs: QwenLM/qwen-code#13928 review 4244450996.
* fix(serve): answer a never-sent command wait on resume, never settle nothing
The await_command checkpoint names every call of the batch before any
outbound entry commits, so an owner that exits between the two leaves
a run with no record. The recovered wait anchored the bound on the
resume itself, waited the full two minutes, and then settled cancelled
— which throws "has no record to revise" — leaving the parked call
unfoldable and the Turn unresumable. A missing record is permanent, so
the resumed poll ends the wait at once with the live arm's own not-sent
answer, word for word (decision 14), and folds the run.
Refs: QwenLM/qwen-code#13928 review 4244450991.
* fix(serve): carry decision 4's command timeout lock in the wait group
The lock that refuses every later command of a Turn once one command
timed out lived only in the owning Turn's memory: after the result and
the folded checkpoint committed, a restart raised a fresh
HostedWorkspaceToolTurn for the same prompt and the next board_write
minted an outbound that ran to consumed — exactly the leader silence
the lock existed to refuse. The fold that carried the timeout now
stamps the group, every later fold carries the stamp, a fresh wait
starts clean, and a replacement owner of the Turn re-arms from the
carried group before it mints the next wait.
Refs: QwenLM/qwen-code#13928 review 4244450997.
* fix(serve): fold the parked wait on a member-facing command refusal
The await_command checkpoint commits before the batch's first outbound
entry, so a budget_exhausted (or byte_limit) refusal that answered the
call but skipped the fold left the Turn's durable wait parked: the next
command call of the same Turn met its own durable wait and surfaced as
a recovery fault instead of the same local refusal. The member-facing
refusal arm now answers and folds, like every arm that runs.
Refs: QwenLM/qwen-code#13928 review 4244451000.
* fix(serve): judge member addBlocks writes against the board as it stands
A well-formed but absent addBlocks target reached the ownership probe
before the reference check, so memberTaskUpdate read undefined.owner
and escaped as a TypeError instead of an answer. Existence now comes
first, in the initial validation and again per replayed place. And a
partially committed task_update replayed only an owner check for its
remaining places: an edge the original validation allowed could since
have closed a cycle on the live board, which the authority rightly
refused — but the refusal escaped execute as a fault, so the relay
retried it forever and the member never got its receipt. Each remaining
place is now re-validated against the board as it stands, and the
authority's deterministic conflict rides the same Not-committed
receipt.
Refs: QwenLM/qwen-code#13928 reviews 4244451002, 4244451008.
* fix(serve): answer a never-sent command wait on resume even when cancelled, never settle nothing
The recovered wait checked the cancelled signal before checking whether
the checkpointed call's outbound ever existed: an owner that cancelled a
recovery over a parked, never-minted call fell into the journal arm,
which treated the missing record as planned and settled cancelled —
throwing "has no record to revise" with neither the tool result nor
the fold committed, stranding the wait at await_command. The
missing-record arm now runs before every signal arm, and the shared
journal answer treats an absent record as the planned arm, never
reaching settle.
Refs: QwenLM/qwen-code#13928 review 4246316365.
* fix(serve): rebuild the command timeout lock from the journaled answer, not just the fold
The lock that refuses every later command of a Turn after one timeout
relied on the fold carrying the timedOut stamp: the dead owner commits
the unknown (or cancelled) tool result first and the fold stamps second,
so a crash between the two leaves a takeover owner holding a group that
still reads timedOut=false. The next command of that prompt then mints a
fresh outbound and rides to consumed — the leader stayed unanswering and
nobody told the model. The owner now also re-arms from this prompt's
journaled timeout answer itself (decision 14's own texts and the locked
leader-unanswering text), so the persisted evidence of the timeout never
depends on the last fold having succeeded.
Refs: QwenLM/qwen-code#13928 review 4246316346.
* fix(serve): re-check the blocker cap per remaining place on a replayed member task_update
The replay loop re-validated existence, ownership and cycle for each
remaining addBlocks place against the board as it stands, but never the
blocker-count limit the initial whole-command check covers: a partially
committed command whose leader since filled its target to the cap met
teamTaskRevisionBody's own "blockedBy exceeds 64 entries" record error, a
deterministic refusal that surfaced as an escape fault — no receipt for
the relay, the same error again at the next retry. Each remaining place
now re-evaluates the current board's picker count against the maximum
before committing, folding the deterministic refusal into the same
Not-committed receipt the cycle and authority checks already fill; true
storage faults still escape.
Refs: QwenLM/qwen-code#13928 review 4246316356.
* fix(serve): stamp decision 4's lock at detection time, never scan tool-result text
Two repair attempts rebuilt the timeout lock from journaled tool-result
text, and both misfired on legitimate data: an ordinary task_list reply
that contains the leader-unanswering words as its subject locked the
Turn's next command, and the missing-record path's planned-not-sent
answer did the same on a healthy continuation. A crash between the
settle(unknown) write and the tool_result commit passed the window too —
the record is already terminal and nothing structural says the Turn
timed out once. The lock now has exactly one durable carrier: the wait
group's own timedOut field, stamped at deadline detection time by
markCommandWaitTimeout before every other commit the timeout arm makes
(settle, tool_result, fold). Recovery re-arms from that field, never
from what text happened to land in the journal, and the prompt still
recognizes the genuinely-timed-out chain and refuses it identically.
Refs: QwenLM/qwen-code#13928 reviews 4247995945, 4247995960.
* fix(serve): keep decision 4's lock on the Turn and settle the timed-out call by its journal
The timeout lock lived on the command wait group, which the Turn's next
command batch replaces and its Runtime work clears, so a takeover in
either window let a later board call mint an outbound and reach the
leader. The lock is now a root `commandLock` naming the Turn that
stamped it: every checkpoint of that Turn carries it, and a checkpoint
of another Turn drops it.
A recovered wait no longer answers every run with the leader-unanswering
text once the lock is on. A run whose outbound exists is the call whose
own wait timed out — its owner may have died between stamping the lock
and settling the entry — so its journal answers it: planned is cancelled
as not sent, accepting becomes unknown, a settled state is said as
committed. Only a run that sent nothing gets the fixed refusal, as in
the live arm, which applies the same rule to a re-driven call.
* fix(serve): never consume a timed-out call on resume, and end the lock with its Turn
A recovered wait answered a timed-out call through the journal even when
its answer was already committed, so a relay reply copied in after the
settle was consumed although the call only ever read the unknown
evidence. A call already answered was settled first, so the lock arm now
only folds it, as the funnel's rule for timed-out calls requires.
The next-turn-ready checkpoint marks its Turn finished, whether it is
written under the settled Turn's id or the next prompt's, so it drops
the lock: an idle Session carries no commandLock key. The carry also
tolerates an in-memory checkpoint shaped without the key.
* fix(serve): let the takeover continue a member's command wait, and answer its committed commands by journal
The takeover reports a parked member Turn as await_command and qwencode
calls continue for it, but the continue route admitted only results_ready
and the agent wait, so every command wait was refused with
continuation_not_ready and resumeCommandWaitRuns was unreachable. The
gate now admits the command wait the same way, through the resume arm
that was already written for it.
The load and settle routes answer an interrupted Turn's owed calls before
reading its round. A member's board calls share the team tools' names,
so a committed command was answered "never ran" while its outbound stayed
planned or accepting for the relay to deliver later. Those routes now
carry the command funnel, and a call whose outbound committed is answered
by its journal — planned cancelled, accepting unknown, a reply as the
reply — through the helper the cancelled wait already used.
* fix(serve): a resumed command call consumes only the reply it commits
The lock arm already folds a timed-out call whose answer was journaled
without consuming it. The unlocked arms kept consuming: a call cancelled
or timed out as unknown, whose owner died before the fold, read the
unknown evidence, yet a reply the relay copied in afterwards was marked
consumed on resume. Both the journal arm and the resumed reply arm now
consume only when the reply is what they commit.
* fix(serve): keep a resumed command wait off the Workspace mount, and cancel it through the recovery cancel
Now that the continue route admits the command wait, its resume fell
into the Runtime-results branch, whose acquire claims the Workspace
mount for a Turn that never held it — a member shares its lead's
Workspace, so a busy mount failed the recovered Turn after its commands
had folded. A command wait whose Turn did not hold the mount takes the
agent wait's branch; one that held it got it back in its resume arm.
A cancelled takeover of a Turn parked on a command wait was classed as
owing no Runtime settlement, so the load settled it plainly and left the
checkpoint at await_command, refusing the next Turn's model start. Like
the agent wait, it now goes to the recovery cancel, which answers each
run by its journal and folds it.
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
This commit is contained in:
parent
18f80f5666
commit
21b78c55d5
41 changed files with 10529 additions and 153 deletions
|
|
@ -129,6 +129,7 @@ import {
|
|||
import { HostedChildAgentSession } from './hosted-child-agent-session.js';
|
||||
import { HostedSessionMessageSession } from './hosted-session-message-session.js';
|
||||
import { HostedTeamSession } from './hosted-team-session.js';
|
||||
import { teamMemberPreamble } from './hosted-member-chain.js';
|
||||
|
||||
const wakeDeps = vi.hoisted(() => ({
|
||||
last: undefined as unknown,
|
||||
|
|
@ -19485,6 +19486,229 @@ describe('Hosted Harness Runtime turn takeover', () => {
|
|||
);
|
||||
});
|
||||
|
||||
/**
|
||||
* H4e-b3: a member's board call parked on `await_command` by an owner
|
||||
* that died mid-wait. The member is a child Session whose first input
|
||||
* carries its member preamble, so its board tools ride the command
|
||||
* channel; the cancel's abandoned answer is sabotaged, so the wait stays
|
||||
* parked with its outbound already settled.
|
||||
*/
|
||||
async function parkCommandWaitTurn(): Promise<void> {
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'warm').mockResolvedValue();
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'acquire').mockResolvedValue();
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'cancel').mockResolvedValue();
|
||||
vi.spyOn(HostedWorkspaceBroker.prototype, 'release').mockResolvedValue();
|
||||
const originalWrite = ManagedSessionRecordSink.prototype.write;
|
||||
const sabotage = vi
|
||||
.spyOn(ManagedSessionRecordSink.prototype, 'write')
|
||||
.mockImplementation(async function (
|
||||
this: ManagedSessionRecordSink,
|
||||
record,
|
||||
) {
|
||||
if (
|
||||
record.type === 'tool_result' &&
|
||||
record.daemonPromptId === PROMPT_ID
|
||||
)
|
||||
throw new Error('settlement unavailable');
|
||||
return originalWrite.call(this, record);
|
||||
});
|
||||
const server = await app(true);
|
||||
const parent = '44444444-4444-4444-8444-444444444444';
|
||||
const created = await headers(supertest(server).post('/session')).send({
|
||||
sessionId: SESSION_ID,
|
||||
sessionScope: 'thread',
|
||||
managedSessionStore: store(),
|
||||
toolProfile: 'hosted-workspace-shell/1',
|
||||
lineage: {
|
||||
parentSessionId: parent,
|
||||
rootSessionId: parent,
|
||||
parentChildRunId: 'run-leaf',
|
||||
depth: 1,
|
||||
},
|
||||
});
|
||||
expect(created.status).toBe(200);
|
||||
const boardCall = {
|
||||
name: 'task_create',
|
||||
callId: 'call-1',
|
||||
args: { subject: 'ship it', description: 'land the fix' },
|
||||
isClientInitiated: false,
|
||||
prompt_id: PROMPT_ID,
|
||||
};
|
||||
state.model.mockImplementationOnce(
|
||||
async ({ toolTurn, signal }) =>
|
||||
toolTurn!.execute(
|
||||
[boardCall],
|
||||
[
|
||||
{
|
||||
functionCall: {
|
||||
id: 'call-1',
|
||||
name: 'task_create',
|
||||
args: boardCall.args,
|
||||
},
|
||||
},
|
||||
],
|
||||
'test-model',
|
||||
signal,
|
||||
) as never,
|
||||
);
|
||||
const prompt = [
|
||||
{
|
||||
type: 'text',
|
||||
text:
|
||||
teamMemberPreamble({ name: 'alice', teamName: 'review' }) +
|
||||
'write the board',
|
||||
},
|
||||
];
|
||||
const payloadDigest = `sha256:${createHash('sha256')
|
||||
.update(JSON.stringify(prompt))
|
||||
.digest('hex')}`;
|
||||
const admitted = await headers(
|
||||
supertest(server).post(`/session/${SESSION_ID}/prompt`),
|
||||
)
|
||||
.set('X-Qwen-Client-Id', created.body.clientId as string)
|
||||
.send({ prompt, promptId: PROMPT_ID, payloadDigest });
|
||||
expect(admitted.status).toBe(202);
|
||||
const key = {
|
||||
tenantId: 'tenant',
|
||||
workspaceId: 'workspace',
|
||||
sessionId: SESSION_ID,
|
||||
};
|
||||
const resources = LocalManagedSessionResourceStore.create({
|
||||
runtimeBaseDir: state.root,
|
||||
sessionKey: key,
|
||||
});
|
||||
const latestCheckpointState = async () => {
|
||||
const journal = await LocalJsonlManagedSessionJournalStore.read(
|
||||
path.join(state.root, `${SESSION_ID}.jsonl`),
|
||||
key,
|
||||
);
|
||||
const checkpoint = journal.events.findLast(
|
||||
(event) => event.kind === 'checkpoint.committed',
|
||||
);
|
||||
if (checkpoint === undefined) return undefined;
|
||||
return JSON.parse(
|
||||
(
|
||||
await resources.read(
|
||||
assertManagedSessionDurableRef(
|
||||
checkpoint.payload['stateRef'],
|
||||
'checkpoint',
|
||||
),
|
||||
)
|
||||
).toString(),
|
||||
);
|
||||
};
|
||||
// Park and outbound both durable before the "crash": the cancel then
|
||||
// settles a committed entry, never a call that had not sent yet.
|
||||
await vi.waitFor(
|
||||
async () => {
|
||||
expect((await latestCheckpointState())?.continuation.phase).toBe(
|
||||
'await_command',
|
||||
);
|
||||
const journal = await LocalJsonlManagedSessionJournalStore.read(
|
||||
path.join(state.root, `${SESSION_ID}.jsonl`),
|
||||
key,
|
||||
);
|
||||
expect(
|
||||
journal.events.some(
|
||||
(event) =>
|
||||
event.kind === 'domain.committed' &&
|
||||
event.payload['domain'] === 'session_command',
|
||||
),
|
||||
).toBe(true);
|
||||
},
|
||||
{ timeout: 10_000 },
|
||||
);
|
||||
await headers(supertest(server).post(`/session/${SESSION_ID}/cancel`)).set(
|
||||
'X-Qwen-Client-Id',
|
||||
created.body.clientId as string,
|
||||
);
|
||||
await vi.waitFor(
|
||||
async () => {
|
||||
const status = await headers(
|
||||
supertest(server).get(`/session/${SESSION_ID}/status`),
|
||||
).set('X-Qwen-Client-Id', created.body.clientId as string);
|
||||
expect(status.body.hasActivePrompt).toBe(false);
|
||||
expect(status.body.recoveryBlocked).toBe(true);
|
||||
},
|
||||
{ timeout: 10_000 },
|
||||
);
|
||||
sabotage.mockRestore();
|
||||
expect(
|
||||
(await headers(supertest(server).delete(`/session/${SESSION_ID}`)))
|
||||
.status,
|
||||
).toBe(204);
|
||||
const savedState = await latestCheckpointState();
|
||||
expect(savedState.continuation.phase).toBe('await_command');
|
||||
expect(savedState.commandWait.runs).toMatchObject([
|
||||
{ functionCallId: 'call-1', folded: false },
|
||||
]);
|
||||
}
|
||||
|
||||
// The command wait recovers like the agent wait: the coordinator's
|
||||
// continue re-enters it and folds the call from its journal. A gate
|
||||
// that admits only the agent wait leaves the member's Turn refused
|
||||
// forever.
|
||||
it('admits a continue for the parked command wait and folds it from the journal', async () => {
|
||||
await parkCommandWaitTurn();
|
||||
const { server, loaded } = await loadReplacement(
|
||||
false,
|
||||
'hosted-workspace-shell/1',
|
||||
);
|
||||
expect(loaded.status).toBe(200);
|
||||
const recovery = loaded.body._meta?.[
|
||||
'qwen.daemon.managedRuntimeRecovery'
|
||||
] as {
|
||||
phase: string;
|
||||
checkpointId: string;
|
||||
activationId: string;
|
||||
executions: Array<Record<string, unknown>>;
|
||||
};
|
||||
expect(recovery.phase).toBe('await_command');
|
||||
expect(recovery.executions).toMatchObject([
|
||||
{
|
||||
functionCallId: 'call-1',
|
||||
outcome: 'known',
|
||||
status: { state: 'executing' },
|
||||
},
|
||||
]);
|
||||
// The member's Turn never held the mount: its resume must not claim
|
||||
// the Workspace it shares with its lead.
|
||||
const acquire = vi
|
||||
.spyOn(HostedWorkspaceBroker.prototype, 'acquire')
|
||||
.mockResolvedValue();
|
||||
const continued = await replacementHeaders(
|
||||
supertest(server).post(`/session/${SESSION_ID}/managed-runtime/continue`),
|
||||
)
|
||||
.set('X-Qwen-Client-Id', loaded.body.clientId as string)
|
||||
.send({
|
||||
promptId: PROMPT_ID,
|
||||
checkpointId: recovery.checkpointId,
|
||||
activationId: recovery.activationId,
|
||||
});
|
||||
expect(continued.status).toBe(200);
|
||||
await vi.waitFor(
|
||||
async () => {
|
||||
const status = await replacementHeaders(
|
||||
supertest(server).get(`/session/${SESSION_ID}/status`),
|
||||
).set('X-Qwen-Client-Id', loaded.body.clientId as string);
|
||||
expect(status.body.hasActivePrompt).toBe(false);
|
||||
expect(status.body.recoveryBlocked).toBe(false);
|
||||
},
|
||||
{ timeout: 10_000 },
|
||||
);
|
||||
const transcript = await replacementHeaders(
|
||||
supertest(server).get(`/session/${SESSION_ID}/transcript`),
|
||||
).set('X-Qwen-Client-Id', loaded.body.clientId as string);
|
||||
// The cancelled entry answers as its journal committed it.
|
||||
expect(JSON.stringify(transcript.body)).toContain(
|
||||
'The command was cancelled before it ran; it never left.',
|
||||
);
|
||||
expect(acquire).not.toHaveBeenCalled();
|
||||
await replacementHeaders(
|
||||
supertest(server).delete(`/session/${SESSION_ID}`),
|
||||
);
|
||||
});
|
||||
|
||||
it.each(
|
||||
(['continue', 'cancel'] as const).flatMap((route) =>
|
||||
(['closing', 'authorizing', 'detached'] as const).map((phase) => ({
|
||||
|
|
|
|||
|
|
@ -55,6 +55,7 @@ import {
|
|||
parseHarnessCheckpointV1,
|
||||
type HarnessRunAuthorization,
|
||||
harnessCheckpointIsAgentWait,
|
||||
harnessCheckpointIsCommandWait,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-harness-checkpoint.js';
|
||||
import { createManagedHarnessHandle } from '@qwen-code/qwen-code-core/managed-runtime/managed-harness-factory.js';
|
||||
import { MANAGED_MCP_MAX_CONNECTIONS } from '@qwen-code/qwen-code-core/managed-runtime/managed-mcp-protocol.js';
|
||||
|
|
@ -141,6 +142,19 @@ import {
|
|||
SessionMessageNotReadyError,
|
||||
withSessionMessageConsumption,
|
||||
} from './hosted-session-message-session.js';
|
||||
import {
|
||||
HostedSessionCommandSession,
|
||||
SessionCommandBlockedError,
|
||||
SessionCommandRefusalError,
|
||||
} from './hosted-session-command-session.js';
|
||||
import {
|
||||
parseSessionCommandReply,
|
||||
type SessionCommandReply,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-session-command-operations.js';
|
||||
import {
|
||||
MANAGED_SESSION_COMMAND_LIMITS,
|
||||
type SessionCommandKind,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-session-command-record.js';
|
||||
import { HostedMonitorSession } from './hosted-monitor-session.js';
|
||||
import {
|
||||
AutomationNotFoundError,
|
||||
|
|
@ -204,11 +218,13 @@ import {
|
|||
answerCommittedTurnCalls,
|
||||
answerResumedTurnCalls,
|
||||
fillParkedRoundAgentGaps,
|
||||
fillParkedRoundCommandGaps,
|
||||
isDurableBlockedVerdict,
|
||||
originalRuntimeBroker,
|
||||
RecoveryDeclined,
|
||||
recoverHostedRuntimeTurn,
|
||||
settleCancelledAgentWaitRuns,
|
||||
settleCancelledCommandWaitRuns,
|
||||
settleInterruptedTurnRuntime,
|
||||
settleParkedTurnCancelled,
|
||||
stopParkedRuntimeExecutions,
|
||||
|
|
@ -337,6 +353,8 @@ interface HostedSession {
|
|||
childAgents?: HostedChildAgentSession;
|
||||
/** H4d-b: the Session's messages along its lineage, beside its children. */
|
||||
messages?: HostedSessionMessageSession;
|
||||
/** H4e-b3: the Session's commands to its parent, on the same chain. */
|
||||
commands?: HostedSessionCommandSession;
|
||||
/** H4d-b: message operations past the closing check, which a close waits
|
||||
* out before it cancels the pending inputs. */
|
||||
messageOperations?: Set<Promise<void>>;
|
||||
|
|
@ -1070,6 +1088,7 @@ export async function settleCrashedWakeTurnAftermath(params: {
|
|||
children: session.childAgents,
|
||||
teams: session.teams,
|
||||
messages: session.messages,
|
||||
commands: session.commands,
|
||||
});
|
||||
// Consume the crashed input: until its turnId settles, every reload
|
||||
// re-classifies it as recovery and re-blocks the Session over the
|
||||
|
|
@ -1490,6 +1509,7 @@ async function settleCancelledHarnessTurn(
|
|||
children: session.childAgents,
|
||||
teams: session.teams,
|
||||
messages: session.messages,
|
||||
commands: session.commands,
|
||||
});
|
||||
await managed.sink.write(
|
||||
record(session, sessionId, 'system', null, {
|
||||
|
|
@ -1527,6 +1547,11 @@ export async function parkNeedsNoRuntimeSettlement(
|
|||
// settlement: the faithful cancel is the abandoned-wait fold, never a
|
||||
// settle that would orphan the ledger.
|
||||
if (harnessCheckpointIsAgentWait(checkpoint)) return false;
|
||||
// So does one parked at a member's command wait (H4e-b3): only the
|
||||
// recovery cancel folds its runs, each answered by its own journal; a
|
||||
// plain settle would leave the checkpoint parked and refuse the next
|
||||
// Turn's model start.
|
||||
if (harnessCheckpointIsCommandWait(checkpoint)) return false;
|
||||
if (
|
||||
!(checkpoint.tools?.items ?? []).every(
|
||||
(item) => item.state === 'settled' && item.consumed,
|
||||
|
|
@ -2050,6 +2075,9 @@ async function verifyWorkspaceRestore(
|
|||
// H4e-b1: the lead's team roster and board.
|
||||
'team_state',
|
||||
'team_task',
|
||||
// H4e-b3: member command ledgers — the outbox entries, the
|
||||
// lead's inbound receipts, and the replies they carry.
|
||||
'session_command',
|
||||
].includes(event.payload['domain'] as string)
|
||||
)
|
||||
throw new Error('Hosted recovery domain is unsupported.');
|
||||
|
|
@ -2916,6 +2944,7 @@ async function executeHostedTurn(
|
|||
childAgents: childAgentsOf(session),
|
||||
messages: session.messages,
|
||||
teams: session.teams,
|
||||
commands: session.commands,
|
||||
memberPlanMode: await ensureMemberPlanMode(session),
|
||||
},
|
||||
)
|
||||
|
|
@ -4105,8 +4134,20 @@ export function registerHostedHarnessSessionRoutes(
|
|||
// H4f cancel would, through the child funnel's own chain.
|
||||
session.childAgents,
|
||||
);
|
||||
// H4d-b: the Session messages along its lineage on the same
|
||||
// writes chain; a child names its parent from its definition.
|
||||
const lineage =
|
||||
session.childDepth !== undefined &&
|
||||
typeof savedLineage?.['parentSessionId'] === 'string' &&
|
||||
typeof savedLineage['parentChildRunId'] === 'string'
|
||||
? {
|
||||
parentSessionId: savedLineage['parentSessionId'],
|
||||
parentChildRunId: savedLineage['parentChildRunId'],
|
||||
}
|
||||
: undefined;
|
||||
// H4d-b: messages share the child orchestrator's write chain, so
|
||||
// a Shell lane that suppressed the agent tool has no funnel.
|
||||
// a Shell lane that suppressed the agent tool has no funnel. The
|
||||
// commands funnel (H4e-b3) rides the same guard.
|
||||
if (session.childAgents) {
|
||||
session.messages = new HostedSessionMessageSession(
|
||||
{
|
||||
|
|
@ -4115,14 +4156,19 @@ export function registerHostedHarnessSessionRoutes(
|
|||
},
|
||||
session.managed.authority.sessionHeader.sessionKey,
|
||||
session.childAgents,
|
||||
session.childDepth !== undefined &&
|
||||
typeof savedLineage?.['parentSessionId'] === 'string' &&
|
||||
typeof savedLineage['parentChildRunId'] === 'string'
|
||||
? {
|
||||
parentSessionId: savedLineage['parentSessionId'],
|
||||
parentChildRunId: savedLineage['parentChildRunId'],
|
||||
}
|
||||
: undefined,
|
||||
lineage,
|
||||
);
|
||||
// H4e-b3: the Session's commands run beside its messages — its
|
||||
// own outbox as a member, and the lead's execute on the same
|
||||
// funnels when its children command it.
|
||||
session.commands = new HostedSessionCommandSession(
|
||||
{
|
||||
authority: session.managed.authority,
|
||||
resources: session.managed.resources,
|
||||
},
|
||||
session.managed.authority.sessionHeader.sessionKey,
|
||||
session.childAgents,
|
||||
lineage,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -4327,6 +4373,7 @@ export function registerHostedHarnessSessionRoutes(
|
|||
session.childConsumption.add(childRunId),
|
||||
teams: session.teams,
|
||||
messages: session.messages,
|
||||
commands: session.commands,
|
||||
});
|
||||
} catch (cause) {
|
||||
// R6 P1: a durable decline freezes for the fleet, but a
|
||||
|
|
@ -4594,6 +4641,7 @@ export function registerHostedHarnessSessionRoutes(
|
|||
children: session.childAgents,
|
||||
teams: session.teams,
|
||||
messages: session.messages,
|
||||
commands: session.commands,
|
||||
});
|
||||
} catch (cause) {
|
||||
writeStderrLineSafe(
|
||||
|
|
@ -5021,6 +5069,7 @@ export function registerHostedHarnessSessionRoutes(
|
|||
children: session.childAgents,
|
||||
teams: session.teams,
|
||||
messages: session.messages,
|
||||
commands: session.commands,
|
||||
});
|
||||
const projected = await managed.sink.project();
|
||||
const current = projected.filter(
|
||||
|
|
@ -6790,6 +6839,178 @@ export function registerHostedHarnessSessionRoutes(
|
|||
});
|
||||
};
|
||||
|
||||
// One command verb onto the Session's journal, for the route below.
|
||||
// Same replay rules as the message verbs; `execute` additionally answers
|
||||
// with the reply document so the relay can copy it back to the member.
|
||||
const runCommandOperation = async (
|
||||
req: Request,
|
||||
res: Response,
|
||||
session: HostedSession,
|
||||
commands: HostedSessionCommandSession,
|
||||
): Promise<void> => {
|
||||
const body = object(req.body);
|
||||
const operationId = body?.['operationId'];
|
||||
const commandId = body?.['commandId'];
|
||||
const kind = body?.['kind'];
|
||||
if (typeof operationId !== 'string' || !HOSTED_UUID.test(operationId)) {
|
||||
return error(res, 400, 'invalid_command_operation');
|
||||
}
|
||||
if (
|
||||
typeof commandId !== 'string' ||
|
||||
commandId.length < 1 ||
|
||||
commandId.length > 320
|
||||
) {
|
||||
return error(res, 400, 'invalid_command_operation');
|
||||
}
|
||||
let reply: SessionCommandReply | undefined;
|
||||
try {
|
||||
switch (kind) {
|
||||
case 'handover': {
|
||||
const targetSessionId = body?.['targetSessionId'];
|
||||
if (
|
||||
typeof targetSessionId !== 'string' ||
|
||||
!HOSTED_UUID.test(targetSessionId)
|
||||
) {
|
||||
return error(res, 400, 'invalid_command_operation');
|
||||
}
|
||||
await commands.handover(commandId, targetSessionId);
|
||||
break;
|
||||
}
|
||||
case 'answered': {
|
||||
try {
|
||||
await commands.answered(
|
||||
commandId,
|
||||
parseSessionCommandReply(body?.['reply']),
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof ManagedSessionRecordError) {
|
||||
return error(res, 400, 'invalid_command_operation');
|
||||
}
|
||||
throw cause;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'cancelled':
|
||||
case 'rejected':
|
||||
case 'unknown':
|
||||
await commands.settle(commandId, kind);
|
||||
break;
|
||||
case 'execute': {
|
||||
const commandKind = body?.['commandKind'];
|
||||
const route = body?.['route'];
|
||||
const childRunId = body?.['childRunId'];
|
||||
const senderSessionId = body?.['senderSessionId'];
|
||||
const argsBase64 = body?.['argsBase64'];
|
||||
const argsDigest = body?.['argsDigest'];
|
||||
if (
|
||||
(commandKind !== 'board_read' &&
|
||||
commandKind !== 'board_write' &&
|
||||
commandKind !== 'member_message' &&
|
||||
commandKind !== 'plan_submit') ||
|
||||
route !== 'to_parent' ||
|
||||
typeof childRunId !== 'string' ||
|
||||
childRunId.length < 1 ||
|
||||
childRunId.length > 320 ||
|
||||
typeof senderSessionId !== 'string' ||
|
||||
!HOSTED_UUID.test(senderSessionId) ||
|
||||
typeof argsBase64 !== 'string' ||
|
||||
argsBase64.length < 1 ||
|
||||
// The base64 of at most the 64 KiB arguments bound.
|
||||
argsBase64.length >
|
||||
Math.ceil(MANAGED_SESSION_COMMAND_LIMITS.maxArgsBytes / 3) * 4 ||
|
||||
typeof argsDigest !== 'string' ||
|
||||
!/^[0-9a-f]{64}$/.test(argsDigest)
|
||||
) {
|
||||
return error(res, 400, 'invalid_command_operation');
|
||||
}
|
||||
const argsBytes = Buffer.from(argsBase64, 'base64');
|
||||
if (
|
||||
createHash('sha256').update(argsBytes).digest('hex') !== argsDigest
|
||||
) {
|
||||
return error(
|
||||
res,
|
||||
409,
|
||||
'session_command_refused',
|
||||
undefined,
|
||||
`Session command ${commandId} names a digest that does not match its arguments.`,
|
||||
);
|
||||
}
|
||||
let args: Record<string, unknown>;
|
||||
try {
|
||||
const parsed: unknown = JSON.parse(argsBytes.toString('utf8'));
|
||||
if (
|
||||
typeof parsed !== 'object' ||
|
||||
parsed === null ||
|
||||
Array.isArray(parsed)
|
||||
) {
|
||||
throw new Error('not an object');
|
||||
}
|
||||
args = parsed as Record<string, unknown>;
|
||||
} catch {
|
||||
return error(
|
||||
res,
|
||||
409,
|
||||
'session_command_refused',
|
||||
undefined,
|
||||
`Session command ${commandId} carries arguments that are not a JSON object.`,
|
||||
);
|
||||
}
|
||||
if (session.teams === undefined) {
|
||||
return error(res, 409, 'hosted_commands_unavailable');
|
||||
}
|
||||
reply = await commands.execute(session.teams, {
|
||||
commandId,
|
||||
kind: commandKind as SessionCommandKind,
|
||||
childRunId,
|
||||
senderSessionId,
|
||||
args,
|
||||
});
|
||||
break;
|
||||
}
|
||||
default:
|
||||
return error(res, 400, 'invalid_command_operation');
|
||||
}
|
||||
} catch (cause) {
|
||||
const message = cause instanceof Error ? cause.message : String(cause);
|
||||
// A command whose sender has not attached yet waits for the
|
||||
// control plane's attach commit: the relay defers it untouched,
|
||||
// never toward `rejected` and never spending an attempt.
|
||||
if (cause instanceof SessionCommandBlockedError) {
|
||||
return error(res, 409, 'session_command_blocked', undefined, message);
|
||||
}
|
||||
// The lead cannot accept the command at all: the relay walks this
|
||||
// entry into `rejected`, never re-executing it.
|
||||
if (cause instanceof SessionCommandRefusalError) {
|
||||
return error(res, 409, 'session_command_refused', undefined, message);
|
||||
}
|
||||
if (
|
||||
cause instanceof ManagedSessionStoreHttpError ||
|
||||
cause instanceof ManagedSessionStoreTransportError ||
|
||||
cause instanceof ManagedSessionWritesStoppedError
|
||||
) {
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted command operation ${kind} of session ${req.params['id']} failed in its store: ${message}`,
|
||||
);
|
||||
return error(res, 503, 'session_command_failed', undefined, message);
|
||||
}
|
||||
if (cause instanceof ManagedSessionConflictError) {
|
||||
return error(res, 409, 'session_command_conflict', undefined, message);
|
||||
}
|
||||
if (cause instanceof ManagedSessionRecordError) {
|
||||
return error(res, 409, 'session_command_record', undefined, message);
|
||||
}
|
||||
writeStderrLineSafe(
|
||||
`qwen serve: Hosted command operation ${kind} of session ${req.params['id']} failed: ${message}`,
|
||||
);
|
||||
return error(res, 503, 'session_command_failed', undefined, message);
|
||||
}
|
||||
res.status(202).json({
|
||||
operationId,
|
||||
state: 'settled',
|
||||
...(reply === undefined ? {} : { reply }),
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* H4d-b: the control plane's message relay onto this Session's journal.
|
||||
* A sender's outbox entry moves through its handover, its acceptance and
|
||||
|
|
@ -6799,23 +7020,45 @@ export function registerHostedHarnessSessionRoutes(
|
|||
* a second input. A turn in flight is not a refusal: the input queues
|
||||
* behind it in the journal. The child result relay's `stop` (H4f) is
|
||||
* idempotent rather than replayed: it settles only what is still waiting.
|
||||
*
|
||||
* H4e-b3: the same route carries the command relay's verbs when the body
|
||||
* names a `commandId` — `handover`, `execute`, `answered` and the three
|
||||
* endings — on the Session's command funnel instead.
|
||||
*/
|
||||
app.post('/session/:id/messages/operations', async (req, res) => {
|
||||
const session = identity(req, sessions);
|
||||
if (!session) return error(res, 404, 'hosted_session_not_found');
|
||||
if (!session.messages)
|
||||
return error(res, 409, 'hosted_messages_unavailable');
|
||||
if (session.blocked)
|
||||
return error(
|
||||
res,
|
||||
409,
|
||||
'hosted_turn_recovery_required',
|
||||
hostedTurnBusyReason(session),
|
||||
);
|
||||
if (session.mcpClosing) return error(res, 409, 'hosted_session_closing');
|
||||
const commanding = typeof object(req.body)?.['commandId'] === 'string';
|
||||
let operation: Promise<void>;
|
||||
if (commanding) {
|
||||
if (session.commands === undefined) {
|
||||
return error(res, 409, 'hosted_commands_unavailable');
|
||||
}
|
||||
if (session.blocked)
|
||||
return error(
|
||||
res,
|
||||
409,
|
||||
'hosted_turn_recovery_required',
|
||||
hostedTurnBusyReason(session),
|
||||
);
|
||||
if (session.mcpClosing) return error(res, 409, 'hosted_session_closing');
|
||||
operation = runCommandOperation(req, res, session, session.commands);
|
||||
} else {
|
||||
if (session.messages === undefined) {
|
||||
return error(res, 409, 'hosted_messages_unavailable');
|
||||
}
|
||||
if (session.blocked)
|
||||
return error(
|
||||
res,
|
||||
409,
|
||||
'hosted_turn_recovery_required',
|
||||
hostedTurnBusyReason(session),
|
||||
);
|
||||
if (session.mcpClosing) return error(res, 409, 'hosted_session_closing');
|
||||
operation = runMessageOperation(req, res, session, session.messages);
|
||||
}
|
||||
// The close waits out an operation that passed the check above, so a
|
||||
// receipt never lands after its pass over the pending inputs.
|
||||
const operation = runMessageOperation(req, res, session, session.messages);
|
||||
const settled = operation.catch(() => undefined);
|
||||
const operations = (session.messageOperations ??= new Set());
|
||||
operations.add(settled);
|
||||
|
|
@ -7075,9 +7318,14 @@ export function registerHostedHarnessSessionRoutes(
|
|||
'results_ready' &&
|
||||
// The agent wait (#13708): still outstanding, or folded past it —
|
||||
// the consumed advance lands as model_output_committed carrying the
|
||||
// wait's group. Both re-enter through the same resume arm below.
|
||||
// wait's group. Both re-enter through the same resume arm below, and
|
||||
// so does the command wait (H4e-b3), which the takeover reports as
|
||||
// await_command.
|
||||
!harnessCheckpointIsAgentWait(
|
||||
continueAuthorization.authorization.checkpoint,
|
||||
) &&
|
||||
!harnessCheckpointIsCommandWait(
|
||||
continueAuthorization.authorization.checkpoint,
|
||||
)
|
||||
) {
|
||||
releaseRecoveredRuntime(session);
|
||||
|
|
@ -7243,6 +7491,7 @@ export function registerHostedHarnessSessionRoutes(
|
|||
childAgents: childAgentsOf(session),
|
||||
messages: session.messages,
|
||||
teams: session.teams,
|
||||
commands: session.commands,
|
||||
memberPlanMode: await ensureMemberPlanMode(session),
|
||||
},
|
||||
);
|
||||
|
|
@ -7314,6 +7563,59 @@ export function registerHostedHarnessSessionRoutes(
|
|||
({ history, resumeParts } = deriveRound());
|
||||
}
|
||||
}
|
||||
// H4e-b3: the command wait resumes exactly like the agent wait —
|
||||
// re-enter each outstanding run's poll with what is left of its
|
||||
// bound, then the round's never-reached calls pair off.
|
||||
if (
|
||||
continueAuthorization.authorization.status === 'runnable' &&
|
||||
continueAuthorization.authorization.checkpoint.commandWait !== null
|
||||
) {
|
||||
const commandWait =
|
||||
continueAuthorization.authorization.checkpoint.commandWait;
|
||||
if (session.commands === undefined)
|
||||
throw new Error('Command wait checkpoint has no command funnel.');
|
||||
const outstanding = commandWait.runs.filter((run) => !run.folded);
|
||||
if (outstanding.length > 0) {
|
||||
await toolTurn.resumeCommandWaitRuns(
|
||||
outstanding,
|
||||
commandWait.holdsWorkspaceMount,
|
||||
'recovered',
|
||||
abort.signal,
|
||||
);
|
||||
} else {
|
||||
// The carried all-folded shape: restate the last resolve so
|
||||
// this handle adopts the takeover activation.
|
||||
const lastRun = commandWait.runs.at(-1);
|
||||
if (lastRun !== undefined) {
|
||||
let adopted;
|
||||
try {
|
||||
adopted = await harness.resolveAwaitCommand(
|
||||
lastRun.commandId,
|
||||
);
|
||||
} catch (cause) {
|
||||
throw new HostedToolRecoveryRequiredError(cause);
|
||||
}
|
||||
if (adopted === null)
|
||||
throw new Error(
|
||||
'Recovered command wait did not adopt the takeover activation.',
|
||||
);
|
||||
}
|
||||
}
|
||||
const filled = await fillParkedRoundCommandGaps({
|
||||
managed: session.managed,
|
||||
sessionId,
|
||||
promptId,
|
||||
cwd: session.cwd,
|
||||
gapText: HOSTED_AGENT_CALL_NOT_REACHED_TEXT,
|
||||
commands: session.commands,
|
||||
children: session.childAgents,
|
||||
messages: session.messages,
|
||||
});
|
||||
if (outstanding.length > 0 || filled > 0) {
|
||||
projected = await session.managed.sink.project();
|
||||
({ history, resumeParts } = deriveRound());
|
||||
}
|
||||
}
|
||||
// A sibling the Runtime reconciliation does not own (a team
|
||||
// call, a background launch) is answered before the resumed round
|
||||
// reads it: by what committed when it committed before the
|
||||
|
|
@ -7328,6 +7630,7 @@ export function registerHostedHarnessSessionRoutes(
|
|||
children: session.childAgents,
|
||||
teams: session.teams,
|
||||
messages: session.messages,
|
||||
commands: session.commands,
|
||||
});
|
||||
if (answered > 0) {
|
||||
projected = await session.managed.sink.project();
|
||||
|
|
@ -7340,12 +7643,20 @@ export function registerHostedHarnessSessionRoutes(
|
|||
}
|
||||
if (
|
||||
continueAuthorization.authorization.status === 'runnable' &&
|
||||
continueAuthorization.authorization.checkpoint.agentWait !== null
|
||||
(continueAuthorization.authorization.checkpoint.agentWait !==
|
||||
null ||
|
||||
(continueAuthorization.authorization.checkpoint.commandWait !==
|
||||
null &&
|
||||
!continueAuthorization.authorization.checkpoint.commandWait
|
||||
.holdsWorkspaceMount))
|
||||
) {
|
||||
// An agent-wait resume has no Runtime results to consume and
|
||||
// must NOT claim the Workspace mount in advance of any tool
|
||||
// work: the unconditional acquire would make the parent refuse
|
||||
// its own later foreground agent call (R2-3). The pending
|
||||
// its own later foreground agent call (R2-3). A command wait is
|
||||
// the same unless its Turn held the mount, which its resume arm
|
||||
// took back already: a member shares its lead's Workspace, and
|
||||
// a mount it never held is not its to claim. The pending
|
||||
// file-history obligation dies with the Turn the terminal
|
||||
// record will close, exactly like the interrupted settle.
|
||||
const savedHistory = await readHostedFileHistory(session.managed);
|
||||
|
|
@ -7585,8 +7896,43 @@ export function registerHostedHarnessSessionRoutes(
|
|||
// to fold, the carried all-consumed shape names none — either way
|
||||
// the round pairs off — while no group means the Runtime park.
|
||||
const agentWait = cancelRunnable.checkpoint.agentWait;
|
||||
const commandWait = cancelRunnable.checkpoint.commandWait;
|
||||
let broker: HostedWorkspaceBroker | undefined;
|
||||
if (agentWait !== null) {
|
||||
if (commandWait !== null) {
|
||||
// H4e-b3's own mirror of the agent-wait cancellation: the
|
||||
// outstanding command runs answer by what their journal
|
||||
// committed (decision 14), the resolve moves the checkpoint
|
||||
// past the wait, and the round's never-reached calls pair off
|
||||
// from the same journal — no Broker binding ever exists to stop.
|
||||
if (session.commands === undefined) {
|
||||
void (async () => {
|
||||
session.active = undefined;
|
||||
throw new Error('Command wait checkpoint has no command funnel.');
|
||||
})();
|
||||
} else {
|
||||
const outstanding = commandWait.runs.filter((run) => !run.folded);
|
||||
if (outstanding.length > 0) {
|
||||
await settleCancelledCommandWaitRuns({
|
||||
managed: session.managed,
|
||||
sessionId,
|
||||
promptId,
|
||||
cwd: session.cwd,
|
||||
runs: outstanding,
|
||||
commands: session.commands,
|
||||
});
|
||||
}
|
||||
await fillParkedRoundCommandGaps({
|
||||
managed: session.managed,
|
||||
sessionId,
|
||||
promptId,
|
||||
cwd: session.cwd,
|
||||
gapText: HOSTED_AGENT_CALL_NOT_ADMITTED_TEXT,
|
||||
commands: session.commands,
|
||||
children: session.childAgents,
|
||||
messages: session.messages,
|
||||
});
|
||||
}
|
||||
} else if (agentWait !== null) {
|
||||
// The agent wait (#13708): no Runtime executions exist to stop or
|
||||
// settle — each outstanding run is folded as the abandoned answer
|
||||
// the live arm itself would have written, and the resolve moves
|
||||
|
|
@ -7651,6 +7997,7 @@ export function registerHostedHarnessSessionRoutes(
|
|||
children: session.childAgents,
|
||||
teams: session.teams,
|
||||
messages: session.messages,
|
||||
commands: session.commands,
|
||||
});
|
||||
// Whatever shape the wait was in, its round is fully answered now:
|
||||
// the folds above landed (or the journaled proof they were never
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ import {
|
|||
type TeamState,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-team-record.js';
|
||||
import type { ManagedSessionDomain } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-records.js';
|
||||
import { assertManagedSessionDomainEnabled } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-records.js';
|
||||
|
||||
// H4e-b2 of #12827: what a team adds on top of H4d-b's Session messages —
|
||||
// a member is its continuation chain, every run of that chain opens its
|
||||
|
|
@ -83,8 +84,12 @@ export function broadcastMessageId(params: {
|
|||
.slice(0, 32)}`;
|
||||
}
|
||||
|
||||
const PREAMBLE_BODY =
|
||||
'You are a member of this team. Your parent, the agent that launched you, is the team leader. Your result is your answer to the last instruction you receive: if another message from the leader arrives after you have answered, your answer to it becomes your whole result, so answer it in full and repeat anything from your earlier answers the leader still needs. To ask the leader something, use send_message with to "parent"; its reply reaches you after you have answered, as your next instruction. You cannot message other members or edit the team\'s task board.';
|
||||
const PREAMBLE_BODY_HEAD =
|
||||
'You are a member of this team. Your parent, the agent that launched you, is the team leader. Your result is your answer to the last instruction you receive: if another message from the leader arrives after you have answered, your answer to it becomes your whole result, so answer it in full and repeat anything from your earlier answers the leader still needs. To ask the leader something, use send_message with to "parent"; its reply reaches you after you have answered, as your next instruction. ';
|
||||
|
||||
const PREAMBLE_TAIL_NO_COMMANDS =
|
||||
"You cannot message other members or edit the team's task board.";
|
||||
const PREAMBLE_TAIL_WITH_COMMANDS = 'You cannot message other members.';
|
||||
|
||||
/**
|
||||
* H4e-b3 decision 5: what the body adds for a plan-mode run, in the
|
||||
|
|
@ -99,6 +104,10 @@ const PLAN_MODE_BODY =
|
|||
* with. Names are sanitized to `[a-z0-9-]`, so they need no escaping.
|
||||
* H4e-b3 decision 5: the open tag carries `plan-mode="required"` while
|
||||
* the member's runs are plan-required (decision 9's composition rule).
|
||||
* The wording near the end also follows what the build declares: once
|
||||
* the `session_command` domain is enabled, the member may touch the
|
||||
* board, so the closing sentence no longer forbids it. The build
|
||||
* carries member messages (b3c) before the messaging sentence lifts.
|
||||
*/
|
||||
export function teamMemberPreamble(
|
||||
member: {
|
||||
|
|
@ -107,8 +116,20 @@ export function teamMemberPreamble(
|
|||
},
|
||||
planMode: boolean = false,
|
||||
): string {
|
||||
let commandsEnabled = false;
|
||||
try {
|
||||
assertManagedSessionDomainEnabled('session_command');
|
||||
commandsEnabled = true;
|
||||
} catch {
|
||||
commandsEnabled = false;
|
||||
}
|
||||
const attribute = planMode ? ' plan-mode="required"' : '';
|
||||
const body = planMode ? PREAMBLE_BODY + PLAN_MODE_BODY : PREAMBLE_BODY;
|
||||
const body =
|
||||
PREAMBLE_BODY_HEAD +
|
||||
(commandsEnabled
|
||||
? PREAMBLE_TAIL_WITH_COMMANDS
|
||||
: PREAMBLE_TAIL_NO_COMMANDS) +
|
||||
(planMode ? PLAN_MODE_BODY : '');
|
||||
return `<team-member name="${member.name}" team="${member.teamName}"${attribute}>\n${body}\n</team-member>\n\n`;
|
||||
}
|
||||
|
||||
|
|
@ -185,6 +206,29 @@ export function memberPlanModeRequired(
|
|||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* b3b decision 5: whether one run's first input declares member tools —
|
||||
* its instruction part opens with a well-formed preamble block, read
|
||||
* after a launch's `[description]` and its blank line, or after a
|
||||
* continuation's `Your next instruction:` marker (history before it is
|
||||
* escaped, so only the run's own block can match). A misread only ever
|
||||
* hides the tools: admission is the lead's, not the declaration's.
|
||||
*/
|
||||
export function firstInputOpensWithMemberPreamble(firstInput: string): boolean {
|
||||
const marker = 'Your next instruction:';
|
||||
const at = firstInput.lastIndexOf(marker);
|
||||
if (at >= 0) {
|
||||
return PREAMBLE_FORM.test(
|
||||
firstInput.slice(at + marker.length).replace(/^\s+/, ''),
|
||||
);
|
||||
}
|
||||
const wrapped = /^\[[\s\S]*?\]\n\n/.exec(firstInput);
|
||||
if (wrapped !== null) {
|
||||
return PREAMBLE_FORM.test(firstInput.slice(wrapped[0].length));
|
||||
}
|
||||
return PREAMBLE_FORM.test(firstInput);
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision 5's replay evidence: whether a committed instruction is the
|
||||
* sent text alone, or one well-formed preamble block followed by it. The
|
||||
|
|
|
|||
|
|
@ -7,15 +7,18 @@
|
|||
import { randomUUID } from 'node:crypto';
|
||||
import type { Part } from '@google/genai';
|
||||
import type { ManagedSession } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-assembly.js';
|
||||
import { ManagedSessionConflictError } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-authority.js';
|
||||
import { createManagedHarnessHandle } from '@qwen-code/qwen-code-core/managed-runtime/managed-harness-factory.js';
|
||||
import type {
|
||||
HarnessAgentWaitRun,
|
||||
HarnessCommandWaitRun,
|
||||
HarnessRunAuthorization,
|
||||
HarnessToolItem,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-harness-checkpoint.js';
|
||||
import {
|
||||
HARNESS_MODEL_START_PHASES,
|
||||
harnessCheckpointIsAgentWait,
|
||||
harnessCheckpointIsCommandWait,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-harness-checkpoint.js';
|
||||
import { managedChildTaskId } from '@qwen-code/qwen-code-core/managed-runtime/managed-extension-projection.js';
|
||||
import {
|
||||
|
|
@ -35,7 +38,12 @@ import {
|
|||
type HostedChildAgentSession,
|
||||
} from './hosted-child-agent-session.js';
|
||||
import type { HostedSessionMessageSession } from './hosted-session-message-session.js';
|
||||
import {
|
||||
sessionCommandEvidenceText,
|
||||
type HostedSessionCommandSession,
|
||||
} from './hosted-session-command-session.js';
|
||||
import { sessionMessageId } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-message-operations.js';
|
||||
import { sessionCommandId } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-command-operations.js';
|
||||
import {
|
||||
HOSTED_TEAM_TOOL_NAMES,
|
||||
type HostedTeamSession,
|
||||
|
|
@ -58,6 +66,7 @@ import {
|
|||
import { parseTeamState } from '@qwen-code/qwen-code-core/managed-runtime/managed-team-record.js';
|
||||
import {
|
||||
HostedChildChains,
|
||||
hostedPromptKey,
|
||||
hostedMemberView,
|
||||
hostedRouteDetails,
|
||||
isEndedMember,
|
||||
|
|
@ -89,7 +98,7 @@ export interface HostedRuntimeRecoveryExecution {
|
|||
}
|
||||
|
||||
export interface HostedRuntimeRecoveryReport {
|
||||
phase: 'await_runtime' | 'await_agent' | 'results_ready';
|
||||
phase: 'await_runtime' | 'await_agent' | 'await_command' | 'results_ready';
|
||||
checkpointId: string;
|
||||
activationId: string;
|
||||
executions: HostedRuntimeRecoveryExecution[];
|
||||
|
|
@ -428,6 +437,7 @@ async function answerAbandonedTurnCalls(input: {
|
|||
children?: HostedChildAgentSession;
|
||||
teams?: HostedTeamSession;
|
||||
messages?: HostedSessionMessageSession;
|
||||
commands?: HostedSessionCommandSession;
|
||||
}): Promise<number> {
|
||||
const uncommitted = input.uncommitted ?? 'all';
|
||||
const records = (await input.session.sink.project()).filter(
|
||||
|
|
@ -459,8 +469,30 @@ async function answerAbandonedTurnCalls(input: {
|
|||
call.name === 'send_message'
|
||||
? sentMessageText(input, functionCallId, callKey)
|
||||
: undefined;
|
||||
// A member's board call shares the team tools' names, but once its
|
||||
// outbound committed (H4e-b3) it is a command: answered by its own
|
||||
// journal, never as a call that never ran — that would leave the
|
||||
// entry live for the relay to deliver after the Turn gave it up.
|
||||
const commandId =
|
||||
input.commands === undefined
|
||||
? undefined
|
||||
: sessionCommandId({
|
||||
senderSessionId:
|
||||
input.session.authority.sessionHeader.sessionKey.sessionId,
|
||||
promptKey: hostedPromptKey(input.promptId),
|
||||
callId: functionCallId,
|
||||
});
|
||||
let parts: Part[];
|
||||
if (sent !== undefined) {
|
||||
if (
|
||||
commandId !== undefined &&
|
||||
input.commands!.command(commandId) !== undefined
|
||||
) {
|
||||
parts = await sessionCommandJournalAnswer(input.commands!, {
|
||||
commandId,
|
||||
functionCallId,
|
||||
toolName: call.name,
|
||||
});
|
||||
} else if (sent !== undefined) {
|
||||
parts = convertToFunctionResponse(call.name, functionCallId, [
|
||||
{ text: sent },
|
||||
]);
|
||||
|
|
@ -651,6 +683,7 @@ export function answerCommittedTurnCalls(input: {
|
|||
children?: HostedChildAgentSession;
|
||||
teams?: HostedTeamSession;
|
||||
messages?: HostedSessionMessageSession;
|
||||
commands?: HostedSessionCommandSession;
|
||||
}): Promise<number> {
|
||||
return answerAbandonedTurnCalls({
|
||||
...input,
|
||||
|
|
@ -676,6 +709,7 @@ export function answerResumedTurnCalls(input: {
|
|||
children?: HostedChildAgentSession;
|
||||
teams?: HostedTeamSession;
|
||||
messages?: HostedSessionMessageSession;
|
||||
commands?: HostedSessionCommandSession;
|
||||
}): Promise<number> {
|
||||
return answerAbandonedTurnCalls({
|
||||
...input,
|
||||
|
|
@ -967,6 +1001,203 @@ export async function fillParkedRoundAgentGaps(input: {
|
|||
return filled;
|
||||
}
|
||||
|
||||
/**
|
||||
* A member command call's answer by what its journal committed (H4e-b3
|
||||
* decision 14): an accepted entry's reply; a planned entry cancelled and
|
||||
* told as not sent; a handed-over one told it reached the leader with its
|
||||
* reply still missing and a write possibly landed; rejected, cancelled
|
||||
* and unknown said so. The settle's conflict is the relay race's own
|
||||
* evidence: once it loses, the journal is re-read and answered as it
|
||||
* committed. It never consumes the entry — no waiting call reads it.
|
||||
*/
|
||||
async function sessionCommandJournalAnswer(
|
||||
commands: HostedSessionCommandSession,
|
||||
run: Pick<HarnessCommandWaitRun, 'commandId' | 'functionCallId' | 'toolName'>,
|
||||
): Promise<Part[]> {
|
||||
for (let attempt = 0; attempt < 4; attempt += 1) {
|
||||
const record = commands.command(run.commandId);
|
||||
const state =
|
||||
record === undefined
|
||||
? 'planned'
|
||||
: (record.run.delivery?.state ?? 'planned');
|
||||
if (state === 'accepted' || state === 'consumed') {
|
||||
const reply = await commands.readReply(run.commandId);
|
||||
if (reply === null) continue;
|
||||
return reply.outcome === 'answered'
|
||||
? convertToFunctionResponse(run.toolName, run.functionCallId, [
|
||||
{ text: reply.text },
|
||||
])
|
||||
: convertToFunctionErrorResponse(
|
||||
run.toolName,
|
||||
run.functionCallId,
|
||||
[],
|
||||
reply.text,
|
||||
);
|
||||
}
|
||||
try {
|
||||
if (record !== undefined && state === 'planned') {
|
||||
await commands.settle(run.commandId, 'cancelled');
|
||||
} else if (state === 'accepting') {
|
||||
await commands.settle(run.commandId, 'unknown');
|
||||
}
|
||||
} catch (cause) {
|
||||
if (cause instanceof ManagedSessionConflictError) continue;
|
||||
throw cause;
|
||||
}
|
||||
return convertToFunctionErrorResponse(
|
||||
run.toolName,
|
||||
run.functionCallId,
|
||||
[],
|
||||
sessionCommandEvidenceText(
|
||||
// The accepted/consumed arm above answered already; what remains
|
||||
// of the delivery vocabulary is exactly the settle-relevant five.
|
||||
state as 'planned' | 'accepting' | 'rejected' | 'cancelled' | 'unknown',
|
||||
),
|
||||
);
|
||||
}
|
||||
throw new Error(`Session command ${run.commandId} never settled.`);
|
||||
}
|
||||
|
||||
/**
|
||||
* The command wait's cancellation settlement (H4e-b3 decision 14): every
|
||||
* outstanding wait run is answered by what its own journal committed —
|
||||
* an accepted entry's reply; a planned entry cancelled and told as not
|
||||
* sent; a handed-over one told it reached the leader with its reply still
|
||||
* missing and a write possibly landed; rejected, cancelled and unknown
|
||||
* said so — or the checkpoint moves past the wait with no terminal
|
||||
* record stranding the Turn. The settle's conflict is the relay race's
|
||||
* own evidence: once it loses, the journal is re-read and answered as it
|
||||
* committed. The journaled set keeps a replayed cancel silent. Shared by
|
||||
* the cancel/close path and the interrupted-turn settle.
|
||||
*/
|
||||
export async function settleCancelledCommandWaitRuns(input: {
|
||||
managed: ManagedSession;
|
||||
sessionId: string;
|
||||
promptId: string;
|
||||
cwd: string;
|
||||
runs: readonly HarnessCommandWaitRun[];
|
||||
commands: HostedSessionCommandSession;
|
||||
}): Promise<void> {
|
||||
const harness = createManagedHarnessHandle(input.managed);
|
||||
const journaled = await journaledToolResultIds(input.managed, input.promptId);
|
||||
for (const run of input.runs) {
|
||||
const parts = await sessionCommandJournalAnswer(input.commands, run);
|
||||
if (!journaled.has(run.functionCallId))
|
||||
await input.managed.sink.write({
|
||||
uuid: randomUUID(),
|
||||
parentUuid: run.modelMessageId,
|
||||
sessionId: input.sessionId,
|
||||
timestamp: new Date().toISOString(),
|
||||
type: 'tool_result',
|
||||
cwd: input.cwd,
|
||||
version: 'hosted-harness/1',
|
||||
daemonPromptId: input.promptId,
|
||||
message: { role: 'user', parts },
|
||||
});
|
||||
await harness.resolveAwaitCommand(run.commandId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The gap fill of a parked command round (H4e-b3 decision 4): a command
|
||||
* batch parks with some calls of its last assistant round never reached
|
||||
* by the dead loop — the checkpoint names every admitted command call,
|
||||
* so the only gaps are the journal-only siblings (`send_message` to
|
||||
* `"parent"`), which fill from what committed, and anything validation
|
||||
* refused, which never ran. A call named by the checkpoint is never a
|
||||
* gap: its fold and its replay-safe marks belong to the wait-arm channel
|
||||
* (settle / resume). Shared by the cancel/close path and the
|
||||
* interrupted-turn settle.
|
||||
*/
|
||||
export async function fillParkedRoundCommandGaps(input: {
|
||||
managed: ManagedSession;
|
||||
sessionId: string;
|
||||
promptId: string;
|
||||
cwd: string;
|
||||
/** The never-admitted sibling's answer wording, caller-named. */
|
||||
gapText: string;
|
||||
commands: HostedSessionCommandSession;
|
||||
children?: HostedChildAgentSession;
|
||||
messages?: HostedSessionMessageSession;
|
||||
}): Promise<number> {
|
||||
const authorization = await input.managed.authority
|
||||
.harnessRunAuthorization()
|
||||
.catch(() => undefined);
|
||||
const waitRuns =
|
||||
authorization?.status === 'runnable'
|
||||
? (authorization.checkpoint.commandWait?.runs ?? [])
|
||||
: [];
|
||||
const namedRounds = new Set(waitRuns.map((run) => run.modelMessageId));
|
||||
const waitedCalls = new Set(waitRuns.map((run) => run.functionCallId));
|
||||
const projected = await input.managed.sink.project();
|
||||
const assistant = projected
|
||||
.filter(
|
||||
(entry) =>
|
||||
entry.daemonPromptId === input.promptId &&
|
||||
entry.type === 'assistant' &&
|
||||
entry.message?.parts?.some((part) => part.functionCall) &&
|
||||
namedRounds.has(entry.uuid),
|
||||
)
|
||||
.at(-1);
|
||||
if (assistant === undefined) return 0;
|
||||
const journaled = await journaledToolResultIds(
|
||||
input.managed,
|
||||
input.promptId,
|
||||
projected,
|
||||
);
|
||||
let filled = 0;
|
||||
for (const part of assistant.message?.parts ?? []) {
|
||||
const callId = part.functionCall?.id;
|
||||
const name = part.functionCall?.name;
|
||||
if (typeof callId !== 'string' || typeof name !== 'string') continue;
|
||||
if (waitedCalls.has(callId)) continue;
|
||||
const foldOwed = !journaled.has(callId);
|
||||
const sent =
|
||||
name === 'send_message'
|
||||
? await sentMessageText(
|
||||
{
|
||||
session: input.managed,
|
||||
promptId: input.promptId,
|
||||
children: input.children,
|
||||
messages: input.messages,
|
||||
},
|
||||
callId,
|
||||
hostedChildRunIdFor(input.promptId, callId),
|
||||
)
|
||||
: undefined;
|
||||
let parts: Part[] | undefined;
|
||||
if (sent !== undefined) {
|
||||
parts = convertToFunctionResponse(name, callId, [{ text: sent }]);
|
||||
} else {
|
||||
parts = convertToFunctionErrorResponse(
|
||||
name,
|
||||
callId,
|
||||
[],
|
||||
['task_list', 'task_create', 'task_update'].includes(name)
|
||||
? sessionCommandEvidenceText('planned')
|
||||
: input.gapText,
|
||||
);
|
||||
}
|
||||
if (foldOwed) {
|
||||
await input.managed.sink.write({
|
||||
uuid: randomUUID(),
|
||||
parentUuid: assistant.uuid,
|
||||
sessionId: input.sessionId,
|
||||
timestamp: new Date().toISOString(),
|
||||
type: 'tool_result',
|
||||
cwd: input.cwd,
|
||||
version: 'hosted-harness/1',
|
||||
model: 'recovered',
|
||||
daemonPromptId: input.promptId,
|
||||
message: { role: 'user', parts },
|
||||
});
|
||||
filled += 1;
|
||||
}
|
||||
journaled.add(callId);
|
||||
}
|
||||
return filled;
|
||||
}
|
||||
|
||||
/**
|
||||
* H5/F5 follow-up: the Runtime-facing settlement of an interrupted Turn
|
||||
* ahead of its terminal record. A Turn that died inside a tool call parked
|
||||
|
|
@ -1013,6 +1244,8 @@ export async function settleInterruptedTurnRuntime(input: {
|
|||
teams?: HostedTeamSession;
|
||||
/** H4d-b: the message funnel that tells which send_message committed. */
|
||||
messages?: HostedSessionMessageSession;
|
||||
/** H4e-b3: the command funnel that tells a command's committed evidence. */
|
||||
commands?: HostedSessionCommandSession;
|
||||
}): Promise<HostedInterruptedTurnRuntime> {
|
||||
const authorization = await input.session.authority.harnessRunAuthorization();
|
||||
// Only a committed, readable checkpoint — or the durable absence of any
|
||||
|
|
@ -1117,6 +1350,37 @@ export async function settleInterruptedTurnRuntime(input: {
|
|||
consume: input.consume,
|
||||
});
|
||||
}
|
||||
if (harnessCheckpointIsCommandWait(authorization.checkpoint)) {
|
||||
// H4e-b3 decision 14: no Runtime executions exist to stop either —
|
||||
// each outstanding command run answers by what its own journal
|
||||
// committed, the resolve moves the checkpoint past the wait, and
|
||||
// the round's never-reached calls pair off from the same journal.
|
||||
if (input.commands === undefined)
|
||||
throw new Error('Command wait checkpoint has no command funnel.');
|
||||
const outstanding = (
|
||||
authorization.checkpoint.commandWait?.runs ?? []
|
||||
).filter((run) => !run.folded);
|
||||
if (outstanding.length > 0) {
|
||||
await settleCancelledCommandWaitRuns({
|
||||
managed: input.session,
|
||||
sessionId: input.sessionId,
|
||||
promptId: input.promptId,
|
||||
cwd: input.cwd,
|
||||
runs: outstanding,
|
||||
commands: input.commands,
|
||||
});
|
||||
}
|
||||
await fillParkedRoundCommandGaps({
|
||||
managed: input.session,
|
||||
sessionId: input.sessionId,
|
||||
promptId: input.promptId,
|
||||
cwd: input.cwd,
|
||||
gapText: HOSTED_AGENT_CALL_NOT_REACHED_TEXT,
|
||||
commands: input.commands,
|
||||
children: input.children,
|
||||
messages: input.messages,
|
||||
});
|
||||
}
|
||||
}
|
||||
// The dead Turn's pending file-history obligation dies with it: keep
|
||||
// the snapshots, drop the marker, or every later load stays refused.
|
||||
|
|
@ -1155,6 +1419,7 @@ export async function settleInterruptedTurnRuntime(input: {
|
|||
children: input.children,
|
||||
teams: input.teams,
|
||||
messages: input.messages,
|
||||
commands: input.commands,
|
||||
});
|
||||
} else {
|
||||
// A Turn whose checkpoint never bound it — a batch of team calls and
|
||||
|
|
@ -1168,6 +1433,7 @@ export async function settleInterruptedTurnRuntime(input: {
|
|||
children: input.children,
|
||||
teams: input.teams,
|
||||
messages: input.messages,
|
||||
commands: input.commands,
|
||||
});
|
||||
}
|
||||
// The handback owed for a taken Workspace survives a settlement split
|
||||
|
|
@ -1365,6 +1631,31 @@ export async function recoverHostedRuntimeTurn(input: {
|
|||
},
|
||||
});
|
||||
}
|
||||
// H4e-b3: the command wait classifies exactly like the agent wait —
|
||||
// a folded run reads as settled, an outstanding one as executing, and
|
||||
// its reply is the relay's durable evidence, so every outcome is
|
||||
// known. No Runtime lease is owed on this load either: the wait owns
|
||||
// no Broker binding; the mount the checkpoint names a Turn took in an
|
||||
// earlier batch returns with its resume, not with the report.
|
||||
if (harnessCheckpointIsCommandWait(checkpoint)) {
|
||||
return recovered({
|
||||
promptId,
|
||||
acquiredRuntime: false,
|
||||
report: {
|
||||
phase: 'await_command',
|
||||
checkpointId: checkpoint.identity.checkpointId,
|
||||
activationId: session.activation.activationId,
|
||||
executions: (checkpoint.commandWait?.runs ?? []).map((run) => ({
|
||||
functionCallId: run.functionCallId,
|
||||
toolName: run.toolName,
|
||||
executionCallId: run.commandId,
|
||||
runtimeSessionId: hostedRuntimeSessionId(promptId),
|
||||
outcome: 'known',
|
||||
status: { state: run.folded ? 'settled' : 'executing' },
|
||||
})),
|
||||
},
|
||||
});
|
||||
}
|
||||
// Settled in the checkpoint while the journal never landed the settle:
|
||||
// it completed and must never be recorded as a failure — answering
|
||||
// inapplicable hands the load route the one case it DOES project
|
||||
|
|
|
|||
618
packages/cli/src/serve/hosted-session-command-session.test.ts
Normal file
618
packages/cli/src/serve/hosted-session-command-session.test.ts
Normal file
|
|
@ -0,0 +1,618 @@
|
|||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { mkdtemp, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { afterEach, describe, expect, it } from 'vitest';
|
||||
import {
|
||||
openManagedSession,
|
||||
type ManagedSession,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-session-assembly.js';
|
||||
import { LocalManagedSessionResourceStore } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-resources.js';
|
||||
import { sessionCommandId } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-command-operations.js';
|
||||
import { ManagedSessionConflictError } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-authority.js';
|
||||
import { HostedChildAgentSession } from './hosted-child-agent-session.js';
|
||||
import {
|
||||
HostedSessionCommandSession,
|
||||
SessionCommandBlockedError,
|
||||
SessionCommandRefusalError,
|
||||
} from './hosted-session-command-session.js';
|
||||
import { HostedTeamSession } from './hosted-team-session.js';
|
||||
|
||||
// H4e-b3: `session_command` is enabled for real, so this suite drives the
|
||||
// funnel with no enablement mock. The member side only needs its own
|
||||
// lineage Session; the lead side additionally needs the member's run
|
||||
// launched and attached in its own journal — that binding is what the
|
||||
// inbound lineage rule proves.
|
||||
|
||||
const roots: string[] = [];
|
||||
const sessions: ManagedSession[] = [];
|
||||
const PARENT = randomUUID();
|
||||
const CHILD = randomUUID();
|
||||
const LEAF_RUN = 'run-1';
|
||||
|
||||
afterEach(async () => {
|
||||
for (const session of sessions.splice(0)) await session.close();
|
||||
for (const root of roots.splice(0))
|
||||
await rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
async function open(sessionId: string): Promise<ManagedSession> {
|
||||
const root = await mkdtemp(path.join(tmpdir(), 'hosted-command-'));
|
||||
roots.push(root);
|
||||
const sessionKey = {
|
||||
tenantId: 'tenant',
|
||||
workspaceId: 'workspace',
|
||||
sessionId,
|
||||
};
|
||||
const resources = LocalManagedSessionResourceStore.create({
|
||||
runtimeBaseDir: root,
|
||||
sessionKey,
|
||||
});
|
||||
const session = await openManagedSession({
|
||||
runtimeBaseDir: root,
|
||||
cwd: root,
|
||||
transcriptPath: path.join(root, 'transcript.jsonl'),
|
||||
sessionId,
|
||||
sessionKey,
|
||||
version: 'test',
|
||||
workerId: 'worker',
|
||||
activationLeaseDurationMs: 60_000,
|
||||
create: {
|
||||
definitionRef: await resources.publish(
|
||||
'managed-definition',
|
||||
Buffer.from('{}'),
|
||||
),
|
||||
rootSnapshotRef: await resources.publish(
|
||||
'managed-root',
|
||||
Buffer.from('{}'),
|
||||
),
|
||||
createdBy: 'test',
|
||||
},
|
||||
});
|
||||
sessions.push(session);
|
||||
return session;
|
||||
}
|
||||
|
||||
interface Side {
|
||||
readonly session: ManagedSession;
|
||||
readonly children: HostedChildAgentSession;
|
||||
readonly commands: HostedSessionCommandSession;
|
||||
}
|
||||
|
||||
async function side(
|
||||
sessionId: string,
|
||||
lineage?: { parentSessionId: string; parentChildRunId: string },
|
||||
): Promise<Side> {
|
||||
const session = await open(sessionId);
|
||||
const store = { authority: session.authority, resources: session.resources };
|
||||
const key = session.authority.sessionHeader.sessionKey;
|
||||
const children = new HostedChildAgentSession(store, key);
|
||||
return {
|
||||
session,
|
||||
children,
|
||||
commands: new HostedSessionCommandSession(store, key, children, lineage),
|
||||
};
|
||||
}
|
||||
|
||||
async function memberSide(): Promise<Side> {
|
||||
return side(CHILD, {
|
||||
parentSessionId: PARENT,
|
||||
parentChildRunId: LEAF_RUN,
|
||||
});
|
||||
}
|
||||
|
||||
const DEFINITION = {
|
||||
definitionId: 'hosted-agent/hosted-workspace-shell/1',
|
||||
definitionRevision: 1,
|
||||
definitionDigest: 'f'.repeat(64),
|
||||
};
|
||||
|
||||
async function leadSideSetup(): Promise<Side> {
|
||||
const lead = await side(PARENT, undefined);
|
||||
// The member's run in the lead's own journal, attached — an attached
|
||||
// child Session run that the inbound lineage rule can prove the
|
||||
// sender against.
|
||||
await lead.children.admit({
|
||||
completion: 'sent',
|
||||
childRunId: LEAF_RUN,
|
||||
ownerScopeId: PARENT,
|
||||
rootSessionId: PARENT,
|
||||
description: 'audit the diff',
|
||||
prompt: 'review the change',
|
||||
definition: DEFINITION,
|
||||
workingDirectory: '.',
|
||||
workspaceMode: 'shared',
|
||||
executionCallId: LEAF_RUN,
|
||||
} as never);
|
||||
await lead.children.dispatchStarted(LEAF_RUN, {
|
||||
dispatchId: `dispatch-${LEAF_RUN}`,
|
||||
runtime: { runtimeBindingId: 'binding-1', generation: '1' },
|
||||
});
|
||||
await lead.children.attach(LEAF_RUN, CHILD);
|
||||
return lead;
|
||||
}
|
||||
|
||||
function commandIdOf(callId: string): string {
|
||||
return sessionCommandId({
|
||||
senderSessionId: CHILD,
|
||||
promptKey: 'prompt-1',
|
||||
callId,
|
||||
});
|
||||
}
|
||||
|
||||
/** The lead with an open team and `bob` on it, running as LEAF_RUN. */
|
||||
async function leadWithTeam(): Promise<
|
||||
Side & { readonly teams: HostedTeamSession }
|
||||
> {
|
||||
const lead = await leadSideSetup();
|
||||
const teams = new HostedTeamSession(
|
||||
{
|
||||
authority: lead.session.authority,
|
||||
resources: lead.session.resources,
|
||||
},
|
||||
lead.session.authority.sessionHeader.sessionKey,
|
||||
);
|
||||
await teams.run('team_create', { team_name: 'Review' }, 'prompt:call-team');
|
||||
const member = teams.admitMember('bob');
|
||||
expect(await teams.join({ ...member, childRunId: LEAF_RUN })).toEqual({
|
||||
joined: true,
|
||||
});
|
||||
return { ...lead, teams };
|
||||
}
|
||||
|
||||
const TASK_LIST_ARGS = { status: 'pending' } as const;
|
||||
const TASK_CREATE_ARGS = {
|
||||
subject: 'ship the fix',
|
||||
description: 'explain the change once',
|
||||
} as const;
|
||||
const TASK_UPDATE_ARGS = {
|
||||
taskId: '1',
|
||||
status: 'in_progress',
|
||||
owner: 'alice',
|
||||
} as const;
|
||||
|
||||
describe('hosted-session-command-session funnel (H4e-b3)', () => {
|
||||
it('commits the outbound planned entry and replays an identical restatement', async () => {
|
||||
const member = await memberSide();
|
||||
const commandId = commandIdOf('call-1');
|
||||
await member.commands.runCommand({
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
args: TASK_UPDATE_ARGS,
|
||||
executionCallId: 'prompt-1:call-1',
|
||||
closing: false,
|
||||
});
|
||||
const committed = member.commands.commands().at(-1)!;
|
||||
expect(committed.direction).toBe('outbound');
|
||||
expect(committed.kind).toBe('board_write');
|
||||
expect(committed.childRunId).toBe(LEAF_RUN);
|
||||
expect(committed.senderSessionId).toBe(CHILD);
|
||||
expect(committed.targetSessionId).toBeNull();
|
||||
expect(committed.run.delivery?.state).toBe('planned');
|
||||
await member.commands.runCommand({
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
args: TASK_UPDATE_ARGS,
|
||||
executionCallId: 'prompt-1:call-1',
|
||||
closing: false,
|
||||
});
|
||||
expect(member.commands.commands()).toHaveLength(1);
|
||||
await expect(
|
||||
member.commands.runCommand({
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
args: { ...TASK_UPDATE_ARGS, status: 'completed' },
|
||||
executionCallId: 'prompt-1:call-1',
|
||||
closing: false,
|
||||
}),
|
||||
).rejects.toThrow(ManagedSessionConflictError);
|
||||
});
|
||||
|
||||
it('refuses a command from a Session without a lineage', async () => {
|
||||
const memberless = await side(CHILD, undefined);
|
||||
await expect(
|
||||
memberless.commands.runCommand({
|
||||
commandId: commandIdOf('call-2'),
|
||||
kind: 'board_read',
|
||||
args: TASK_LIST_ARGS,
|
||||
executionCallId: 'prompt-1:call-2',
|
||||
closing: false,
|
||||
}),
|
||||
).rejects.toThrow(/Only a child Session can command its parent/);
|
||||
});
|
||||
|
||||
it('refuses a command from a closing Session', async () => {
|
||||
const member = await memberSide();
|
||||
await expect(
|
||||
member.commands.runCommand({
|
||||
commandId: commandIdOf('call-3'),
|
||||
kind: 'board_read',
|
||||
args: TASK_LIST_ARGS,
|
||||
executionCallId: 'prompt-1:call-3',
|
||||
closing: true,
|
||||
}),
|
||||
).rejects.toThrow(/closing and takes no more commands/);
|
||||
});
|
||||
|
||||
it('refuses arguments over the byte bound with byte_limit', async () => {
|
||||
const member = await memberSide();
|
||||
await expect(
|
||||
member.commands.runCommand({
|
||||
commandId: commandIdOf('call-4'),
|
||||
kind: 'board_write',
|
||||
args: { subject: 'x'.repeat(64 * 1024) },
|
||||
executionCallId: 'prompt-1:call-4',
|
||||
closing: false,
|
||||
}),
|
||||
).rejects.toThrow(/byte_limit/);
|
||||
});
|
||||
|
||||
it('moves the entry through handover, answered and consumed, each replayable', async () => {
|
||||
const member = await memberSide();
|
||||
const commandId = commandIdOf('call-5');
|
||||
await member.commands.runCommand({
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
args: TASK_CREATE_ARGS,
|
||||
executionCallId: 'prompt-1:call-5',
|
||||
closing: false,
|
||||
});
|
||||
await member.commands.handover(commandId, PARENT);
|
||||
expect(member.commands.command(commandId)?.run.delivery?.state).toBe(
|
||||
'accepting',
|
||||
);
|
||||
await member.commands.handover(commandId, PARENT);
|
||||
await expect(
|
||||
member.commands.handover(commandId, randomUUID()),
|
||||
).rejects.toThrow(ManagedSessionConflictError);
|
||||
|
||||
const reply = {
|
||||
outcome: 'answered' as const,
|
||||
text: 'Task #1 created: "ship the fix"',
|
||||
};
|
||||
await member.commands.answered(commandId, reply);
|
||||
expect(member.commands.command(commandId)?.run.delivery?.state).toBe(
|
||||
'accepted',
|
||||
);
|
||||
await member.commands.answered(commandId, reply);
|
||||
await expect(
|
||||
member.commands.answered(commandId, {
|
||||
outcome: 'refused',
|
||||
text: 'no',
|
||||
}),
|
||||
).rejects.toThrow(ManagedSessionConflictError);
|
||||
|
||||
expect(await member.commands.readReply(commandId)).toEqual(reply);
|
||||
await member.commands.commitConsumed(commandId);
|
||||
expect(member.commands.command(commandId)?.run.delivery?.state).toBe(
|
||||
'consumed',
|
||||
);
|
||||
});
|
||||
|
||||
it('settles only along the map: cancelled from planned, unknown and rejected from accepting', async () => {
|
||||
const member = await memberSide();
|
||||
const commandId = commandIdOf('call-6');
|
||||
await member.commands.runCommand({
|
||||
commandId,
|
||||
kind: 'board_read',
|
||||
args: TASK_LIST_ARGS,
|
||||
executionCallId: 'prompt-1:call-6',
|
||||
closing: false,
|
||||
});
|
||||
await member.commands.settle(commandId, 'cancelled');
|
||||
await expect(member.commands.handover(commandId, PARENT)).rejects.toThrow(
|
||||
ManagedSessionConflictError,
|
||||
);
|
||||
await expect(
|
||||
member.commands.answered(commandId, {
|
||||
outcome: 'answered',
|
||||
text: 'reached later',
|
||||
}),
|
||||
).rejects.toThrow(ManagedSessionConflictError);
|
||||
|
||||
const other = commandIdOf('call-7');
|
||||
await member.commands.runCommand({
|
||||
commandId: other,
|
||||
kind: 'board_write',
|
||||
args: TASK_CREATE_ARGS,
|
||||
executionCallId: 'prompt-1:call-7',
|
||||
closing: false,
|
||||
});
|
||||
await member.commands.handover(other, PARENT);
|
||||
await member.commands.settle(other, 'unknown');
|
||||
await member.commands.settle(other, 'rejected');
|
||||
const record = member.commands.command(other)!;
|
||||
expect(record.run.delivery?.state).toBe('rejected');
|
||||
});
|
||||
|
||||
it('enforces the per-run command budget with budget_exhausted', async () => {
|
||||
const member = await memberSide();
|
||||
const commandId = commandIdOf('call-8');
|
||||
await member.commands.runCommand({
|
||||
commandId,
|
||||
kind: 'board_read',
|
||||
args: TASK_LIST_ARGS,
|
||||
executionCallId: 'prompt-1:call-8',
|
||||
closing: false,
|
||||
});
|
||||
// Mint 256 entries by hand: a direct count of outbound records is the
|
||||
// rule's own evidence, so the suite plants the history instead of
|
||||
// driving 256 funnels calls.
|
||||
for (let index = 0; index < 255; index += 1) {
|
||||
await member.commands.runCommand({
|
||||
commandId: `cmd_extra_${index}`,
|
||||
kind: 'board_read',
|
||||
args: TASK_LIST_ARGS,
|
||||
executionCallId: `prompt-1:call-extra-${index}`,
|
||||
closing: false,
|
||||
});
|
||||
}
|
||||
await expect(
|
||||
member.commands.runCommand({
|
||||
commandId: commandIdOf('call-9'),
|
||||
kind: 'board_read',
|
||||
args: TASK_LIST_ARGS,
|
||||
executionCallId: 'prompt-1:call-9',
|
||||
closing: false,
|
||||
}),
|
||||
).rejects.toThrow(/budget_exhausted/);
|
||||
}, 30_000);
|
||||
|
||||
it("commits the leader's receipt against an attached run and refetches its reply", async () => {
|
||||
const lead = await leadSideSetup();
|
||||
const commandId = commandIdOf('call-10');
|
||||
const reply = {
|
||||
outcome: 'answered' as const,
|
||||
text: 'Task #1 created: "ship the fix"',
|
||||
};
|
||||
await lead.commands.commitInboundReceipt({
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
args: TASK_CREATE_ARGS,
|
||||
reply,
|
||||
});
|
||||
expect(await lead.commands.existingReply(commandId)).toEqual(reply);
|
||||
// Restating the same receipt replays silently.
|
||||
await lead.commands.commitInboundReceipt({
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
args: TASK_CREATE_ARGS,
|
||||
reply,
|
||||
});
|
||||
expect(lead.commands.commands()).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('executes a board write, commits the receipt, and answers a re-execution from it', async () => {
|
||||
const lead = await leadWithTeam();
|
||||
const commandId = commandIdOf('call-11');
|
||||
const execute = () =>
|
||||
lead.commands.execute(lead.teams, {
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
args: TASK_CREATE_ARGS,
|
||||
});
|
||||
const first = await execute();
|
||||
expect(first).toEqual({
|
||||
outcome: 'answered',
|
||||
text: 'Task #1 created: "ship the fix"',
|
||||
});
|
||||
expect(lead.commands.command(commandId)?.direction).toBe('inbound');
|
||||
// The receipt is the whole answer: a re-execution writes nothing more.
|
||||
expect(await execute()).toEqual(first);
|
||||
expect(
|
||||
lead.session.authority.extensionRecordsInDomain('team_task'),
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('executes a board update with a string task id and replays from it', async () => {
|
||||
const lead = await leadWithTeam();
|
||||
await lead.commands.execute(lead.teams, {
|
||||
commandId: commandIdOf('call-31'),
|
||||
kind: 'board_write',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
args: TASK_CREATE_ARGS,
|
||||
});
|
||||
const commandId = commandIdOf('call-32');
|
||||
const execute = () =>
|
||||
lead.commands.execute(lead.teams, {
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
// The member tool contract is a string id ("3" or "#3"): an
|
||||
// in-progress claim on the unowned task auto-claims her.
|
||||
args: { taskId: '1', status: 'in_progress' },
|
||||
});
|
||||
const first = await execute();
|
||||
expect(first).toEqual({
|
||||
outcome: 'answered',
|
||||
text: 'Task #1 updated (status: in_progress, owner: bob).',
|
||||
});
|
||||
expect(await execute()).toEqual(first);
|
||||
// The update revised the one task; a re-execution writes nothing more.
|
||||
expect(
|
||||
lead.session.authority.extensionRecordsInDomain('team_task'),
|
||||
).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("rejects a command whose sender is not the run's attached Session", async () => {
|
||||
const lead = await leadWithTeam();
|
||||
await expect(
|
||||
lead.commands.execute(lead.teams, {
|
||||
commandId: commandIdOf('call-12'),
|
||||
kind: 'board_write',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: randomUUID(),
|
||||
args: TASK_CREATE_ARGS,
|
||||
}),
|
||||
).rejects.toThrow(SessionCommandRefusalError);
|
||||
expect(
|
||||
lead.session.authority.extensionRecordsInDomain('team_task'),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('blocks a command whose run has not attached yet, committing nothing', async () => {
|
||||
const lead = await leadWithTeam();
|
||||
// The relay's execute can outrun the control plane's attach commit:
|
||||
// the run exists pre-attach.
|
||||
await lead.children.admit({
|
||||
completion: 'sent',
|
||||
childRunId: 'run-late',
|
||||
ownerScopeId: PARENT,
|
||||
rootSessionId: PARENT,
|
||||
description: 'join late',
|
||||
prompt: 'join late',
|
||||
definition: DEFINITION,
|
||||
workingDirectory: '.',
|
||||
workspaceMode: 'shared',
|
||||
executionCallId: 'run-late',
|
||||
} as never);
|
||||
await lead.children.dispatchStarted('run-late', {
|
||||
dispatchId: 'dispatch-run-late',
|
||||
runtime: { runtimeBindingId: 'binding-late', generation: '1' },
|
||||
});
|
||||
const commandId = commandIdOf('call-17');
|
||||
await expect(
|
||||
lead.commands.execute(lead.teams, {
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
childRunId: 'run-late',
|
||||
senderSessionId: randomUUID(),
|
||||
args: TASK_CREATE_ARGS,
|
||||
}),
|
||||
).rejects.toThrow(SessionCommandBlockedError);
|
||||
// A wait commits nothing: no receipt, no task.
|
||||
expect(lead.commands.command(commandId)).toBeUndefined();
|
||||
expect(
|
||||
lead.session.authority.extensionRecordsInDomain('team_task'),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('refuses a sender off the open team and commits the refused receipt', async () => {
|
||||
const lead = await leadWithTeam();
|
||||
const OUTSIDER = randomUUID();
|
||||
await lead.children.admit({
|
||||
completion: 'sent',
|
||||
childRunId: 'run-outsider',
|
||||
ownerScopeId: PARENT,
|
||||
rootSessionId: PARENT,
|
||||
description: 'watch the wall',
|
||||
prompt: 'watch',
|
||||
definition: DEFINITION,
|
||||
workingDirectory: '.',
|
||||
workspaceMode: 'shared',
|
||||
executionCallId: 'run-outsider',
|
||||
} as never);
|
||||
await lead.children.dispatchStarted('run-outsider', {
|
||||
dispatchId: 'dispatch-run-outsider',
|
||||
runtime: { runtimeBindingId: 'binding-2', generation: '1' },
|
||||
});
|
||||
await lead.children.attach('run-outsider', OUTSIDER);
|
||||
const commandId = commandIdOf('call-13');
|
||||
const reply = await lead.commands.execute(lead.teams, {
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
childRunId: 'run-outsider',
|
||||
senderSessionId: OUTSIDER,
|
||||
args: TASK_CREATE_ARGS,
|
||||
});
|
||||
expect(reply).toEqual({
|
||||
outcome: 'refused',
|
||||
text: 'This Session is not a member of an open team.',
|
||||
});
|
||||
const receipt = lead.commands.command(commandId);
|
||||
expect(receipt?.direction).toBe('inbound');
|
||||
expect(receipt?.run.delivery?.state).toBe('consumed');
|
||||
expect(
|
||||
lead.session.authority.extensionRecordsInDomain('team_task'),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('answers a board read for a running member without committing a receipt', async () => {
|
||||
const lead = await leadWithTeam();
|
||||
const commandId = commandIdOf('call-14');
|
||||
const reply = await lead.commands.execute(lead.teams, {
|
||||
commandId,
|
||||
kind: 'board_read',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
args: {},
|
||||
});
|
||||
expect(reply.outcome).toBe('answered');
|
||||
expect(reply.text).toContain('--- Team "review" members ---');
|
||||
expect(reply.text).toContain('bob: running');
|
||||
expect(lead.commands.command(commandId)).toBeUndefined();
|
||||
});
|
||||
|
||||
it('refuses the kinds a later batch owns and commits the refusal', async () => {
|
||||
const lead = await leadWithTeam();
|
||||
const commandId = commandIdOf('call-15');
|
||||
const reply = await lead.commands.execute(lead.teams, {
|
||||
commandId,
|
||||
kind: 'member_message',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
args: { to: 'alice', text: 'hi' },
|
||||
});
|
||||
expect(reply.outcome).toBe('refused');
|
||||
expect(reply.text).toContain('not accepted yet');
|
||||
expect(lead.commands.command(commandId)?.direction).toBe('inbound');
|
||||
});
|
||||
|
||||
it("answers a re-execution from the command's own committed writes", async () => {
|
||||
const lead = await leadWithTeam();
|
||||
const commandId = commandIdOf('call-16');
|
||||
// Half-applied: the derived create landed before any receipt did.
|
||||
await lead.teams.memberTaskCreate(commandId, TASK_CREATE_ARGS);
|
||||
const reply = await lead.commands.execute(lead.teams, {
|
||||
commandId,
|
||||
kind: 'board_write',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
args: TASK_CREATE_ARGS,
|
||||
});
|
||||
expect(reply).toEqual({
|
||||
outcome: 'answered',
|
||||
text: 'Task #1 created: "ship the fix"',
|
||||
});
|
||||
expect(
|
||||
lead.session.authority.extensionRecordsInDomain('team_task'),
|
||||
).toHaveLength(1);
|
||||
expect(lead.commands.command(commandId)?.direction).toBe('inbound');
|
||||
});
|
||||
|
||||
it('refuses a fresh command from a run that ended, with the fixed sentence', async () => {
|
||||
const lead = await leadWithTeam();
|
||||
await lead.children.settleCompleted(LEAF_RUN, {
|
||||
result: Buffer.from('done', 'utf8'),
|
||||
receipt: Buffer.from('{"outcome":"settled"}', 'utf8'),
|
||||
});
|
||||
const reply = await lead.commands.execute(lead.teams, {
|
||||
commandId: commandIdOf('call-17'),
|
||||
kind: 'board_write',
|
||||
childRunId: LEAF_RUN,
|
||||
senderSessionId: CHILD,
|
||||
args: TASK_CREATE_ARGS,
|
||||
});
|
||||
expect(reply).toEqual({
|
||||
outcome: 'refused',
|
||||
text: 'Your run has ended or is being stopped; it takes no more team commands.',
|
||||
});
|
||||
expect(
|
||||
lead.session.authority.extensionRecordsInDomain('team_task'),
|
||||
).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
680
packages/cli/src/serve/hosted-session-command-session.ts
Normal file
680
packages/cli/src/serve/hosted-session-command-session.ts
Normal file
|
|
@ -0,0 +1,680 @@
|
|||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { createHash } from 'node:crypto';
|
||||
import { isDeepStrictEqual } from 'node:util';
|
||||
import { ManagedSessionConflictError } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-authority.js';
|
||||
import {
|
||||
answeredCommandReply,
|
||||
inboundCommandBody,
|
||||
MANAGED_SESSION_COMMAND_RUNTIME_LIMITS,
|
||||
outboundCommandAnsweredBody,
|
||||
outboundCommandBody,
|
||||
outboundCommandConsumedBody,
|
||||
outboundCommandFinishedBody,
|
||||
outboundCommandHandoverBody,
|
||||
refusedCommandReply,
|
||||
sessionCommandReplyFromBytes,
|
||||
type SessionCommandReply,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-session-command-operations.js';
|
||||
import {
|
||||
MANAGED_SESSION_COMMAND_LIMITS,
|
||||
parseSessionCommand,
|
||||
type SessionCommand,
|
||||
type SessionCommandKind,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-session-command-record.js';
|
||||
import {
|
||||
isChildSessionRun,
|
||||
parseChildRun,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-child-run-record.js';
|
||||
import {
|
||||
ManagedSessionRecordError,
|
||||
type ManagedSessionKey,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-session-records.js';
|
||||
import {
|
||||
type HostedChildAgentSession,
|
||||
type HostedChildAgentStore,
|
||||
} from './hosted-child-agent-session.js';
|
||||
import type { HostedSessionLineage } from './hosted-session-message-session.js';
|
||||
import {
|
||||
HostedChildChains,
|
||||
hostedMemberView,
|
||||
hostedOpenTeamChainMember,
|
||||
} from './hosted-member-chain.js';
|
||||
import {
|
||||
HostedTeamRefusal,
|
||||
type HostedTeamSession,
|
||||
} from './hosted-team-session.js';
|
||||
|
||||
// H4e-b3 of #12827: the command channel a member Session takes to its
|
||||
// leader. The member commits its outbound entry and parks; the control
|
||||
// plane's message relay fixes the target at the handover (`handover`),
|
||||
// drives the leader's `execute`, and copies the reply back (`answered`).
|
||||
// The leader's `execute` runs on this funnel too: an already committed
|
||||
// receipt answers first, half-applied board writes re-validate or commit
|
||||
// their remaining places, and a fresh command gates on the member being
|
||||
// a live chain head of the open team. Every write rides the children
|
||||
// funnel's one writes chain. See
|
||||
// docs/design/2026-10-11-managed-agent-team-member-commands.md.
|
||||
|
||||
const ARGS_RESOURCE_KIND = 'managed-command-args';
|
||||
const REPLY_RESOURCE_KIND = 'managed-command-reply';
|
||||
|
||||
/**
|
||||
* Decision 4's timeout lock: once one command call of a Turn timed out,
|
||||
* every later command call of that Turn refuses at once, so a Turn never
|
||||
* holds the mount through a string of waits.
|
||||
*/
|
||||
export const SESSION_COMMAND_LEADER_UNANSWERING_TEXT =
|
||||
'The leader is not answering; end your turn.';
|
||||
|
||||
/**
|
||||
* Decision 14's member-facing answer of a command that ended without a
|
||||
* reply — the live wait's arm, the resumed wait and the interrupted-turn
|
||||
* settle all say exactly this, word for word.
|
||||
*/
|
||||
export function sessionCommandEvidenceText(
|
||||
state: 'planned' | 'accepting' | 'rejected' | 'cancelled' | 'unknown',
|
||||
): string {
|
||||
switch (state) {
|
||||
case 'planned':
|
||||
return 'The leader did not take the command before the wait ended; it was not sent and nothing happened. Send it again in a fresh turn.';
|
||||
case 'accepting':
|
||||
return 'The command reached the leader, but its reply did not arrive in time; a write may have taken effect. End your turn and say so in your result; read the board again before repeating the write.';
|
||||
case 'rejected':
|
||||
return 'The leader rejected the command; it was not run.';
|
||||
case 'cancelled':
|
||||
return 'The command was cancelled before it ran; it never left.';
|
||||
default:
|
||||
return "The command's outcome is unknown; the leader may have run it and a write may have taken effect. Read the board again before repeating the write.";
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The lead cannot accept the command at all — a lineage or evidence
|
||||
* mismatch. The route maps it to `session_command_refused`, and the relay
|
||||
* walks such an entry into `rejected`, never re-executing it.
|
||||
*/
|
||||
export class SessionCommandRefusalError extends Error {}
|
||||
|
||||
/**
|
||||
* The command cannot run yet: its sender's run exists but has not
|
||||
* attached. The relay's execute may outrun the control plane's attach
|
||||
* commit, so this is a wait, never a rejection — the route maps it to
|
||||
* `session_command_blocked` and the relay defers without an attempt.
|
||||
*/
|
||||
export class SessionCommandBlockedError extends Error {}
|
||||
|
||||
function digest(record: unknown): string {
|
||||
return createHash('sha256').update(JSON.stringify(record)).digest('hex');
|
||||
}
|
||||
|
||||
/**
|
||||
* A reply is at most 64 KiB as stored JSON; a longer task list or board
|
||||
* answer is shortened with a marker rather than refused. Leaves enough
|
||||
* headroom for the reply's own `outcome`/`text` envelope.
|
||||
*/
|
||||
function cutCommandReplyText(text: string): string {
|
||||
const cap = MANAGED_SESSION_COMMAND_LIMITS.maxReplyBytes - 512;
|
||||
if (Buffer.byteLength(text, 'utf8') <= cap) return text;
|
||||
const marker = '\n… (the reply grew past its size limit and was cut)';
|
||||
const budget = cap - Buffer.byteLength(marker, 'utf8');
|
||||
const cut = new TextDecoder()
|
||||
.decode(Buffer.from(text, 'utf8').subarray(0, budget))
|
||||
.replace(/\uFFFD+$/, '');
|
||||
return cut + marker;
|
||||
}
|
||||
|
||||
export class HostedSessionCommandSession {
|
||||
constructor(
|
||||
private readonly store: HostedChildAgentStore,
|
||||
private readonly key: ManagedSessionKey,
|
||||
private readonly children: HostedChildAgentSession,
|
||||
/** Present exactly on a child Session, which may command its parent. */
|
||||
readonly lineage: HostedSessionLineage | undefined,
|
||||
) {}
|
||||
|
||||
/** The last committed body of one command chain in this journal. */
|
||||
command(commandId: string): SessionCommand | undefined {
|
||||
const existing = this.store.authority.extensionRecord(
|
||||
'session_command',
|
||||
commandId,
|
||||
);
|
||||
return existing ? parseSessionCommand(existing.record) : undefined;
|
||||
}
|
||||
|
||||
/** Every command chain in this journal. */
|
||||
commands(): readonly SessionCommand[] {
|
||||
return this.store.authority
|
||||
.extensionRecordsInDomain('session_command')
|
||||
.map((entry) => parseSessionCommand(entry.record));
|
||||
}
|
||||
|
||||
/**
|
||||
* The wait bound's anchor (decision 4): when this journal committed the
|
||||
* command's outbound entry, read from the commit event itself so a
|
||||
* resumed wait measures what is left, never a fresh bound. The store's
|
||||
* narrowed view stops at commits; the anchor is a journal event read,
|
||||
* taken through the authority it actually is.
|
||||
*/
|
||||
outboundCommittedAt(commandId: string): number | undefined {
|
||||
const authority = this.store.authority as unknown as {
|
||||
eventsInSequenceRange(
|
||||
from: number,
|
||||
through: number,
|
||||
): ReadonlyArray<{
|
||||
readonly kind: string;
|
||||
readonly occurredAt: number;
|
||||
readonly payload: Record<string, unknown>;
|
||||
}>;
|
||||
readonly committedSequence: number;
|
||||
};
|
||||
for (const event of authority.eventsInSequenceRange(
|
||||
1,
|
||||
authority.committedSequence,
|
||||
)) {
|
||||
if (
|
||||
event.kind === 'domain.committed' &&
|
||||
event.payload['domain'] === 'session_command' &&
|
||||
event.payload['operationId'] === `${commandId}:1`
|
||||
) {
|
||||
return event.occurredAt;
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* A member's board read/write: the outbound `planned` entry, committed
|
||||
* before the call begins waiting, so recovery re-enters it even when
|
||||
* the Turn crashes right after. A re-driven call names the same
|
||||
* `commandId` with the same kind and arguments and replays.
|
||||
*/
|
||||
async runCommand(params: {
|
||||
readonly commandId: string;
|
||||
readonly kind: SessionCommandKind;
|
||||
readonly args: Record<string, unknown>;
|
||||
readonly executionCallId: string;
|
||||
/** The member's Session is closing: no more commands. */
|
||||
readonly closing: boolean;
|
||||
}): Promise<void> {
|
||||
const lineage = this.lineage;
|
||||
if (lineage === undefined) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'Only a child Session can command its parent.',
|
||||
);
|
||||
}
|
||||
const argsBytes = Buffer.from(JSON.stringify(params.args), 'utf8');
|
||||
if (argsBytes.byteLength > MANAGED_SESSION_COMMAND_LIMITS.maxArgsBytes) {
|
||||
throw new ManagedSessionRecordError(
|
||||
`Session command arguments exceed ${MANAGED_SESSION_COMMAND_LIMITS.maxArgsBytes} bytes (byte_limit).`,
|
||||
);
|
||||
}
|
||||
await this.children.serialized(async () => {
|
||||
const sent = this.command(params.commandId);
|
||||
if (sent !== undefined) {
|
||||
if (
|
||||
sent.direction !== 'outbound' ||
|
||||
sent.kind !== params.kind ||
|
||||
sent.argsDigest !== digest(params.args)
|
||||
) {
|
||||
throw new ManagedSessionConflictError(
|
||||
`Session command ${params.commandId} was sent with different evidence.`,
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (params.closing) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'This Session is closing and takes no more commands.',
|
||||
);
|
||||
}
|
||||
const volume = this.commands().filter(
|
||||
(command) =>
|
||||
command.direction === 'outbound' &&
|
||||
command.childRunId === lineage.parentChildRunId,
|
||||
).length;
|
||||
if (volume >= MANAGED_SESSION_COMMAND_RUNTIME_LIMITS.maxPerRun) {
|
||||
throw new ManagedSessionRecordError(
|
||||
`This Session has issued ${MANAGED_SESSION_COMMAND_RUNTIME_LIMITS.maxPerRun} commands on this run (budget_exhausted).`,
|
||||
);
|
||||
}
|
||||
const argsRef = await this.store.resources.publish(
|
||||
ARGS_RESOURCE_KIND,
|
||||
argsBytes,
|
||||
);
|
||||
await this.commit(
|
||||
outboundCommandBody({
|
||||
commandId: params.commandId,
|
||||
kind: params.kind,
|
||||
childRunId: lineage.parentChildRunId,
|
||||
senderSessionId: this.key.sessionId,
|
||||
argsRef,
|
||||
executionCallId: params.executionCallId,
|
||||
}),
|
||||
`${params.commandId}:1`,
|
||||
'sendSessionCommand',
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/** The relay fixed the target: planned → accepting, once and identically. */
|
||||
async handover(commandId: string, targetSessionId: string): Promise<void> {
|
||||
if (
|
||||
this.lineage !== undefined &&
|
||||
targetSessionId !== this.lineage.parentSessionId
|
||||
) {
|
||||
throw new ManagedSessionConflictError(
|
||||
`Session command ${commandId} names ${targetSessionId}, not this Session's parent.`,
|
||||
);
|
||||
}
|
||||
return this.revise(commandId, (previous) => {
|
||||
const state = previous.run.delivery?.state;
|
||||
if (state === 'planned') {
|
||||
return outboundCommandHandoverBody(previous, targetSessionId);
|
||||
}
|
||||
if (
|
||||
previous.targetSessionId === targetSessionId &&
|
||||
state !== 'cancelled'
|
||||
) {
|
||||
return previous;
|
||||
}
|
||||
throw new ManagedSessionConflictError(
|
||||
`Session command ${commandId} cannot be handed to ${targetSessionId} from ${state}.`,
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The relay copied the leader's reply back: the outbound moves to
|
||||
* `accepted` with the reply pair. Checked one byte at a time so the
|
||||
* stored evidence is the leader's reply, never the relay's paraphrase.
|
||||
*/
|
||||
async answered(commandId: string, reply: SessionCommandReply): Promise<void> {
|
||||
const replyBytes = Buffer.from(JSON.stringify(reply), 'utf8');
|
||||
if (replyBytes.byteLength > MANAGED_SESSION_COMMAND_LIMITS.maxReplyBytes) {
|
||||
throw new SessionCommandRefusalError(
|
||||
`Session command reply exceeds ${MANAGED_SESSION_COMMAND_LIMITS.maxReplyBytes} bytes (byte_limit).`,
|
||||
);
|
||||
}
|
||||
const replyDigest = createHash('sha256').update(replyBytes).digest('hex');
|
||||
await this.children.serialized(async () => {
|
||||
const existing = this.store.authority.extensionRecord(
|
||||
'session_command',
|
||||
commandId,
|
||||
);
|
||||
if (existing === undefined) {
|
||||
throw new ManagedSessionRecordError(
|
||||
`Session command ${commandId} has no record to revise.`,
|
||||
);
|
||||
}
|
||||
const previous = parseSessionCommand(existing.record);
|
||||
const state = previous.run.delivery?.state;
|
||||
if (
|
||||
(state === 'accepted' || state === 'consumed') &&
|
||||
previous.replyDigest === replyDigest
|
||||
) {
|
||||
return;
|
||||
}
|
||||
if (state !== 'accepting' && state !== 'unknown') {
|
||||
throw new ManagedSessionConflictError(
|
||||
`Session command ${commandId} cannot be answered from ${state}.`,
|
||||
);
|
||||
}
|
||||
const replyRef = await this.store.resources.publish(
|
||||
REPLY_RESOURCE_KIND,
|
||||
replyBytes,
|
||||
);
|
||||
await this.commit(
|
||||
outboundCommandAnsweredBody(previous, replyRef),
|
||||
`${commandId}:${existing.revision + 1}`,
|
||||
'commitSessionCommand',
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The outbox entry's terminal step: `rejected`, `unknown`, or
|
||||
* `cancelled` at the relay's give-up or the tool turn's timeout
|
||||
* withdrawal. `cancelled` happens only before the handover; `unknown`
|
||||
* and `rejected` only after.
|
||||
*/
|
||||
settle(
|
||||
commandId: string,
|
||||
state: 'cancelled' | 'rejected' | 'unknown',
|
||||
): Promise<void> {
|
||||
const from: Record<typeof state, readonly string[]> = {
|
||||
cancelled: ['planned'],
|
||||
rejected: ['accepting', 'unknown'],
|
||||
unknown: ['accepting'],
|
||||
};
|
||||
return this.revise(commandId, (previous) => {
|
||||
const current = previous.run.delivery?.state ?? '';
|
||||
if (current === state) return previous;
|
||||
if (!from[state].includes(current)) {
|
||||
throw new ManagedSessionConflictError(
|
||||
`Session command ${commandId} cannot reach ${state} from ${current}.`,
|
||||
);
|
||||
}
|
||||
return outboundCommandFinishedBody(previous, state);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The parked call read the reply: the outbox entry is consumed, ending
|
||||
* the command for everyone. Calls that ended by timeout, refusal or
|
||||
* give-up never call this.
|
||||
*/
|
||||
commitConsumed(commandId: string): Promise<void> {
|
||||
return this.revise(commandId, (previous) => {
|
||||
const state = previous.run.delivery?.state;
|
||||
if (state === 'accepted') return outboundCommandConsumedBody(previous);
|
||||
if (state === 'consumed') return previous;
|
||||
throw new ManagedSessionConflictError(
|
||||
`Session command ${commandId} cannot be consumed from ${state}.`,
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The parked call's view: the reply when `answered` landed, null while
|
||||
* the relay owes one or while the wait ended by timeout, refusal or
|
||||
* give-up.
|
||||
*/
|
||||
async readReply(commandId: string): Promise<SessionCommandReply | null> {
|
||||
const command = this.command(commandId);
|
||||
if (command === undefined || command.direction !== 'outbound') {
|
||||
return null;
|
||||
}
|
||||
const state = command.run.delivery?.state;
|
||||
if (
|
||||
(state !== 'accepted' && state !== 'consumed') ||
|
||||
command.replyRef === null
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return sessionCommandReplyFromBytes(
|
||||
await this.store.resources.read(command.replyRef),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The leader's receipt for an executing command, committed after the
|
||||
* run completes. `execute`'s idempotence anchor: a later `execute`
|
||||
* finds it and refetches the stored reply rather than recommitting.
|
||||
*/
|
||||
async commitInboundReceipt(params: {
|
||||
readonly commandId: string;
|
||||
readonly kind: Exclude<SessionCommandKind, 'board_read'>;
|
||||
readonly childRunId: string;
|
||||
readonly senderSessionId: string;
|
||||
readonly args: Record<string, unknown>;
|
||||
readonly reply: SessionCommandReply;
|
||||
}): Promise<void> {
|
||||
await this.children.serialized(async () => {
|
||||
const existing = this.command(params.commandId);
|
||||
if (existing !== undefined) {
|
||||
if (
|
||||
existing.direction !== 'inbound' ||
|
||||
existing.kind !== params.kind ||
|
||||
existing.replyDigest !== digest(params.reply)
|
||||
) {
|
||||
throw new ManagedSessionConflictError(
|
||||
`Session command ${params.commandId} was answered with different evidence.`,
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
const argsRef = await this.store.resources.publish(
|
||||
ARGS_RESOURCE_KIND,
|
||||
Buffer.from(JSON.stringify(params.args), 'utf8'),
|
||||
);
|
||||
const replyRef = await this.store.resources.publish(
|
||||
REPLY_RESOURCE_KIND,
|
||||
Buffer.from(JSON.stringify(params.reply), 'utf8'),
|
||||
);
|
||||
await this.commit(
|
||||
inboundCommandBody({
|
||||
commandId: params.commandId,
|
||||
kind: params.kind,
|
||||
childRunId: params.childRunId,
|
||||
senderSessionId: params.senderSessionId,
|
||||
targetSessionId: this.key.sessionId,
|
||||
argsRef,
|
||||
replyRef,
|
||||
}),
|
||||
`${params.commandId}:receipt`,
|
||||
'commitSessionCommandReceipt',
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* `execute`'s first check: an inbound receipt for `commandId` exists and
|
||||
* its stored reply is the answer, again.
|
||||
*/
|
||||
async existingReply(
|
||||
commandId: string,
|
||||
): Promise<SessionCommandReply | undefined> {
|
||||
const receipt = this.command(commandId);
|
||||
if (
|
||||
receipt === undefined ||
|
||||
receipt.direction !== 'inbound' ||
|
||||
receipt.replyRef === null
|
||||
) {
|
||||
return undefined;
|
||||
}
|
||||
return sessionCommandReplyFromBytes(
|
||||
await this.store.resources.read(receipt.replyRef),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision 3: the lead's execution of one command, run on this lead
|
||||
* Session's funnels with no model Turn and no lineage. Its answer is
|
||||
* derived in three steps, each replay-stable:
|
||||
*
|
||||
* 1. a committed inbound receipt answers again, stored byte for byte;
|
||||
* 2. otherwise any board write the command itself committed (its derived
|
||||
* ids) is found, and what remains re-validates against the current
|
||||
* board — places that no longer fit are named "Not committed";
|
||||
* 3. otherwise the fresh admission gates: the sender must be a member
|
||||
* of an open team, on the running, unstopped head of its chain.
|
||||
*
|
||||
* A board read commits no receipt: it changes nothing and a re-run may
|
||||
* answer a newer board. A board write commits its receipt under the
|
||||
* same commandId, refused replies included. `member_message` and
|
||||
* `plan_submit` belong to later batches and are refused deterministically.
|
||||
*/
|
||||
async execute(
|
||||
teams: HostedTeamSession,
|
||||
params: {
|
||||
readonly commandId: string;
|
||||
readonly kind: SessionCommandKind;
|
||||
readonly childRunId: string;
|
||||
readonly senderSessionId: string;
|
||||
readonly args: Record<string, unknown>;
|
||||
},
|
||||
): Promise<SessionCommandReply> {
|
||||
const existing = await this.existingReply(params.commandId);
|
||||
if (existing !== undefined) return existing;
|
||||
// The H4d-b inbound rule: the run must be this Session's own child
|
||||
// run, attached (once) to the sender. A relay that names anything
|
||||
// else has its own bookkeeping bug, so the command is rejected, never
|
||||
// run. A run that attached and since settled still proves its sender —
|
||||
// its fresh commands fail the running gate below instead.
|
||||
const runRecord = this.store.authority.extensionRecord(
|
||||
'child_run',
|
||||
params.childRunId,
|
||||
);
|
||||
const parsed =
|
||||
runRecord === undefined ? undefined : parseChildRun(runRecord.record);
|
||||
const child =
|
||||
parsed !== undefined && isChildSessionRun(parsed) ? parsed : undefined;
|
||||
if (child !== undefined && child.childSessionId === null) {
|
||||
throw new SessionCommandBlockedError(
|
||||
`Session command ${params.commandId} names a run that has not attached yet.`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
child === undefined ||
|
||||
child.childSessionId !== params.senderSessionId
|
||||
) {
|
||||
throw new SessionCommandRefusalError(
|
||||
`Session command ${params.commandId} names no run of this Session attached to ${params.senderSessionId}.`,
|
||||
);
|
||||
}
|
||||
const receipt = async (
|
||||
reply: SessionCommandReply,
|
||||
): Promise<SessionCommandReply> => {
|
||||
if (params.kind !== 'board_read') {
|
||||
await this.commitInboundReceipt({
|
||||
commandId: params.commandId,
|
||||
kind: params.kind as Exclude<SessionCommandKind, 'board_read'>,
|
||||
childRunId: params.childRunId,
|
||||
senderSessionId: params.senderSessionId,
|
||||
args: params.args,
|
||||
reply,
|
||||
});
|
||||
}
|
||||
return reply;
|
||||
};
|
||||
if (params.kind === 'member_message' || params.kind === 'plan_submit') {
|
||||
return receipt(
|
||||
refusedCommandReply(
|
||||
'This team command is not accepted yet: members cannot send it to their leader.',
|
||||
),
|
||||
);
|
||||
}
|
||||
const chains = new HostedChildChains(
|
||||
this.store.authority.extensionRecordsInDomain('child_run'),
|
||||
);
|
||||
const membership = hostedOpenTeamChainMember(
|
||||
this.store.authority.extensionRecordsInDomain('team_state'),
|
||||
chains,
|
||||
params.childRunId,
|
||||
);
|
||||
const runsNow = (() => {
|
||||
if (membership === undefined) return false;
|
||||
const view = hostedMemberView(chains, membership.rootChildRunId);
|
||||
return (
|
||||
view.head?.childRunId === params.childRunId &&
|
||||
view.state === 'running' &&
|
||||
!view.stopping
|
||||
);
|
||||
})();
|
||||
if (params.kind === 'board_read') {
|
||||
if (membership === undefined) {
|
||||
return refusedCommandReply(
|
||||
'This Session is not a member of an open team.',
|
||||
);
|
||||
}
|
||||
if (!runsNow) {
|
||||
return refusedCommandReply(
|
||||
'Your run has ended or is being stopped; it takes no more team commands.',
|
||||
);
|
||||
}
|
||||
return answeredCommandReply(
|
||||
cutCommandReplyText(
|
||||
await teams.memberBoardList(membership.name, params.args),
|
||||
),
|
||||
);
|
||||
}
|
||||
// board_write: `taskId` arrives as a string ("3" or "#3") per the
|
||||
// member tool schema — its presence discriminates the update against
|
||||
// the create. An update writes one revision per `[self, ...blocks]`,
|
||||
// each under `${commandId}:${place + 1}` as the funnel does, so the
|
||||
// half-applied probe carries exactly that many landings.
|
||||
const updating =
|
||||
typeof params.args['taskId'] === 'string' &&
|
||||
params.args['taskId'].trim().length > 0;
|
||||
const placeCount = (value: unknown): number =>
|
||||
Array.isArray(value) &&
|
||||
value.every((each) => typeof each === 'string' && each.trim().length > 0)
|
||||
? value.length
|
||||
: 0;
|
||||
const derivedCommitted = updating
|
||||
? Array.from(
|
||||
{ length: 1 + placeCount(params.args['addBlocks']) },
|
||||
(_, index) => `${params.commandId}:${index + 1}`,
|
||||
).some((recordId) => teams.commandWriteCommitted(recordId))
|
||||
: teams.commandWriteCommitted(`${params.commandId}:create`);
|
||||
if (!derivedCommitted) {
|
||||
if (membership === undefined) {
|
||||
return receipt(
|
||||
refusedCommandReply('This Session is not a member of an open team.'),
|
||||
);
|
||||
}
|
||||
if (!runsNow) {
|
||||
return receipt(
|
||||
refusedCommandReply(
|
||||
'Your run has ended or is being stopped; it takes no more team commands.',
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
try {
|
||||
const text = updating
|
||||
? await teams.memberTaskUpdate(
|
||||
membership?.name ?? '',
|
||||
params.commandId,
|
||||
params.args,
|
||||
)
|
||||
: await teams.memberTaskCreate(params.commandId, params.args);
|
||||
return receipt(answeredCommandReply(cutCommandReplyText(text)));
|
||||
} catch (cause) {
|
||||
if (cause instanceof HostedTeamRefusal) {
|
||||
return receipt(refusedCommandReply(cutCommandReplyText(cause.message)));
|
||||
}
|
||||
throw cause;
|
||||
}
|
||||
}
|
||||
|
||||
private revise(
|
||||
commandId: string,
|
||||
step: (previous: SessionCommand) => SessionCommand,
|
||||
): Promise<void> {
|
||||
return this.children.serialized(async () => {
|
||||
const existing = this.store.authority.extensionRecord(
|
||||
'session_command',
|
||||
commandId,
|
||||
);
|
||||
if (existing === undefined) {
|
||||
throw new ManagedSessionRecordError(
|
||||
`Session command ${commandId} has no record to revise.`,
|
||||
);
|
||||
}
|
||||
const previous = parseSessionCommand(existing.record);
|
||||
const next = step(previous);
|
||||
if (isDeepStrictEqual(previous, next)) return;
|
||||
await this.commit(
|
||||
next,
|
||||
`${commandId}:${existing.revision + 1}`,
|
||||
'commitSessionCommand',
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
private async commit(
|
||||
record: SessionCommand,
|
||||
commandId: string,
|
||||
operation: string,
|
||||
): Promise<void> {
|
||||
await this.store.authority.commitExtensionRecord(
|
||||
{
|
||||
operation,
|
||||
commandId,
|
||||
sessionKey: this.key,
|
||||
contentDigest: digest(record),
|
||||
},
|
||||
{
|
||||
domain: 'session_command',
|
||||
record,
|
||||
},
|
||||
{ class: 'trusted_entry' },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -940,6 +940,118 @@ describe('the board', () => {
|
|||
).toContain('Cannot filter by owner');
|
||||
});
|
||||
|
||||
// A well-formed number with no task is a refusal, never a crash:
|
||||
// existence is proved before the ownership probe reads the task.
|
||||
it('refuses a well-formed addBlocks target that does not exist instead of crashing', async () => {
|
||||
await createTeam();
|
||||
await addMember('alice');
|
||||
await teams.run(
|
||||
'task_create',
|
||||
{ subject: 'one', description: 'do one' },
|
||||
'prompt:t1',
|
||||
);
|
||||
expect(
|
||||
await refusal(
|
||||
teams.memberTaskUpdate('alice', 'cmd_missing', {
|
||||
taskId: '1',
|
||||
addBlocks: ['999'],
|
||||
}),
|
||||
),
|
||||
).toBe('Cannot update task #1: referenced task #999 not found.');
|
||||
});
|
||||
|
||||
// A partially committed task_update replays its remaining places
|
||||
// against the board as it stands now: the leader legally made task 2
|
||||
// block task 1 after the subject place committed, so the replayed
|
||||
// edge would close a cycle — it rides the receipt, never the fault.
|
||||
it('replays the remaining addBlocks places against the board as it stands now', async () => {
|
||||
await createTeam();
|
||||
await addMember('alice');
|
||||
await teams.run(
|
||||
'task_create',
|
||||
{ subject: 'one', description: 'do one' },
|
||||
'prompt:t1',
|
||||
);
|
||||
await teams.run(
|
||||
'task_create',
|
||||
{ subject: 'two', description: 'do two' },
|
||||
'prompt:t2',
|
||||
);
|
||||
// The original call committed only the subject place.
|
||||
await teams.memberTaskUpdate('alice', 'cmd_board', {
|
||||
taskId: '1',
|
||||
subject: 'renamed',
|
||||
});
|
||||
await teams.run(
|
||||
'task_update',
|
||||
{ taskId: '1', addBlockedBy: ['2'] },
|
||||
'prompt:leader-edge',
|
||||
);
|
||||
const result = await teams.memberTaskUpdate('alice', 'cmd_board', {
|
||||
taskId: '1',
|
||||
subject: 'renamed',
|
||||
addBlocks: ['2'],
|
||||
});
|
||||
expect(result).toContain('Not committed');
|
||||
expect(result).toContain('dependency cycle');
|
||||
const two = parseTeamTask(
|
||||
session.authority.extensionRecord('team_task', 'prompt:call-team#2')!
|
||||
.record,
|
||||
);
|
||||
expect(two.blockedBy).toHaveLength(0);
|
||||
});
|
||||
|
||||
// The blocker-count limit follows the same replay rule: the remaining
|
||||
// place is validated against the current board, so a crowded target
|
||||
// gets its Not-committed entry instead of a fault escaping execute.
|
||||
it('replays a remaining addBlocks place by its blocker cap against the current board', async () => {
|
||||
await createTeam();
|
||||
await addMember('alice');
|
||||
await teams.run(
|
||||
'task_create',
|
||||
{ subject: 'one', description: 'do one' },
|
||||
'prompt:t1',
|
||||
);
|
||||
await teams.run(
|
||||
'task_create',
|
||||
{ subject: 'two', description: 'do two' },
|
||||
'prompt:t2',
|
||||
);
|
||||
// The original call committed only the subject place.
|
||||
await teams.memberTaskUpdate('alice', 'cmd_cap', {
|
||||
taskId: '1',
|
||||
subject: 'renamed',
|
||||
});
|
||||
// The leader filled task 2 with the maximum since: 64 blockers.
|
||||
const blockers: string[] = [];
|
||||
for (let index = 0; index < 64; index += 1) {
|
||||
const number = index + 3;
|
||||
blockers.push(`${number}`);
|
||||
await teams.run(
|
||||
'task_create',
|
||||
{ subject: `filler-${number}`, description: 'f' },
|
||||
`prompt:cap-${number}`,
|
||||
);
|
||||
}
|
||||
await teams.run(
|
||||
'task_update',
|
||||
{ taskId: '2', addBlockedBy: blockers },
|
||||
'prompt:cap-fill',
|
||||
);
|
||||
const result = await teams.memberTaskUpdate('alice', 'cmd_cap', {
|
||||
taskId: '1',
|
||||
subject: 'renamed',
|
||||
addBlocks: ['2'],
|
||||
});
|
||||
expect(result).toContain('Not committed');
|
||||
expect(result).toContain('would exceed 64 blockers');
|
||||
const two = parseTeamTask(
|
||||
session.authority.extensionRecord('team_task', 'prompt:call-team#2')!
|
||||
.record,
|
||||
);
|
||||
expect(two.blockedBy).toHaveLength(64);
|
||||
});
|
||||
|
||||
it('opens new tasks only in an active team', async () => {
|
||||
expect(
|
||||
await refusal(
|
||||
|
|
|
|||
|
|
@ -121,10 +121,10 @@ export interface HostedTeamStops {
|
|||
}
|
||||
|
||||
/** The Legacy board tools' schema bounds, in characters. */
|
||||
const SUBJECT_CHARS = 200;
|
||||
const DESCRIPTION_CHARS = 10_000;
|
||||
const ACTIVE_FORM_CHARS = 200;
|
||||
const TASK_STATUSES = ['pending', 'in_progress', 'completed', 'deleted'];
|
||||
export const SUBJECT_CHARS = 200;
|
||||
export const DESCRIPTION_CHARS = 10_000;
|
||||
export const ACTIVE_FORM_CHARS = 200;
|
||||
export const TASK_STATUSES = ['pending', 'in_progress', 'completed', 'deleted'];
|
||||
const UNSAFE_METADATA_KEYS = ['__proto__', 'constructor', 'prototype'];
|
||||
const TRUSTED: ManagedSessionActor = { class: 'trusted_entry' };
|
||||
|
||||
|
|
@ -599,6 +599,12 @@ export class HostedTeamSession {
|
|||
throw new HostedTeamRefusal(
|
||||
`"${MANAGED_TEAM_LEADER}" is reserved for the team leader. Choose a different teammate name.`,
|
||||
);
|
||||
// H4e-b3 decision 7: `parent` is how a member addresses its leader, so
|
||||
// no member may take the word other members address their leader with.
|
||||
if (name === 'parent')
|
||||
throw new HostedTeamRefusal(
|
||||
`"parent" is reserved for the Session that accepts the members' messages. Choose a different teammate name.`,
|
||||
);
|
||||
if (name.length > MANAGED_TEAM_LIMITS.maxNameLength)
|
||||
throw new HostedTeamRefusal(
|
||||
`Teammate name "${name}" is longer than ${MANAGED_TEAM_LIMITS.maxNameLength} characters.`,
|
||||
|
|
@ -1272,6 +1278,466 @@ export class HostedTeamSession {
|
|||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b3: the member's view of the board for `task_list`. The board
|
||||
* carries everything the lead's view does, minus the child task ids
|
||||
* behind each roster name — a member reads names, states and task
|
||||
* numbers, never the run another member is attached to.
|
||||
*/
|
||||
async memberBoardList(
|
||||
_memberName: string,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<string> {
|
||||
return this.serial(async () => this.memberListIn(args));
|
||||
}
|
||||
|
||||
private memberListIn(args: Record<string, unknown>): string {
|
||||
const team = this.openTeam();
|
||||
if (team === undefined)
|
||||
throw new HostedTeamRefusal(
|
||||
'You are not a member of an open team. Ask the leader how the team ended.',
|
||||
);
|
||||
let owner: string | undefined;
|
||||
if (typeof args['owner'] === 'string' && args['owner'].trim()) {
|
||||
owner = sanitizeName(args['owner']);
|
||||
if (!owner)
|
||||
throw new HostedTeamRefusal(
|
||||
'Cannot filter by owner: owner must include at least one letter, number, or hyphen.',
|
||||
);
|
||||
}
|
||||
const tasks = this.tasksOf(team.teamId);
|
||||
const byId = new Map(tasks.map((task) => [task.taskId, task]));
|
||||
const blocker =
|
||||
typeof args['blockedBy'] === 'string' && args['blockedBy'].trim()
|
||||
? `${team.teamId}#${taskNumber(args['blockedBy'])}`
|
||||
: undefined;
|
||||
const lines = tasks
|
||||
.filter(
|
||||
(task) =>
|
||||
task.status !== 'deleted' &&
|
||||
(args['status'] === undefined || task.status === args['status']) &&
|
||||
(owner === undefined || task.owner === owner) &&
|
||||
(blocker === undefined ||
|
||||
teamTaskOpenBlockers(task, byId).includes(blocker)),
|
||||
)
|
||||
.map((task) => {
|
||||
const open = teamTaskOpenBlockers(task, byId).map(
|
||||
(each) => `#${byId.get(each)!.number}`,
|
||||
);
|
||||
return (
|
||||
`#${task.number} [${task.status}]` +
|
||||
(task.owner ? ` @${task.owner}` : '') +
|
||||
` — ${task.subject}` +
|
||||
(open.length > 0 ? ` (blocked by ${open.join(', ')})` : '')
|
||||
);
|
||||
});
|
||||
const chains = this.chains();
|
||||
return [
|
||||
...(lines.length > 0 ? lines : ['No tasks found.']),
|
||||
'',
|
||||
`--- Team "${team.name}" members ---`,
|
||||
...(team.members.length > 0
|
||||
? team.members.map((member) => this.memberRosterLine(member, chains))
|
||||
: ['No members yet.']),
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
/** The roster line a member sees: state only, never the child task id. */
|
||||
private memberRosterLine(
|
||||
member: { readonly name: string; readonly childRunId: string },
|
||||
chains: HostedChildChains,
|
||||
): string {
|
||||
const view = this.memberView(member.childRunId, chains);
|
||||
const head = view.head;
|
||||
const label = view.stopping
|
||||
? 'running (stopping)'
|
||||
: view.stopRequestedCompletion
|
||||
? 'completed (stop requested)'
|
||||
: view.state;
|
||||
const reason =
|
||||
(view.state === 'failed' || view.state === 'cancelled') && head
|
||||
? head.stopReason
|
||||
: null;
|
||||
const note =
|
||||
view.neverStarted === undefined ? '' : ` (its latest task did not start)`;
|
||||
return `${member.name}: ${label}` + (reason ? ` (${reason})` : '') + note;
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b3 decision 6: a member's `task_create`. One write, replay-stable
|
||||
* under `${commandId}:create` so a TaskCreateCommand re-run of
|
||||
* `execute` finds it and takes no second task. Refuses on a closing
|
||||
* team or any rule violation, with the member-facing texts.
|
||||
*/
|
||||
async memberTaskCreate(
|
||||
commandId: string,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<string> {
|
||||
return this.serial(async () => {
|
||||
const created = this.committed('createTeamTask', `${commandId}:create`);
|
||||
if (created !== undefined) {
|
||||
const task = this.task(created);
|
||||
return `Task #${task.number} created: "${task.subject}"`;
|
||||
}
|
||||
const team = this.openTeam();
|
||||
if (team === undefined)
|
||||
throw new HostedTeamRefusal(
|
||||
'You are not a member of an open team. Ask the leader how the team ended.',
|
||||
);
|
||||
if (team.lifecycle !== 'active')
|
||||
throw new HostedTeamRefusal(
|
||||
`Team "${team.name}" is closing and takes no new tasks.`,
|
||||
);
|
||||
const number =
|
||||
Math.max(0, ...this.tasksOf(team.teamId).map((task) => task.number)) +
|
||||
1;
|
||||
const metadata = metadataBytes(mergeMetadata({}, args['metadata']));
|
||||
const task = teamTaskOpenBody({
|
||||
teamId: team.teamId,
|
||||
taskId: `${team.teamId}#${number}`,
|
||||
number,
|
||||
subject: args['subject'] as string,
|
||||
descriptionRef: await this.publishText(args['description'] as string),
|
||||
activeForm: (args['activeForm'] as string | undefined) ?? null,
|
||||
metadataRef: await this.publishMetadata(metadata),
|
||||
});
|
||||
await this.commit(
|
||||
'team_task',
|
||||
task,
|
||||
'createTeamTask',
|
||||
`${commandId}:create`,
|
||||
);
|
||||
return `Task #${number} created: "${task.subject}"`;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b3 decision 6: a member's `task_update`, the Legacy teammate
|
||||
* rules. Status/owner/subject/description, addBlockedBy, addBlocks and
|
||||
* delete are writes the member may do only on its own or unowned tasks;
|
||||
* activeForm and metadata are free for any task; owner resolves to the
|
||||
* member's own name or `""`. An unowned task moved to `in_progress`
|
||||
* takes the member as its owner. Deleting freezes tasks just like the
|
||||
* lead's flow. Each write place is replay-stable under
|
||||
* `${commandId}:${place + 1}`.
|
||||
*/
|
||||
async memberTaskUpdate(
|
||||
memberName: string,
|
||||
commandId: string,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<string> {
|
||||
return this.serial(async () => {
|
||||
const team = this.openTeam();
|
||||
if (team === undefined)
|
||||
throw new HostedTeamRefusal(
|
||||
'You are not a member of an open team. Ask the leader how the team ended.',
|
||||
);
|
||||
const number = taskNumber(args['taskId'] as string);
|
||||
const deleting = args['status'] === 'deleted';
|
||||
const blocks = deleting
|
||||
? []
|
||||
: ((args['addBlocks'] as string[] | undefined) ?? []).map(taskNumber);
|
||||
const blockedBy = deleting
|
||||
? []
|
||||
: ((args['addBlockedBy'] as string[] | undefined) ?? []).map(
|
||||
taskNumber,
|
||||
);
|
||||
const board = new Map(
|
||||
this.tasksOf(team.teamId).map((task) => [task.number, task]),
|
||||
);
|
||||
const commandId_ = (place: number) => `${commandId}:${place + 1}`;
|
||||
const pending = [number, ...blocks]
|
||||
.map((_, place) => place)
|
||||
.filter(
|
||||
(place) =>
|
||||
this.committed('updateTeamTask', commandId_(place)) === undefined,
|
||||
);
|
||||
const replaying = pending.length < blocks.length + 1;
|
||||
if (!replaying)
|
||||
this.validateMemberUpdate(
|
||||
team,
|
||||
board,
|
||||
memberName,
|
||||
number,
|
||||
args,
|
||||
blocks,
|
||||
blockedBy,
|
||||
);
|
||||
const notCommitted: string[] = [];
|
||||
for (const place of pending) {
|
||||
// Each remaining place is re-validated against the board as it
|
||||
// stands now — the board may have moved since the place before
|
||||
// it committed: a target can be gone, and an edge the original
|
||||
// validation allowed can now close a cycle. Deterministic
|
||||
// refusals ride the receipt; they never escape as faults.
|
||||
const liveBoard = this.memberBoardOf(team.teamId);
|
||||
if (place > 0) {
|
||||
const targetNumber = blocks[place - 1]!;
|
||||
const target = liveBoard.get(targetNumber);
|
||||
if (target === undefined) {
|
||||
notCommitted.push(`#${targetNumber} not found`);
|
||||
continue;
|
||||
}
|
||||
if (!this.memberMayWrite(target, memberName)) {
|
||||
notCommitted.push(
|
||||
`#${target.number} is owned by ${target.owner ?? 'someone'} (not unowned for you)`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
const selfTask = liveBoard.get(number);
|
||||
if (selfTask === undefined) {
|
||||
notCommitted.push(`#${number} not found`);
|
||||
continue;
|
||||
}
|
||||
const cycle = dependencyCycle(liveBoard, number, [targetNumber], []);
|
||||
if (cycle !== null) {
|
||||
notCommitted.push(
|
||||
`#${targetNumber} would create a dependency cycle (${cycle
|
||||
.map((each) => `#${each}`)
|
||||
.join(' → ')})`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
const current = this.task(target.taskId);
|
||||
const nextBlocks = new Set([...current.blockedBy, selfTask.taskId]);
|
||||
if (nextBlocks.size > MANAGED_TEAM_LIMITS.maxBlockers) {
|
||||
notCommitted.push(
|
||||
`#${targetNumber} would exceed ${MANAGED_TEAM_LIMITS.maxBlockers} blockers`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
await this.commit(
|
||||
'team_task',
|
||||
teamTaskRevisionBody(current, {
|
||||
addBlockedBy: [selfTask.taskId],
|
||||
}),
|
||||
'updateTeamTask',
|
||||
commandId_(place),
|
||||
);
|
||||
} catch (cause) {
|
||||
// Inside this serial scope the authority's conflict is the
|
||||
// board's own verdict for this shape: fold it like the
|
||||
// checks above, never a rotate-and-fault.
|
||||
if (cause instanceof ManagedSessionConflictError) {
|
||||
notCommitted.push(cause.message);
|
||||
continue;
|
||||
}
|
||||
throw cause;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
this.validateMemberUpdate(
|
||||
team,
|
||||
liveBoard,
|
||||
memberName,
|
||||
number,
|
||||
args,
|
||||
blocks,
|
||||
blockedBy,
|
||||
true,
|
||||
);
|
||||
} catch (cause) {
|
||||
if (!(cause instanceof HostedTeamRefusal)) throw cause;
|
||||
notCommitted.push(cause.message);
|
||||
continue;
|
||||
}
|
||||
const current = this.task(board.get(number)!.taskId);
|
||||
const revision = deleting
|
||||
? teamTaskRevisionBody(current, { status: 'deleted' })
|
||||
: await this.memberUpdatedTask(
|
||||
current,
|
||||
memberName,
|
||||
args,
|
||||
blockedBy,
|
||||
board,
|
||||
);
|
||||
await this.commit(
|
||||
'team_task',
|
||||
revision,
|
||||
'updateTeamTask',
|
||||
commandId_(place),
|
||||
);
|
||||
}
|
||||
const task = this.task(board.get(number)!.taskId);
|
||||
if (deleting) return `Task #${number} deleted.`;
|
||||
const done = `Task #${number} updated (status: ${task.status}${task.owner ? `, owner: ${task.owner}` : ''}).`;
|
||||
if (notCommitted.length === 0) return done;
|
||||
return `${done} Not committed: ${notCommitted.join('; ')}.`;
|
||||
});
|
||||
}
|
||||
|
||||
private memberBoardOf(teamId: string): ReadonlyMap<number, TeamTask> {
|
||||
return new Map(
|
||||
this.tasksOf(teamId).map((each) => [each.number, each] as const),
|
||||
);
|
||||
}
|
||||
|
||||
private memberMayWrite(task: TeamTask, memberName: string): boolean {
|
||||
return task.owner === null || task.owner === memberName;
|
||||
}
|
||||
|
||||
/**
|
||||
* The member rules of decision 6 for one board write: status, owner,
|
||||
* subject, description, addBlockedBy and delete all require the task to
|
||||
* be the member's own or unowned; addBlocks also requires every named
|
||||
* task to be the member's own or unowned; activeForm and metadata are
|
||||
* unrestricted.
|
||||
*/
|
||||
private validateMemberUpdate(
|
||||
team: TeamState,
|
||||
board: ReadonlyMap<number, TeamTask>,
|
||||
memberName: string,
|
||||
number: number,
|
||||
args: Record<string, unknown>,
|
||||
blocks: readonly number[],
|
||||
blockedBy: readonly number[],
|
||||
_revisited = false,
|
||||
): void {
|
||||
const task = board.get(number);
|
||||
if (task === undefined || task.status === 'deleted') {
|
||||
throw new HostedTeamRefusal(`Task #${number} not found.`);
|
||||
}
|
||||
const deleting = args['status'] === 'deleted';
|
||||
const touchesOwnerTaskRules =
|
||||
args['status'] !== undefined ||
|
||||
args['owner'] !== undefined ||
|
||||
args['subject'] !== undefined ||
|
||||
args['description'] !== undefined ||
|
||||
(args['addBlockedBy'] as unknown[] | undefined)?.length !== undefined;
|
||||
if (touchesOwnerTaskRules && !this.memberMayWrite(task, memberName)) {
|
||||
throw new HostedTeamRefusal(
|
||||
`Task #${number} is owned by ${task.owner}; you can change only tasks you own or that are unowned.`,
|
||||
);
|
||||
}
|
||||
// Existence comes before ownership: the probe below reads the tasks
|
||||
// this gate proves present, and an absent number is a refusal,
|
||||
// never an undefined read.
|
||||
const live = (eachNumber: number) => {
|
||||
const each = board.get(eachNumber);
|
||||
return each !== undefined && each.status !== 'deleted';
|
||||
};
|
||||
const missing = [...new Set([...blocks, ...blockedBy])]
|
||||
.filter((each) => !live(each))
|
||||
.sort((left, right) => left - right);
|
||||
if (missing.length > 0)
|
||||
throw new HostedTeamRefusal(
|
||||
`Cannot update task #${number}: referenced task${missing.length === 1 ? '' : 's'} ${missing.map((each) => `#${each}`).join(', ')} not found.`,
|
||||
);
|
||||
if (blocks.length > 0) {
|
||||
const held = blocks
|
||||
.map((each) => board.get(each)!)
|
||||
.find((each) => !this.memberMayWrite(each, memberName));
|
||||
if (held)
|
||||
throw new HostedTeamRefusal(
|
||||
`Task #${held.number} is owned by ${held.owner}; you can block only tasks you own or that are unowned.`,
|
||||
);
|
||||
}
|
||||
if (
|
||||
args['owner'] !== undefined &&
|
||||
args['owner'] !== '' &&
|
||||
sanitizeName(args['owner'] as string) !== memberName
|
||||
) {
|
||||
throw new HostedTeamRefusal(
|
||||
'Task owner must be your own name or "". Assign work to another member instead with send_message.',
|
||||
);
|
||||
}
|
||||
if (deleting) return;
|
||||
if (blocks.includes(number) || blockedBy.includes(number))
|
||||
throw new HostedTeamRefusal(
|
||||
`Cannot update task #${number}: a task cannot block or be blocked by itself.`,
|
||||
);
|
||||
const crowded = [
|
||||
[number, blockedBy] as const,
|
||||
...blocks.map((each) => [each, [number]] as const),
|
||||
].find(
|
||||
([each, added]) =>
|
||||
new Set([
|
||||
...board.get(each)!.blockedBy,
|
||||
...added.map((other) => board.get(other)!.taskId),
|
||||
]).size > MANAGED_TEAM_LIMITS.maxBlockers,
|
||||
);
|
||||
if (crowded)
|
||||
throw new HostedTeamRefusal(
|
||||
`Cannot update task #${number}: task #${crowded[0]} would have more than ${MANAGED_TEAM_LIMITS.maxBlockers} blockers.`,
|
||||
);
|
||||
const cycle = dependencyCycle(board, number, blocks, blockedBy);
|
||||
if (cycle)
|
||||
throw new HostedTeamRefusal(
|
||||
`Cannot update task #${number}: this would create a dependency cycle (${cycle.map((each) => `#${each}`).join(' → ')}).`,
|
||||
);
|
||||
const status =
|
||||
(args['status'] as TeamTaskStatus | undefined) ?? task.status;
|
||||
const resultingOwner =
|
||||
args['owner'] === undefined
|
||||
? status === 'in_progress' && task.owner === null
|
||||
? memberName
|
||||
: task.owner
|
||||
: args['owner'] === ''
|
||||
? null
|
||||
: memberName;
|
||||
if (status === 'in_progress' && resultingOwner === null)
|
||||
throw new HostedTeamRefusal(
|
||||
'Cannot move a task to in_progress without an owner. Set owner to your own name, or leave the task pending.',
|
||||
);
|
||||
// The lead's owner rule still applies wherever the final owner is named.
|
||||
if (resultingOwner !== null && resultingOwner !== MANAGED_TEAM_LEADER) {
|
||||
const member = team.members.find((each) => each.name === resultingOwner);
|
||||
if (member === undefined && resultingOwner === memberName)
|
||||
throw new HostedTeamRefusal(
|
||||
"You are not on this team's roster, so you cannot own a task here.",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private async memberUpdatedTask(
|
||||
task: TeamTask,
|
||||
memberName: string,
|
||||
args: Record<string, unknown>,
|
||||
blockedBy: readonly number[],
|
||||
board: ReadonlyMap<number, TeamTask>,
|
||||
): Promise<TeamTask> {
|
||||
const status =
|
||||
(args['status'] as TeamTaskStatus | undefined) ?? task.status;
|
||||
const owner =
|
||||
args['owner'] === undefined
|
||||
? status === 'in_progress' && task.owner === null
|
||||
? memberName
|
||||
: undefined
|
||||
: args['owner'] === ''
|
||||
? null
|
||||
: memberName;
|
||||
return teamTaskRevisionBody(task, {
|
||||
subject: args['subject'] as string | undefined,
|
||||
descriptionRef:
|
||||
args['description'] === undefined
|
||||
? undefined
|
||||
: await this.publishText(args['description'] as string),
|
||||
activeForm: args['activeForm'] as string | undefined,
|
||||
metadataRef:
|
||||
args['metadata'] === undefined
|
||||
? undefined
|
||||
: await this.publishMetadata(
|
||||
metadataBytes(
|
||||
mergeMetadata(
|
||||
task.metadataRef === null
|
||||
? {}
|
||||
: (JSON.parse(
|
||||
(
|
||||
await this.store.resources.read(task.metadataRef)
|
||||
).toString('utf8'),
|
||||
) as Record<string, unknown>),
|
||||
args['metadata'],
|
||||
),
|
||||
),
|
||||
),
|
||||
owner,
|
||||
status: args['status'] as TeamTaskStatus | undefined,
|
||||
addBlockedBy: blockedBy.map((each) => board.get(each)!.taskId),
|
||||
});
|
||||
}
|
||||
|
||||
/** The lead's child Session runs, indexed for one call. */
|
||||
private chains(): HostedChildChains {
|
||||
return new HostedChildChains(
|
||||
|
|
@ -1319,6 +1785,20 @@ export class HostedTeamSession {
|
|||
: this.store.resources.publish('managed-team-metadata', bytes);
|
||||
}
|
||||
|
||||
/**
|
||||
* The command funnel's `execute` half-applied-write probe (decision 3):
|
||||
* whether one board write this command derived is already committed —
|
||||
* a create's `:create` or one update place. When any is, `execute`
|
||||
* answers from what is committed and re-validates what remains instead
|
||||
* of running the fresh admission gates again.
|
||||
*/
|
||||
commandWriteCommitted(recordId: string): boolean {
|
||||
return (
|
||||
this.committed('createTeamTask', recordId) !== undefined ||
|
||||
this.committed('updateTeamTask', recordId) !== undefined
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The record a call's command already committed (decision 11): such a
|
||||
* command is done, and a write is computed only for one that is not.
|
||||
|
|
|
|||
1291
packages/cli/src/serve/hosted-workspace-tool-turn.commands.test.ts
Normal file
1291
packages/cli/src/serve/hosted-workspace-tool-turn.commands.test.ts
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -62,7 +62,11 @@ import {
|
|||
type ManagedSessionDurableRef,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-session-records.js';
|
||||
import type { ManagedHarnessHandle } from '@qwen-code/qwen-code-core/managed-runtime/managed-harness-factory.js';
|
||||
import type { HarnessAgentWaitRun } from '@qwen-code/qwen-code-core/managed-runtime/managed-harness-checkpoint.js';
|
||||
import {
|
||||
harnessCommandLocked,
|
||||
type HarnessAgentWaitRun,
|
||||
type HarnessCommandWaitRun,
|
||||
} from '@qwen-code/qwen-code-core/managed-runtime/managed-harness-checkpoint.js';
|
||||
import {
|
||||
HTTP_MANAGED_SESSION_STORE_CONTRACT,
|
||||
ManagedSessionStoreHttpError,
|
||||
|
|
@ -104,11 +108,15 @@ import { sessionMessageId } from '@qwen-code/qwen-code-core/managed-runtime/mana
|
|||
import { MANAGED_SESSION_MESSAGE_LIMITS } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-message-record.js';
|
||||
import { sanitizeName } from '@qwen-code/qwen-code-core/agents/team/teamHelpers.js';
|
||||
import {
|
||||
ACTIVE_FORM_CHARS,
|
||||
DESCRIPTION_CHARS,
|
||||
HOSTED_TEAM_TOOLS,
|
||||
HOSTED_TEAM_TOOL_NAMES,
|
||||
HostedTeamRefusal,
|
||||
hostedTeamArgumentError,
|
||||
hostedTeamMembership,
|
||||
SUBJECT_CHARS,
|
||||
TASK_STATUSES,
|
||||
type HostedTeamMembership,
|
||||
type HostedTeamSession,
|
||||
} from './hosted-team-session.js';
|
||||
|
|
@ -125,6 +133,7 @@ import {
|
|||
HostedChildChains,
|
||||
broadcastMessageId,
|
||||
broadcastRunId,
|
||||
firstInputOpensWithMemberPreamble,
|
||||
hostedCallKey,
|
||||
hostedPromptKey,
|
||||
hostedRouteDetails,
|
||||
|
|
@ -132,6 +141,13 @@ import {
|
|||
neverStartedPhrase,
|
||||
teamMemberPreamble,
|
||||
} from './hosted-member-chain.js';
|
||||
import { sessionCommandId } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-command-operations.js';
|
||||
import type { SessionCommandKind } from '@qwen-code/qwen-code-core/managed-runtime/managed-session-command-record.js';
|
||||
import {
|
||||
SESSION_COMMAND_LEADER_UNANSWERING_TEXT,
|
||||
sessionCommandEvidenceText,
|
||||
type HostedSessionCommandSession,
|
||||
} from './hosted-session-command-session.js';
|
||||
import { MANAGED_TEAM_LEADER } from '@qwen-code/qwen-code-core/managed-runtime/managed-team-record.js';
|
||||
import { HostedMonitorLoop } from './hosted-monitor-loop.js';
|
||||
import { HostedMonitorRemoteExecutor } from './hosted-monitor-remote-executor.js';
|
||||
|
|
@ -297,6 +313,24 @@ function sessionMessageAdmissionsEnabled(): boolean {
|
|||
}
|
||||
}
|
||||
|
||||
// H4e-b3: the member board tools exist only while the session_command
|
||||
// domain is enabled for commits.
|
||||
function sessionCommandAdmissionsEnabled(): boolean {
|
||||
try {
|
||||
assertManagedSessionDomainEnabled('session_command');
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b3 decision 4: a command's wait bound, measured from the outbound
|
||||
* entry's own commit — well inside the Turn deadline and the relay's
|
||||
* inactivity bound.
|
||||
*/
|
||||
export const COMMAND_WAIT_MS = 120_000;
|
||||
|
||||
// H4e-b1: the team tools, and the agent tool's `name`, exist only while
|
||||
// both lead-side team domains are enabled for commits.
|
||||
function teamAdmissionsEnabled(): boolean {
|
||||
|
|
@ -864,6 +898,134 @@ export const HOSTED_SEND_MESSAGE_TO_PARENT_TOOL: FunctionDeclaration = {
|
|||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* H4e-b3 decision 6: a member's board read, as a `board_read` command to
|
||||
* its leader. The schema mirrors the lead's `task_list` exactly.
|
||||
*/
|
||||
export const HOSTED_MEMBER_TASK_LIST_TOOL: FunctionDeclaration = {
|
||||
name: 'task_list',
|
||||
description:
|
||||
'List your team board, optionally filtered, followed by each member with the state of its current run — the board and the roster as your leader holds them. Board changes are not pushed to you: read the board again when you need the current state.',
|
||||
parametersJsonSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
status: {
|
||||
type: 'string',
|
||||
enum: ['pending', 'in_progress', 'completed'],
|
||||
description: 'Filter by task status.',
|
||||
},
|
||||
owner: {
|
||||
type: 'string',
|
||||
description: 'Filter by owner name.',
|
||||
},
|
||||
blockedBy: {
|
||||
type: 'string',
|
||||
description: 'Filter for tasks still blocked by this task number.',
|
||||
},
|
||||
},
|
||||
additionalProperties: false,
|
||||
},
|
||||
};
|
||||
|
||||
/** H4e-b3 decision 6: a member's task creation, as a `board_write` command. */
|
||||
export const HOSTED_MEMBER_TASK_CREATE_TOOL: FunctionDeclaration = {
|
||||
name: 'task_create',
|
||||
description:
|
||||
'Create a pending task on your team board. The leader records it and answers with the new task number. Members are not told about board changes: put new work for another member in your final answer, or ask the leader to hand it out.',
|
||||
parametersJsonSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
subject: {
|
||||
type: 'string',
|
||||
maxLength: SUBJECT_CHARS,
|
||||
description: 'Short title for the task.',
|
||||
},
|
||||
description: {
|
||||
type: 'string',
|
||||
maxLength: DESCRIPTION_CHARS,
|
||||
description: 'Detailed description of the task.',
|
||||
},
|
||||
activeForm: {
|
||||
type: 'string',
|
||||
maxLength: ACTIVE_FORM_CHARS,
|
||||
description: 'Present tense label for UI (e.g., "Running tests").',
|
||||
},
|
||||
metadata: {
|
||||
type: 'object',
|
||||
description: 'Optional arbitrary metadata.',
|
||||
},
|
||||
},
|
||||
required: ['subject', 'description'],
|
||||
additionalProperties: false,
|
||||
},
|
||||
};
|
||||
|
||||
/** H4e-b3 decision 6: a member's task revision, as a `board_write` command. */
|
||||
export const HOSTED_MEMBER_TASK_UPDATE_TOOL: FunctionDeclaration = {
|
||||
name: 'task_update',
|
||||
description:
|
||||
'Update a board task: status, owner, subject, description, active form, metadata or dependencies; set status to "deleted" to remove it. Status, owner, subject and description change only on a task you own or that has no owner, and edges only reach such tasks; activeForm and metadata change on any task; owner may be only your own name or "" to unassign, and moving an unowned task to in_progress makes you its owner. A task in progress needs an owner. The answer is the revised task, or the refusal.',
|
||||
parametersJsonSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
taskId: {
|
||||
type: 'string',
|
||||
description: 'Number of the task to update, as "3" or "#3".',
|
||||
},
|
||||
status: {
|
||||
type: 'string',
|
||||
enum: TASK_STATUSES,
|
||||
description: 'New task status.',
|
||||
},
|
||||
owner: {
|
||||
type: 'string',
|
||||
description:
|
||||
'New owner: your own name, or an empty string to unassign.',
|
||||
},
|
||||
subject: {
|
||||
type: 'string',
|
||||
maxLength: SUBJECT_CHARS,
|
||||
description: 'Updated task title.',
|
||||
},
|
||||
description: {
|
||||
type: 'string',
|
||||
maxLength: DESCRIPTION_CHARS,
|
||||
description: 'Updated task description.',
|
||||
},
|
||||
activeForm: {
|
||||
type: 'string',
|
||||
maxLength: ACTIVE_FORM_CHARS,
|
||||
description: 'Present tense label for UI.',
|
||||
},
|
||||
metadata: {
|
||||
type: 'object',
|
||||
description: 'Metadata to merge. Set a key to null to delete it.',
|
||||
},
|
||||
addBlocks: {
|
||||
type: 'array',
|
||||
items: { type: 'string' },
|
||||
description: 'Numbers of the tasks this task blocks.',
|
||||
},
|
||||
addBlockedBy: {
|
||||
type: 'array',
|
||||
items: { type: 'string' },
|
||||
description: 'Numbers of the tasks that block this task.',
|
||||
},
|
||||
},
|
||||
required: ['taskId'],
|
||||
additionalProperties: false,
|
||||
},
|
||||
};
|
||||
|
||||
export const HOSTED_MEMBER_BOARD_TOOLS: readonly FunctionDeclaration[] = [
|
||||
HOSTED_MEMBER_TASK_LIST_TOOL,
|
||||
HOSTED_MEMBER_TASK_CREATE_TOOL,
|
||||
HOSTED_MEMBER_TASK_UPDATE_TOOL,
|
||||
];
|
||||
|
||||
export const HOSTED_MEMBER_COMMAND_TOOL_NAMES: readonly string[] =
|
||||
HOSTED_MEMBER_BOARD_TOOLS.map((tool) => tool.name!);
|
||||
|
||||
/**
|
||||
* H4e-b1: the agent tool of a Session whose team tools are declared. With
|
||||
* `name`, the child is a member of the Session's active team.
|
||||
|
|
@ -1089,6 +1251,7 @@ interface HostedPreparedRequest {
|
|||
agentBackground: boolean;
|
||||
team: boolean;
|
||||
message: boolean;
|
||||
command: boolean;
|
||||
}
|
||||
|
||||
export class HostedWorkspaceToolTurn {
|
||||
|
|
@ -1140,6 +1303,16 @@ export class HostedWorkspaceToolTurn {
|
|||
private readonly childConsumption: (childRunId: string) => void;
|
||||
private readonly messages?: HostedSessionMessageSession;
|
||||
private readonly teams?: HostedTeamSession;
|
||||
/** H4e-b3: the member's command funnel to its parent, beside its messages. */
|
||||
private readonly commands?: HostedSessionCommandSession;
|
||||
/** b3b: whether this Session's run carries a member preamble (once read). */
|
||||
private memberPreamble?: boolean;
|
||||
/** b3b: the member board vocabulary declared, refreshed per declarations. */
|
||||
private readonly commandToolNames = new Set<string>();
|
||||
/** b3b decision 4: batches of this Turn whose command park committed. */
|
||||
private readonly commandParks = new Set<string>();
|
||||
/** b3b decision 4: a command call of this Turn timed out (decision 4's lock). */
|
||||
private commandTimedOut = false;
|
||||
/**
|
||||
* H4e-b3 decisions 5/9: this run's first input carried an unescaped
|
||||
* `plan-mode="required"` preamble (assembly read it off the journal).
|
||||
|
|
@ -1188,6 +1361,7 @@ export class HostedWorkspaceToolTurn {
|
|||
};
|
||||
messages?: HostedSessionMessageSession;
|
||||
teams?: HostedTeamSession;
|
||||
commands?: HostedSessionCommandSession;
|
||||
/**
|
||||
* H4e-b3 decisions 5/9: this run's first input carried an unescaped
|
||||
* `plan-mode="required"` preamble; its calls admit reads only.
|
||||
|
|
@ -1209,6 +1383,7 @@ export class HostedWorkspaceToolTurn {
|
|||
extras?.childAgents?.queueConsumption ?? (() => undefined);
|
||||
this.messages = extras?.messages;
|
||||
this.teams = extras?.teams;
|
||||
this.commands = extras?.commands;
|
||||
this.memberPlanMode = extras?.memberPlanMode === true;
|
||||
this.publication =
|
||||
publicationOrShell && 'owner' in publicationOrShell
|
||||
|
|
@ -1316,6 +1491,16 @@ export class HostedWorkspaceToolTurn {
|
|||
? [HOSTED_SEND_MESSAGE_TO_PARENT_TOOL]
|
||||
: []),
|
||||
];
|
||||
// H4e-b3: a member's board tools are its commands to the leader,
|
||||
// declared only on a lineage Session whose run's first input carries
|
||||
// its member preamble (decision 5) — a convenience beside the lead's
|
||||
// own admission of every command.
|
||||
this.commandToolNames.clear();
|
||||
if (await this.memberCommandsDeclared()) {
|
||||
this.advertised.push(...HOSTED_MEMBER_BOARD_TOOLS);
|
||||
for (const tool of HOSTED_MEMBER_BOARD_TOOLS)
|
||||
if (tool.name !== undefined) this.commandToolNames.add(tool.name);
|
||||
}
|
||||
return this.advertised;
|
||||
}
|
||||
|
||||
|
|
@ -1333,6 +1518,66 @@ export class HostedWorkspaceToolTurn {
|
|||
: undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* b3b decision 5: the member board tools ride the command channel, and
|
||||
* only a lineage Session whose run's first input opens with a
|
||||
* well-formed member preamble declares them.
|
||||
*/
|
||||
private async memberCommandsDeclared(): Promise<boolean> {
|
||||
if (this.commands?.lineage === undefined) return false;
|
||||
if (!sessionCommandAdmissionsEnabled()) return false;
|
||||
this.memberPreamble ??= await this.firstInputHasMemberPreamble();
|
||||
return this.memberPreamble;
|
||||
}
|
||||
|
||||
/**
|
||||
* The member marker (decision 5): this Session's own first accepted
|
||||
* input's text, checked for the run's preamble block. Anything
|
||||
* unreadable hides the tools — it never grants them.
|
||||
*/
|
||||
private async firstInputHasMemberPreamble(): Promise<boolean> {
|
||||
const authority = this.session.authority;
|
||||
const first = authority
|
||||
.eventsInSequenceRange(1, authority.committedSequence)
|
||||
.find((event) => event.kind === 'input.accepted');
|
||||
const contentRef = first?.payload['contentRef'];
|
||||
if (contentRef === undefined) return false;
|
||||
const ref = assertManagedSessionDurableRef(
|
||||
contentRef,
|
||||
'member launch input',
|
||||
);
|
||||
if (ref.kind !== 'managed-input') return false;
|
||||
try {
|
||||
const parsed: unknown = JSON.parse(
|
||||
(await this.session.resources.read(ref)).toString('utf8'),
|
||||
);
|
||||
// The admission committers write the launch envelope as an
|
||||
// OpenAI message-content array; a plain {text} envelope rides
|
||||
// the same gate for the test-path fixtures.
|
||||
let text: string | undefined;
|
||||
if (Array.isArray(parsed)) {
|
||||
const part = parsed.find(
|
||||
(each): each is { type: string; text: string } =>
|
||||
typeof each === 'object' &&
|
||||
each !== null &&
|
||||
(each as Record<string, unknown>)['type'] === 'text' &&
|
||||
typeof (each as Record<string, unknown>)['text'] === 'string',
|
||||
);
|
||||
text = part?.text;
|
||||
} else if (
|
||||
typeof parsed === 'object' &&
|
||||
parsed !== null &&
|
||||
typeof (parsed as Record<string, unknown>)['text'] === 'string'
|
||||
) {
|
||||
text = (parsed as Record<string, unknown>)['text'] as string;
|
||||
}
|
||||
if (text === undefined) return false;
|
||||
return firstInputOpensWithMemberPreamble(text);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b1's batch rules for a member launch: its name is a string, no
|
||||
* other launch in the batch takes the same name, and the batch neither
|
||||
|
|
@ -1978,6 +2223,7 @@ export class HostedWorkspaceToolTurn {
|
|||
agentBackground: true,
|
||||
team: false,
|
||||
message: false,
|
||||
command: false,
|
||||
});
|
||||
}
|
||||
this.logicalRound = plan.round;
|
||||
|
|
@ -2222,8 +2468,15 @@ export class HostedWorkspaceToolTurn {
|
|||
let agentAdmitted = false;
|
||||
let agentBackground = true;
|
||||
let messageAdmitted = false;
|
||||
// b3b: on a member Session the board vocabulary is its command
|
||||
// channel's, never the lead funnel's — the two never meet on one
|
||||
// profile, but a name collision must route to the command arm.
|
||||
const command =
|
||||
mcpInput === undefined && this.commandToolNames.has(call.name);
|
||||
const team =
|
||||
mcpInput === undefined && HOSTED_TEAM_TOOL_NAMES.includes(call.name);
|
||||
!command &&
|
||||
mcpInput === undefined &&
|
||||
HOSTED_TEAM_TOOL_NAMES.includes(call.name);
|
||||
// H4e-b3 decision 9: a run whose first input carried an unescaped
|
||||
// plan-mode preamble admits reads only; every other call is
|
||||
// refused before it runs, ahead of any per-tool arm below.
|
||||
|
|
@ -2550,6 +2803,9 @@ export class HostedWorkspaceToolTurn {
|
|||
validationError = `Hosted send_message message exceeds ${MANAGED_SESSION_MESSAGE_LIMITS.maxContentBytes} bytes.`;
|
||||
}
|
||||
input = teamForm ? hostedTeamMessageInput(args) : { ...args };
|
||||
} else if (command) {
|
||||
validationError = this.commandArgumentError(call, source);
|
||||
input = { ...call.args };
|
||||
} else if (team) {
|
||||
validationError = hostedTeamArgumentError(call.name, call.args);
|
||||
input = { ...call.args };
|
||||
|
|
@ -2599,6 +2855,7 @@ export class HostedWorkspaceToolTurn {
|
|||
agentBackground,
|
||||
message: mcpInput === undefined && messageAdmitted,
|
||||
team,
|
||||
command,
|
||||
};
|
||||
});
|
||||
};
|
||||
|
|
@ -2663,7 +2920,10 @@ export class HostedWorkspaceToolTurn {
|
|||
!this.acquired &&
|
||||
requests.some(
|
||||
(request) =>
|
||||
request.agent !== true && request.message !== true && !request.team,
|
||||
request.agent !== true &&
|
||||
request.message !== true &&
|
||||
!request.team &&
|
||||
request.command !== true,
|
||||
)
|
||||
) {
|
||||
// Acquisition may have taken effect even when its reply is lost.
|
||||
|
|
@ -2970,9 +3230,11 @@ export class HostedWorkspaceToolTurn {
|
|||
if (refusals[ordinal] !== undefined) continue;
|
||||
// A child-agent launch has no Runtime execution to reserve: the
|
||||
// control plane's relay owns its side effect, so it never enters
|
||||
// the Broker pipeline below. Nor does a session message or a team
|
||||
// tool, which commit only to this Session's journal.
|
||||
if (request.agent || request.message || request.team) continue;
|
||||
// the Broker pipeline below. Nor does a session message, a team
|
||||
// tool, or a session command, which commit only to this Session's
|
||||
// journal.
|
||||
if (request.agent || request.message || request.team || request.command)
|
||||
continue;
|
||||
if (request.mcp) {
|
||||
const renewed = this.mcp!.toolInput(
|
||||
request.call.name,
|
||||
|
|
@ -3349,6 +3611,27 @@ export class HostedWorkspaceToolTurn {
|
|||
renewTimer.unref();
|
||||
}
|
||||
const responses: Part[] = [];
|
||||
// b3b decision 4: the batch's command park names every command call
|
||||
// that survived validation; the first of them commits the checkpoint.
|
||||
const commandRuns: HarnessCommandWaitRun[] = requests.flatMap(
|
||||
(request, index) =>
|
||||
request.command && refusals[index] === undefined
|
||||
? [
|
||||
{
|
||||
commandId: sessionCommandId({
|
||||
senderSessionId:
|
||||
this.session.authority.sessionHeader.sessionKey.sessionId,
|
||||
promptKey: hostedPromptKey(this.promptId),
|
||||
callId: request.call.callId,
|
||||
}),
|
||||
functionCallId: request.call.callId,
|
||||
toolName: request.call.name,
|
||||
modelMessageId: messageId,
|
||||
folded: false,
|
||||
},
|
||||
]
|
||||
: [],
|
||||
);
|
||||
for (const [index, request] of requests.entries()) {
|
||||
const refused = refusal(index);
|
||||
if (refused) {
|
||||
|
|
@ -3368,6 +3651,17 @@ export class HostedWorkspaceToolTurn {
|
|||
);
|
||||
continue;
|
||||
}
|
||||
if (request.command) {
|
||||
responses.push(
|
||||
...(await this.acceptSessionCommand(
|
||||
request,
|
||||
model,
|
||||
signal,
|
||||
commandRuns,
|
||||
)),
|
||||
);
|
||||
continue;
|
||||
}
|
||||
if (request.team) {
|
||||
responses.push(
|
||||
...(await this.acceptTeamTool(request, model, signal)),
|
||||
|
|
@ -4523,6 +4817,519 @@ export class HostedWorkspaceToolTurn {
|
|||
return parts;
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b3 decision 4 and 6: one member board command's argument check,
|
||||
* and the batch rule — a command call shares its batch only with other
|
||||
* command calls and with the journal-only `send_message` to `"parent"`,
|
||||
* since a command parks the Turn while the leader answers.
|
||||
*/
|
||||
private commandArgumentError(
|
||||
call: ToolCallRequestInfo,
|
||||
batch: readonly ToolCallRequestInfo[],
|
||||
): string | undefined {
|
||||
const keys: Record<string, readonly string[]> = {
|
||||
task_list: ['status', 'owner', 'blockedBy'],
|
||||
task_create: ['subject', 'description', 'activeForm', 'metadata'],
|
||||
task_update: [
|
||||
'taskId',
|
||||
'status',
|
||||
'owner',
|
||||
'subject',
|
||||
'description',
|
||||
'activeForm',
|
||||
'metadata',
|
||||
'addBlocks',
|
||||
'addBlockedBy',
|
||||
],
|
||||
};
|
||||
const unsupported = Object.keys(call.args).find(
|
||||
(key) => !keys[call.name]?.includes(key),
|
||||
);
|
||||
if (unsupported !== undefined)
|
||||
return `Hosted ${call.name} received unsupported argument ${JSON.stringify(unsupported)}.`;
|
||||
if (
|
||||
batch.some(
|
||||
(other) =>
|
||||
other !== call &&
|
||||
!this.commandToolNames.has(other.name) &&
|
||||
other.name !== 'send_message',
|
||||
)
|
||||
)
|
||||
return `Hosted ${call.name} cannot share a batch with Workspace or Runtime calls; a command parks the Turn while the leader answers, so run board commands in their own batch or beside send_message to "parent".`;
|
||||
if (call.name === 'task_create') {
|
||||
if (
|
||||
typeof call.args['subject'] !== 'string' ||
|
||||
!(call.args['subject'] as string).trim()
|
||||
)
|
||||
return 'Hosted task_create requires a nonempty subject.';
|
||||
if (
|
||||
typeof call.args['description'] !== 'string' ||
|
||||
!(call.args['description'] as string).trim()
|
||||
)
|
||||
return 'Hosted task_create requires a nonempty description.';
|
||||
}
|
||||
if (
|
||||
call.name === 'task_update' &&
|
||||
(typeof call.args['taskId'] !== 'string' ||
|
||||
!(call.args['taskId'] as string).trim())
|
||||
)
|
||||
return 'Hosted task_update requires the taskId of the task to update.';
|
||||
if (
|
||||
call.args['status'] !== undefined &&
|
||||
!TASK_STATUSES.includes(call.args['status'] as string)
|
||||
)
|
||||
return `Hosted ${call.name} status must be one of ${TASK_STATUSES.join(', ')}.`;
|
||||
return undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b3 decision 4: one member board command. Before the batch's first
|
||||
* command call runs, the Turn commits the `await_command` checkpoint
|
||||
* naming every command call of the batch; each call then commits its
|
||||
* outbound entry and polls its own journal for the reply, as
|
||||
* `await_agent` does for a child's acceptance, never holding the funnel
|
||||
* chain the relay's `handover` and `answered` commit on. The wait ends
|
||||
* at the reply, at 2 minutes from the entry's own commit, or with the
|
||||
* Turn; once one call of the Turn timed out, every later command call
|
||||
* of it refuses at once.
|
||||
*/
|
||||
/** Decision 4's lock is the Turn's durable state, not just Turn
|
||||
* memory: every checkpoint of the Turn carries it, so a replacement
|
||||
* owner re-arms from whichever one it resumes, and a timed-out leader
|
||||
* stays unanswering for every later command, across restarts as across
|
||||
* seconds. */
|
||||
private async syncCommandTimedOutFromDurable(): Promise<void> {
|
||||
if (this.commandTimedOut) return;
|
||||
const authorization =
|
||||
await this.session.authority.harnessRunAuthorization();
|
||||
if (
|
||||
authorization.status === 'runnable' &&
|
||||
harnessCommandLocked(authorization.checkpoint, this.promptId)
|
||||
)
|
||||
this.commandTimedOut = true;
|
||||
}
|
||||
|
||||
private async acceptSessionCommand(
|
||||
request: { call: ToolCallRequestInfo },
|
||||
model: string,
|
||||
signal: AbortSignal,
|
||||
batchRuns: readonly HarnessCommandWaitRun[],
|
||||
): Promise<Part[]> {
|
||||
const { name, callId, args } = request.call;
|
||||
const commands = this.commands;
|
||||
if (commands === undefined)
|
||||
throw new Error('A member command call has no command funnel.');
|
||||
await this.syncCommandTimedOutFromDurable();
|
||||
const kind: SessionCommandKind =
|
||||
name === 'task_list' ? 'board_read' : 'board_write';
|
||||
const key = this.session.authority.sessionHeader.sessionKey;
|
||||
const commandId = sessionCommandId({
|
||||
senderSessionId: key.sessionId,
|
||||
promptKey: hostedPromptKey(this.promptId),
|
||||
callId,
|
||||
});
|
||||
// Before this batch's first command call runs, the park names every
|
||||
// command call of the batch. Batch siblings replay the same run set;
|
||||
// a fully folded earlier batch left no durable wait, and a later
|
||||
// batch of this Turn parks afresh under its own run set.
|
||||
const parkKey = batchRuns.map((run) => run.commandId).join(':');
|
||||
if (!this.commandParks.has(parkKey)) {
|
||||
const argsRef = await this.session.resources.publish(
|
||||
'managed-command-args',
|
||||
Buffer.from(JSON.stringify(args), 'utf8'),
|
||||
);
|
||||
await this.harness.commitAwaitCommand(
|
||||
batchRuns,
|
||||
{ turnId: this.promptId, promptId: this.promptId },
|
||||
{
|
||||
holdsWorkspaceMount: this.acquired,
|
||||
attempt: {
|
||||
attemptId: batchRuns[0]?.modelMessageId ?? this.promptId,
|
||||
routeRef: argsRef,
|
||||
},
|
||||
},
|
||||
);
|
||||
this.commandParks.add(parkKey);
|
||||
}
|
||||
const answer = async (text: string, failed: boolean): Promise<Part[]> => {
|
||||
const parts = failed
|
||||
? convertToFunctionErrorResponse(name, callId, [], text)
|
||||
: convertToFunctionResponse(name, callId, [{ text }]);
|
||||
await this.commit('tool_result', parts, model);
|
||||
return parts;
|
||||
};
|
||||
if (this.commandTimedOut) {
|
||||
// The lock refuses only a call that sent nothing. A call whose
|
||||
// outbound already exists is the one whose own wait timed out,
|
||||
// named again by a re-driven batch: its journal answers it.
|
||||
const parts =
|
||||
commands.command(commandId) === undefined
|
||||
? await answer(SESSION_COMMAND_LEADER_UNANSWERING_TEXT, true)
|
||||
: await this.answerCommandByJournal(
|
||||
commands,
|
||||
request,
|
||||
commandId,
|
||||
model,
|
||||
);
|
||||
// The locked run still folds, or the Turn's wait never completes —
|
||||
// its answer rides the batch the same as a call that ran.
|
||||
await this.harness.resolveAwaitCommand(commandId, { timedOut: true });
|
||||
return parts;
|
||||
}
|
||||
if (signal.aborted) {
|
||||
return answer('The turn was cancelled before this tool call ran.', true);
|
||||
}
|
||||
try {
|
||||
await commands.runCommand({
|
||||
commandId,
|
||||
kind,
|
||||
args: { ...args },
|
||||
executionCallId: this.childRunIdFor(callId),
|
||||
closing: this.session.authority.currentActivation?.phase !== 'active',
|
||||
});
|
||||
} catch (cause) {
|
||||
// byte_limit and budget_exhausted are the funnel's member-facing
|
||||
// refusals; a store fault is recovery's, never the call's answer.
|
||||
if (cause instanceof ManagedSessionRecordError) {
|
||||
const parts = await answer(cause.message, true);
|
||||
// A normal local refusal still owes the parked wait its fold —
|
||||
// a durable wait it leaves behind turns the Turn's next call
|
||||
// into a recovery fault.
|
||||
await this.harness.resolveAwaitCommand(commandId);
|
||||
return parts;
|
||||
}
|
||||
throw cause;
|
||||
}
|
||||
// The bound measures from the entry's own commit, so a resumed wait
|
||||
// keeps what is left instead of a fresh 2 minutes.
|
||||
const anchor = commands.outboundCommittedAt(commandId) ?? Date.now();
|
||||
const deadline = anchor + COMMAND_WAIT_MS;
|
||||
for (;;) {
|
||||
if (signal.aborted) {
|
||||
const parts = await this.answerCommandByJournal(
|
||||
commands,
|
||||
request,
|
||||
commandId,
|
||||
model,
|
||||
);
|
||||
await this.harness.resolveAwaitCommand(commandId);
|
||||
return parts;
|
||||
}
|
||||
const record = commands.command(commandId);
|
||||
const state = record?.run.delivery?.state;
|
||||
if (state === 'accepted' || state === 'consumed') {
|
||||
const reply = await commands.readReply(commandId);
|
||||
if (reply !== null) {
|
||||
const parts = await this.commitCommandReply(
|
||||
request,
|
||||
reply.text,
|
||||
reply.outcome === 'refused',
|
||||
model,
|
||||
);
|
||||
await commands.commitConsumed(commandId);
|
||||
await this.harness.resolveAwaitCommand(commandId);
|
||||
return parts;
|
||||
}
|
||||
}
|
||||
if (
|
||||
state === 'rejected' ||
|
||||
state === 'cancelled' ||
|
||||
state === 'unknown'
|
||||
) {
|
||||
const parts = await this.answerCommandByJournal(
|
||||
commands,
|
||||
request,
|
||||
commandId,
|
||||
model,
|
||||
);
|
||||
await this.harness.resolveAwaitCommand(commandId);
|
||||
return parts;
|
||||
}
|
||||
if (Date.now() >= deadline) {
|
||||
this.commandTimedOut = true;
|
||||
// The lock is stamped at detection time — before this Turn's next
|
||||
// commit, so a crash in the settle/tool_result window between the
|
||||
// accepting-to-unknown write and its answer can never un-lock it.
|
||||
await this.harness.markCommandWaitTimeout();
|
||||
const parts = await this.answerCommandByJournal(
|
||||
commands,
|
||||
request,
|
||||
commandId,
|
||||
model,
|
||||
);
|
||||
// The fold restates the Turn's lock: a replacement owner of this
|
||||
// Turn re-arms from it and refuses the next command too.
|
||||
await this.harness.resolveAwaitCommand(commandId, {
|
||||
timedOut: true,
|
||||
});
|
||||
return parts;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision 14: the command call's answer by what its journal committed.
|
||||
* A planned entry is cancelled and answered as not sent; a handed-over
|
||||
* one says the command reached the leader and a write may have taken
|
||||
* effect; rejected, cancelled and unknown are said so, with a reply
|
||||
* that landed mid-race still read. The settle's conflict is the race's
|
||||
* own evidence, so once it loses the journal is re-read and answered.
|
||||
*/
|
||||
private async answerCommandByJournal(
|
||||
commands: HostedSessionCommandSession,
|
||||
request: { call: Pick<ToolCallRequestInfo, 'name' | 'callId'> },
|
||||
commandId: string,
|
||||
model: string,
|
||||
journaled?: ReadonlySet<string>,
|
||||
): Promise<Part[]> {
|
||||
const { name, callId } = request.call;
|
||||
for (let attempt = 0; attempt < 4; attempt += 1) {
|
||||
const record = commands.command(commandId);
|
||||
if (record === undefined) {
|
||||
// 未发送:没有记录要结算,直接给了计划态的答案,不再走
|
||||
// settle(那只会抛 "has no record to revise")。
|
||||
const unsent = convertToFunctionErrorResponse(
|
||||
name,
|
||||
callId,
|
||||
[],
|
||||
sessionCommandEvidenceText('planned'),
|
||||
);
|
||||
if (!journaled?.has(callId))
|
||||
await this.commit('tool_result', unsent, model);
|
||||
return unsent;
|
||||
}
|
||||
const state = record.run.delivery?.state;
|
||||
if (state === 'accepted' || state === 'consumed') {
|
||||
const reply = await commands.readReply(commandId);
|
||||
if (reply !== null) {
|
||||
const parts = await this.commitCommandReply(
|
||||
request,
|
||||
reply.text,
|
||||
reply.outcome === 'refused',
|
||||
model,
|
||||
journaled,
|
||||
);
|
||||
// A call consumes only the reply it commits now: one already
|
||||
// answered may have read the not-sent or unknown evidence before
|
||||
// the relay's reply landed, and never read this one.
|
||||
if (!journaled?.has(callId)) await commands.commitConsumed(commandId);
|
||||
return parts;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
try {
|
||||
if (state === 'planned') await commands.settle(commandId, 'cancelled');
|
||||
if (state === 'accepting') await commands.settle(commandId, 'unknown');
|
||||
} catch (cause) {
|
||||
if (cause instanceof ManagedSessionConflictError) continue;
|
||||
throw cause;
|
||||
}
|
||||
const parts = convertToFunctionErrorResponse(
|
||||
name,
|
||||
callId,
|
||||
[],
|
||||
sessionCommandEvidenceText(
|
||||
// The accepted/consumed arm above returned already; what remains
|
||||
// of the delivery vocabulary is exactly the settle-relevant five.
|
||||
state as
|
||||
| 'planned'
|
||||
| 'accepting'
|
||||
| 'rejected'
|
||||
| 'cancelled'
|
||||
| 'unknown',
|
||||
),
|
||||
);
|
||||
if (!journaled?.has(callId))
|
||||
await this.commit('tool_result', parts, model);
|
||||
return parts;
|
||||
}
|
||||
throw new Error(`Session command ${commandId} never settled.`);
|
||||
}
|
||||
|
||||
/** A long board answer folds to the inline bound, as the team tools'. */
|
||||
private async commitCommandReply(
|
||||
request: { call: Pick<ToolCallRequestInfo, 'name' | 'callId'> },
|
||||
text: string,
|
||||
failed: boolean,
|
||||
model: string,
|
||||
journaled?: ReadonlySet<string>,
|
||||
): Promise<Part[]> {
|
||||
const { name, callId } = request.call;
|
||||
const make = (each: string): Part[] =>
|
||||
failed
|
||||
? convertToFunctionErrorResponse(name, callId, [], each)
|
||||
: convertToFunctionResponse(name, callId, [{ text: each }]);
|
||||
let parts = make(text);
|
||||
for (
|
||||
let head = Math.floor(text.length / 2);
|
||||
!this.messageFitsInline('tool_result', parts, model);
|
||||
head = Math.floor(head / 2)
|
||||
) {
|
||||
if (head === 0) throw new Error('Command answer cannot be recorded.');
|
||||
parts = make(
|
||||
text.slice(0, head) +
|
||||
'\n… (truncated to fit the Session record; narrow task_list with its filters)',
|
||||
);
|
||||
}
|
||||
if (!journaled?.has(callId)) await this.commit('tool_result', parts, model);
|
||||
return parts;
|
||||
}
|
||||
|
||||
/**
|
||||
* The recovered arm of the command wait (decision 4): re-enter the poll
|
||||
* the dead owner's acceptSessionCommand ran, for every run the
|
||||
* checkpoint still owes, with what is left of the bound measured from
|
||||
* the entry's own commit. A restart lost the Turn's in-memory mount,
|
||||
* so the lease the checkpoint's flag names comes back first, and the
|
||||
* journaled set keeps a replayed fold silent — the resolve still runs
|
||||
* so the wait advances. A call already answered never consumes: what
|
||||
* it read is the journaled answer, which may predate the reply.
|
||||
*/
|
||||
async resumeCommandWaitRuns(
|
||||
runs: readonly HarnessCommandWaitRun[],
|
||||
holdsWorkspaceMount: boolean,
|
||||
model: string,
|
||||
signal: AbortSignal,
|
||||
): Promise<void> {
|
||||
const commands = this.commands;
|
||||
if (commands === undefined)
|
||||
throw new Error('Recovered command wait has no command funnel.');
|
||||
if (holdsWorkspaceMount && !this.acquired) {
|
||||
await this.acquire(false, signal);
|
||||
}
|
||||
const journaled = await journaledToolResultIds(this.session, this.promptId);
|
||||
await this.syncCommandTimedOutFromDurable();
|
||||
for (const run of runs) {
|
||||
if (run.folded) continue;
|
||||
const request = {
|
||||
call: { name: run.toolName, callId: run.functionCallId },
|
||||
};
|
||||
const anchor = commands.outboundCommittedAt(run.commandId) ?? Date.now();
|
||||
const deadline = anchor + COMMAND_WAIT_MS;
|
||||
for (;;) {
|
||||
const record = commands.command(run.commandId);
|
||||
if (this.commandTimedOut) {
|
||||
if (record === undefined) {
|
||||
// A call the lock refused before it sent anything: the same
|
||||
// fixed answer the live arm gives every later call.
|
||||
const locked = convertToFunctionErrorResponse(
|
||||
run.toolName,
|
||||
run.functionCallId,
|
||||
[],
|
||||
SESSION_COMMAND_LEADER_UNANSWERING_TEXT,
|
||||
);
|
||||
if (!journaled.has(run.functionCallId))
|
||||
await this.commit('tool_result', locked, model);
|
||||
} else if (!journaled.has(run.functionCallId)) {
|
||||
// The call whose own wait timed out: the dead owner stamped
|
||||
// the lock at detection and may have died before settling
|
||||
// it. Its journal answers it — a planned entry cancelled as
|
||||
// not sent, a handed-over one unknown — never the later
|
||||
// calls' refusal, or the entry stays live for the relay to
|
||||
// deliver after the wait has ended. A call already answered
|
||||
// was settled first (the timeout arm settles, then answers),
|
||||
// so a reply the relay copied in since is not what it read
|
||||
// and is never consumed: it only folds.
|
||||
await this.answerCommandByJournal(
|
||||
commands,
|
||||
request,
|
||||
run.commandId,
|
||||
model,
|
||||
journaled,
|
||||
);
|
||||
}
|
||||
await this.harness.resolveAwaitCommand(run.commandId, {
|
||||
timedOut: true,
|
||||
});
|
||||
break;
|
||||
}
|
||||
if (record === undefined) {
|
||||
// The checkpoint names a call whose outbound was never
|
||||
// committed — the dead owner exited between the two steps,
|
||||
// and no process will ever commit it now. This answer holds
|
||||
// under any signal: a cancelled recovery must not fall into
|
||||
// arm that would try (and fail) to settle a record that does
|
||||
// not exist.
|
||||
const unsent = convertToFunctionErrorResponse(
|
||||
run.toolName,
|
||||
run.functionCallId,
|
||||
[],
|
||||
sessionCommandEvidenceText('planned'),
|
||||
);
|
||||
if (!journaled.has(run.functionCallId))
|
||||
await this.commit('tool_result', unsent, model);
|
||||
await this.harness.resolveAwaitCommand(run.commandId);
|
||||
break;
|
||||
}
|
||||
if (signal.aborted) {
|
||||
await this.answerCommandByJournal(
|
||||
commands,
|
||||
request,
|
||||
run.commandId,
|
||||
model,
|
||||
journaled,
|
||||
);
|
||||
await this.harness.resolveAwaitCommand(run.commandId);
|
||||
break;
|
||||
}
|
||||
const state = record.run.delivery?.state;
|
||||
if (state === 'accepted' || state === 'consumed') {
|
||||
const reply = await commands.readReply(run.commandId);
|
||||
if (reply !== null) {
|
||||
await this.commitCommandReply(
|
||||
request,
|
||||
reply.text,
|
||||
reply.outcome === 'refused',
|
||||
model,
|
||||
journaled,
|
||||
);
|
||||
// The same rule as the journal arm: only the reply committed
|
||||
// now is the one the call read.
|
||||
if (!journaled.has(run.functionCallId))
|
||||
await commands.commitConsumed(run.commandId);
|
||||
await this.harness.resolveAwaitCommand(run.commandId);
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (
|
||||
state === 'rejected' ||
|
||||
state === 'cancelled' ||
|
||||
state === 'unknown'
|
||||
) {
|
||||
await this.answerCommandByJournal(
|
||||
commands,
|
||||
request,
|
||||
run.commandId,
|
||||
model,
|
||||
journaled,
|
||||
);
|
||||
await this.harness.resolveAwaitCommand(run.commandId);
|
||||
break;
|
||||
}
|
||||
if (Date.now() >= deadline) {
|
||||
this.commandTimedOut = true;
|
||||
// The lock is stamped at detection time — before the journal
|
||||
// arm's next commit, so a crash between the settle and the
|
||||
// tool_result write can never un-lock it.
|
||||
await this.harness.markCommandWaitTimeout();
|
||||
await this.answerCommandByJournal(
|
||||
commands,
|
||||
request,
|
||||
run.commandId,
|
||||
model,
|
||||
journaled,
|
||||
);
|
||||
await this.harness.resolveAwaitCommand(run.commandId, {
|
||||
timedOut: true,
|
||||
});
|
||||
break;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The foreground arm: the call's answer always re-derives from the
|
||||
* committed chain — a crashed Session resumes into this same wait and
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@
|
|||
"schedule": null,
|
||||
"automation_run": "automation_run",
|
||||
"session_message": null,
|
||||
"session_command": null,
|
||||
"team_state": null,
|
||||
"team_task": null,
|
||||
"team_message": null,
|
||||
|
|
|
|||
|
|
@ -31,6 +31,7 @@
|
|||
"team_message",
|
||||
"team_plan",
|
||||
"session_message",
|
||||
"session_command",
|
||||
"session_metadata",
|
||||
"file_history",
|
||||
"session_source"
|
||||
|
|
|
|||
|
|
@ -62,6 +62,7 @@
|
|||
"team_message",
|
||||
"team_plan",
|
||||
"session_message",
|
||||
"session_command",
|
||||
"session_metadata",
|
||||
"file_history",
|
||||
"session_source"
|
||||
|
|
@ -519,6 +520,7 @@
|
|||
"team_message",
|
||||
"team_plan",
|
||||
"session_message",
|
||||
"session_command",
|
||||
"session_metadata",
|
||||
"file_history",
|
||||
"session_source"
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -67,6 +67,11 @@ import {
|
|||
isSessionMessageSuccessor,
|
||||
parseSessionMessage,
|
||||
} from './managed-session-message-record.js';
|
||||
import {
|
||||
isSessionCommandStart,
|
||||
isSessionCommandSuccessor,
|
||||
parseSessionCommand,
|
||||
} from './managed-session-command-record.js';
|
||||
import {
|
||||
isTeamMessageStart,
|
||||
isTeamMessageSuccessor,
|
||||
|
|
@ -279,6 +284,15 @@ export const MANAGED_EXTENSION_RECORD_BODIES: Readonly<
|
|||
isStart: isSessionMessageStart,
|
||||
isSuccessor: isSessionMessageSuccessor,
|
||||
}),
|
||||
session_command: Object.freeze({
|
||||
taskKindOf: () => null,
|
||||
parse: (value: unknown) => {
|
||||
const record = parseSessionCommand(value);
|
||||
return { record, recordId: record.commandId, run: record.run };
|
||||
},
|
||||
isStart: isSessionCommandStart,
|
||||
isSuccessor: isSessionCommandSuccessor,
|
||||
}),
|
||||
team_state: Object.freeze({
|
||||
taskKindOf: () => null,
|
||||
parse: (value: unknown) => {
|
||||
|
|
|
|||
|
|
@ -24,6 +24,12 @@ import {
|
|||
type HarnessAgentWaitGroup,
|
||||
type HarnessCheckpointV1,
|
||||
} from './managed-harness-checkpoint.js';
|
||||
import {
|
||||
createAwaitCommandHarnessCheckpoint,
|
||||
createConsumedCommandWaitHarnessCheckpoint,
|
||||
harnessCommandLocked,
|
||||
type HarnessCommandWaitGroup,
|
||||
} from './managed-harness-checkpoint.js';
|
||||
import {
|
||||
ManagedSessionRecordError,
|
||||
type ManagedSessionDurableRef,
|
||||
|
|
@ -1211,3 +1217,480 @@ describe('harness checkpoint v1', () => {
|
|||
expect(parseHarnessCheckpointV1(waitBytes).agentWait?.runs).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
function commandWaitOf(
|
||||
commandIds: readonly string[],
|
||||
holdsWorkspaceMount = false,
|
||||
): HarnessCommandWaitGroup {
|
||||
return {
|
||||
runs: commandIds.map((commandId, index) => ({
|
||||
commandId,
|
||||
functionCallId: `fc-${index + 1}`,
|
||||
toolName: 'task_update',
|
||||
modelMessageId: 'msg-1',
|
||||
folded: false,
|
||||
})),
|
||||
holdsWorkspaceMount,
|
||||
};
|
||||
}
|
||||
|
||||
describe('command wait', () => {
|
||||
it('round-trips an await_command checkpoint and its folded advancement', () => {
|
||||
const wait = createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: commandWaitOf(['cmd_1', 'cmd_2'], true),
|
||||
});
|
||||
expect(wait.continuation.phase).toBe('await_command');
|
||||
expect(wait.approval).toBeNull();
|
||||
expect(wait.agentWait).toBeNull();
|
||||
expect(wait.commandWait?.runs).toHaveLength(2);
|
||||
expect(wait.commandWait?.holdsWorkspaceMount).toBe(true);
|
||||
expect(parseHarnessCheckpointV1(bytesOf(wait))).toEqual(wait);
|
||||
|
||||
const partial = createConsumedCommandWaitHarnessCheckpoint({
|
||||
previous: wait,
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
commandId: 'cmd_1',
|
||||
});
|
||||
expect(partial.continuation.phase).toBe('await_command');
|
||||
expect(partial.commandWait?.runs).toMatchObject([
|
||||
{ commandId: 'cmd_1', folded: true },
|
||||
{ commandId: 'cmd_2', folded: false },
|
||||
]);
|
||||
expect(partial.commandWait?.holdsWorkspaceMount).toBe(true);
|
||||
expect(parseHarnessCheckpointV1(bytesOf(partial))).toEqual(partial);
|
||||
|
||||
const advanced = createConsumedCommandWaitHarnessCheckpoint({
|
||||
previous: partial,
|
||||
checkpointId: 'ckpt-7',
|
||||
coveredSequence: 6,
|
||||
previousCheckpointId: 'ckpt-6',
|
||||
commandId: 'cmd_2',
|
||||
});
|
||||
expect(advanced.continuation.phase).toBe('model_output_committed');
|
||||
expect(advanced.commandWait?.runs.every((run) => run.folded)).toBe(true);
|
||||
expect(parseHarnessCheckpointV1(bytesOf(advanced))).toEqual(advanced);
|
||||
});
|
||||
|
||||
// Decision 4: the fold that carried the timeout stamps the Turn's lock,
|
||||
// and every later fold of the same wait carries it — a replacement
|
||||
// owner re-arms from it.
|
||||
it('carries the timeout lock on every fold after the one that stamped it', () => {
|
||||
const wait = createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: commandWaitOf(['cmd_1', 'cmd_2']),
|
||||
});
|
||||
expect(wait.commandLock).toBeNull();
|
||||
const stamped = createConsumedCommandWaitHarnessCheckpoint({
|
||||
previous: wait,
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
commandId: 'cmd_1',
|
||||
timedOut: true,
|
||||
});
|
||||
expect(stamped.continuation.phase).toBe('await_command');
|
||||
expect(stamped.commandLock).toEqual({ turnId: 'turn-1' });
|
||||
const advanced = createConsumedCommandWaitHarnessCheckpoint({
|
||||
previous: stamped,
|
||||
checkpointId: 'ckpt-7',
|
||||
coveredSequence: 6,
|
||||
previousCheckpointId: 'ckpt-6',
|
||||
commandId: 'cmd_2',
|
||||
});
|
||||
expect(advanced.continuation.phase).toBe('model_output_committed');
|
||||
expect(advanced.commandLock).toEqual({ turnId: 'turn-1' });
|
||||
expect(parseHarnessCheckpointV1(bytesOf(advanced))).toEqual(advanced);
|
||||
});
|
||||
|
||||
// The lock is the Turn's, not the wait group's: the Turn's next command
|
||||
// batch replaces the group and its Runtime work clears it, and neither
|
||||
// may unlock the Turn. The Turn's end drops it.
|
||||
it("keeps the timeout lock through the Turn's later checkpoints and drops it when the Turn ends", () => {
|
||||
const wait = createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
});
|
||||
const locked = createConsumedCommandWaitHarnessCheckpoint({
|
||||
previous: wait,
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
commandId: 'cmd_1',
|
||||
timedOut: true,
|
||||
});
|
||||
expect(harnessCommandLocked(locked, 'turn-1')).toBe(true);
|
||||
const nextBatch = createAwaitCommandHarnessCheckpoint({
|
||||
previous: locked,
|
||||
checkpointId: 'ckpt-7',
|
||||
coveredSequence: 6,
|
||||
previousCheckpointId: 'ckpt-6',
|
||||
attempt: null,
|
||||
commandWait: commandWaitOf(['cmd_2']),
|
||||
});
|
||||
expect(nextBatch.commandWait?.runs.map((run) => run.commandId)).toEqual([
|
||||
'cmd_2',
|
||||
]);
|
||||
expect(harnessCommandLocked(nextBatch, 'turn-1')).toBe(true);
|
||||
expect(
|
||||
harnessCommandLocked(
|
||||
parseHarnessCheckpointV1(bytesOf(nextBatch)),
|
||||
'turn-1',
|
||||
),
|
||||
).toBe(true);
|
||||
const runtime = createAwaitRuntimeHarnessCheckpoint({
|
||||
previous: locked,
|
||||
checkpointId: 'ckpt-7',
|
||||
coveredSequence: 6,
|
||||
previousCheckpointId: 'ckpt-6',
|
||||
attempt: committedAttempt(),
|
||||
...inProgressRuntime(),
|
||||
});
|
||||
expect(runtime.commandWait).toBeNull();
|
||||
expect(harnessCommandLocked(runtime, 'turn-1')).toBe(true);
|
||||
expect(
|
||||
harnessCommandLocked(
|
||||
parseHarnessCheckpointV1(bytesOf(runtime)),
|
||||
'turn-1',
|
||||
),
|
||||
).toBe(true);
|
||||
// The ready checkpoint the settled Turn writes under its own id ends
|
||||
// it too, so an idle Session carries no lock key.
|
||||
const settled = createNextTurnReadyHarnessCheckpoint({
|
||||
previous: locked,
|
||||
checkpointId: 'ckpt-7',
|
||||
coveredSequence: 6,
|
||||
previousCheckpointId: 'ckpt-6',
|
||||
activationId: 'act-1',
|
||||
turnId: 'turn-1',
|
||||
promptId: null,
|
||||
});
|
||||
expect(settled.commandLock).toBeNull();
|
||||
expect(
|
||||
Object.keys(JSON.parse(bytesOf(settled).toString('utf8'))),
|
||||
).not.toContain('commandLock');
|
||||
const nextTurn = createNextTurnReadyHarnessCheckpoint({
|
||||
previous: locked,
|
||||
checkpointId: 'ckpt-7',
|
||||
coveredSequence: 6,
|
||||
previousCheckpointId: 'ckpt-6',
|
||||
activationId: 'act-1',
|
||||
turnId: 'turn-2',
|
||||
promptId: null,
|
||||
});
|
||||
expect(nextTurn.commandLock).toBeNull();
|
||||
expect(harnessCommandLocked(nextTurn, 'turn-2')).toBe(false);
|
||||
});
|
||||
|
||||
it('omits the commandLock key while the Turn is unlocked and rejects a malformed lock', () => {
|
||||
const bytes = bytesOf(seed());
|
||||
expect(Object.keys(JSON.parse(bytes.toString('utf8')))).not.toContain(
|
||||
'commandLock',
|
||||
);
|
||||
expect(parseHarnessCheckpointV1(bytes).commandLock).toBeNull();
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(seed(), (value) => {
|
||||
value['commandLock'] = { turnId: 'turn-1', timedOut: true };
|
||||
}),
|
||||
),
|
||||
).toThrow(/commandLock/);
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(seed(), (value) => {
|
||||
value['commandLock'] = { turnId: '' };
|
||||
}),
|
||||
),
|
||||
).toThrow(/commandLock\.turnId/);
|
||||
// The group no longer holds the lock: a stale stamp there is refused.
|
||||
const wait = createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
});
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(wait, (value) => {
|
||||
(value['commandWait'] as Record<string, unknown>)['timedOut'] = true;
|
||||
}),
|
||||
),
|
||||
).toThrow(/commandWait/);
|
||||
});
|
||||
|
||||
it('treats a checkpoint without a commandWait key as carrying none', () => {
|
||||
const bytes = mutate(seed(), (value) => {
|
||||
delete value['commandWait'];
|
||||
});
|
||||
expect(parseHarnessCheckpointV1(bytes).commandWait).toBeNull();
|
||||
});
|
||||
|
||||
it('rejects an await_command checkpoint against the other wait families', () => {
|
||||
expect(() =>
|
||||
createAwaitCommandHarnessCheckpoint({
|
||||
previous: createAwaitActionHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
approval: requestedUserApproval(),
|
||||
}),
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
attempt: null,
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
}),
|
||||
).toThrow(/cannot keep a requested approval/);
|
||||
expect(() =>
|
||||
createAwaitCommandHarnessCheckpoint({
|
||||
previous: createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
}),
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
attempt: null,
|
||||
commandWait: commandWaitOf(['cmd_2']),
|
||||
}),
|
||||
).toThrow(/cannot replace an unfinished command wait/);
|
||||
expect(() =>
|
||||
createAwaitCommandHarnessCheckpoint({
|
||||
previous: createAwaitAgentHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
agentWait: agentWaitOf(['run-1']),
|
||||
}),
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
attempt: null,
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
}),
|
||||
).toThrow(/cannot replace an unfinished agent wait/);
|
||||
expect(() =>
|
||||
createAwaitCommandHarnessCheckpoint({
|
||||
previous: createAwaitRuntimeHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
...inProgressRuntime(),
|
||||
}),
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
attempt: null,
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
}),
|
||||
).toThrow(/cannot carry in-flight Runtime work/);
|
||||
expect(() =>
|
||||
createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: null,
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
}),
|
||||
).toThrow(/requires the attempt/);
|
||||
});
|
||||
|
||||
it('replaces an all-folded carried group, never an unfinished wait', () => {
|
||||
const first = createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
});
|
||||
const folded = createConsumedCommandWaitHarnessCheckpoint({
|
||||
previous: first,
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
commandId: 'cmd_1',
|
||||
});
|
||||
const next = createAwaitCommandHarnessCheckpoint({
|
||||
previous: folded,
|
||||
checkpointId: 'ckpt-7',
|
||||
coveredSequence: 6,
|
||||
previousCheckpointId: 'ckpt-6',
|
||||
attempt: null,
|
||||
commandWait: commandWaitOf(['cmd_2', 'cmd_3']),
|
||||
});
|
||||
expect(next.continuation.phase).toBe('await_command');
|
||||
expect(next.commandWait?.runs.map((run) => run.commandId)).toEqual([
|
||||
'cmd_2',
|
||||
'cmd_3',
|
||||
]);
|
||||
});
|
||||
|
||||
it('write-side guards refuse the two rejected shapes', () => {
|
||||
const wait = createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
});
|
||||
expect(() =>
|
||||
createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: { runs: [], holdsWorkspaceMount: false },
|
||||
}),
|
||||
).toThrow(/requires at least one run/);
|
||||
expect(() =>
|
||||
createModelOutputCommittedHarnessCheckpoint({
|
||||
previous: wait,
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
}),
|
||||
).toThrow(/cannot carry an unfolded command wait/);
|
||||
const folded = createConsumedCommandWaitHarnessCheckpoint({
|
||||
previous: wait,
|
||||
checkpointId: 'ckpt-6',
|
||||
coveredSequence: 5,
|
||||
previousCheckpointId: 'ckpt-5',
|
||||
commandId: 'cmd_1',
|
||||
});
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(folded, (value) => {
|
||||
value['commandWait'] = { runs: [], holdsWorkspaceMount: false };
|
||||
}),
|
||||
),
|
||||
).toThrow(/requires at least one run/);
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(folded, (value) => {
|
||||
value['commandWait'] = JSON.parse(
|
||||
JSON.stringify({
|
||||
runs: folded.commandWait!.runs,
|
||||
holdsWorkspaceMount: 'maybe',
|
||||
}),
|
||||
);
|
||||
}),
|
||||
),
|
||||
).toThrow(/holdsWorkspaceMount must be a boolean/);
|
||||
expect(
|
||||
createModelOutputCommittedHarnessCheckpoint({
|
||||
previous: folded,
|
||||
checkpointId: 'ckpt-7',
|
||||
coveredSequence: 6,
|
||||
previousCheckpointId: 'ckpt-6',
|
||||
}).commandWait?.runs,
|
||||
).toMatchObject([{ commandId: 'cmd_1', folded: true }]);
|
||||
});
|
||||
|
||||
it('rejects the command group at every other phase too', () => {
|
||||
const group = commandWaitOf(['cmd_1']);
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(seed(), (value) => {
|
||||
value['commandWait'] = JSON.parse(JSON.stringify(group));
|
||||
}),
|
||||
),
|
||||
).toThrow(/before_model cannot carry a command wait/);
|
||||
const awaited = createAwaitActionHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
approval: requestedUserApproval(),
|
||||
});
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(awaited, (value) => {
|
||||
value['commandWait'] = JSON.parse(JSON.stringify(group));
|
||||
}),
|
||||
),
|
||||
).toThrow(/await_action cannot carry a command wait/);
|
||||
const wait = createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: group,
|
||||
});
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(wait, (value) => {
|
||||
value['approval'] = JSON.parse(
|
||||
JSON.stringify(requestedUserApproval()),
|
||||
);
|
||||
}),
|
||||
),
|
||||
).toThrow(/cannot keep a requested approval/);
|
||||
expect(() =>
|
||||
parseHarnessCheckpointV1(
|
||||
mutate(wait, (value) => {
|
||||
value['commandWait'] = JSON.parse(
|
||||
JSON.stringify({
|
||||
runs: [{ ...group.runs[0], folded: true }],
|
||||
holdsWorkspaceMount: false,
|
||||
}),
|
||||
);
|
||||
}),
|
||||
),
|
||||
).toThrow(/all-folded command wait/);
|
||||
});
|
||||
|
||||
it('omits the commandWait key whenever no group rides the checkpoint', () => {
|
||||
const bytes = bytesOf(seed());
|
||||
expect(Object.keys(JSON.parse(bytes.toString('utf8')))).not.toContain(
|
||||
'commandWait',
|
||||
);
|
||||
expect(parseHarnessCheckpointV1(bytes).commandWait).toBeNull();
|
||||
const wait = createAwaitCommandHarnessCheckpoint({
|
||||
previous: seed(),
|
||||
checkpointId: 'ckpt-5',
|
||||
coveredSequence: 4,
|
||||
previousCheckpointId: 'ckpt-4',
|
||||
attempt: committedAttempt(),
|
||||
commandWait: commandWaitOf(['cmd_1']),
|
||||
});
|
||||
const waitBytes = bytesOf(wait);
|
||||
expect(Object.keys(JSON.parse(waitBytes.toString('utf8')))).toContain(
|
||||
'commandWait',
|
||||
);
|
||||
expect(parseHarnessCheckpointV1(waitBytes).commandWait?.runs).toHaveLength(
|
||||
1,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ export const HARNESS_CHECKPOINT_PHASES = [
|
|||
'await_action',
|
||||
'await_runtime',
|
||||
'await_agent',
|
||||
'await_command',
|
||||
'results_ready',
|
||||
'turn_settled',
|
||||
] as const;
|
||||
|
|
@ -233,6 +234,49 @@ export interface HarnessAgentWaitGroup {
|
|||
readonly runs: readonly HarnessAgentWaitRun[];
|
||||
}
|
||||
|
||||
/**
|
||||
* One member command call this Turn waits on (H4e-b3). The wait rides its
|
||||
* own group rather than `tools.items`, same lifetime rule as the agent
|
||||
* wait: the durable evidence is the command's outbox identity plus the
|
||||
* call's journal identity, and a resume re-derives the round from the
|
||||
* journal itself, so no call-site position is carried. `folded` marks the
|
||||
* reply fold done; the call stays listed — never filtered out — until a
|
||||
* later all-folded group is replaced by a new command wait or the Turn
|
||||
* ends, so a second crash re-enters idempotently.
|
||||
*/
|
||||
export interface HarnessCommandWaitRun {
|
||||
readonly commandId: string;
|
||||
readonly functionCallId: string;
|
||||
readonly toolName: string;
|
||||
readonly modelMessageId: string;
|
||||
readonly folded: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* The parked member-command waits of one Turn (safety point:
|
||||
* await_command). `holdsWorkspaceMount` answers whether the Turn keeps the
|
||||
* Workspace mount it took in an earlier batch, so a resumed wait takes the
|
||||
* Runtime lease back when it did — `execute` never needs the mount, so
|
||||
* holding it through the wait cannot deadlock with the leader.
|
||||
*/
|
||||
export interface HarnessCommandWaitGroup {
|
||||
readonly runs: readonly HarnessCommandWaitRun[];
|
||||
readonly holdsWorkspaceMount: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision 4's durable lock (H4e-b3): once one command call of a Turn
|
||||
* timed out, every later command call of that Turn is refused at once.
|
||||
* It is the Turn's state, not the wait group's — a later command batch
|
||||
* replaces the group and Runtime work clears it — so it rides every
|
||||
* checkpoint of the Turn that stamped it, and a replacement owner re-arms
|
||||
* from whichever checkpoint it resumes. A checkpoint of another Turn
|
||||
* drops it.
|
||||
*/
|
||||
export interface HarnessCommandLockGroup {
|
||||
readonly turnId: string;
|
||||
}
|
||||
|
||||
export interface HarnessOutputGroup {
|
||||
readonly llmContentRef: ManagedSessionDurableRef | null;
|
||||
readonly physicalStatus: string | null;
|
||||
|
|
@ -264,6 +308,8 @@ export interface HarnessCheckpointV1 {
|
|||
readonly runtime: HarnessRuntimeGroup | null;
|
||||
readonly approval: HarnessApprovalGroup | null;
|
||||
readonly agentWait: HarnessAgentWaitGroup | null;
|
||||
readonly commandWait: HarnessCommandWaitGroup | null;
|
||||
readonly commandLock: HarnessCommandLockGroup | null;
|
||||
readonly output: HarnessOutputGroup;
|
||||
readonly followUp: HarnessFollowUpGroup;
|
||||
}
|
||||
|
|
@ -301,6 +347,8 @@ const ROOT_KEYS = [
|
|||
'runtime',
|
||||
'approval',
|
||||
'agentWait',
|
||||
'commandWait',
|
||||
'commandLock',
|
||||
'output',
|
||||
'followUp',
|
||||
] as const;
|
||||
|
|
@ -963,6 +1011,122 @@ function parseAgentWait(
|
|||
return { runs };
|
||||
}
|
||||
|
||||
function parseCommandWaitRun(
|
||||
value: ManagedSessionJsonValue,
|
||||
label: string,
|
||||
): HarnessCommandWaitRun {
|
||||
const record = object(value, label);
|
||||
assertNoUnknownKeys(
|
||||
record,
|
||||
['commandId', 'functionCallId', 'toolName', 'modelMessageId', 'folded'],
|
||||
label,
|
||||
);
|
||||
if (typeof record['folded'] !== 'boolean') {
|
||||
fail(`${label}.folded must be a boolean.`);
|
||||
}
|
||||
return {
|
||||
commandId: assertManagedSessionStableId(
|
||||
record['commandId'],
|
||||
`${label}.commandId`,
|
||||
),
|
||||
functionCallId: assertManagedSessionStableId(
|
||||
record['functionCallId'],
|
||||
`${label}.functionCallId`,
|
||||
),
|
||||
toolName: assertManagedSessionStableId(
|
||||
record['toolName'],
|
||||
`${label}.toolName`,
|
||||
),
|
||||
modelMessageId: assertManagedSessionStableId(
|
||||
record['modelMessageId'],
|
||||
`${label}.modelMessageId`,
|
||||
),
|
||||
folded: record['folded'],
|
||||
};
|
||||
}
|
||||
|
||||
function parseCommandWait(
|
||||
value: ManagedSessionJsonValue | undefined,
|
||||
): HarnessCommandWaitGroup | null {
|
||||
// Checkpoints committed before this group existed carry no key at all;
|
||||
// their writers could not have minted a command wait, so absent means
|
||||
// null — the same mixed-version rule as the agent wait.
|
||||
if (value === null || value === undefined) return null;
|
||||
const record = object(value, 'commandWait');
|
||||
assertNoUnknownKeys(record, ['runs', 'holdsWorkspaceMount'], 'commandWait');
|
||||
const runs = jsonArray(record['runs'], 'commandWait.runs').map(
|
||||
(item, index) => parseCommandWaitRun(item, `commandWait.runs[${index}]`),
|
||||
);
|
||||
if (runs.length === 0) {
|
||||
fail('command wait requires at least one run.');
|
||||
}
|
||||
uniqueIds(
|
||||
runs.map((run) => run.commandId),
|
||||
'commandWait.runs.commandId',
|
||||
);
|
||||
uniqueIds(
|
||||
runs.map((run) => run.functionCallId),
|
||||
'commandWait.runs.functionCallId',
|
||||
);
|
||||
const mount = record['holdsWorkspaceMount'];
|
||||
if (typeof mount !== 'boolean') {
|
||||
fail('commandWait.holdsWorkspaceMount must be a boolean.');
|
||||
}
|
||||
return { runs, holdsWorkspaceMount: mount };
|
||||
}
|
||||
|
||||
function parseCommandLock(
|
||||
value: ManagedSessionJsonValue | undefined,
|
||||
): HarnessCommandLockGroup | null {
|
||||
// Absent means unlocked: only a Turn that timed out writes the key.
|
||||
if (value === null || value === undefined) return null;
|
||||
const record = object(value, 'commandLock');
|
||||
assertNoUnknownKeys(record, ['turnId'], 'commandLock');
|
||||
return {
|
||||
turnId: assertManagedSessionStableId(
|
||||
record['turnId'],
|
||||
'commandLock.turnId',
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The lock `previous` hands to its Turn's next checkpoint: kept within
|
||||
* the Turn that stamped it, dropped by any other Turn's checkpoint.
|
||||
*/
|
||||
function carriedCommandLock(
|
||||
previous: HarnessCheckpointV1,
|
||||
): HarnessCommandLockGroup | null {
|
||||
// An in-memory checkpoint shaped before the key existed carries none.
|
||||
const lock = previous.commandLock ?? null;
|
||||
return lock !== null && lock.turnId === previous.identity.turnId
|
||||
? lock
|
||||
: null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decision 4's lock for the checkpoint's own Turn. Only a Turn-bound
|
||||
* checkpoint can carry it: a command wait always names its Turn.
|
||||
*/
|
||||
export function harnessCommandLockOf(
|
||||
checkpoint: HarnessCheckpointV1,
|
||||
): HarnessCommandLockGroup {
|
||||
if (checkpoint.identity.turnId === null) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'a command timeout lock requires the Turn it binds.',
|
||||
);
|
||||
}
|
||||
return { turnId: checkpoint.identity.turnId };
|
||||
}
|
||||
|
||||
/** Whether decision 4's lock binds the Turn `turnId`. */
|
||||
export function harnessCommandLocked(
|
||||
checkpoint: HarnessCheckpointV1,
|
||||
turnId: string,
|
||||
): boolean {
|
||||
return checkpoint.commandLock?.turnId === turnId;
|
||||
}
|
||||
|
||||
function parseParentHistory(
|
||||
value: ManagedSessionJsonValue | undefined,
|
||||
): HarnessOutputGroup['parentHistory'] {
|
||||
|
|
@ -1116,6 +1280,9 @@ function assertPhaseShape(checkpoint: HarnessCheckpointV1): void {
|
|||
if (checkpoint.agentWait !== null) {
|
||||
fail('before_model cannot carry an agent wait.');
|
||||
}
|
||||
if (checkpoint.commandWait !== null) {
|
||||
fail('before_model cannot carry a command wait.');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -1145,6 +1312,13 @@ function assertPhaseShape(checkpoint: HarnessCheckpointV1): void {
|
|||
) {
|
||||
fail('model_output_committed cannot carry an unconsumed agent wait.');
|
||||
}
|
||||
// Same rule for a folded command wait (H4e-b3).
|
||||
if (
|
||||
checkpoint.commandWait !== null &&
|
||||
checkpoint.commandWait.runs.some((run) => !run.folded)
|
||||
) {
|
||||
fail('model_output_committed cannot carry an unfolded command wait.');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -1164,6 +1338,9 @@ function assertPhaseShape(checkpoint: HarnessCheckpointV1): void {
|
|||
if (checkpoint.agentWait !== null) {
|
||||
fail('await_action cannot carry an agent wait.');
|
||||
}
|
||||
if (checkpoint.commandWait !== null) {
|
||||
fail('await_action cannot carry a command wait.');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -1177,6 +1354,9 @@ function assertPhaseShape(checkpoint: HarnessCheckpointV1): void {
|
|||
if (checkpoint.agentWait !== null) {
|
||||
fail('await_runtime cannot carry an agent wait.');
|
||||
}
|
||||
if (checkpoint.commandWait !== null) {
|
||||
fail('await_runtime cannot carry a command wait.');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -1201,6 +1381,35 @@ function assertPhaseShape(checkpoint: HarnessCheckpointV1): void {
|
|||
if (inFlightTools || dispatchRuntime) {
|
||||
fail('await_agent cannot carry in-flight Runtime work.');
|
||||
}
|
||||
if (checkpoint.commandWait !== null) {
|
||||
fail('await_agent cannot carry a command wait.');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// The command wait (H4e-b3): a member's batch parked on the leader's
|
||||
// replies. It obeys the same exclusions as the agent wait, and may keep
|
||||
// the Workspace mount only as a flag — the mount is the Turn's, taken in
|
||||
// an earlier batch, so it never shows as in-flight Runtime work here.
|
||||
if (phase === 'await_command') {
|
||||
if (
|
||||
checkpoint.commandWait === null ||
|
||||
checkpoint.commandWait.runs.length === 0
|
||||
) {
|
||||
fail('await_command requires the command wait runs.');
|
||||
}
|
||||
if (checkpoint.commandWait.runs.every((run) => run.folded)) {
|
||||
fail('await_command cannot keep an all-folded command wait.');
|
||||
}
|
||||
if (checkpoint.approval?.state === 'requested') {
|
||||
fail('await_command cannot keep a requested approval.');
|
||||
}
|
||||
if (inFlightTools || dispatchRuntime) {
|
||||
fail('await_command cannot carry in-flight Runtime work.');
|
||||
}
|
||||
if (checkpoint.agentWait !== null) {
|
||||
fail('await_command cannot carry an agent wait.');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -1217,6 +1426,9 @@ function assertPhaseShape(checkpoint: HarnessCheckpointV1): void {
|
|||
if (checkpoint.agentWait !== null) {
|
||||
fail('results_ready cannot carry an agent wait.');
|
||||
}
|
||||
if (checkpoint.commandWait !== null) {
|
||||
fail('results_ready cannot carry a command wait.');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -1224,10 +1436,11 @@ function assertPhaseShape(checkpoint: HarnessCheckpointV1): void {
|
|||
inFlightTools ||
|
||||
dispatchRuntime ||
|
||||
checkpoint.approval?.state === 'requested' ||
|
||||
checkpoint.agentWait !== null
|
||||
checkpoint.agentWait !== null ||
|
||||
checkpoint.commandWait !== null
|
||||
) {
|
||||
fail(
|
||||
'turn_settled cannot carry in-flight tools, a requested approval, or an agent wait.',
|
||||
'turn_settled cannot carry in-flight tools, a requested approval, an agent wait, or a command wait.',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
@ -1247,6 +1460,8 @@ function parseHarnessCheckpointObject(
|
|||
runtime: parseRuntime(record['runtime']),
|
||||
approval: parseApproval(record['approval']),
|
||||
agentWait: parseAgentWait(record['agentWait']),
|
||||
commandWait: parseCommandWait(record['commandWait']),
|
||||
commandLock: parseCommandLock(record['commandLock']),
|
||||
output: parseOutput(record['output']),
|
||||
followUp: parseFollowUp(record['followUp']),
|
||||
};
|
||||
|
|
@ -1283,6 +1498,22 @@ export function harnessCheckpointIsAgentWait(
|
|||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The checkpoint carries a live command wait (H4e-b3): still outstanding
|
||||
* at `await_command`, or the same wait a breath past its last fold at
|
||||
* `model_output_committed` with the all-folded group carried. Recovery and
|
||||
* the route's phase gate share this predicate, never their own copies.
|
||||
*/
|
||||
export function harnessCheckpointIsCommandWait(
|
||||
checkpoint: HarnessCheckpointV1,
|
||||
): boolean {
|
||||
return (
|
||||
checkpoint.continuation.phase === 'await_command' ||
|
||||
(checkpoint.continuation.phase === 'model_output_committed' &&
|
||||
checkpoint.commandWait !== null)
|
||||
);
|
||||
}
|
||||
|
||||
export function tryParseHarnessCheckpointV1(
|
||||
bytes: Buffer,
|
||||
): HarnessCheckpointParseResult {
|
||||
|
|
@ -1335,12 +1566,17 @@ export function encodeHarnessCheckpointV1(
|
|||
): Buffer {
|
||||
// Mixed-version compat (#13708): a pre-`agentWait` reader rejects
|
||||
// unknown root keys, and an absent key parses as null on both sides —
|
||||
// so the key rides only the checkpoints that actually carry the group.
|
||||
if (checkpoint.agentWait === null) {
|
||||
const { agentWait: _omitted, ...rest } = checkpoint;
|
||||
return Buffer.from(JSON.stringify(rest), 'utf8');
|
||||
// so each optional group's key rides only the checkpoints that actually
|
||||
// carry it. `commandWait` and `commandLock` (H4e-b3) follow the same
|
||||
// rule, so a pre-b3b Harness never meets a checkpoint it cannot read
|
||||
// until a member command is used.
|
||||
const record: Partial<Record<keyof HarnessCheckpointV1, unknown>> = {
|
||||
...checkpoint,
|
||||
};
|
||||
for (const key of ['agentWait', 'commandWait', 'commandLock'] as const) {
|
||||
if (record[key] === null) delete record[key];
|
||||
}
|
||||
return Buffer.from(JSON.stringify(checkpoint), 'utf8');
|
||||
return Buffer.from(JSON.stringify(record), 'utf8');
|
||||
}
|
||||
|
||||
function formatHarnessIdentity(input: {
|
||||
|
|
@ -1445,6 +1681,8 @@ export function createInitialHarnessCheckpoint(input: {
|
|||
runtime: null,
|
||||
approval: null,
|
||||
agentWait: null,
|
||||
commandWait: null,
|
||||
commandLock: null,
|
||||
output: {
|
||||
llmContentRef: null,
|
||||
physicalStatus: null,
|
||||
|
|
@ -1502,6 +1740,10 @@ export function createNextTurnReadyHarnessCheckpoint(input: {
|
|||
runtime: null,
|
||||
approval: null,
|
||||
agentWait: null,
|
||||
commandWait: null,
|
||||
// The Turn that stamped decision 4's lock has finished, whichever Turn
|
||||
// comes next.
|
||||
commandLock: null,
|
||||
output: {
|
||||
llmContentRef: null,
|
||||
physicalStatus: null,
|
||||
|
|
@ -1545,6 +1787,8 @@ export function createAwaitActionHarnessCheckpoint(input: {
|
|||
runtime: input.previous.runtime,
|
||||
approval: input.approval,
|
||||
agentWait: null,
|
||||
commandWait: null,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
|
|
@ -1579,6 +1823,14 @@ export function createAwaitAgentHarnessCheckpoint(input: {
|
|||
'await_agent cannot replace an unfinished agent wait.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
input.previous.commandWait !== null &&
|
||||
input.previous.commandWait.runs.some((run) => !run.folded)
|
||||
) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'await_agent cannot replace an unfinished command wait.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
input.previous.tools?.items.some((item) => item.state === 'in_progress')
|
||||
) {
|
||||
|
|
@ -1615,6 +1867,166 @@ export function createAwaitAgentHarnessCheckpoint(input: {
|
|||
runtime: input.previous.runtime,
|
||||
approval: null,
|
||||
agentWait: input.agentWait,
|
||||
commandWait: null,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The command-wait safety point (H4e-b3): one or more member command calls
|
||||
* of the batch are parked on the leader's replies. The relay owns their
|
||||
* execution; the next model request stays blocked until every call is
|
||||
* folded from the journal. A carried all-folded group from the previous
|
||||
* wait is replaced — its folds are already durable in the journal — while
|
||||
* any unfolded call makes the replacement a conflict. `holdsWorkspaceMount`
|
||||
* tells a resumed wait whether it owes the Turn's Runtime lease back.
|
||||
*/
|
||||
export function createAwaitCommandHarnessCheckpoint(input: {
|
||||
readonly previous: HarnessCheckpointV1;
|
||||
readonly checkpointId: string;
|
||||
readonly coveredSequence: number;
|
||||
readonly previousCheckpointId: string | null;
|
||||
readonly attempt: HarnessAttemptGroup | null;
|
||||
readonly commandWait: HarnessCommandWaitGroup;
|
||||
}): HarnessCheckpointV1 {
|
||||
if (input.previous.approval?.state === 'requested') {
|
||||
throw new ManagedSessionRecordError(
|
||||
'await_command cannot keep a requested approval.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
input.previous.commandWait !== null &&
|
||||
input.previous.commandWait.runs.some((run) => !run.folded)
|
||||
) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'await_command cannot replace an unfinished command wait.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
input.previous.agentWait !== null &&
|
||||
input.previous.agentWait.runs.some((run) => !run.consumed)
|
||||
) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'await_command cannot replace an unfinished agent wait.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
input.previous.tools?.items.some((item) => item.state === 'in_progress')
|
||||
) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'await_command cannot carry in-flight Runtime work.',
|
||||
);
|
||||
}
|
||||
const attempt = input.attempt ?? input.previous.attempt;
|
||||
if (attempt === null) {
|
||||
throw new ManagedSessionRecordError('await_command requires the attempt.');
|
||||
}
|
||||
if (input.commandWait.runs.length === 0) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'await_command requires at least one run.',
|
||||
);
|
||||
}
|
||||
return {
|
||||
identity: {
|
||||
...input.previous.identity,
|
||||
checkpointId: input.checkpointId,
|
||||
coveredSequence: input.coveredSequence,
|
||||
previousCheckpointId: input.previousCheckpointId,
|
||||
},
|
||||
resume: {
|
||||
...input.previous.resume,
|
||||
throughSequence: input.coveredSequence,
|
||||
},
|
||||
continuation: {
|
||||
phase: 'await_command',
|
||||
pendingEventIds: [],
|
||||
},
|
||||
attempt,
|
||||
tools: input.previous.tools,
|
||||
runtime: input.previous.runtime,
|
||||
approval: null,
|
||||
agentWait: null,
|
||||
commandWait: input.commandWait,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Marks one parked command call folded after its reply committed. The
|
||||
* wait's phase stays `await_command` while calls remain; with every call
|
||||
* folded the continuation advances to `model_output_committed` — the folded
|
||||
* replies owe the next model round, and the all-folded group rides along
|
||||
* so a second crash re-enters idempotently.
|
||||
*/
|
||||
export function createConsumedCommandWaitHarnessCheckpoint(input: {
|
||||
readonly previous: HarnessCheckpointV1;
|
||||
readonly checkpointId: string;
|
||||
readonly coveredSequence: number;
|
||||
readonly previousCheckpointId: string | null;
|
||||
readonly commandId: string;
|
||||
readonly timedOut?: boolean;
|
||||
}): HarnessCheckpointV1 {
|
||||
if (input.previous.continuation.phase !== 'await_command') {
|
||||
throw new ManagedSessionRecordError(
|
||||
'folded command wait requires an await_command checkpoint.',
|
||||
);
|
||||
}
|
||||
if (input.previous.commandWait === null) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'folded command wait requires the command wait runs.',
|
||||
);
|
||||
}
|
||||
const waited = input.previous.commandWait.runs.find(
|
||||
(run) => run.commandId === input.commandId,
|
||||
);
|
||||
if (waited === undefined) {
|
||||
throw new ManagedSessionRecordError(
|
||||
`await_command has no run ${input.commandId}.`,
|
||||
);
|
||||
}
|
||||
if (waited.folded) {
|
||||
throw new ManagedSessionRecordError(
|
||||
`command wait run ${input.commandId} is already folded.`,
|
||||
);
|
||||
}
|
||||
const runs = input.previous.commandWait.runs.map((run) =>
|
||||
run.commandId === input.commandId ? { ...run, folded: true } : run,
|
||||
);
|
||||
const allFolded = runs.every((run) => run.folded);
|
||||
return {
|
||||
identity: {
|
||||
...input.previous.identity,
|
||||
checkpointId: input.checkpointId,
|
||||
coveredSequence: input.coveredSequence,
|
||||
previousCheckpointId: input.previousCheckpointId,
|
||||
},
|
||||
resume: {
|
||||
...input.previous.resume,
|
||||
throughSequence: input.coveredSequence,
|
||||
},
|
||||
continuation: {
|
||||
phase: allFolded ? 'model_output_committed' : 'await_command',
|
||||
pendingEventIds: [],
|
||||
},
|
||||
attempt: input.previous.attempt,
|
||||
tools: input.previous.tools,
|
||||
runtime: input.previous.runtime,
|
||||
approval: null,
|
||||
agentWait: null,
|
||||
commandWait: {
|
||||
runs,
|
||||
holdsWorkspaceMount: input.previous.commandWait.holdsWorkspaceMount,
|
||||
},
|
||||
// A fold carrying the timeout stamps the Turn's lock; any other fold
|
||||
// keeps the lock the Turn already holds.
|
||||
commandLock:
|
||||
input.timedOut === true
|
||||
? harnessCommandLockOf(input.previous)
|
||||
: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
|
|
@ -1681,6 +2093,8 @@ export function createConsumedAgentWaitHarnessCheckpoint(input: {
|
|||
runtime: input.previous.runtime,
|
||||
approval: null,
|
||||
agentWait: { runs },
|
||||
commandWait: null,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
|
|
@ -1724,6 +2138,8 @@ export function createAwaitRuntimeHarnessCheckpoint(input: {
|
|||
runtime: input.runtime,
|
||||
approval: null,
|
||||
agentWait: null,
|
||||
commandWait: null,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
|
|
@ -1800,6 +2216,8 @@ export function createResultsReadyHarnessCheckpoint(input: {
|
|||
},
|
||||
approval: null,
|
||||
agentWait: null,
|
||||
commandWait: null,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
|
|
@ -1850,6 +2268,8 @@ export function createTurnSettledHarnessCheckpoint(input: {
|
|||
runtime: input.previous.runtime,
|
||||
approval: null,
|
||||
agentWait: null,
|
||||
commandWait: null,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
|
|
@ -1937,6 +2357,8 @@ export function createConsumedRuntimeResultsHarnessCheckpoint(input: {
|
|||
runtime: input.previous.runtime,
|
||||
approval: null,
|
||||
agentWait: null,
|
||||
commandWait: null,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
|
|
@ -1967,6 +2389,15 @@ export function createModelOutputCommittedHarnessCheckpoint(input: {
|
|||
'model_output_committed cannot carry an unconsumed agent wait.',
|
||||
);
|
||||
}
|
||||
const carriedCommands = input.previous.commandWait ?? null;
|
||||
if (
|
||||
carriedCommands !== null &&
|
||||
carriedCommands.runs.some((run) => !run.folded)
|
||||
) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'model_output_committed cannot carry an unfolded command wait.',
|
||||
);
|
||||
}
|
||||
return {
|
||||
identity: {
|
||||
...input.previous.identity,
|
||||
|
|
@ -1987,6 +2418,8 @@ export function createModelOutputCommittedHarnessCheckpoint(input: {
|
|||
runtime: input.previous.runtime,
|
||||
approval: null,
|
||||
agentWait: input.previous.agentWait,
|
||||
commandWait: input.previous.commandWait,
|
||||
commandLock: carriedCommandLock(input.previous),
|
||||
output: input.previous.output,
|
||||
followUp: input.previous.followUp,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -24,6 +24,7 @@ import {
|
|||
HARNESS_TURN_COMPLETE_BOUNDARY,
|
||||
parseHarnessCheckpointV1,
|
||||
type HarnessAgentWaitRun,
|
||||
type HarnessCommandWaitRun,
|
||||
} from './managed-harness-checkpoint.js';
|
||||
import {
|
||||
openManagedSession,
|
||||
|
|
@ -1830,3 +1831,252 @@ describe('agent wait', () => {
|
|||
await awaiting.close();
|
||||
});
|
||||
});
|
||||
|
||||
function commandWaitRun(
|
||||
commandId: string,
|
||||
overrides: Partial<HarnessCommandWaitRun> = {},
|
||||
): HarnessCommandWaitRun {
|
||||
return {
|
||||
commandId,
|
||||
functionCallId: 'fc-1',
|
||||
toolName: 'task_update',
|
||||
modelMessageId: 'msg-1',
|
||||
folded: false,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe('command wait', () => {
|
||||
it('commits the wait at admission and replays the identical restated set', async () => {
|
||||
const session = await open(await createWorkspace());
|
||||
const { handle, turn } = await modelOutputCommittedTurn(session);
|
||||
const boundary = await handle.commitAwaitCommand(
|
||||
[commandWaitRun('cmd_1')],
|
||||
turn,
|
||||
{ holdsWorkspaceMount: false },
|
||||
);
|
||||
expect(boundary.kind).toBe('durable_wait');
|
||||
expect(session.authority.latestCheckpoint?.boundary).toBe(
|
||||
HARNESS_DURABLE_WAIT_BOUNDARY,
|
||||
);
|
||||
const parsed = parseHarnessCheckpointV1(
|
||||
(await session.authority.readCheckpointState())!,
|
||||
);
|
||||
expect(parsed.continuation.phase).toBe('await_command');
|
||||
expect(parsed.commandWait?.runs).toMatchObject([
|
||||
{ commandId: 'cmd_1', functionCallId: 'fc-1', folded: false },
|
||||
]);
|
||||
expect(parsed.commandWait?.holdsWorkspaceMount).toBe(false);
|
||||
await expect(handle.ensureRunnable()).rejects.toMatchObject({
|
||||
reason: 'invalid_state',
|
||||
});
|
||||
await expect(handle.requestBoundary()).resolves.toMatchObject({
|
||||
kind: 'durable_wait',
|
||||
checkpointId: boundary.checkpointId,
|
||||
});
|
||||
const replayed = await handle.commitAwaitCommand(
|
||||
[commandWaitRun('cmd_1')],
|
||||
turn,
|
||||
{ holdsWorkspaceMount: false },
|
||||
);
|
||||
expect(replayed.checkpointId).toBe(boundary.checkpointId);
|
||||
await session.close();
|
||||
});
|
||||
|
||||
it('refuses a conflicting restatement, a sibling durable wait, and malformed batches', async () => {
|
||||
const session = await open(await createWorkspace());
|
||||
const { handle, turn } = await modelOutputCommittedTurn(session);
|
||||
await handle.commitAwaitCommand([commandWaitRun('cmd_1')], turn, {
|
||||
holdsWorkspaceMount: false,
|
||||
});
|
||||
await expect(
|
||||
handle.commitAwaitCommand([commandWaitRun('cmd_2')], turn, {
|
||||
holdsWorkspaceMount: false,
|
||||
}),
|
||||
).rejects.toThrow(/different commands/);
|
||||
await expect(
|
||||
handle.commitAwaitRuntime(await runtimeCommit(session)),
|
||||
).rejects.toThrow(/must resolve before/);
|
||||
await session.close();
|
||||
|
||||
const second = await open(await createWorkspace());
|
||||
const handleTwo = createManagedHarnessHandle(second);
|
||||
await handleTwo.ensureRunnable();
|
||||
await handleTwo.commitDurableWait(waitCommit(await waitRefs(second)));
|
||||
await expect(
|
||||
handleTwo.commitAwaitCommand(
|
||||
[commandWaitRun('cmd_1')],
|
||||
{ turnId: 'turn-1', promptId: 'turn-1' },
|
||||
{ holdsWorkspaceMount: false },
|
||||
),
|
||||
).rejects.toThrow(/must resolve before the command wait/);
|
||||
await second.close();
|
||||
|
||||
const third = await open(await createWorkspace());
|
||||
const handleFake = await modelOutputCommittedTurn(third);
|
||||
await expect(
|
||||
handleFake.handle.commitAwaitCommand(
|
||||
[commandWaitRun('cmd_1')],
|
||||
{ turnId: 'turn-2', promptId: 'turn-1' },
|
||||
{ holdsWorkspaceMount: false },
|
||||
),
|
||||
).rejects.toThrow(/cannot change the current unfinished turn/);
|
||||
await expect(
|
||||
handleFake.handle.commitAwaitCommand(
|
||||
[commandWaitRun('cmd_1'), commandWaitRun('cmd_1')],
|
||||
handleFake.turn,
|
||||
{ holdsWorkspaceMount: false },
|
||||
),
|
||||
).rejects.toThrow(/must be unique/);
|
||||
await expect(
|
||||
handleFake.handle.commitAwaitCommand(
|
||||
[commandWaitRun('cmd_1', { folded: true })],
|
||||
handleFake.turn,
|
||||
{ holdsWorkspaceMount: false },
|
||||
),
|
||||
).rejects.toThrow(/already folded/);
|
||||
await third.close();
|
||||
});
|
||||
|
||||
it('marks calls folded, restates idempotently, and advances the activation only at the last fold', async () => {
|
||||
const session = await open(await createWorkspace());
|
||||
const { handle, turn } = await modelOutputCommittedTurn(session);
|
||||
await handle.commitAwaitCommand(
|
||||
[
|
||||
commandWaitRun('cmd_1'),
|
||||
commandWaitRun('cmd_2', { functionCallId: 'fc-2' }),
|
||||
],
|
||||
turn,
|
||||
{ holdsWorkspaceMount: true },
|
||||
);
|
||||
const partial = await handle.resolveAwaitCommand('cmd_1');
|
||||
expect(partial?.continuation.phase).toBe('await_command');
|
||||
expect(partial?.commandWait?.runs).toMatchObject([
|
||||
{ commandId: 'cmd_1', folded: true },
|
||||
{ commandId: 'cmd_2', folded: false },
|
||||
]);
|
||||
expect(partial?.commandWait?.holdsWorkspaceMount).toBe(true);
|
||||
expect(session.authority.latestCheckpoint?.boundary).toBe(
|
||||
HARNESS_DURABLE_WAIT_BOUNDARY,
|
||||
);
|
||||
const restated = await handle.resolveAwaitCommand('cmd_1');
|
||||
expect(restated?.identity.checkpointId).toBe(
|
||||
partial?.identity.checkpointId,
|
||||
);
|
||||
const takeover = await session.replaceActivation();
|
||||
const successor = createManagedHarnessHandle(session);
|
||||
const advanced = await successor.resolveAwaitCommand('cmd_2');
|
||||
expect(advanced?.continuation.phase).toBe('model_output_committed');
|
||||
expect(advanced?.commandWait?.runs.every((run) => run.folded)).toBe(true);
|
||||
expect(advanced?.identity.activationId).toBe(takeover.activationId);
|
||||
expect(session.authority.latestCheckpoint?.boundary).toBeNull();
|
||||
// The carried all-folded group replays silently under the same
|
||||
// activation; a fresh activation would owe the replay its identity.
|
||||
const replayed = await successor.resolveAwaitCommand('cmd_1');
|
||||
expect(replayed?.identity.activationId).toBe(takeover.activationId);
|
||||
expect(replayed?.identity.checkpointId).toBe(
|
||||
advanced?.identity.checkpointId,
|
||||
);
|
||||
await session.close();
|
||||
});
|
||||
|
||||
it('stamps the timeout lock on the fold that carried it and keeps it past a takeover', async () => {
|
||||
const session = await open(await createWorkspace());
|
||||
const { handle, turn } = await modelOutputCommittedTurn(session);
|
||||
await handle.commitAwaitCommand([commandWaitRun('cmd_1')], turn, {
|
||||
holdsWorkspaceMount: false,
|
||||
});
|
||||
const folded = await handle.resolveAwaitCommand('cmd_1', {
|
||||
timedOut: true,
|
||||
});
|
||||
expect(folded?.commandLock).toEqual({ turnId: turn.turnId });
|
||||
expect(folded?.continuation.phase).toBe('model_output_committed');
|
||||
await session.replaceActivation();
|
||||
const settled = await session.authority.harnessRunAuthorization();
|
||||
if (settled.status !== 'runnable')
|
||||
throw new Error(`expected runnable, got ${settled.status}`);
|
||||
expect(settled.checkpoint.commandLock).toEqual({ turnId: turn.turnId });
|
||||
await session.close();
|
||||
});
|
||||
|
||||
// The lock is the Turn's, not the wait group's (decision 4): the Turn's
|
||||
// next command batch replaces the group and its Runtime work clears it,
|
||||
// and a takeover at either point must still find the Turn locked.
|
||||
it("keeps the timeout lock across the Turn's next command batch and its Runtime work", async () => {
|
||||
const session = await open(await createWorkspace());
|
||||
const { handle, turn } = await modelOutputCommittedTurn(session);
|
||||
await handle.commitAwaitCommand([commandWaitRun('cmd_1')], turn, {
|
||||
holdsWorkspaceMount: false,
|
||||
});
|
||||
await handle.markCommandWaitTimeout();
|
||||
await handle.resolveAwaitCommand('cmd_1', { timedOut: true });
|
||||
// The next batch parks afresh — and dies before its refusal folds.
|
||||
const parked = await handle.commitAwaitCommand(
|
||||
[commandWaitRun('cmd_2', { functionCallId: 'fc-2' })],
|
||||
turn,
|
||||
{ holdsWorkspaceMount: false },
|
||||
);
|
||||
expect(parked.kind).toBe('durable_wait');
|
||||
await session.replaceActivation();
|
||||
const successor = createManagedHarnessHandle(session);
|
||||
const fresh = await session.authority.harnessRunAuthorization();
|
||||
if (fresh.status !== 'runnable')
|
||||
throw new Error(`expected runnable, got ${fresh.status}`);
|
||||
expect(fresh.checkpoint.commandWait?.runs[0]?.commandId).toBe('cmd_2');
|
||||
expect(fresh.checkpoint.commandLock).toEqual({ turnId: turn.turnId });
|
||||
await successor.resolveAwaitCommand('cmd_2');
|
||||
// Ordinary Runtime work of the same Turn clears the wait group.
|
||||
await successor.commitAwaitRuntime(await runtimeCommit(session));
|
||||
const runtime = await session.authority.harnessRunAuthorization();
|
||||
if (runtime.status !== 'runnable')
|
||||
throw new Error(`expected runnable, got ${runtime.status}`);
|
||||
expect(runtime.checkpoint.continuation.phase).toBe('await_runtime');
|
||||
expect(runtime.checkpoint.commandWait).toBeNull();
|
||||
expect(runtime.checkpoint.commandLock).toEqual({ turnId: turn.turnId });
|
||||
await session.replaceActivation();
|
||||
const after = await session.authority.harnessRunAuthorization();
|
||||
if (after.status !== 'runnable')
|
||||
throw new Error(`expected runnable, got ${after.status}`);
|
||||
expect(after.checkpoint.commandLock).toEqual({ turnId: turn.turnId });
|
||||
await session.close();
|
||||
});
|
||||
|
||||
it('stamps the timeout lock at detection time without folding any run', async () => {
|
||||
const session = await open(await createWorkspace());
|
||||
const { handle, turn } = await modelOutputCommittedTurn(session);
|
||||
await handle.commitAwaitCommand(
|
||||
[
|
||||
commandWaitRun('cmd_1'),
|
||||
commandWaitRun('cmd_2', { functionCallId: 'fc-2' }),
|
||||
],
|
||||
turn,
|
||||
{ holdsWorkspaceMount: false },
|
||||
);
|
||||
const stamped = await handle.markCommandWaitTimeout();
|
||||
expect(stamped?.commandLock).toEqual({ turnId: turn.turnId });
|
||||
expect(stamped?.commandWait?.runs.every((run) => !run.folded)).toBe(true);
|
||||
expect(stamped?.continuation.phase).toBe('await_command');
|
||||
// Restating on an already-stamped group answers the same checkpoint.
|
||||
const restated = await handle.markCommandWaitTimeout();
|
||||
expect(restated?.identity.checkpointId).toBe(
|
||||
stamped?.identity.checkpointId,
|
||||
);
|
||||
await session.close();
|
||||
});
|
||||
|
||||
it('resolves nothing outside the command wait', async () => {
|
||||
const session = await open(await createWorkspace());
|
||||
const { handle } = await modelOutputCommittedTurn(session);
|
||||
await expect(handle.resolveAwaitCommand('cmd_1')).resolves.toBeNull();
|
||||
await session.close();
|
||||
|
||||
const awaiting = await open(await createWorkspace());
|
||||
const handleTwo = createManagedHarnessHandle(awaiting);
|
||||
await handleTwo.ensureRunnable();
|
||||
await handleTwo.commitDurableWait(waitCommit(await waitRefs(awaiting)));
|
||||
await expect(handleTwo.resolveAwaitCommand('cmd_1')).rejects.toThrow(
|
||||
/not an await_command phase/,
|
||||
);
|
||||
await awaiting.close();
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -8,8 +8,10 @@ import { createHash } from 'node:crypto';
|
|||
import {
|
||||
createAwaitActionHarnessCheckpoint,
|
||||
createAwaitAgentHarnessCheckpoint,
|
||||
createAwaitCommandHarnessCheckpoint,
|
||||
createAwaitRuntimeHarnessCheckpoint,
|
||||
createConsumedAgentWaitHarnessCheckpoint,
|
||||
createConsumedCommandWaitHarnessCheckpoint,
|
||||
createConsumedRuntimeResultsHarnessCheckpoint,
|
||||
createHookStoppedRuntimeHarnessCheckpoint,
|
||||
createInitialHarnessCheckpoint,
|
||||
|
|
@ -20,9 +22,12 @@ import {
|
|||
HARNESS_DURABLE_WAIT_BOUNDARY,
|
||||
HARNESS_MODEL_START_PHASES,
|
||||
HARNESS_TURN_COMPLETE_BOUNDARY,
|
||||
harnessCommandLocked,
|
||||
harnessCommandLockOf,
|
||||
type HarnessActionSource,
|
||||
type HarnessAgentWaitRun,
|
||||
type HarnessCheckpointV1,
|
||||
type HarnessCommandWaitRun,
|
||||
type HarnessRunAuthorization,
|
||||
} from './managed-harness-checkpoint.js';
|
||||
import { managedRuntimeDispatchGate } from './managed-runtime-dispatch-gate.js';
|
||||
|
|
@ -232,6 +237,55 @@ export interface ManagedHarnessHandle {
|
|||
* checkpoint.
|
||||
*/
|
||||
resolveAwaitAgent(childRunId: string): Promise<HarnessCheckpointV1 | null>;
|
||||
/**
|
||||
* Commits the command-wait safety point (H4e-b3): the batch's member
|
||||
* command calls as `await_command` with `durable_wait`. A command batch
|
||||
* is admitted before the first call of the batch runs, so the checkpoint
|
||||
* must name every command call of the batch — replay-safe the way
|
||||
* `commitAwaitAgent` is: an identical restated run set answers the same
|
||||
* boundary, a conflicting set conflicts. `turn` binds the wait exactly
|
||||
* as it binds the agent wait. `holdsWorkspaceMount` reports whether the
|
||||
* Turn keeps the Workspace mount it took in an earlier batch, so a
|
||||
* resumed wait takes the Runtime lease back. The batch entries must be
|
||||
* unique on `commandId` and on `functionCallId` and none already folded.
|
||||
*/
|
||||
commitAwaitCommand(
|
||||
runs: readonly HarnessCommandWaitRun[],
|
||||
turn: { readonly turnId: string; readonly promptId: string },
|
||||
options: {
|
||||
readonly holdsWorkspaceMount: boolean;
|
||||
readonly attempt?: {
|
||||
readonly attemptId: string;
|
||||
readonly routeRef: ManagedSessionDurableRef;
|
||||
};
|
||||
},
|
||||
): Promise<HarnessDurableWaitBoundary>;
|
||||
/**
|
||||
* Marks one parked command call folded after its reply committed. The
|
||||
* wait stays `await_command` while calls remain; with every call folded
|
||||
* the continuation advances to `model_output_committed`. Returns null
|
||||
* when no durable wait is in progress; rejects when the open durable
|
||||
* wait is not the command wait; restating an already-folded call
|
||||
* answers the same checkpoint. `options.timedOut` stamps decision 4's
|
||||
* durable lock on the Turn, so a replacement owner of the same Turn
|
||||
* refuses the next command at once.
|
||||
*/
|
||||
resolveAwaitCommand(
|
||||
commandId: string,
|
||||
options?: { readonly timedOut?: boolean },
|
||||
): Promise<HarnessCheckpointV1 | null>;
|
||||
/**
|
||||
* Stamps decision 4's durable lock on the parked command wait's Turn at
|
||||
* detection time — before the timed-out call settles or answers. The
|
||||
* lock is the Turn's, never the wait group's: every later checkpoint of
|
||||
* the Turn carries it, through later command batches and Runtime work
|
||||
* alike, so a replacement owner refuses the Turn's next command; the
|
||||
* evidence of that refusal is structural, never a scan of journaled
|
||||
* text. No-op where the open durable wait is not a command wait, and
|
||||
* never folds any run. Restating it on an already-locked Turn is a
|
||||
* silent replay.
|
||||
*/
|
||||
markCommandWaitTimeout(): Promise<HarnessCheckpointV1 | null>;
|
||||
/**
|
||||
* Commits safety point C: an admitted Runtime tool as `await_runtime` with
|
||||
* `durable_wait`. The original executionCallId is dispatched at most once.
|
||||
|
|
@ -382,10 +436,11 @@ class LocalManagedHarnessHandle implements ManagedHarnessHandle {
|
|||
latest.boundary === HARNESS_DURABLE_WAIT_BOUNDARY &&
|
||||
phase !== 'await_action' &&
|
||||
phase !== 'await_runtime' &&
|
||||
phase !== 'await_agent'
|
||||
phase !== 'await_agent' &&
|
||||
phase !== 'await_command'
|
||||
) {
|
||||
throw new ManagedSessionConflictError(
|
||||
'durable-wait checkpoint is not an await_action, await_runtime, or await_agent phase.',
|
||||
'durable-wait checkpoint is not an await_action, await_runtime, await_agent, or await_command phase.',
|
||||
);
|
||||
}
|
||||
return {
|
||||
|
|
@ -735,6 +790,162 @@ class LocalManagedHarnessHandle implements ManagedHarnessHandle {
|
|||
});
|
||||
}
|
||||
|
||||
async commitAwaitCommand(
|
||||
runs: readonly HarnessCommandWaitRun[],
|
||||
turn: { readonly turnId: string; readonly promptId: string },
|
||||
options: {
|
||||
readonly holdsWorkspaceMount: boolean;
|
||||
readonly attempt?: {
|
||||
readonly attemptId: string;
|
||||
readonly routeRef: ManagedSessionDurableRef;
|
||||
};
|
||||
},
|
||||
): Promise<HarnessDurableWaitBoundary> {
|
||||
if (runs.length === 0) {
|
||||
throw new ManagedSessionConflictError(
|
||||
'Command wait batch must contain at least one run.',
|
||||
);
|
||||
}
|
||||
// Boundary-side validation the parser keeps only on the read path:
|
||||
// refuse here, or the duplicate / already-folded batch becomes a
|
||||
// checkpoint durable-blocked forever at the next read.
|
||||
const seenCommands = new Set<string>();
|
||||
const seenCalls = new Set<string>();
|
||||
for (const run of runs) {
|
||||
if (
|
||||
seenCommands.has(run.commandId) ||
|
||||
seenCalls.has(run.functionCallId)
|
||||
) {
|
||||
throw new ManagedSessionConflictError(
|
||||
'Command wait batch entries must be unique on commandId and functionCallId.',
|
||||
);
|
||||
}
|
||||
seenCommands.add(run.commandId);
|
||||
seenCalls.add(run.functionCallId);
|
||||
if (run.folded) {
|
||||
throw new ManagedSessionConflictError(
|
||||
`Command wait batch run ${run.commandId} is already folded.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return this.mutateCheckpoint(async () => {
|
||||
this.assertNotDetached();
|
||||
this.assertCurrentActivation();
|
||||
const latest = this.authority.latestCheckpoint;
|
||||
if (latest?.boundary === HARNESS_DURABLE_WAIT_BOUNDARY) {
|
||||
const previous = (await this.requireRunnableAuthorization()).checkpoint;
|
||||
// Replay discipline: the identical restated run set against the
|
||||
// same wait answers the same boundary — a re-driven batch names the
|
||||
// same commands again; anything else conflicts, never a second
|
||||
// wait and never a rewrite.
|
||||
if (previous.continuation.phase === 'await_command') {
|
||||
const waited = previous.commandWait?.runs ?? [];
|
||||
const same =
|
||||
runs.length === waited.length &&
|
||||
runs.every((run) =>
|
||||
waited.some(
|
||||
(existing) =>
|
||||
existing.commandId === run.commandId &&
|
||||
existing.functionCallId === run.functionCallId,
|
||||
),
|
||||
) &&
|
||||
waited.every((existing) =>
|
||||
runs.some(
|
||||
(run) =>
|
||||
run.commandId === existing.commandId &&
|
||||
run.functionCallId === existing.functionCallId,
|
||||
),
|
||||
);
|
||||
if (same) {
|
||||
return {
|
||||
kind: 'durable_wait',
|
||||
checkpointId: latest.checkpointId,
|
||||
coveredSequence: latest.coveredSequence,
|
||||
activationId: this.activation.activationId,
|
||||
epoch: this.activation.epoch,
|
||||
} satisfies HarnessDurableWaitBoundary;
|
||||
}
|
||||
throw new ManagedSessionConflictError(
|
||||
'harness is already waiting on different commands.',
|
||||
);
|
||||
}
|
||||
throw new ManagedSessionConflictError(
|
||||
'the durable wait in progress must resolve before the command wait.',
|
||||
);
|
||||
}
|
||||
const runnable = await this.ensureRunnableUnlocked();
|
||||
const startsTurn =
|
||||
runnable.continuation.phase === 'before_model' ||
|
||||
runnable.continuation.phase === 'turn_settled';
|
||||
if (
|
||||
(turn.turnId !== runnable.identity.turnId ||
|
||||
turn.promptId !== runnable.identity.promptId) &&
|
||||
!startsTurn
|
||||
) {
|
||||
throw new ManagedSessionConflictError(
|
||||
'a command wait cannot change the current unfinished turn.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
runnable.identity.activationId !== this.activation.activationId &&
|
||||
!startsTurn
|
||||
) {
|
||||
throw new ManagedSessionConflictError(
|
||||
'a command wait cannot continue a prior activation.',
|
||||
);
|
||||
}
|
||||
const previous = {
|
||||
...runnable,
|
||||
identity: {
|
||||
...runnable.identity,
|
||||
...turn,
|
||||
activationId: this.activation.activationId,
|
||||
},
|
||||
};
|
||||
const identity = this.nextCheckpointIdentity();
|
||||
const attempt = options.attempt;
|
||||
const checkpoint = createAwaitCommandHarnessCheckpoint({
|
||||
previous,
|
||||
...identity,
|
||||
attempt:
|
||||
previous.attempt ??
|
||||
(attempt
|
||||
? {
|
||||
attemptId: attempt.attemptId,
|
||||
routeRef: attempt.routeRef,
|
||||
capabilityRef: null,
|
||||
samplingRef: null,
|
||||
outputState: 'output_committed',
|
||||
usageRef: null,
|
||||
budgetConsumed: 0,
|
||||
}
|
||||
: null),
|
||||
commandWait: {
|
||||
runs,
|
||||
holdsWorkspaceMount: options.holdsWorkspaceMount,
|
||||
},
|
||||
});
|
||||
await this.commitHarnessCheckpoint(
|
||||
`harness:await_command:${this.activation.activationId}:${runs.map((run) => run.commandId).join(',')}:${identity.coveredSequence}`,
|
||||
checkpoint,
|
||||
HARNESS_DURABLE_WAIT_BOUNDARY,
|
||||
);
|
||||
const committed = this.authority.latestCheckpoint;
|
||||
if (committed === undefined) {
|
||||
throw new ManagedSessionConflictError(
|
||||
'command wait was committed but no checkpoint was recorded.',
|
||||
);
|
||||
}
|
||||
return {
|
||||
kind: 'durable_wait',
|
||||
checkpointId: committed.checkpointId,
|
||||
coveredSequence: committed.coveredSequence,
|
||||
activationId: this.activation.activationId,
|
||||
epoch: this.activation.epoch,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
async resolveAwaitAgent(
|
||||
childRunId: string,
|
||||
): Promise<HarnessCheckpointV1 | null> {
|
||||
|
|
@ -833,6 +1044,142 @@ class LocalManagedHarnessHandle implements ManagedHarnessHandle {
|
|||
});
|
||||
}
|
||||
|
||||
async resolveAwaitCommand(
|
||||
commandId: string,
|
||||
options?: { readonly timedOut?: boolean },
|
||||
): Promise<HarnessCheckpointV1 | null> {
|
||||
return this.mutateCheckpoint(async () => {
|
||||
this.assertNotDetached();
|
||||
this.assertCurrentActivation();
|
||||
const latest = this.authority.latestCheckpoint;
|
||||
if (latest?.boundary !== HARNESS_DURABLE_WAIT_BOUNDARY) {
|
||||
// The wait already advanced: the carried all-folded group on
|
||||
// model_output_committed. Restating a folded call replays
|
||||
// silently under the parking activation, but a fresh activation
|
||||
// owes the Turn-bound commits its identity.
|
||||
const previous = (await this.requireRunnableAuthorization()).checkpoint;
|
||||
if (
|
||||
previous.continuation.phase !== 'model_output_committed' ||
|
||||
previous.commandWait === null ||
|
||||
!previous.commandWait.runs.every((run) => run.folded) ||
|
||||
!previous.commandWait.runs.some((run) => run.commandId === commandId)
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
if (previous.identity.activationId === this.activation.activationId) {
|
||||
return previous;
|
||||
}
|
||||
const identity = this.nextCheckpointIdentity();
|
||||
const adopted: HarnessCheckpointV1 = {
|
||||
...previous,
|
||||
resume: {
|
||||
...previous.resume,
|
||||
throughSequence: identity.coveredSequence,
|
||||
},
|
||||
identity: {
|
||||
...previous.identity,
|
||||
checkpointId: identity.checkpointId,
|
||||
coveredSequence: identity.coveredSequence,
|
||||
previousCheckpointId: identity.previousCheckpointId,
|
||||
activationId: this.activation.activationId,
|
||||
},
|
||||
};
|
||||
await this.commitHarnessCheckpoint(
|
||||
`harness:model_output_committed:${this.activation.activationId}:${commandId}:${identity.coveredSequence}`,
|
||||
adopted,
|
||||
null,
|
||||
);
|
||||
return adopted;
|
||||
}
|
||||
const previous = (await this.requireRunnableAuthorization()).checkpoint;
|
||||
if (previous.continuation.phase !== 'await_command') {
|
||||
throw new ManagedSessionConflictError(
|
||||
'durable-wait checkpoint is not an await_command phase.',
|
||||
);
|
||||
}
|
||||
const waited = previous.commandWait?.runs.find(
|
||||
(candidate) => candidate.commandId === commandId,
|
||||
);
|
||||
if (waited?.folded === true) return previous;
|
||||
const identity = this.nextCheckpointIdentity();
|
||||
// A replacement owner that folds the last owed call feeds the Turn's
|
||||
// next tool batch to the model: adopt the Turn's identity on the
|
||||
// advancing commit, or that batch is refused as work of a prior
|
||||
// activation. A fold before the last run keeps the parking
|
||||
// activation — the wait is still the dead owner's in flight.
|
||||
const willAdvance = previous.commandWait!.runs.every(
|
||||
(run) => run.commandId === commandId || run.folded,
|
||||
);
|
||||
const adopted =
|
||||
willAdvance &&
|
||||
previous.identity.activationId !== this.activation.activationId
|
||||
? {
|
||||
...previous,
|
||||
identity: {
|
||||
...previous.identity,
|
||||
activationId: this.activation.activationId,
|
||||
},
|
||||
}
|
||||
: previous;
|
||||
const checkpoint = createConsumedCommandWaitHarnessCheckpoint({
|
||||
previous: adopted,
|
||||
...identity,
|
||||
commandId,
|
||||
timedOut: options?.timedOut,
|
||||
});
|
||||
const advanced =
|
||||
checkpoint.continuation.phase === 'model_output_committed';
|
||||
await this.commitHarnessCheckpoint(
|
||||
`harness:${checkpoint.continuation.phase}:${this.activation.activationId}:${commandId}:${identity.coveredSequence}`,
|
||||
checkpoint,
|
||||
advanced ? null : HARNESS_DURABLE_WAIT_BOUNDARY,
|
||||
);
|
||||
return checkpoint;
|
||||
});
|
||||
}
|
||||
|
||||
async markCommandWaitTimeout(): Promise<HarnessCheckpointV1 | null> {
|
||||
return this.mutateCheckpoint(async () => {
|
||||
this.assertNotDetached();
|
||||
this.assertCurrentActivation();
|
||||
const latest = this.authority.latestCheckpoint;
|
||||
if (latest?.boundary !== HARNESS_DURABLE_WAIT_BOUNDARY) {
|
||||
return null;
|
||||
}
|
||||
const previous = (await this.requireRunnableAuthorization()).checkpoint;
|
||||
if (previous.continuation.phase !== 'await_command') {
|
||||
return null;
|
||||
}
|
||||
if (
|
||||
previous.identity.turnId !== null &&
|
||||
harnessCommandLocked(previous, previous.identity.turnId)
|
||||
) {
|
||||
return previous;
|
||||
}
|
||||
const identity = this.nextCheckpointIdentity();
|
||||
const checkpoint: HarnessCheckpointV1 = {
|
||||
...previous,
|
||||
commandLock: harnessCommandLockOf(previous),
|
||||
identity: {
|
||||
...previous.identity,
|
||||
checkpointId: identity.checkpointId,
|
||||
coveredSequence: identity.coveredSequence,
|
||||
previousCheckpointId: identity.previousCheckpointId,
|
||||
},
|
||||
resume: {
|
||||
...previous.resume,
|
||||
throughSequence: identity.coveredSequence,
|
||||
},
|
||||
};
|
||||
await this.commitHarnessCheckpoint(
|
||||
`harness:await_command:${this.activation.activationId}:timedout:${identity.coveredSequence}`,
|
||||
checkpoint,
|
||||
HARNESS_DURABLE_WAIT_BOUNDARY,
|
||||
);
|
||||
return checkpoint;
|
||||
});
|
||||
}
|
||||
|
||||
async adoptPreparedContinuation(
|
||||
promptId: string,
|
||||
settledResults?: boolean,
|
||||
|
|
|
|||
|
|
@ -103,6 +103,10 @@ import {
|
|||
parseSessionMessage,
|
||||
type SessionMessage,
|
||||
} from './managed-session-message-record.js';
|
||||
import {
|
||||
parseSessionCommand,
|
||||
type SessionCommand,
|
||||
} from './managed-session-command-record.js';
|
||||
import {
|
||||
MANAGED_TEAM_LEADER,
|
||||
parseTeamMessage,
|
||||
|
|
@ -1661,6 +1665,13 @@ export class LocalManagedSessionAuthority {
|
|||
);
|
||||
}
|
||||
}
|
||||
if (request.domain === 'session_command' && request.input !== undefined) {
|
||||
// H4e-b3: a command is answered by the leader, never by waking a
|
||||
// model, so no revision of it ever carries an input.
|
||||
throw new ManagedSessionConflictError(
|
||||
'A session command never carries an input.',
|
||||
);
|
||||
}
|
||||
// Refused before publishing, so a retry loop leaves no body behind.
|
||||
this.assertCommandWritable(command);
|
||||
this.assertExpectedSequence(command);
|
||||
|
|
@ -2463,6 +2474,42 @@ export class LocalManagedSessionAuthority {
|
|||
}
|
||||
}
|
||||
}
|
||||
if (domain === 'session_command') {
|
||||
// H4e-b3: each journal proves what it holds. The leader's receipt
|
||||
// binds to a child run of its own journal; a member's own lineage
|
||||
// lives with the control plane, whose store checks the outbound side.
|
||||
const command = parsed.record as SessionCommand;
|
||||
const self = this.sessionKey.sessionId;
|
||||
if (
|
||||
command.direction === 'outbound' &&
|
||||
command.senderSessionId !== self
|
||||
) {
|
||||
reject('Outbound session command must be sent by this Session.');
|
||||
}
|
||||
if (command.direction === 'inbound') {
|
||||
if (command.targetSessionId !== self) {
|
||||
reject('Inbound session command must be addressed to this Session.');
|
||||
}
|
||||
const named = this.extensionRecord('child_run', command.childRunId);
|
||||
const child =
|
||||
named === undefined ? undefined : parseChildRun(named.record);
|
||||
if (child === undefined || !isChildSessionRun(child)) {
|
||||
reject(
|
||||
'Session command must name a child Session run of this Session.',
|
||||
);
|
||||
}
|
||||
if (child.childSessionId === null) {
|
||||
reject(
|
||||
'Session command from a child arrives only once its run attached.',
|
||||
);
|
||||
}
|
||||
if (command.senderSessionId !== child.childSessionId) {
|
||||
reject(
|
||||
'Session command from a child must come from the Session its run attached.',
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (domain === 'team_state') {
|
||||
// H4e: a team lives in its lead's journal, and each member joins as a
|
||||
// live child Session run of the lead that no other team lists.
|
||||
|
|
@ -2851,6 +2898,9 @@ export class LocalManagedSessionAuthority {
|
|||
refs = [acceptance.contentRef, acceptance.terminalReceiptRef];
|
||||
} else if (domain === 'session_message') {
|
||||
refs = [parseSessionMessage(record).contentRef];
|
||||
} else if (domain === 'session_command') {
|
||||
const command = parseSessionCommand(record);
|
||||
refs = [command.argsRef, command.replyRef];
|
||||
} else if (domain === 'team_task') {
|
||||
const task = parseTeamTask(record);
|
||||
refs = [task.descriptionRef, task.metadataRef];
|
||||
|
|
|
|||
|
|
@ -0,0 +1,244 @@
|
|||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { createHash } from 'node:crypto';
|
||||
import type {
|
||||
ExtensionDeliveryState,
|
||||
ExtensionRun,
|
||||
} from './managed-extension-record.js';
|
||||
import type {
|
||||
SessionCommand,
|
||||
SessionCommandKind,
|
||||
} from './managed-session-command-record.js';
|
||||
import {
|
||||
ManagedSessionRecordError,
|
||||
type ManagedSessionDurableRef,
|
||||
type ManagedSessionJsonValue,
|
||||
} from './managed-session-records.js';
|
||||
|
||||
// H4e-b3 of #12827: the pure construction side of the member command
|
||||
// channel. Every revision body the member, its leader and the control
|
||||
// plane's message relay commit is built here; the verb funnel in
|
||||
// packages/cli/src/serve/hosted-session-command-session.ts serializes them
|
||||
// onto `commitExtensionRecord`. See
|
||||
// docs/design/2026-10-11-managed-agent-team-member-commands.md.
|
||||
|
||||
/**
|
||||
* The member mints the command id from its own Session, the collapsed
|
||||
* prompt key of its Turn and the call, so the id is unique across Sessions
|
||||
* and a re-driven call names the same command again (replay stability).
|
||||
* `promptKey` arrives already collapsed the way H4e-b2's call keys are
|
||||
* (`hostedPromptKey` in hosted-member-chain.ts).
|
||||
*/
|
||||
export function sessionCommandId(params: {
|
||||
readonly senderSessionId: string;
|
||||
readonly promptKey: string;
|
||||
readonly callId: string;
|
||||
}): string {
|
||||
const digest = createHash('sha256')
|
||||
.update(`${params.senderSessionId}\0${params.promptKey}\0${params.callId}`)
|
||||
.digest('hex');
|
||||
return `cmd_${digest.slice(0, 32)}`;
|
||||
}
|
||||
|
||||
function commandRun(
|
||||
executionCallId: string | null,
|
||||
state: ExtensionDeliveryState,
|
||||
): ExtensionRun {
|
||||
return Object.freeze({
|
||||
state: 'settled',
|
||||
reason: null,
|
||||
definition: null,
|
||||
executionCallId,
|
||||
effectId: null,
|
||||
dispatchId: null,
|
||||
deliveryId: null,
|
||||
execution: null,
|
||||
runtime: null,
|
||||
delivery: Object.freeze({ target: 'session', state }),
|
||||
});
|
||||
}
|
||||
|
||||
/** The member's outbox entry, revision 1: planned, its target unfixed. */
|
||||
export function outboundCommandBody(params: {
|
||||
readonly commandId: string;
|
||||
readonly kind: SessionCommandKind;
|
||||
readonly childRunId: string;
|
||||
readonly senderSessionId: string;
|
||||
readonly argsRef: ManagedSessionDurableRef;
|
||||
readonly executionCallId: string;
|
||||
}): SessionCommand {
|
||||
return Object.freeze({
|
||||
direction: 'outbound',
|
||||
commandId: params.commandId,
|
||||
route: 'to_parent',
|
||||
kind: params.kind,
|
||||
childRunId: params.childRunId,
|
||||
senderSessionId: params.senderSessionId,
|
||||
targetSessionId: null,
|
||||
argsRef: params.argsRef,
|
||||
argsDigest: params.argsRef.digest,
|
||||
replyRef: null,
|
||||
replyDigest: null,
|
||||
run: commandRun(params.executionCallId, 'planned'),
|
||||
});
|
||||
}
|
||||
|
||||
/** The relay claimed the command: the target is fixed at the handover. */
|
||||
export function outboundCommandHandoverBody(
|
||||
previous: SessionCommand,
|
||||
targetSessionId: string,
|
||||
): SessionCommand {
|
||||
return Object.freeze({
|
||||
...previous,
|
||||
targetSessionId,
|
||||
run: commandRun(previous.run.executionCallId, 'accepting'),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The leader answered the command: the reply is copied back with the
|
||||
* `accepted` revision (the relay's `answered` verb).
|
||||
*/
|
||||
export function outboundCommandAnsweredBody(
|
||||
previous: SessionCommand,
|
||||
replyRef: ManagedSessionDurableRef,
|
||||
): SessionCommand {
|
||||
return Object.freeze({
|
||||
...previous,
|
||||
replyRef,
|
||||
replyDigest: replyRef.digest,
|
||||
run: commandRun(previous.run.executionCallId, 'accepted'),
|
||||
});
|
||||
}
|
||||
|
||||
/** The parked call read the reply: the outbox entry is consumed. */
|
||||
export function outboundCommandConsumedBody(
|
||||
previous: SessionCommand,
|
||||
): SessionCommand {
|
||||
return Object.freeze({
|
||||
...previous,
|
||||
run: commandRun(previous.run.executionCallId, 'consumed'),
|
||||
});
|
||||
}
|
||||
|
||||
/** Any other single delivery step of an outbox entry. */
|
||||
export function outboundCommandFinishedBody(
|
||||
previous: SessionCommand,
|
||||
state: 'cancelled' | 'rejected' | 'unknown',
|
||||
): SessionCommand {
|
||||
return Object.freeze({
|
||||
...previous,
|
||||
run: commandRun(previous.run.executionCallId, state),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The leader's receipt, revision 1: opens consumed with the reply the
|
||||
* leader knew at once. A command that changes nothing (a board read)
|
||||
* commits no receipt — only the member's journal keeps that reply.
|
||||
*/
|
||||
export function inboundCommandBody(params: {
|
||||
readonly commandId: string;
|
||||
readonly kind: Exclude<SessionCommandKind, 'board_read'>;
|
||||
readonly childRunId: string;
|
||||
readonly senderSessionId: string;
|
||||
readonly targetSessionId: string;
|
||||
readonly argsRef: ManagedSessionDurableRef;
|
||||
readonly replyRef: ManagedSessionDurableRef;
|
||||
}): SessionCommand {
|
||||
return Object.freeze({
|
||||
direction: 'inbound',
|
||||
commandId: params.commandId,
|
||||
route: 'to_parent',
|
||||
kind: params.kind,
|
||||
childRunId: params.childRunId,
|
||||
senderSessionId: params.senderSessionId,
|
||||
targetSessionId: params.targetSessionId,
|
||||
argsRef: params.argsRef,
|
||||
argsDigest: params.argsRef.digest,
|
||||
replyRef: params.replyRef,
|
||||
replyDigest: params.replyRef.digest,
|
||||
run: commandRun(null, 'consumed'),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The reply document, `{ "outcome": "answered" | "refused", "text": "…" }`:
|
||||
* `answered` is the call's result, `refused` ends the call with a tool
|
||||
* error. A refusal is still a reply. The shape is deliberately closed so
|
||||
* the leader-side producer and the member-side reader agree in both
|
||||
* languages.
|
||||
*/
|
||||
export interface SessionCommandReply {
|
||||
readonly outcome: 'answered' | 'refused';
|
||||
readonly text: string;
|
||||
}
|
||||
|
||||
export function answeredCommandReply(text: string): SessionCommandReply {
|
||||
return Object.freeze({ outcome: 'answered', text });
|
||||
}
|
||||
|
||||
export function refusedCommandReply(text: string): SessionCommandReply {
|
||||
return Object.freeze({ outcome: 'refused', text });
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses the reply document carried under a `replyRef` resource.
|
||||
*/
|
||||
export function parseSessionCommandReply(value: unknown): SessionCommandReply {
|
||||
const json = value as { outcome?: unknown; text?: unknown };
|
||||
if (
|
||||
typeof json !== 'object' ||
|
||||
json === null ||
|
||||
Object.keys(json).length !== 2 ||
|
||||
(json.outcome !== 'answered' && json.outcome !== 'refused') ||
|
||||
typeof json.text !== 'string'
|
||||
) {
|
||||
throw new ManagedSessionRecordError(
|
||||
'Session command reply must be { outcome: answered | refused, text: string }.',
|
||||
);
|
||||
}
|
||||
return Object.freeze({ outcome: json.outcome, text: json.text });
|
||||
}
|
||||
|
||||
/** The reply's serialized form, canonical so both sides digest it alike. */
|
||||
export function sessionCommandReplyBytes(reply: SessionCommandReply): Buffer {
|
||||
return Buffer.from(JSON.stringify(reply), 'utf8');
|
||||
}
|
||||
|
||||
/** Reads the reply resource bytes back into a reply document. */
|
||||
export function sessionCommandReplyFromBytes(
|
||||
bytes: Buffer,
|
||||
): SessionCommandReply {
|
||||
return parseSessionCommandReply(
|
||||
JSON.parse(bytes.toString('utf8')) as ManagedSessionJsonValue,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* The runtime's own bound on one member: commands, reads included, one of
|
||||
* its runs may ever make. It bounds what a looping model can ask of the
|
||||
* relay and of the leader's journal.
|
||||
*/
|
||||
export const MANAGED_SESSION_COMMAND_RUNTIME_LIMITS = Object.freeze({
|
||||
maxPerRun: 256,
|
||||
} as const);
|
||||
|
||||
/**
|
||||
* Whether an outbox entry still owes its relay pass — the unfinished set of
|
||||
* the per-sender ordering: planned or accepting. An `accepted` entry is
|
||||
* finished for the relay (the parked call consumes it itself), and a
|
||||
* finished entry (`rejected`, `cancelled`, `unknown` or `consumed`) never
|
||||
* displaces a later command of the same sender.
|
||||
*/
|
||||
export function isUnfinishedCommand(command: SessionCommand): boolean {
|
||||
const state = command.run.delivery?.state;
|
||||
return (
|
||||
command.direction === 'outbound' &&
|
||||
(state === 'planned' || state === 'accepting')
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,241 @@
|
|||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { MANAGED_EXTENSION_RECORD_BODIES } from './managed-extension-projection.js';
|
||||
import { parseSessionCommand } from './managed-session-command-record.js';
|
||||
import {
|
||||
inboundCommandBody,
|
||||
isUnfinishedCommand,
|
||||
outboundCommandBody,
|
||||
sessionCommandId,
|
||||
} from './managed-session-command-operations.js';
|
||||
import {
|
||||
MANAGED_SESSION_ENABLED_DOMAINS,
|
||||
assertManagedSessionDomainEnabled,
|
||||
} from './managed-session-records.js';
|
||||
import { sessionMessageId } from './managed-session-message-operations.js';
|
||||
|
||||
interface Fixture {
|
||||
id: string;
|
||||
domain: 'session_command';
|
||||
template: 'outbound' | 'inbound';
|
||||
patch: Record<string, unknown>;
|
||||
valid: boolean;
|
||||
start: boolean;
|
||||
/** The clause substring both validators must report on an invalid case. */
|
||||
error?: string;
|
||||
}
|
||||
const fixtures = JSON.parse(
|
||||
readFileSync(
|
||||
new URL(
|
||||
'./contracts/managed-session-command-record-v1.fixtures.json',
|
||||
import.meta.url,
|
||||
),
|
||||
'utf8',
|
||||
),
|
||||
) as {
|
||||
contract: string;
|
||||
keys: readonly string[];
|
||||
fixedKeys: readonly string[];
|
||||
templates: Record<string, Record<string, unknown>>;
|
||||
cases: Fixture[];
|
||||
successors: Array<
|
||||
Omit<Fixture, 'patch' | 'start'> & {
|
||||
before: Record<string, unknown>;
|
||||
after: Record<string, unknown>;
|
||||
}
|
||||
>;
|
||||
};
|
||||
|
||||
function merge(
|
||||
base: Record<string, unknown>,
|
||||
patch: Record<string, unknown>,
|
||||
): Record<string, unknown> {
|
||||
const value = structuredClone(base ?? {});
|
||||
for (const [key, replacement] of Object.entries(patch)) {
|
||||
value[key] =
|
||||
replacement !== null &&
|
||||
typeof replacement === 'object' &&
|
||||
!Array.isArray(replacement)
|
||||
? merge(
|
||||
value[key] as Record<string, unknown>,
|
||||
replacement as Record<string, unknown>,
|
||||
)
|
||||
: replacement;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
/** Resolves a fixture's template, or fails loudly: a silent miss degenerates. */
|
||||
function templateOf(fixture: { id: string; template: string }) {
|
||||
const template = fixtures.templates[fixture.template];
|
||||
if (template === undefined) {
|
||||
throw new Error(`fixture ${fixture.id} names an unknown template`);
|
||||
}
|
||||
return template;
|
||||
}
|
||||
|
||||
describe('managed-session-command-record/1 shared contract', () => {
|
||||
it('projects no task and is enabled for submission (H4e-b3)', () => {
|
||||
const body = MANAGED_EXTENSION_RECORD_BODIES.session_command!;
|
||||
for (const template of ['outbound', 'inbound']) {
|
||||
expect(
|
||||
body.taskKindOf(parseSessionCommand(fixtures.templates[template])),
|
||||
).toBeNull();
|
||||
}
|
||||
expect(MANAGED_SESSION_ENABLED_DOMAINS).toContain('session_command');
|
||||
expect(() =>
|
||||
assertManagedSessionDomainEnabled('session_command'),
|
||||
).not.toThrow();
|
||||
expect(fixtures.contract).toBe('managed-session-command-record/1');
|
||||
});
|
||||
|
||||
it('pins the closed keys and the fixed keys', () => {
|
||||
expect([...fixtures.keys].sort()).toEqual(
|
||||
[
|
||||
'argsDigest',
|
||||
'argsRef',
|
||||
'childRunId',
|
||||
'commandId',
|
||||
'direction',
|
||||
'kind',
|
||||
'replyDigest',
|
||||
'replyRef',
|
||||
'route',
|
||||
'run',
|
||||
'senderSessionId',
|
||||
'targetSessionId',
|
||||
].sort(),
|
||||
);
|
||||
expect([...fixtures.fixedKeys].sort()).toEqual(
|
||||
[
|
||||
'argsDigest',
|
||||
'argsRef',
|
||||
'childRunId',
|
||||
'commandId',
|
||||
'direction',
|
||||
'kind',
|
||||
'route',
|
||||
'senderSessionId',
|
||||
].sort(),
|
||||
);
|
||||
for (const template of Object.values(fixtures.templates)) {
|
||||
expect(Object.keys(template).sort()).toEqual([...fixtures.keys].sort());
|
||||
}
|
||||
});
|
||||
|
||||
it.each(fixtures.cases)('$id', (fixture) => {
|
||||
const body = MANAGED_EXTENSION_RECORD_BODIES[fixture.domain]!;
|
||||
const record = merge(templateOf(fixture), fixture.patch);
|
||||
if (fixture.valid) {
|
||||
const parsed = body.parse(record);
|
||||
expect(parsed.record).toEqual(record);
|
||||
expect(parsed.recordId).toBe((record as { commandId: string }).commandId);
|
||||
expect(Object.isFrozen(parsed.record)).toBe(true);
|
||||
for (const value of Object.values(
|
||||
parsed.record as Record<string, unknown>,
|
||||
)) {
|
||||
if (
|
||||
typeof value === 'object' &&
|
||||
value !== null &&
|
||||
!Array.isArray(value)
|
||||
) {
|
||||
expect(Object.isFrozen(value)).toBe(true);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
expect(() => body.parse(record)).toThrow(fixture.error as string);
|
||||
}
|
||||
expect(body.isStart(record)).toBe(fixture.start);
|
||||
});
|
||||
|
||||
it.each(fixtures.successors)('$id', (fixture) => {
|
||||
const template = templateOf(fixture);
|
||||
expect(
|
||||
MANAGED_EXTENSION_RECORD_BODIES[fixture.domain]!.isSuccessor(
|
||||
merge(template, fixture.before),
|
||||
merge(template, fixture.after),
|
||||
),
|
||||
).toBe(fixture.valid);
|
||||
});
|
||||
|
||||
it('commands never meet the message or task id namespaces', () => {
|
||||
const commandId = sessionCommandId({
|
||||
senderSessionId: 'session-member',
|
||||
promptKey: 'prompt-1',
|
||||
callId: 'call-1',
|
||||
});
|
||||
expect(commandId).toMatch(/^cmd_[0-9a-f]{32}$/);
|
||||
const messageId = sessionMessageId({
|
||||
senderSessionId: 'session-member',
|
||||
turnId: 'prompt-1',
|
||||
callId: 'call-1',
|
||||
});
|
||||
expect(messageId).toMatch(/^msg_[0-9a-f]{32}$/);
|
||||
expect(commandId).not.toBe(messageId);
|
||||
});
|
||||
|
||||
it('derives replay-stable ids and reports unfinished outbox entries', () => {
|
||||
const one = sessionCommandId({
|
||||
senderSessionId: 'session-member',
|
||||
promptKey: 'prompt-1',
|
||||
callId: 'call-1',
|
||||
});
|
||||
const two = sessionCommandId({
|
||||
senderSessionId: 'session-member',
|
||||
promptKey: 'prompt-1',
|
||||
callId: 'call-1',
|
||||
});
|
||||
expect(one).toBe(two);
|
||||
const outbound = parseSessionCommand(
|
||||
outboundCommandBody({
|
||||
commandId: one,
|
||||
kind: 'board_write',
|
||||
childRunId: 'run-1',
|
||||
senderSessionId: 'session-member',
|
||||
argsRef: {
|
||||
resourceId: 'args-1',
|
||||
kind: 'managed-command-args',
|
||||
schemaVersion: 1,
|
||||
byteLength: 2,
|
||||
digest:
|
||||
'cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc',
|
||||
},
|
||||
executionCallId: 'call-1',
|
||||
}),
|
||||
);
|
||||
expect(isUnfinishedCommand(outbound)).toBe(true);
|
||||
const inbound = parseSessionCommand(
|
||||
inboundCommandBody({
|
||||
commandId: one,
|
||||
kind: 'board_write',
|
||||
childRunId: 'run-1',
|
||||
senderSessionId: 'session-member',
|
||||
targetSessionId: 'session-lead',
|
||||
argsRef: {
|
||||
resourceId: 'args-copy-1',
|
||||
kind: 'managed-command-args',
|
||||
schemaVersion: 1,
|
||||
byteLength: 2,
|
||||
digest:
|
||||
'cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc',
|
||||
},
|
||||
replyRef: {
|
||||
resourceId: 'reply-1',
|
||||
kind: 'managed-command-reply',
|
||||
schemaVersion: 1,
|
||||
byteLength: 10,
|
||||
digest:
|
||||
'dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd',
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(inbound.run.delivery?.state).toBe('consumed');
|
||||
expect(isUnfinishedCommand(inbound)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,335 @@
|
|||
/**
|
||||
* @license
|
||||
* Copyright 2026 Qwen Team
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
import {
|
||||
isExtensionRunSuccessor,
|
||||
parseExtensionRun,
|
||||
type ExtensionRun,
|
||||
} from './managed-extension-record.js';
|
||||
import {
|
||||
assertManagedSessionDigest,
|
||||
assertManagedSessionDurableRef,
|
||||
assertManagedSessionStableId,
|
||||
ManagedSessionRecordError,
|
||||
type ManagedSessionDurableRef,
|
||||
type ManagedSessionJsonValue,
|
||||
} from './managed-session-records.js';
|
||||
|
||||
// The `managed-session_command` record body, schema version 1 (H4e-b3 of
|
||||
// #12827): one command a member Session asks its leader to execute on the
|
||||
// team. Each command is committed twice, once in each journal by that
|
||||
// journal's own writer: the member's `outbound` record is the durable
|
||||
// outbox entry and the leader's `inbound` record is the receipt, both keyed
|
||||
// by the same `commandId`. A receipt carries a reply and no input — a
|
||||
// command never wakes a model, and only a command that changes anything
|
||||
// gets a receipt at all. The shared fixtures in
|
||||
// contracts/managed-session-command-record-v1.fixtures.json pin it, and
|
||||
// ManagedSessionCommandRecords in packages/sdk-java replays the same cases.
|
||||
// See docs/design/2026-10-11-managed-agent-team-member-commands.md.
|
||||
|
||||
export const MANAGED_SESSION_COMMAND_LIMITS = Object.freeze({
|
||||
/**
|
||||
* Bytes of stored arguments, within the durable inline bound. A producer
|
||||
* bounds bytes before it commits; an oversized call is a tool error.
|
||||
*/
|
||||
maxArgsBytes: 64 * 1024,
|
||||
/**
|
||||
* Bytes of stored reply. A longer board listing is cut with a marker by
|
||||
* the funnel, never refused.
|
||||
*/
|
||||
maxReplyBytes: 64 * 1024,
|
||||
} as const);
|
||||
|
||||
/** Which side of the command a record is: the outbox entry or the receipt. */
|
||||
export type SessionCommandDirection = 'outbound' | 'inbound';
|
||||
/** A command only ever travels from a member to its leader. */
|
||||
export type SessionCommandRoute = 'to_parent';
|
||||
/** What the command asks the leader to do. */
|
||||
export type SessionCommandKind =
|
||||
| 'board_read'
|
||||
| 'board_write'
|
||||
| 'member_message'
|
||||
| 'plan_submit';
|
||||
|
||||
/** The body of a `managed-session_command` schema version 1 record. */
|
||||
export interface SessionCommand {
|
||||
readonly direction: SessionCommandDirection;
|
||||
readonly commandId: string;
|
||||
readonly route: SessionCommandRoute;
|
||||
readonly kind: SessionCommandKind;
|
||||
/** The lineage edge: the child run in the leader Session's journal. */
|
||||
readonly childRunId: string;
|
||||
readonly senderSessionId: string;
|
||||
/** Fixed at handover on the outbound side; this Session on the inbound. */
|
||||
readonly targetSessionId: string | null;
|
||||
/** The holding Session's own copy of the call's arguments. */
|
||||
readonly argsRef: ManagedSessionDurableRef;
|
||||
readonly argsDigest: string;
|
||||
/**
|
||||
* The reply, `{ "outcome": "answered" | "refused", "text": "…" }`, set
|
||||
* once on the outbound's `accepted` revision and at the opening of the
|
||||
* inbound receipt; `null` before and never on a failed delivery.
|
||||
*/
|
||||
readonly replyRef: ManagedSessionDurableRef | null;
|
||||
readonly replyDigest: string | null;
|
||||
readonly run: ExtensionRun;
|
||||
}
|
||||
|
||||
const BODY_KEYS = [
|
||||
'argsDigest',
|
||||
'argsRef',
|
||||
'childRunId',
|
||||
'commandId',
|
||||
'direction',
|
||||
'kind',
|
||||
'replyDigest',
|
||||
'replyRef',
|
||||
'route',
|
||||
'run',
|
||||
'senderSessionId',
|
||||
'targetSessionId',
|
||||
] as const;
|
||||
/**
|
||||
* Every key but the run is fixed across revisions, except `targetSessionId`
|
||||
* and the reply pair, which are set once.
|
||||
*/
|
||||
const FIXED_KEYS = [
|
||||
'argsDigest',
|
||||
'argsRef',
|
||||
'childRunId',
|
||||
'commandId',
|
||||
'direction',
|
||||
'kind',
|
||||
'route',
|
||||
'senderSessionId',
|
||||
] as const;
|
||||
|
||||
function fail(message: string): never {
|
||||
throw new ManagedSessionRecordError(message);
|
||||
}
|
||||
|
||||
function closed<Key extends string>(
|
||||
value: unknown,
|
||||
keys: readonly Key[],
|
||||
): Record<Key, ManagedSessionJsonValue> {
|
||||
if (
|
||||
typeof value !== 'object' ||
|
||||
value === null ||
|
||||
(Object.getPrototypeOf(value) !== Object.prototype &&
|
||||
Object.getPrototypeOf(value) !== null) ||
|
||||
Object.keys(value).length !== keys.length ||
|
||||
Object.keys(value).some((key) => !keys.includes(key as Key))
|
||||
)
|
||||
fail(`Session command must have exactly the keys ${keys.join(', ')}.`);
|
||||
return { ...value } as Record<Key, ManagedSessionJsonValue>;
|
||||
}
|
||||
|
||||
function nullableId(value: ManagedSessionJsonValue, label: string) {
|
||||
return value === null ? null : assertManagedSessionStableId(value, label);
|
||||
}
|
||||
|
||||
function accepts(check: () => boolean): boolean {
|
||||
try {
|
||||
return check();
|
||||
} catch (error) {
|
||||
if (error instanceof ManagedSessionRecordError) return false;
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function same(left: unknown, right: unknown): boolean {
|
||||
return JSON.stringify(left) === JSON.stringify(right);
|
||||
}
|
||||
|
||||
/**
|
||||
* Parses the body of a `managed-session_command` schema version 1 record.
|
||||
* The reply is the act's result: the run is settled from its first
|
||||
* revision and only its session delivery moves. The outbound run names the
|
||||
* commanding call; the inbound run names nothing.
|
||||
*/
|
||||
export function parseSessionCommand(value: unknown): SessionCommand {
|
||||
const body = closed(value, BODY_KEYS);
|
||||
const run = parseExtensionRun(body.run);
|
||||
const direction = body.direction;
|
||||
if (direction !== 'outbound' && direction !== 'inbound') {
|
||||
fail('Session command direction must be outbound or inbound.');
|
||||
}
|
||||
const route = body.route;
|
||||
if (route !== 'to_parent') {
|
||||
fail('Session command route must be to_parent.');
|
||||
}
|
||||
const kind = body.kind;
|
||||
if (
|
||||
kind !== 'board_read' &&
|
||||
kind !== 'board_write' &&
|
||||
kind !== 'member_message' &&
|
||||
kind !== 'plan_submit'
|
||||
) {
|
||||
fail(
|
||||
'Session command kind must be one of board_read, board_write, member_message, plan_submit.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
run.definition !== null ||
|
||||
run.effectId !== null ||
|
||||
run.dispatchId !== null ||
|
||||
run.deliveryId !== null ||
|
||||
run.execution !== null ||
|
||||
run.runtime !== null
|
||||
) {
|
||||
fail('Session command run must be purely logical.');
|
||||
}
|
||||
if (run.state !== 'settled') {
|
||||
fail('Session command run must be settled.');
|
||||
}
|
||||
const delivery = run.delivery;
|
||||
// The run carries no deliveryId, so the delivery is a session one.
|
||||
if (delivery === null) {
|
||||
fail('Session command delivery must be a session delivery.');
|
||||
}
|
||||
const commandId = assertManagedSessionStableId(body.commandId, 'commandId');
|
||||
const childRunId = assertManagedSessionStableId(
|
||||
body.childRunId,
|
||||
'childRunId',
|
||||
);
|
||||
const senderSessionId = assertManagedSessionStableId(
|
||||
body.senderSessionId,
|
||||
'senderSessionId',
|
||||
);
|
||||
const targetSessionId = nullableId(body.targetSessionId, 'targetSessionId');
|
||||
const argsRef = Object.freeze(
|
||||
assertManagedSessionDurableRef(body.argsRef, 'argsRef'),
|
||||
);
|
||||
if (argsRef.byteLength > MANAGED_SESSION_COMMAND_LIMITS.maxArgsBytes) {
|
||||
fail(
|
||||
`Session command arguments exceed ${MANAGED_SESSION_COMMAND_LIMITS.maxArgsBytes} bytes.`,
|
||||
);
|
||||
}
|
||||
const argsDigest = assertManagedSessionDigest(body.argsDigest, 'argsDigest');
|
||||
if (argsDigest !== argsRef.digest) {
|
||||
fail("Session command argsDigest must name the arguments' digest.");
|
||||
}
|
||||
const replyRef =
|
||||
body.replyRef === null
|
||||
? null
|
||||
: Object.freeze(
|
||||
assertManagedSessionDurableRef(body.replyRef, 'replyRef'),
|
||||
);
|
||||
const replyDigest =
|
||||
body.replyDigest === null
|
||||
? null
|
||||
: assertManagedSessionDigest(body.replyDigest, 'replyDigest');
|
||||
if ((replyRef === null) !== (replyDigest === null)) {
|
||||
fail('Session command names its reply as a pair, or not at all.');
|
||||
}
|
||||
if (replyRef !== null) {
|
||||
if (replyRef.byteLength > MANAGED_SESSION_COMMAND_LIMITS.maxReplyBytes) {
|
||||
fail(
|
||||
`Session command reply exceeds ${MANAGED_SESSION_COMMAND_LIMITS.maxReplyBytes} bytes.`,
|
||||
);
|
||||
}
|
||||
if (replyDigest !== replyRef.digest) {
|
||||
fail("Session command replyDigest must name the reply's digest.");
|
||||
}
|
||||
}
|
||||
if (targetSessionId !== null && targetSessionId === senderSessionId) {
|
||||
fail('Session command cannot address its own sender.');
|
||||
}
|
||||
if (direction === 'outbound') {
|
||||
if (run.executionCallId === null) {
|
||||
fail('Outbound session command run must name its commanding call.');
|
||||
}
|
||||
if (
|
||||
targetSessionId === null &&
|
||||
delivery.state !== 'planned' &&
|
||||
delivery.state !== 'cancelled'
|
||||
) {
|
||||
fail(
|
||||
'Outbound session command must fix its target once its delivery is claimed.',
|
||||
);
|
||||
}
|
||||
if (
|
||||
(replyRef !== null) !==
|
||||
(delivery.state === 'accepted' || delivery.state === 'consumed')
|
||||
) {
|
||||
fail('Outbound session command names its reply exactly once accepted.');
|
||||
}
|
||||
} else {
|
||||
if (kind === 'board_read') {
|
||||
fail(
|
||||
'Inbound session command is never a board read; reads commit nothing.',
|
||||
);
|
||||
}
|
||||
if (run.executionCallId !== null) {
|
||||
fail('Inbound session command run must name no call.');
|
||||
}
|
||||
if (delivery.state !== 'consumed') {
|
||||
fail('Inbound session command opens consumed with its reply.');
|
||||
}
|
||||
if (targetSessionId === null) {
|
||||
fail('Inbound session command must name its target.');
|
||||
}
|
||||
if (replyRef === null) {
|
||||
fail('Inbound session command opens with its reply.');
|
||||
}
|
||||
}
|
||||
return Object.freeze({
|
||||
direction,
|
||||
commandId,
|
||||
route,
|
||||
kind,
|
||||
childRunId,
|
||||
senderSessionId,
|
||||
targetSessionId,
|
||||
argsRef,
|
||||
argsDigest,
|
||||
replyRef,
|
||||
replyDigest,
|
||||
run,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether `value` may open a command chain: an outbox entry still planned,
|
||||
* or a receipt already consumed with its reply — never an empty wait.
|
||||
*/
|
||||
export function isSessionCommandStart(value: unknown): boolean {
|
||||
return accepts(() => {
|
||||
const command = parseSessionCommand(value);
|
||||
return (
|
||||
command.run.delivery?.state ===
|
||||
(command.direction === 'outbound' ? 'planned' : 'consumed')
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether `next` may follow `previous`: the command, its edge, its sender
|
||||
* and its arguments never change, the target and the reply are set once,
|
||||
* and the delivery takes one shared step at a time. A receipt opens
|
||||
* consumed, so only an identical restatement follows it.
|
||||
*/
|
||||
export function isSessionCommandSuccessor(
|
||||
previous: unknown,
|
||||
next: unknown,
|
||||
): boolean {
|
||||
return accepts(() => {
|
||||
const before = parseSessionCommand(previous);
|
||||
const after = parseSessionCommand(next);
|
||||
if (FIXED_KEYS.some((key) => !same(before[key], after[key]))) return false;
|
||||
if (
|
||||
(before.targetSessionId !== null &&
|
||||
before.targetSessionId !== after.targetSessionId) ||
|
||||
(before.replyRef !== null && !same(before.replyRef, after.replyRef)) ||
|
||||
(before.replyDigest !== null && before.replyDigest !== after.replyDigest)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (!isExtensionRunSuccessor(before.run, after.run)) return false;
|
||||
if (before.direction === 'outbound') return true;
|
||||
return same(before.run, after.run);
|
||||
});
|
||||
}
|
||||
|
|
@ -840,8 +840,9 @@ describe('managed session per-kind rules', () => {
|
|||
});
|
||||
|
||||
it('registers the v1 domains including file history and session source', () => {
|
||||
expect(MANAGED_SESSION_DOMAINS).toHaveLength(33);
|
||||
expect(new Set(MANAGED_SESSION_DOMAINS).size).toBe(33);
|
||||
expect(MANAGED_SESSION_DOMAINS).toHaveLength(34);
|
||||
expect(new Set(MANAGED_SESSION_DOMAINS).size).toBe(34);
|
||||
expect(MANAGED_SESSION_DOMAINS).toContain('session_command');
|
||||
});
|
||||
|
||||
it('validates the lifecycle target state', () => {
|
||||
|
|
|
|||
|
|
@ -112,6 +112,7 @@ export const MANAGED_SESSION_DOMAINS = [
|
|||
'team_message',
|
||||
'team_plan',
|
||||
'session_message',
|
||||
'session_command',
|
||||
'session_metadata',
|
||||
'file_history',
|
||||
'session_source',
|
||||
|
|
@ -144,6 +145,8 @@ export const MANAGED_SESSION_ENABLED_DOMAINS: readonly ManagedSessionDomain[] =
|
|||
// H4e-b1: a lead's team and its task board.
|
||||
'team_state',
|
||||
'team_task',
|
||||
// H4e-b3: produced by member board tools and the control plane's relay.
|
||||
'session_command',
|
||||
// H3 (#13856): the Monitor runtime, enabled after its physical
|
||||
// acceptance pass; the hold semantics land with the gate.
|
||||
'monitor_run',
|
||||
|
|
|
|||
|
|
@ -115,10 +115,12 @@ public interface HarnessConnector extends AutoCloseable {
|
|||
* result relay's stop of a stopped run's message turns (stop). The
|
||||
* Hosted side settles it before answering. A receive or consume starts
|
||||
* work in the Session, so it takes the same Workspace admission as a
|
||||
* submit.
|
||||
* submit. H4e-b3's command operations ride the same route when the body
|
||||
* names a command id, and a settled {@code execute} answers with the
|
||||
* reply the relay copies back.
|
||||
*/
|
||||
default void runMessageOperation(String tenantId, String sessionId,
|
||||
Map<String, Object> body) {
|
||||
default Map<String, Object> runMessageOperation(String tenantId,
|
||||
String sessionId, Map<String, Object> body) {
|
||||
throw new UnsupportedOperationException(
|
||||
"Message operations are unavailable");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -583,15 +583,18 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
|||
}
|
||||
|
||||
@Override
|
||||
public void runMessageOperation(String tenantId, String sessionId,
|
||||
Map<String, Object> body) {
|
||||
public Map<String, Object> runMessageOperation(String tenantId,
|
||||
String sessionId, Map<String, Object> body) {
|
||||
try {
|
||||
// A receipt admits an input and wakes a turn in its target, and
|
||||
// the consume reconciliation reloads a Session whose wake pump
|
||||
// then runs the waiting input: both drive new work, so they take
|
||||
// the automation relay's Workspace checks. The sender's own
|
||||
// steps are journal writes. Every verb re-attaches a Session a
|
||||
// prior control-plane process attached, since the relay's
|
||||
// steps are journal writes, and H4e-b3's command verbs (a body
|
||||
// naming commandId) commit no input and never start work, so
|
||||
// they keep the same journal admission as the sender's steps
|
||||
// without the Workspace view. Every verb re-attaches a Session
|
||||
// a prior control-plane process attached, since the relay's
|
||||
// ledger outlives that process.
|
||||
Object kind = body.get("kind");
|
||||
if ("stop".equals(kind)) {
|
||||
|
|
@ -599,15 +602,14 @@ public class QwenHostedHarnessConnector implements HarnessConnector {
|
|||
// load can adopt a new generation, which closes the old one.
|
||||
HarnessSessionRef ref = messageStopAttachment(tenantId,
|
||||
sessionId);
|
||||
client().runMessageOperation(ref, body);
|
||||
return;
|
||||
return client().runMessageOperation(ref, body);
|
||||
}
|
||||
if ("receive".equals(kind) || "consume".equals(kind)) {
|
||||
requireReadyForNewWork(tenantId, sessionId, false);
|
||||
}
|
||||
reattachTakenOver(tenantId, sessionId);
|
||||
HarnessSessionRef ref = attachment(tenantId, sessionId, true);
|
||||
client().runMessageOperation(ref, body);
|
||||
return client().runMessageOperation(ref, body);
|
||||
} catch (HostedHarnessGenerationException error) {
|
||||
adoptGeneration(error);
|
||||
throw error;
|
||||
|
|
|
|||
|
|
@ -111,7 +111,8 @@ public class SessionMessageRelay {
|
|||
// entry since, so the delivery and the body it decides by are the
|
||||
// ones committed now.
|
||||
SessionMessageRelayStore.CurrentRecord current = store.currentRecord(
|
||||
row.tenantId(), row.senderSessionId(), row.messageId());
|
||||
row.tenantId(), row.senderSessionId(), pending.domain(),
|
||||
row.messageId());
|
||||
if (current == null) {
|
||||
return;
|
||||
}
|
||||
|
|
@ -130,7 +131,8 @@ public class SessionMessageRelay {
|
|||
? body.path("targetSessionId").textValue() : null;
|
||||
boolean delivered = "accepted".equals(current.deliveryState())
|
||||
|| target != null && store.deliveryState(
|
||||
row.tenantId(), target, row.messageId()) != null;
|
||||
row.tenantId(), target, "session_message",
|
||||
row.messageId()) != null;
|
||||
store.classify(row, owner, delivered ? "done" : "orphaned",
|
||||
senderStatus == null ? "sender session is gone"
|
||||
: "sender session is " + senderStatus,
|
||||
|
|
@ -150,15 +152,19 @@ public class SessionMessageRelay {
|
|||
now + LEASE_MS, now);
|
||||
return;
|
||||
}
|
||||
switch (current.deliveryState()) {
|
||||
case "planned" -> handover(row, body, now);
|
||||
case "accepting", "unknown" -> deliver(row, body,
|
||||
target(body), now);
|
||||
case "accepted" -> awaitConsumption(row, body, now);
|
||||
default -> store.classify(row, owner, "done", null, now);
|
||||
if ("session_command".equals(pending.domain())) {
|
||||
workCommand(row, body, current, now);
|
||||
} else {
|
||||
switch (current.deliveryState()) {
|
||||
case "planned" -> handover(row, body, now);
|
||||
case "accepting", "unknown" -> deliver(row, body,
|
||||
target(body), now);
|
||||
case "accepted" -> awaitConsumption(row, body, now);
|
||||
default -> store.classify(row, owner, "done", null, now);
|
||||
}
|
||||
}
|
||||
} catch (RuntimeException error) {
|
||||
defer(row, error, now);
|
||||
defer(row, pending.domain(), error, now);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -287,7 +293,7 @@ public class SessionMessageRelay {
|
|||
private void deliver(MessageRow row, JsonNode body, String target,
|
||||
long now) {
|
||||
String inputId = inputIdOf(row.messageId());
|
||||
if (store.deliveryState(row.tenantId(), target,
|
||||
if (store.deliveryState(row.tenantId(), target, "session_message",
|
||||
row.messageId()) == null) {
|
||||
if (!"ACTIVE".equals(records.sessionStatus(row.tenantId(),
|
||||
target))) {
|
||||
|
|
@ -370,7 +376,7 @@ public class SessionMessageRelay {
|
|||
private void awaitConsumption(MessageRow row, JsonNode body, long now) {
|
||||
String target = target(body);
|
||||
if ("consumed".equals(store.deliveryState(row.tenantId(), target,
|
||||
row.messageId()))) {
|
||||
"session_message", row.messageId()))) {
|
||||
senderOperation(row, "consumed", Map.of());
|
||||
store.classify(row, owner, "done", null, now);
|
||||
return;
|
||||
|
|
@ -425,6 +431,142 @@ public class SessionMessageRelay {
|
|||
operation);
|
||||
}
|
||||
|
||||
private void senderCommandOperation(MessageRow row, String kind,
|
||||
Map<String, Object> fields) {
|
||||
Map<String, Object> operation = new LinkedHashMap<>();
|
||||
operation.put("operationId", UUID.randomUUID().toString());
|
||||
operation.put("commandId", row.messageId());
|
||||
operation.put("kind", kind);
|
||||
operation.putAll(fields);
|
||||
harness.runMessageOperation(row.tenantId(), row.senderSessionId(),
|
||||
operation);
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b3: the command pass. `planned` fixes the parent of the member's
|
||||
* lineage; `accepting` and `unknown` drive the leader's `execute`,
|
||||
* whose settled reply is then copied back with `answered`. A command
|
||||
* never walks `consume` — the parked tool call reads its own reply.
|
||||
* The leader's own `session_command_refused` is terminal and rejects
|
||||
* the entry at once; every other 4xx is a retryable blocked Session —
|
||||
* the closing and recovery gates the design's route table already
|
||||
* carries, which the relay defers without spending an attempt. One
|
||||
* sender's commands pass strictly in order: an older unclassified one
|
||||
* keeps a newer one waiting, so a write the member reads after a
|
||||
* timeout can never land behind a later command of the same run.
|
||||
*/
|
||||
private void workCommand(MessageRow row, JsonNode body,
|
||||
SessionMessageRelayStore.CurrentRecord current, long now) {
|
||||
if (store.olderOpenCommand(row.tenantId(), row.senderSessionId(),
|
||||
row.messageId())) {
|
||||
store.scheduleRetry(row, owner, now + HEARTBEAT_MS,
|
||||
now + LEASE_MS, now);
|
||||
return;
|
||||
}
|
||||
String state = current.deliveryState();
|
||||
String target = body.path("targetSessionId").isTextual()
|
||||
? body.path("targetSessionId").textValue()
|
||||
: null;
|
||||
if ("planned".equals(state)) {
|
||||
SessionMessageRelayStore.Lineage lineage = store.lineage(
|
||||
row.tenantId(), row.senderSessionId());
|
||||
if (lineage == null) {
|
||||
// A command came only from a lineage Session, so a missing
|
||||
// one cannot be proven undeliverable: wait, never reject.
|
||||
store.scheduleRetry(row, owner, now + HEARTBEAT_MS,
|
||||
now + LEASE_MS, now);
|
||||
return;
|
||||
}
|
||||
if (!body.required("childRunId").asText().equals(
|
||||
lineage.parentChildRunId())) {
|
||||
throw new IllegalStateException(
|
||||
"sender's recorded lineage does not name run "
|
||||
+ body.required("childRunId").asText());
|
||||
}
|
||||
target = lineage.parentSessionId();
|
||||
if (!"ACTIVE".equals(records.sessionStatus(row.tenantId(),
|
||||
target))) {
|
||||
senderCommandOperation(row, "cancelled", Map.of());
|
||||
store.classify(row, owner, "done",
|
||||
"parent session is not active", now);
|
||||
return;
|
||||
}
|
||||
senderCommandOperation(row, "handover",
|
||||
Map.of("targetSessionId", target));
|
||||
store.advance(row, owner, "relaying", target, now,
|
||||
now + LEASE_MS, now);
|
||||
}
|
||||
if (!"planned".equals(state) && !"accepting".equals(state)) {
|
||||
// Decision 2: the sender already holds this command's
|
||||
// terminal answer — its own timeout moved accepting to
|
||||
// cancelled or unknown while the entry was in flight. The
|
||||
// pass owes the ledger row only its classification now,
|
||||
// never a second execute, whatever the row's state.
|
||||
store.classify(row, owner,
|
||||
"unknown".equals(state) ? "unknown" : "done",
|
||||
"command is already " + state, now);
|
||||
return;
|
||||
}
|
||||
if (target == null) {
|
||||
throw new IllegalStateException(
|
||||
"a handed-over command names no target");
|
||||
}
|
||||
if (!"ACTIVE".equals(records.sessionStatus(row.tenantId(), target))) {
|
||||
// A executed write may have landed: never say it did not.
|
||||
senderCommandOperation(row, "unknown", Map.of());
|
||||
store.classify(row, owner, "unknown",
|
||||
"target session closed before the execution", now);
|
||||
return;
|
||||
}
|
||||
JsonNode argsRef = body.required("argsRef");
|
||||
byte[] args = records.readResourceBytes(row.tenantId(),
|
||||
argsRef.required("resourceId").asText());
|
||||
if (args == null) {
|
||||
throw new IllegalStateException(
|
||||
"command arguments are not readable yet");
|
||||
}
|
||||
Map<String, Object> execute = new LinkedHashMap<>();
|
||||
execute.put("operationId", UUID.randomUUID().toString());
|
||||
execute.put("commandId", row.messageId());
|
||||
execute.put("kind", "execute");
|
||||
execute.put("commandKind", body.required("kind").asText());
|
||||
execute.put("route", "to_parent");
|
||||
execute.put("childRunId", body.required("childRunId").asText());
|
||||
execute.put("senderSessionId", row.senderSessionId());
|
||||
execute.put("argsBase64", Base64.getEncoder().encodeToString(args));
|
||||
execute.put("argsDigest", body.required("argsDigest").asText());
|
||||
Map<String, Object> settled;
|
||||
try {
|
||||
settled = harness.runMessageOperation(row.tenantId(), target,
|
||||
execute);
|
||||
} catch (DaemonHttpException error) {
|
||||
if ("session_command_refused".equals(error.getErrorCode())) {
|
||||
// The lead cannot accept the command at all: rejected, and
|
||||
// never re-executed.
|
||||
senderCommandOperation(row, "rejected", Map.of());
|
||||
store.classify(row, owner, "done", error.getMessage(), now);
|
||||
return;
|
||||
}
|
||||
if ("hosted_commands_unavailable".equals(error.getErrorCode())
|
||||
|| "hosted_turn_recovery_required"
|
||||
.equals(error.getErrorCode())
|
||||
|| "hosted_session_closing".equals(error.getErrorCode())
|
||||
|| "session_command_blocked".equals(error.getErrorCode())) {
|
||||
// A blocked or closing lead is a wait, not a failure — the
|
||||
// attempt budget is for real faults. A sender whose run
|
||||
// has not attached yet waits the same way: the control
|
||||
// plane's attach commit may trail this execute.
|
||||
store.scheduleRetry(row, owner, now + HEARTBEAT_MS,
|
||||
now + LEASE_MS, now);
|
||||
return;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
Object reply = settled.get("reply");
|
||||
senderCommandOperation(row, "answered", Map.of("reply", reply));
|
||||
store.classify(row, owner, "done", null, now);
|
||||
}
|
||||
|
||||
/**
|
||||
* A failed step counts an attempt. Past the bound the relay gives up,
|
||||
* but first ends the entry on its sender, so nothing still reads it as
|
||||
|
|
@ -433,7 +575,8 @@ public class SessionMessageRelay {
|
|||
* its receipt exists and otherwise unknown. A give-up whose sender
|
||||
* step faltered owes it and retries, never classifying over it.
|
||||
*/
|
||||
private void defer(MessageRow row, RuntimeException error, long now) {
|
||||
private void defer(MessageRow row, String domain,
|
||||
RuntimeException error, long now) {
|
||||
if (row.attempts() + 1 >= MAX_ATTEMPTS) {
|
||||
try {
|
||||
// The failed step may have moved the entry (a handover that
|
||||
|
|
@ -441,9 +584,51 @@ public class SessionMessageRelay {
|
|||
// committed now.
|
||||
SessionMessageRelayStore.CurrentRecord latest =
|
||||
store.currentRecord(row.tenantId(),
|
||||
row.senderSessionId(), row.messageId());
|
||||
row.senderSessionId(), domain,
|
||||
row.messageId());
|
||||
String state = latest == null ? null : latest.deliveryState();
|
||||
if ("planned".equals(state)) {
|
||||
if ("session_command".equals(domain)) {
|
||||
// H4e-b3: never handed over is cancelled, as a message's
|
||||
// is. A handed-over command is answered from the
|
||||
// leader's own receipt when that committed — the reply
|
||||
// never gets paraphrased — and otherwise unknown.
|
||||
if ("planned".equals(state)) {
|
||||
senderCommandOperation(row, "cancelled", Map.of());
|
||||
} else if (latest != null
|
||||
&& !"cancelled".equals(state)) {
|
||||
JsonNode body = readJson(records.readResource(
|
||||
row.tenantId(), latest.recordResourceId()),
|
||||
"command body");
|
||||
String target = body.path("targetSessionId")
|
||||
.isTextual()
|
||||
? body.path("targetSessionId")
|
||||
.textValue()
|
||||
: null;
|
||||
SessionMessageRelayStore.CurrentRecord receipt =
|
||||
target == null ? null
|
||||
: store.currentRecord(row.tenantId(),
|
||||
target, domain,
|
||||
row.messageId());
|
||||
if (receipt != null) {
|
||||
JsonNode receiptBody = readJson(
|
||||
records.readResource(row.tenantId(),
|
||||
receipt.recordResourceId()),
|
||||
"command receipt");
|
||||
JsonNode replyRef = receiptBody
|
||||
.required("replyRef");
|
||||
JsonNode reply = readJson(
|
||||
records.readResource(row.tenantId(),
|
||||
replyRef.required("resourceId")
|
||||
.asText()),
|
||||
"command reply");
|
||||
senderCommandOperation(row, "answered",
|
||||
Map.of("reply", mapper
|
||||
.convertValue(reply, Map.class)));
|
||||
} else {
|
||||
senderCommandOperation(row, "unknown", Map.of());
|
||||
}
|
||||
}
|
||||
} else if ("planned".equals(state)) {
|
||||
senderOperation(row, "cancelled", Map.of());
|
||||
} else if ("accepting".equals(state)
|
||||
|| "unknown".equals(state)) {
|
||||
|
|
@ -451,7 +636,7 @@ public class SessionMessageRelay {
|
|||
row.tenantId(), latest.recordResourceId()),
|
||||
"message body");
|
||||
if (store.deliveryState(row.tenantId(), target(body),
|
||||
row.messageId()) != null) {
|
||||
domain, row.messageId()) != null) {
|
||||
senderOperation(row, "accepted",
|
||||
Map.of("inputId", inputIdOf(row.messageId())));
|
||||
} else if ("accepting".equals(state)) {
|
||||
|
|
@ -465,9 +650,9 @@ public class SessionMessageRelay {
|
|||
}
|
||||
store.classify(row, owner, "unknown", error.getMessage(), now);
|
||||
LOG.warn("session message relay gives up tenant={} sender={}"
|
||||
+ " message={} after={} failure={}",
|
||||
row.tenantId(), row.senderSessionId(), row.messageId(),
|
||||
row.attempts(), error.getMessage());
|
||||
+ " domain={} message={} after={} failure={}",
|
||||
row.tenantId(), row.senderSessionId(), domain,
|
||||
row.messageId(), row.attempts(), error.getMessage());
|
||||
return;
|
||||
}
|
||||
long delay = Math.min(MAX_BACKOFF_MS,
|
||||
|
|
|
|||
|
|
@ -85,6 +85,11 @@ public final class ManagedExtensionProjection {
|
|||
body -> body.get("messageId").textValue(),
|
||||
ManagedSessionMessageRecords::isMessageStart,
|
||||
ManagedSessionMessageRecords::isMessageSuccessor)),
|
||||
entry("session_command", new Body(record -> null,
|
||||
ManagedSessionCommandRecords::requireCommand,
|
||||
body -> body.get("commandId").textValue(),
|
||||
ManagedSessionCommandRecords::isCommandStart,
|
||||
ManagedSessionCommandRecords::isCommandSuccessor)),
|
||||
entry("team_state", new Body(record -> null,
|
||||
ManagedTeamRecords::requireState,
|
||||
body -> body.get("teamId").textValue(),
|
||||
|
|
|
|||
|
|
@ -55,6 +55,10 @@ public class ManagedExtensionRecordStore {
|
|||
* store holds a child's lineage, so it is a rollbackable non-commit. */
|
||||
public static final String ERROR_MESSAGE_LINEAGE =
|
||||
"session_message_lineage_refused";
|
||||
/** A session command its Session's lineage refuses (H4e-b3): the same
|
||||
* carve-out as the message channel's, under its own code. */
|
||||
public static final String ERROR_COMMAND_LINEAGE =
|
||||
"session_command_lineage_refused";
|
||||
public static final String ERROR_REJECTED =
|
||||
"managed_session_extension_record_rejected";
|
||||
public static final String OPENING_COMMAND_QUERY = "SELECT COUNT(*) FROM"
|
||||
|
|
@ -755,6 +759,14 @@ public class ManagedExtensionRecordStore {
|
|||
JsonNode ref = record.get("contentRef");
|
||||
requireReference(resources.apply(ref.get("resourceId").textValue()), ref);
|
||||
}
|
||||
if (domain.equals("session_command")) {
|
||||
JsonNode argsRef = record.get("argsRef");
|
||||
requireReference(resources.apply(argsRef.get("resourceId").textValue()), argsRef);
|
||||
JsonNode replyRef = record.get("replyRef");
|
||||
if (replyRef != null && !replyRef.isNull()) {
|
||||
requireReference(resources.apply(replyRef.get("resourceId").textValue()), replyRef);
|
||||
}
|
||||
}
|
||||
if (List.of("team_task", "team_message", "team_plan").contains(domain)) {
|
||||
for (String field : List.of("descriptionRef", "metadataRef", "contentRef",
|
||||
"planRef", "feedbackRef")) {
|
||||
|
|
@ -936,6 +948,10 @@ public class ManagedExtensionRecordStore {
|
|||
requireSessionMessage(tenantId, sessionId, scopeKey, record,
|
||||
previous == null, resources);
|
||||
}
|
||||
if (domain.equals("session_command")) {
|
||||
requireSessionCommand(tenantId, sessionId, scopeKey, record,
|
||||
previous == null, resources);
|
||||
}
|
||||
if (domain.startsWith("team_")) {
|
||||
requireTeamRecord(sessionId, scopeKey, domain, record,
|
||||
previous == null ? null
|
||||
|
|
@ -1088,6 +1104,10 @@ public class ManagedExtensionRecordStore {
|
|||
"An inbound session message opens together with its"
|
||||
+ " input, and no other revision carries one.");
|
||||
}
|
||||
if (domain.equals("session_command")) {
|
||||
require(inputIds.get().isEmpty(),
|
||||
"A session command never carries an input.");
|
||||
}
|
||||
JsonNode run = record.get("run");
|
||||
TaskProjection projection = ManagedExtensionProjection.project(
|
||||
previous == null ? null : previous.projection(), run,
|
||||
|
|
@ -1325,6 +1345,68 @@ public class ManagedExtensionRecordStore {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* H4e-b3's session command rules, exactly the message channel's two
|
||||
* arms under the command's own evidence: the member journal's
|
||||
* outbound entry must follow its stamped lineage to the parent, and
|
||||
* the lead journal's inbound receipt must name one of its own child
|
||||
* Session runs, attached to the claimed sender.
|
||||
*/
|
||||
private void requireSessionCommand(String tenantId, String sessionId,
|
||||
String scopeKey, JsonNode command, boolean opening,
|
||||
Function<String, StoredResource> resources) {
|
||||
boolean outbound = "outbound".equals(command.get("direction")
|
||||
.textValue());
|
||||
String target = command.get("targetSessionId").isNull() ? null
|
||||
: command.get("targetSessionId").textValue();
|
||||
String sender = command.get("senderSessionId").textValue();
|
||||
String childRunId = command.get("childRunId").textValue();
|
||||
require(!outbound || sender.equals(sessionId),
|
||||
"Outbound session command must be sent by this Session.");
|
||||
require(outbound || sessionId.equals(target),
|
||||
"Inbound session command must be addressed to this"
|
||||
+ " Session.");
|
||||
if (outbound) {
|
||||
// to_parent on the member's own journal: the lineage is
|
||||
// stamped at creation and never changes, so a plain read
|
||||
// decides, exactly like the message channel's.
|
||||
List<String[]> lineage = jdbc.query("SELECT parent_session_id,"
|
||||
+ " parent_child_run_id FROM managed_agent_session"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?",
|
||||
(result, row) -> new String[] {
|
||||
result.getString("parent_session_id"),
|
||||
result.getString("parent_child_run_id") },
|
||||
tenantId, sessionId);
|
||||
String parent = lineage.isEmpty() ? null : lineage.get(0)[0];
|
||||
String edge = lineage.isEmpty() ? null : lineage.get(0)[1];
|
||||
boolean follows = parent != null && childRunId.equals(edge)
|
||||
&& (target == null || target.equals(parent));
|
||||
if (!follows) {
|
||||
throw new ApiException(HttpStatus.CONFLICT,
|
||||
ERROR_COMMAND_LINEAGE,
|
||||
"Session command to a parent must follow this"
|
||||
+ " Session's lineage.");
|
||||
}
|
||||
return;
|
||||
}
|
||||
// to_parent's receipt on the lead's journal: exactly H4d-b's
|
||||
// inbound child branch — the run belongs to this Session, and it
|
||||
// attached the claimed sender.
|
||||
JsonNode child = recordBody(scopeKey, sessionId, "child_run",
|
||||
childRunId, resources);
|
||||
require(child != null
|
||||
&& ManagedExtensionRecords.isChildSessionRun(child),
|
||||
"Session command must name a child Session run of this"
|
||||
+ " Session.");
|
||||
JsonNode attachedNode = child.get("childSessionId");
|
||||
require(!attachedNode.isNull(),
|
||||
"Session command from a child arrives only once its run"
|
||||
+ " attached.");
|
||||
require(sender.equals(attachedNode.textValue()),
|
||||
"Session command from a child must come from the Session"
|
||||
+ " its run attached.");
|
||||
}
|
||||
|
||||
/**
|
||||
* H4d's continueChildRun: a continuation opens a new run after a
|
||||
* completed one of this Session, in its scope, tree, workspace and
|
||||
|
|
|
|||
|
|
@ -33,7 +33,8 @@ public final class ManagedExtensionRecords {
|
|||
"schedule", "automation_run", "child_run", "child_acceptance",
|
||||
"memory_job", "monitor_run", "goal_state", "todo_state",
|
||||
"plan_mode", "team_state", "team_task", "team_message",
|
||||
"team_plan", "session_message", "session_metadata",
|
||||
"team_plan", "session_message", "session_command",
|
||||
"session_metadata",
|
||||
"file_history", "session_source");
|
||||
/** The event kinds of managed-session/1, as the authority writes them. */
|
||||
public static final List<String> EVENT_KINDS = List.of("input.accepted",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,220 @@
|
|||
package com.alibaba.qwen.code.managedagent.store;
|
||||
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedExtensionRecords.InvalidRecordException;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import java.util.function.BooleanSupplier;
|
||||
|
||||
/**
|
||||
* The {@code managed-session_command} body (H4e-b3 of #12827): one
|
||||
* command a member Session sends its leader, committed once in each
|
||||
* journal by that journal's own writer — the member's {@code outbound}
|
||||
* outbox entry and the lead's {@code inbound} receipt, both keyed by the
|
||||
* same {@code commandId}. The shared fixtures in packages/core pin the
|
||||
* validator, and managed-session-command-record.ts there replays the same
|
||||
* cases. H4e-b3's runtime produces it: the member board tools and the
|
||||
* session message relay's command pass.
|
||||
*/
|
||||
public final class ManagedSessionCommandRecords {
|
||||
/** Bytes of stored arguments, bound exactly like a message's content. */
|
||||
public static final long MAX_ARGS_BYTES = 64 * 1024;
|
||||
/** Bytes of the stored reply document, bound never refused. */
|
||||
public static final long MAX_REPLY_BYTES = 64 * 1024;
|
||||
|
||||
private static final Set<String> KEYS = Set.of("argsDigest", "argsRef",
|
||||
"childRunId", "commandId", "direction", "kind", "replyDigest",
|
||||
"replyRef", "route", "run", "senderSessionId",
|
||||
"targetSessionId");
|
||||
/** Every key but the run is fixed across revisions, except
|
||||
* targetSessionId and the reply pair, which are set once. */
|
||||
private static final List<String> FIXED = List.of("argsDigest",
|
||||
"argsRef", "childRunId", "commandId", "direction", "kind",
|
||||
"route", "senderSessionId");
|
||||
private static final Set<String> KINDS = Set.of("board_read",
|
||||
"board_write", "member_message", "plan_submit");
|
||||
|
||||
private ManagedSessionCommandRecords() {
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks the body of a managed-session_command domain record. Nothing
|
||||
* executes for a command record itself — the board act is the lead's —
|
||||
* so the run is settled from its first revision and only its session
|
||||
* delivery moves. The outbound run names the commanding call; the
|
||||
* inbound run, the receipt of an executed command, names nothing.
|
||||
*/
|
||||
public static void requireCommand(JsonNode record) {
|
||||
ManagedExtensionRecords.closed(record, KEYS, "Session command");
|
||||
JsonNode run = record.get("run");
|
||||
ManagedExtensionRecords.requireRun(run);
|
||||
String direction = record.get("direction").textValue();
|
||||
require("outbound".equals(direction) || "inbound".equals(direction),
|
||||
"Session command direction must be outbound or inbound");
|
||||
require("to_parent".equals(record.get("route").textValue()),
|
||||
"Session command route must be to_parent");
|
||||
String kind = record.get("kind").textValue();
|
||||
require(KINDS.contains(kind),
|
||||
"Session command kind must be one of board_read,"
|
||||
+ " board_write, member_message, plan_submit");
|
||||
require(run.get("definition").isNull()
|
||||
&& run.get("effectId").isNull()
|
||||
&& run.get("dispatchId").isNull()
|
||||
&& run.get("deliveryId").isNull()
|
||||
&& run.get("execution").isNull()
|
||||
&& run.get("runtime").isNull(),
|
||||
"Session command run must be purely logical");
|
||||
require("settled".equals(run.get("state").textValue()),
|
||||
"Session command run must be settled");
|
||||
JsonNode delivery = run.get("delivery");
|
||||
// The run carries no deliveryId, so the delivery is a session one.
|
||||
require(!delivery.isNull(),
|
||||
"Session command delivery must be a session delivery");
|
||||
String state = delivery.get("state").textValue();
|
||||
ManagedExtensionRecords.id(record.get("commandId"), "commandId");
|
||||
ManagedExtensionRecords.id(record.get("childRunId"), "childRunId");
|
||||
String sender = ManagedExtensionRecords.id(
|
||||
record.get("senderSessionId"), "senderSessionId");
|
||||
String target = nullableId(record.get("targetSessionId"),
|
||||
"targetSessionId");
|
||||
JsonNode argsRef = record.get("argsRef");
|
||||
ManagedExtensionRecords.durableRef(argsRef, "argsRef");
|
||||
require(argsRef.get("byteLength").longValue() <= MAX_ARGS_BYTES,
|
||||
"Session command arguments exceed " + MAX_ARGS_BYTES
|
||||
+ " bytes");
|
||||
ManagedExtensionRecords.digest(record.get("argsDigest"),
|
||||
"argsDigest");
|
||||
require(record.get("argsDigest").textValue().equals(
|
||||
argsRef.get("digest").textValue()),
|
||||
"Session command argsDigest must name the arguments'"
|
||||
+ " digest");
|
||||
JsonNode replyRef = record.get("replyRef");
|
||||
require((replyRef == null || replyRef.isNull())
|
||||
== record.get("replyDigest").isNull(),
|
||||
"Session command names its reply as a pair, or not at all");
|
||||
if (replyRef != null && !replyRef.isNull()) {
|
||||
ManagedExtensionRecords.durableRef(replyRef, "replyRef");
|
||||
require(replyRef.get("byteLength").longValue()
|
||||
<= MAX_REPLY_BYTES,
|
||||
"Session command reply exceeds " + MAX_REPLY_BYTES
|
||||
+ " bytes");
|
||||
ManagedExtensionRecords.digest(record.get("replyDigest"),
|
||||
"replyDigest");
|
||||
require(record.get("replyDigest").textValue().equals(
|
||||
replyRef.get("digest").textValue()),
|
||||
"Session command replyDigest must name the reply's"
|
||||
+ " digest");
|
||||
}
|
||||
require(target == null || !sender.equals(target),
|
||||
"Session command cannot address its own sender");
|
||||
boolean reply = replyRef != null && !replyRef.isNull();
|
||||
if ("outbound".equals(direction)) {
|
||||
require(!run.get("executionCallId").isNull(),
|
||||
"Outbound session command run must name its commanding"
|
||||
+ " call");
|
||||
require(target != null || "planned".equals(state)
|
||||
|| "cancelled".equals(state),
|
||||
"Outbound session command must fix its target once its"
|
||||
+ " delivery is claimed");
|
||||
require(reply == ("accepted".equals(state)
|
||||
|| "consumed".equals(state)),
|
||||
"Outbound session command names its reply exactly once"
|
||||
+ " accepted");
|
||||
} else {
|
||||
require(!"board_read".equals(kind),
|
||||
"Inbound session command is never a board read; reads"
|
||||
+ " commit nothing");
|
||||
require(run.get("executionCallId").isNull(),
|
||||
"Inbound session command run must name no call");
|
||||
require("consumed".equals(state),
|
||||
"Inbound session command opens consumed with its reply");
|
||||
require(target != null,
|
||||
"Inbound session command must name its target");
|
||||
require(reply,
|
||||
"Inbound session command opens with its reply");
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether {@code record} may open a command chain: an outbox entry
|
||||
* still planned, or a receipt that opens consumed — the only inbound
|
||||
* state.
|
||||
*/
|
||||
public static boolean isCommandStart(JsonNode record) {
|
||||
return accepts(() -> {
|
||||
requireCommand(record);
|
||||
String state = record.get("run").get("delivery").get("state")
|
||||
.textValue();
|
||||
if ("outbound".equals(record.get("direction").textValue())) {
|
||||
return "planned".equals(state);
|
||||
}
|
||||
return "consumed".equals(state);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether {@code after} may follow {@code before}: the command, its
|
||||
* edge, its sender and its arguments never change, the target and the
|
||||
* reply pair are set once, and the delivery takes one shared step at
|
||||
* a time. A receipt opens consumed, so only an identical restatement
|
||||
* follows it.
|
||||
*/
|
||||
public static boolean isCommandSuccessor(JsonNode before,
|
||||
JsonNode after) {
|
||||
return accepts(() -> {
|
||||
requireCommand(before);
|
||||
requireCommand(after);
|
||||
for (String key : FIXED) {
|
||||
if (!ManagedExtensionRecords.same(before.get(key),
|
||||
after.get(key))) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
if (!before.get("targetSessionId").isNull()
|
||||
&& !before.get("targetSessionId").equals(
|
||||
after.get("targetSessionId"))) {
|
||||
return false;
|
||||
}
|
||||
if (!before.get("replyRef").isNull()
|
||||
&& !before.get("replyRef").equals(after.get("replyRef"))) {
|
||||
return false;
|
||||
}
|
||||
if (!before.get("replyDigest").isNull()
|
||||
&& !before.get("replyDigest").equals(
|
||||
after.get("replyDigest"))) {
|
||||
return false;
|
||||
}
|
||||
if (!ManagedExtensionRecords.isRunSuccessor(before.get("run"),
|
||||
after.get("run"))) {
|
||||
return false;
|
||||
}
|
||||
if ("outbound".equals(before.get("direction").textValue())) {
|
||||
return true;
|
||||
}
|
||||
return ManagedExtensionRecords.same(before.get("run"),
|
||||
after.get("run"));
|
||||
});
|
||||
}
|
||||
|
||||
private static boolean accepts(BooleanSupplier check) {
|
||||
try {
|
||||
return check.getAsBoolean();
|
||||
} catch (InvalidRecordException | NullPointerException
|
||||
| ClassCastException refused) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private static String nullableId(JsonNode node, String label) {
|
||||
if (node == null || node.isNull()) {
|
||||
return null;
|
||||
}
|
||||
return ManagedExtensionRecords.id(node, label);
|
||||
}
|
||||
|
||||
private static void require(boolean check, String message) {
|
||||
if (!check) {
|
||||
throw new InvalidRecordException(message);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -17,9 +17,12 @@ public class SessionMessageRelayStore {
|
|||
/** The failed steps after which the relay gives an entry up. */
|
||||
public static final int MAX_ATTEMPTS = 64;
|
||||
|
||||
/** One outbox entry the delivery-pending index surfaced. */
|
||||
/** One outbox entry the delivery-pending index surfaced. H4e-b3 adds
|
||||
* the domain: command entries share every field and never collide
|
||||
* with message ids, which would duplicate across the ledger. */
|
||||
public record PendingMessage(String tenantId, String senderSessionId,
|
||||
String messageId, String deliveryState, String recordResourceId) {
|
||||
String messageId, String domain, String deliveryState,
|
||||
String recordResourceId) {
|
||||
}
|
||||
|
||||
/** One relay ledger row. */
|
||||
|
|
@ -38,17 +41,23 @@ public class SessionMessageRelayStore {
|
|||
String recordResourceId) {
|
||||
}
|
||||
|
||||
/** The domains this ledger drives: H4d-b's channel and H4e-b3's. */
|
||||
private static final List<String> DOMAINS = List.of("session_message",
|
||||
"session_command");
|
||||
|
||||
// The entries still owing a handover. Only outbox entries reach these
|
||||
// states (a receipt opens accepted), and a terminal ledger row means
|
||||
// the relay already finished with the entry.
|
||||
// states (a receipt opens accepted; a command receipt opens consumed),
|
||||
// and a terminal ledger row means the relay already finished with the
|
||||
// entry. The query's domain binds one channel: session_message, or
|
||||
// H4e-b3's session_command, which discovery drives independently.
|
||||
private static final String HANDOVER_SQL =
|
||||
"SELECT r.tenant_id, r.session_id, r.record_id, r.delivery_state,"
|
||||
+ " r.record_resource_id"
|
||||
"SELECT r.tenant_id, r.session_id, r.record_id, r.domain,"
|
||||
+ " r.delivery_state, r.record_resource_id"
|
||||
+ " FROM qwen_managed_session_extension_record r"
|
||||
+ " LEFT JOIN qwen_managed_session_message_relay l"
|
||||
+ " ON l.sender_session_id = r.session_id"
|
||||
+ " AND l.message_id = r.record_id"
|
||||
+ " WHERE r.domain = 'session_message'"
|
||||
+ " WHERE r.domain = ?"
|
||||
+ " AND r.delivery_state IN ('planned', 'accepting',"
|
||||
+ " 'unknown')"
|
||||
+ " AND (l.state IS NULL OR l.state NOT IN ('done',"
|
||||
|
|
@ -66,8 +75,8 @@ public class SessionMessageRelayStore {
|
|||
// still `relaying` over an accepted entry lost its advance after the
|
||||
// sender's step committed; it waits like a `delivered` one.
|
||||
private static final String AWAITING_SQL =
|
||||
"SELECT r.tenant_id, r.session_id, r.record_id, r.delivery_state,"
|
||||
+ " r.record_resource_id"
|
||||
"SELECT r.tenant_id, r.session_id, r.record_id, r.domain,"
|
||||
+ " r.delivery_state, r.record_resource_id"
|
||||
+ " FROM qwen_managed_session_message_relay l"
|
||||
+ " JOIN qwen_managed_session_extension_record r"
|
||||
+ " ON r.tenant_id = l.tenant_id"
|
||||
|
|
@ -89,13 +98,17 @@ public class SessionMessageRelayStore {
|
|||
}
|
||||
|
||||
/** The owed-work page for one worker: due, not terminal, and never
|
||||
* leased to another worker. */
|
||||
* leased to another worker. Messages take their accepted reads from
|
||||
* the awaiting queue; commands never — their pass ends at answered. */
|
||||
public List<PendingMessage> findPendingMessages(String workerId, long now,
|
||||
int limit) {
|
||||
List<PendingMessage> page = new java.util.ArrayList<>(jdbc.query(
|
||||
HANDOVER_SQL, this::pending, now, now, workerId, limit));
|
||||
HANDOVER_SQL, this::pending, "session_message", now, now,
|
||||
workerId, limit));
|
||||
page.addAll(jdbc.query(AWAITING_SQL, this::pending, now, now,
|
||||
workerId, limit));
|
||||
page.addAll(jdbc.query(HANDOVER_SQL, this::pending,
|
||||
"session_command", now, now, workerId, limit));
|
||||
return page;
|
||||
}
|
||||
|
||||
|
|
@ -103,6 +116,7 @@ public class SessionMessageRelayStore {
|
|||
throws java.sql.SQLException {
|
||||
return new PendingMessage(result.getString("tenant_id"),
|
||||
result.getString("session_id"), result.getString("record_id"),
|
||||
result.getString("domain"),
|
||||
result.getString("delivery_state"),
|
||||
result.getString("record_resource_id"));
|
||||
}
|
||||
|
|
@ -111,28 +125,28 @@ public class SessionMessageRelayStore {
|
|||
* worker that claimed it: the page's own snapshot can be a revision
|
||||
* behind, and a body without the target it fixed reads wrong. */
|
||||
public CurrentRecord currentRecord(String tenantId, String sessionId,
|
||||
String messageId) {
|
||||
String domain, String messageId) {
|
||||
List<CurrentRecord> rows = jdbc.query(
|
||||
"SELECT delivery_state, record_resource_id FROM"
|
||||
+ " qwen_managed_session_extension_record"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND domain = 'session_message' AND record_id = ?",
|
||||
+ " AND domain = ? AND record_id = ?",
|
||||
(result, row) -> new CurrentRecord(
|
||||
result.getString("delivery_state"),
|
||||
result.getString("record_resource_id")),
|
||||
tenantId, sessionId, messageId);
|
||||
tenantId, sessionId, domain, messageId);
|
||||
return rows.isEmpty() ? null : rows.getFirst();
|
||||
}
|
||||
|
||||
/** One record's delivery state as committed now, in any journal. */
|
||||
public String deliveryState(String tenantId, String sessionId,
|
||||
String messageId) {
|
||||
String domain, String messageId) {
|
||||
List<String> rows = jdbc.query(
|
||||
"SELECT delivery_state FROM qwen_managed_session_extension_record"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND domain = 'session_message' AND record_id = ?",
|
||||
+ " AND domain = ? AND record_id = ?",
|
||||
(result, row) -> result.getString("delivery_state"),
|
||||
tenantId, sessionId, messageId);
|
||||
tenantId, sessionId, domain, messageId);
|
||||
return rows.isEmpty() ? null : rows.getFirst();
|
||||
}
|
||||
|
||||
|
|
@ -149,6 +163,29 @@ public class SessionMessageRelayStore {
|
|||
return rows.isEmpty() ? null : rows.getFirst();
|
||||
}
|
||||
|
||||
/** Whether an older command of the same sender is still unclassified:
|
||||
* one sender's commands pass strictly in order. The open window is
|
||||
* the outbound's planned/accepting span on an unclassified row — a
|
||||
* terminal record never blocks again, whatever row state its last
|
||||
* step left, and a terminal row never blocks again, whatever the
|
||||
* record says. */
|
||||
public boolean olderOpenCommand(String tenantId, String senderSessionId,
|
||||
String commandId) {
|
||||
return !jdbc.query("SELECT l.message_id FROM"
|
||||
+ " qwen_managed_session_message_relay l"
|
||||
+ " JOIN qwen_managed_session_extension_record r"
|
||||
+ " ON r.tenant_id = l.tenant_id"
|
||||
+ " AND r.session_id = l.sender_session_id"
|
||||
+ " AND r.domain = 'session_command'"
|
||||
+ " AND r.record_id = l.message_id"
|
||||
+ " WHERE l.tenant_id = ? AND l.sender_session_id = ?"
|
||||
+ " AND l.message_id < ?"
|
||||
+ " AND l.state NOT IN ('done', 'unknown', 'orphaned')"
|
||||
+ " AND r.delivery_state IN ('planned', 'accepting')",
|
||||
(result, row) -> result.getString(1), tenantId,
|
||||
senderSessionId, commandId).isEmpty();
|
||||
}
|
||||
|
||||
/** Insert-or-claim: returns the row this worker now owns, or null when
|
||||
* another live claim holds it. */
|
||||
public MessageRow claim(String tenantId, String senderSessionId,
|
||||
|
|
|
|||
|
|
@ -135,8 +135,17 @@ class HostedTeamLeadMySqlIT {
|
|||
assertThat(team(lead).path("lifecycle").asText()).isEqualTo("deleted");
|
||||
|
||||
for (String member : List.of("alice", "bob")) {
|
||||
assertThat(toolNames(requestsOf("H4EB1_MEMBER_" + member + "_WAIT").get(0))).contains("read_file")
|
||||
.doesNotContain("agent").doesNotContainAnyElementsOf(TEAM_TOOLS);
|
||||
List<String> tools = toolNames(requestsOf(
|
||||
"H4EB1_MEMBER_" + member + "_WAIT").get(0));
|
||||
// H4e-b3: members now read and write the same board the lead
|
||||
// does; the three board verbs ride the lineage lane, while
|
||||
// the team-level creational names stay lead-only.
|
||||
assertThat(tools).contains("read_file")
|
||||
.containsAll(List.of("task_create", "task_list",
|
||||
"task_update"))
|
||||
.doesNotContain("agent")
|
||||
.doesNotContainAnyElementsOf(
|
||||
List.of("team_create", "team_delete"));
|
||||
}
|
||||
assertThat(toolAnswer("lead-alice")).contains("It joined team \"review\" as \"alice\"");
|
||||
assertThat(toolAnswer("lead-list-running")).contains("#1 [in_progress] @alice — Audit A")
|
||||
|
|
@ -204,16 +213,27 @@ class HostedTeamLeadMySqlIT {
|
|||
long stateRevision = revision(lead, "team_state");
|
||||
int leadRequests = requestsOf("H4EB1_LEAD_CLOSE").size();
|
||||
|
||||
String operation = request("POST", "/v1/agents/sessions/" + lead + "/close", null, "close-lead", 202)
|
||||
.path("id").asText();
|
||||
String operation = null;
|
||||
try {
|
||||
operation = request("POST", "/v1/agents/sessions/" + lead + "/close", null, "close-lead", 202)
|
||||
.path("id").asText();
|
||||
} catch (AssertionError failure) {
|
||||
// Without workspace execution no close operation can bind, and
|
||||
// the cascade this case watches lives behind it — abort there.
|
||||
org.junit.jupiter.api.Assumptions.assumeTrue(
|
||||
!String.valueOf(failure.getMessage()).contains("workspace_unavailable"),
|
||||
"workspace execution unavailable under the runtime broker's opt-out");
|
||||
throw failure;
|
||||
}
|
||||
String closeOperation = operation;
|
||||
waiting().atMost(Duration.ofSeconds(15)).pollInterval(Duration.ofMillis(200)).untilAsserted(() ->
|
||||
assertThat(body(rows(lead, "child_run").get(0)).path("stopRequested").asBoolean()).isTrue());
|
||||
// The stop is recorded, but for as long as the member's Turn runs the
|
||||
// close stays owed: the cascade's retries (1, 2 and 4 s apart) are
|
||||
// refused, and nothing interrupts that Turn or closes its Session.
|
||||
waiting().atMost(Duration.ofSeconds(25)).pollInterval(Duration.ofMillis(250)).untilAsserted(() -> {
|
||||
assertCloseOwed(lead, operation, child);
|
||||
assertThat(closeAttempts(operation)).isGreaterThanOrEqualTo(4);
|
||||
assertCloseOwed(lead, closeOperation, child);
|
||||
assertThat(closeAttempts(closeOperation)).isGreaterThanOrEqualTo(4);
|
||||
});
|
||||
|
||||
releases.get("carol").countDown();
|
||||
|
|
@ -221,7 +241,7 @@ class HostedTeamLeadMySqlIT {
|
|||
// the relay misses it, the attempt after waits out a 16 s backoff.
|
||||
waiting().atMost(Duration.ofSeconds(45)).pollInterval(Duration.ofMillis(500)).untilAsserted(() -> {
|
||||
assertNoModelFailure();
|
||||
String status = operationStatus(lead, operation);
|
||||
String status = operationStatus(lead, closeOperation);
|
||||
if (List.of("failed", "recovery_blocked").contains(status))
|
||||
throw new IllegalStateException("The lead's close ended " + status);
|
||||
assertThat(status).isEqualTo("completed");
|
||||
|
|
@ -664,12 +684,25 @@ class HostedTeamLeadMySqlIT {
|
|||
* Closes the lead and waits until the tenant owes no operation and no
|
||||
* child result: the relay and the operation recovery scan every tenant,
|
||||
* so work left behind would be picked up by the next Spring context.
|
||||
* A workspace Session's close operation needs the runtime broker's
|
||||
* workspace execution, unavailable under this fixture's opt-out —
|
||||
* tolerate the 202 only when it binds an operation to wait on.
|
||||
*/
|
||||
private void closeAndSettle(String lead) throws Exception {
|
||||
String operation = request("POST", "/v1/agents/sessions/" + lead + "/close", null, "close-" + lead, 202)
|
||||
.path("id").asText();
|
||||
waiting().atMost(Duration.ofSeconds(20)).pollInterval(Duration.ofMillis(500)).untilAsserted(() ->
|
||||
assertThat(operationStatus(lead, operation)).isEqualTo("completed"));
|
||||
String operation = null;
|
||||
try {
|
||||
operation = request("POST", "/v1/agents/sessions/" + lead + "/close", null, "close-" + lead, 202)
|
||||
.path("id").asText();
|
||||
} catch (AssertionError failure) {
|
||||
if (!String.valueOf(failure.getMessage()).contains("workspace_unavailable")) {
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
if (operation != null) {
|
||||
String op = operation;
|
||||
waiting().atMost(Duration.ofSeconds(20)).pollInterval(Duration.ofMillis(500)).untilAsserted(() ->
|
||||
assertThat(operationStatus(lead, op)).isEqualTo("completed"));
|
||||
}
|
||||
awaitTenantSettled();
|
||||
}
|
||||
|
||||
|
|
@ -678,7 +711,8 @@ class HostedTeamLeadMySqlIT {
|
|||
assertThat(jdbc.queryForList("SELECT operation_kind, state, delivery_state FROM managed_agent_operation"
|
||||
+ " WHERE tenant_id = ? AND delivery_state IN ('PENDING', 'LEASED', 'BLOCKED')", tenant)).isEmpty();
|
||||
assertThat(jdbc.queryForList("SELECT child_run_id, state FROM qwen_managed_child_result_relay"
|
||||
+ " WHERE tenant_id = ? AND state NOT IN ('done', 'orphaned', 'unknown')", tenant)).isEmpty();
|
||||
+ " WHERE tenant_id = ? AND state NOT IN ('done', 'orphaned', 'unknown', 'close_debt')",
|
||||
tenant)).isEmpty();
|
||||
// A child run the relay has not yet picked up has no ledger row.
|
||||
assertThat(jdbc.queryForList("SELECT r.record_id FROM qwen_managed_session_extension_record r"
|
||||
+ " LEFT JOIN qwen_managed_child_result_relay l ON l.parent_session_id = r.session_id"
|
||||
|
|
@ -759,6 +793,12 @@ class HostedTeamLeadMySqlIT {
|
|||
"--qwen.managed-agent.runtime-broker.workspace-mounts[0].storage-id=storage-0",
|
||||
"--qwen.managed-agent.runtime-broker.workspace-mounts[0].root=" + workspace));
|
||||
if (approvals) arguments.add("--qwen.managed-agent.harness.approval-mode=default");
|
||||
// The broker's durable local-process mode only exists on Linux;
|
||||
// elsewhere it must be opted out, which starts the broker without
|
||||
// workspace execution (workspace closes then keep answering the
|
||||
// tolerated 409). Linux CI keeps the mode and its workspace.
|
||||
if (!System.getProperty("os.name", "").toLowerCase(java.util.Locale.ROOT).contains("linux"))
|
||||
arguments.add("--qwen.managed-agent.runtime-broker.durable-local-process=false");
|
||||
spring = (ServletWebServerApplicationContext) new SpringApplicationBuilder(ManagedAgentServerApplication.class)
|
||||
.initializers(context -> context.getBeanFactory().registerSingleton("h4eb1Authentication",
|
||||
authentication()))
|
||||
|
|
|
|||
|
|
@ -0,0 +1,928 @@
|
|||
/**
|
||||
* Copyright 2026 Alibaba Group
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*/
|
||||
|
||||
package com.alibaba.qwen.code.managedagent;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.awaitility.Awaitility.await;
|
||||
|
||||
import com.alibaba.qwen.code.managedagent.api.AuthenticatedTenantActor;
|
||||
import com.alibaba.qwen.code.managedagent.api.WorkspaceSelection;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedAgentStore;
|
||||
import com.alibaba.qwen.code.runtimebroker.WorkspaceExecutionProfile;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.fasterxml.jackson.databind.node.ArrayNode;
|
||||
import com.fasterxml.jackson.databind.node.ObjectNode;
|
||||
import com.sun.net.httpserver.HttpExchange;
|
||||
import com.sun.net.httpserver.HttpServer;
|
||||
import jakarta.servlet.Filter;
|
||||
import jakarta.servlet.http.HttpServletRequest;
|
||||
import jakarta.servlet.http.HttpServletRequestWrapper;
|
||||
import java.io.IOException;
|
||||
import java.net.InetSocketAddress;
|
||||
import java.net.ServerSocket;
|
||||
import java.net.URI;
|
||||
import java.net.http.HttpClient;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.nio.file.attribute.PosixFilePermissions;
|
||||
import java.security.Principal;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.UUID;
|
||||
import java.util.concurrent.CopyOnWriteArraySet;
|
||||
import java.util.concurrent.ExecutorService;
|
||||
import java.util.concurrent.Executors;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.concurrent.atomic.AtomicBoolean;
|
||||
import java.util.concurrent.atomic.AtomicReference;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
import org.awaitility.core.ConditionFactory;
|
||||
import org.junit.jupiter.api.AfterEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.Timeout;
|
||||
import org.junit.jupiter.api.extension.AfterTestExecutionCallback;
|
||||
import org.junit.jupiter.api.extension.RegisterExtension;
|
||||
import org.junit.jupiter.api.io.CleanupMode;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import org.springframework.boot.builder.SpringApplicationBuilder;
|
||||
import org.springframework.boot.web.servlet.FilterRegistrationBean;
|
||||
import org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext;
|
||||
import org.springframework.core.Ordered;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.transaction.PlatformTransactionManager;
|
||||
import org.springframework.transaction.support.TransactionTemplate;
|
||||
|
||||
/**
|
||||
* H4e-b3: the member board commands end to end. A real Spring server on
|
||||
* MySQL drives the packaged Hosted Harness through its own
|
||||
* HarnessConnector, with the embedded Runtime Broker behind the relay's
|
||||
* command pass; only the model is scripted. The lead creates a team and
|
||||
* launches two named members. Each member's scripted answers carry the
|
||||
* board tools as commands, so each member writes the board and then
|
||||
* reads it back through the relay (`planned` on the member, a H4e-b3
|
||||
* receipt on the lead, `accepted` and `consumed` back on the member);
|
||||
* the lead's own board read shows both writes. One after another:
|
||||
* alice's in-progress auto-claim and bob's plain create. A third member
|
||||
* commits a command and parks, her Harness dies, the new worker wakes
|
||||
* her wait, and the relay's answer after that lands her report anyway.
|
||||
*/
|
||||
class HostedTeamMemberCommandMySqlIT {
|
||||
private static final String TOKEN = "h4eb3-fixture";
|
||||
private static final String DIGEST = "sha256:" + "b".repeat(64);
|
||||
private static final Pattern MEMBER = Pattern.compile(
|
||||
"H4EB3_CMD_MEMBER_([a-z]+)_([A-Z]+)");
|
||||
private static final Pattern TEAMMATE = Pattern.compile(
|
||||
"<teammate>([^<]+)</teammate>");
|
||||
|
||||
private final ObjectMapper json = new ObjectMapper();
|
||||
private final HttpClient http = HttpClient.newHttpClient();
|
||||
private final String tenant = "h4eb3-" + UUID.randomUUID();
|
||||
private final AtomicReference<Throwable> modelFailure =
|
||||
new AtomicReference<>();
|
||||
// Member reports seen by the scripted lead across her wake turns. A
|
||||
// wake turn's request names only the report that woke it — the tags
|
||||
// never accumulate across turns, so the record of "every report has
|
||||
// arrived" is a cross-request accumulation, not a per-request parse.
|
||||
private final Set<String> seenReports = new CopyOnWriteArraySet<>();
|
||||
private final AtomicBoolean bobLaunched = new AtomicBoolean();
|
||||
private final AtomicBoolean boardReadFired = new AtomicBoolean();
|
||||
@TempDir(cleanup = CleanupMode.ON_SUCCESS)
|
||||
private Path temporary;
|
||||
private ServletWebServerApplicationContext spring;
|
||||
private HttpServer model;
|
||||
private ExecutorService modelThreads;
|
||||
private volatile String leadSession;
|
||||
private volatile Process harness;
|
||||
private volatile JdbcTemplate jdbc;
|
||||
private int harnessPort;
|
||||
private int brokerPort;
|
||||
private int port;
|
||||
|
||||
@RegisterExtension
|
||||
final AfterTestExecutionCallback harnessLogOnFailure = context -> {
|
||||
Path log = temporary.resolve("harness.log");
|
||||
if (context.getExecutionException().isPresent()
|
||||
&& Files.exists(log)) {
|
||||
String text = Files.readString(log);
|
||||
System.out.println("--- harness.log (tail) ---\n"
|
||||
+ text.substring(Math.max(0, text.length() - 20_000)));
|
||||
}
|
||||
};
|
||||
|
||||
@AfterEach
|
||||
void tearDown() throws Exception {
|
||||
if (harness != null) {
|
||||
harness.destroy();
|
||||
harness.waitFor(20, TimeUnit.SECONDS);
|
||||
}
|
||||
if (spring != null) spring.close();
|
||||
if (model != null) model.stop(0);
|
||||
if (modelThreads != null) modelThreads.shutdownNow();
|
||||
}
|
||||
|
||||
/**
|
||||
* Two members write the board with member commands and read it back:
|
||||
* alice's task opens pending and turns in-progress with her as its
|
||||
* owner when she claims it, bob's second task stays pending; the
|
||||
* lead's own board read shows both. The members' outbox entries all
|
||||
* settle consumed on their journals, and the lead never sees a
|
||||
* session_command it did not receipt.
|
||||
*/
|
||||
@Test
|
||||
@Timeout(240)
|
||||
void membersReadAndWriteTheBoardTheirLeadReads() throws Exception {
|
||||
boot();
|
||||
String lead = createLead("H4EB3_CMD_LIFECYCLE");
|
||||
waiting().atMost(Duration.ofSeconds(180))
|
||||
.pollInterval(Duration.ofSeconds(1)).untilAsserted(() -> {
|
||||
assertNoModelFailure();
|
||||
assertNoFailedTurn(lead);
|
||||
assertNoFailedChild(lead);
|
||||
assertThat(seenReports)
|
||||
.containsAll(List.of("alice", "bob"));
|
||||
});
|
||||
|
||||
List<Map<String, Object>> board = rows(lead, "team_task");
|
||||
assertThat(board).hasSize(2);
|
||||
JsonNode aliceTask = board.stream()
|
||||
.filter(row -> "Audit one".equals(
|
||||
body(row).path("subject").asText()))
|
||||
.findFirst().map(this::body).orElse(null);
|
||||
JsonNode bobTask = board.stream()
|
||||
.filter(row -> "Doc sync".equals(
|
||||
body(row).path("subject").asText()))
|
||||
.findFirst().map(this::body).orElse(null);
|
||||
assertThat(aliceTask).isNotNull();
|
||||
assertThat(aliceTask.path("status").asText())
|
||||
.isEqualTo("in_progress");
|
||||
assertThat(aliceTask.path("owner").asText()).isEqualTo("alice");
|
||||
assertThat(bobTask).isNotNull();
|
||||
assertThat(bobTask.path("status").asText()).isEqualTo("pending");
|
||||
assertThat(bobTask.path("owner").asText()).isEqualTo("null");
|
||||
|
||||
// Every member's command did the full pass: handover → execute →
|
||||
// answered → consumed, never left pending.
|
||||
int stalled = jdbc.queryForObject(
|
||||
"SELECT COUNT(*) FROM qwen_managed_session_extension_record"
|
||||
+ " WHERE tenant_id = ? AND domain = 'session_command'"
|
||||
+ " AND delivery_state IN ('planned', 'accepting')",
|
||||
Integer.class, tenant);
|
||||
assertThat(stalled).isZero();
|
||||
closeAndSettle(lead);
|
||||
}
|
||||
|
||||
/**
|
||||
* carol commits her command and parks; the Harness dies before her
|
||||
* relay could hand it over, the new worker's wake re-enters her wait,
|
||||
* the relay hands it over and executes it, and her report still
|
||||
* lands — the relay's answer never asked the dead process anything.
|
||||
*/
|
||||
@Test
|
||||
@Timeout(150)
|
||||
void memberHarnessRestartDuringCommandWait() throws Exception {
|
||||
boot();
|
||||
String lead = createLead("H4EB3_CMD_RESTART");
|
||||
// Wait until carol's park is durable: the await_command
|
||||
// checkpoint is committed before her outbox is, so sighting it
|
||||
// fires the kill while the relay's next scan is still ahead —
|
||||
// the Harness dies before the handover. The lead's own Turn must
|
||||
// have settled first: a Harness dying mid-model-round loses that
|
||||
// Turn for good (its input re-drives as a fresh one), and a
|
||||
// FAILED row would break the acceptance below.
|
||||
waiting().atMost(Duration.ofSeconds(60))
|
||||
.pollInterval(Duration.ofMillis(50)).untilAsserted(() -> {
|
||||
assertNoModelFailure();
|
||||
assertNoFailedTurn(lead);
|
||||
assertNoFailedChild(lead);
|
||||
assertThat(openTurns(lead)).isEmpty();
|
||||
assertThat(parkedOnCommand(onlyChildOf(lead))).isTrue();
|
||||
});
|
||||
|
||||
harness.descendants().forEach(ProcessHandle::destroyForcibly);
|
||||
harness.destroyForcibly();
|
||||
harness.waitFor(20, TimeUnit.SECONDS);
|
||||
startHarness();
|
||||
|
||||
waiting().atMost(Duration.ofSeconds(100))
|
||||
.pollInterval(Duration.ofSeconds(1)).untilAsserted(() -> {
|
||||
assertNoModelFailure();
|
||||
assertNoFailedTurn(lead);
|
||||
assertNoFailedChild(lead);
|
||||
// Her report after the dead generation is durable on
|
||||
// the lead's journal: the result was accepted, never
|
||||
// asked of the dead process.
|
||||
assertThat(childResultAccepted(lead)).isTrue();
|
||||
List<JsonNode> commands = commandsOf(lead, "carol");
|
||||
assertThat(commands).isNotEmpty();
|
||||
assertThat(commands.get(0).path("run").path("delivery")
|
||||
.path("state").asText()).isEqualTo("consumed");
|
||||
});
|
||||
closeAndSettle(lead);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------ scenario plumbing
|
||||
|
||||
private String createLead(String marker) throws Exception {
|
||||
TransactionTemplate transaction = new TransactionTemplate(
|
||||
spring.getBean(PlatformTransactionManager.class));
|
||||
ManagedAgentStore store = spring.getBean(ManagedAgentStore.class);
|
||||
transaction.execute(status -> {
|
||||
// The trusted internal TeamLead lane: no public Shell gate,
|
||||
// the internal team-lead profile ships with its own line
|
||||
// (#13674's public Shell gate admits only externally created
|
||||
// Shell Session lanes).
|
||||
var created = store.insertInternalTeamLeadSessionCommand(tenant,
|
||||
"actor", "create-" + marker, DIGEST, null,
|
||||
new WorkspaceSelection("workspace-0", "child"));
|
||||
leadSession = created.sessionId();
|
||||
return created.sessionId();
|
||||
});
|
||||
waiting().atMost(Duration.ofSeconds(30))
|
||||
.pollInterval(Duration.ofSeconds(1)).ignoreExceptions()
|
||||
.untilAsserted(() -> request("POST",
|
||||
"/v1/agents/sessions/" + leadSession + "/events",
|
||||
Map.of("type", "agent.session.input.message",
|
||||
"input",
|
||||
List.of(Map.of("type", "input_text", "text",
|
||||
marker))),
|
||||
"input-" + marker, 202));
|
||||
return leadSession;
|
||||
}
|
||||
|
||||
private void closeAndSettle(String lead) throws Exception {
|
||||
// A workspace Session's close operation needs the runtime
|
||||
// broker's workspace execution, unavailable under this suite's
|
||||
// macOS opt-out (durable-local-process=false); Linux CI answers
|
||||
// 202 and runs the operation through.
|
||||
JsonNode closed = null;
|
||||
try {
|
||||
closed = request("POST", "/v1/agents/sessions/" + lead
|
||||
+ "/close", null, "close-" + lead, 202);
|
||||
} catch (AssertionError failure) {
|
||||
if (!String.valueOf(failure.getMessage())
|
||||
.contains("workspace_unavailable")) {
|
||||
throw failure;
|
||||
}
|
||||
}
|
||||
if (closed != null) {
|
||||
String operation = closed.path("id").asText();
|
||||
waiting().atMost(Duration.ofSeconds(20))
|
||||
.pollInterval(Duration.ofMillis(500))
|
||||
.untilAsserted(() ->
|
||||
assertThat(operationStatus(lead, operation))
|
||||
.isEqualTo("completed"));
|
||||
}
|
||||
waiting().atMost(Duration.ofSeconds(20))
|
||||
.pollInterval(Duration.ofMillis(500)).untilAsserted(() -> {
|
||||
assertThat(jdbc.queryForList(
|
||||
"SELECT operation_kind, state, delivery_state FROM"
|
||||
+ " managed_agent_operation WHERE tenant_id"
|
||||
+ " = ? AND delivery_state IN ('PENDING',"
|
||||
+ " 'LEASED', 'BLOCKED')", tenant))
|
||||
.isEmpty();
|
||||
// close_debt means the member's result was delivered
|
||||
// and only her Workspace close is owed — the only
|
||||
// terminal shape that path can take anywhere this
|
||||
// fixture's runtime broker keeps workspace execution
|
||||
// off, macOS and CI alike.
|
||||
assertThat(jdbc.queryForList(
|
||||
"SELECT child_run_id, state FROM"
|
||||
+ " qwen_managed_child_result_relay WHERE"
|
||||
+ " tenant_id = ? AND state NOT IN"
|
||||
+ " ('done','orphaned','unknown',"
|
||||
+ " 'close_debt')", tenant)).isEmpty();
|
||||
assertThat(jdbc.queryForList(
|
||||
"SELECT session_id, status FROM managed_agent_turn"
|
||||
+ " WHERE tenant_id = ? AND status NOT IN"
|
||||
+ " ('COMPLETED', 'FAILED', 'CANCELLED')",
|
||||
tenant)).isEmpty();
|
||||
});
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------ the model
|
||||
|
||||
private void modelReply(HttpExchange exchange) throws IOException {
|
||||
try {
|
||||
JsonNode body = json.readTree(exchange.getRequestBody());
|
||||
String first = firstUserText(body);
|
||||
Matcher member = MEMBER.matcher(first);
|
||||
if (member.find()) {
|
||||
String name = member.group(1);
|
||||
reply(exchange, memberPlan(name, body));
|
||||
return;
|
||||
}
|
||||
if (first.contains("H4EB3_CMD_LIFECYCLE")) {
|
||||
reply(exchange, leadPlan("LIFECYCLE", body));
|
||||
return;
|
||||
}
|
||||
if (first.contains("H4EB3_CMD_RESTART")) {
|
||||
reply(exchange, leadPlan("RESTART", body));
|
||||
return;
|
||||
}
|
||||
throw new AssertionError("Unknown model request: " + first);
|
||||
} catch (Exception | AssertionError failure) {
|
||||
modelFailure.compareAndSet(null, failure);
|
||||
try {
|
||||
// Never synthesize a tool call on the way out: a fixture
|
||||
// failure ends the Turn quietly while the latch reports it.
|
||||
reply(exchange, List.of());
|
||||
} catch (Exception ignored) {
|
||||
}
|
||||
} finally {
|
||||
exchange.close();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The lead's next move: create the team, launch the members as their
|
||||
* rule says (a member launch beside a team create is its batch
|
||||
* refusal), read the board once every report has arrived, then end.
|
||||
* bob launches only after alice's own report has landed: a wake that
|
||||
* arrives beside a busy Turn rides its followUp and settles tagged
|
||||
* but unrendered, so every member wake must find the lead idle for
|
||||
* its tag to reach a model request (and the lead-list that reads the
|
||||
* board after both).
|
||||
*/
|
||||
private Step leadPlan(String kind, JsonNode body) {
|
||||
long rounds = countRoleTool(body);
|
||||
seenReports.addAll(reportedIn(body));
|
||||
Set<String> reported = new LinkedHashSet<>(seenReports);
|
||||
boolean wake = List.of("alice", "bob", "carol").stream()
|
||||
.anyMatch(reported::contains);
|
||||
if (!wake) {
|
||||
if (rounds == 0) {
|
||||
return new Step(List.of(call("lead-create", "team_create",
|
||||
Map.of("team_name", "review"))), null);
|
||||
}
|
||||
if (rounds == 1) {
|
||||
return new Step(List.of("LIFECYCLE".equals(kind)
|
||||
? call("lead-alice", "agent",
|
||||
Map.of("description", "alice review",
|
||||
"prompt",
|
||||
"H4EB3_CMD_MEMBER_alice_DONE audit one",
|
||||
"name", "alice"))
|
||||
: call("lead-carol", "agent",
|
||||
Map.of("description", "carol review",
|
||||
"prompt",
|
||||
"H4EB3_CMD_MEMBER_carol_DONE slow audit",
|
||||
"name", "carol"))), null);
|
||||
}
|
||||
return new Step(List.of(), null);
|
||||
}
|
||||
if ("LIFECYCLE".equals(kind)) {
|
||||
if (reported.containsAll(List.of("alice", "bob"))
|
||||
&& boardReadFired.compareAndSet(false, true)) {
|
||||
return new Step(
|
||||
List.of(call("lead-list", "task_list", Map.of())),
|
||||
null);
|
||||
}
|
||||
if (reported.contains("alice")
|
||||
&& bobLaunched.compareAndSet(false, true)) {
|
||||
return new Step(List.of(call("lead-bob", "agent",
|
||||
Map.of("description", "bob review", "prompt",
|
||||
"H4EB3_CMD_MEMBER_bob_DONE doc sync",
|
||||
"name", "bob"))), null);
|
||||
}
|
||||
return new Step(List.of(), null);
|
||||
}
|
||||
return new Step(List.of(), null);
|
||||
}
|
||||
|
||||
/** The member names whose reports have reached the lead in this body. */
|
||||
private static Set<String> reportedIn(JsonNode body) {
|
||||
Set<String> names = new LinkedHashSet<>();
|
||||
for (JsonNode message : body.path("messages")) {
|
||||
if (!"user".equals(message.path("role").asText())) continue;
|
||||
names.addAll(teammates(message.toString()));
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
/**
|
||||
* A member's Turn plan: first its board calls, then its report text.
|
||||
* Alice claims the task her own create minted — her create's reply is
|
||||
* read for its number, so bob's write in flight never re-homes it.
|
||||
*/
|
||||
private Step memberPlan(String name, JsonNode body) {
|
||||
long results = countRoleTool(body);
|
||||
switch (name) {
|
||||
case "alice" -> {
|
||||
if (results == 0) {
|
||||
return new Step(List.of(call("alice-create", "task_create",
|
||||
Map.of("subject", "Audit one",
|
||||
"description", "audit one"))), null);
|
||||
}
|
||||
if (results == 1) {
|
||||
return new Step(List.of(call("alice-claim", "task_update",
|
||||
Map.of("taskId", createdNumber(body), "status",
|
||||
"in_progress")),
|
||||
call("alice-list", "task_list", Map.of())), null);
|
||||
}
|
||||
return new Step(List.of(), reportFor(name));
|
||||
}
|
||||
case "bob" -> {
|
||||
if (results == 0) {
|
||||
return new Step(List.of(
|
||||
call("bob-create", "task_create",
|
||||
Map.of("subject", "Doc sync",
|
||||
"description", "sync the docs")),
|
||||
call("bob-list", "task_list", Map.of())), null);
|
||||
}
|
||||
return new Step(List.of(), reportFor(name));
|
||||
}
|
||||
case "carol" -> {
|
||||
if (results == 0) {
|
||||
return new Step(List.of(call("carol-create", "task_create",
|
||||
Map.of("subject", "Slow audit",
|
||||
"description", "a slow audit"))), null);
|
||||
}
|
||||
return new Step(List.of(), reportFor(name));
|
||||
}
|
||||
default -> throw new AssertionError("Unknown member " + name);
|
||||
}
|
||||
}
|
||||
|
||||
private record Step(List<Map<String, Object>> calls, String text) {
|
||||
}
|
||||
|
||||
/** A tool answer as plain text, whatever JSON escaping carried it. */
|
||||
private static String plain(String text) {
|
||||
String out = text;
|
||||
for (int pass = 0; pass < 4; pass++)
|
||||
out = out.replace("\\\\", "\\").replace("\\\"", "\"")
|
||||
.replace("\\n", "\n");
|
||||
return out;
|
||||
}
|
||||
|
||||
/** The number a member's own task_create opened, from its reply. */
|
||||
private String createdNumber(JsonNode body) {
|
||||
for (JsonNode message : body.path("messages")) {
|
||||
if (!"tool".equals(message.path("role").asText())) continue;
|
||||
String text = plain(message.path("content").toString());
|
||||
Matcher number = Pattern.compile(
|
||||
"Task #(\\d+) created").matcher(text);
|
||||
if (number.find()) return number.group(1);
|
||||
}
|
||||
throw new AssertionError("member task_create reply lacks its number");
|
||||
}
|
||||
|
||||
/** A member's final text after its calls: her result to the leader. */
|
||||
private static String reportFor(String name) {
|
||||
return name.toUpperCase() + "_REPORT all clean";
|
||||
}
|
||||
|
||||
private static long countRoleTool(JsonNode body) {
|
||||
long results = 0;
|
||||
for (JsonNode message : body.path("messages"))
|
||||
if ("tool".equals(message.path("role").asText())) results += 1;
|
||||
return results;
|
||||
}
|
||||
|
||||
private static Map<String, Object> call(String id, String name,
|
||||
Map<String, Object> args) {
|
||||
return Map.of("id", id, "name", name, "args", args);
|
||||
}
|
||||
|
||||
private void reply(HttpExchange exchange, Step step) throws IOException {
|
||||
reply(exchange, step.calls(), step.text());
|
||||
}
|
||||
|
||||
private void reply(HttpExchange exchange,
|
||||
List<Map<String, Object>> calls) throws IOException {
|
||||
reply(exchange, calls, null);
|
||||
}
|
||||
|
||||
private void reply(HttpExchange exchange,
|
||||
List<Map<String, Object>> calls, String text) throws IOException {
|
||||
ObjectNode chunk = json.createObjectNode().put("id", "h4eb3")
|
||||
.put("object", "chat.completion.chunk").put("created", 0)
|
||||
.put("model", "h4eb3-fixture");
|
||||
ObjectNode choice = chunk.putArray("choices").addObject()
|
||||
.put("index", 0);
|
||||
ObjectNode delta = choice.putObject("delta").put("role",
|
||||
"assistant");
|
||||
if (calls.isEmpty()) {
|
||||
delta.put("content", text == null ? "" : text);
|
||||
} else {
|
||||
ArrayNode toolCalls = delta.putArray("tool_calls");
|
||||
for (int index = 0; index < calls.size(); index++) {
|
||||
Map<String, Object> call = calls.get(index);
|
||||
toolCalls.addObject().put("index", index)
|
||||
.put("id", call.get("id").toString())
|
||||
.put("type", "function")
|
||||
.putObject("function")
|
||||
.put("name", call.get("name").toString())
|
||||
.put("arguments",
|
||||
json.writeValueAsString(call.get("args")));
|
||||
}
|
||||
}
|
||||
choice.putNull("finish_reason");
|
||||
String first = "data: " + json.writeValueAsString(chunk) + "\n\n";
|
||||
choice.putObject("delta");
|
||||
choice.put("finish_reason", calls.isEmpty() ? "stop" : "tool_calls");
|
||||
byte[] response = (first + "data: "
|
||||
+ json.writeValueAsString(chunk) + "\n\ndata: [DONE]\n\n")
|
||||
.getBytes(StandardCharsets.UTF_8);
|
||||
exchange.getResponseHeaders().set("Content-Type",
|
||||
"text/event-stream");
|
||||
exchange.sendResponseHeaders(200, response.length);
|
||||
exchange.getResponseBody().write(response);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------- queries
|
||||
|
||||
private static String firstUserText(JsonNode body) {
|
||||
for (JsonNode message : body.path("messages")) {
|
||||
if (!"user".equals(message.path("role").asText())) continue;
|
||||
String text = message.path("content").isTextual()
|
||||
? message.path("content").asText()
|
||||
: message.path("content").toString();
|
||||
if (text.contains("H4EB3_")) return text;
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
private static Set<String> teammates(String text) {
|
||||
Set<String> names = new LinkedHashSet<>();
|
||||
Matcher matcher = TEAMMATE.matcher(text.replace("\\u003c", "<")
|
||||
.replace("\\u003e", ">"));
|
||||
while (matcher.find()) names.add(matcher.group(1));
|
||||
return names;
|
||||
}
|
||||
|
||||
/** Every command chain a member committed, newest revision each. */
|
||||
private List<JsonNode> commandsOf(String lead, String member) {
|
||||
List<String> children = jdbc.queryForList(
|
||||
"SELECT session_id FROM managed_agent_session"
|
||||
+ " WHERE tenant_id = ? AND parent_session_id = ?",
|
||||
String.class, tenant, lead);
|
||||
for (String child : children) {
|
||||
List<Map<String, Object>> commands = rows(child,
|
||||
"session_command");
|
||||
if (!commands.isEmpty()) {
|
||||
JsonNode task = team(lead);
|
||||
if (task != null && task.toString().contains("\"" + member
|
||||
+ "\"")) {
|
||||
List<JsonNode> out = new ArrayList<>();
|
||||
for (Map<String, Object> row : commands)
|
||||
out.add(body(row));
|
||||
return out;
|
||||
}
|
||||
}
|
||||
}
|
||||
return List.of();
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> rows(String session, String domain) {
|
||||
return jdbc.queryForList(
|
||||
"SELECT r.record_id, r.revision, CAST(res.inline_bytes AS CHAR)"
|
||||
+ " AS body FROM"
|
||||
+ " qwen_managed_session_extension_record r"
|
||||
+ " JOIN qwen_managed_session_resource res ON"
|
||||
+ " res.session_scope_key = r.session_scope_key"
|
||||
+ " AND res.resource_id = r.record_resource_id"
|
||||
+ " WHERE r.tenant_id = ? AND r.session_id = ?"
|
||||
+ " AND r.domain = ? ORDER BY r.record_id",
|
||||
tenant, session, domain);
|
||||
}
|
||||
|
||||
private JsonNode body(Map<String, Object> row) {
|
||||
try {
|
||||
return json.readTree(row.get("body").toString());
|
||||
} catch (IOException failure) {
|
||||
throw new AssertionError(failure);
|
||||
}
|
||||
}
|
||||
|
||||
private JsonNode team(String session) {
|
||||
List<Map<String, Object>> rows = rows(session, "team_state");
|
||||
return rows.isEmpty() ? null : body(rows.get(0));
|
||||
}
|
||||
|
||||
private String operationStatus(String session, String operation)
|
||||
throws Exception {
|
||||
return request("GET", "/v1/agents/sessions/" + session + "/operations/"
|
||||
+ operation, null, null, 200).path("status").asText();
|
||||
}
|
||||
|
||||
private void assertNoModelFailure() {
|
||||
Throwable failure = modelFailure.get();
|
||||
if (failure != null) {
|
||||
throw new IllegalStateException("The model fixture failed",
|
||||
failure);
|
||||
}
|
||||
}
|
||||
|
||||
private void assertNoFailedChild(String lead) {
|
||||
for (Map<String, Object> row : rows(lead, "child_run")) {
|
||||
JsonNode run = body(row);
|
||||
if ("failed".equals(run.at("/run/state").asText())) {
|
||||
throw new IllegalStateException("A member failed: " + run);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void assertNoFailedTurn(String session) {
|
||||
List<Map<String, Object>> failed = jdbc.queryForList(
|
||||
"SELECT turn_id, error_code FROM managed_agent_turn"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND status = 'FAILED'",
|
||||
tenant, session);
|
||||
if (!failed.isEmpty()) {
|
||||
throw new IllegalStateException("Turns failed: " + failed);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The session's in-flight Turns: every terminal write (SETTLED,
|
||||
* FAILED, CANCELLED) sets completed_at.
|
||||
*/
|
||||
private List<Map<String, Object>> openTurns(String session) {
|
||||
return jdbc.queryForList(
|
||||
"SELECT turn_id, status FROM managed_agent_turn"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND completed_at IS NULL",
|
||||
tenant, session);
|
||||
}
|
||||
|
||||
/** The restart fixture's single member: the lead's only child. */
|
||||
private String onlyChildOf(String lead) {
|
||||
List<String> children = jdbc.queryForList(
|
||||
"SELECT session_id FROM managed_agent_session"
|
||||
+ " WHERE tenant_id = ? AND parent_session_id = ?",
|
||||
String.class, tenant, lead);
|
||||
if (children.size() != 1) {
|
||||
throw new IllegalStateException("expected one member session: "
|
||||
+ children);
|
||||
}
|
||||
return children.get(0);
|
||||
}
|
||||
|
||||
/** Whether the lead's journal accepted a member result durably. */
|
||||
private boolean childResultAccepted(String lead) {
|
||||
Integer accepted = jdbc.queryForObject(
|
||||
"SELECT COUNT(*) FROM qwen_managed_session_journal_tx"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND operation = 'acceptChildResult'",
|
||||
Integer.class, tenant, lead);
|
||||
return accepted != null && accepted > 0;
|
||||
}
|
||||
|
||||
/** Whether the Session parks a Turn on an `await_command` checkpoint. */
|
||||
private boolean parkedOnCommand(String session) {
|
||||
Integer parked = jdbc.queryForObject(
|
||||
"SELECT COUNT(*) FROM qwen_managed_session_resource"
|
||||
+ " WHERE tenant_id = ? AND session_id = ?"
|
||||
+ " AND kind = 'managed-checkpoint'"
|
||||
+ " AND CAST(inline_bytes AS CHAR) LIKE"
|
||||
+ " '%await_command%'",
|
||||
Integer.class, tenant, session);
|
||||
return parked != null && parked > 0;
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------- the stack
|
||||
|
||||
private void boot() throws Exception {
|
||||
Path cli = Path.of(System.getProperty("qwen.cli.entry",
|
||||
"../../../dist/cli.js")).toAbsolutePath();
|
||||
assertThat(cli).as("Build and bundle the CLI first").isRegularFile();
|
||||
String node = System.getProperty("node.executable");
|
||||
assertThat(node).as("Pass -Dnode.executable with a Node.js 22+ path")
|
||||
.isNotBlank();
|
||||
assertThat(System.getProperty("mysql.url"))
|
||||
.as("Pass -Dmysql.url for a disposable MySQL schema")
|
||||
.isNotBlank();
|
||||
temporary = temporary.toRealPath();
|
||||
Files.createDirectory(temporary.resolve("broker"),
|
||||
PosixFilePermissions.asFileAttribute(
|
||||
PosixFilePermissions.fromString("rwx------")));
|
||||
Path workspace = Files
|
||||
.createDirectory(temporary.resolve("workspace-a"));
|
||||
Files.createDirectory(workspace.resolve("child"));
|
||||
// The Runtime Broker resolves its workspace cwd eagerly; the decoy
|
||||
// directory must exist before the Spring context asks for it.
|
||||
Files.createDirectory(temporary.resolve("harness-decoy"));
|
||||
port = freePort();
|
||||
harnessPort = freePort();
|
||||
brokerPort = freePort();
|
||||
model = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
|
||||
modelThreads = Executors.newCachedThreadPool();
|
||||
model.setExecutor(modelThreads);
|
||||
model.createContext("/v1/chat/completions", this::modelReply);
|
||||
model.start();
|
||||
startSpring(cli, node, workspace);
|
||||
jdbc.update(
|
||||
"INSERT INTO managed_workspace_registry (tenant_id, workspace_id,"
|
||||
+ " workspace_generation, storage_id, display_name,"
|
||||
+ " config_ref, policy_ref, state) VALUES (?,"
|
||||
+ " 'workspace-0', 1, 'storage-0', 'H4e-b3', ?, ?,"
|
||||
+ " 'ACTIVE')",
|
||||
tenant, WorkspaceExecutionProfile.CONFIG_REF,
|
||||
WorkspaceExecutionProfile.POLICY_REF);
|
||||
jdbc.update(
|
||||
"INSERT INTO managed_workspace_access (tenant_id, workspace_id,"
|
||||
+ " actor_id, role) VALUES (?, 'workspace-0', ?,"
|
||||
+ " 'OPERATOR')",
|
||||
tenant, "actor".getBytes(StandardCharsets.UTF_8));
|
||||
startHarness();
|
||||
}
|
||||
|
||||
private void startSpring(Path cli, String node, Path workspace) {
|
||||
List<String> arguments = new ArrayList<>(List.of(
|
||||
"--server.address=127.0.0.1", "--server.port=" + port,
|
||||
"--spring.datasource.url="
|
||||
+ System.getProperty("mysql.url"),
|
||||
"--spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver",
|
||||
"--spring.datasource.username=" + System.getProperty("mysql.user"),
|
||||
"--spring.datasource.password="
|
||||
+ System.getProperty("mysql.password", ""),
|
||||
"--qwen.managed-agent.session-store.enabled=true",
|
||||
"--qwen.managed-agent.session-store.base-url=http://127.0.0.1:"
|
||||
+ port,
|
||||
"--qwen.managed-agent.session-store.workspace-id=unused-global-workspace",
|
||||
"--qwen.managed-agent.harness.enabled=true",
|
||||
"--qwen.managed-agent.harness.workspace-files-enabled=true",
|
||||
"--qwen.managed-agent.harness.base-url=http://127.0.0.1:"
|
||||
+ harnessPort,
|
||||
"--qwen.managed-agent.harness.token=" + TOKEN,
|
||||
"--qwen.managed-agent.harness.capability-digest=" + DIGEST,
|
||||
"--qwen.managed-agent.runtime-broker.enabled=true",
|
||||
"--qwen.managed-agent.runtime-broker.port=" + brokerPort,
|
||||
"--qwen.managed-agent.runtime-broker.token=" + TOKEN,
|
||||
"--qwen.managed-agent.runtime-broker.durable-local-process=false",
|
||||
"--qwen.managed-agent.runtime-broker.trusted-local-reboot-recovery=false",
|
||||
"--qwen.managed-agent.runtime-broker.workspace-cwd="
|
||||
+ temporary.resolve("harness-decoy"),
|
||||
"--qwen.managed-agent.runtime-broker.state-directory="
|
||||
+ temporary.resolve("broker"),
|
||||
"--qwen.managed-agent.runtime-broker.credential-key-id=h4eb3-fixture",
|
||||
"--qwen.managed-agent.runtime-broker.credential-key="
|
||||
+ "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=",
|
||||
"--qwen.managed-agent.runtime-broker.node-executable=" + node,
|
||||
"--qwen.managed-agent.runtime-broker.worker-entry=" + cli,
|
||||
"--qwen.managed-agent.runtime-broker.cli-entry=" + cli,
|
||||
"--qwen.managed-agent.runtime-broker.workspace-mounts[0].tenant-id="
|
||||
+ tenant,
|
||||
"--qwen.managed-agent.runtime-broker.workspace-mounts[0].storage-id=storage-0",
|
||||
"--qwen.managed-agent.runtime-broker.workspace-mounts[0].root="
|
||||
+ workspace));
|
||||
spring = (ServletWebServerApplicationContext)
|
||||
new SpringApplicationBuilder(ManagedAgentServerApplication.class)
|
||||
.initializers(context -> context.getBeanFactory()
|
||||
.registerSingleton("h4eb3Authentication",
|
||||
authentication()))
|
||||
.run(arguments.toArray(String[]::new));
|
||||
jdbc = spring.getBean(JdbcTemplate.class);
|
||||
}
|
||||
|
||||
private void startHarness() throws Exception {
|
||||
String node = System.getProperty("node.executable");
|
||||
Path cli = Path.of(System.getProperty("qwen.cli.entry",
|
||||
"../../../dist/cli.js")).toAbsolutePath();
|
||||
Path home = Files.createDirectory(
|
||||
temporary.resolve("home-" + UUID.randomUUID()));
|
||||
Path config = Files.createDirectory(home.resolve(".qwen"));
|
||||
String modelUrl = "http://127.0.0.1:" + model.getAddress().getPort()
|
||||
+ "/v1";
|
||||
Files.writeString(config.resolve("settings.json"),
|
||||
json.writeValueAsString(Map.of(
|
||||
"security",
|
||||
Map.of("auth", Map.of("selectedType", "openai")),
|
||||
"model", Map.of("name", "h4eb3-fixture"),
|
||||
"telemetry", Map.of("enabled", false),
|
||||
"modelProviders",
|
||||
Map.of("openai", List.of(Map.of("id", "h4eb3-fixture",
|
||||
"envKey", "OPENAI_API_KEY", "baseUrl",
|
||||
modelUrl))))));
|
||||
Path log = temporary.resolve("harness.log");
|
||||
ProcessBuilder builder = new ProcessBuilder(node, cli.toString(),
|
||||
"serve", "--profile", "hosted-harness", "--http-bridge",
|
||||
"--hostname", "127.0.0.1", "--port",
|
||||
Integer.toString(harnessPort), "--require-auth", "--no-web",
|
||||
"--workspace", temporary.resolve("harness-decoy").toString(),
|
||||
"--managed-runtime-broker-url", "http://127.0.0.1:"
|
||||
+ brokerPort,
|
||||
"--managed-runtime-broker-token", TOKEN)
|
||||
.directory(temporary.resolve("harness-decoy").toFile())
|
||||
.redirectErrorStream(true).redirectOutput(log.toFile());
|
||||
Map<String, String> environment = builder.environment();
|
||||
environment.clear();
|
||||
if (System.getenv("PATH") != null)
|
||||
environment.put("PATH", System.getenv("PATH"));
|
||||
environment.putAll(Map.of(
|
||||
"HOME", home.toString(),
|
||||
"USERPROFILE", home.toString(),
|
||||
"QWEN_HOME", config.toString(),
|
||||
"QWEN_RUNTIME_DIR", temporary.resolve("runtime").toString(),
|
||||
"OPENAI_API_KEY", "fake-local-key",
|
||||
"OPENAI_BASE_URL", modelUrl,
|
||||
"QWEN_SERVER_TOKEN", TOKEN,
|
||||
"QWEN_HOSTED_HARNESS_CAPABILITY_DIGEST", DIGEST));
|
||||
environment.put("TMPDIR", temporary.toString());
|
||||
environment.put("TMP", temporary.toString());
|
||||
environment.put("TEMP", temporary.toString());
|
||||
for (String name : List.of("QWEN_CODE_SYSTEM_SETTINGS_PATH",
|
||||
"QWEN_CODE_SYSTEM_DEFAULTS_PATH",
|
||||
"QWEN_CODE_TRUSTED_FOLDERS_PATH"))
|
||||
environment.put(name,
|
||||
temporary.resolve(name + ".json").toString());
|
||||
harness = builder.start();
|
||||
waiting().atMost(Duration.ofSeconds(30)).untilAsserted(() -> {
|
||||
if (!harness.isAlive()) {
|
||||
throw new IllegalStateException(
|
||||
"Hosted Harness exited: " + Files.readString(log));
|
||||
}
|
||||
int status;
|
||||
try {
|
||||
// --require-auth gates every route too — the loopback
|
||||
// /health probe needs the same bearer as the routes.
|
||||
HttpResponse<String> ready = http.send(
|
||||
HttpRequest.newBuilder(URI.create("http://127.0.0.1:"
|
||||
+ harnessPort + "/health"))
|
||||
.header("authorization", "Bearer " + TOKEN)
|
||||
.build(),
|
||||
HttpResponse.BodyHandlers.ofString());
|
||||
status = ready.statusCode();
|
||||
} catch (IOException ignored) {
|
||||
status = -1;
|
||||
}
|
||||
assertThat(status).isEqualTo(200);
|
||||
});
|
||||
}
|
||||
|
||||
private JsonNode request(String method, String path, Object body,
|
||||
String key, int expected) throws Exception {
|
||||
HttpRequest.Builder request = HttpRequest.newBuilder(
|
||||
URI.create("http://127.0.0.1:" + port + path))
|
||||
.timeout(Duration.ofSeconds(15))
|
||||
.header("X-Qwen-Tenant-Id", tenant)
|
||||
.header("Accept", "application/json")
|
||||
.header("Content-Type", "application/json")
|
||||
.header("X-H4eb3-Fixture-Actor", "actor")
|
||||
.method(method,
|
||||
body == null ? HttpRequest.BodyPublishers.noBody()
|
||||
: HttpRequest.BodyPublishers.ofString(
|
||||
json.writeValueAsString(body)));
|
||||
if (key != null) request.header("Idempotency-Key", key);
|
||||
HttpResponse<String> response = http.send(request.build(),
|
||||
HttpResponse.BodyHandlers.ofString());
|
||||
assertThat(response.statusCode())
|
||||
.as("%s %s: %s", method, path, response.body())
|
||||
.isEqualTo(expected);
|
||||
return json.readTree(response.body());
|
||||
}
|
||||
|
||||
private FilterRegistrationBean<Filter> authentication() {
|
||||
FilterRegistrationBean<Filter> filter = new FilterRegistrationBean<>(
|
||||
(request, response, chain) -> chain.doFilter(
|
||||
new HttpServletRequestWrapper(
|
||||
(HttpServletRequest) request) {
|
||||
@Override
|
||||
public Principal getUserPrincipal() {
|
||||
String actor = getHeader(
|
||||
"X-H4eb3-Fixture-Actor");
|
||||
return actor == null ? null
|
||||
: new AuthenticatedTenantActor() {
|
||||
public String tenantId() {
|
||||
return tenant;
|
||||
}
|
||||
|
||||
public String actorId() {
|
||||
return actor;
|
||||
}
|
||||
|
||||
public String getName() {
|
||||
return actor;
|
||||
}
|
||||
};
|
||||
}
|
||||
}, response));
|
||||
filter.setOrder(Ordered.HIGHEST_PRECEDENCE);
|
||||
filter.setAsyncSupported(true);
|
||||
filter.addUrlPatterns("/v1/agents/*");
|
||||
return filter;
|
||||
}
|
||||
|
||||
private static int freePort() throws IOException {
|
||||
try (ServerSocket socket = new ServerSocket(0)) {
|
||||
return socket.getLocalPort();
|
||||
}
|
||||
}
|
||||
|
||||
private static ConditionFactory waiting() {
|
||||
return await().failFast("The case's @Timeout interrupted it",
|
||||
() -> Thread.currentThread().isInterrupted());
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
package com.alibaba.qwen.code.managedagent;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertNull;
|
||||
import static org.junit.jupiter.api.Assertions.assertThrows;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedExtensionProjection;
|
||||
import com.alibaba.qwen.code.managedagent.store.ManagedExtensionRecords.InvalidRecordException;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.fasterxml.jackson.databind.node.ObjectNode;
|
||||
import java.io.IOException;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Objects;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
/**
|
||||
* Replays the language-neutral managed-session-command-record/1 fixtures
|
||||
* (H4e-b3 of #12827) that the TypeScript module replays too.
|
||||
*/
|
||||
class ManagedSessionCommandRecordContractTest {
|
||||
private static final ObjectMapper JSON = new ObjectMapper();
|
||||
|
||||
@Test
|
||||
void pinsTheClosedAndFixedKeys() throws IOException {
|
||||
JsonNode fixtures = fixtures();
|
||||
assertEquals("managed-session-command-record/1",
|
||||
fixtures.required("contract").textValue());
|
||||
assertEquals(List.of("argsDigest", "argsRef", "childRunId",
|
||||
"commandId", "direction", "kind", "replyDigest", "replyRef",
|
||||
"route", "run", "senderSessionId", "targetSessionId"),
|
||||
jsonList(fixtures.required("keys")));
|
||||
assertEquals(List.of("argsDigest", "argsRef", "childRunId",
|
||||
"commandId", "direction", "kind", "route",
|
||||
"senderSessionId"),
|
||||
jsonList(fixtures.required("fixedKeys")));
|
||||
}
|
||||
|
||||
@Test
|
||||
void validatesTheSharedRecordsAndStarts() throws IOException {
|
||||
JsonNode fixtures = fixtures();
|
||||
var body = ManagedExtensionProjection.RECORD_BODIES
|
||||
.get("session_command");
|
||||
for (JsonNode fixture : fixtures.get("cases")) {
|
||||
String id = fixture.get("id").textValue();
|
||||
JsonNode record = merge(template(fixtures, fixture),
|
||||
fixture.get("patch"));
|
||||
if (fixture.get("valid").booleanValue()) {
|
||||
body.require().accept(record);
|
||||
assertNull(body.taskKindOf().apply(record), id);
|
||||
assertEquals(record.get("commandId").textValue(),
|
||||
body.recordId().apply(record), id);
|
||||
} else {
|
||||
InvalidRecordException refused = assertThrows(
|
||||
InvalidRecordException.class,
|
||||
() -> body.require().accept(record), id);
|
||||
// Every invalid case names the clause that must refuse it,
|
||||
// so a masked guard can never slip a fixture green.
|
||||
assertTrue(refused.getMessage()
|
||||
.contains(fixture.get("error").textValue()),
|
||||
id + ": " + refused.getMessage());
|
||||
}
|
||||
assertEquals(fixture.get("start").booleanValue(),
|
||||
body.isStart().test(record), id);
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void validatesTheSharedSuccessors() throws IOException {
|
||||
JsonNode fixtures = fixtures();
|
||||
for (JsonNode fixture : fixtures.get("successors")) {
|
||||
JsonNode base = template(fixtures, fixture);
|
||||
assertEquals(fixture.get("valid").booleanValue(),
|
||||
ManagedExtensionProjection.RECORD_BODIES
|
||||
.get("session_command").isSuccessor().test(
|
||||
merge(base, fixture.get("before")),
|
||||
merge(base, fixture.get("after"))),
|
||||
fixture.get("id").textValue());
|
||||
}
|
||||
}
|
||||
|
||||
private static JsonNode template(JsonNode fixtures, JsonNode fixture) {
|
||||
return Objects.requireNonNull(fixtures.get("templates")
|
||||
.get(fixture.get("template").textValue()),
|
||||
() -> fixture.get("id").textValue()
|
||||
+ " names an unknown template");
|
||||
}
|
||||
|
||||
private static List<String> jsonList(JsonNode node) {
|
||||
List<String> values = new ArrayList<>();
|
||||
node.forEach(each -> values.add(each.textValue()));
|
||||
return values;
|
||||
}
|
||||
|
||||
static JsonNode fixtures() throws IOException {
|
||||
Path directory = Path.of("").toAbsolutePath();
|
||||
while (directory != null) {
|
||||
Path path = directory.resolve(
|
||||
"packages/core/src/managed-runtime/contracts/managed-session-command-record-v1.fixtures.json");
|
||||
if (Files.exists(path)) {
|
||||
return JSON.readTree(Files.readString(path));
|
||||
}
|
||||
directory = directory.getParent();
|
||||
}
|
||||
throw new IOException("Session command contract fixtures not found");
|
||||
}
|
||||
|
||||
static JsonNode merge(JsonNode base, JsonNode patch) {
|
||||
ObjectNode value = base == null || !base.isObject()
|
||||
? JSON.createObjectNode() : base.deepCopy();
|
||||
patch.fields().forEachRemaining(entry -> value.set(entry.getKey(),
|
||||
entry.getValue().isObject()
|
||||
? merge(value.get(entry.getKey()), entry.getValue())
|
||||
: entry.getValue().deepCopy()));
|
||||
return value;
|
||||
}
|
||||
}
|
||||
|
|
@ -11,6 +11,7 @@ import com.alibaba.qwen.code.managedagent.store.SessionMessageRelayStore.Pending
|
|||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.Arrays;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
|
@ -43,6 +44,17 @@ class SessionMessageRelayStoreTest {
|
|||
|
||||
private void message(String sessionId, String messageId, String state,
|
||||
String body) {
|
||||
extension("session_message", sessionId, messageId, state, body);
|
||||
}
|
||||
|
||||
/** A member's outbound command entry, as the open team tools commit it. */
|
||||
private void command(String sessionId, String commandId, String state,
|
||||
String body) {
|
||||
extension("session_command", sessionId, commandId, state, body);
|
||||
}
|
||||
|
||||
private void extension(String domain, String sessionId, String recordId,
|
||||
String state, String body) {
|
||||
String resource = "resource-" + IDS.incrementAndGet();
|
||||
resource(sessionId, resource, body.getBytes(StandardCharsets.UTF_8));
|
||||
jdbc.update("INSERT INTO qwen_managed_session_extension_record"
|
||||
|
|
@ -50,11 +62,10 @@ class SessionMessageRelayStoreTest {
|
|||
+ " workspace_id, session_id, domain, record_id,"
|
||||
+ " operation_hash, revision, record_resource_id,"
|
||||
+ " delivery_target, delivery_state, created_at)"
|
||||
+ " VALUES ('scope', ?, ?, 'workspace', ?,"
|
||||
+ " 'session_message', ?, 'h', 1, ?, 'session', ?,"
|
||||
+ " ?)",
|
||||
sessionId + "/" + messageId, TENANT, sessionId, messageId,
|
||||
resource, state, IDS.incrementAndGet());
|
||||
+ " VALUES ('scope', ?, ?, 'workspace', ?, ?, ?,"
|
||||
+ " 'h', 1, ?, 'session', ?, ?)",
|
||||
sessionId + "/" + recordId, TENANT, sessionId, domain,
|
||||
recordId, resource, state, IDS.incrementAndGet());
|
||||
}
|
||||
|
||||
private static String outbound(String route, String childRunId) {
|
||||
|
|
@ -176,9 +187,59 @@ class SessionMessageRelayStoreTest {
|
|||
assertThat(store.findPendingMessages("owner-b", 1_000, 50).stream()
|
||||
.map(PendingMessage::messageId))
|
||||
.doesNotContain("msg_waiting");
|
||||
assertThat(store.deliveryState(TENANT, target, "msg_receipt"))
|
||||
.isEqualTo("accepted");
|
||||
assertThat(store.deliveryState(TENANT, target, "msg_none")).isNull();
|
||||
assertThat(
|
||||
store.deliveryState(TENANT, target, "session_message",
|
||||
"msg_receipt")).isEqualTo("accepted");
|
||||
assertThat(store.deliveryState(TENANT, target, "session_message",
|
||||
"msg_none")).isNull();
|
||||
}
|
||||
|
||||
@Test
|
||||
void discoversCommandsAndMessagesOfOneSenderWithTheirDomains() {
|
||||
String sender = UUID.randomUUID().toString();
|
||||
message(sender, "msg_planned", "planned", outbound("to_child", "r"));
|
||||
command(sender, "cmd_handover", "planned",
|
||||
"{\"route\":\"to_parent\"}");
|
||||
command(sender, "cmd_executing", "accepting",
|
||||
"{\"route\":\"to_parent\"}");
|
||||
command(sender, "cmd_finished", "consumed",
|
||||
"{\"route\":\"to_parent\"}");
|
||||
Map<String, String> page = new java.util.HashMap<>();
|
||||
for (PendingMessage entry : store.findPendingMessages("owner-a",
|
||||
1_000, 50))
|
||||
if (entry.tenantId().equals(TENANT)
|
||||
&& entry.senderSessionId().equals(sender))
|
||||
page.put(entry.messageId(), entry.domain());
|
||||
assertThat(page).containsExactlyInAnyOrderEntriesOf(Map.of(
|
||||
"msg_planned", "session_message",
|
||||
"cmd_handover", "session_command",
|
||||
"cmd_executing", "session_command"));
|
||||
}
|
||||
|
||||
// One sender's commands pass strictly in order — but only across the
|
||||
// open window: an unclassified row over a planned/accepting record.
|
||||
// A cancelled entry's leftover relaying row never gets another scan
|
||||
// (HANDOVER_SQL no longer sees the record), so it must not hold the
|
||||
// younger command behind it; a classified row never holds either.
|
||||
@Test
|
||||
void ordersCommandsByTheirOpenWindowOnly() {
|
||||
String sender = UUID.randomUUID().toString();
|
||||
command(sender, "cmd_1_cancelled", "cancelled",
|
||||
"{\"route\":\"to_parent\"}");
|
||||
command(sender, "cmd_2_open", "accepting",
|
||||
"{\"route\":\"to_parent\"}");
|
||||
command(sender, "cmd_3_new", "planned", "{\"route\":\"to_parent\"}");
|
||||
store.claim(TENANT, sender, "cmd_1_cancelled", "owner-a", 31_000,
|
||||
1_000);
|
||||
assertThat(store.olderOpenCommand(TENANT, sender, "cmd_2_open"))
|
||||
.isFalse();
|
||||
store.claim(TENANT, sender, "cmd_2_open", "owner-a", 31_000, 1_000);
|
||||
assertThat(store.olderOpenCommand(TENANT, sender, "cmd_3_new"))
|
||||
.isTrue();
|
||||
store.classify(store.find(TENANT, sender, "cmd_2_open"), "owner-a",
|
||||
"unknown", "gave up", 2_000);
|
||||
assertThat(store.olderOpenCommand(TENANT, sender, "cmd_3_new"))
|
||||
.isFalse();
|
||||
}
|
||||
|
||||
@Test
|
||||
|
|
|
|||
|
|
@ -104,14 +104,15 @@ class ChildResultRelayTest {
|
|||
}
|
||||
|
||||
@Override
|
||||
public void runMessageOperation(String tenantId, String sessionId,
|
||||
Map<String, Object> body) {
|
||||
public Map<String, Object> runMessageOperation(String tenantId,
|
||||
String sessionId, Map<String, Object> body) {
|
||||
if (refuseMessageCode != null) {
|
||||
throw refusal(refuseMessageCode);
|
||||
}
|
||||
Map<String, Object> recorded = new LinkedHashMap<>(body);
|
||||
recorded.put("sessionId", sessionId);
|
||||
messageOperations.add(recorded);
|
||||
return Map.of();
|
||||
}
|
||||
|
||||
@Override
|
||||
|
|
|
|||
|
|
@ -58,6 +58,8 @@ class SessionMessageRelayTest {
|
|||
volatile String refuseKind;
|
||||
volatile String refuseCode;
|
||||
volatile String refuseSecondKind;
|
||||
/** The settled body a verb comes back with (the execute reply). */
|
||||
volatile Object reply;
|
||||
volatile boolean available = true;
|
||||
|
||||
@Override
|
||||
|
|
@ -98,14 +100,15 @@ class SessionMessageRelayTest {
|
|||
}
|
||||
|
||||
@Override
|
||||
public void runMessageOperation(String tenantId, String sessionId,
|
||||
Map<String, Object> body) {
|
||||
public Map<String, Object> runMessageOperation(String tenantId,
|
||||
String sessionId, Map<String, Object> body) {
|
||||
if (refuseKind != null && refuseKind.equals(body.get("kind"))
|
||||
|| refuseSecondKind != null
|
||||
&& refuseSecondKind.equals(body.get("kind"))) {
|
||||
throw refusal(refuseCode);
|
||||
}
|
||||
calls.add(new Call(sessionId, Map.copyOf(body)));
|
||||
return reply == null ? Map.of() : Map.of("reply", reply);
|
||||
}
|
||||
|
||||
static DaemonHttpException refusal(String code) {
|
||||
|
|
@ -131,7 +134,7 @@ class SessionMessageRelayTest {
|
|||
row = new AtomicReference<>(new MessageRow(TENANT, PARENT, MESSAGE,
|
||||
null, "relaying", "owner", 31_000L, 0, 0, null));
|
||||
pending = new AtomicReference<>(new PendingMessage(TENANT, PARENT,
|
||||
MESSAGE, "planned", "resource-message"));
|
||||
MESSAGE, "session_message", "planned", "resource-message"));
|
||||
when(store.findPendingMessages(anyString(), anyLong(),
|
||||
Mockito.anyInt())).thenAnswer(ignored -> List.of(pending.get()));
|
||||
when(store.claim(anyString(), anyString(), anyString(), anyString(),
|
||||
|
|
@ -168,7 +171,7 @@ class SessionMessageRelayTest {
|
|||
return null;
|
||||
}).when(store).defer(any(MessageRow.class), anyString(),
|
||||
anyLong(), any(), anyLong(), anyLong());
|
||||
when(store.currentRecord(anyString(), anyString(), anyString()))
|
||||
when(store.currentRecord(anyString(), anyString(), anyString(), anyString()))
|
||||
.thenAnswer(ignored -> new SessionMessageRelayStore
|
||||
.CurrentRecord(pending.get().deliveryState(),
|
||||
"resource-message"));
|
||||
|
|
@ -345,7 +348,7 @@ class SessionMessageRelayTest {
|
|||
void routesAChildMessageToItsRecordedParent() {
|
||||
row.set(new MessageRow(TENANT, CHILD, MESSAGE, null, "relaying",
|
||||
"owner", 31_000L, 0, 0, null));
|
||||
pending.set(new PendingMessage(TENANT, CHILD, MESSAGE, "planned",
|
||||
pending.set(new PendingMessage(TENANT, CHILD, MESSAGE, "session_message", "planned",
|
||||
"resource-message"));
|
||||
body("to_parent", CHILD, null);
|
||||
when(store.lineage(TENANT, CHILD))
|
||||
|
|
@ -374,7 +377,7 @@ class SessionMessageRelayTest {
|
|||
void leavesAStoppedChildsOwnMessageToItsClose() {
|
||||
row.set(new MessageRow(TENANT, CHILD, MESSAGE, null, "relaying",
|
||||
"owner", 31_000L, 0, 0, null));
|
||||
pending.set(new PendingMessage(TENANT, CHILD, MESSAGE, "planned",
|
||||
pending.set(new PendingMessage(TENANT, CHILD, MESSAGE, "session_message", "planned",
|
||||
"resource-message"));
|
||||
body("to_parent", CHILD, null);
|
||||
when(store.lineage(TENANT, CHILD))
|
||||
|
|
@ -397,7 +400,7 @@ class SessionMessageRelayTest {
|
|||
void handsOverAStoppedChildsMessageOnceNoMessageWorkIsOwed() {
|
||||
row.set(new MessageRow(TENANT, CHILD, MESSAGE, null, "relaying",
|
||||
"owner", 31_000L, 0, 0, null));
|
||||
pending.set(new PendingMessage(TENANT, CHILD, MESSAGE, "planned",
|
||||
pending.set(new PendingMessage(TENANT, CHILD, MESSAGE, "session_message", "planned",
|
||||
"resource-message"));
|
||||
body("to_parent", CHILD, null);
|
||||
when(store.lineage(TENANT, CHILD))
|
||||
|
|
@ -413,10 +416,10 @@ class SessionMessageRelayTest {
|
|||
// start the message its stop is settling: it waits for the close.
|
||||
@Test
|
||||
void neverReloadsAStoppedRunsChildToConsume() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepted",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepted",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
when(store.deliveryState(TENANT, CHILD, MESSAGE))
|
||||
when(store.deliveryState(TENANT, CHILD, "session_message", MESSAGE))
|
||||
.thenReturn("accepted");
|
||||
childRun("running", CHILD, true);
|
||||
childOwesMessageWork(1);
|
||||
|
|
@ -431,7 +434,7 @@ class SessionMessageRelayTest {
|
|||
|
||||
@Test
|
||||
void holdsAChildMessageThatItsParentCannotTakeYet() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepting",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepting",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
harness.refuseKind = "receive";
|
||||
|
|
@ -445,7 +448,7 @@ class SessionMessageRelayTest {
|
|||
|
||||
@Test
|
||||
void rejectsAMessageItsTargetRefusesForGood() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepting",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepting",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
harness.refuseKind = "receive";
|
||||
|
|
@ -457,10 +460,10 @@ class SessionMessageRelayTest {
|
|||
|
||||
@Test
|
||||
void replaysACommittedReceiptWithoutAnotherInput() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepting",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepting",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
when(store.deliveryState(TENANT, CHILD, MESSAGE))
|
||||
when(store.deliveryState(TENANT, CHILD, "session_message", MESSAGE))
|
||||
.thenReturn("accepted");
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("accepted");
|
||||
|
|
@ -472,17 +475,17 @@ class SessionMessageRelayTest {
|
|||
// sender's last step follows the receipt.
|
||||
@Test
|
||||
void advancesTheSenderOnlyOnceTheTargetConsumed() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepted",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepted",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
when(store.deliveryState(TENANT, CHILD, MESSAGE))
|
||||
when(store.deliveryState(TENANT, CHILD, "session_message", MESSAGE))
|
||||
.thenReturn("accepted");
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("consume");
|
||||
assertThat(harness.calls.getFirst().sessionId()).isEqualTo(CHILD);
|
||||
verify(store).scheduleRetry(any(MessageRow.class), anyString(),
|
||||
anyLong(), anyLong(), anyLong());
|
||||
when(store.deliveryState(TENANT, CHILD, MESSAGE))
|
||||
when(store.deliveryState(TENANT, CHILD, "session_message", MESSAGE))
|
||||
.thenReturn("consumed");
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("consume", "consumed");
|
||||
|
|
@ -491,10 +494,10 @@ class SessionMessageRelayTest {
|
|||
|
||||
@Test
|
||||
void waitsForAMessageNotReadYetWithoutSpendingAnAttempt() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepted",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepted",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
when(store.deliveryState(TENANT, CHILD, MESSAGE))
|
||||
when(store.deliveryState(TENANT, CHILD, "session_message", MESSAGE))
|
||||
.thenReturn("accepted");
|
||||
harness.refuseKind = "consume";
|
||||
harness.refuseCode = "session_message_not_ready";
|
||||
|
|
@ -505,10 +508,10 @@ class SessionMessageRelayTest {
|
|||
|
||||
@Test
|
||||
void finishesAMessageWhoseReadingTurnEndedIncomplete() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepted",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepted",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
when(store.deliveryState(TENANT, CHILD, MESSAGE))
|
||||
when(store.deliveryState(TENANT, CHILD, "session_message", MESSAGE))
|
||||
.thenReturn("accepted");
|
||||
harness.refuseKind = "consume";
|
||||
harness.refuseCode = "session_message_record";
|
||||
|
|
@ -520,7 +523,7 @@ class SessionMessageRelayTest {
|
|||
// A target's store fault answers 503: retried, never a rejection.
|
||||
@Test
|
||||
void retriesATargetStoreFaultInsteadOfRejecting() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepting",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepting",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
harness.refuseKind = "receive";
|
||||
|
|
@ -535,7 +538,7 @@ class SessionMessageRelayTest {
|
|||
// anywhere: retried, never rejected over a session named "null".
|
||||
@Test
|
||||
void neverDeliversAHandedOverMessageWithoutItsTarget() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepting",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepting",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, null);
|
||||
relay.scan();
|
||||
|
|
@ -555,7 +558,7 @@ class SessionMessageRelayTest {
|
|||
// child going away.
|
||||
@Test
|
||||
void rejectsAHandedOverMessageWhoseRunIsBeingStopped() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepting",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepting",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
childRun("running", CHILD, true);
|
||||
|
|
@ -567,10 +570,10 @@ class SessionMessageRelayTest {
|
|||
// The receipt landed before the sender closed: delivered, not orphaned.
|
||||
@Test
|
||||
void finishesAReceivedMessageWhoseSenderClosedBeforeItsAcceptance() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepting",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepting",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
when(store.deliveryState(TENANT, CHILD, MESSAGE))
|
||||
when(store.deliveryState(TENANT, CHILD, "session_message", MESSAGE))
|
||||
.thenReturn("accepted");
|
||||
when(records.sessionStatus(TENANT, PARENT)).thenReturn("CLOSED");
|
||||
relay.scan();
|
||||
|
|
@ -580,10 +583,10 @@ class SessionMessageRelayTest {
|
|||
|
||||
@Test
|
||||
void neverWidensAnUnconsumedMessageOfAClosedTarget() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepted",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepted",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
when(store.deliveryState(TENANT, CHILD, MESSAGE))
|
||||
when(store.deliveryState(TENANT, CHILD, "session_message", MESSAGE))
|
||||
.thenReturn("accepted");
|
||||
when(records.sessionStatus(TENANT, CHILD)).thenReturn("CLOSED");
|
||||
relay.scan();
|
||||
|
|
@ -604,7 +607,7 @@ class SessionMessageRelayTest {
|
|||
// stays the consumption truth.
|
||||
@Test
|
||||
void finishesADeliveredMessageWhoseSenderClosed() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepted",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepted",
|
||||
"resource-message"));
|
||||
when(records.sessionStatus(TENANT, PARENT)).thenReturn("CLOSED");
|
||||
relay.scan();
|
||||
|
|
@ -612,6 +615,243 @@ class SessionMessageRelayTest {
|
|||
assertThat(row.get().state()).isEqualTo("done");
|
||||
}
|
||||
|
||||
/** One outbound command as the member's funnel commits it. */
|
||||
private void commandBody(String kind, Object target) {
|
||||
String targetJson = target == null ? "null"
|
||||
: "\"" + target + "\"";
|
||||
when(records.readResource(TENANT, "resource-message")).thenReturn(
|
||||
"{\"direction\":\"outbound\",\"route\":\"to_parent\","
|
||||
+ "\"childRunId\":\"run-1\",\"kind\":\"" + kind
|
||||
+ "\",\"commandId\":\"" + MESSAGE
|
||||
+ "\",\"argsRef\":{\"resourceId\":\"resource-args\"},"
|
||||
+ "\"argsDigest\":\"a1\",\"targetSessionId\":"
|
||||
+ targetJson + "}");
|
||||
}
|
||||
|
||||
private PendingMessage commandPending(String state) {
|
||||
return new PendingMessage(TENANT, PARENT, MESSAGE, "session_command",
|
||||
state, "resource-message");
|
||||
}
|
||||
|
||||
// H4e-b3: a command does handover → execute → answered, never consume:
|
||||
// the parked tool call reads its own reply from the journal.
|
||||
@Test
|
||||
void carriesACommandFromHandoverThroughExecuteToItsReply() {
|
||||
pending.set(commandPending("planned"));
|
||||
commandBody("board_write", null);
|
||||
when(store.lineage(TENANT, PARENT))
|
||||
.thenReturn(new Lineage(CHILD, "run-1"));
|
||||
when(records.readResourceBytes(TENANT, "resource-args"))
|
||||
.thenReturn(CONTENT);
|
||||
harness.reply = Map.of("outcome", "answered", "text", "done");
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("handover", "execute",
|
||||
"answered");
|
||||
assertThat(harness.calls.get(0).sessionId()).isEqualTo(PARENT);
|
||||
assertThat(harness.calls.get(0).body()).containsEntry("commandId",
|
||||
MESSAGE);
|
||||
Call execute = harness.calls.get(1);
|
||||
assertThat(execute.sessionId()).isEqualTo(CHILD);
|
||||
assertThat(execute.body()).containsEntry("commandKind", "board_write")
|
||||
.containsEntry("route", "to_parent")
|
||||
.containsEntry("childRunId", "run-1")
|
||||
.containsEntry("senderSessionId", PARENT)
|
||||
.containsEntry("argsDigest", "a1")
|
||||
.containsEntry("argsBase64",
|
||||
Base64.getEncoder().encodeToString(CONTENT));
|
||||
assertThat(harness.calls.get(2).sessionId()).isEqualTo(PARENT);
|
||||
assertThat(harness.calls.get(2).body().get("reply"))
|
||||
.isEqualTo(Map.of("outcome", "answered", "text", "done"));
|
||||
assertThat(row.get().state()).isEqualTo("done");
|
||||
assertThat(row.get().attempts()).isZero();
|
||||
}
|
||||
|
||||
// The leader's own session_command_refused is terminal: rejected,
|
||||
// never re-executed.
|
||||
@Test
|
||||
void rejectsACommandItsLeaderRefusesForGood() {
|
||||
pending.set(commandPending("accepting"));
|
||||
commandBody("board_write", CHILD);
|
||||
when(records.readResourceBytes(TENANT, "resource-args"))
|
||||
.thenReturn(CONTENT);
|
||||
harness.refuseKind = "execute";
|
||||
harness.refuseCode = "session_command_refused";
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("rejected");
|
||||
assertThat(row.get().state()).isEqualTo("done");
|
||||
}
|
||||
|
||||
// A blocked or closing lead is a wait, not a failure: retried without
|
||||
// spending an attempt.
|
||||
@Test
|
||||
void defersABlockedLeaderWithoutSpendingAnAttempt() {
|
||||
pending.set(commandPending("accepting"));
|
||||
commandBody("board_read", CHILD);
|
||||
when(records.readResourceBytes(TENANT, "resource-args"))
|
||||
.thenReturn(CONTENT);
|
||||
harness.refuseKind = "execute";
|
||||
harness.refuseCode = "hosted_turn_recovery_required";
|
||||
relay.scan();
|
||||
assertThat(harness.calls).isEmpty();
|
||||
assertThat(row.get().attempts()).isZero();
|
||||
assertThat(row.get().state()).isEqualTo("relaying");
|
||||
}
|
||||
|
||||
@Test
|
||||
void defersAClosingLeaderWithoutSpendingAnAttempt() {
|
||||
pending.set(commandPending("accepting"));
|
||||
commandBody("board_read", CHILD);
|
||||
when(records.readResourceBytes(TENANT, "resource-args"))
|
||||
.thenReturn(CONTENT);
|
||||
harness.refuseKind = "execute";
|
||||
harness.refuseCode = "hosted_session_closing";
|
||||
relay.scan();
|
||||
assertThat(harness.calls).isEmpty();
|
||||
assertThat(row.get().attempts()).isZero();
|
||||
}
|
||||
|
||||
// The sender's run has not attached yet: the control plane's attach
|
||||
// commit may trail this execute — a wait, never a rejection.
|
||||
@Test
|
||||
void defersACommandWhoseSenderHasNotAttachedYet() {
|
||||
pending.set(commandPending("accepting"));
|
||||
commandBody("board_read", CHILD);
|
||||
when(records.readResourceBytes(TENANT, "resource-args"))
|
||||
.thenReturn(CONTENT);
|
||||
harness.refuseKind = "execute";
|
||||
harness.refuseCode = "session_command_blocked";
|
||||
relay.scan();
|
||||
assertThat(harness.calls).isEmpty();
|
||||
assertThat(row.get().attempts()).isZero();
|
||||
assertThat(row.get().state()).isEqualTo("relaying");
|
||||
}
|
||||
|
||||
// One sender's commands pass strictly in order: an older unclassified
|
||||
// one keeps a newer one waiting.
|
||||
@Test
|
||||
void holdsAYoungerCommandBehindAnUnclassifiedOlderOne() {
|
||||
pending.set(commandPending("planned"));
|
||||
commandBody("board_write", null);
|
||||
when(store.olderOpenCommand(TENANT, PARENT, MESSAGE))
|
||||
.thenReturn(true);
|
||||
relay.scan();
|
||||
assertThat(harness.calls).isEmpty();
|
||||
assertThat(row.get().attempts()).isZero();
|
||||
verify(store).scheduleRetry(any(MessageRow.class), anyString(),
|
||||
anyLong(), anyLong(), anyLong());
|
||||
}
|
||||
|
||||
// Decision 2: the member's own timeout may settle the outbound before
|
||||
// the relay ever classifies its row. A terminal record is classified
|
||||
// on sight — never re-executed — whatever the ledger row holds.
|
||||
@Test
|
||||
void classifiesAnAlreadyUnknownCommandWithoutExecutingItAgain() {
|
||||
pending.set(commandPending("unknown"));
|
||||
commandBody("board_write", CHILD);
|
||||
when(records.readResourceBytes(TENANT, "resource-args"))
|
||||
.thenReturn(CONTENT);
|
||||
relay.scan();
|
||||
assertThat(harness.calls).isEmpty();
|
||||
assertThat(row.get().state()).isEqualTo("unknown");
|
||||
assertThat(row.get().lastError())
|
||||
.isEqualTo("command is already unknown");
|
||||
}
|
||||
|
||||
@Test
|
||||
void classifiesAnAlreadyCancelledCommandWithoutExecutingIt() {
|
||||
pending.set(commandPending("cancelled"));
|
||||
commandBody("board_write", CHILD);
|
||||
when(records.readResourceBytes(TENANT, "resource-args"))
|
||||
.thenReturn(CONTENT);
|
||||
relay.scan();
|
||||
assertThat(harness.calls).isEmpty();
|
||||
assertThat(row.get().state()).isEqualTo("done");
|
||||
assertThat(row.get().lastError())
|
||||
.isEqualTo("command is already cancelled");
|
||||
}
|
||||
|
||||
@Test
|
||||
void cancelsACommandWhoseParentClosedBeforeTheHandover() {
|
||||
pending.set(commandPending("planned"));
|
||||
commandBody("board_write", null);
|
||||
when(store.lineage(TENANT, PARENT))
|
||||
.thenReturn(new Lineage(CHILD, "run-1"));
|
||||
when(records.sessionStatus(TENANT, CHILD)).thenReturn("CLOSED");
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("cancelled");
|
||||
assertThat(row.get().state()).isEqualTo("done");
|
||||
}
|
||||
|
||||
// A give-up answers a handed-over command from the leader's own
|
||||
// receipt — the reply, never a paraphrase.
|
||||
@Test
|
||||
void givesUpAHandedOverCommandFromItsReceipt() {
|
||||
pending.set(commandPending("accepting"));
|
||||
commandBody("board_write", CHILD);
|
||||
harness.refuseKind = "execute";
|
||||
harness.refuseCode = "session_command_failed";
|
||||
row.set(new MessageRow(TENANT, PARENT, MESSAGE, CHILD, "relaying",
|
||||
"owner", 31_000L, 63, 0, null));
|
||||
when(store.currentRecord(anyString(), anyString(), anyString(),
|
||||
anyString()))
|
||||
.thenAnswer(invocation -> PARENT
|
||||
.equals(invocation.getArgument(1))
|
||||
? new SessionMessageRelayStore.CurrentRecord(
|
||||
"accepting", "resource-message")
|
||||
: new SessionMessageRelayStore.CurrentRecord(
|
||||
"consumed", "resource-receipt"));
|
||||
when(records.readResource(TENANT, "resource-receipt")).thenReturn(
|
||||
"{\"direction\":\"inbound\",\"replyRef\":{\"resourceId\":"
|
||||
+ "\"resource-reply\"}}");
|
||||
when(records.readResource(TENANT, "resource-reply")).thenReturn(
|
||||
"{\"outcome\":\"answered\",\"text\":\"says\"}");
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("answered");
|
||||
assertThat(harness.calls.getFirst().body().get("reply"))
|
||||
.isEqualTo(Map.of("outcome", "answered", "text", "says"));
|
||||
assertThat(row.get().state()).isEqualTo("unknown");
|
||||
}
|
||||
|
||||
@Test
|
||||
void givesUpAHandedOverCommandWithoutItsReceiptAsUnknown() {
|
||||
pending.set(commandPending("accepting"));
|
||||
commandBody("board_write", CHILD);
|
||||
harness.refuseKind = "execute";
|
||||
harness.refuseCode = "session_command_failed";
|
||||
row.set(new MessageRow(TENANT, PARENT, MESSAGE, CHILD, "relaying",
|
||||
"owner", 31_000L, 63, 0, null));
|
||||
when(store.currentRecord(anyString(), anyString(), anyString(),
|
||||
anyString()))
|
||||
.thenAnswer(invocation -> PARENT
|
||||
.equals(invocation.getArgument(1))
|
||||
? new SessionMessageRelayStore.CurrentRecord(
|
||||
"accepting", "resource-message")
|
||||
: null);
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("unknown");
|
||||
assertThat(row.get().state()).isEqualTo("unknown");
|
||||
}
|
||||
|
||||
// A give-up answers a never-handed command as cancelled on its sender.
|
||||
@Test
|
||||
void givesUpAPlannedCommandAsCancelledOnItsSender() {
|
||||
pending.set(commandPending("planned"));
|
||||
commandBody("board_write", null);
|
||||
when(store.lineage(TENANT, PARENT))
|
||||
.thenReturn(new Lineage(CHILD, "run-1"));
|
||||
harness.refuseKind = "handover";
|
||||
harness.refuseCode = "session_command_failed";
|
||||
when(store.currentRecord(anyString(), anyString(), anyString(),
|
||||
anyString()))
|
||||
.thenAnswer(invocation -> new SessionMessageRelayStore
|
||||
.CurrentRecord("planned", "resource-message"));
|
||||
row.set(new MessageRow(TENANT, PARENT, MESSAGE, null, "relaying",
|
||||
"owner", 31_000L, 63, 0, null));
|
||||
relay.scan();
|
||||
assertThat(kinds()).containsExactly("cancelled");
|
||||
assertThat(row.get().state()).isEqualTo("unknown");
|
||||
}
|
||||
|
||||
// A give-up ends the entry on its sender first, so nothing reads it as
|
||||
// owing a handover any more: never handed over → cancelled.
|
||||
@Test
|
||||
|
|
@ -629,7 +869,7 @@ class SessionMessageRelayTest {
|
|||
|
||||
@Test
|
||||
void givesUpAHandedOverMessageAsUnknownOnItsSender() {
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "accepting",
|
||||
pending.set(new PendingMessage(TENANT, PARENT, MESSAGE, "session_message", "accepting",
|
||||
"resource-message"));
|
||||
body("to_child", PARENT, CHILD);
|
||||
harness.refuseKind = "receive";
|
||||
|
|
@ -648,7 +888,7 @@ class SessionMessageRelayTest {
|
|||
@Test
|
||||
void givesUpFromTheEntryAsTheFailedStepLeftIt() {
|
||||
body("to_child", PARENT, CHILD);
|
||||
when(store.currentRecord(anyString(), anyString(), anyString()))
|
||||
when(store.currentRecord(anyString(), anyString(), anyString(), anyString()))
|
||||
.thenAnswer(ignored -> new SessionMessageRelayStore
|
||||
.CurrentRecord(kinds().contains("handover")
|
||||
? "accepting" : "planned",
|
||||
|
|
|
|||
|
|
@ -42,8 +42,11 @@ public final class HarnessRuntimeRecovery {
|
|||
public boolean isContinuationReady() {
|
||||
// await_agent: every wait run is observable (the relay ledger), so
|
||||
// outcomes are always known; the continue route re-enters the wait
|
||||
// and folds what the checkpoint still owes (#13708).
|
||||
if ("await_agent".equals(phase)) {
|
||||
// and folds what the checkpoint still owes (#13708). H4e-b3's
|
||||
// await_command waits are the same shape: the member's own
|
||||
// session_command journal is the ledger, and every run's reply is
|
||||
// durable evidence once it lands.
|
||||
if ("await_agent".equals(phase) || "await_command".equals(phase)) {
|
||||
return !executions.isEmpty() && executions.stream().allMatch(
|
||||
execution -> "known".equals(execution.getOutcome()));
|
||||
}
|
||||
|
|
@ -57,6 +60,7 @@ public final class HarnessRuntimeRecovery {
|
|||
public boolean isCancellationReady() {
|
||||
return ("await_runtime".equals(phase)
|
||||
|| "await_agent".equals(phase)
|
||||
|| "await_command".equals(phase)
|
||||
|| "results_ready".equals(phase)) && !executions.isEmpty()
|
||||
&& executions.stream().allMatch(execution ->
|
||||
"known".equals(execution.getOutcome()));
|
||||
|
|
|
|||
|
|
@ -57,7 +57,8 @@ public final class HostedHarnessClient implements AutoCloseable {
|
|||
private static final Set<String> RUNTIME_RECOVERY_OUTCOMES =
|
||||
Set.of("known", "unknown");
|
||||
private static final Set<String> RUNTIME_RECOVERY_PHASES =
|
||||
Set.of("await_runtime", "await_agent", "results_ready");
|
||||
Set.of("await_runtime", "await_agent", "await_command",
|
||||
"results_ready");
|
||||
private static final Set<String> RUNTIME_EXECUTION_STATES =
|
||||
Set.of("prepared", "executing", "cancel_requested", "settled");
|
||||
private static final AtomicLong CLIENT_SEQUENCE = new AtomicLong();
|
||||
|
|
@ -397,11 +398,13 @@ public final class HostedHarnessClient implements AutoCloseable {
|
|||
/**
|
||||
* H4d-b: one session message operation onto the Session's journal (the
|
||||
* message relay's verbs). The Hosted side commits through its funnel
|
||||
* and answers 202 once settled; a non-2xx answer surfaces as a
|
||||
* and answers 202 once settled, carrying the settled body — H4e-b3's
|
||||
* command verbs answer it with the lead's reply document, which the
|
||||
* relay copies back to the member. A non-2xx answer surfaces as a
|
||||
* {@link DaemonHttpException}, and anything ambiguous is an unknown
|
||||
* outcome for the caller to retry, never to guess at.
|
||||
*/
|
||||
public void runMessageOperation(HarnessSessionRef session,
|
||||
public Map<String, Object> runMessageOperation(HarnessSessionRef session,
|
||||
Map<String, Object> body) {
|
||||
HarnessSessionRef ref = requireSessionRef(session);
|
||||
String operation = "POST /session/:id/messages/operations";
|
||||
|
|
@ -425,6 +428,7 @@ public final class HostedHarnessClient implements AutoCloseable {
|
|||
throw new DaemonProtocolException(
|
||||
"Hosted Harness settled a different message operation");
|
||||
}
|
||||
return json;
|
||||
} catch (DaemonProtocolException e) {
|
||||
throw new MutationOutcomeUnknownException(operation, e);
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue