* feat(memory): opt-in extraction cooldown and selector skip experiments
Moved out of #13020, where they had been bundled with unrelated
deferred-tool work. Content is the #13020 head (ffd9df8928) for these
files and hunks, applied onto current main; history and review threads
for these changes live on #13020.
- #13004: after a no-op auto-memory extraction, an opt-in cooldown
(QWEN_CODE_MEMORY_EXTRACT_NOOP_COOLDOWN_TURNS, off by default) skips
extraction for a bounded number of turns while the prompt stays below
the compaction warn threshold. Skipped turns stay inside the
extractor's window (CACHE_SAFE_HISTORY_TAIL_ENTRIES), and a run that
makes no progress advances the window instead of freezing it.
- #13003: an opt-in flag (recall-experiment.ts, off by default) skips
the recall selector when there is a single strong hit; the result is
delivered as the fast phase, and a low-cardinality selector_skipped
attribute is added to the recall log and metrics.
Refs #13004
Refs #13003
* fix(core): Preserve the zero-tool extraction cursor guard
* fix(memory): Avoid extraction from runtime reminders
* fix(memory): Preserve user text across reminder parts
* fix(memory): Bind extraction cursor to history
* test(memory): Require exact same-length replacement
* fix(memory): Wait for queued tasks during drain
* fix(memory): Consume ineligible history tails
* docs(memory): Record tail context acceptance limit
* fix(memory): Recover reported writes after extraction failure
* docs(memory): Clarify best-effort refresh recovery
* fix(memory): Suspend repeated failed extraction windows
* fix(core): close the cooldown-arm and review-round findings on the memory PR
- The no-op cooldown now arms on a completed no-op whose cursor ADVANCED
(extract.ts reports `cursorAdvanced`), not on reaching the live end —
the windowed arm stops short of history.length by construction, so a
tool-heavy session never armed it and the experiment saved nothing
exactly where #13004 measured the waste (R1-1).
- A media-only user turn counts as user content for the extraction gate;
an empty-text read advanced the cursor past a turn no extractor ever
sees, and the cursor only moves forward (R1-11).
- The queued trailing extraction's rejection is observed with a warn log
instead of vanishing into track()'s in-flight removal (R2-9).
- `selector_skipped` is stamped only on structured-mode recalls, where a
skip decision exists; legacy recalls left the field at a constant false
and mixed into the experiment's control series. The metric dimension
attaches only when the knob is on AND the field is a boolean (R2-14).
- The docs now say so: the structured-recall contract (both languages)
marks selector steps conditional under the knob, and the telemetry
inventory lists selector_skipped with its gating conditions (R2-11).
- Pins: cooldown arm on a windowed no-op + held-cursor non-arm + arm-point
accumulation across consecutive skips (manager.test.ts); media-only user
turn + cursorAdvanced assertions (extract.test.ts); legacy-unset event +
dimension drop (recall.test.ts, loggers.test.ts).
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
* fix(memory): Count pending backlog from the extraction cursor
* fix(memory): Flush pending extraction before compaction
* fix(memory): Narrow selector experiment and withdraw unsafe cooldown
* fix(memory): widen the #13003 skip guard, narrow selector_skipped
Two review findings on the opt-in skip-selector guard, each reproduced at
b4e0bba5b8 before fixing.
The uniqueness test counted only the published fast list -- the top
MAX_RELEVANT_DOCS of the lexical ranking -- while the selector it suppresses
is handed modelCandidates, whose lexical half is the top
MAX_MODEL_CANDIDATE_DOCS - RECENT_MODEL_CANDIDATE_RESERVE, roughly 36x wider.
A second strong title/keyword match ranked 6th or lower was therefore
invisible to the guard: an ambiguous recall skipped the selector and still
reported selector_skipped: true, landing in the treatment arm as a "unique"
strong hit. selectModelCandidateDocuments now also returns the strong-match
count over the lexical array it already scored. The count is computed only
while the guard is live, so knob-off recalls pay nothing on a path that
carries a latency budget, and the flag is now read once per recall so the
count and the guard cannot disagree.
logRecallResult also stamped selector_skipped: false on the empty-query /
empty-corpus / non-positive-limit short circuit, where the selector is never
called. Those trivially fast recalls joined the ablation's control arm, which
the treatment arm structurally cannot contain, and recordMemoryRecallMetrics
puts only strategy and selector_skipped on the counter and histogram, so the
samples could not be sliced back out. logRecallResult now takes the caller's
decision state and omits the field when no skip decision was made; the
heuristic fallback keys off selectorStartedAt so a selector failure still
reports false as a genuine control sample.
Both fixes are mutation-checked: reverting either one reds exactly the tests
added for it (1 failure and 2 failures respectively).
Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmurnh6s41i
* style(memory): let Prettier reflow the suppressed-selector assertion
Lint & Static failed at step "Run Prettier" on 34894db1d6 with
"[warn] packages/core/src/memory/recall.test.ts / Code style issues found
in 1 file". Prettier 3.6.1 reproduces it locally and --write produces this
two-line reflow of the `mock.calls[0]?.[2] ?? []` member chain; the
expression is unchanged, so this is formatting only.
Verified: prettier --check now passes on this file and on all 13 files the
branch changes. Test execution is left to CI (no node_modules in this
worktree).
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-conflict/jmurrend91n
* fix(memory): require a token boundary for a short title in the #13003 skip gate
R3-1: the skip gate made matchesTitleOrKeyword the final authority on whether
the model selector runs at all, but its title arm is an unbounded substring
test while the keyword arm already guards short Latin strings with a
token-boundary regex. A memory titled `ai` therefore matched the query
`explain provider fallback.`, and as the unique strong hit it suppressed the
selector: every other memory the selector would have added was lost, and the
recall entered the ablation's treatment arm as a genuine "unique strong hit"
when the match was a coincidence.
The injected document itself is not what this fixes. isStrongFastMatch keeps
the loose arm, so a coincidental short-title substring was and still is
published as a fast candidate; the refined selector result is filtered against
fastDeliveredRefs (core/client.ts:1656), so it can only add to the fast phase,
never retract it. What the gate changed is that the match also cancelled the
selector, and that is what is closed here.
Reproduced at 279c5440eb: the new test fails there with the selector never
called, and passes after the fix (88/88).
The guard is applied only at the gate, so lexical ranking and the default
(knob-off) path are unchanged and the design doc's "default behavior is
unchanged" claim still holds. The gate's title test reuses the keyword arm's
existing boundary rule instead of adding a second matcher; both design docs
now state it. lexicalStrongMatchCount deliberately keeps the loose arm and now
says so, because a looser count under-fires the skip, which is the safe
direction for an ablation.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmurtwo5e1r
* fix(memory): make the #13003 selector-skip gate require a real unique hit
The strict title arm delegated to `includesKeyword`, whose token-boundary
rule is gated on `/^[a-z0-9]{1,2}$/`, so for any Latin title of three or
more characters the strict and loose arms were byte-identical: a memory
titled `log` cancelled the model selector for a query containing
`catalog`. Apply the boundary rule to any single-word Latin title when
`requireTitleBoundary` is set; multi-word and non-Latin titles keep the
loose arm, and ranking callers are untouched.
The uniqueness count also ran over `lexical`, while the selector the gate
suppresses is handed `modelCandidates` — whose `recent` reserve is built
from `eligible`. A doc whose only strong-match evidence is a keyword that
is a proper substring of a query token scores 0, never enters `lexical`,
and so was never counted, yet it sat in the suppressed selector's pool.
Count the pool the selector actually receives, and rename the field to say
what it counts (title/keyword matches, deliberately not `isStrongFastMatch`
— that predicate's loose arm would empty the experiment's treatment arm).
Both changes only ever raise the count, so the gate skips strictly less
often.
State the real gate condition in both design docs, and pin
`selectorDecided` in both directions: each constant mutant of the
heuristic-fallback argument now fails a named assertion.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmus4mgzr28
* fix(memory): require a token boundary for every non-CJK value in the #13003 skip gate
The strict arm keyed its boundary rule on value shape: `^[a-z0-9]+$` for a
title and `^[a-z0-9]{1,2}$` for a keyword. That left the coincidental inner
substring the gate exists to reject reachable through every other shape: a
keyword of three or more characters (`log` inside `catalog`, the collision the
gate's own rationale names), a multi-word Latin title (`log conventions` inside
`catalog conventions`), and a single-word non-ASCII title (`ток` inside
`поток`). Each suppressed the selector, dropped the refined phase over up to
200 candidates, and put a sample that is not a unique strong hit into the
ablation's treatment arm as `selector_skipped: true`.
Express the strictness once, on the script split this module's own tokenizer
already draws: a non-CJK value goes through `includesAtBoundary`, a CJK value
keeps `includes`, because a boundary rule needs a `[^\p{L}\p{N}]` neighbour
while Han, Kana, and Hangul are `\p{L}`. Both arms share it and it stays
behind the gate flag, so ranking callers and knob-off recall are unchanged.
Pinned by three tests that are red without the fix. The design-doc sentences
scoped the rule to single-word Latin titles; both twins now describe both arms.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Patrol-Run: qwen-pr-closeout/jmusfc9vi2o
* fix(core): preserve memory selector freshness and delivery phase
* fix(memory): reject concurrent document changes during reread
* fix(memory): tighten selector skip boundaries and log attributes
* fix(memory): clarify unstable reads and cover selector edges
---------
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: yiliang114 <yiliang114@users.noreply.github.com>
Co-authored-by: Qwen-Coder <qwen-coder @alibabacloud.com>
5374641323