qwen-code/scripts/release-script-utils.js
Shaojin Wen d5c22d336b
ci(sdk-java): guard Flyway version uniqueness and watch red main runs (#12965)
* ci(sdk-java): guard Flyway version uniqueness and watch red main runs

Two in-flight PRs each added a V16 managed-agent-server migration and
merged 23 minutes apart; each branch was green in isolation, and Flyway
refused to boot the merged result, leaving main red for two hours while
three more merges stacked on it (#12940).

A new flyway-migrations job runs scripts/check-flyway-migrations.js on
every SDK Java pull request and push: a filename scan of the two
migration locations that share one version sequence, failing in seconds
and naming both files when two claim one version. Versions compare the
way Flyway compares them — numerically per segment — so V016 collides
with V16, and a location rename that would empty the guard fails instead
of passing vacuously.

The main-CI failure watcher now watches the SDK Java workflow, whose
path-filtered push run on main is the only signal for a merge result
neither PR could fail, and the failure-signature helper reads Surefire's
per-test error lines, so a red Java lane files one deduped issue naming
the failing tests instead of a per-commit stub.

Refs #12940

* ci(sdk-java): harden the Flyway guard and dedupe its main-red issues

Review on #12965:

- compare versions across every module of one invocation — Flyway resolves
  classpath:db/migration across jars, so a version claimed in two modules
  collides exactly like two files in one module
- fail when the SQL location yields no migration even if the Java location
  still does, so renaming one location away can no longer pass vacuously
- escape %/CR/LF in module names and file paths before emitting ::error::
  (the runner parses workflow commands from stderr too, and the job scans
  contributor-controlled filenames on pull_request)
- match the migration suffix case-insensitively, as Flyway does
- keep the fully qualified Surefire id as the dedupe identity so same-named
  tests in different packages no longer merge into one issue; the cosmetic
  package strip moves to shortenForTitle
- give the guard's ::error:: line a stable flyway:<module>:<version>
  identity, so a standing red dedupes to one issue instead of one per
  stacked merge, and render the failed jobs in the test-keyed body
- pin the workflow_run watch entry to sdk-java.yml's name key

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* ci(sdk-java): fix the Flyway dedupe on runner-rendered logs and close review findings

The Actions runner consumes ::error:: workflow commands and writes
##[error] into the downloadable job log, so the dedupe pattern anchored
on ^::error:: never matched production input and every merge stacked on
a standing red would have opened a fresh autofix issue. The anchor now
accepts both forms, the guard's two non-collision failure modes (moved
migration location, missing module directory) get their own stable
module+mode identity, and the synthesized flyway id is written to be
read since it doubles as the issue title and body bullet.

The issue body's "- Failed jobs:" block is now machine-owned like
"## Also failing": stripped and re-emitted from the current run on every
merge instead of freezing the lanes recorded at filing time, with the
empty-list case pinned against a dangling heading.

The guard itself now reports a multi-module collision once after all
modules are scanned, naming every claimant (a second line would file a
second issue), never prints a clean summary for a claimant, and only
requires the SQL location of a module that owns a db/migration*
directory — which lets the workflow invocation cover runtime-broker, the
module that shares the classpath:db/migration namespace without carrying
migrations today.

escapeWorkflowCommand moves to scripts/release-script-utils.js (the
byte-identical copy would have drifted from the tested original), and
the filename-forgery tests gain a runner-decoding oracle and a
capability probe so the Windows lane skips instead of failing on
filenames Win32 cannot hold.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>

* ci(sdk-java): close the guard's blind spots and validate its dedupe captures

Review and sandboxed verification on #12965 found the moved-location
probe keying on the two exact directory names: a rename outside the
migration* family (db/migration -> db/changelog) passed silently with
the whole sequence inside it, a renamed Java location printed "all
versions unique" from a scan that skipped it, and a Java-only module
red-lit as "moved" with its real collision never named. The probe is
now derived from the LOCATIONS table: a versioned-migration file
anywhere under a source root but outside the configured location fails
with the files named, the sibling check is per location and fires only
when the location directory itself is gone, and a repeated module
argument no longer fabricates a collision of a file with itself.

The invocation now names qwencode too — managed-agent-server depends on
it at compile scope, the same namespace premise that admitted
runtime-broker — and the .size-baseline entry is re-recorded at the
file's current size.

On the consumer side, both Flyway patterns feed their captures through a
module-shape validation before they become a dedupe identity, so a
filename-borne forgery rendered by the runner's annotation decode can no
longer inject a second identity into the issue this files, and the
guard's run-level diagnosis is ordered ahead of first-seen test ids so a
mass failure in another lane cannot crowd it past the search and body
caps. A closure pin keeps the spawned guard importable in the
dependency-free helper lane.

Refs #12940

* ci(sdk-java): close the Windows-drive forgery and split the stray-file message

The drive-prefix alternative in the module validator re-opened the
forgery it closed: backslash and colon are ordinary characters in a
POSIX filename, so a migration name could supply a `C:\evil` prefix and
pass the path-shape check. No lane that runs this consumer ever
produces a backslashed module — the producer and analyze jobs are
ubuntu-latest and the suite runs only on the Linux lint_and_static
lane — so the branch is dropped, backslashes are rejected outright, and
the fixtures hand the guard a slash-bearing argument (a no-op on POSIX)
instead of teaching the consumer a shape only a forgery can produce.
The unsafe-character half of the validator is now pinned for
whitespace, backticks and backslashes across both call sites.

A populated location with a migration-shaped stray elsewhere under the
source root no longer prints "found no migration under": the guard
names the stray as the cause, and the consumer's config identity learns
the new phrase, so the standing red still files one issue. The dep-free
closure pin now walks the guard's relative-import closure instead of a
hardcoded depth-one list, so a new hop carrying an npm import cannot
slip through.

Refs #12940

* ci(sdk-java): re-record the sdk-java.yml size baseline after merging main

The rebase onto 51b80dadbc folded #12945's Hosted latency steps into the
workflow; the file is 22735 bytes now, and the ratchet requires the
number in the same PR.

* ci(sdk-java): anchor the flyway dedupe identity to runner-emitted lines

Sandboxed verification found the module-shape validator closing the
filename door but not the directory one: the guard prints file paths,
so a LF inside a directory component leaves the forged text after a
real `/` — slash-bearing and past the shape. The runner timestamps
every line it writes, and a decoded annotation continuation carries
none, so both flyway arms now accept their line only when it came from
the runner itself. The shape validation stays as the second layer for
every other channel, and its inline fixtures are timestamped so they
still reach it. The Surefire arm's matching exposure is pre-existing —
the vitest and pytest patterns share it — and is left to a follow-up.

Refs #12940

* ci(sdk-java): close the forgery family at the guard's emission sites

The third Critical in the family, and the round that ends it. The
guard interpolates contributor-controlled path bytes into ::error::
command data, and the runner percent-decodes that data where the
consumer parses — so percent-escaping control bytes was never enough:
a directory name carries the LF, the slash and every pattern byte the
decoded continuation needs. No charset rule over the consumer's module
shape can close that — the forged module is shape-indistinguishable —
and double-escaping regresses the genuine LF arm into the suppression
arm. What closes every measured arm is bounding the emitted alphabet
at the four emission sites: nothing outside a migration path's real
characters reaches the command data, so no decode pass can restore
pattern text from it.

The suppression arm needs no newline at all and no runner behaviour:
spaces on the timestamped line run the duplicate-version arm's lazy
module capture all the way to the payload, the shape rejects it, and
an unconditional continue used to discard the line's genuine config
identity with it — the duplicate storm this change exists to end, from
the opposite direction. The flyway arm now falls through to the config
arm when its capture is rejected.

The stray-file test's path assertion is separator-agnostic, per the
round-6 review's note that the suite must pass on the Windows lane.

Refs #12940

* ci(sdk-java): re-record the sdk-java.yml size baseline after merging main

The merge folded #13083's workflow growth into the file (23,524 bytes
now); the ratchet asks for the number in the same PR.

---------

Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com>
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
2026-10-02 06:49:12 +00:00

139 lines
3.9 KiB
JavaScript

/**
* @license
* Copyright 2026 Qwen Team
* SPDX-License-Identifier: Apache-2.0
*/
import crypto from 'node:crypto';
import fs from 'node:fs';
import path from 'node:path';
import { pipeline } from 'node:stream/promises';
import { fileURLToPath } from 'node:url';
export function fail(message) {
throw new Error(`ERROR: ${message}`);
}
export async function sha256File(filePath) {
const hash = crypto.createHash('sha256');
await pipeline(fs.createReadStream(filePath), hash);
return hash.digest('hex');
}
/**
* Parse a SHA256SUMS file. Handles:
* - optional leading UTF-8 BOM (uploaded via Windows tools)
* - binary-prefix markers (`*` before filename)
* - empty lines and CRLF / LF line endings
*/
export function parseSha256Sums(content) {
// Strip a leading UTF-8 BOM so a SHA256SUMS file uploaded via a Windows tool
// that prepends one still reports a useful "Missing checksum entry" error
// instead of "Malformed SHA256SUMS line 1".
const normalized = content.replace(/^\uFEFF/, '');
const checksums = new Map();
for (const [index, line] of normalized.split(/\r?\n/).entries()) {
const trimmed = line.trim();
if (!trimmed) {
continue;
}
const match = /^([0-9a-fA-F]{64})\s+\*?(.+)$/.exec(trimmed);
if (!match) {
fail(`Malformed SHA256SUMS line ${index + 1}: ${trimmed}`);
}
if (checksums.has(match[2])) {
fail(`Duplicate SHA256SUMS entry for: ${match[2]}`);
}
checksums.set(match[2], match[1].toLowerCase());
}
return checksums;
}
/**
* Escape a value embedded in a `::error::`/`::warning::` workflow command.
* The value can be contributor-controlled (a fork PR's filename, model
* output), and the runner parses workflow commands from stderr too: a raw LF
* would emit a second, forged command from the step. `%` must be replaced
* first, or the `%0A`/`%0D` sequences the other two arms emit get
* double-encoded.
*/
export function escapeWorkflowCommand(text) {
return String(text)
.replace(/%/g, '%25')
.replace(/\r/g, '%0D')
.replace(/\n/g, '%0A');
}
export function readOptionValue(argv, index, optionName) {
const value = argv[index + 1];
if (!value || value.startsWith('-')) {
fail(`${optionName} requires a value`);
}
return value;
}
export function isMainModule(importMetaUrl) {
const filename = fileURLToPath(importMetaUrl);
return process.argv[1] && path.resolve(process.argv[1]) === filename;
}
/**
* Parse CLI arguments. Supports:
* - --flag → args[def.key] = true
* - --key value → args[def.key] = value
* - --key=value → args[def.key] = value
* - -h, --help → args.help = true (always recognised)
*
* @param {string[]} argv
* @param {Record<string, {key: string, type: 'flag'|'value'}>} definitions
* @returns {{help: false} & Record<string, any>}
*/
export function parseArgs(argv, definitions) {
const args = { help: false };
for (let index = 0; index < argv.length; index += 1) {
const raw = argv[index];
if (raw === '--help' || raw === '-h') {
args.help = true;
continue;
}
// --key=value form
const eqIndex = raw.indexOf('=');
if (eqIndex >= 0) {
const key = raw.slice(0, eqIndex);
const value = raw.slice(eqIndex + 1);
if (key === '--help' || key === '-h') {
fail(`${key} does not accept a value`);
}
const def = definitions[key];
if (!def) {
fail(`Unknown option: ${key}`);
}
if (def.type === 'flag') {
fail(`${key} does not accept a value`);
}
if (!value || value.startsWith('-')) {
fail(`${key} requires a value`);
}
args[def.key] = value;
continue;
}
const def = definitions[raw];
if (!def) {
fail(`Unknown option: ${raw}`);
}
if (def.type === 'flag') {
args[def.key] = true;
continue;
}
args[def.key] = readOptionValue(argv, index, raw);
index += 1;
}
return args;
}