Windows on ARM: install the native ARM64 CUDA stack on NVIDIA hosts (#10282)

* Windows on ARM: install the native ARM64 CUDA stack on NVIDIA hosts

NVIDIA's Windows-on-ARM parts (GB10 / N1X, the RTX Spark laptops) run CUDA
natively, but the installer treated every ARM64 Windows host as "no GPU wheels
here" and fell back to x64 Python under emulation. That fallback looks like it
works: torch loads and sees the GPU. It then fails on the first kernel, because
triton-windows' bundled ptxas is built for the x64 CUDA 12.8 toolkit and cannot
target the sm_121 these parts report, so every Unsloth kernel fails to compile.
llama.cpp separately picked a CPU-only bundle regardless of the GPU.

Detection is evidence based rather than arch based: the installer asks the wheel
indexes what actually exists for this interpreter, and only takes the native path
when both a win_arm64 CUDA torch and an obtainable pyarrow are found. Anything
else keeps the historical x64 path, so a host that cannot be served natively is
never left with a half configured venv. The torch index is probed in order:
download.pytorch.org first, so this special case ends by itself the day PyTorch
publishes these wheels, then NVIDIA's GA channel, then its nightly.

Requirement pins with no win_arm64 wheel are split on
platform_machine == "ARM64" rather than loosened. Every other platform
(linux x86_64 and aarch64, win x64, macOS arm64 and x86_64) evaluates to the
pins it had before, checked by evaluating the markers per platform as well as by
the test suite.

Also:

* llama.cpp prebuilt selection learns a windows-arm64-cuda install kind and
  falls through to upstream's win-cuda-13.4-arm64 asset, so these hosts get a
  CUDA bundle instead of the CPU one. When neither is hash gated the fallback is
  still taken, but says so in the log rather than silently.
* setup.ps1 keeps an installed cu134 torch on win_arm64 instead of "repairing"
  it to a cu130 that has no wheel for this platform, and asks the chosen index
  whether it publishes torchaudio instead of assuming it does not.
* install.ps1 checks before the studio handoff that the installed package
  understands this host, and stops with an explanation if it does not. The two
  halves ship through different channels (the script is fetched from unsloth.ai,
  the package comes from PyPI), and an older setup.ps1 handed a working +cu134
  torch will reinstall a cu130 that does not exist here, leaving the environment
  with no torch at all. UNSLOTH_WOA_NATIVE=0 opts back out to the x64 path.

Verified on an RTX Spark N1X laptop: clean install into an empty
UNSLOTH_STUDIO_HOME ends EXIT=0 with torch 2.14.0+cu134, torchvision 0.29.0,
torchaudio 2.11.0, triton-windows 3.8.0.post28 and the
llama-b10715-bin-win-cuda-13.4-arm64 bundle; LoRA and QLoRA both train; Studio
serves and reports CUDA; Qwen3.8-27B-GGUF UD-Q4_K_XL answers at 8.1 tok/s with
the GPU at 94-95 percent through generation.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: key the new filters on the interpreter, not the machine

Review of the branch turned up one real regression and several sharp edges. All of
them are on paths this PR added; none change what a host did before it.

The regression. `_is_windows_arm64()` answers for the MACHINE, and says yes even
under an emulated x64 Python -- its docstring says so, since that is what it is for.
But the requirement markers this PR adds key on `platform_machine == "ARM64"`, which
is the INTERPRETER's arch, and every Windows-on-ARM install predating native support
is running an emulated x64 interpreter, because install.ps1 deliberately fetched one
there. So on exactly those existing installs the two mechanisms disagreed: the marker
rows correctly selected the x64 pins while the new skip list dropped ten packages
(xformers, tiktoken, tensorboard, librosa, openai-whisper, hf_transfer, sqlite-vec,
mecab and the rest) that have working win_amd64 wheels and install fine today. Anyone
taking the new UNSLOTH_WOA_NATIVE=0 escape hatch would have hit it too.

`_is_win_arm64_interpreter()` answers the other question, from
`sysconfig.get_platform()` -- the same authority pip and uv tag wheels with -- and
the two call sites this PR added now use it. The pre-existing uses of the machine
predicate are left alone, so nothing outside this PR shifts.

The rest:

* setup.ps1 answers `Test-WinArm64Venv` from the machine arch before launching any
  interpreter. The callers sit in the stale-venv path, which on x64 previously
  executed nothing at all; probing a venv's python there to learn something already
  known gave a corrupt or lock-blocked interpreter a chance to hang the install.
* `$expectedKinds` gains windows-arm64-cuda only on the arch that can carry it.
  Accepting it on x64 would keep an install that machine cannot run, where today it
  is removed as mismatched.
* The wheelhouse reuses a staged wheel only if it opens as a zip. An interrupted run
  leaves a truncated file, and skipping on mere existence handed the resolver that
  same corpse on every retry. Both downloads also take a timeout now.
* UV_OVERRIDE / UV_FIND_LINKS / PIP_FIND_LINKS are cleared when they point into this
  StudioHome's own woa directory, so a second `irm | iex` in one shell cannot inherit
  a previous run's wheelhouse. Matched by path rather than cleared outright: an
  inherited UV_FIND_LINKS that a user set for their own wheelhouse is left alone,
  since clearing it would break air-gapped installs on every Windows host.
* hf_transfer in no-torch-runtime.txt and cryptography in studio.txt (and its mirror
  in pyproject) get the same marker split as the other rows. That file installs
  --no-deps and the extra is installable directly, so neither was covered by the
  constraints file.
* `windows_cuda_arch_for_host` tests is_windows as well as is_arm64. Every caller is
  already inside a Windows branch, but `is_arm64` is also true on macOS arm64 and
  Linux aarch64, so the answer should not depend on the caller remembering that.
* The published legacy-minor scan uses the arch being resolved instead of a
  hardcoded x64, which was harmless only while no arm64 CUDA artifact existed.
* No `windows_cuda_selection` log line on x64, where the arch is the unspoken
  default and the line would be new output in every existing install log.

Verified: requirement markers resolve identically to origin/main across 65
environments (linux x86_64/aarch64/arm64/armv7l/ppc64le/s390x, win32
AMD64/x86/ARM64EC, darwin arm64/x86_64, freebsd, python 3.10 to 3.14), differing
only on win32/ARM64; the studio-extra mirroring guard passes; both PowerShell files
parse; and a simulated interpreter matrix confirms the win_arm64 skip list applies
on a native ARM64 interpreter and on nothing else, including the emulated x64 case
that regressed.

* Keep the cross-platform parity greps matching

Two parity tests read install.ps1 and setup.ps1 as text, and both changes in this
PR moved the strings they look for. The behaviour they guard is intact in each
case, so the fix is on this side rather than in the assertions.

test_install_ps1_bounds_unknown_leaf_pinned_torch pins the literal
"@_torchSpecs --default-index $TorchIndexUrl", which is how it checks that the
bounded companion specs actually reach uv. The Windows on ARM branch inserted
@_torchExtraArgs between the two halves. Those arguments are all flags and their
values, never positional packages, so uv accepts them anywhere on the line;
moving them after the index restores the literal and changes nothing about the
resolve.

test_setup_ps1_scrub_covers_pip_fallback slices a fixed 2500 character window
from "function Fast-Install" and requires the pip fallback to appear before the
finally that restores the scrub. The flag stripping loop added here is sixteen
lines and pushed the finally out of that window: the fallback is still inside the
try, but the test could no longer see it. Lifting the loop into
Remove-UvOnlyResolverFlags puts Fast-Install back near its original size, with
the finally at offset 1695 against a 1629 baseline on main.

The helper was exercised against the inlined version it replaces on all six
shapes: no flags, --index-strategy in both spellings, --prerelease in both
spellings, and an empty list. Output is identical, including the empty-in
empty-out case that every caller in the tree takes today.

* Exempt the Windows on ARM drop list from the xFormers pin check

test_installer_never_installs_an_unpinned_xformers flags every quoted xformers
literal in install.ps1 that is not the exact pin. The Windows on ARM block added
one, but it is a removal list: those names are written into the requirement
overrides so the package vanishes from the resolve, since win_arm64 has no
xFormers wheel and no buildable sdist. A version pin there would be meaningless,
and the name has to match the requirement line exactly for the override to bind.

The test already carves out wheel filenames on the same grounds, that a filename
is not a spec. This adds the drop list beside it rather than loosening the
assertion. Checked that it stays narrow: a bare "xformers" appended at end of
file is still caught, and so is a loose "xformers>=0" placed on the line
immediately above the drop list, while the real file is clean.

* Windows on ARM: fix six ways the native stack picks the wrong thing

Six defects found in review, all on the new WoA path and none reachable on any
other host.

Probe the interpreter that was actually selected. Initialize-WoaNativeCudaTorch
answered for $PythonVersion, but Find-CompatiblePython falls back to another
supported minor whenever the requested one is not installed. Every native
decision is keyed to an interpreter tag, so a 3.13 answer carried into a 3.12
venv staged a cp313 pyarrow wheel and then pinned pyarrow== to it, forcing the
Arrow sdist this path exists to avoid. Re-probe once the interpreter is settled,
key the staged tags on the venv's own minor, and clear every script-scope answer
on entry so a second probe cannot inherit the first one's.

Redact the selected torch index. The status line printed
$script:WoaTorchIndexUrl verbatim, and it can be UNSLOTH_TORCH_INDEX_URL with
userinfo or a token query. It was the only index line in the file that did not
redact. Remove-IndexUrlCredentials moves up to the other helpers, since
PowerShell resolves a function only once its definition has executed.

Hand uv a space-free override path. uv reads UV_OVERRIDE as a space-separated
list of files, which this script splits on \s+ itself, and the default
StudioHome sits under %USERPROFILE%. An overrides file under a path with a space
was read as two paths, neither of which exists, and every later uv call failed
with "File not found". Same defect and same remedy as
backend/utils/uv_path_safety.py (issue #6503), which the Python half already
applies. pip splits its repeatable options the same way; UV_FIND_LINKS is
comma-separated and is untouched.

Do not enable the native stack for an x64 venv. The migration branches keep a
healthy legacy environment as it is, and on this host those are x64 by design.
Get-TorchIndexUrl returns the win_arm64 index on the flag alone while the spec
lift is gated on the venv's platform tag, so a migrated venv was asked for the
bounded x64 specs from an ARM64-only index and aborted with no fallback. Ask the
venv the same question the spec branch asks.

Accept every spelling of a local wheelhouse. UNSLOTH_WOA_WHEELHOUSE is
documented as taking a directory or a base URL, but only drive-plus-backslash
and UNC counted as local, so C:/wheels went to Invoke-RestMethod and the native
path was dropped on a host whose wheels were present. Ask the filesystem instead.

Count a wheelhouse wheel only for the interpreter it was built for. Staging
copies every win_arm64 wheel, cp311 through cp314, but both availability checks
keyed on the distribution name alone. A cp311 tiktoken in a cp313 venv cleared
its skip and its requirement override, and the resolve fell to an unbuildable
sdist. Filter on PEP 425 tags on both sides. WINDOWS_ARM64_SKIP_UNBLOCKED_BY
takes the whole blocker set rather than one name, since librosa needs numba as
well as llvmlite and whisper needs tiktoken too; its openai-whisper key was also
never matched, being looked up canonically.

Let an explicit CPU pin through the CUDA preservation guard. It ran ahead of
every pin check on both sides, so UNSLOTH_TORCH_INDEX_URL=.../cpu was dropped
even though download.pytorch.org publishes win_arm64 /cpu torch and torchvision.
In setup.ps1 it also left PinChangedForceReinstall false, which skips the
dependency pass entirely, so the run kept the CUDA build and reported success.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pin that an emulated x64 interpreter still gets the torch flavor repair

The CUDA preservation shortcut reads the interpreter, not the machine, and nothing
in the suite held it to that. Swapping _is_win_arm64_interpreter for
_is_windows_arm64 at the guard leaves 223 of 224 tests here passing, which is the
whole margin between a correct guard and one that strands every Windows on ARM
install predating native support. Those run an emulated x64 python against
ordinary win_amd64 wheels from download.pytorch.org, so the repair has somewhere
to go and must still reach them.

The fixture drove both predicates from one flag, which is right for every case
already here, since a native ARM64 venv is the only place both are true. They
separate on an ARM64 machine running an x64 interpreter, so win_arm64_interpreter
overrides just that half and defaults to the existing flag.

Verified by mutation: with the guard reading the machine instead, the new test is
the only one of the 224 that fails.

* Windows on ARM: pick the retained llama.cpp kind from the venv arch

$expectedKinds asked Get-HostMachineArch, which answers for the machine. A
Windows-on-ARM host on the x64 fallback runs the prebuilt helper under an
emulated x64 python and installs windows-cuda, so the machine's answer would
call that bundle mismatched and discard a working runtime. Test-WinArm64Venv is
the same predicate the torch path uses, and it short-circuits free on x64.

The block stays inert either way: write_prebuilt_metadata persists no
install_kind, so $existingKind is always null.

* Windows on ARM: pin the marker split, drop a write-only variable

Cross-platform verification of the requirement splits, plus one dead variable
the earlier fix left behind.

WoaPythonMinor was written in three places and read in none. It was the variable
the staging tag was meant to use; that moved to WoaVenvMinor, which is keyed on
the venv's own interpreter. Leaving a write-only "the minor we probed" next to
the real one only invites a future reader to take the stale answer.

test_windows_arm64_requirement_markers pins the mechanism the whole split rests
on. Every package needing a different version on win_arm64 is two rows whose
markers are exact complements, so exactly one is live everywhere. Both ways of
breaking that are silent: an overlap makes pip intersect two specifiers and can
render the row unsatisfiable, and a gap drops the package on a platform nobody
tested. The tests evaluate every row of every requirements file and every
pyproject extra across 60 environments.

Two of the checks exist because a scope-limited test skips itself when the bug
appears. Spell the machine "arm64" rather than "ARM64" and the conjunction with
sys_platform == "win32" becomes unsatisfiable, the split stops existing, and any
check that looks for a split finds nothing to look at; Windows on ARM quietly
takes the x64 pins and source-builds. That is caught semantically, by asserting
no row is dead in every environment, rather than by matching marker text --
torchcodec legitimately pairs 'arm64' with sys_platform == 'darwin' in a
separate disjunct. And the expected rows are asserted per source file, because
studio.txt and pyproject[studio] mirror each other, so a global check stays
green when only one of them loses a row.

Verified by mutation: overlapping rows, a lowercase arm64 in either file,
aarch64 for ARM64, a deleted WoA row, and either deliberate exclusion
(xformers, hf_transfer) losing its marker are each caught, and each by a
different assertion.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: close seven gaps the last round left open

All seven are on code the previous rounds added, which is where a fresh pass
should look.

The resolver paths have to be readable by uv or the native stack is not viable.
uv splits UV_OVERRIDE on whitespace and accepts no quoting of any kind -- bare,
"quoted", 'quoted' and back\ slashed all fail identically on uv 0.10.7 -- so the
8.3 short form was the whole mitigation, and 8.3 generation can be disabled on
the volume. A spaced StudioHome then has no space-free spelling and every later
uv call dies on a truncated path. Decide that up front and take the x64 path,
which never sets these variables, rather than staging wheels into a venv that
cannot resolve. Same rule the pyarrow probe already applies.

That short form then has to be recognised on the way back in. The block that
drops a previous run's resolver variables matched only the long spelling, so
after a native install under a spaced home it no longer recognised its own
values and the next run in that shell inherited them. Both spellings are
checked; a wheelhouse the user set for themselves is still left alone.

The probe follows the interpreter that ends up owning the venv. Re-probing for
the minor Find-CompatiblePython settled on was not enough: the ARM64 bootstrap
below it installs $PythonVersion, which can be a third minor again, and the
index and pyarrow source would both have been confirmed for one no longer in
play.

setup.ps1 is told which index install.ps1 used. It otherwise derives the CUDA
index from the driver, and download.pytorch.org publishes no win_arm64 CUDA
wheel at all, so a repair or a missing companion on a native venv resolved
against an index that cannot serve it. Passed as UNSLOTH_WOA_SELECTED_TORCH_INDEX
rather than UNSLOTH_WOA_TORCH_INDEX_URL, which is the user's input override: a
second run in the same shell would otherwise read this run's choice as a pin.

A wheel's ABI has to agree, not just its interpreter tag. cp313-cp313t is the
free-threaded build and uv rejects it on a regular cp313, so counting it as
available dropped the package's override and sent the resolve to the sdist the
override existed to avoid. The Python half already checked this; the PowerShell
half did not.

Blockers decide even when the package's own wheel is hosted. tensorboard and
librosa publish py3-none-any wheels that the staging step copies, so a
wheelhouse can hold the package and still lack grpcio or numba; unskipping it
there walks straight into the blocker's sdist.

UNSLOTH_LLAMA_ARM64_CUDA=0 is honoured on the direct-upstream path too. It was
checked only in the fork resolver, so the documented escape hatch silently did
nothing with --published-repo ggml-org/llama.cpp.

Each fix is mutation-tested: reverting either Python-side change fails a
specific new test, and the PowerShell changes are asserted structurally and by
executing the extracted helpers.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: build probe URLs with Join-UrlPath

A tokenized mirror is a supported form for UNSLOTH_TORCH_INDEX_URL and for the
wheelhouse, and "$base/$leaf" puts the leaf inside the token value:
https://mirror/whl?token=abc became .../whl?token=abc/torch/, so the preflight
read the mirror root or an error and disabled the native stack on a host whose
wheels were there. Join-UrlPath already solves this for the xFormers URLs; it
moves up next to the other Windows-on-ARM helpers, since PowerShell resolves a
function only once its definition has executed and these probes run far above
its old position. Applied to the torch/torchaudio preflight and to all three
wheelhouse URLs, which had the same shape.

* Windows on ARM: abandon the native route before building, and carry the index across shells

Two review findings on the previous head.

Abandon the native route earlier. The check for a StudioHome uv cannot read --
a path containing a space on a volume with 8.3 short names disabled, which uv
splits UV_OVERRIDE on -- sat next to the resolver files it guards, which is
after the ARM64 venv has been created. The install therefore paid for a venv it
then tore down. Test-WoaResolverPathsUsable now runs inside
Initialize-WoaNativeCudaTorch, immediately after the GPU probe, so the route is
never chosen in the first place and every re-probe honours it. The verdict is
cached, so a user with several interpreters reads the explanation once.

Carry the chosen index across shells. install.ps1 exports the index it probed,
but a later `unsloth studio update` runs in a fresh shell where that variable is
gone, and Windows on ARM is the one platform whose CUDA wheels are not on
download.pytorch.org at all, so the index cannot be re-derived from the driver.
The manifest now records it, and setup.ps1 falls back to it when the handover
variable is absent.

That is a deliberate exception to the rule install_manifest.py documents for
itself -- the flavor, never the index URL it came from, since a pinned index can
carry a token -- so it is narrowed to the case that motivated it. Only
credential-free https://pypi.nvidia.com URLs are written, with no userinfo,
query or fragment, and setup.ps1 re-checks the same thing on read so a
hand-edited manifest cannot redirect a torch install to another host. A mirror
the user pinned is deliberately not persisted; they supply it through their own
environment as before.

tests/studio/install/test_woa_torch_index_persistence.py covers both halves of
the guard, the Python that writes and the PowerShell that reads, including the
older manifests that carry no such key and the truncated ones a killed installer
leaves behind.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: restore the generated requirement overrides on a fresh-shell update

The torch index was not the only thing install.ps1 hands over through a
process-scoped variable. It also writes StudioHome\woa\overrides.txt, stages a
win_arm64 wheelhouse beside it, and exports both through UV_OVERRIDE /
UV_FIND_LINKS / PIP_FIND_LINKS. A direct `unsloth studio update` runs in a fresh
shell where all three are gone, so the dependency pass resolved without them.

That matters because studio.txt pulls ddgs, which requires
httpx[brotli,http2,socks], whose brotli extra is Brotli on CPython, and neither
Brotli nor brotlicffi publishes a win_arm64 wheel. Confirmed against the
published metadata and against the resolver: uv pip compile for
httpx[brotli]==0.28.1 targeting aarch64-pc-windows-msvc resolves 8 packages
including brotli==1.2.0, and 7 with the overrides applied. With no wheel to
install, those 8 mean an sdist build of a C extension -- the exact failure this
path exists to avoid. Only reachable since this PR: an emulated x64 venv has a
win_amd64 brotli wheel.

Restore-WoaResolverEnvironment sits next to the torch-index recovery so both
halves of the lost handover are put back in one place, before the torch installs
and the dependency pass, which is where install.ps1 has them set for its own run.
It restores rather than re-derives: both files persist under StudioHome\woa\, and
the generated file is the only accurate record of what was dropped, since the
drops depend on what the wheelhouse turned out to hold. No-op unless
Test-WinArm64Venv, and it returns untouched if any of the three variables is
already set, so install.ps1 driving the resolve in the same process is never
clobbered. A native venv whose overrides file was deleted warns and points at
install.ps1 rather than guessing the list.

Get-UvSafePath is duplicated from install.ps1 because neither script can
dot-source the other; a test asserts the two copies stay identical, since a copy
that drifts would hand uv a path install.ps1 had already ruled unreadable.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: re-export the recovered index, restore each resolver variable, and cover no-torch

Three follow-ups on the fresh-shell recovery.

Re-export the recovered index. install_python_stack.py rewrites the manifest at
the end of every run from UNSLOTH_WOA_SELECTED_TORCH_INDEX, so recovering the
index into a local variable only meant the first fresh-shell update wrote a
manifest without it -- erasing the one thing the manifest exists to carry, and
making the whole mechanism work exactly once. It is now put back in the
environment right after it is recovered, guarded so an empty recovery exports
nothing. The value is always one install.ps1 chose or one write_manifest's
credential-free allowlist already accepted, so re-exporting widens nothing.

Restore each resolver variable independently. Returning when ANY of the three
was set was aimed at the install.ps1 case, where all three arrive together, but
it also fired for a shell carrying an unrelated UV_FIND_LINKS or PIP_FIND_LINKS
-- a corporate wheel mirror has nothing to do with the win_arm64 drop list, and
should not cost the user the brotli exclusions. Each is now restored on its own
and a caller's value is still never overwritten, so the install.ps1 path is
unchanged: every branch no-ops there.

Cover no-torch mode. Both recoveries sat inside `if (-not $NoTorchMode)`, and
neither is about torch. install_python_stack.py installs studio.txt at function
scope with no NO_TORCH gate, and ddgs is not in NO_TORCH_SKIP_PACKAGES, so
UNSLOTH_NO_TORCH=1 resolves the same brotli chain; the manifest is rewritten in
that mode too. Both have moved above the guard. The torch-index USES stay inside
it, where they belong.

The new tests assert the placement rather than the text, and one asserts the
studio.txt pass is still unconditional so this reasoning cannot go stale
unnoticed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: read the manifest before dropping it, publish the index torch came from

Three follow-ups, the first of which made the persistence added earlier a no-op.

Read the index before the manifest is deleted. setup.ps1 drops
unsloth_install_manifest.json before it replaces pip, torch and triton, so a run
killed in those leaves the venv marked half-built rather than behind a marker
that verifies. The recovery was placed below that drop and read a file that no
longer existed, so it returned empty every time -- on precisely the fresh-shell
path the persisted index exists to serve, and silently, since empty is also the
correct answer for an older install that recorded nothing. The recovery now runs
as soon as the dependency guard opens, before the drop. It stays inside that
guard: a run that installs nothing has nothing to recover for.

Publish the index torch was actually installed from. The native path resolves
from the channel install.ps1 probed, while $TorchInstallIndexUrl still names the
driver-derived family, so the handoff sent install_python_stack.py's repair to an
index with no win_arm64 CUDA wheel and named the flavor cu130 when the installed
wheel is +cu134 -- a repair could have tried to correct a working venv to a build
that does not exist for this platform. $_effectiveTorchIndexUrl is initialised to
$TorchInstallIndexUrl and overwritten only by the branch that installs torch, so
every other path publishes exactly what it published before. On the NVIDIA channel
the leaf is nvtorch_oot, not a CUDA family name, so no flavor tag is published at
all -- the honest answer, and what the block already documents for an unknown leaf.

Gate the llama.cpp ARM64 CUDA branch on the opt-out. resolve_upstream_asset_choice
is reached through resolve_asset_choice on the fallback paths, so leaving it
ungated meant UNSLOTH_LLAMA_ARM64_CUDA=0 worked in direct_upstream_release_plan
and silently did not work here. The same gate is already on the equivalent
branches at lines 1126 and 3897.

The new tests assert ordering and wiring rather than the function in isolation,
which is what let the first of these through.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: rebuild a migrated x64 venv as ARM64, and widen the llama.cpp CUDA opt-out

Rebuild a migrated legacy environment. The migration branches keep a healthy
~/.unsloth/studio/.venv exactly as it is, and on a Windows-on-ARM host those are
x64 by design: every install predating the native stack deliberately
bootstrapped an emulated x64 interpreter. The venv-platform guard then saw
win-amd64 and stood down to that same stack, so an upgrade left an NVIDIA user
on a Triton that cannot compile for sm_121. Only a SECOND installer run
recovered, because migrating makes the layout new and the new-layout branch
already does preserve-and-recreate.

Do that in the first run instead. Just above the venv-creation block, a migrated
environment that is not win-arm64 is preserved with the same
Start-StudioVenvRollback and the existing creation path builds an ARM64 venv.
Scoped to a venv THIS run migrated: a new-layout venv was already moved aside,
and one created moments ago came from the interpreter this run chose.

$_Migrated is cleared on success. It gates an upgrade-in-place far below that
installs unsloth with --no-deps --reinstall-package; left set, that would run
against a freshly created empty venv and leave an unsloth with no dependencies.
If the rollback throws, the environment is kept and the existing guard disables
native exactly as before -- a user's environment is not worth trading for a
native stack, so the platform guard keeps the final say.

Widen UNSLOTH_LLAMA_ARM64_CUDA to every ARM64 CUDA bundle. This is a deliberate
scope change, not only a bug fix. The helper documented itself as governing the
upstream bundle when the fork publishes no checksum, but what the flag promises
the user is the CPU bundle, and gating only the unverified tail meant the escape
hatch would switch itself off the day the fork published an approved
windows-arm64-cuda artifact, since the published branch returns first. The
docstring now says what the code does. The inner check inside that branch is
unreachable once the branch is gated, so it is gone rather than left reading as
a fallback that no longer exists.

The opt-out tests walk the AST rather than grepping, because the same attribute
guards the x64, Linux and macOS paths, which must not be gated: they assert both
that every Windows-ARM64 NVIDIA branch is gated and that no other one is.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: gate the pandas 3 row on Python 3.11, and keep the CPU llama bundle when CUDA is opted out

pandas 3 needs Python 3.11. Every pandas 3.0.x is requires-python >=3.11 and its
win_arm64 wheels start at cp311, which is why the Windows-on-ARM row exists at
all -- but the row carried no python_version bound while requires-python still
admits 3.9 and 3.10, so `pip install unsloth[studio]` was unsatisfiable on an
ARM64 3.9 or 3.10 interpreter. Split on python_version as well as platform,
which keeps the partition exact: below 3.11 an ARM64 host takes 2.3.3 and
source-builds, slow but resolvable, as it did before win_arm64 wheels existed.
Narrowing requires-python instead would drop 3.9/3.10 everywhere to fix one
platform. The same two rows appear in studio.txt and single-env/constraints.txt
with the same defect, so all three move together.

The 603-case marker suite passed throughout this bug's life: the partition was
exact and every environment had exactly one live row -- that row just could not
install anywhere. So the suite gains the axis it was missing, asserting that a
live row is installable on the interpreter it was selected for, checked against
the package's own requires-python, plus the complement that no ARM64 Python is
left with no pandas row at all.

Keep the ARM64 CPU llama.cpp bundle when CUDA is opted out. With
UNSLOTH_LLAMA_ARM64_CUDA=0 the selector installs a windows-arm64 CPU bundle on an
NVIDIA Windows-on-ARM host, but the mismatch check expected only
windows-arm64-cuda there, so it deleted a correct install on every setup and
update -- and an update that cannot download then leaves no llama.cpp at all.
The previous commit made this reachable in practice by widening the opt-out to
the published artifact.

The expected kind now follows the flag, INSTEAD of the CUDA kind rather than as
well as it: accepting both would keep a bundle installed before the flag was set,
so the opt-out would appear not to work on exactly the machine that already had
CUDA. Falsy spellings are _upstream_arm64_cuda_allowed's, with a test asserting
the two stay identical, since this is now one contract in two languages. The x64
arm is untouched and covered as a negative control.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: pair torchaudio with torch, prove wheels are CUDA, gate prereleases on nightly

Take torchaudio only as a matched pair. The GA channel serves torch 2.14.0+cu134
beside torchaudio 2.11.0+cu134, torchaudio 2.11 dropped the exact torch== pin
that 2.10 still carried, and the native specs are torch>=2.4 / torchaudio>=2.4 --
so nothing stopped the resolver installing that pair and leaving torchaudio's
extension to load against a libtorch three minors newer. install.ps1 already
names this hazard on the bounded-trio branch; the native branch was not covered
by it. The probe now reads versions rather than existence and enables torchaudio
only when its major.minor pairs with torch's, the relationship the old pin
encoded. No matching pair means no torchaudio, which is what every
Windows-on-ARM install did before this channel published one; it lights up on
its own the day NVIDIA ships a matching pair.

Establish CUDA positively. The probe rejected a literal "+cpu", which misses two
things: PEP 503 hrefs encode "+" as %2B, and PyPI's own win_arm64 torch wheels
carry no local version at all, so they are not "+cpu" and were read as CUDA --
sending the host native on CPU-only torch, which is worse than staying emulated
because the GPU then goes unused with no fallback. Filenames are unescaped and
must carry a +cuNNN local version. A private index whose CUDA wheels carry no
+cu tag now reads as unknown and takes the x64 path: a behaviour change, and the
safe direction, since that path works.

Gate --prerelease=allow on the nightly channel, as install.ps1 already does.
setup.ps1 passed it unconditionally and its comment ("the wheels are nightlies")
predated the GA channel. "allow" means every prerelease, and it rode on a command
carrying unsafe-best-match and an explicit PyPI extra index, so a prerelease of
torch or of any shared dependency could outrank the stable GA build on every
setup and update. The other two flags stay unconditional; the GA channel needs
them.

Verified against both live NVIDIA channels (temp/sim10282/probe_test.ps1) and
covered offline with stubbed index pages for the parsing and pairing rules.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: five resolver and validation fixes

Reject abi3 wheels on free-threaded interpreters. The exact-minor branch
accepted "abi3" unconditionally, shadowing the guarded branch below it, so
cp313-abi3 was installable on 3.13t while cp311-abi3 was correctly refused.
Free-threaded builds do not implement the stable ABI (CPython #111506, PEP 703),
so this marked a blocker available, dropped its skip and sent the resolver at a
wheel it cannot use. PEP 803's abi3t is 3.15 and outside requires-python, so it
is documented rather than matched. The test simulates both builds instead of
reading whichever interpreter runs the suite.

Validate UNSLOTH_PYARROW_WHEEL. Every other branch of Get-WoaPyarrowSource
checks the interpreter and platform tags; this one took any file that existed,
so an x64 wheel, a wheel for another minor or a truncated download selected the
native path and then failed at resolution, having already given up the working
x64 stack. Same checks as the siblings plus the zip signature, read two bytes
off a stream rather than loading tens of megabytes.

Keep the caller's resolver configuration. The purge above deliberately preserves
anything that is not this StudioHome's own, and the assignments then overwrote
it. Folded, not appended: uv combines override files rather than letting a later
one win, and two files naming one package without distinguishing markers is an
error -- and this file declares torch and torchvision. Same shape as
New-UnslothTorchOverridesFile. Find-links have no such conflict semantics, so
those are appended with the caller's kept and ours first. install_python_stack.py
split find-links on os.pathsep alone, which would have read an appended
"dirA,dirB" as one unusable path; it now splits on comma, whitespace and
os.pathsep. The generated file is UTF-8 now that a caller's line can be
non-ASCII.

Accept the automatic ARM64 CPU fallback. resolve_asset_choice falls through to
the published windows-arm64 bundle when no ARM64 CUDA asset exists, so demanding
the CUDA kind deleted a healthy fallback every run. Widening strands nobody: the
installer's already-satisfied check is per candidate and CUDA is attempted
first, which a test now pins. The opt-out arm stays exclusive.

Make the manifest writer and reader accept the same URLs. urlsplit().hostname
strips the port, so https://pypi.nvidia.com:443/... was persisted and then
refused by setup.ps1, losing the index on the next fresh-shell update. The
writer now requires netloc == hostname, admitting exactly what the reader admits
and dropping userinfo with it. Declining is preferred to silently rewriting a
caller's URL; the default channels carry no port.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: probe and stage the interpreter's ABI tag, not just its minor

A free-threaded build installs cp313-cp313t wheels, but every probe here derived
its tag from the minor alone and searched for cp313-cp313. The two halves then
failed in opposite directions: the index probe found the ordinary GIL wheels and
enabled the native stack, while the wheelhouse staging kept exactly those wheels
and discarded the cp313t ones the venv could have installed -- all after the x64
fallback had been given up, so the run failed at resolution with nowhere to go.

Reachable: Find-CompatiblePython's python3/python PATH scan and its `py -0p`
enumeration both accept any interpreter whose --version matches, which a
free-threaded build does, and the native path ranks ARM64 builds first.

Test-PythonFreeThreaded asks the selected interpreter for Py_GIL_DISABLED and
Get-WoaAbiTag turns the answer into cp313 or cp313t, threaded through the torch,
torchaudio and pyarrow probes and through the staging scan, which also records
the venv's own ABI. A free-threaded host therefore declines the native path
today, because no index publishes those wheels, and starts working by itself the
day one does -- better than refusing such interpreters outright. Unknown answers
GIL, the historical assumption, so nothing moves for a GIL host: there the two
tags are the same string.

The staging scan also stops accepting abi3 on a free-threaded venv, the rule
already applied to _wheel_matches_interpreter, so the two now agree.

Verified against the live GA channel: a GIL 3.13 still resolves torch
2.14.0+cu134 and a free-threaded 3.13 finds nothing.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: re-probe on ABI change, merge caller overrides, honour an explicit pin

Re-probe when the ABI changes, not only the minor. The previous round threaded
the interpreter's ABI through every probe but left the re-probe guard keyed on
the minor alone -- so the one case where the minor cannot change, a 3.13t chosen
for the default 3.13 request, was exactly the case that never reached the call
passing -FreeThreaded. The initial probe's GIL assumption stood, native mode was
enabled on cp313 wheels, and the venv was then built from an interpreter that
installs cp313t. $WoaProbedFreeThreaded is tracked beside $WoaProbedMinor and
either changing re-probes, including after Install-PythonFromPythonOrg, where
the ABI can change while the minor does not. Detection is scoped to an ARM64
host so no other Windows host spends a subprocess on an answer that cannot
matter there.

Merge caller overrides on an update instead of standing down. install.ps1 folds;
setup.ps1 skipped the generated file entirely whenever UV_OVERRIDE was set, which
keeps the caller's file and loses the win_arm64 drop list -- so ddgs ->
httpx[brotli] -> Brotli went back to an sdist with no win_arm64 wheel. uv
combines override files rather than letting one win, and errors on a duplicate
package, so the two cases differ: disjoint files are both handed over, needing no
rewriting and leaving each file's relative references resolving against its own
directory, and only an actual conflict is merged, with this platform's entries
winning because they name what it cannot build. Get-RequirementName(s) do PEP 503
normalisation so Brotli and brotli_cffi compare as one name, and a merge that
cannot be written falls back to the caller's file with a warning.

Let an explicit pin outrank the persisted index. UNSLOTH_TORCH_INDEX_URL and
UNSLOTH_TORCH_INDEX_FAMILY both land in $PinnedTorchIndexUrl, and the recovered
WoA index was winning over both -- so a user switching to another CUDA mirror was
silently still served by the previously recorded channel. The recovery is a
memory of what install.ps1 chose, not a decision the user made, so it is now
consulted only when nothing is pinned.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: keep caller resolver entries, survive a bad handoff URL, drop a stale torchaudio, check wheelhouse versions

- install.ps1: the resolver purge removes only the entries pointing into this
  StudioHome's woa directory instead of the whole variable, so a caller's
  mirror appended behind ours survives a second run in one shell.
- install_manifest.py: parse the WoA index URL inside a try/except ValueError;
  a malformed authority no longer aborts the whole manifest write.
- setup.ps1: on a native ARM64 install that leaves torchaudio out of the trio,
  uninstall an already-installed torchaudio whose major.minor no longer
  matches torch.
- install_python_stack.py: the find-links scan records versions, and a hosted
  wheel clears a win_arm64 skip only when one of its versions satisfies the
  pin in the requirements file being installed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: match wheel tags as fields, evaluate markers, bound the purge, version the drop list

- install.ps1: one Test-WoaWheelTags parses the PEP 425 fields and expands each
  dot-separated tag set, replacing seven substring probes. "*cp313-cp313*" also
  matched cp313-cp313t, so a free-threaded wheel could select and then be staged
  for a GIL interpreter, failing only after the ARM64 venv was committed to.
- install.ps1: the resolver purge treats only the prefix itself or a descendant
  past a path boundary as owned, so <StudioHome>\woa-mirror survives.
- install.ps1: the override drop list records wheel versions and consults the
  floor the released metadata puts on the name (xformers>=0.0.22.post7) before
  removing a drop; a lower hosted version keeps it.
- install_python_stack.py: requirement markers are evaluated for the running
  interpreter, so the inactive MeCab row no longer overwrites the active one.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: honour any explicit pin, preflight PyAV, keep caller override files in place

- setup.ps1: the CUDA-preservation shortcut exempts any explicit torch index
  pin, not only a /cpu one. It distrusts the inferred host-family comparison; a
  pin at a cu129 mirror is a stated instruction, and the narrower test skipped
  the dependency pass and kept cu134 while reporting success.
- install.ps1: on a free-threaded interpreter, native mode requires an
  installable win_arm64 av (PyAV) wheel. The wheel constraints.txt asks for is
  cp311-abi3, which no free-threaded build can use; PyAV publishes cp314-cp314t
  but no cp313t, so the index is asked rather than assumed. GIL builds are
  unchanged and never run the probe.
- install.ps1: a caller UV_OVERRIDE file that declares none of our packages is
  passed to uv where it sits, so its relative -r and wheel paths keep resolving.
  Only a conflicting file is folded, and its relative references are rebased
  onto the directory they were written for.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: read the machine arch from the registry, not from the process

Fresh install of this branch on the GB10 laptop went down the x64 path: plain
"NVIDIA GPU detected" and cu130 from download.pytorch.org, which publishes no
win_arm64 wheel, with no line saying why. Running the probe's gates one at a time
found it: Get-HostMachineArch answered x86_64, so Initialize-WoaNativeCudaTorch
returned on its first check before printing anything, and every later gate,
nvidia-smi, the NVIDIA index, the pyarrow wheel, would have passed.

Every signal it trusted follows the process, and the install ran from an x64
emulated shell. There PROCESSOR_ARCHITECTURE is AMD64, PROCESSOR_ARCHITEW6432 is
empty because it is a WOW64 variable and x64 emulation on ARM64 is not WOW64, and
.NET Framework's OSArchitecture reports X64, a bug fixed only in .NET 5. VS Code's
integrated terminal is often x64 on these machines, so this is a real user path,
not a test artefact. The machine-scope PROCESSOR_ARCHITECTURE from the registry
is the one value emulation cannot misreport, and it now leads the signal list in
install.ps1, in setup.ps1, and in setup.ps1's Get-UvHostArch so the uv an update
picks matches the one install.ps1 chose. The per-process signals stay as
fallbacks; on x64 hardware every answer is unchanged.

setup.ps1 is the one that made this urgent. Test-WinArm64Venv short-circuits on
the same function, so from an x64 terminal an update of a native ARM64 venv would
have skipped the CUDA preservation guard and repaired a working cu134 torch to a
cu130 that cannot install there.

The python side had the identical hole: _is_windows_arm64 read the same three
signals and its docstring promised ARCHITEW6432 would be set under emulation,
which it is not. It reads the registry first now. Two of its three callers drop
torchaudio on the grounds that no win_arm64 wheel exists, which is a property of
the interpreter, not the machine: an emulated x64 venv on an ARM64 box installs
win_amd64 wheels and download.pytorch.org publishes torchaudio for those. Both
move to _is_win_arm64_interpreter. The third caller prints the Vulkan note, a
machine fact, and stays.

Verified with the real functions lifted from install.ps1 and run under both an x64
emulated and a native ARM64 PowerShell: both now answer arm64 and select the
nvtorch_oot index. Under the x64 and the ARM64 CPython on the same box the machine
predicate is True for both and the interpreter predicate is True only for the
ARM64 one, which is the disagreement the wheel filters rely on.

Tests: the arch-probe cases patch the registry helper so they answer the same on
an ARM64 host as on x64 CI, and gain the emulated case with every per-process
signal saying AMD64. The Vulkan-advice fixture takes the host as a parameter
instead of reading the box it runs on; three of its tests had failed on every
ARM64 dev box for that reason.

* Make the marker-evaluation test independent of the host running it

test_pins_are_read_canonically_and_markers_evaluated wrote a row under
`sys_platform != 'win32'` and asserted the pin survived because "its marker holds
on this host". It holds on a Linux runner and on no Windows machine, which is
every machine this file is about; on the GB10 laptop it fails with a KeyError.

The reader answers True, False or None for a marker, and only False drops the
row, so the test now uses one marker that holds everywhere and one that holds
nowhere, and asserts both directions. Same coverage, no dependence on the box.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: prepend the managed wheelhouse, rebase merged overrides, tolerate a self-copy

- setup.ps1: the resolver recovery prepends $StudioHome\woa\wheels to an existing
  UV_FIND_LINKS / PIP_FIND_LINKS instead of standing down. find-links are
  additional search locations, so skipping ours because a caller had set a mirror
  left the staged win_arm64 wheels out of the search and sent pyarrow to an sdist.
- setup.ps1: Resolve-WoaOverrideLine is carried over from install.ps1 and every
  line folded into overrides.merged.txt is rebased against the caller file's
  directory, so a relative -r, -c, -f or wheel path keeps resolving.
- install.ps1: both wheel staging copies compare normalised paths first. A
  wheelhouse pointed at the managed wheel directory made Copy-Item overwrite an
  item with itself, which under ErrorActionPreference Stop aborted the install.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: open the supplied wheel, and honour a configured win_arm64 mirror

- install.ps1: UNSLOTH_PYARROW_WHEEL is validated with Test-ZipArchiveReadable
  rather than a two-byte PK sniff. An interrupted download still carries that
  signature, and accepting one selected the native path, staged the broken file
  and failed the pyarrow resolve with the x64 stack already given up.
- setup.ps1: UNSLOTH_WOA_TORCH_INDEX_URL is read ahead of the handover variable
  and the manifest. write_manifest persists only NVIDIA's own channels, since
  any other URL could carry a credential, so a corporate win_arm64 mirror
  survives a fresh shell nowhere else; without this a direct update fell back to
  the driver-derived index, which publishes no win_arm64 CUDA wheel.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: guard the third staging copy against a self-copy

UNSLOTH_PYARROW_WHEEL may point at the already-cached wheel under
$StudioHome\woa\wheels, which is how an offline run reuses the managed cache.
Copy-Item refuses to overwrite an item with itself; here the throw is caught, so
instead of aborting it disabled native mode after the ARM64 interpreter and venv
had already been chosen, leaving the run on a CUDA index with no win_arm64 torch.
Test-WoaSamePath now guards all three staging copies, and a counting test keeps a
fourth from being added without one.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make the new marker and path tests independent of the host running them

Same defect as the pin-reading test: a fixture marker of
sys_platform != 'darwin' or python_version < '3.14' picks the other row on a
macOS 3.14 box, and literal POSIX expectations for Resolve-WoaOverrideLine are
wrong on Windows, where GetFullPath returns a drive-qualified backslash path.
The marker fixtures now use one marker that holds everywhere and one that holds
nowhere, and the rewriter expectations are derived with os.path.abspath.

* Windows on ARM: trim the commentary

Comment-only pass over the branch: collapse wrapped explanations, drop
restatements of the line below them, and shorten test docstrings that
carried more than their first sentence. Every fact a comment was carrying
is kept.

The comment-only property is checked two ways: comment_tools.py for the
Python files, and a PowerShell tokeniser comparison for install.ps1 and
setup.ps1 (32293 and 29539 non-comment tokens, identical before and after).

One test changes: test_no_staging_copy_is_left_unguarded anchored its
search on a comment, so a comment pass could break it. It now anchors on
the code the region starts with, and still fails when a guard is removed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: six review fixes

- _version_satisfies now answers with packaging when it is importable, with
  prereleases off unless the specifier asks for one, which is the policy uv
  resolves under. A wheelhouse tiktoken-0.13.0rc1 reduced to (0, 13, 0) and
  read as satisfying ==0.13.0, so tiktoken was unskipped and the resolve fell
  to the ARM64 sdist. The numeric fallback now refuses a version it cannot
  model rather than guessing.

- Every pyarrow candidate is floored at the ARM64 constraint (>=21.0.0). A
  tag-compatible 19.x in the wheelhouse selected the native path, and staging
  then wrote an exact pyarrow== override that the constraint rejects, after
  the ARM64 venv had been built. A test keeps the two numbers in step.

- --prerelease=allow is decided by the version of the wheel the probe
  selected, not by whether the index URL says nightly. A mirror of NVIDIA's
  prerelease channel need not say so, and without the flag uv takes the
  stable win_arm64 CPU torch from the PyPI extra index instead of the CUDA
  build. install.ps1 hands the answer to setup.ps1, which cannot probe.

- The ARM64 CUDA-preservation shortcut in install_python_stack.py exempts
  every explicit pin, not only a /cpu one. A user asking for cu129 by URL or
  family was silently left on the cu134 build. setup.ps1 already did this.

- Both requirement scanners follow a nested -r. uv reads those includes, so a
  conflicting package one level down made two override files look disjoint;
  handing uv both is an error, and the caller's capped torch could win. The
  fold flattens the include, rebasing each line onto the file it was read
  from.

- install.ps1's torchaudio and prerelease flags apply only while the index
  they were measured on is still in force. Change UNSLOTH_TORCH_INDEX_URL and
  re-run an update in the same shell and the pin outranks the handover for
  the install, while a stale 1 asked the new index for a torchaudio wheel it
  does not publish and aborted the whole torch update.

Each fix is mutation-tested. Suite: 5902 passed, four pre-existing failures
unrelated to this branch (managed node runtime, tokenizers negative control).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: a version floor has to order prereleases

Test-WoaVersionAtLeast compared the numeric release and then .postN, so
21.0.0rc1 cleared a 21.0.0 floor. Staging writes an exact pyarrow==
override from whichever candidate it picks, and pyarrow>=21.0.0 rejects
that rc, which is the failure the floor was added to prevent.

A marker attached to the release now sorts below it. A larger release is
unaffected, so a wheelhouse nightly still clears an older floor: the
pyarrow 24.0.0.dev260 an end-to-end GB10 run staged is named in a test,
since a floor that rejected it would break a verified install.

The eleven-row ordering table is cross-checked against packaging in the
same class rather than asserted from memory, because the PowerShell
cannot import it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* GPU driver: an all-[N/A] compute-apps listing is "cannot attribute", not "nothing appeared"

Ran the Kaggle GPU payload against a fresh native install on the GB10 laptop.
Every workflow passed except cli_run, which said `unsloth run` served from the
CPU because no process appeared on the card holding more than 0 MiB. It had:
with the exact flags the CLI passes (-ngl -1 --threads 2) the same llama-server
generates at 7.7 to 8.3 ms per token, inside the --gpu-layers 99 band (6.5 to
7.1) and far from a two-thread CPU control (18.4). llama-server --list-devices
shows CUDA0 with 46 GB.

The verdict was blind, not wrong about the rule. On Windows (WDDM) and on
unified-memory parts nvidia-smi lists every CUDA process with used_gpu_memory
[N/A], and parse_compute_apps drops those rows, so a server that did appear
during the window looked like nothing appeared and the device-delta fallback
never engaged, because that branch only runs when nvidia-smi cannot enumerate
at all.

Listing alone is not evidence either: a -ngl 0 server is listed too, since a
CUDA build creates a context regardless. Measured here: the device counter
reads 132 MiB idle, 272 MiB with that bare context, 560 MiB with the 270M model
offloaded, so the existing 200 MiB fallback threshold separates the two.

count_listed_pids tells listed-but-unattributed apart from empty, and
nvidia_compute_apps returns None for it, which is what the verdict already
treats as "judge on the device delta". Readable listings are unchanged, and
parse_compute_apps keeps dropping [N/A] rows, so its tests stand. Tests added
for the count, for the None, and for the verdict on the numbers measured on
this box.

test_a_box_without_nvidia_smi_reports_no_gpu_rather_than_crashing fails on this
host on unmodified main too, since a real nvidia-smi leaks through its
simulation; not touched here.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: sqlite-vec cannot resolve in the studio extra

pyproject pinned sqlite-vec==0.1.9 unconditionally, so a native win_arm64
interpreter running pip install "unsloth[studio]" hit a row it cannot
satisfy: sqlite-vec publishes win_amd64 wheels only and no sdist at all,
so there is nothing to build and no version to floor to. The runtime skip
list already covers the name, but that filtering never runs for a direct
extra install, which resolves pyproject's rows.

Dropped on win_arm64 with the same negative marker cryptography and
pymupdf use, mirrored into studio.txt so the two stay in step. Nothing
regresses: rag_db.py guards the import and the router mounts either way,
reporting RAG unavailable.

Guarded generally for the studio extra: no skip-list package may be left
live on win_arm64, and a marker that drops one must drop it nowhere else.
Scoped to studio on purpose, since the other extras are x64 recipes
pinning win_amd64 wheel URLs and were never a claim about this platform.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: three review fixes

- The resolver purge split every variable on [,\s] and rejoined each with
  its own separator. uv reads UV_FIND_LINKS comma-separated, so a caller
  path like C:\private wheels is ONE directory there: the shared split
  tore it into two fragments, dropped the managed entry, and rejoined the
  pieces with commas, quietly losing an air-gapped mirror. Each variable
  is now split the way the tool that reads it splits it, in install.ps1
  and in _find_links_wheel_versions, where the same class of split made
  wheels in a space-containing directory invisible to the skip list.

- The native ARM64 index arguments were gated on $UseUv, so when uv could
  not be obtained at all the pip fallback got only --index-url pointing at
  a channel that publishes just the torch trio, leaving their shared
  dependencies unresolvable. Remove-UvOnlyResolverFlags already exists to
  make that list safe for pip, dropping --index-strategy and rewriting
  --prerelease=allow as --pre, so the arguments are built either way.

- The manifest was the only durable record of the WoA index, and the
  dependency pass deletes it before rebuilding it. A run that died in that
  window left the next update with no handover and no manifest, so it fell
  to the driver-derived cu130 and failed on an index with no win_arm64
  wheel, on every retry. A marker beside overrides.txt, written before the
  manifest is dropped, now carries it. It persists exactly what the
  manifest persists, checked on write as well as on read, so a mirror
  pinned with a token is not written to disk and a hand-edited file cannot
  redirect a torch install.

Seven mutations, all killed. The first attempt at the marker tests passed
with the write-side guard removed, since the read guard alone defends a
round-trip; a test that inspects the file on disk covers it now.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: two review fixes

- New-UnslothTorchOverridesFile kept relative references working by writing
  the merge into the caller's override directory, which only helped when
  every override file shared one. Round 19 made two the normal case on the
  native path, since a non-conflicting caller file is now kept where it
  sits, so the merge fell through to %TEMP% and every relative include or
  wheel path resolved against nothing. Each folded line is rebased against
  the file it came from instead, and nested includes are flattened for the
  same reason, so where the merge lands no longer changes its meaning.

- The dependency pass fell back to pip whenever uv failed or was missing.
  On the native stack that does not recover: the overrides lift the
  released torch cap, which no win_arm64 CUDA wheel satisfies, and drop the
  packages with no win_arm64 build, and pip has no override mechanism to
  translate them into. Constraints cannot stand in, they narrow a
  requirement rather than replace one. Both fallback routes now refuse with
  the reason and name the remedy instead of resolving the wrong stack.

Five mutations, all killed. Restored one behaviour the new scanner had
changed: comment lines are carried through a fold again, which
test_unsloth_torch_override.ps1 was right to be checking.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: four review fixes

- An include's inline comment is not part of its path. "-r nested.txt
  # corporate pins" is a valid line, and capturing the comment meant the
  file never opened, so a torch conflict inside it went unseen and the two
  override files were handed to uv as disjoint. Stripped only where
  whitespace precedes the hash, so "-r a#b.txt" keeps its filename.

- An index the marker may not record now CLEARS it rather than leaving the
  last one in place. A host that moved from NVIDIA's channel to a
  credentialed corporate mirror kept a marker naming the old public
  channel, and since the manifest does not record the mirror either, the
  next fresh-shell update read the marker and put torch back on a source
  the user had left. An unrecordable index inherits nothing.

- The local wheelhouse mirror counted a wheel on its filename, which is all
  _find_links_wheel_versions reads, so a truncated archive took its package
  off the ARM64 skip list and uv failed the whole dependency pass on it.
  It is opened before it is copied now, the same check the remote branch
  already applied.

- The GPU harness read a mixed nvidia-smi listing as proof of CPU: a
  readable row on one card keeps the mapping nonempty, so the all-[N/A]
  guard stays quiet and the newly launched server, reported as [N/A] on a
  WDDM or unified-memory card, is simply missing from it. A pid that
  appeared but carries no figure now defers to the device-wide delta, as
  the all-[N/A] case does. Both samples come from one nvidia-smi call, so
  the attributed mapping and the listed pids describe the same moment.

Seven mutations, all killed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: two review fixes

- The local wheelhouse pyarrow was selected and staged on its filename.
  Last round's archive check covered the OPTIONAL mirror, where a bad wheel
  costs one feature; this is the mandatory one, and staging writes an exact
  pyarrow== override from whichever file it picks, so a truncated wheel
  chose the native route and then failed the resolve with x64 already given
  up. The probe and the staging now apply the same filter, tags and
  readability, so they cannot pick different files.

- The device-wide VRAM total is only a measurement of this process when
  this process owns the card. Under --studio-concurrent a training leg
  shares it and both allocates and frees inside the window, which is how
  one run read the delta as -182.0 MiB while the server held 2.6 GB, so
  accepting a rise there would pass a CPU-served run on somebody else's
  memory. Both fallbacks to the total now refuse when anything was on the
  card before the launch. Decided on evidence rather than configuration:
  the payload is not told which half it is, but a pid already present says
  so, and an empty card keeps the fallback the [N/A] parts need.

Six mutations, all killed. Two tests from last round asserted the mixed
listing case with a co-tenant present, which is exactly the scenario that
must not pass; they now use an owned card, and the shared variants are
their own cases.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: quote a rebased override path that needs it

Resolve-WoaOverrideLine stripped the quotes off an -r/-c/-f target and
wrote the absolute path back bare. These options take one file argument,
so a space truncated it and the dependency pass failed opening a path cut
at its first space.

Two ways in, and only one is about quoting. A caller who quoted the value
had the quotes discarded. A caller who had no reason to quote a plain
relative name gets a space anyway when it rebases onto a directory that
has one, which no amount of preserving would have caught. The rebased
value is quoted whenever it contains whitespace, so both are covered, and
setup.ps1's parity copy is kept identical by a test.

The first version of this had the option token disappear: -match on the
rebased value replaces , so reading the groups afterwards returned
a bare path where -c <file> had been. The file already carries a comment
warning about exactly that. The groups are copied out first now, and a
test asserts the option survives.

Four mutations, all killed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: four review fixes

- The marker recorded the WoA chain, which never consults the generic
  UNSLOTH_TORCH_INDEX_URL pin the torch steps actually prefer. A run pinned
  elsewhere installed from the pin and then recorded an NVIDIA channel it
  had not used, and the next fresh shell went back to it. The pin decides
  what is recorded now, and an unrecordable one clears the marker.

- _find_links_wheel_versions read PIP_FIND_LINKS as well as UV_FIND_LINKS.
  uv does not consume it, so a wheel hosted only there was counted as
  available, dropped off the skip list, and then invisible to the uv
  resolve that followed. pip cannot be the resolver on this path either,
  since pip_install refuses the fallback once the overrides are in force.

- A transitive blocker had no direct row to satisfy, so any hosted version
  counted. tensorboard 2.21.0 requires grpcio>=1.74.0 and librosa 0.11.0
  requires numba>=0.51.0: a wheelhouse holding an older one lifted the
  skip, and the optional package's own metadata then failed the whole
  extras pass. The floors are recorded with the release they were read
  from, and a test fails if extras.txt moves off that pin. llvmlite has no
  entry deliberately: nothing states a floor for it.

- The PyPI pyarrow route cleared native mode on a wheel it then forgot, so
  no exact override was written. With only pyarrow>=21.0.0 in force, uv
  takes the newest release, and one that ships an sdist for this
  interpreter builds Arrow from source. The matched wheel is pinned now,
  and the probe clears the name on entry so a re-probe cannot inherit it.

Six mutations, all killed. The first attempt at the second fix keyed on
USE_UV, which is False until the bootstrap runs, so it read the wrong
variable everywhere else; the list is unconditional now, which is correct
because uv is the only resolver that reaches it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: open the downloaded pyarrow wheel before keeping native mode

A mirror can serve a truncated body with a 200, and Invoke-WebRequest
reports that as success. This is the mandatory wheel: the exact pyarrow==
override is written from it, so an unreadable download kept the native
route and then failed uv on that override, after x64 had been given up.
The optional remote wheels below already opened what they kept; this one
was the last route that did not.

It throws into the existing catch, so the failure falls back to x64 rather
than continuing, and the bad file is removed first -- left in the managed
directory it would be read by the resolver on every later run, and by
_find_links_wheel_versions as proof the package is available.

Three mutations, all killed.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Record the effective torch index in the manifest, not just the marker

install_python_stack.py writes UNSLOTH_WOA_SELECTED_TORCH_INDEX into the
manifest as woa_torch_index, and the read chain in setup.ps1 prefers the
manifest over the marker. Exporting the WoA chain while saving the pin left
the manifest shadowing a marker that had already been corrected, so the next
fresh shell went back to the index the run had not installed from.

Resolve the value once from Get-PinnedTorchIndexUrl, falling back to the WoA
chain, and write both records from it. The torchaudio and prerelease flags
install.ps1 leaves in the shell describe the index it probed, so drop them
when the resolved index differs from that one.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Drop the generated requirement snapshots that reached the branch

_filter_requirements writes its skip-filtered copy beside the source so relative
-r/-c includes still resolve, and pip_install unlinks them in a finally. A test
that calls the helper directly, or an install killed mid-run, leaves them in the
checkout, where a git add -A picks them up. Nine were committed that way.

They are a snapshot of a file already in the tree, so a stale one reads as a
second, silently divergent copy of the pins. Remove them, ignore the pattern, and
add two tests: one that no tracked file matches the generated shape, one that the
ignore rule covers it. The shape is matched exactly rather than by a leading dot,
so a real hidden requirements file such as .constraints.txt stays in scope.

* Persist an explicit pin, install hosted optionals, scope the GPU samples

Three separate holes.

setup.ps1 ran the index persistence block only when the WoA chain had something
in it. A venv installed through a credentialed mirror has nothing to recover, so
a later run pinned at an NVIDIA channel installed from the pin and recorded
neither record, and the next fresh shell fell back to the driver-derived index
that publishes no win_arm64 CUDA wheel. Either record now opens the block.

install.ps1 reported that a hosted optional wheel was keeping its package, but
only declined to emit the removal override. hf_transfer and xformers are
excluded on win_arm64 by marker, so no requirement survived for the wheel to
satisfy, and torchcodec's only requirement line was filtered out here
unconditionally. Make that filter conditional on a hosted wheel, and install the
distributions the metadata cannot ask for, --no-deps and best effort.

The Kaggle GPU harness sampled every physical card while the payload was pinned
to one, so an unrelated process on a hidden card could supply the memory that
carried a CPU-served run past the assertion. Both samplers now pass
CUDA_VISIBLE_DEVICES to nvidia-smi -i.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pin the probed CUDA trio, and validate the wheelhouse xformers

uv documents --index-strategy unsafe-best-match as selecting the best version
from the combined candidate set of every index, and the PyPI extra index is
there because NVIDIA's channel publishes only torch, torchvision and torchaudio.
So torch>=2.4 takes PyPI's stable win_arm64 CPU wheel the moment it is one
release ahead of the selected channel, and the native GPU path is replaced by a
CPU build that imports perfectly. Probe torchvision as well as torch and
torchaudio, keep all three versions, and install them pinned exactly, local +cu
tag included. An unreadable version keeps the floor it had before.

The optional wheelhouse installs added last commit took the bare name, so any
tag-compatible wheel landed: below the declared xformers floor, or built for
another torch and CUDA pair, whose extension then fails to load while the step
reports success. Select the newest hosted version that clears the floor,
install that version, and for xformers compare cpp_lib.json against the resident
torch, removing it on a mismatch the way the post-repair resync already does.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pin the repair path too, pair the companion wheels, fix two availability rules

The forced-repair path in setup.ps1 installs the trio with the same
unsafe-best-match and public PyPI extra index as the install path, so open-ended
specs there had the defect the install path was just fixed for. Re-probe the
effective index at repair time and pin what it publishes; a repair can happen in
a fresh shell long after the install, so a remembered version would be stale.
The probe is a parity copy of install.ps1's, compared body for body by a test.

Pinning both torch and torchvision exactly made the pair mandatory, and NVIDIA's
nightly channel publishes each project on its own schedule while nightly
torchvision metadata pins its exact torch. Select the companions as builds of the
torch that was chosen, matching on the dev stamp and the local tag, and fall back
to the floor when the index carries no companion from that build.

Test-WoaVersionAtLeast read .dev as a pre-release of the release, so
0.0.22.post7.dev0 came out at or above 0.0.22.post7. PEP 440 hangs .devN off
whatever precedes it. Track it separately and as an ordinal.

The win_arm64 skip list was decided from the local wheelhouse alone, so a native
CPython 3.14 host dropped librosa even though llvmlite and numba publish cp314
win_arm64 wheels. Record what the public index resolves for this interpreter.

Also removes a file named - from the root: a workspace simulation ran
uv pip compile -o -, which writes a file rather than to stdout.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten the Windows on ARM comments

Post-convergence comment pass over the three files that carry the added prose:
collapse the multi-line rationales to the fact each one is there to state, and
delete the ones that restate the line below them. 782 added comment lines to
569. Comments, docstrings and whitespace only, verified with comment_tools.

* Stop the override file from undoing the CUDA pins, and two probe fixes

uv's --overrides replace a requirement's version even when it is named directly
on the command line: verified on 0.10.7, an override of packaging>=20 beat a CLI
packaging==24.0. The generated file carries torch>=2.4 and torchvision>=0.19, so
it discarded the exact CUDA pins the probe selected and best-match then took
PyPI's newer CPU wheel. Build the value for that one command without the trio,
passing every other file through, and restore it in a finally so the later
unsloth resolve still gets the drop list.

nvidia-smi failing to answer is not evidence the GPU is gone. A transient
failure during a direct update dropped windows-arm64-cuda from the expected
kinds, deleted the working llama.cpp tree and installed the CPU bundle. Accept a
win_arm64 venv on a persisted WoA CUDA index as evidence too; only NVIDIA's own
channels are ever persisted, so a cpu pin cannot claim it.

The public-index wheel table is only availability if the resolve looks at PyPI.
Offline, or pointed at an exclusive corporate index, unblocking librosa dropped
the skip and then failed the extras pass on an unavailable numba chain.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Take a wheel from PyPI when PyPI builds it, not from the wheelhouse

The Windows on ARM wheelhouse exists for distributions PyPI has no win_arm64 build
of. regex is not one of them: PyPI has published win_arm64 regex since 2025.7.29, and
the wheelhouse copy is the same version. The staging directory is first in
UV_FIND_LINKS, so ours won the tie and the venv got our binary of a wheel the project
releases itself. Nothing needed it there either: regex is in no skip list, no drop
list and no constraint, so it unblocked nothing.

Staging now asks PyPI first and skips the copy when PyPI publishes that project at or
above the version being staged. Version-aware in both directions, because the reverse
is a downgrade: an upstream release BEHIND the wheelhouse leaves ours in place.

The probe is PyPI-only on purpose. Test-WoaWheelAvailable falls back to the wheelhouse,
and a wheel is by definition present in the wheelhouse it is being staged from, so
reusing it would answer $true every time and the guard would never fire. Its PyPI leg
became Test-WoaPyPIWheel and it now calls that, so there is still one place that knows
how to read the simple index.

Tag matching is relaxed for this caller alone. hf_transfer ships cp38-abi3 and
sqlite_vec ships py3-none, both of which import on cp313; the exact-tag test would call
them foreign and go on shipping ours the day upstream publishes one. Free-threaded
venvs keep the exact test, which is what it was written for. A wheel this venv could
not have used is never judged: a cp312 wheel is not made redundant by a cp313 wheel.

pyarrow keeps its own path. Get-WoaPyarrowSource already probes PyPI before the
wheelhouse and reports which one it used, which is why the generic loop skips it.

Two comments were wrong and are corrected. pyarrow has never published a win_arm64
wheel at any version, so the constraints floor is the floor of the wheels we stage
rather than of anything upstream ships, and Get-WoaPyarrowSource checks an explicitly
supplied UNSLOTH_PYARROW_WHEEL before PyPI, not after.

* Fix uninstall.ps1 Split-Path calls that throw on Windows PowerShell 5.1

Split-Path puts -LiteralPath in its own parameter set, which carries only
-Resolve and -Credential. -Parent belongs to the -Path set, so naming both
resolves no parameter set and the call throws AmbiguousParameterSet. It
throws at runtime rather than at parse time, which is how eight of these
shipped in scripts/uninstall.ps1.

Two of them sat outside a try/catch, so a real uninstall printed a red
error and left the derived path null: the legacy sibling build at
<parent>\stable-diffusion.cpp was never removed, and the empty path then
tripped the deny list into "refusing to remove unsafe path:" with nothing
after the colon. The other six were swallowed by their surrounding
try/catch and failed silently. That left _RootFromConf returning nothing,
so studio.conf-based root discovery never worked, and it disabled the
parent-of-USERPROFILE rule in _IsUnsafeRoot, so C:\Users read as a safe
recursive-delete target.

Drop -Parent rather than switching to -Path. -LiteralPath on its own
already returns the parent, and -Path globs, so an install root containing
[ ] would be read as a wildcard.

Verified on this box under Windows PowerShell 5.1.28000.2704 against a
real env-mode install: the root, its studio.db, auth, outputs and exports
all go, an Unsloth-owned sibling stable-diffusion.cpp now goes with them,
and an unowned checkout at the same path is still kept.

The new test is static because CI has no Windows PowerShell 5.1 to run
the scripts under. It covers install.ps1 and studio/setup.ps1 too, which
are already clean.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Give the uninstaller's remove retries a long enough budget

Uninstalling a Studio that had actually been used left the whole install
tree behind. The stop pass kills the server, but torch inductor's compile
workers keep handles on the .py files they wrote under
<root>\TORCHINDUCTOR_CACHE_DIR for several seconds after that. Those are
plain data handles on files the worker never loaded as a module, so
neither the ExecutablePath pass nor the loaded-module pass in
_StopProcessesLockingRoots has a process to kill, and the only thing left
to do is wait.

_RemovePath waited 700ms three times, so it gave up after about 2.1s and
reported 'could not remove', leaving the tree and studio.db on disk with
a summary telling the user to delete them by hand. Replace the flat sleep
with an escalating backoff that tops out near 20s, which covers the
teardown with room to spare and still bounds the run.

* Say that a Windows on ARM desktop build needs clang

Building studio/src-tauri for aarch64-pc-windows-msvc with only MSVC BuildTools
installed fails with

    ring@0.17.14: failed to find tool "clang": program not found

because ring and aws-lc-sys assemble their aarch64 sources through cc-rs, which
calls clang rather than cl.exe on that target. The error names the crate rather
than the missing toolchain, and the prerequisite was written down nowhere: not in
the workflows, not in CONTRIBUTING.md, not in the manifest.

Recorded in build.rs, which is the file whose job this is and the one a reader
reaches for when the build fails. No behaviour change.

Not a hypothetical: the desktop app was built and launched on a Windows ARM64 box
for this branch, and this was the only thing standing in the way. It builds and
runs natively there once clang is present, PE machine 0xAA64, window up and the
frontend rendering off tauri.localhost.

* Fix the two CI reds: a dead import and a pid-file race

Import hoist lint: run_studio_gpu.py still imported count_listed_pids after the
listing helpers grew up around it. nvidia_compute_apps_listing now returns the
pids themselves via listed_pids, because a MIXED listing needs names rather than
a count, so the count import had no caller left and only survived in a comment.
Dropped both. count_listed_pids itself stays: gpu_assert still exports it, its
tests still cover it, and listed_pids documents itself against it.

Tauri rust tests: stop_update_kills_descendants_after_the_group_leader_exits
panicked with ParseIntError { kind: Empty }. The test child runs
`echo $! > "$1"`, and the redirection creates that file before writing to it, so
waiting on is_file() can win the race and read an empty string. It is a real
flake rather than anything about this branch, and it does not reproduce
reliably: the same test passed on main minutes earlier. It surfaced here only
because the Tauri workflow is path filtered on studio/src-tauri and the clang
note is the first change this branch makes there, so this was its first run.

Wait for a pid that parses instead of for a path that exists, over the same kind
of budget. A child that never writes one now fails with a sentence rather than
an unwrap panic.

* Close three holes in the PyPI-first wheelhouse guard

Review of the guard I added in 7fe37c6 turned up three ways it could do harm,
one of them worse than the duplication it was written to fix.

The bad one: skipping a wheel made it stop counting as available. $WoaWheelNames
is rebuilt by scanning the staging directory, and a $WoaDropCandidates name that
is missing from it gets emitted as `name ; platform_machine == "AMD64"`, which
EXCLUDES the package on ARM64. So the day PyPI published a win_arm64 hf_transfer
or brotli, the guard would have turned "installed from our wheelhouse" into "not
installed at all" -- the opposite of preferring upstream, firing on exactly the
event the guard exists for. A skipped wheel is now recorded in
$script:WoaPyPIProvided with the version PyPI matched, and that is merged into
$WoaWheelNames, so availability reads the same whether the wheel came from the
wheelhouse or from PyPI. The xformers floor still applies to the PyPI version,
so a release below it keeps the drop.

abi3 was treated as universally compatible. It is forward compatible from the
version it was built against, so a cp314-abi3 wheel does not import on cp313,
and calling ours redundant against one would have left the package
uninstallable. Test-WoaWheelTagsUsable now applies the same minimum rule the
staging scan already used for abi3 and py3N, and free-threaded venvs keep the
exact-tag test, which is what it was written for.

PyPI publishing a wheel is only availability if the resolve will look at PyPI.
UV_INDEX_URL and UV_DEFAULT_INDEX REPLACE the default index, and offline flags
remove it, so on a corporate mirror the guard was dropping our wheel and leaving
it obtainable from nowhere. Test-WoaResolveReachesPyPI mirrors
_public_pypi_is_reachable in studio/install_python_stack.py, which reasons about
the same five variables for the same reason. An unreachable pypi.org already
failed safe; reachable but unused did not.

The new tests execute the helpers under pwsh instead of reading install.ps1 as
text, because two of these three were wired correctly and answered wrongly, which
is precisely what a source-shape test cannot see. Hermetic: the live PyPI leg is
exercised on hardware, not in CI.

* Don't recursively delete ~/.unsloth/studio unless it is ours

The custom-root branch refuses a path that carries none of the four ownership
sentinels ("refusing to remove non-Unsloth path"), and then the default root a
few lines down is removed with no check at all. So a bare run, the documented
one-liner with no UNSLOTH_STUDIO_HOME set, recursively deletes whatever happens
to sit at ~/.unsloth/studio, and the empty-dir prune takes ~/.unsloth with it.

That path is not far-fetched to own. On a machine where Studio was only ever
installed in env mode, ~/.unsloth/studio is an ordinary place for someone to
keep notes or a checkout, and nothing warns them that running the uninstaller
for their custom install will also clear it out. Hit it here for real: a run
against a disposable env-mode install removed an unrelated directory that held
only a log file, and reported nothing unusual while doing it.

Gate it on _IsStudioRoot, the same test the custom root gets. An interrupted
install that lost every sentinel is now left on disk with a message instead of
removed, which is the failure direction that does not destroy data.

Both scripts had it; uninstall.sh line 588 was the same bare call.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Pair stable companions by release line, honour uv config files, and clean up the override swap

Stable torch, torchvision and torchaudio releases all carry an empty dev stamp, so
the CUDA local tag alone paired a companion from any release the index still
served and the exact-pin install then asked for a pair that does not exist.
Test-WoaWheelPairsWithTorch now applies the release mapping for stable builds:
torchvision 0.(M+15) pairs with torch 2.M and torchaudio agrees on major.minor.
Nightlies still pair by dev stamp. The setup.ps1 parity copy is identical.

Only environment variables were consulted before assuming a resolve reaches
public PyPI. uv also discovers uv.toml and pyproject [tool.uv] from the current
directory and its parents, then the user and system files, so a no-index or an
exclusive default-index set there still answered "PyPI". Both the PowerShell
probe (Test-WoaResolveReachesPyPI, now also gating the pyarrow source probe)
and the Python one (_public_pypi_is_reachable) read that configuration, honour
UV_NO_CONFIG and UV_CONFIG_FILE, let an index in the environment decide alone,
and resolve doubt to "not PyPI".

The filtered torch-step override was written with GetTempFileName(), which
follows %TEMP% and so could produce a quoted, uv-rejected path, and the copies
were never deleted. New-WoaTorchStepOverrideValue now writes under the already
validated WoA directory, passes every path through Get-UvSafePath, and reports
the copies so the caller removes them in finally. setup.ps1 never performed
this swap around its own CUDA trio install, so the restored overrides.txt
floors undid its exact pins too; it now swaps the same way.

On the manifest fast path the dependency block is skipped, so the llama.cpp
NVIDIA-evidence check read an unset index and a transient nvidia-smi failure
deleted a working CUDA bundle. The check now reads the pin, manifest or marker
itself when the pass did not run.

sqlite-vec joins the explicit ARM64 optional install map: its requirement is
marker-excluded on win_arm64 everywhere the torch path applies, so a staged
wheel was ignored. The Kaggle harness stops the Studio server before the CLI
baseline regardless of skip_ui, so its own llama-server is not counted as a
co-tenant.

Tests cover each change and the free-threaded probe test inlines the helpers
Test-WoaWheelAvailable now routes through.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Decide "public PyPI" by host, not by substring

An exclusive index whose URL merely contains the name, such as
https://pypi.org.corp.example/simple or .../api/pypi/pypi.org/simple, passed
the substring test in both the PowerShell and the Python reachability probes,
so the pyarrow route preferred a wheel the resolve could never fetch. Both now
parse the URL and require the host to be exactly pypi.org. TestPyPI, which does
not carry these packages, is therefore not public PyPI either.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Rank uv's [pip] index settings above the top-level ones, and drop a redundant wheel's managed copy

uv pip gives [pip].no-index and [pip].index-url precedence over their top-level
twins, and an [[index]] entry with default = true beats [pip].index-url
(verified on uv 0.10.7 with dry-run resolves). Both readers assigned whichever
value came first in the file, so a uv.toml that set the top-level key ahead of
the [pip] one could report public PyPI as reachable while the real resolve went
to an exclusive index. Each value is now collected and ranked at the end, in
both install.ps1 and install_python_stack.py.

A wheelhouse wheel that PyPI now publishes was skipped at staging time but a
copy already in the managed directory stayed there, and that directory is
first in UV_FIND_LINKS, so it still won the tie: the offline-cache mode points
the wheelhouse at that very directory, and a URL wheelhouse leaves the copy an
earlier install staged. The managed copy is removed in both modes; an external
wheelhouse file is never touched.

test_woa_wheelhouse_guard_behaviour inlines every helper the reachability probe
now calls, so a missing one no longer answers True from inside a non-terminating
error.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Run the two uninstaller tests that no workflow ran

cross-platform-parity-ci already triggered on uninstall*.ps1 and
scripts/uninstall*.ps1, so editing the uninstaller queued a run that then
asserted nothing about it. test_uninstall_prebuilt_parity.ps1 and
test_uninstall_reparse_stop_roots.ps1 were referenced by no workflow at all.
Both are hermetic -- they parse scripts/uninstall.ps1 with the AST and execute
only the helper under test -- so they cost a few seconds on a row that is
already running.

Worth being precise about what this does not buy, because the gap is wider than
the wiring. Neither test executes the uninstaller BODY, and neither does any
other: every uninstaller test we have asserts source shape. A runtime
parameter-binding fault is therefore invisible to all of them, which is exactly
how `Split-Path -LiteralPath $p -Parent` reached users. It binds to no parameter
set and throws on every uninstall, on 5.1 and on 7 alike, but it parses, so the
board stayed green.

The reparse test also could not run unprivileged. It created its home with
New-Item -ItemType SymbolicLink, which needs elevation or Developer Mode, so it
died on NewItemSymbolicLinkElevationRequired before the first subtree check. It
now runs over both reparse kinds. A junction needs no privilege and is the shape
a user is likelier to have -- mklink /J is what gets reached for when moving a
studio home to another volume -- and _ManagedPathsUnderReparseTargets reads
nothing but .Target, which both kinds supply rooted. A kind that cannot be
created is reported as SKIP rather than passed over, and a final check fails the
run if neither kind was exercised, so an environment that can make no links
cannot report a clean pass having asserted nothing.

* Read a uv config the way TOML is actually written

Test-WoaResolveReachesPyPI decides whether PyPI is in the resolve at all, and a
wrong "yes" is the expensive answer: it lets the redundancy guard delete the
wheelhouse copy of a wheel nothing can then fetch, and makes Get-WoaPyarrowSource
answer "pypi" and skip a usable local pyarrow, which gates the whole native path.
The subset parser behind it answered "yes" for six shapes of config that had
replaced or disabled PyPI. All six failed in that direction.

Held against tomllib over 41 fixtures, the parser and a correct reading now agree
everywhere; before this they disagreed on exactly these six:

  no-index = true# offline lab                 a comment needs no space in front
  index-url = "https://corp/simple"#corp       same, on the value side
  "index-url" = "https://corp/simple"          a quoted key is the same key
  pip.no-index = true                          a dotted key is [pip]'s no-index
  pip.index-url = "https://corp/simple"        same
  [tool.uv] + pip.no-index = true              same, in pyproject.toml

The comment cut was `(^|\s)#`, which misses a `#` with nothing before it. That
cannot be widened to a bare `#`, though, because the same line may carry one
inside a string: `index-url = "https://host/simple#frag"` is a fragment, not a
comment, and the old pattern got that right only by accident. Both halves need a
scan that knows where the quotes are, so Remove-WoaTomlComment does that, and
Split-WoaTomlKey does the matching job for keys -- unquoting them and splitting
on dots, so `pip.no-index` is routed into the [pip] scope instead of being
compared, whole, against the literal `no-index` and never matching.

The two new helpers are added to the behaviour test's HELPERS tuple in the same
commit. Leaving them out would have been the bug that tuple exists to prevent: a
helper the prelude does not extract is a command-not-found inside
Test-WoaResolveReachesPyPI, which aborts the statement, skips the branch, and
answers True.

Coverage was the reason these survived. The PowerShell side had no config-file
fixtures at all -- test_pypi_counts_only_when_the_resolve_would_reach_it drives
environment variables only -- while the Python twin has had file fixtures and a
real tomllib all along. Sixteen cases now cover both directions of the scan, the
[pip] precedence both ways, a lookalike host, and the shapes that must keep
answering True so this cannot be mistaken for a blanket "not reachable".

* Put the uv user config where uv would look for it on Windows

The user-file row of test_a_configured_exclusive_source_is_not_pypi failed on
Windows and always would have. The fixture deletes APPDATA and then points only
XDG_CONFIG_HOME at a directory, but _uv_config_files reads %APPDATA%\uv\uv.toml
on Windows and $XDG_CONFIG_HOME/uv/uv.toml elsewhere, so on Windows the file was
written somewhere uv never looks, nothing was discovered, and the assertion that
a user-level no-index takes PyPI out of the resolve could not hold. It stayed
green because tests/python runs on the Linux row, where XDG is the right answer.

Both names now point at one directory, so the case is real on every platform.

test_project_outranks_user_for_a_scalar needed the same treatment for a sharper
reason: it wrote the user file to the old xdg path, which after this change is
not a config location anywhere, so it would have passed on the project file
alone while asserting nothing about precedence. It now writes to the same place
and first asserts the file is one _uv_config_files discovers, because a
precedence test whose lower-ranked file was never found is a test that cannot
fail.

* Don't strand an old install behind the new ownership gate

bcc0ffd stopped the uninstaller recursively deleting ~/.unsloth/studio unless
one of install.ps1's sentinels says the directory is ours. That was right, and
the list it used was too short.

On Windows share\studio.conf is never written -- only install.sh writes it --
so the three sentinels that actually decide a Windows root all postdate the
bin\ shim dir and the unsloth_studio rename. An install from before those has
neither: the venv lived at <root>\.venv, the venv's Scripts dir went on PATH in
place of a shim dir, and .unsloth-studio-owned did not exist yet. install.ps1
still migrates exactly that layout, at "found legacy Unsloth environment", so
it is not hypothetical. Uninstalling one refused, printed the user's own
install as a non-Unsloth path, and left the tree and studio.db on disk.

tests/sh/test_uninstall_arg_guard.sh has been asserting this all along -- its
fixture is a pre-marker unsloth_studio venv and "no arguments removes the
install" fails at eda3801.

So accept two more things Unsloth put there: the owner marker inside the legacy
.venv, and either venv dir carrying the unsloth console script pip generates for
the CLI. That is the property the gate is testing. A bare .venv, a venv merely
named unsloth_studio and a hand-made studio directory still have none of them
and are still refused.

(cherry picked from commit 05c6cdf9a23273bb7a5e9fe4900b59dbe6cda3a1)

* Recognise the abi3 pyarrow upstream is going to publish

pyarrow is the mandatory gate for the whole native path: staging turns the chosen
wheel into an exact `pyarrow==` override, so whichever file we select is what the
resolve gets. Today that is our own 24.0.0.dev260, a pre-release, while upstream
stable is 25.0.1 -- acceptable only because PyPI publishes no win_arm64 pyarrow
at all yet.

It will. apache/arrow#48539 adds the wheel and is held behind apache/arrow#50398,
whose plan is an abi3 floor of 3.11, so the first one on PyPI will be named
pyarrow-<v>-cp311-abi3-win_arm64.whl. Every pyarrow candidate went through
Test-WoaPyarrowWheelUsable, which gated on Test-WoaWheelTags -- exact PEP 425
fields -- and an exact test answers false for cp311-abi3 on every interpreter we
build. The day upstream shipped, we would have gone on selecting our own dev
build and pinning it, and nothing anywhere would have said so. Silent, permanent,
and pointing the wrong way for a PR whose whole aim is that a new user on this
hardware installs from upstream wheels.

Test-WoaWheelTagsUsable is the predicate that already answers this correctly, and
it keeps both properties exactness was buying: an abi3 wheel built against a
NEWER interpreter than the venv is still refused, because abi3 reaches forward
from its build version and not backward, and abi3 is refused outright on
free-threaded builds, which have no stable ABI (CPython #111506).

One function changed, because all six call sites already funnel through it -- the
supplied wheel, the PyPI scan, the wheelhouse directory and index, and the two in
staging. test_every_pyarrow_candidate_goes_through_the_floor counts them and
still reads 7.

Four cases cover the shape upstream will ship and the two directions it must not
loosen, and a fifth asserts the free-threaded refusal on its own. The preludes
that lift Test-WoaPyarrowWheelUsable out of install.ps1 gained
Test-WoaWheelTagsUsable in the same commit, for the reason that keeps recurring
in this file: a helper the prelude does not lift is a command-not-found, which
aborts the statement rather than answering, and the caller then reports success.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Windows on ARM: make the purge and merge tests speak the host's path separator

These tests are gated on pwsh being present, not on the platform, so on a Windows
box with PowerShell installed they run and 11 of them fail. None of it is the
product: the purge builds its owned prefix with Join-Path, which uses the host
separator, and the tests hand it a hardcoded POSIX home, so "C:\...\unsloth\woa"
was compared against ".../unsloth/woa" and never matched. In a real run both
sides come from the same Join-Path, so the purge is correct either way.

Build the fake home and the entries the way the host does. The same cases now
cover the separator the machine actually uses, and on POSIX the values are
byte-identical to before.

The two merge tests failed for a second reason: their fakepython stub is a
shebang script, which Windows cannot execute, so the merge got no pins back and
the failure was an empty path rather than anything about rebasing. Write a .cmd
stub there instead; PowerShell's & runs it and the stub ignores -c regardless.

473 passed, 0 skipped on win-arm64 with PowerShell 5.1.

(cherry picked from commit 8dc13a7692c325408a9cb56eb94b612e71f45a2f)

* Test the Windows on ARM install path on Windows on ARM

Everything this repo knows about win_arm64 has been tested only on x64 runners:
ubuntu-latest and windows-latest. That is the wrong machine for all of it, and it
is why several real bugs sat behind a green board.

Three of them could not have failed anywhere else. The purge builds its owned
prefix with Join-Path, which uses the HOST separator, so a test hardcoding a POSIX
home compared "C:\...\woa" against ".../woa" and never matched -- eleven tests
passed on Linux and failed on Windows for a reason that had nothing to do with the
purge. The uv config parser, which exists because Windows PowerShell 5.1 ships no
TOML reader, read six shapes of config wrongly, every one of them answering "PyPI
is reachable" for a config that had taken PyPI out of the resolve. And the
uninstaller's ownership gate refused genuinely old installs, leaving the user's
tree and studio.db on disk while calling their own install a non-Unsloth path.

windows-11-arm has been free for public repositories since 2025-08-07, and this
leg is cheap because none of this needs a GPU, CUDA or a compiler: it is
arithmetic over wheel tags, path handling and PowerShell parsing. A full install
or a Tauri build needs much more and is deliberately not attempted here.

Two rows, because the shells are not interchangeable and each is what something
real uses. pwsh 7 is what CI has always run. Windows PowerShell 5.1 is what the
CLI launches setup.ps1 with, and it is the only shell on a stock Windows box, so a
developer on this hardware runs 5.1 whether or not anyone planned for it.
cross-platform-parity-ci already keeps a 5.1 row for #8335 on the same reasoning.
The suites locate their shell with shutil.which("pwsh"), which 5.1 does not
provide, so that row puts a copy of powershell.exe on PATH under that name --
without it the row would SKIP everything and report green having run nothing. A
copy rather than a .cmd shim, because CreateProcess will not launch a .cmd without
a shell, and Python is what starts them.

Both architectures are asserted rather than assumed. A leg that silently ran on
x64 would be worse than no leg, since it would report green for precisely the
platform it does not cover, and setup-python needs an explicit architecture: arm64
because the image ships an x64 Python that runs under emulation and reports
win-amd64 from sysconfig -- half of these assertions are about wheel tags, so an
emulated interpreter would quietly test the wrong platform. A third check pins
that the shell resolved for the row is really the major version that row is for.

Dry run of this exact selection on win_arm64 under PowerShell 5.1: 671 passed, 0
skipped.

One note recorded in the workflow for later: GitHub moves the windows-11-arm image
to Visual Studio 2026 between 2026-09-21 and 2026-09-30. This job invokes no
compiler so it cannot be affected, but a leg that builds studio/src-tauri would
be, and would also need clang on PATH for ring and aws-lc-sys.

* Only ask for a junction where junctions exist

My own regression, caught by the parity job the same commit wired these tests
into: parity (ubuntu-latest) went red on

  FAIL  Junction : a linked root yields the sd.cpp tree under its physical target
  FAIL  Junction : ... and the venv under it

A junction is a Windows-only reparse type, and New-Item -ItemType Junction does
not throw on Linux pwsh. It quietly produces a plain directory with no .Target,
so the SKIP guard -- which only catches a failure -- never fired, and the row ran
and failed every positive assertion. The negative ones passed for the wrong
reason: _ManagedPathsUnderReparseTargets returns nothing for a path with no
target, which is also what "the bare physical target is NOT in scope" wants.

Choose the kinds by platform instead of by catching an exception that never
comes, and treat $IsWindows being undefined as Windows, since that is Windows
PowerShell 5.1 and 5.1 runs nowhere else.

Also verify the link after creating it. Created is not the same as usable: the
helper reads nothing but .Target, so a link without one cannot exercise
anything, and reporting that as a failure blames the helper for the filesystem's
answer. A kind that produces no target is now a SKIP, and the existing check that
at least one kind ran still fails a run that exercised nothing.

Windows unprivileged: Junction all pass, SymbolicLink SKIP (needs elevation),
overall pass. Linux keeps the SymbolicLink row, which passed on that job already.

* Assert the uninstaller's ownership gate on POSIX too

The gate that decides whether ~/.unsloth/studio is recursively deleted exists in
both uninstallers, and when it regressed it regressed in both. Verified on real
Linux (aarch64, bash 5.2.21): against scripts/uninstall.sh before the fix, three
layouts Unsloth itself created were refused --

  FAIL: legacy .venv carrying the owner marker (got foreign, want own)
  FAIL: pre-marker unsloth_studio venv        (got foreign, want own)
  FAIL: pre-marker legacy .venv               (got foreign, want own)

-- and against the current one all thirteen checks pass. So this was never a
Windows problem: a Linux or macOS user with an install predating the
unsloth_studio rename was told their own install was a non-Unsloth path and left
with the tree and studio.db on disk.

There was no POSIX test for it. tests/studio/test_uninstall_legacy_layout_gate.ps1
covers the Windows side; this is its twin, following the extract-via-sed pattern of
test_uninstall_shared_icon.sh so the uninstaller body, which kills processes and
deletes trees, is never executed.

Both directions are asserted, because a gate is only as good as what it refuses.
The negative cases include a venv that merely carries an unrelated console script,
so accepting bin/unsloth cannot quietly become "any venv with anything in it", and
a bare .venv, which is the case the gate was added for and which stays refused
because it is genuinely indistinguishable from a user's own project venv.

Wired into the parity workflow's Linux row. Worth noting how it went unnoticed:
the shell uninstaller tests are named individually in workflows and four of them
are named in none, while consolidated-tests-ci excludes tests/sh outright. A test
nobody runs is indistinguishable from a test that passes.

* Cut the comment volume in the Windows on ARM installer

* Cut the comment volume in the studio setup script

* Cut the comment volume in the python installers and the CI workflows

* Cut the comment volume in the uninstaller and its tests

* Cut the comment volume in the tests, requirements and tauri sources

* Second comment reduction sweep across the Windows on ARM changes

* Third comment reduction sweep over the Windows on ARM changes

* Rebase editable and bare local-directory override lines too

New-UnslothTorchOverridesFile used to write its merge beside the caller's
UV_OVERRIDE file when there was exactly one, so relative references in it kept
resolving. The merge moved to %TEMP% with per-line rebasing, which also fixes
the multi-directory case that was always broken, but Resolve-WoaOverrideLine
did not know -e/--editable or a bare ./pkg directory requirement, so those two
forms started pointing at nothing. Reached every Windows host with UV_OVERRIDE
set, not only Windows on ARM.

* Fold 64 copies of the PowerShell invocation into one helper

Every test that runs a lifted PowerShell function spelled out the same five-line
subprocess.run and the same `assert done.returncode == 0, done.stderr`, 64 times.
_ps and _ps_ok replace them, and the timeout stays a parameter because five sites
need 180 and one needs 60.

369 lines out of the file, no test added or removed: 502 collected before and
after. Mechanically substituted, then checked the tests still catch what they are
for, by breaking install.ps1 three ways and comparing which tests fail. The
failing node ID sets are identical before and after (27 for a Split-WoaTomlKey
that never splits, 8 for a Test-WoaWheelTagsUsable that always says yes), so the
tests still execute PowerShell rather than passing vacuously.

* Test the default wheelhouse URL, which nothing tested

Found by mutation: changing the default to a working but wrong host left all
1492 tests passing. It is load bearing twice over. It is the only source for the
pyarrow that gates the native path, so a typo silently sends every Windows on
ARM host back to the emulated x64 stack with no error anywhere, and it is
fetched over the network, so a wrong host is a wrong download.

Pins the literal, that it is an https resolve/main URL under our own org rather
than a repo page that would serve HTML the staging code saves as a .whl, and
runs the assignment for eight override shapes: unset, empty, a mirror, one and
several trailing slashes, surrounding whitespace, and a local directory whose
trailing backslash TrimEnd('/') must not touch.

The same mutation now fails 3 tests instead of 0.

* Keep windows-vulkan in the ARM64 kind lists main just widened

main added windows-vulkan to every branch of the llama.cpp kind check, because
any x64 Windows host can land on a Vulkan bundle and a guard without it deletes
a working install on every setup run. That reasoning holds for the ARM64 arms
this branch adds, so it goes in $_nvidiaKinds rather than being appended at the
one call site: the opted-out arm needs it as much as the CUDA one.

Test expectations and the source-slice anchors move with it.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Move the uninstaller fixes out to their own PR

The ownership gate, the ~/.unsloth/studio guard, the Split-Path call that throws
and the remove-retry budget are not about Windows on ARM, and two of them change
behaviour on Linux and macOS. Reviewing them as a footnote to a Windows
installer feature was the wrong shape. They are now #10471, against main, with
the parity CI wiring that runs them.

These files go back to main exactly. Also drops this workflow's references to
test_uninstall_legacy_layout_gate.ps1, which now lives in that PR; the runner
loop already skips a test file it cannot find, so it can be listed again once
#10471 lands.

* Move the Kaggle GPU harness fix out to its own PR

An all-[N/A] compute-apps listing means nvidia-smi cannot attribute memory, not
that the card is idle, which is how WDDM and unified-memory parts report. That is
a GPU attribution fix in the test harness and touches no installer code, so it
reviews better on its own: #10472.

These files go back to main exactly.

* Read the two installer scripts once, not 117 times

62 tests re-read install.ps1 and 55 re-read setup.ps1, each on its own line, and
none of them mutate what they get back. Two module constants replace all of it,
and _ps_function serves from them rather than opening the file again on each of
its 60 calls. The wheelhouse-tag tests get the same treatment for
sys.version_info and the cp tag derived from it, restated in nearly every test.

130 lines out, no test added or removed: 1502 collected before and after.

Substituted on NAME tokens through tokenize, never on raw text, so an occurrence
of "text" inside a PowerShell string literal could not be caught by it. Verified
by mutation, not by the suite going green: breaking install.ps1 three ways fails
exactly the same node IDs as before, 3 for the default wheelhouse URL, 27 for a
Split-WoaTomlKey that never splits, 8 for a Test-WoaWheelTagsUsable that always
says yes.

Formatted with scripts/run_ruff_format.py and confirmed idempotent.

* Lower the win_arm64 floors to the first release that actually has a wheel

Three floors were set one release too high, and each one excludes a wheel that
exists.

scikit-learn is the one that bites. 1.8.0 publishes six win_arm64 wheels
including cp313-cp313t; 1.9.0 publishes five and dropped the free-threaded one.
So >=1.9.0 leaves a free-threaded 3.13 with no binary candidate and an sdist to
compile, which is what the ARM64 rows exist to avoid. The comment two lines up
already said "from 1.8 only" and contradicted its own pin.

av first ships win_arm64 at 17.0.0 (cp311-abi3, plus a cp314t), not 17.1.
No release has a 3.13t wheel, so that half of the comment now says so.

bitsandbytes is a comment fix only: the wheel starts at 0.50.0, and the pin was
already below both.

tests/studio/install/test_diffusers_pin.py pins all three against the index
rather than against an audit, and rejects a second ARM64 row for the same
project so a future edit cannot leave two floors disagreeing. It fails on each
of the three old floors.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Translate every spelling of --prerelease for the pip fallback, not one

Remove-UvOnlyResolverFlags handled --prerelease two ways, and both were wrong
away from the single spelling the two call sites use.

The space-separated form dropped the flag and its value and emitted nothing, so
a pip fallback given `--prerelease allow` resolved without prereleases and could
not see the torch build the run had just asked for. uv accepts that form
everywhere it accepts the = form.

The = form emitted --pre for any value. Of uv's five, only allow means what pip's
--pre means: disallow is pip's default and if-necessary, explicit and
if-necessary-or-explicit have no pip flag at all. So --prerelease=disallow became
--pre and inverted the caller's intent.

Both are latent today, since install.ps1 and setup.ps1 both pass
--prerelease=allow, which is the one case the old code got right. The helper
takes a caller-supplied argument list, so the grammar it accepts should be uv's.

The test walks ten argument lists through the real function: both spellings of
both flags, four --prerelease values, a bare value that must not be swallowed by
a flag it does not belong to, and the two flags combined. Four fail on the old
implementation.

* Stop the llama.cpp gate deleting what the selector just installed on ARM64

setup.ps1 deletes an existing llama.cpp tree whose recorded install_kind is not
in a set it computes for the host, then reinstalls via
install_llama_prebuilt.py. Nothing made those two agree, and where they do not
the disagreement is permanent: every update deletes the tree and refetches the
identical bundle, forever, on a machine that is working.

Walking the product of what either side branches on finds two disagreements on
main, both only reachable from Windows ARM64, which is why they lasted:

  arm64 + nvidia  selector installs windows-arm64, gate expected windows-cuda
  arm64 + rocm    selector installs windows-arm64, gate expected windows-rocm

The first is every Windows ARM64 machine with an NVIDIA GPU, and this branch
already fixed it by widening the gate to the ARM64 kinds.

The second is fixed here. No ROCm bundle exists for Windows ARM64 at all:
upstream ships hip-radeon-x64 and we publish none, so the selector falls through
to the ARM64 CPU bundle while the gate went on expecting windows-rocm or
windows-hip. The two ROCm kinds stay in the set for the day one is published.

tests/studio/install/test_llama_kind_gate_matches_selector.py asserts the only
relationship that matters between the two sets, over the twelve host
combinations, rather than the text of either. Offline:
direct_upstream_release_plan takes a release dict, and the gate's own block is
lifted out of setup.ps1 with its inputs replaced. It fails 3 checks against the
gate before this commit.

While confirming this, the x64-fallback churn loop I had noted earlier turned
out not to exist. An emulated x64 Python on an ARM64 box reports AMD64, so that
venv is an x64 host to the selector, it installs windows-cuda, and the gate
expects windows-cuda. All four x64 rows agree, before and after.

* Fold the repeated script and source boilerplate in the WoA persistence tests

Four whole installer and stack files were still being re-read per test; they
are read once now. The 62 copies of the join-a-list-of-lines idiom and the 48
run-then-take-the-last-line pairs become _script and _ps_last, and the three
copies of the parity normaliser become one _ps_copies helper.

* Give the wheelhouse tag tests one cache fixture and one read of each source

The find-links listing was cleared by hand in eleven try/finally blocks and a
dozen bookend pairs; an autouse fixture now empties it before and after every
test, and the clears that sit between two states inside a test stay. install.ps1,
install_python_stack.py and extras.txt are read once, and the wheelhouse builder
becomes a fixture rather than a helper taking three fixtures per call.

* Read each source file once in the marker and guard behaviour tests

install_python_stack.py was exec'd twice in the marker tests and install.ps1 was
re-read and re-scanned for ten helper bodies on every one of the guard tests'
thirty-five pwsh calls.

* Use the hoisted source constants directly rather than aliasing them

Sixteen tests opened with a one line rebinding of a module constant to a shorter
local name; the constant is now named at the point of use.

* Let the prerelease gate test tell producing the flag from comparing to it

Remove-UvOnlyResolverFlags now names --prerelease=allow to translate it into
pip's --pre. That is not a call site adding the flag, it runs after the gate has
already allowed it, so the array-shape rule the gate test enforces does not
apply to it.

* Verify the Windows ARM64 CUDA bundle against GitHub's asset digest

This was the one archive the installer would install unchecked. Every other
prebuilt goes through apply_approved_hashes, which drops an attempt our own
checksum manifest does not cover. The ARM64 CUDA branch could not: we publish no
windows-arm64-cuda artifact yet, so nothing covers upstream's zip, and refusing
outright would mean no CUDA llama.cpp on this hardware at all. So it logged that
it was installing without a hash, and did.

GitHub's release API now reports a per-asset digest. Checked against
ggml-org/llama.cpp b10853: 27 of 27 assets carry one, including
llama-b10853-bin-win-cuda-13.4-arm64.zip and the cudart archive it pairs with.

That is weaker than our manifest, which we compute ourselves rather than read
from the same host that serves the bytes, and the comment says so. It still pins
the download to what the API listed, which is enough for this branch to stop
being the exception. An asset GitHub states no digest for is now refused and the
install falls through to the ARM64 CPU bundle, so no path installs an unverified
archive.

Three things kept deliberately narrow:

  * A separate github_release_asset_digests rather than a second return value
    from github_release_assets. That one is mocked in a dozen suites and used on
    every platform; this is read on one branch.
  * Any failure fetching the digests is an empty mapping, which that branch reads
    as refuse. A rate limit costs the CUDA bundle, not the install, and cannot be
    mistaken for "no digest needed".
  * release_asset_digests requires the sha256: prefix rather than stripping it
    for convenience. normalize_sha256_digest accepts a bare 64-hex string, and an
    unprefixed digest is one whose algorithm GitHub did not state.

tests/studio/install/test_arm64_cuda_digest_verification.py covers the digest
parser over seven rejected shapes, the drop and unpair rules, the fail-closed
fetcher, and asserts against the source that every return in the resolver is
hash gated. That last one fails on the branch before this commit, on the literal
`return upstream_arm64_cuda`.

143 failures before and after across the four llama selection suites, identical
sets, so nothing here changes any other platform's routing.

* Bound the early GPU probes, gate the wheel on the driver, and tighten the resolver and index checks

install.ps1
- Test-WoaNvidiaPresent and the new Get-WoaDriverCudaVersion run nvidia-smi through the bounded runner.
- A CUDA wheel whose major exceeds the driver's falls back to the x64 stack with a driver-update message.
- The av probe is gated on the resolver reaching PyPI; public PyPI supplied as an extra index (env or uv config) keeps the probe in play.
- A managed wheel is redundant only when PyPI has the exact version, and the managed scan lifts win_arm64 or any wheels only.
- On an equal numeric release a final build outranks a .dev one; later stamps win among dev builds.
- The exact CUDA trio pin ignores UV_EXCLUDE_NEWER for that one install, since the index carries no upload dates.
- A pyarrow staging failure after the venv exists is a hard stop with the opt-out named.
- Rebased file: references become file URIs, so spaces are encoded.

studio/setup.ps1
- The merged override file is recorded, removed at the end of the run and on failure, and any stale copy is removed first.
- WinArm64NoAudio follows the effective-index probe; retention pairs torchaudio with the new torch by both predicates.
- Same tie-break, file URI and exclude-newer handling as the installer.

studio/install_python_stack.py, install_manifest.py, install_llama_prebuilt.py
- Extra indexes that are public PyPI count; the resident xformers check runs after a failed refresh.
- The NVIDIA index is persisted with scheme and host normalised.
- Legacy bundle ordering synthesises names for the arch being resolved and passes it through.

.github/workflows/windows-arm64-ci.yml
- The ARM job fails when a required test is reported as skipped.

Tests cover each change, including a real Exit-SetupFailure run under UNSLOTH_TAURI_MODE.

* Make a paired torchvision part of the native gate, and read a supplied wheel's name from its archive

install.ps1
- An index qualifies for the native path only when it publishes a torchvision paired with its torch; otherwise the next candidate is tried, and with none pairing the x64 stack is kept with the reason logged. The vision pin is the version the gate accepted.
- UNSLOTH_PYARROW_WHEEL saved under another name (.bin, no extension) has its wheel name read from the archive before the project and tag checks, as staging already does.

studio/setup.ps1
- When the effective index pairs no torchvision with its newest torch, the fresh-shell probe keeps the installed torch and torchvision pair instead of pinning a torch whose torchvision would resolve against another build.

Tests cover the gate order, the x64 fallback, the kept pair, and archives named .bin or without an extension.

* Rank prereleases in PEP 440 order, probe a reselected interpreter, and put the caller's resolver variables back

install.ps1
- Within one numeric release the wheel pick follows PEP 440 order in full: dev below a below b below rc below the final build, later numbers winning within a kind. Same in setup.ps1's copy.
- After the re-probe flips native mode, the reselected interpreter is probed before it is taken; one with no stack of its own is declined and the accepted interpreter keeps its answer.
- UV_OVERRIDE, UV_FIND_LINKS and PIP_FIND_LINKS are snapshotted before the first assignment and restored in the script-level finally, so an `irm | iex` session does not keep resolving with Studio's overrides and wheelhouse.

studio/backend/plugins/data-designer-unstructured-seed
- The plugin admits pandas 3, which constraints.txt installs on Windows on ARM; it only reads and writes parquet.

Tests cover the ordering, the reselection, the restore, and the plugin range against the constraints.

* Format the three files this branch owns with the ruff the hooks actually pin

.pre-commit-config.yaml pins ruff v0.15.18. I had been running
scripts/run_ruff_format.py against a local ruff 0.6.9, and the two disagree
about where an assert's message goes:

  0.6.9     assert (
                cond
            ), "message"

  0.15.18   assert cond, (
                "message"
            )

So every local format pass wrote the older shape and pre-commit.ci rewrote it on
push. This puts the three files that are ours at the pinned formatter's fixed
point instead.

Verified rather than assumed: with 0.15.18, running the wrapper twice leaves
every file byte-identical, so the wrapper is idempotent and it was only the
version that was wrong.

Four more files still differ under 0.15.18 and are left alone, because they
differ identically on origin/main and so are not this branch's drift:
studio/prebuilt_core.py, tests/studio/install/test_cuda_repair.py,
tests/studio/install/test_rdna1_unsupported_message_8529.py and
tests/studio/install/test_windows_torch_flavor_invariant.py.

855 tests pass across the three.

* Let the dependency index follow the resolver policy, widen the ARM CI filters, and survive an upstream listing failure

install.ps1 and studio/setup.ps1
- torch's shared dependencies resolve from the index the caller's resolver policy names (environment, then uv config), from public PyPI when it names none, and from the find-links wheelhouse alone under no-index. Invoke-InstallCommand clears the inherited index settings whenever --default-index is passed, so the policy is restated on the command line. setup.ps1 carries the readers as parity copies.

.github/workflows/windows-arm64-ci.yml
- The path filters name pyproject.toml and the whole requirements tree the ARM tests read.

studio/install_llama_prebuilt.py
- A failure listing the upstream release assets on a Windows ARM64 NVIDIA host costs the CUDA bundle only; the planner falls through to the published ARM64 CPU bundle.

Tests cover the policy readers in both scripts, the filters, and the fall-through.

* Judge index reachability and the dependency index for the resolver that runs the pass

uv reads UV_* and its configuration files and ignores PIP_*; pip reads PIP_* alone. Reading both sets together reported public PyPI reachable from a PIP_EXTRA_INDEX_URL that uv, the resolver in use, never consults, and then dropped the librosa and tensorboard skip on wheels the resolve could not fetch.

- studio/install_python_stack.py: the probe branches on USE_UV; the uv path also honours UV_NO_INDEX.
- install.ps1: the probe reads uv's policy only, since this script resolves with uv; Get-WoaDependencyIndexArgs takes the resolver and reads that resolver's variables, uv's configuration files for uv alone.
- studio/setup.ps1: the parity copy, with the resolver it will use passed in.

Tests cover the cross-resolver case in all three places and the pip fallback's own policy.

* Keep markers out of rebased file references, clean the merged overrides on every exit, and stop on an unwritable wheel directory

install.ps1 and studio/setup.ps1
- A PEP 508 marker on a `pkg @ file:...` override is kept aside while the path is rebased and re-attached after the URI.
- setup.ps1 traps a terminating error after the resolver restore, removes the merged override file, and rethrows.

install.ps1
- A wheel directory that cannot be created after the ARM64 venv exists is a stop with the reason, not a stand-down into an index with no win_arm64 wheel. The directory is verified after New-Item, which can return without creating it when a file is in the way.
- The trio command names the staged wheelhouse with --find-links, since the inherited UV_FIND_LINKS is cleared for a pinned index; UV_NO_INDEX yields for that one command and is restored after it.
- The two probe warnings that print an index URL redact its credentials.

studio/backend/requirements/no-torch-runtime.txt
- sqlite-vec carries the negative Windows ARM64 marker, as it does in studio.txt.

Tests cover each change; the marker table claims the new row.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make the five Windows-on-ARM tests actually run on Windows

All five assert Windows-on-ARM behaviour and none of them could execute their
own fake on Windows, so they failed on the one platform they are about while
passing on Linux. windows-arm64-ci.yml runs this file on a win-arm64 runner
under both shells and asserts sysconfig.get_platform() first, so these were red,
not skipped.

Test-WoaNvidiaPresent. The fake was an extensionless file containing #!/bin/sh.
Windows has no shebang handling and PATHEXT covers no extensionless name, so the
probe found nothing at all. Now written as nvidia-smi.cmd on Windows, which is
the idiom the rest of this suite already uses, and the sh script elsewhere. The
hung-probe case shared the same fake and so was a false pass: it asserted the
call returns, and a probe that finds no executable returns immediately. It now
actually blocks, via ping rather than timeout, since timeout /t needs a console.

The installed-pair probe. The block runs <VenvDir>/Scripts/python.exe by that
exact name, and the fake was a shell script called python.exe, which Windows
will not execute because it is not a PE. All three cases therefore asserted the
floor rather than the branch they were written for. Replaced with a real venv
carrying real .dist-info metadata, so importlib.metadata answers the way it
would after a real install and the interpreter answers the tag query itself.
Confirmed it now bites: breaking the installed-pair read fails
test_the_installed_pair_is_kept, which it could not do before.

The file: URL row. Its expectation was built by pasting a rebased path after
"file://". A rebased Windows path is C:\opt\corp\ov\dist\a.whl, and "file://"
plus that is not a file URL: the authority would be C: and the separators are
wrong. The rewriter emits file:///C:/opt/corp/ov/dist/a.whl, which is correct,
so the code was right and the expectation was not. Split into its own test that
builds the URL with pathlib as_uri instead of concatenating.

26 pass here on native Windows ARM64 under PowerShell 5.1.

* Add soxr to librosa's unblock set: it has no win_arm64 wheel either

librosa is on the win_arm64 skip list and comes off it once its blockers are
installable. The blocker set was llvmlite and numba, but librosa 0.11.0, which is
what extras.txt pins, requires soxr>=0.3.2 as well, and soxr has published no
win_arm64 wheel in any release up to 1.1.0. Verified against the PyPI JSON API
for the pinned version rather than for latest, since only the pinned release's
metadata is the one that applies.

So hosting cp313 numba and llvmlite would have un-skipped librosa, and the extras
pass would then have gone looking for soxr and found only an sdist. Building that
needs the toolchain the whole skip list exists to avoid. Nobody has hit it
because numba and llvmlite publish win_arm64 for cp314 only, so librosa has never
actually un-skipped on a cp313 host, which is what kept it latent.

soxr also gets a floor entry, read from librosa 0.11.0's own metadata, matching
the provenance convention the other two floors already follow.

openai-whisper is left alone deliberately: its metadata asks for numba and
tiktoken and does not mention soxr.

Upstream is dofuuz/python-soxr PR 36, "Add Windows ARM64 wheels", still open.

The three existing tests that drove librosa off the skip list now stub the index
answer for soxr, so they stay about what they were written for: the interpreter
tag deciding availability, the empty-wheelhouse early return, and losing the
index offline. The new truth gets its own test, which fails when soxr is taken
back out of the tuple.

165 pass.

* Name the wheelhouse on the setup.ps1 trio command, read pip's config files, evict a stale xformers without a hosted wheel for PR #10282

setup.ps1: Fast-Install clears UV_FIND_LINKS and PIP_FIND_LINKS beside the other inherited index settings whenever --index-url is given and leaves UV_NO_INDEX alone, so a native CUDA update under --no-index saw neither the staged wheelhouse nor the CUDA index. $WinArm64IndexArgs now carries --find-links for the wheelhouse when the directory exists, and UV_NO_INDEX yields for the trio command through the existing cutoff save and restore, as install.ps1 already does.

install_python_stack.py: the pip path of the PyPI reachability check read PIP_* only. It now consults `pip config list` when the environment is silent: an `[install]` or `[global]` no-index or exclusive index-url set in pip's files means PyPI is not in play, a PyPI extra-index-url keeps it in play, and a pip config that cannot be read keeps the skip. :env: rows are skipped since the variables were already read.

install_python_stack.py: the resident xformers compatibility check ran only after a hosted install attempt, so a copy an earlier run installed against another torch stayed resident whenever the wheelhouse stopped offering a usable wheel. The check is now a helper that also runs when nothing usable is hosted.

Tests: setup.ps1 twin of the no-index trio class (find-links on the command line, UV_NO_INDEX yield and restore, off-ARM64 untouched), the pip fallback rows carry --find-links through the pip translation, a pip configuration class with stubbed and real `pip config list` output, and the wheelhouse optional rows for eviction without a hosted wheel. Twelve mutants on the three changes are killed.

* Search every supported nvidia-smi location for the driver version

Test-WoaNvidiaPresent searched PATH, System32 and the NVSMI folder, but
Get-WoaDriverCudaVersion searched PATH and System32 only. On a host that carries
nvidia-smi.exe under %ProgramFiles%\NVIDIA Corporation\NVSMI alone, the presence
probe reported a GPU and the version probe returned nothing, so
Initialize-WoaNativeCudaTorch skipped its CUDA-major compatibility guard and could
select a cu13x ARM64 wheel against a driver that only advertises CUDA 12.

Factor the candidate list into Get-WoaNvidiaSmiPath and have both probes call it, so
the two cannot disagree. Same two locations the main GPU detection already searches.

* Inject Get-WoaNvidiaSmiPath into the scripts that paste the probes' real bodies

The pwsh tests compose real function bodies into a bare script. Routing both nvidia-smi
probes through the shared lookup gave those bodies a call the compositions did not
inject, leaving it unresolved.

One of the two would have failed loudly. The other passes for the wrong reason:
PowerShell writes the unresolved-command error and continues with $null, so
Get-WoaDriverCudaVersion returns $null before it ever calls nvidia-smi and the '[]'
assertion still holds, with the timeout the test exists to bound never exercised.

Both are marked requires_pwsh, so on a host without pwsh they skip and the suite is
green either way. Add a guard that needs no pwsh: it fails if a composition injects a
body calling Get-WoaNvidiaSmiPath without also injecting the helper.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Read uv's inline index spelling, and stop guessing when it cannot be read

Read-WoaUvTomlIndexKeys returned $null on sight of an `index` key, so
`index = [{ url = "...", default = true }]` -- valid, documented uv config -- read as
Unreadable. Get-WoaDependencyIndexArgs never looked at Unreadable and fell through to
its https://pypi.org/simple default, silently replacing whatever mirror the file
configured. Test-WoaResolveReachesPyPI had honoured the same flag all along, so the two
consumers of one signal disagreed, and the unsafe one built the install command.

Parse the inline form, conservatively: a single-line, brace-balanced array of flat
inline tables only. A multi-line array, a nested value, a bare entry beside a table, a
missing url, a non-boolean default or an explicit index all still return $null and stay
Unreadable, because guessing is the only outcome worse than not parsing.

Then make the consumers agree, per call site, since they genuinely differ. With no env
default and an unreadable policy, name no dependency index rather than substituting
PyPI. install.ps1's trio step calls uv directly, so uv reads the file itself and the
empty answer is strictly better than today: no substitution and no failure.
setup.ps1's runs under Fast-Install with --index-url, which sets UV_NO_CONFIG and
scrubs UV_*, so uv cannot read it and an empty answer would leave the trio index as the
only source; that one stops with a message naming the file, the spelling, and both ways
out. Get-WoaUvConfigIndexPolicy now records which file it could not read so the message
can name it.

Blast radius, traced before writing: a previously-$null read now returns a value, which
moves Test-WoaResolveReachesPyPI and therefore wheel pruning. A mirror URL still gives
false and still disables pruning, so that case is unmoved. A genuine pypi.org URL now
gives true and enables it, which is correct and was previously suppressed by a parse
failure rather than by a decision.

Verified against pwsh 7.4.6 rather than skipped: the inline reader is exercised on five
accepted shapes and six ambiguous ones. Running them for real caught a bare string entry
beside a table being accepted, because the scanner entered string mode before testing
depth. Full tests/studio/install plus the wheelhouse tags suite gives an identical
failure set to HEAD, 173 either side, none new.

* Apply the pinned formatter's kwarg spacing to the new inline-index cases

The ruff-format-with-kwargs hook pins ruff==0.6.9, which wraps the new parametrize
tuples one element per line. Formatting only; the 733 tests in the two files are
unchanged and still pass, and a second run of the hook is a no-op.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Default the Windows on ARM wheelhouse to the published one for PR #10282

The default named unsloth/windows-arm64-wheels, which does not exist, so no host took the native path without UNSLOTH_WOA_WHEELHOUSE set. The wheelhouse now lives under danielhanchen/unsloth-blackwell-docker: pyarrow 25.0.1 for cp311 through cp314, built from the Arrow release tag with apache/arrow#48539's build script on a windows-11-arm runner, and sqlite-vec 0.1.9 repacked with an ARM64 DLL built from its release tag. Every wheel was smoke tested on the ARM64 runner, every native image inside is ARM64, and install.ps1 ran its native path against the published URL end to end (pyarrow 25.0.1, torch 2.14.0+cu134, sqlite_vec 0.1.9 all importing in the win-arm64 venv). The test that pins the URL follows.

* Share the PowerShell test harness and parametrize the Windows on ARM persistence tests for PR #10282

* Shorten the comments the Windows on ARM installer added for PR #10282

* Parametrize and share fixtures across the Windows on ARM wheelhouse tests for PR #10282

* Keep explicit uv indexes out of the dependency index, prune wheels an earlier wheelhouse left for PR #10282

Explicit indexes: uv serves an index marked explicit = true only to packages pinned to it via [tool.uv.sources], so it is neither the default nor an extra for the trio's shared dependencies. The block form [[index]] reader in install.ps1 and setup.ps1 read url and default only and flushed such an entry as a general extra, which Get-WoaDependencyIndexArgs then passed as --extra-index-url to the native torch resolve; the inline table reader refused the key outright, which turned the documented PyTorch pattern into an unreadable policy. Both readers and the Python reader now skip an explicit entry; explicit together with default = true also removes PyPI as the default, which is not modelled, so that combination stays doubt.

Stale wheels: the managed woa\wheels directory persists across runs, so after UNSLOTH_WOA_WHEELHOUSE changed a wheel the earlier wheelhouse staged stayed in it, the hosted scan read it as current and UV_FIND_LINKS installed it from a source no longer configured. Staging now reconciles the directory against the wheelhouse listing it just read, keeping the listed wheels and the selected pyarrow wheel; an unreadable listing (offline) keeps the staged copies, and the directory used as its own wheelhouse lists everything it holds.

Tests: block-form and inline rows on both scripts plus the Python reader, the dependency args never naming an explicit index, and the reconciliation across a local wheelhouse, a URL wheelhouse, an offline listing and the self-sourced directory. Ten mutants on the change are killed.

* Shorten the comments the explicit-index and wheelhouse reconciliation changes added for PR #10282

* Hand PyPI-provided wheels to the skip list, install a hosted torchcodec, order xformers after the final torch repair, and key the pip refusal to the generated overrides for PR #10282

* Keep folded caller overrides in a per-run file rather than the persistent WoA overrides for PR #10282

* Backtrack to the newest paired torch and keep inline comments when rebasing override lines for PR #10282

* Form the rebase test's expected paths per host and let the ARM64 CI print its pytest output for PR #10282

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add a signed Windows on ARM wheelhouse workflow that builds, Authenticode signs and publishes the pyarrow and sqlite-vec win_arm64 wheels for PR #10282

* Publish the Windows on ARM wheelhouse under the Windows-ARM64 release tag for PR #10282

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
This commit is contained in:
Daniel Han 2026-09-16 04:32:53 -07:00 • committed by GitHub
parent f621d12a21
commit b535eca250
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
40 changed files with 15085 additions and 239 deletions

44
.github/scripts/repack_sqlite_vec.py vendored Normal file
View file

@ -0,0 +1,44 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Repack the PyPI sqlite-vec win_amd64 wheel with an ARM64 vec0.dll under the win_arm64 tag.
The wheel is pure Python plus one DLL, generated upstream by sqlite-dist: replacing the DLL,
the WHEEL tag and RECORD is the whole port. Usage: repack_sqlite_vec.py <amd64 wheel> <vec0.dll> <out dir>
"""
import base64, hashlib, struct, sys, zipfile
from pathlib import Path
src, dll, out = Path(sys.argv[1]), Path(sys.argv[2]), Path(sys.argv[3])
assert src.name.endswith("-py3-none-win_amd64.whl"), src.name
dst = out / src.name.replace("-win_amd64.whl", "-win_arm64.whl")
data = dll.read_bytes()
e_lfanew = struct.unpack_from("<I", data, 0x3C)[0]
assert data[e_lfanew : e_lfanew + 4] == b"PE\0\0", "not a PE file"
machine = struct.unpack_from("<H", data, e_lfanew + 4)[0]
assert machine == 0xAA64, f"vec0.dll machine is {machine:#x}, not ARM64 (0xaa64)"
def record_line(name: str, blob: bytes) -> str:
digest = base64.urlsafe_b64encode(hashlib.sha256(blob).digest()).rstrip(b"=").decode()
return f"{name},sha256={digest},{len(blob)}"
out.mkdir(parents = True, exist_ok = True)
with zipfile.ZipFile(src) as zin, zipfile.ZipFile(dst, "w", zipfile.ZIP_DEFLATED) as zout:
names = zin.namelist()
record_name = next(n for n in names if n.endswith(".dist-info/RECORD"))
lines = []
for name in names:
if name == record_name:
continue
blob = zin.read(name)
if name.endswith("/vec0.dll"):
blob = data
elif name.endswith(".dist-info/WHEEL"):
text = blob.decode()
assert "Tag: py3-none-win_amd64" in text, text
blob = text.replace("Tag: py3-none-win_amd64", "Tag: py3-none-win_arm64").encode()
zout.writestr(name, blob)
lines.append(record_line(name, blob))
lines.append(f"{record_name},,")
zout.writestr(record_name, "\n".join(lines) + "\n")
print(dst.name, dst.stat().st_size, "bytes")

205
.github/scripts/sign_wheel_binaries.py vendored Normal file
View file

@ -0,0 +1,205 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Authenticode-sign the native binaries inside a wheel and repack it.
A .whl cannot itself be Authenticode signed: it is a zip, and Windows has no subject
interface package for one. What CAN be signed are the PE images inside it -- the .pyd
extension modules and the .dll files they link -- which is what Windows actually loads and
what SmartScreen and an enterprise WDAC policy judge. So this unpacks the wheel, signs each
PE we built, and packs it back under the same filename.
RECORD is regenerated, because it has to be: signing appends a certificate table, so every
signed member's sha256 and length change, and a wheel whose RECORD disagrees with its
contents is one `pip check`, `uv pip install --strict` or reproducibility audit away from
being called corrupt.
Files that already carry a certificate are left alone. delvewheel vendors Microsoft's own
redistributables (msvcp140.dll and friends) into pyarrow, already signed by Microsoft;
re-signing those would replace their signature with ours and claim authorship of somebody
else's binary.
The signing itself is delegated, never reimplemented: --signer names the command, and CI
passes studio/src-tauri/windows/sign-with-trusted-signing.ps1, the same helper the desktop
release uses, so the endpoint, the identity and the Azure retry policy have one definition.
"""
from __future__ import annotations
import argparse
import base64
import hashlib
import shutil
import struct
import subprocess
import sys
import tempfile
import zipfile
from pathlib import Path
#: IMAGE_FILE_MACHINE_ARM64. The wheels this signs are win_arm64.
MACHINE_ARM64 = 0xAA64
#: Data directory index 4 is the Certificate Table -- a non-zero size means "already signed".
CERTIFICATE_TABLE_INDEX = 4
def _pe_header_offset(blob: bytes) -> "int | None":
"""Offset of the PE signature, or None when this is not a PE image at all."""
if len(blob) < 0x40 or blob[:2] != b"MZ":
return None
try:
offset = struct.unpack_from("<I", blob, 0x3C)[0]
except struct.error:
return None
if offset <= 0 or offset + 24 > len(blob) or blob[offset : offset + 4] != b"PE\0\0":
return None
return offset
def pe_machine(blob: bytes) -> "int | None":
"""The COFF machine type, or None when the bytes are not a PE image."""
offset = _pe_header_offset(blob)
if offset is None:
return None
return struct.unpack_from("<H", blob, offset + 4)[0]
def is_signed(blob: bytes) -> bool:
"""Does this PE image already carry a certificate table?
Read from the optional header's data directory rather than by running a tool, so the
answer does not depend on a Windows host or on the file being unpacked first.
"""
offset = _pe_header_offset(blob)
if offset is None:
return False
magic_at = offset + 24
if magic_at + 2 > len(blob):
return False
magic = struct.unpack_from("<H", blob, magic_at)[0]
# PE32+ has 8 more bytes of fixed fields before the data directory than PE32 does.
directory_at = magic_at + (112 if magic == 0x20B else 96) + CERTIFICATE_TABLE_INDEX * 8
if directory_at + 8 > len(blob):
return False
_, size = struct.unpack_from("<II", blob, directory_at)
return size > 0
def record_line(name: str, blob: bytes) -> str:
"""One RECORD row: urlsafe base64 sha256 with the padding stripped, per PEP 376."""
digest = base64.urlsafe_b64encode(hashlib.sha256(blob).digest()).rstrip(b"=").decode()
return f"{name},sha256={digest},{len(blob)}"
def sign_file(path: Path, signer: "list[str]") -> None:
"""Run the signing command over one file, failing the build if it cannot sign."""
result = subprocess.run([*signer, str(path)], text = True)
if result.returncode != 0:
raise SystemExit(f"signing failed for {path.name} (exit {result.returncode})")
def sign_wheel(
wheel: Path,
out_dir: Path,
signer: "list[str]",
*,
require_machine: "int | None" = MACHINE_ARM64,
) -> "tuple[Path, list[str], list[str]]":
"""Sign every unsigned PE inside `wheel` and write the result into `out_dir`.
Returns (written wheel, names signed, names skipped as already signed).
"""
out_dir.mkdir(parents = True, exist_ok = True)
destination = out_dir / wheel.name
signed: list[str] = []
skipped: list[str] = []
with tempfile.TemporaryDirectory() as scratch:
work = Path(scratch)
with zipfile.ZipFile(wheel) as archive:
names = archive.namelist()
payload = {name: archive.read(name) for name in names}
for name in names:
blob = payload[name]
machine = pe_machine(blob)
if machine is None:
continue
if require_machine is not None and machine != require_machine:
raise SystemExit(
f"{wheel.name}: {name} is machine {machine:#x}, not {require_machine:#x}"
)
if is_signed(blob):
skipped.append(name)
continue
# Signed on disk under its own name: signtool and trusted-signing-cli both work
# in place, and the suffix decides which subject interface package is used.
staged = work / Path(name).name
staged.write_bytes(blob)
sign_file(staged, signer)
resigned = staged.read_bytes()
if not is_signed(resigned):
raise SystemExit(f"{wheel.name}: {name} has no certificate after signing")
payload[name] = resigned
signed.append(name)
staged.unlink()
if not signed and not skipped:
raise SystemExit(f"{wheel.name}: no PE images found -- nothing to sign")
record_name = next((n for n in names if n.endswith(".dist-info/RECORD")), None)
if record_name is None:
raise SystemExit(f"{wheel.name}: no .dist-info/RECORD")
# Member order preserved: the installer does not care, but a diff between the signed
# and unsigned wheel should show the signatures and nothing else.
with zipfile.ZipFile(destination, "w", zipfile.ZIP_DEFLATED) as out:
lines = []
for name in names:
if name == record_name:
continue
out.writestr(name, payload[name])
lines.append(record_line(name, payload[name]))
lines.append(f"{record_name},,")
out.writestr(record_name, "\n".join(lines) + "\n")
return destination, signed, skipped
def main(argv: "list[str] | None" = None) -> int:
parser = argparse.ArgumentParser(description = __doc__)
parser.add_argument("wheels", nargs = "+", type = Path)
parser.add_argument("--out", required = True, type = Path)
parser.add_argument(
"--signer",
required = True,
help = "signing command; the file to sign is appended as the last argument",
)
parser.add_argument(
"--expect-machine",
default = "arm64",
choices = ("arm64", "any"),
help = "refuse a wheel carrying a PE for another architecture (default: arm64)",
)
args = parser.parse_args(argv)
import shlex
signer = shlex.split(args.signer, posix = False)
require = MACHINE_ARM64 if args.expect_machine == "arm64" else None
failures = 0
for wheel in args.wheels:
written, signed, skipped = sign_wheel(wheel, args.out, signer, require_machine = require)
print(f"{written.name}: signed {len(signed)}, already signed {len(skipped)}")
for name in signed:
print(f" signed {name}")
for name in skipped:
print(f" kept {name} (already carries a certificate)")
if not signed:
print(f"::warning::{wheel.name} had nothing to sign")
failures += 1
return 1 if failures and len(args.wheels) == failures else 0
if __name__ == "__main__":
sys.exit(main())

167
.github/workflows/windows-arm64-ci.yml vendored Normal file
View file

@ -0,0 +1,167 @@
# The native Windows on ARM install path, run on Windows on ARM: wheel-tag arithmetic, path handling and
# PowerShell parsing, none of which fails the same way on x64. No GPU, CUDA or compiler, so a full install or a Tauri build is out of scope (ring and aws-lc-sys need clang for their aarch64 sources).
name: Windows ARM64 CI
on:
push:
branches: [main]
paths:
- 'install.ps1'
- 'scripts/uninstall.ps1'
- 'scripts/uninstall.sh'
- 'studio/install_python_stack.py'
- 'studio/setup.ps1'
- 'pyproject.toml'
- 'studio/backend/requirements/**'
- 'tests/studio/install/test_woa_wheelhouse_prefers_pypi.py'
- 'tests/studio/install/test_woa_wheelhouse_guard_behaviour.py'
- 'tests/studio/install/test_woa_torch_index_persistence.py'
- 'tests/studio/install/test_diffusers_pin.py'
- 'tests/python/test_windows_arm64_wheelhouse_tags.py'
- 'tests/studio/test_uninstall_arg_guard.ps1'
- 'tests/studio/test_uninstall_dual_install_icon.ps1'
- 'tests/studio/test_uninstall_prebuilt_parity.ps1'
- 'tests/studio/test_uninstall_reparse_stop_roots.ps1'
- '.github/workflows/windows-arm64-ci.yml'
pull_request:
paths:
- 'install.ps1'
- 'scripts/uninstall.ps1'
- 'scripts/uninstall.sh'
- 'studio/install_python_stack.py'
- 'studio/setup.ps1'
- 'pyproject.toml'
- 'studio/backend/requirements/**'
- 'tests/studio/install/test_woa_wheelhouse_prefers_pypi.py'
- 'tests/studio/install/test_woa_wheelhouse_guard_behaviour.py'
- 'tests/studio/install/test_woa_torch_index_persistence.py'
- 'tests/studio/install/test_diffusers_pin.py'
- 'tests/python/test_windows_arm64_wheelhouse_tags.py'
- 'tests/studio/test_uninstall_arg_guard.ps1'
- 'tests/studio/test_uninstall_dual_install_icon.ps1'
- 'tests/studio/test_uninstall_prebuilt_parity.ps1'
- 'tests/studio/test_uninstall_reparse_stop_roots.ps1'
- '.github/workflows/windows-arm64-ci.yml'
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.ref == 'refs/heads/main' && github.sha || '' }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
permissions:
contents: read
jobs:
woa:
# Free for public repositories since 2025-08-07; a fork on a private plan fails to start.
runs-on: windows-11-arm
timeout-minutes: 40
strategy:
fail-fast: false
matrix:
# Both shells: pwsh 7 is what CI has always run, and Windows PowerShell 5.1 is the only shell on a stock Windows box, which the CLI launches setup.ps1 with (see #8335).
powershell: [pwsh-7, windows-powershell-5.1]
name: woa (${{ matrix.powershell }})
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: The runner really is ARM64
shell: pwsh
run: |
$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture
$proc = [System.Runtime.InteropServices.RuntimeInformation]::ProcessArchitecture
# Machine scope: an x64-emulated process reports AMD64 in its own environment block.
$machine = [Environment]::GetEnvironmentVariable(
"PROCESSOR_ARCHITECTURE", [EnvironmentVariableTarget]::Machine)
Write-Host "OSArchitecture=$arch ProcessArchitecture=$proc machine=$machine"
if ("$arch" -ne "Arm64") {
Write-Host "::error::this job is not running on ARM64 (OSArchitecture=$arch)"
exit 1
}
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
# Required: the image ships an emulated x64 Python whose sysconfig says win-amd64.
architecture: 'arm64'
- name: The interpreter really is ARM64
shell: pwsh
run: |
python -c "import platform, sysconfig, sys; print(sys.version); print('machine', platform.machine()); print('platform', sysconfig.get_platform())"
python -c "import sysconfig, sys; p = sysconfig.get_platform(); sys.exit(0 if p == 'win-arm64' else (print(f'::error::sysconfig reports {p}, expected win-arm64') or 1))"
- run: python -m pip install -U pip pytest packaging
# 5.1 has no `pwsh` on PATH and the suites locate their shell with shutil.which("pwsh"), so without this they SKIP and report green. A copy, because CreateProcess will not run a .cmd.
- name: Put Windows PowerShell 5.1 on PATH as pwsh
if: matrix.powershell == 'windows-powershell-5.1'
shell: pwsh
run: |
$dir = Join-Path $env:RUNNER_TEMP "wps51"
New-Item -ItemType Directory -Path $dir -Force | Out-Null
$src = Join-Path $env:SystemRoot "System32\WindowsPowerShell\v1.0\powershell.exe"
if (-not (Test-Path -LiteralPath $src)) {
Write-Host "::error::Windows PowerShell 5.1 was not found at $src"
exit 1
}
Copy-Item -LiteralPath $src -Destination (Join-Path $dir "pwsh.exe") -Force
"$dir" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- name: Which PowerShell the suites will use
shell: pwsh
run: |
$exe = (Get-Command pwsh).Source
$ver = & $exe -NoProfile -Command '$PSVersionTable.PSVersion.ToString()'
Write-Host "pwsh resolves to $exe (version $ver)"
$wanted = if ("${{ matrix.powershell }}" -eq "pwsh-7") { "7" } else { "5" }
if (-not $ver.StartsWith($wanted)) {
Write-Host "::error::expected PowerShell $wanted.x for this row, got $ver"
exit 1
}
# A skip is a failure here: these files exist for this row, and -rs only REPORTS skipped cases, so the summary line is read back and any skipped count fails the step.
- name: WoA installer and wheelhouse tests
shell: pwsh
env:
UNSLOTH_NO_TORCH: '1'
run: |
& python -m pytest `
tests/studio/install/test_woa_wheelhouse_prefers_pypi.py `
tests/studio/install/test_woa_wheelhouse_guard_behaviour.py `
tests/studio/install/test_woa_torch_index_persistence.py `
tests/studio/install/test_diffusers_pin.py `
tests/python/test_windows_arm64_wheelhouse_tags.py `
-q -rs 2>&1 | Tee-Object -Variable pytestOut
$rc = $LASTEXITCODE
$summary = ($pytestOut | Where-Object { $_ -match '\d+ (passed|failed|error)' } | Select-Object -Last 1)
if ($rc -ne 0) { exit $rc }
if ($summary -match '(\d+) skipped') {
Write-Host "::error::$($Matches[1]) test(s) skipped on the row they exist for: $summary"
exit 1
}
- name: Uninstaller tests
shell: pwsh
run: |
$failed = @()
foreach ($t in @(
"test_uninstall_arg_guard",
"test_uninstall_dual_install_icon",
"test_uninstall_prebuilt_parity",
"test_uninstall_reparse_stop_roots"
)) {
$path = "tests/studio/$t.ps1"
if (-not (Test-Path -LiteralPath $path)) { continue }
Write-Host "::group::$t"
& (Get-Command pwsh).Source -NoProfile -File $path
$code = $LASTEXITCODE
Write-Host "::endgroup::"
if ($code -ne 0) { $failed += "$t (exit $code)" }
}
if ($failed.Count -gt 0) {
Write-Host "::error::failing uninstaller tests: $($failed -join ', ')"
exit 1
}

634
.github/workflows/woa-wheelhouse.yml vendored Normal file
View file

@ -0,0 +1,634 @@
# The Windows on ARM wheelhouse: build, SIGN and publish the two win_arm64 wheels the
# native ARM64 installer cannot get anywhere else.
#
# pyarrow -- mandatory. `datasets` imports it, and PyPI has published no win_arm64 wheel
# in any release (apache/arrow#48539 adds the CI support and is still open).
# Its sdist needs Arrow C++, vcpkg and the MSVC ARM64 toolchain, which is
# ~45 min on this runner and flatly out of reach on a user's machine.
# sqlite-vec -- optional (RAG). PyPI ships one py3-none-win_amd64 wheel: pure Python plus a
# single x64 vec0.dll. The port is that DLL rebuilt for ARM64.
#
# Signed, because these are the only binaries in the stack we produce ourselves. Everything
# else on that install comes from PyPI or from NVIDIA. An unsigned .pyd we host and tell the
# installer to fetch is the weakest link in the chain, and Authenticode is what Windows,
# SmartScreen and a WDAC policy actually check. The procedure is release-desktop.yml's,
# unchanged: the digest-pinned trusted-signing-cli, sign-with-trusted-signing.ps1, and the
# release-signing environment gating the Azure secrets.
#
# A .whl cannot be Authenticode signed (a zip has no subject interface package), so what is
# signed is every PE inside it; sign_wheel_binaries.py repacks and regenerates RECORD.
#
# Dispatch only. This holds signing credentials and writes a release, so it must never be
# reachable from a pull request, and `publish: false` is the default: a run then builds,
# signs and verifies, and leaves the wheels as artifacts to inspect.
name: Windows on ARM wheelhouse
on:
workflow_dispatch:
inputs:
arrow_tag:
description: "apache/arrow tag to build pyarrow from"
default: "apache-arrow-25.0.1"
sqlite_vec_tag:
description: "asg017/sqlite-vec tag"
default: "v0.1.9"
release_tag:
description: "release tag to publish under"
default: "Windows-ARM64"
publish:
description: "upload to the release (otherwise artifacts only)"
type: boolean
default: false
permissions:
contents: read
concurrency:
group: woa-wheelhouse-${{ github.ref }}
cancel-in-progress: false
env:
# Windows ARM64 support in Arrow's wheel build is CI-only and still unmerged, so it is
# overlaid onto the release tag. Both halves move together or not at all.
ARROW_WOA_REPO: MugundanMCW/arrow
ARROW_WOA_REF: Pyarrow_WoA
jobs:
build-pyarrow:
name: pyarrow win_arm64 / cp${{ matrix.python }}
runs-on: windows-11-arm
timeout-minutes: 300
strategy:
fail-fast: false
matrix:
python: ['3.11', '3.12', '3.13', '3.14']
env:
CMAKE_BUILD_TYPE: release
ARCH: ARM64
steps:
- name: Harden runner (audit)
uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with:
egress-policy: audit
- uses: actions/checkout@v4
with:
repository: apache/arrow
ref: ${{ inputs.arrow_tag }}
path: arrow
- uses: actions/checkout@v4
with:
repository: ${{ env.ARROW_WOA_REPO }}
ref: ${{ env.ARROW_WOA_REF }}
path: arrow-woa
sparse-checkout: |
ci/scripts
ci/vcpkg
sparse-checkout-cone-mode: false
# Every edit below asserts on the text it replaces, so an upstream rename fails this
# step rather than silently producing an x64 wheel with an arm64 filename.
- name: Overlay the ARM64 build support onto the release tag
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
Copy-Item arrow-woa/ci/scripts/python_wheel_windows_build.bat arrow/ci/scripts/ -Force
Copy-Item arrow-woa/ci/scripts/python_wheel_windows_test.bat arrow/ci/scripts/ -Force
$json = 'arrow/ci/vcpkg/vcpkg.json'
$t = Get-Content $json -Raw
$edits = @(
@('"supports": "x64 | (arm64 & !windows)"', '"supports": "x64 | arm64"'),
@(" `"xsimd`",", " {`n `"name`": `"xsimd`",`n `"platform`": `"!(arm64 & windows)`"`n },")
)
foreach ($e in $edits) {
if (-not $t.Contains($e[0])) { throw "vcpkg.json no longer contains: $($e[0])" }
$t = $t.Replace($e[0], $e[1])
}
Set-Content $json -Value $t -Encoding ascii -NoNewline
python -c "import json; json.load(open('$json'))"
# The upstream script hardcodes VS 2022 Enterprise and its generator. This image
# moves to VS 2026, so both come from the installed toolset instead.
$vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
$vs = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.ARM64 -property installationPath
if (-not $vs) { throw 'no Visual Studio with the ARM64 C++ tools' }
$major = [int]((& $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.ARM64 -property installationVersion) -split '\.')[0]
$generator = switch ($major) { 17 { 'Visual Studio 17 2022' } 18 { 'Visual Studio 18 2026' } default { throw "unknown Visual Studio major $major" } }
$bat = 'arrow/ci/scripts/python_wheel_windows_build.bat'
$b = Get-Content $bat -Raw
$edits = @(
@('set VCVARS_BAT=C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsarm64.bat', "set VCVARS_BAT=$vs\VC\Auxiliary\Build\vcvarsarm64.bat"),
@('set CMAKE_GENERATOR=Visual Studio 17 2022', "set CMAKE_GENERATOR=$generator")
)
foreach ($e in $edits) {
if (-not $b.Contains($e[0])) { throw "build.bat no longer contains: $($e[0])" }
$b = $b.Replace($e[0], $e[1])
}
Set-Content $bat -Value $b -Encoding ascii
if (-not (Test-Path "$vs\VC\Auxiliary\Build\vcvarsarm64.bat")) { throw "no vcvarsarm64.bat under $vs" }
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
architecture: arm64
- name: Python build dependencies
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
python -c "import sys, sysconfig; print(sys.version, sysconfig.get_platform())"
python -m pip install --upgrade pip
python -m pip install -r arrow/python/requirements-wheel-build.txt
"PYTHON_CMD=$env:pythonLocation\python.exe" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
- name: Version from the C++ tree
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$cm = Get-Content arrow/cpp/CMakeLists.txt -Raw
if ($cm -notmatch 'set\(ARROW_VERSION "([0-9.]+)(-SNAPSHOT)?"\)') { throw 'no ARROW_VERSION' }
$v = $Matches[1]; if ($Matches[2]) { $v = "$v.dev${{ github.run_number }}" }
Write-Host "pyarrow version: $v"
"SETUPTOOLS_SCM_PRETEND_VERSION=$v" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# Trimmed to what `datasets` reaches: this removes gRPC and the cloud SDKs from the
# vcpkg build entirely, which is most of the wall clock and most of the attack surface.
- name: Trim the build to what datasets needs
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$bat = 'arrow/ci/scripts/python_wheel_windows_build.bat'
$t = Get-Content $bat -Raw
foreach ($flag in 'ARROW_AZURE','ARROW_FLIGHT','ARROW_GCS','ARROW_HDFS','ARROW_ORC','ARROW_S3','ARROW_SUBSTRAIT','ARROW_TENSORFLOW','ARROW_WITH_OPENTELEMETRY') {
$before = $t
$t = $t -replace "set $flag=ON", "set $flag=OFF"
if ($t -eq $before) { throw "did not find 'set $flag=ON'" }
}
Set-Content $bat -Value $t -Encoding ascii
Select-String -Path $bat -Pattern '^set ARROW_' | ForEach-Object { $_.Line }
- name: Cache vcpkg packages
uses: actions/cache@v4
with:
path: C:\vcpkg\installed
key: vcpkg-arm64-static-md-trimmed-${{ inputs.arrow_tag }}-${{ hashFiles('arrow/ci/vcpkg/vcpkg.json') }}
restore-keys: |
vcpkg-arm64-static-md-trimmed-${{ inputs.arrow_tag }}-
vcpkg-arm64-static-md-trimmed-
# Retried: the ports fetch source tarballs from archive.apache.org and friends, and one
# timed-out download fails the whole step. Packages already built survive an attempt.
- name: vcpkg dependencies (base + json + parquet)
shell: cmd
run: |
cd arrow\ci\vcpkg
set attempt=0
:again
set /a attempt+=1
C:\vcpkg\vcpkg.exe install ^
--triplet arm64-windows-static-md ^
--x-install-root=C:\vcpkg\installed ^
--x-manifest-root=. ^
--x-feature=json ^
--x-feature=parquet && exit /B 0
if %attempt% GEQ 3 exit /B 1
echo vcpkg install failed, attempt %attempt% of 3; retrying in 60s
timeout /t 60 /nobreak >nul
goto again
- name: Build Arrow C++ and the pyarrow wheel
shell: cmd
run: |
call "arrow\ci\scripts\python_wheel_windows_build.bat"
# A win_arm64 filename says nothing about what was linked in, so every native image is
# opened and its COFF machine checked before anything downstream trusts the wheel.
- name: Verify the wheel is really ARM64
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$whl = Get-ChildItem arrow/python/repaired_wheels/*.whl | Select-Object -First 1
Write-Host "wheel: $($whl.Name) ($([math]::Round($whl.Length/1MB)) MB)"
if ($whl.Name -notmatch '-win_arm64\.whl$') { throw "not a win_arm64 wheel: $($whl.Name)" }
python -c @"
import struct, sys, zipfile
z = zipfile.ZipFile(sys.argv[1]); bad = []; n = 0
for name in z.namelist():
if not name.lower().endswith(('.pyd', '.dll')): continue
d = z.read(name); off = struct.unpack_from('<I', d, 0x3C)[0]
machine = struct.unpack_from('<H', d, off + 4)[0]; n += 1
if machine != 0xAA64: bad.append((name, hex(machine)))
print(n, 'native images, all ARM64' if not bad else bad)
sys.exit(1 if bad or n == 0 else 0)
"@ $whl.FullName
if ($LASTEXITCODE -ne 0) { exit 1 }
- name: Smoke test in a fresh venv
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$whl = Get-ChildItem arrow/python/repaired_wheels/*.whl | Select-Object -First 1
python -m venv smoke
smoke\Scripts\python.exe -m pip install --quiet $whl.FullName numpy
smoke\Scripts\python.exe -c @"
import pyarrow as pa, pyarrow.parquet as pq, pyarrow.dataset as ds, pyarrow.compute as pc, pyarrow.csv, pyarrow.json, sysconfig
print('pyarrow', pa.__version__, sysconfig.get_platform())
t = pa.table({'a': [1, 2, 3], 'b': ['x', 'y', 'z']})
pq.write_table(t, 'smoke.parquet')
back = pq.read_table('smoke.parquet')
assert back.equals(t), back
print('sum a =', pc.sum(back['a']).as_py())
print('dataset rows =', ds.dataset('smoke.parquet').count_rows())
print('compression:', [c for c in ('snappy','zstd','lz4','brotli','gzip','bz2') if pa.Codec.is_available(c)])
"@
if ($LASTEXITCODE -ne 0) { exit 1 }
# The reason the wheel exists at all, so it is a gate and not a nice-to-have.
- name: Does datasets work on it
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
smoke\Scripts\python.exe -m pip install --quiet datasets
smoke\Scripts\python.exe -c @"
from datasets import Dataset
d = Dataset.from_dict({'text': ['hello', 'world'], 'n': [1, 2]})
d = d.map(lambda r: {'len': len(r['text'])})
d.to_parquet('ds.parquet')
print('datasets round-trip ok:', Dataset.from_parquet('ds.parquet').to_dict())
"@
if ($LASTEXITCODE -ne 0) { exit 1 }
- uses: actions/upload-artifact@v4
with:
name: unsigned-pyarrow-cp${{ matrix.python }}
path: arrow/python/repaired_wheels/*.whl
if-no-files-found: error
build-sqlite-vec:
name: sqlite-vec win_arm64
runs-on: windows-11-arm
timeout-minutes: 30
steps:
- name: Harden runner (audit)
uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with:
egress-policy: audit
- uses: actions/checkout@v4
with:
repository: asg017/sqlite-vec
ref: ${{ inputs.sqlite_vec_tag }}
path: sv
fetch-depth: 0
- uses: actions/checkout@v4
with:
path: tools
sparse-checkout: |
.github/scripts
sparse-checkout-cone-mode: false
- uses: actions/setup-python@v5
with:
python-version: '3.13'
architecture: arm64
# scripts/vendor.sh without the unzip dependency. The amalgamation URL is read from
# that script rather than hardcoded here, so the tag decides which sqlite is vendored.
- name: Vendor the sqlite amalgamation
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$vendor = Get-Content sv/scripts/vendor.sh -Raw
if ($vendor -notmatch '(https://www\.sqlite\.org/\d{4}/sqlite-amalgamation-\d+\.zip)') {
throw "could not read the amalgamation URL out of scripts/vendor.sh"
}
$url = $Matches[1]
Write-Host "amalgamation: $url"
Invoke-WebRequest -Uri $url -OutFile amalgamation.zip -UseBasicParsing -MaximumRetryCount 3 -RetryIntervalSec 5
Expand-Archive amalgamation.zip -DestinationPath amalgamation
New-Item -ItemType Directory -Force -Path sv/vendor | Out-Null
Get-ChildItem amalgamation/*/* | Move-Item -Destination sv/vendor
Get-ChildItem sv/vendor
- name: Generate sqlite-vec.h from its template
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
Set-Location sv
$version = (Get-Content VERSION -Raw).Trim()
$parts = $version.Split('.')
$source = (git log -n 1 --pretty=format:%H -- VERSION)
$date = (git log -n 1 --pretty=format:%cI -- VERSION)
$h = Get-Content sqlite-vec.h.tmpl -Raw
$h = $h.Replace('${VERSION}', $version).Replace('${DATE}', $date).Replace('${SOURCE}', $source)
$h = $h.Replace('${VERSION_MAJOR}', $parts[0]).Replace('${VERSION_MINOR}', $parts[1]).Replace('${VERSION_PATCH}', ($parts[2] -split '-')[0])
if ($h -match '\$\{') { throw "unreplaced placeholder in sqlite-vec.h" }
Set-Content sqlite-vec.h -Value $h -Encoding ascii
"SQLITE_VEC_VERSION=$version" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
- name: Locate the ARM64 MSVC environment
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
$vs = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.ARM64 -property installationPath
if (-not $vs) { $vs = & $vswhere -latest -products * -property installationPath }
$bat = Join-Path $vs 'VC\Auxiliary\Build\vcvarsarm64.bat'
if (-not (Test-Path $bat)) { throw "no vcvarsarm64.bat under $vs" }
Write-Host "vcvars: $bat"
"VCVARS_BAT=$bat" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
# The upstream release workflow's own cl line, run natively for arm64.
- name: Compile vec0.dll for ARM64
shell: cmd
run: |
call "%VCVARS_BAT%" || exit /B 1
cd sv
mkdir dist
cl.exe /nologo /W4 /O2 /LD /Ivendor sqlite-vec.c /Fe:dist\vec0.dll || exit /B 1
dir dist
# The PyPI wheel is pure Python plus that one DLL, generated by sqlite-dist, so the
# port is a DLL swap, a tag rewrite and a regenerated RECORD -- not a rebuild.
- name: Repack the PyPI wheel with the ARM64 DLL
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
python -m pip download --quiet --no-deps --only-binary=:all: --platform win_amd64 --python-version 3.13 -d amd64 "sqlite-vec==$env:SQLITE_VEC_VERSION"
$src = Get-ChildItem amd64/*.whl | Select-Object -First 1
python tools/.github/scripts/repack_sqlite_vec.py $src.FullName sv/dist/vec0.dll wheel
Get-ChildItem wheel
- name: Load it through sqlite3 and run a vec0 query
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$whl = Get-ChildItem wheel/*.whl | Select-Object -First 1
python -m venv smoke
smoke\Scripts\python.exe -m pip install --quiet $whl.FullName
smoke\Scripts\python.exe -c @"
import sqlite3, sqlite_vec, sysconfig
print('platform', sysconfig.get_platform(), 'sqlite', sqlite3.sqlite_version)
db = sqlite3.connect(':memory:')
db.enable_load_extension(True)
sqlite_vec.load(db)
db.enable_load_extension(False)
print('vec_version', db.execute('select vec_version()').fetchone()[0])
db.execute('create virtual table v using vec0(embedding float[4])')
for rid, vec in [(1, [1, 0, 0, 0]), (2, [0, 1, 0, 0]), (3, [0.9, 0.1, 0, 0])]:
db.execute('insert into v(rowid, embedding) values (?, ?)', (rid, sqlite_vec.serialize_float32(vec)))
got = db.execute('select rowid, distance from v where embedding match ? order by distance limit 2', (sqlite_vec.serialize_float32([1, 0, 0, 0]),)).fetchall()
print('knn', got)
assert [r[0] for r in got] == [1, 3], got
"@
if ($LASTEXITCODE -ne 0) { exit 1 }
- uses: actions/upload-artifact@v4
with:
name: unsigned-sqlite-vec
path: wheel/*.whl
if-no-files-found: error
sign:
name: Sign the native binaries
needs: [build-pyarrow, build-sqlite-vec]
# x64, deliberately. The wheels are ARM64 but Authenticode does not care what the host
# is, and trusted-signing-cli is an x64 binary whose digest is pinned for windows-latest
# in release-desktop.yml. Signing here runs the same verified bytes rather than the same
# bytes under emulation.
runs-on: windows-latest
timeout-minutes: 45
# The Azure Trusted Signing secrets are read by this job and no other. Naming the
# environment is what makes them gateable; add required reviewers in repo settings to
# close the gap, exactly as release-desktop.yml documents.
environment: release-signing
steps:
- name: Harden runner (audit)
uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with:
egress-policy: audit
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
pattern: unsigned-*
merge-multiple: true
path: unsigned
- name: List what was built
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$wheels = Get-ChildItem unsigned/*.whl
$wheels | ForEach-Object { "{0,-52} {1,8:N0} KB" -f $_.Name, ($_.Length / 1KB) }
# Four pyarrow (cp311-cp314) and one sqlite-vec. A short matrix leg would otherwise
# publish a wheelhouse missing an interpreter, which reads as "no wheel" downstream.
if ($wheels.Count -ne 5) { throw "expected 5 wheels, got $($wheels.Count)" }
# Same pin as release-desktop.yml. The URL is reproducibility, the digest is the gate:
# a release asset can be replaced after upload, and a replaced one must fail the run
# rather than sign with an unknown binary. Bump both together.
- name: Install trusted-signing-cli
shell: pwsh
env:
TRUSTED_SIGNING_CLI_URL: "https://github.com/Levminer/artifact-signing-cli/releases/download/0.10.0/trusted-signing-cli.exe"
TRUSTED_SIGNING_CLI_SHA256: "8c9d750ca582891a7433925dcf302d5d3761fbed757b46c4bf5b417562dccb0e"
run: |
$ErrorActionPreference = "Stop"
$dir = Join-Path $env:RUNNER_TEMP "trusted-signing-cli"
New-Item -ItemType Directory -Force -Path $dir | Out-Null
$dest = Join-Path $dir "trusted-signing-cli.exe"
Invoke-WebRequest -Uri $env:TRUSTED_SIGNING_CLI_URL -OutFile $dest -MaximumRetryCount 3 -RetryIntervalSec 5
$actual = (Get-FileHash -Path $dest -Algorithm SHA256).Hash.ToLower()
$expected = $env:TRUSTED_SIGNING_CLI_SHA256.ToLower()
if ($actual -ne $expected) {
Remove-Item -Force $dest
Write-Output "::error::trusted-signing-cli digest mismatch, refusing to sign with an unverified binary. Expected $expected, got $actual."
exit 1
}
Write-Output "verified trusted-signing-cli sha256=$actual"
$dir | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
- name: Verify trusted-signing-cli
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$cli = Get-Command trusted-signing-cli -ErrorAction SilentlyContinue
if (-not $cli) {
Write-Output "::error::trusted-signing-cli is not on PATH. Check the install step above."
exit 1
}
# The helper invokes it by bare name, so whatever PATH resolves is what signs. Only
# the digest-verified copy may do that.
$verified = [IO.Path]::GetFullPath((Join-Path (Join-Path $env:RUNNER_TEMP "trusted-signing-cli") "trusted-signing-cli.exe"))
if ([IO.Path]::GetFullPath($cli.Source) -ne $verified) {
Write-Output "::error::trusted-signing-cli resolved to $($cli.Source), not the verified $verified."
exit 1
}
try { & $cli.Source --version }
catch {
$detail = ($_.Exception.Message -replace '\s+', ' ').Trim()
Write-Output "::error::trusted-signing-cli could not be started. Underlying error: $detail"
exit 1
}
if ($LASTEXITCODE -ne 0) {
Write-Output "::error::trusted-signing-cli is on PATH but exited $LASTEXITCODE."
exit 1
}
- name: Sign every native binary inside the wheels
shell: pwsh
env:
AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
AZURE_TRUSTED_SIGNING_ACCOUNT_NAME: ${{ secrets.AZURE_TRUSTED_SIGNING_ACCOUNT_NAME }}
AZURE_CERTIFICATE_PROFILE_NAME: ${{ secrets.AZURE_CERTIFICATE_PROFILE_NAME }}
run: |
$ErrorActionPreference = "Stop"
python .github/scripts/sign_wheel_binaries.py (Get-ChildItem unsigned/*.whl | ForEach-Object { $_.FullName }) `
--out signed `
--signer "pwsh -NoProfile -File studio/src-tauri/windows/sign-with-trusted-signing.ps1 -Path"
if ($LASTEXITCODE -ne 0) { Write-Output "::error::signing failed"; exit 1 }
# Independent of the signing tool's own report: unpack what will actually be published
# and ask Windows whether each signature validates on this machine.
- name: Verify the published wheels validate
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$bad = 0; $ok = 0; $ms = 0
foreach ($whl in Get-ChildItem signed/*.whl) {
$dir = Join-Path $env:RUNNER_TEMP ("verify-" + $whl.BaseName)
Remove-Item -Recurse -Force $dir -ErrorAction SilentlyContinue
Expand-Archive -Path $whl.FullName -DestinationPath $dir
foreach ($pe in Get-ChildItem $dir -Recurse -Include *.pyd, *.dll) {
$sig = Get-AuthenticodeSignature $pe.FullName
if ($sig.Status -ne 'Valid') {
Write-Output "::error::$($whl.Name) :: $($pe.Name) is $($sig.Status)"
$bad++
continue
}
if ($sig.SignerCertificate.Subject -match 'Unsloth') { $ok++ } else { $ms++ }
}
}
Write-Output "signed by us: $ok, upstream-signed (Microsoft redistributables): $ms, invalid: $bad"
if ($bad -gt 0) { exit 1 }
if ($ok -eq 0) { Write-Output "::error::nothing carries our signature"; exit 1 }
# RECORD is rewritten by the signing step, so this proves the wheel is still installable
# and that its own manifest agrees with its contents.
- name: Install each wheel and check its RECORD
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
python -c @"
import base64, hashlib, pathlib, sys, zipfile
bad = 0
for whl in sorted(pathlib.Path('signed').glob('*.whl')):
z = zipfile.ZipFile(whl)
record = next(n for n in z.namelist() if n.endswith('.dist-info/RECORD'))
rows = {}
for line in z.read(record).decode().splitlines():
if not line.strip():
continue
name, digest, _, = (line.split(',') + ['', ''])[:3]
rows[name] = digest
for name in z.namelist():
if name == record:
continue
want = rows.get(name)
if not want:
print('MISSING from RECORD:', whl.name, name); bad += 1; continue
got = 'sha256=' + base64.urlsafe_b64encode(hashlib.sha256(z.read(name)).digest()).rstrip(b'=').decode()
if got != want:
print('DIGEST MISMATCH:', whl.name, name); bad += 1
print(whl.name, 'RECORD ok,', len(rows), 'entries')
sys.exit(1 if bad else 0)
"@
if ($LASTEXITCODE -ne 0) { exit 1 }
- name: Build index.txt and SHA256SUMS
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
$wheels = Get-ChildItem signed/*.whl | Sort-Object Name
# Flat find-links listing: install.ps1 reads this, then fetches each name beside it.
($wheels | ForEach-Object { $_.Name }) -join "`n" | Set-Content signed/index.txt -Encoding ascii
($wheels | ForEach-Object { "{0} {1}" -f (Get-FileHash $_.FullName -Algorithm SHA256).Hash.ToLower(), $_.Name }) -join "`n" |
Set-Content signed/SHA256SUMS -Encoding ascii
Get-Content signed/index.txt
Get-Content signed/SHA256SUMS
- uses: actions/upload-artifact@v4
with:
name: woa-wheelhouse-signed
path: |
signed/*.whl
signed/index.txt
signed/SHA256SUMS
if-no-files-found: error
publish:
name: Publish the wheelhouse
needs: sign
if: inputs.publish
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: write
steps:
- name: Harden runner (audit)
uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0
with:
egress-policy: audit
- uses: actions/download-artifact@v4
with:
name: woa-wheelhouse-signed
path: wheelhouse
# A release, not a branch: GitHub reports a per-asset `digest` through the release API,
# which is the same verification the installer already applies to the llama.cpp bundles.
- name: Create or update the release
shell: bash
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ inputs.release_tag }}
run: |
set -euo pipefail
if ! gh release view "$TAG" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
gh release create "$TAG" \
--repo "$GITHUB_REPOSITORY" \
--title "Windows on ARM wheelhouse" \
--notes "Signed win_arm64 wheels for the native Windows on ARM installer: pyarrow (cp311-cp314) and sqlite-vec. Neither publishes a win_arm64 wheel on PyPI. The native binaries inside each wheel are Authenticode signed through Azure Trusted Signing; SHA256SUMS lists the digest of every wheel as published."
fi
# --clobber: this tag is a rolling wheelhouse, and a rebuilt wheel for the same
# interpreter is meant to replace its predecessor rather than fail the upload.
gh release upload "$TAG" --repo "$GITHUB_REPOSITORY" --clobber \
wheelhouse/*.whl wheelhouse/index.txt wheelhouse/SHA256SUMS
- name: Report the wheelhouse URL
shell: bash
env:
TAG: ${{ inputs.release_tag }}
run: |
set -euo pipefail
url="https://github.com/$GITHUB_REPOSITORY/releases/download/$TAG"
{
echo "### Windows on ARM wheelhouse published"
echo
echo 'Point the installer at it with:'
echo
echo '```powershell'
echo "\$env:UNSLOTH_WOA_WHEELHOUSE = \"$url\""
echo '```'
echo
echo '| wheel | sha256 |'
echo '| --- | --- |'
while read -r digest name; do echo "| $name | \`$digest\` |"; done < wheelhouse/SHA256SUMS
} >> "$GITHUB_STEP_SUMMARY"

4
.gitignore vendored
View file

@ -255,6 +255,10 @@ Speech_12dB_s16.flac
# so a test run or an Unsloth start rewrites it and dirties the checkout.
.studio-test-root/
# _filter_requirements() in studio/install_python_stack.py writes its skip-filtered copy beside
# the source file so relative -r/-c includes still resolve. A killed install leaves them behind.
studio/backend/requirements/.*-filtered-*.txt
# launch.py --outdir scratch from a local run
/out/

File diff suppressed because it is too large Load diff

View file

@ -149,7 +149,9 @@ studio = [
"pydantic==2.13.4",
"packaging==26.3",
"matplotlib==3.10.9",
"pandas==2.3.3",
# win_arm64 pandas wheels start at 3.0; the 3.11 floor is pandas 3's own requires-python.
"pandas==2.3.3 ; sys_platform != 'win32' or platform_machine != 'ARM64' or python_version < '3.11'",
"pandas>=3.0,<4 ; sys_platform == 'win32' and platform_machine == 'ARM64' and python_version >= '3.11'",
"nest_asyncio==1.6.0",
"datasets==4.3.0",
"pyjwt==2.13.0",
@ -161,15 +163,21 @@ studio = [
"diceware==1.0.1",
"ddgs==9.14.4 ; python_version >= '3.10'",
"ddgs==9.8.0 ; python_version < '3.10'",
"cryptography>=42.0.0",
"cryptography>=42.0.0; sys_platform != 'win32' or platform_machine != 'ARM64'",
# Windows on ARM: 46.0.7 publishes no win_arm64 wheel and the sdist needs Rust.
"cryptography>=42.0.0,<=46.0.3; sys_platform == 'win32' and platform_machine == 'ARM64'",
"boto3>=1.34.0",
"httpx>=0.27.0",
"fastmcp>=3.0.2",
"gguf==0.19.0 ; python_version >= '3.10'",
"gguf==0.18.0 ; python_version < '3.10'",
"av>=12.3.0",
"sqlite-vec==0.1.9",
"pymupdf==1.27.2.3",
# No win_arm64 wheel and no sdist either, so it cannot be floored or built. Dropped rather
# than pinned: the RAG store already treats a missing sqlite_vec as "RAG unavailable".
"sqlite-vec==0.1.9 ; sys_platform != 'win32' or platform_machine != 'ARM64'",
# Mirrors studio.txt: pymupdf publishes a win_arm64 wheel from 1.28 only.
"pymupdf==1.27.2.3 ; sys_platform != 'win32' or platform_machine != 'ARM64'",
"pymupdf>=1.28.2 ; sys_platform == 'win32' and platform_machine == 'ARM64'",
"pymupdf4llm==0.3.4",
"python-docx==1.2.0",
"markdown-it-py>=3.0",
@ -177,7 +185,9 @@ studio = [
triton = [
"triton>=3.0.0 ; ('linux' in sys_platform)",
# triton-windows has win_arm64 wheels from 3.8.0.post28, so a floor works here, not a marker.
"triton-windows ; (sys_platform == 'win32') and (platform_machine == 'AMD64' or platform_machine == 'x86_64')",
"triton-windows>=3.8.0.post28 ; (sys_platform == 'win32') and platform_machine == 'ARM64'",
]
huggingfacenotorch = [
@ -194,7 +204,9 @@ huggingfacenotorch = [
"accelerate>=0.34.1",
"peft>=0.18.0,!=0.11.0",
"huggingface_hub>=0.34.0",
"hf_transfer",
# A Rust extension with no win_arm64 wheel. huggingface_hub imports it only under
# HF_HUB_ENABLE_HF_TRANSFER=1, and hf_xet ships win_arm64 and covers fast downloads.
"hf_transfer ; sys_platform != 'win32' or platform_machine != 'ARM64'",
# 0.39 is the floor the diffusion stack already assumes: Krea2Pipeline (the Krea 2
# workflow raises without it), the cache_context child-registry behavior, and the
# Flux2 / Z-Image pipelines. An unversioned entry let an upgrade keep an older
@ -287,8 +299,11 @@ huggingface = [
# handles instead.
windows = [
"unsloth[huggingface]",
# bitsandbytes ships a win_arm64 wheel from 0.50.0, so this row covers it.
"bitsandbytes>=0.45.5,!=0.46.0,!=0.48.0 ; (sys_platform == 'win32')",
"xformers>=0.0.22.post7 ; (sys_platform == 'win32')",
# No win_arm64 wheel and no sdist that builds without CUDA's Windows-ARM toolchain, so this
# row would fail the whole extra. Torch's SDPA is the fallback.
"xformers>=0.0.22.post7 ; (sys_platform == 'win32') and platform_machine != 'ARM64'",
]
base = [
"unsloth[huggingface]",

View file

@ -12,7 +12,8 @@ description = "Local Data Designer unstructured seed reader plugin"
requires-python = ">=3.11"
dependencies = [
"data-designer-engine>=0.5.4,<0.6",
"pandas>=2,<3",
# <4: Windows on ARM installs pandas 3 (constraints.txt), and this plugin only reads and writes parquet.
"pandas>=2,<4",
"pymupdf>=1.24.0",
"pymupdf4llm>=0.0.17",
"mammoth>=1.8.0",

View file

@ -2,7 +2,9 @@
flatbuffers==25.12.19
# Also needed by sentence_transformers (installed with --no-deps in extras-no-deps.txt);
# librosa pulls it in too, but is skipped in no-torch mode.
scikit-learn==1.7.1
# scikit-learn publishes win_arm64 wheels from 1.8 only; 1.7.1 would source-build there.
scikit-learn==1.7.1; sys_platform != "win32" or platform_machine != "ARM64"
scikit-learn>=1.8.0; sys_platform == "win32" and platform_machine == "ARM64"
# Additional extras
jiwer==4.0.0 # WER/CER metrics for vision OCR save-merge benchmarks
@ -18,7 +20,9 @@ openai-whisper==20250625
# av<16 because 16+ builds its macOS arm64 wheels against macosx_14_0 and so has
# no installable wheel on macOS 13. Pinning past the cap makes that leg
# unsatisfiable. Lift both together.
av==15.1.0
# See constraints.txt: win_arm64 av wheels start at 17.0.0.
av==15.1.0; sys_platform != "win32" or platform_machine != "ARM64"
av>=17.0.0; sys_platform == "win32" and platform_machine == "ARM64"
uroman==1.3.1.1 # 4.0 MB - used for Outetts.
# 19.9 MB - used for Outetts. No release ships a macOS cp314 wheel (0.996.12 added
# cp314 for Linux and win_amd64 only), so a 3.14 macOS host has no binary candidate

View file

@ -48,7 +48,8 @@ accelerate>=0.34.1
peft>=0.18.0,!=0.11.0
huggingface_hub>=1.23.0,<2.0; python_version >= "3.10"
huggingface_hub==0.36.2; python_version < "3.10"
hf_transfer==0.1.9
# No win_arm64 wheel and this file installs --no-deps, so pip would need cargo plus MSVC.
hf_transfer==0.1.9; sys_platform != "win32" or platform_machine != "ARM64"
# diffusers: see diffusers-pin.txt, installed by its own step (this file is
# installed --no-deps, and the pin has to apply on the torch path too).
@ -96,8 +97,12 @@ pillow==11.3.0; python_version < "3.10"
# RAG store + document parsing, mirroring studio.txt. Pinned here because
# this file installs --no-deps; without them Unsloth runs with RAG disabled.
sqlite-vec==0.1.9
pymupdf==1.27.2.3
# sqlite-vec publishes win_amd64 only and no sdist; the RAG store treats a missing sqlite_vec as
# "RAG unavailable". Same marker as studio.txt: this file is installed directly on an update.
sqlite-vec==0.1.9; sys_platform != "win32" or platform_machine != "ARM64"
# pymupdf publishes a win_arm64 wheel from 1.28 only.
pymupdf==1.27.2.3; sys_platform != "win32" or platform_machine != "ARM64"
pymupdf>=1.28.2; sys_platform == "win32" and platform_machine == "ARM64"
# 0.3.x keeps pymupdf-layout (which pulls onnxruntime) an optional extra; the
# lockstep 1.27.x line makes it a hard dep we do not need for to_markdown().
pymupdf4llm==0.3.4

View file

@ -13,7 +13,10 @@ huggingface-hub==0.36.2; python_version < "3.10"
# Unsloth stack
datasets==4.3.0
pyarrow==23.0.1
# pyarrow has never published a win_arm64 wheel (apache/arrow#48539). The floor below is the
# floor of the wheels install.ps1 stages, not of anything upstream ships.
pyarrow==23.0.1; sys_platform != "win32" or platform_machine != "ARM64"
pyarrow>=21.0.0; sys_platform == "win32" and platform_machine == "ARM64"
# FastMCP compat
fastmcp>=3.0.2
@ -34,7 +37,9 @@ anyio<4.14.0
# guards the call. install_python_stack.py repairs an install that is already past the cap.
accelerate<1.15.0; sys_platform == "win32"
pandas==2.3.3
# pandas publishes win_arm64 wheels from 3.0 only; 2.3.x would source-build there.
pandas==2.3.3; sys_platform != "win32" or platform_machine != "ARM64" or python_version < "3.11"
pandas>=3.0,<4; sys_platform == "win32" and platform_machine == "ARM64" and python_version >= "3.11"
# av (PyAV) 16+ builds its macOS arm64 wheels against macosx_14_0, so on macOS 13 none
# are installable and the resolver falls back to a source build, which needs FFmpeg
@ -45,10 +50,16 @@ pandas==2.3.3
# The remaining sdist-only macOS defaults are pure Python, hence allowlisted in
# .github/scripts/clean-machine-assert.sh instead; cryptography below is the one
# other package that would compile.
av<16
av<16; sys_platform != "win32" or platform_machine != "ARM64"
# The opposite case: av publishes win_arm64 wheels from 17.0.0 only (cp311-abi3,
# plus a cp314t; there is no 3.13t wheel on any release).
av>=17.0.0; sys_platform == "win32" and platform_machine == "ARM64"
# cryptography 49.0.0 dropped the macosx_10_9_universal2 wheel for arm64-only, so
# x86_64 macOS has no wheel and builds the sdist, needing Rust plus a working
# linker. 48.0.1 is the newest release with a universal2 wheel. Lift when
# cryptography ships an x86_64-capable macOS wheel again.
cryptography<49; sys_platform == "darwin" and platform_machine == "x86_64"
# 46.0.3 is the newest cryptography with a win_arm64 wheel; raise the cap when upstream resumes.
cryptography<=46.0.3; sys_platform == "win32" and platform_machine == "ARM64"

View file

@ -12,8 +12,10 @@ packaging==26.3
matplotlib==3.10.9
# Held at 2.3.3 rather than the 3.0.x line: single-env/constraints.txt pins
# pandas==2.3.3, and 2.3.3 is also the newest release that still installs on
# 3.9. Bump both together.
pandas==2.3.3
# 3.9. Bump both together. Windows on ARM is the exception: win_arm64 pandas wheels
# start at 3.0, and the 3.11 floor there is pandas 3's own requires-python.
pandas==2.3.3; sys_platform != "win32" or platform_machine != "ARM64" or python_version < "3.11"
pandas>=3.0,<4; sys_platform == "win32" and platform_machine == "ARM64" and python_version >= "3.11"
nest_asyncio==1.6.0
datasets==4.3.0
pyjwt==2.13.0
@ -37,7 +39,10 @@ structlog>=24.1.0
diceware==1.0.1
ddgs==9.14.4; python_version >= "3.10"
ddgs==9.8.0; python_version < "3.10"
cryptography>=42.0.0
cryptography>=42.0.0; sys_platform != "win32" or platform_machine != "ARM64"
# Windows on ARM: 46.0.7 has no win_arm64 wheel and its sdist needs Rust. Matches
# constraints.txt, which a plain pip install of the extra never reaches.
cryptography>=42.0.0,<=46.0.3; sys_platform == "win32" and platform_machine == "ARM64"
boto3>=1.34.0 # optional: S3 dataset loading
httpx>=0.27.0
fastmcp>=3.0.2
@ -53,8 +58,12 @@ gguf==0.18.0; python_version < "3.10"
av>=12.3.0
# RAG (knowledge bases, hybrid retrieval). sentence-transformers lives in
# extras-no-deps.txt; these add the lexical+dense store and document parsing.
sqlite-vec==0.1.9
pymupdf==1.27.2.3
# sqlite-vec publishes win_amd64 only and no sdist, so there is nothing to floor to. The RAG
# store treats a missing sqlite_vec as "RAG unavailable".
sqlite-vec==0.1.9; sys_platform != "win32" or platform_machine != "ARM64"
# pymupdf publishes a win_arm64 wheel from 1.28 only.
pymupdf==1.27.2.3; sys_platform != "win32" or platform_machine != "ARM64"
pymupdf>=1.28.2; sys_platform == "win32" and platform_machine == "ARM64"
# 0.3.x keeps pymupdf-layout (which pulls onnxruntime) an optional extra; the
# lockstep 1.27.x line makes it a hard dep we do not need for to_markdown().
pymupdf4llm==0.3.4

View file

@ -13,6 +13,8 @@ INSTALL_KIND_BACKENDS: dict[str, str] = {
"linux-cuda": "cuda",
"linux-arm64-cuda": "cuda",
"windows-cuda": "cuda",
# Distinct from windows-cuda: the arm64 bundle ships its own ARM64 cudart/cublas.
"windows-arm64-cuda": "cuda",
"linux-rocm": "rocm",
"windows-hip": "rocm",
"windows-rocm": "rocm",

View file

@ -776,10 +776,52 @@ def checkout_friendly_ref(ref_kind: str | None, ref: str | None) -> str | None:
return ref
def windows_cuda_upstream_asset_names(llama_tag: str, runtime: str) -> list[str]:
# Upstream names these llama-<tag>-bin-win-cuda-<runtime>-<arch>.zip, arch in {x64, arm64}.
WINDOWS_CUDA_ARCHS = ("x64", "arm64")
_WINDOWS_CUDA_INSTALL_KINDS = ("windows-cuda", "windows-arm64-cuda")
def _upstream_arm64_cuda_allowed() -> bool:
"""May a Windows ARM64 NVIDIA host install a CUDA llama.cpp bundle at all? Defaults
to yes, because the only other answer is a CPU-only build on a CUDA machine.
UNSLOTH_LLAMA_ARM64_CUDA=0/false/no/off opts out.
This governs every ARM64 CUDA bundle, published or upstream. It started narrower --
only the unverified upstream bundle, since that is the one with a provenance question
-- but what the flag promises the user is a CPU bundle, and scoping it to the
unverified case meant the escape hatch would switch itself off the day the fork
published an approved windows-arm64-cuda artifact. A setting that silently stops
working on someone else's release is worse than one that is merely blunt.
"""
raw = (os.environ.get("UNSLOTH_LLAMA_ARM64_CUDA") or "").strip().lower()
return raw not in {"0", "false", "no", "off"}
def windows_cuda_arch_for_host(host: HostInfo) -> str:
"""CUDA prebuilt arch token for a Windows host: arm64 on Windows on ARM, else x64.
``is_arm64`` is also true for macOS arm64 and Linux aarch64, so the Windows test
is part of the question rather than the caller's job. Every caller today is already
inside a Windows branch; this keeps the answer right if one ever is not.
"""
return "arm64" if (host.is_windows and host.is_arm64) else "x64"
def windows_cuda_install_kind_for_arch(arch: str) -> str:
"""install_kind for a Windows CUDA prebuilt of `arch`. Kept separate from the x64
kind so the runtime-overlay, health-check and marker tables can tell the two apart
(their payloads differ: the arm64 bundle carries no cuBLASLt-for-x64 DLL set)."""
return "windows-arm64-cuda" if arch == "arm64" else "windows-cuda"
def windows_cuda_upstream_asset_names(
llama_tag: str,
runtime: str,
arch: str = "x64",
) -> list[str]:
return [
f"llama-{llama_tag}-bin-win-cuda-{runtime}-x64.zip",
f"cudart-llama-bin-win-cuda-{runtime}-x64.zip",
f"llama-{llama_tag}-bin-win-cuda-{runtime}-{arch}.zip",
f"cudart-llama-bin-win-cuda-{runtime}-{arch}.zip",
]
@ -787,29 +829,35 @@ def windows_cuda_asset_aliases(
asset_name: str, *, compatibility_tag: str | None = None
) -> list[str]:
aliases: list[str] = []
_arch_re = "|".join(WINDOWS_CUDA_ARCHS)
legacy_match = re.fullmatch(
r"llama-(?P<tag>[^/]+)-bin-win-cuda-(?P<runtime>\d+\.\d+)-x64\.zip",
rf"llama-(?P<tag>[^/]+)-bin-win-cuda-(?P<runtime>\d+\.\d+)-(?P<arch>{_arch_re})\.zip",
asset_name,
)
if legacy_match:
runtime = legacy_match.group("runtime")
aliases.append(f"cudart-llama-bin-win-cuda-{runtime}-x64.zip")
arch = legacy_match.group("arch")
aliases.append(f"cudart-llama-bin-win-cuda-{runtime}-{arch}.zip")
if compatibility_tag:
aliases.append(f"llama-{compatibility_tag}-bin-win-cuda-{runtime}-x64.zip")
aliases.append(f"llama-{compatibility_tag}-bin-win-cuda-{runtime}-{arch}.zip")
return aliases
current_match = re.fullmatch(
r"cudart-llama-bin-win-cuda-(?P<runtime>\d+\.\d+)-x64\.zip",
rf"cudart-llama-bin-win-cuda-(?P<runtime>\d+\.\d+)-(?P<arch>{_arch_re})\.zip",
asset_name,
)
if current_match and compatibility_tag:
runtime = current_match.group("runtime")
aliases.append(f"llama-{compatibility_tag}-bin-win-cuda-{runtime}-x64.zip")
arch = current_match.group("arch")
aliases.append(f"llama-{compatibility_tag}-bin-win-cuda-{runtime}-{arch}.zip")
return aliases
def _published_windows_cuda_runtime(
upstream_assets: dict[str, str], major: int, driver: tuple[int, int] | None
upstream_assets: dict[str, str],
major: int,
driver: tuple[int, int] | None,
arch: str = "x64",
) -> str | None:
"""Highest cuda-<major>.<minor> published upstream that `driver` can run by
default CUDA compatibility, i.e. (major, minor) <= driver. None if nothing
@ -820,7 +868,7 @@ def _published_windows_cuda_runtime(
return None
best: int | None = None
for name in upstream_assets:
m = re.search(r"-bin-win-cuda-(\d+)\.(\d+)-x64\.zip$", name)
m = re.search(rf"-bin-win-cuda-(\d+)\.(\d+)-{re.escape(arch)}\.zip$", name)
if m and int(m.group(1)) == major:
minor = int(m.group(2))
if (major, minor) <= driver and (best is None or minor > best):
@ -1151,6 +1199,20 @@ def direct_upstream_release_plan(
)
)
elif host.is_windows and host.is_arm64:
# Upstream ships -arm64 CUDA bundles for these parts; try them first, CPU bundle still last.
if host.has_usable_nvidia and _upstream_arm64_cuda_allowed():
torch_preference = detect_torch_cuda_runtime_preference(host)
attempts.extend(
windows_cuda_attempts(
host,
release_tag,
assets,
torch_preference.runtime_line,
torch_preference.selection_log,
arch = "arm64",
)
)
attempts[:] = _drop_blackwell_incapable_windows_cuda(host, attempts)
# Upstream ggml-org/llama.cpp ships llama-bNNNN-bin-win-cpu-arm64.zip
# (visible in the b9334 release manifest). Without this branch the
# selector returned 0 attempts and the installer fell back to a
@ -1376,6 +1438,22 @@ def detected_linux_runtime_lines() -> tuple[list[str], dict[str, list[str]]]:
release_asset_map = _core.release_asset_map
release_asset_digests = _core.release_asset_digests
def github_release_asset_digests(repo: str, tag: str) -> dict[str, str]:
"""asset name -> sha256, from the release GitHub already serves us.
Separate call rather than a second return value from github_release_assets: that one is
mocked in a dozen suites and used on every platform, while this is read on one branch
only. Any failure is an empty mapping, which that branch reads as "refuse", so a rate
limit or an outage costs the CUDA bundle rather than installing it unchecked.
"""
try:
return release_asset_digests(github_release(repo, tag))
except Exception as exc:
log(f"could not read release asset digests for {repo}@{tag}: {exc}")
return {}
def parse_published_artifact(raw: Any) -> PublishedLlamaArtifact | None:
@ -3103,8 +3181,14 @@ def windows_cuda_attempts(
upstream_assets: dict[str, str],
preferred_runtime_line: str | None,
selection_preamble: Iterable[str] = (),
arch: str | None = None,
) -> list[AssetChoice]:
if arch is None:
arch = windows_cuda_arch_for_host(host)
install_kind = windows_cuda_install_kind_for_arch(arch)
selection_log = list(selection_preamble)
if arch != "x64":
selection_log.append(f"windows_cuda_selection: arch={arch}")
driver_runtime = pick_windows_cuda_runtime(host)
detected_runtime_lines, runtime_dirs = detected_windows_runtime_lines()
compatible_runtime_lines = compatible_windows_runtime_lines(host)
@ -3189,7 +3273,9 @@ def windows_cuda_attempts(
# Track whatever minor llama.cpp actually ships for this major
# (cuda13 -> 13.1, 13.3, ...). Skip the line when the release has no
# matching asset instead of guessing a now-missing name.
runtime = _published_windows_cuda_runtime(upstream_assets, major, host.driver_cuda_version)
runtime = _published_windows_cuda_runtime(
upstream_assets, major, host.driver_cuda_version, arch
)
if runtime is None:
selection_log.append(
f"windows_cuda_selection: no driver-supported asset for {runtime_line}"
@ -3197,7 +3283,7 @@ def windows_cuda_attempts(
continue
selected_name = None
asset_url = None
for candidate_name in windows_cuda_upstream_asset_names(llama_tag, runtime):
for candidate_name in windows_cuda_upstream_asset_names(llama_tag, runtime, arch):
asset_url = upstream_assets.get(candidate_name)
if asset_url:
selected_name = candidate_name
@ -3205,7 +3291,7 @@ def windows_cuda_attempts(
if not asset_url or not selected_name:
selection_log.append(
"windows_cuda_selection: skip missing assets "
+ ",".join(windows_cuda_upstream_asset_names(llama_tag, runtime))
+ ",".join(windows_cuda_upstream_asset_names(llama_tag, runtime, arch))
)
continue
# Pair the cudart bundle when upstream ships it. Without this
@ -3215,7 +3301,7 @@ def windows_cuda_attempts(
runtime_archive_name: str | None = None
runtime_archive_url: str | None = None
if selected_name.startswith("llama-"):
cudart_name = f"cudart-llama-bin-win-cuda-{runtime}-x64.zip"
cudart_name = f"cudart-llama-bin-win-cuda-{runtime}-{arch}.zip"
cudart_url = upstream_assets.get(cudart_name)
if cudart_url and cudart_url != asset_url:
runtime_archive_name = cudart_name
@ -3239,7 +3325,7 @@ def windows_cuda_attempts(
name = selected_name,
url = asset_url,
source_label = "upstream",
install_kind = "windows-cuda",
install_kind = install_kind,
runtime_line = runtime_line,
runtime_name = runtime_archive_name,
runtime_url = runtime_archive_url,
@ -3252,13 +3338,15 @@ def windows_cuda_attempts(
def _windows_cuda_attempt_covers_blackwell(
attempt: AssetChoice, min_toolkit: tuple[int, int] = _BLACKWELL_MIN_TOOLKIT
) -> bool:
"""True if a windows-cuda attempt is Blackwell-capable (app bundles via
"""True if a windows CUDA attempt is Blackwell-capable (app bundles via
declared SMs; legacy upstream bundles via toolkit minor >= min_toolkit:
12.8 for the family, 12.9 for sm_103/sm_121)."""
if attempt.install_kind != "windows-cuda":
if attempt.install_kind not in _WINDOWS_CUDA_INSTALL_KINDS:
return False
# Legacy bundle: the toolkit minor binds (12.4 cannot offload Blackwell).
m = re.search(r"-bin-win-cuda-(\d+)\.(\d+)-x64\.zip$", attempt.name)
m = re.search(
rf"-bin-win-cuda-(\d+)\.(\d+)-(?:{'|'.join(WINDOWS_CUDA_ARCHS)})\.zip$", attempt.name
)
if m is not None:
return (int(m.group(1)), int(m.group(2))) >= min_toolkit
# App-named bundles carry no minor and declare their SM coverage directly.
@ -3272,17 +3360,19 @@ _blackwell_min_toolkit_for_host = _core.blackwell_min_toolkit_for_host
def _drop_blackwell_incapable_windows_cuda(
host: HostInfo, attempts: list[AssetChoice]
) -> list[AssetChoice]:
"""On a Blackwell host, drop windows-cuda attempts that can't offload
"""On a Blackwell host, drop windows CUDA attempts that can't offload
Blackwell (e.g. cuda-12.4): they load and validate but run a slow non-native
path (RTX 5090: 7.1 vs 551.2 tok/s on cuda-13.3). Non-cuda attempts pass
through so the host can still fall back to an honest CPU install."""
through so the host can still fall back to an honest CPU install. Windows on ARM
is covered too: every NVIDIA part in that class (GB10 / N1X) is Blackwell, and its
only CUDA bundles are cuda-13.x, so nothing legitimate is filtered there."""
if not _host_is_blackwell(host):
return attempts
min_toolkit = _blackwell_min_toolkit_for_host(host)
return [
attempt
for attempt in attempts
if attempt.install_kind != "windows-cuda"
if attempt.install_kind not in _WINDOWS_CUDA_INSTALL_KINDS
or _windows_cuda_attempt_covers_blackwell(attempt, min_toolkit)
]
@ -3292,10 +3382,14 @@ def published_windows_cuda_attempts(
release: PublishedReleaseBundle,
preferred_runtime_line: str | None,
selection_preamble: Iterable[str] = (),
arch: str | None = None,
) -> list[AssetChoice]:
if arch is None:
arch = windows_cuda_arch_for_host(host)
install_kind = windows_cuda_install_kind_for_arch(arch)
selection_log = list(release.selection_log) + list(selection_preamble)
published_artifacts = [
artifact for artifact in release.artifacts if artifact.install_kind == "windows-cuda"
artifact for artifact in release.artifacts if artifact.install_kind == install_kind
]
artifacts_by_runtime: dict[str, list[PublishedLlamaArtifact]] = {}
for artifact in published_artifacts:
@ -3310,8 +3404,10 @@ def published_windows_cuda_attempts(
# actually provides on the host, preferring the torch line. Routing them
# through the synthetic-minor path wrongly dropped cuda13 on a 13.0 driver.
legacy_minors: list[str] = []
# Scoped to the arch being resolved: hardcoded x64 would mis-order the arm64 runtime lines.
_legacy_pattern = rf"-bin-win-cuda-(\d+\.\d+)-{re.escape(arch)}\.zip$"
for artifact in published_artifacts:
m = re.search(r"-bin-win-cuda-(\d+\.\d+)-x64\.zip$", artifact.asset_name)
m = re.search(_legacy_pattern, artifact.asset_name)
if m:
legacy_minors.append(m.group(1))
if legacy_minors:
@ -3321,11 +3417,12 @@ def published_windows_cuda_attempts(
host,
release.upstream_tag,
{
f"llama-{release.upstream_tag}-bin-win-cuda-{minor}-x64.zip": "published"
f"llama-{release.upstream_tag}-bin-win-cuda-{minor}-{arch}.zip": "published"
for minor in legacy_minors
},
preferred_runtime_line,
selection_log,
arch = arch,
)
if attempt.runtime_line
]
@ -3363,7 +3460,10 @@ def published_windows_cuda_attempts(
asset_url = release.assets.get(artifact.asset_name)
if not asset_url:
continue
am = re.search(r"-bin-win-cuda-(\d+)\.(\d+)-x64\.zip$", artifact.asset_name)
# Two groups (major, minor), for THIS arch: the x64 spelling never matched an arm64 bundle.
am = re.search(
rf"-bin-win-cuda-(\d+)\.(\d+)-{re.escape(arch)}\.zip$", artifact.asset_name
)
# Legacy upstream-named bundles encode the minor; gate it against the
# driver. app-named bundles carry no minor and are driver-gated at the
# runtime-line level by windows_cuda_attempts above.
@ -3410,7 +3510,7 @@ def published_windows_cuda_attempts(
runtime_archive_url: str | None = None
if am is not None and artifact.asset_name.startswith("llama-"):
runtime = f"{am.group(1)}.{am.group(2)}"
cudart_name = f"cudart-llama-bin-win-cuda-{runtime}-x64.zip"
cudart_name = f"cudart-llama-bin-win-cuda-{runtime}-{arch}.zip"
cudart_url = release.assets.get(cudart_name)
if cudart_url and cudart_url != asset_url:
runtime_archive_name = cudart_name
@ -3430,7 +3530,7 @@ def published_windows_cuda_attempts(
name = artifact.asset_name,
url = asset_url,
source_label = "published",
install_kind = "windows-cuda",
install_kind = install_kind,
runtime_line = runtime_line,
runtime_name = runtime_archive_name,
runtime_url = runtime_archive_url,
@ -3446,7 +3546,10 @@ def published_windows_cuda_attempts(
def resolve_windows_cuda_choices(
host: HostInfo, llama_tag: str, upstream_assets: dict[str, str]
host: HostInfo,
llama_tag: str,
upstream_assets: dict[str, str],
arch: str | None = None,
) -> list[AssetChoice]:
torch_preference = detect_torch_cuda_runtime_preference(host)
attempts = windows_cuda_attempts(
@ -3455,6 +3558,7 @@ def resolve_windows_cuda_choices(
upstream_assets,
torch_preference.runtime_line,
torch_preference.selection_log,
arch = arch,
)
return attempts
@ -3811,6 +3915,28 @@ def resolve_upstream_asset_choice(host: HostInfo, llama_tag: str) -> AssetChoice
install_kind = "windows-cpu",
)
if host.is_windows and host.is_arm64:
# CUDA first on an NVIDIA host, then the CPU bundle; without this branch every Windows ARM64 host fell through to a source build.
if host.has_usable_nvidia and _upstream_arm64_cuda_allowed():
attempts = _drop_blackwell_incapable_windows_cuda(
host,
resolve_windows_cuda_choices(host, llama_tag, upstream_assets, arch = "arm64"),
)
if attempts:
return attempts[0]
log("no compatible Windows ARM64 CUDA asset was found -- falling back to CPU")
upstream_name = f"llama-{llama_tag}-bin-win-cpu-arm64.zip"
if upstream_name not in upstream_assets:
raise PrebuiltFallback("upstream Windows ARM64 CPU asset was not found")
return AssetChoice(
repo = UPSTREAM_REPO,
tag = llama_tag,
name = upstream_name,
url = upstream_assets[upstream_name],
source_label = "upstream",
install_kind = "windows-arm64",
)
if host.is_macos and host.is_arm64:
upstream_name = f"llama-{llama_tag}-bin-macos-arm64.tar.gz"
if upstream_name not in upstream_assets:
@ -3918,7 +4044,72 @@ def resolve_release_asset_choice(
else:
published_choice = published_asset_choice_for_kind(release, "windows-cpu")
elif host.is_windows and host.is_arm64:
# Windows arm64 has no GPU prebuilt, so it always takes the CPU bundle.
# Prefer a CUDA bundle, as x64 does: the published windows-arm64-cuda artifact first, then
# upstream's -arm64 zip, both hash-gated. The opt-out gates the WHOLE branch.
if host.has_usable_nvidia and _upstream_arm64_cuda_allowed():
torch_preference = detect_torch_cuda_runtime_preference(host)
published_arm64_cuda = _drop_blackwell_incapable_windows_cuda(
host,
published_windows_cuda_attempts(
host,
release,
torch_preference.runtime_line,
torch_preference.selection_log,
arch = "arm64",
),
)
if published_arm64_cuda:
try:
return apply_approved_hashes(published_arm64_cuda, checksums)
except PrebuiltFallback as exc:
log(
"published Windows ARM64 CUDA assets ignored for install planning: "
f"{release.repo}@{release.release_tag} ({exc})"
)
# Same rule as the digest fetch below: a rate limit, an outage or a malformed payload from the
# release API costs the CUDA bundle, never the install; the ARM64 CPU bundle is still there.
try:
upstream_arm64_assets = github_release_assets(UPSTREAM_REPO, llama_tag)
except Exception as exc:
log(
f"could not list the upstream {UPSTREAM_REPO}@{llama_tag} release assets "
f"({exc}); falling through to the ARM64 CPU bundle."
)
upstream_arm64_assets = {}
upstream_arm64_cuda = _drop_blackwell_incapable_windows_cuda(
host,
resolve_windows_cuda_choices(
host,
llama_tag,
upstream_arm64_assets,
arch = "arm64",
),
)
if upstream_arm64_cuda:
try:
return apply_approved_hashes(upstream_arm64_cuda, checksums)
except PrebuiltFallback as exc:
# The fork publishes no windows-arm64-cuda bundle yet, so take upstream ggml-org's, the same release this fork
# repackages. Verified against GitHub's per-asset digest; an asset it states no digest for is refused, not installed.
verified = _apply_release_digests(
upstream_arm64_cuda,
github_release_asset_digests(UPSTREAM_REPO, llama_tag),
)
if verified:
log(
"no approved checksum covers a Windows ARM64 CUDA bundle "
f"({exc}); installing the upstream {UPSTREAM_REPO} bundle "
f"{verified[0].name}, verified against the release asset digest "
"GitHub reports. Set UNSLOTH_LLAMA_ARM64_CUDA=0 to use the CPU "
"bundle instead."
)
return verified
log(
"no approved checksum covers a Windows ARM64 CUDA bundle "
f"({exc}), and GitHub reports no asset digest for the upstream "
f"{UPSTREAM_REPO} bundle either; refusing to install it unverified "
"and falling through to the ARM64 CPU bundle."
)
published_choice = published_asset_choice_for_kind(release, "windows-arm64")
elif host.is_macos and host.is_arm64:
published_choice = published_asset_choice_for_kind(release, "macos-arm64")
@ -4283,8 +4474,10 @@ def paired_runtime_dll_patterns(choice: AssetChoice) -> list[str]:
into the install. Used for the second copy_globs pass in
install_from_archives, narrower than runtime_patterns_for_choice so
the runtime archive cannot overwrite main-archive payload like
llama-server.exe. Only Windows CUDA has paired runtimes today."""
if choice.install_kind == "windows-cuda":
llama-server.exe. Only Windows CUDA has paired runtimes today (x64 and arm64
alike: upstream ships a cudart-...-arm64.zip beside the arm64 binary bundle,
carrying the same DLL names built for ARM64)."""
if choice.install_kind in _WINDOWS_CUDA_INSTALL_KINDS:
return ["cudart64_*.dll", "cublas64_*.dll", "cublasLt64_*.dll"]
return []
@ -4325,6 +4518,7 @@ def runtime_patterns_for_install_kind(
if install_kind in {
"windows-cpu",
"windows-cuda",
"windows-arm64-cuda",
"windows-hip",
"windows-vulkan",
"windows-rocm",
@ -6073,6 +6267,7 @@ def validate_server(
"linux-rocm",
"linux-vulkan",
"windows-cuda",
"windows-arm64-cuda",
"windows-hip",
"windows-vulkan",
"windows-rocm",
@ -6405,6 +6600,34 @@ def collect_system_report(host: HostInfo, choice: AssetChoice | None, install_di
return "\n".join(lines)
def _apply_release_digests(
attempts: Iterable[AssetChoice], digests: dict[str, str]
) -> list[AssetChoice]:
"""apply_approved_hashes against GitHub's per-asset digests instead of our manifest.
Same two rules, because they are the ones that matter: an attempt with no digest is
dropped rather than installed unverified, and a paired runtime archive with no digest
unpairs rather than riding along unchecked. Returns [] when nothing survives, which the
one caller reads as "refuse and fall through", not as "install anyway".
"""
kept: list[AssetChoice] = []
for attempt in attempts:
digest = digests.get(attempt.name)
if digest is None:
continue
attempt.expected_sha256 = digest
if attempt.runtime_name and attempt.runtime_url:
runtime_digest = digests.get(attempt.runtime_name)
if runtime_digest is None:
attempt.runtime_name = None
attempt.runtime_url = None
attempt.runtime_sha256 = None
else:
attempt.runtime_sha256 = runtime_digest
kept.append(attempt)
return kept
def apply_approved_hashes(
attempts: Iterable[AssetChoice], checksums: ApprovedReleaseChecksums
) -> list[AssetChoice]:
@ -7367,7 +7590,7 @@ def runtime_payload_health_groups(
return groups
if install_kind in {"windows-cpu", "windows-arm64"}:
return _windows_shared_groups(source_label, tag)
if install_kind == "windows-cuda":
if install_kind in {"windows-cuda", "windows-arm64-cuda"}:
groups = _windows_shared_groups(source_label, tag) + [["ggml-cuda.dll"]]
# Require the complete cudart trio only when it was paired with this install.
if runtime_name:

View file

@ -27,6 +27,7 @@ import re
import sys
import time
from pathlib import Path
from urllib.parse import urlsplit
from typing import Dict, List, Optional, Sequence, Tuple
MANIFEST_NAME = "unsloth_install_manifest.json"
@ -635,6 +636,9 @@ PROTECTED_MANIFEST_KEYS: Tuple[str, ...] = (
"no_torch",
"expected_torch_tag",
"expected_torch_tag_pinned",
# Validated on write (NVIDIA's own https channel, no userinfo/query/fragment) and read back
# as an index to install from, so caller-composed evidence must not be able to name it.
"woa_torch_index",
)
@ -646,6 +650,7 @@ def write_manifest(
no_torch: Optional[bool] = None,
expected_torch_tag: Optional[str] = None,
expected_torch_tag_pinned: Optional[bool] = None,
woa_torch_index: Optional[str] = None,
extra: Optional[Dict[str, object]] = None,
) -> Optional[Path]:
"""Record a completed install. Never raises: no manifest reads as incomplete,
@ -679,6 +684,25 @@ def write_manifest(
# eGPU with no repair offered. Absent means unknown, as with every other additive key.
if expected_torch_tag_pinned is not None:
payload["expected_torch_tag_pinned"] = bool(expected_torch_tag_pinned)
# Windows on ARM has no CUDA wheels on download.pytorch.org, so a fresh shell cannot re-derive.
# Only NVIDIA's own channels, with no userinfo, query or fragment: a mirror is not persisted.
if woa_torch_index:
candidate = str(woa_torch_index).strip()
# urlsplit raises on a malformed authority, which would break the never-raises contract. Scheme and host compare case-insensitively (RFC 3986); the path keeps its case.
try:
parsed = urlsplit(candidate)
_woa_ok = (
parsed.scheme.lower() == "https"
and parsed.hostname == "pypi.nvidia.com"
and parsed.netloc.lower() == parsed.hostname
and not parsed.query
and not parsed.fragment
)
except ValueError:
_woa_ok = False
# netloc, not hostname: hostname strips ":443", so a value with a port is refused. Equality drops userinfo with it.
if _woa_ok:
payload["woa_torch_index"] = "https://pypi.nvidia.com" + parsed.path.rstrip("/")
# The dependency pass's own evidence (`pass_inputs`, `step_results`, `pip_check_ok`, ...):
# additive, never authoritative, every consumer re-verifies on disk. None is dropped so "absent
# means unknown" holds, and nothing may shadow a field verify_install reads.

View file

@ -83,15 +83,37 @@ _PLATFORM_HAS_TORCHCODEC_WHEEL = (
PLATFORM_LACKS_TORCHCODEC_WHEEL = not _PLATFORM_HAS_TORCHCODEC_WHEEL
def _machine_arch_from_registry() -> str:
"""The machine-scope PROCESSOR_ARCHITECTURE, which an emulated process cannot
misreport. Every per-process signal follows the process: under x64 emulation on
ARM64 Windows the process copy says AMD64, PROCESSOR_ARCHITEW6432 is unset (it is a
WOW64-only variable), and platform.machine() says AMD64 too. Empty when unreadable
or off Windows."""
if not IS_WINDOWS:
return ""
try:
import winreg
with winreg.OpenKey(
winreg.HKEY_LOCAL_MACHINE,
r"SYSTEM\CurrentControlSet\Control\Session Manager\Environment",
) as key:
return str(winreg.QueryValueEx(key, "PROCESSOR_ARCHITECTURE")[0] or "")
except Exception:
return ""
def _is_windows_arm64() -> bool:
"""Windows on ARM, machine arch rather than process arch: platform.machine() reports
AMD64 under an emulated x64 Python, and PROCESSOR_ARCHITEW6432 is ARM64 in exactly
that case. Mirrors Get-HostMachineArch in install.ps1 / setup.ps1."""
"""Windows on ARM, machine arch rather than process arch. The registry value leads
because the per-process signals all say AMD64 under an emulated x64 Python; they stay
as fallbacks for a native interpreter. Mirrors Get-HostMachineArch in install.ps1 /
setup.ps1. Wheel availability is an interpreter question, not a machine one: see
``_is_win_arm64_interpreter``."""
if not IS_WINDOWS:
return False
return any(
(value or "").strip().lower() in {"arm64", "aarch64"}
for value in (
_machine_arch_from_registry(),
os.environ.get("PROCESSOR_ARCHITEW6432"),
os.environ.get("PROCESSOR_ARCHITECTURE"),
platform.machine(),
@ -99,6 +121,44 @@ def _is_windows_arm64() -> bool:
)
@functools.lru_cache(maxsize = None)
def _is_win_arm64_interpreter() -> bool:
"""Windows on ARM, the arch of THIS INTERPRETER rather than of the machine.
The distinction decides which wheels exist. ``_is_windows_arm64`` above answers
for the machine, and is true even under an emulated x64 Python -- which is what
every install predating native ARM64 support is running, because install.ps1
deliberately fetched an x64 interpreter there. Such a venv wants the x64 wheels
and gets them: ``platform_machine == "ARM64"`` in a requirement marker is the
INTERPRETER's arch, so the marker rows and this predicate have to agree or the
same machine is served two different answers.
``sysconfig.get_platform()`` is what pip and uv tag wheels with, so it is the
same authority; ``platform.machine()`` is the fallback and reports AMD64 under
emulation, which is the answer we want there.
"""
if not IS_WINDOWS:
return False
try:
tag = (sysconfig.get_platform() or "").strip().lower()
if tag:
return tag == "win-arm64"
except Exception:
pass
return (platform.machine() or "").strip().lower() in {"arm64", "aarch64"}
def _windows_arm64_has_torchaudio() -> bool:
"""Does the CUDA index this install used publish a win_arm64 torchaudio?
NVIDIA's GA out-of-tree channel does (2.11.0+cu134); its nightly channel and
download.pytorch.org do not. install.ps1 answers this in UNSLOTH_WOA_HAS_TORCHAUDIO.
Unset means "assume not": asking for a wheel that does not exist makes the whole trio
unresolvable, while skipping one that does costs only audio support.
"""
return (os.environ.get("UNSLOTH_WOA_HAS_TORCHAUDIO") or "").strip() == "1"
# ── ROCm / AMD GPU support ─────────────────────────────────────────────────────
_ROCM_TORCH_INDEX: dict[tuple[int, int], str] = {
(7, 2): "rocm7.2", # torch 2.11.0
@ -3119,8 +3179,7 @@ def _hsa_spoofed_physical_gfx(
re-probe."""
if physical == [inferred_gfx]:
_safe_print(
f" {source} reports {inferred_gfx} -- {probed} is a spoof of the "
f"physical arch.\n"
f" {source} reports {inferred_gfx} -- {probed} is a spoof of the physical arch.\n"
)
return inferred_gfx
# Say so rather than leaving "Checking whether..." hanging: on a real gfx1100
@ -3937,8 +3996,7 @@ def _ensure_cuda_torch() -> None:
if _target_span is None or not _span_covers(_target_span, _sms):
return
_why = (
f"torch is {_family} but this host has GPUs outside its "
f"sm_{_span[0]}-{_span[1]} range"
f"torch is {_family} but this host has GPUs outside its sm_{_span[0]}-{_span[1]} range"
)
elif (
_marker == "cpu"
@ -4971,6 +5029,10 @@ def _ensure_expected_torch_flavor(expected: "str | None" = None) -> bool:
"""
if NO_TORCH:
return True
if _is_win_arm64_interpreter() and _explicit_torch_index_url() is None:
_installed = _probe_installed_torch_version()
if _installed and _is_cuda_family_leaf(_torch_flavor_tag(_installed)):
return True
# rocm/xpu/cpu fall THROUGH: an explicit GPU pin sets _TORCH_BACKEND, and rejecting it
# here would skip the invariant on the hosts that asked for that family.
if _TORCH_BACKEND not in ("", "cuda", "rocm", "xpu", "cpu"):
@ -5045,9 +5107,9 @@ def _ensure_expected_torch_flavor(expected: "str | None" = None) -> bool:
_torch_pkg, _vision_pkg, _audio_pkg = (
_XPU_TORCH_PKG_SPEC if expected == "xpu" else _TORCH_FLAVOR_REPAIR_PKG_SPEC
)
# No win_arm64 torchaudio wheel exists on any index ($WinArm64NoAudio in setup.ps1).
# Keyed on the INTERPRETER, not the machine: an emulated x64 venv installs win_amd64 wheels.
_trio = [_torch_pkg, _vision_pkg, _audio_pkg]
if _is_windows_arm64():
if _is_win_arm64_interpreter() and not _windows_arm64_has_torchaudio():
_trio = [_torch_pkg, _vision_pkg]
_label_before = str(installed_version)
# --force-reinstall, not install.ps1's uv-only --reinstall-package: pip_install falls
@ -5392,10 +5454,8 @@ def _ensure_rocm_torch() -> None:
_torch_pkg, _vision_pkg, _audio_pkg = _WINDOWS_ROCM_TORCH_PKG_SPECS.get(
gfx_arch, ("torch", "torchvision", "torchaudio")
)
# Same win_arm64 exception setup.ps1 applies: no torchaudio wheel exists
# there, so asking for one makes the trio unresolvable.
_rocm_trio = [_torch_pkg, _vision_pkg, _audio_pkg]
if _is_windows_arm64():
if _is_win_arm64_interpreter():
_rocm_trio = [_torch_pkg, _vision_pkg]
# Nonfatal: a transient AMD-index failure must not abort the install.
# --force-reinstall resolves before uninstalling, so a failed index keeps the
@ -6392,6 +6452,35 @@ def run(
return result
# First line of the overrides file install.ps1 generates; setup.ps1's merge copies it first.
WOA_OVERRIDES_HEADER = "# Generated by install.ps1 for Windows on ARM"
def _woa_overrides_are_load_bearing() -> bool:
"""Is this a native win_arm64 resolve whose correctness depends on UV_OVERRIDE?
install.ps1 writes those overrides to lift the released torch cap -- no win_arm64 CUDA
wheel satisfies it -- and to drop the packages with no win_arm64 build. pip has no
override mechanism at all: constraints only narrow a requirement, they cannot replace
one, so there is nothing to translate them into. Falling back to pip on this stack does
not recover, it silently resolves the wrong thing.
Judged by the generated file, not by the variable: a caller's own override on a run that
never configured the stack (--no-torch, a direct run) keeps the fallback every host has.
"""
if not _is_win_arm64_interpreter():
return False
for path in os.environ.get("UV_OVERRIDE", "").split():
try:
with open(path, encoding = "utf-8", errors = "replace") as handle:
first = handle.readline()
except OSError:
continue
if first.lstrip("\ufeff").startswith(WOA_OVERRIDES_HEADER):
return True
return False
def _report_failed_command(label: str, result: subprocess.CompletedProcess[bytes]) -> None:
"""Print a failed command's redacted output and exit with its code."""
_step("error", f"{label} failed (exit code {result.returncode})", _red)
@ -6458,6 +6547,706 @@ def _purge_recordless_distributions(output: "bytes | str | None") -> list[str]:
# Packages to skip on Windows (require special build steps)
WINDOWS_SKIP_PACKAGES = {"triton_kernels"}
# No win_arm64 wheel and no sdist buildable without MSVC / Rust / LLVM / FFmpeg. All optional:
# tensorboard needs grpcio; librosa and openai-whisper need numba -> llvmlite (cp314 only).
# Lowercase entries only: _filter_requirements lowercases the line and compares verbatim.
WINDOWS_ARM64_SKIP_PACKAGES = {
"mecab",
"sqlite-vec",
"tiktoken",
"tensorboard",
"librosa",
"openai-whisper",
"torch-c-dlpack-ext",
"pytorch_tokenizers",
"hf_transfer",
"xformers",
}
def _wheel_matches_interpreter(filename: str) -> bool:
"""Can THIS interpreter install the wheel named ``filename``?
A wheelhouse is not built for one interpreter: install.ps1 stages every win_arm64
wheel it finds, cp311 through cp314, as the wheelhouse published them. So a filename
is not proof on its own -- a cp311 tiktoken is invisible to a cp313 resolver, and
counting it as available drops the skip and sends the resolve to an sdist that cannot
build here. PEP 425: a wheel is installable when one of its (python, abi, platform)
triples is one the interpreter supports, and each filename field may be a
"."-separated set expanded as their cartesian product. Unparseable means not
installable, which only leaves the conservative skip in place.
"""
stem = filename[:-4] if filename.lower().endswith(".whl") else filename
parts = stem.split("-")
if len(parts) < 5:
return False
py_tags, abi_tags, plat_tags = (set(field.split(".")) for field in parts[-3:])
this_platform = (sysconfig.get_platform() or "").replace("-", "_").replace(".", "_").lower()
if "any" not in plat_tags and this_platform not in plat_tags:
return False
major, minor = sys.version_info[:2]
free_threaded = bool(sysconfig.get_config_var("Py_GIL_DISABLED"))
this_cpython = f"cp{major}{minor}"
this_abi = f"{this_cpython}t" if free_threaded else this_cpython
for py_tag in py_tags:
for abi_tag in abi_tags:
# abi3 excluded HERE too: this branch shadows the one below (CPython #111506).
if py_tag == this_cpython and (
abi_tag in ("none", this_abi) or (abi_tag == "abi3" and not free_threaded)
):
return True
if abi_tag == "none":
pure = re.fullmatch(r"py(\d)(\d*)", py_tag)
if pure and int(pure.group(1)) == major and int(pure.group(2) or 0) <= minor:
return True
elif abi_tag == "abi3" and not free_threaded:
stable = re.fullmatch(r"cp(\d)(\d+)", py_tag)
if stable and int(stable.group(1)) == major and 2 <= int(stable.group(2)) <= minor:
return True
return False
@functools.lru_cache(maxsize = 1)
def _find_links_wheel_versions() -> "dict[str, frozenset[str]]":
"""Canonical name -> versions of the wheels in the configured find-links directories.
install.ps1 points UV_FIND_LINKS / PIP_FIND_LINKS at a local wheelhouse on Windows on
ARM, holding the wheels PyPI does not publish for win_arm64. Anything served there is
installable, so it must not also be filtered out as unavailable -- but only the wheels
tagged for this interpreter are, which is what the resolver will agree to.
The VERSIONS come back too, not just the names: every requirement these gate is
``==``-pinned, and a wheelhouse holding tiktoken 0.12.0 against a ``tiktoken==0.13.0``
line satisfies nothing -- the resolver goes to PyPI, finds no win_arm64 wheel for the
pinned version, and falls to an sdist that cannot build here. That is the exact
failure the skip list exists to prevent, so the caller checks the pin.
"""
versions: "dict[str, set[str]]" = {}
# UV_FIND_LINKS ONLY: uv does not consume PIP_FIND_LINKS. Comma-split, the way uv reads it.
for value, separator in ((os.environ.get("UV_FIND_LINKS"), ","),):
for entry in re.split(separator, value or ""):
entry = entry.strip().strip('"')
if not entry or "://" in entry:
continue # a URL index cannot be listed cheaply; treat it as unknown
try:
for wheel in Path(entry).glob("*.whl"):
if not _wheel_matches_interpreter(wheel.name):
continue
fields = wheel.name[:-4].split("-")
if len(fields) < 5:
continue # not a wheel filename; _wheel_matches_interpreter agrees
name = _canonical_dist_name(fields[0])
versions.setdefault(name, set()).add(fields[1])
except OSError:
continue
return {name: frozenset(vers) for name, vers in versions.items()}
def _find_links_wheel_names() -> frozenset[str]:
"""Just the names from :func:`_find_links_wheel_versions`."""
return frozenset(_find_links_wheel_versions())
_find_links_wheel_names.cache_clear = _find_links_wheel_versions.cache_clear # type: ignore[attr-defined]
def _parse_release(version: str) -> "tuple[int, ...] | None":
"""The numeric release segment of a PEP 440 version, or None when it has none."""
match = re.match(r"^\s*v?(\d+(?:\.\d+)*)", version or "")
if not match:
return None
return tuple(int(part) for part in match.group(1).split("."))
def _version_satisfies(version: str, specifier: str) -> "bool | None":
"""Does ``version`` satisfy the PEP 440 specifier set ``specifier``?
None means "cannot tell" -- an epoch, an arbitrary-equality clause, anything this
deliberately small comparison does not model. The caller treats that as it treated
every version before this existed, so an exotic pin is no worse off than it was.
"""
specifier = (specifier or "").strip()
if not specifier:
return True
for module_name in ("packaging.specifiers", "pip._vendor.packaging.specifiers"):
try:
module = importlib.import_module(module_name)
except Exception:
continue
try:
spec_set = module.SpecifierSet(specifier)
# Prereleases off unless the specifier names one; packaging's default admits 0.13.0rc1.
return bool(spec_set.contains(version, prereleases = bool(spec_set.prereleases)))
except Exception:
break
if not re.fullmatch(r"\s*v?\d+(?:\.\d+)*\s*", version or ""):
return False
got = _parse_release(version)
# "!" that is not part of "!=" is a PEP 440 epoch, which _parse_release does not model.
if got is None or "!" in (version or "") or "!" in specifier.replace("!=", ""):
return None
for clause in specifier.split(","):
clause = clause.strip()
if not clause:
continue
match = re.fullmatch(r"(==|!=|>=|<=|~=|>|<)\s*([^\s]+)", clause)
if not match:
return None
op, want_raw = match.group(1), match.group(2).rstrip(".*")
wildcard = match.group(2).endswith(".*")
want = _parse_release(want_raw)
if want is None:
return None
# Pad to a common length so 2.11 and 2.11.0 compare equal, as PEP 440 says.
width = max(len(got), len(want))
lhs = got + (0,) * (width - len(got))
rhs = want + (0,) * (width - len(want))
if wildcard:
# ==1.2.* / !=1.2.*: only the prefix is compared.
prefix = got[: len(want)] + (0,) * max(0, len(want) - len(got))
ok = prefix == want
if op == "==":
pass
elif op == "!=":
ok = not ok
else:
return None
elif op == "==":
ok = lhs == rhs
elif op == "!=":
ok = lhs != rhs
elif op == ">=":
ok = lhs >= rhs
elif op == "<=":
ok = lhs <= rhs
elif op == ">":
ok = lhs > rhs
elif op == "<":
ok = lhs < rhs
else: # ~=X.Y[.Z] is ">=X.Y[.Z], ==X.Y.*" one level up
if len(want) < 2:
return None
ok = lhs >= rhs and got[: len(want) - 1] == want[: len(want) - 1]
if not ok:
return False
return True
def _marker_is_active(marker: str) -> "bool | None":
"""Does ``marker`` hold for THIS interpreter? None when it cannot be decided.
packaging is what pip and uv evaluate markers with, so borrowing it keeps the answer
identical to the resolver's; pip vendors a copy, which is the fallback for a venv
where packaging is not installed in its own right. Neither available means the
caller keeps every clause instead of picking one.
"""
if not marker:
return True
for module_name in ("packaging.markers", "pip._vendor.packaging.markers"):
try:
module = importlib.import_module(module_name)
except Exception:
continue
try:
return bool(module.Marker(marker).evaluate())
except Exception:
return None
return None
def _requirement_pins(req: "Path | None") -> "dict[str, list[str]]":
"""Canonical name -> the version specifiers a requirements file states for it.
A name can appear more than once, split by marker -- extras.txt carries
``MeCab==0.996.13`` and ``MeCab==0.996.5`` on complementary markers -- so keeping one
specifier per name let the row for another platform overwrite the row that actually
applies. Markers are evaluated for this interpreter and inactive rows dropped;
when they cannot be evaluated every clause is kept, and the caller takes any of them
as satisfied, which is no stricter than the name-only check that came before.
"""
pins: "dict[str, list[str]]" = {}
if req is None:
return pins
try:
text = req.read_text(encoding = "utf-8-sig")
except OSError:
return pins
for line in text.splitlines():
line = line.split("#", 1)[0].strip()
if not line or line.startswith("-") or "@" in line:
continue
line, _, marker = line.partition(";")
if _marker_is_active(marker.strip()) is False:
continue
line = line.strip()
match = re.match(r"^([A-Za-z0-9][A-Za-z0-9._-]*)\s*(?:\[[^\]]*\])?\s*(.*)$", line)
if not match:
continue
pins.setdefault(_canonical_dist_name(match.group(1)), []).append(match.group(2).strip())
return pins
# Skipped for their DEPENDENCIES: whisper's metadata needs tiktoken unconditionally, so filtering the direct line does not stop the sdist arriving transitively.
WINDOWS_ARM64_SKIP_UNBLOCKED_BY = {
"tensorboard": ("grpcio",),
# soxr as well as the numba pair: librosa 0.11.0 requires soxr>=0.3.2 and soxr has never published a win_arm64 wheel, so un-skipping librosa would build it from an sdist.
"librosa": ("llvmlite", "numba", "soxr"),
"openai_whisper": ("llvmlite", "numba", "tiktoken"),
}
# The blocker versions the packages' OWN metadata demands: too old, and the extras pass fails.
# {blocker: (specifier, package it was read from, that package's pinned version)}. llvmlite arrives through numba.
WINDOWS_ARM64_BLOCKER_FLOORS: "dict[str, tuple[str, str, str]]" = {
"grpcio": (">=1.74.0", "tensorboard", "2.21.0"),
"numba": (">=0.51.0", "librosa", "0.11.0"),
"soxr": (">=0.3.2", "librosa", "0.11.0"),
}
# Excluded on win_arm64 by MARKER, so a hosted wheel has no requirement to satisfy. Floors given, because --no-deps otherwise takes whatever happens to be hosted.
WINDOWS_ARM64_WHEELHOUSE_OPTIONALS = {
"hf-transfer": "",
"xformers": ">=0.0.22.post7",
"sqlite-vec": "",
}
def _wheelhouse_hosts(name: str) -> bool:
"""Does the resolver's own find-links carry a wheel for this distribution?"""
return bool(_find_links_wheel_versions().get(_canonical_dist_name(name)))
def _wheelhouse_best_version(name: str, floor: str) -> "str | None":
"""The newest hosted version that clears ``floor``, or None if none does.
An unreadable comparison keeps the answer it would have had before this floor
existed, matching _windows_arm64_skip_packages: an exotic version is no worse off.
"""
hosted = _find_links_wheel_versions().get(_canonical_dist_name(name)) or ()
usable = [
version
for version in hosted
if not floor or _version_satisfies(version, floor) is not False
]
if not usable:
return None
try:
from packaging.version import Version
return max(usable, key = Version)
except Exception:
return sorted(usable)[-1]
def _wheelhouse_torchcodec_version(torch_version: "str | None") -> "str | None":
"""The hosted torchcodec inside the window this torch selects, or None.
win_arm64 only: elsewhere the platform's own wheel decides. An unknown torch has no
window, so it gets no answer either.
"""
if not _is_win_arm64_interpreter() or not torch_version:
return None
spec = _select_torchcodec_spec(torch_version)
if spec is None:
return None
return _wheelhouse_best_version("torchcodec", spec.split("torchcodec", 1)[1])
def _install_wheelhouse_optionals() -> None:
"""Install the hosted optionals the metadata cannot ask for. Best effort.
--no-deps: the graph is already resolved and installed by the time this runs, and
xformers names torch, so resolving here could walk the win_arm64 CUDA build off to
whatever PyPI offers. A failure leaves the feature off, which is where it was.
Pinned to the selected version rather than installed by bare name, so the floor
checked here is the version that actually lands.
"""
if not _is_win_arm64_interpreter():
return
for name, floor in WINDOWS_ARM64_WHEELHOUSE_OPTIONALS.items():
version = _wheelhouse_best_version(name, floor)
if version is None:
if _wheelhouse_hosts(name):
_note(f"windows on arm: the wheelhouse {name} is below {floor}; leaving it off")
# Nothing to refresh with, but the copy an earlier run installed is still resident.
if _canonical_dist_name(name) == "xformers":
_evict_xformers_built_for_another_torch()
continue
installed = pip_install_try(
f"Installing {name}=={version} from the Windows on ARM wheelhouse",
"--no-deps",
"--no-cache-dir",
f"{name}=={version}",
constrain = False,
)
if not installed:
_note(f"windows on arm: could not install the wheelhouse {name}; feature stays off")
# Checked even when the refresh failed: the copy an earlier torch left behind is still resident.
if _canonical_dist_name(name) == "xformers" and _evict_xformers_built_for_another_torch():
continue
if not installed:
continue
_note(f"windows on arm: installed {name}=={version} from the wheelhouse")
def _evict_xformers_built_for_another_torch() -> bool:
"""Remove a resident xFormers whose extension was built against another torch. True iff removed.
xFormers links its extension against ONE (torch, CUDA) pair; beside any other it is mute,
and a package install never uninstalls what an earlier run left behind.
"""
built_for = _resident_xformers_build_torch()
resident = str(_probe_installed_torch_version() or "")
if not (built_for and resident and built_for != resident):
return False
_uninstall_distribution("xformers")
_note(
f"windows on arm: the wheelhouse xformers was built for torch "
f"{built_for}, not {resident} -- removed; attention uses torch SDPA"
)
return True
WINDOWS_ARM64_PUBLIC_INDEX_WHEELS: "dict[str, dict[str, str]]" = {
"llvmlite": {"cp314": "0.49.0"},
"numba": {"cp314": "0.67.0"},
}
def _uv_config_files() -> "list[tuple[Path, str]]":
"""The persistent configuration uv would discover, most specific first, as (path, table).
uv reads uv.toml or pyproject.toml [tool.uv] from the current directory or the nearest
parent (uv.toml wins over pyproject.toml in the same directory), then the user file
(%APPDATA%\\uv\\uv.toml on Windows, $XDG_CONFIG_HOME/uv/uv.toml elsewhere), then the
system file (%PROGRAMDATA%\\uv\\uv.toml, /etc/uv/uv.toml). UV_CONFIG_FILE names one file
instead of discovering; UV_NO_CONFIG discovers nothing. `table` is the prefix the index
keys sit under: "" for uv.toml, "tool.uv" for pyproject.toml.
"""
if os.environ.get("UV_NO_CONFIG", "").strip().lower() not in ("", "0", "false"):
return []
explicit = os.environ.get("UV_CONFIG_FILE", "").strip()
if explicit:
return [(Path(explicit), "")]
found: "list[tuple[Path, str]]" = []
here = Path.cwd()
for d in (here, *here.parents):
uv_toml = d / "uv.toml"
if uv_toml.is_file():
found.append((uv_toml, ""))
break
pyproject = d / "pyproject.toml"
if pyproject.is_file():
try:
text = pyproject.read_text(encoding = "utf-8")
except OSError:
text = ""
if re.search(r"(?m)^\s*\[+tool\.uv(\.|\])", text):
found.append((pyproject, "tool.uv"))
break
if IS_WINDOWS:
user = os.environ.get("APPDATA", "")
system = os.environ.get("PROGRAMDATA", "")
if user:
found.append((Path(user) / "uv" / "uv.toml", ""))
if system:
found.append((Path(system) / "uv" / "uv.toml", ""))
else:
xdg = os.environ.get("XDG_CONFIG_HOME", "") or str(Path.home() / ".config")
found.append((Path(xdg) / "uv" / "uv.toml", ""))
found.append((Path("/etc/uv/uv.toml"), ""))
return [(p, table) for p, table in found if p.is_file()]
def _uv_config_index_policy() -> "dict[str, object]":
"""{no_index, default_index, unreadable} from uv's discovered configuration.
Project outranks user outranks system for a scalar, so the first file that sets a key
decides it. Only the keys that decide where a resolve looks are read: no-index and
default-index (index-url is the older spelling), at the top level and under [pip], and
an [[index]] entry carrying default = true. A file this cannot parse is reported rather
than guessed at.
"""
policy: "dict[str, object]" = {
"no_index": None,
"default_index": None,
"unreadable": False,
"extra_indexes": [],
}
try:
import tomllib
except ImportError: # 3.10: the native path is 3.11+, so only the x64 fallback lands here
policy["unreadable"] = bool(_uv_config_files())
return policy
for path, table in _uv_config_files():
try:
with open(path, "rb") as fh:
data = tomllib.load(fh)
except (OSError, ValueError):
policy["unreadable"] = True
continue
section = data
for part in [p for p in table.split(".") if p]:
section = section.get(part, {}) if isinstance(section, dict) else {}
if not isinstance(section, dict):
continue
# uv pip (0.10.7): [pip] scalars beat top-level, and [[index]] default = true beats both.
pip_scope = section.get("pip", {}) if isinstance(section.get("pip"), dict) else {}
file_no_index = None
for scope in (pip_scope, section):
if file_no_index is None and isinstance(scope.get("no-index"), bool):
file_no_index = scope["no-index"]
file_default = None
extras: list[str] = []
indexes = section.get("index")
if isinstance(indexes, list):
for entry in indexes:
if not isinstance(entry, dict) or not isinstance(entry.get("url"), str):
continue
if entry.get("explicit") is True:
# uv: explicit serves only packages pinned via [tool.uv.sources]; with default = true it also removes PyPI as the default (not modelled: doubt).
if entry.get("default") is True:
policy["unreadable"] = True
continue
if entry.get("default") is True:
if file_default is None:
file_default = entry["url"]
else:
extras.append(entry["url"])
for scope in (pip_scope, section):
value = scope.get("extra-index-url")
if isinstance(value, str):
extras.append(value)
elif isinstance(value, list):
extras.extend(v for v in value if isinstance(v, str))
policy["extra_indexes"] = list(policy["extra_indexes"]) + extras
if file_default is None:
for scope in (pip_scope, section):
for key in ("default-index", "index-url"):
if file_default is None and isinstance(scope.get(key), str):
file_default = scope[key]
if policy["no_index"] is None and file_no_index is not None:
policy["no_index"] = file_no_index
if policy["default_index"] is None and file_default is not None:
policy["default_index"] = file_default
return policy
def _url_is_public_pypi(url: str) -> bool:
"""The host, not a substring: "https://pypi.org.corp.example/simple" and
".../api/pypi/pypi.org/simple" both contain the name and neither is public PyPI."""
try:
from urllib.parse import urlsplit
host = urlsplit(url.strip()).hostname
except ValueError:
return False
return host is not None and host.lower() == "pypi.org"
def _public_pypi_is_reachable() -> bool:
"""Can this resolution actually reach public PyPI?
The table below records what PyPI publishes, which is only availability if PyPI is where
the resolve will look. Offline, or pointed at an exclusive corporate index, those wheels
are neither cached nor served: unblocking librosa there drops the skip and then fails the
whole extras pass on an unavailable numba, which is exactly what the skip prevents.
Judged for the resolver that runs the pass. uv reads UV_* and its configuration files and
ignores PIP_*; pip reads PIP_* and ignores UV_*. Mixing the two reported PyPI reachable
from a PIP_EXTRA_INDEX_URL that uv, the resolver in use, never consults. A default index
REPLACES PyPI; an extra index adds to it, so PyPI named there is still consulted.
Environment outranks uv's configuration files. Doubt resolves to False: that answer keeps
the skip, the other fails the extras pass.
"""
if USE_UV:
return _uv_reaches_public_pypi()
return _pip_reaches_public_pypi()
def _env_flag(name: str) -> bool:
return os.environ.get(name, "").strip().lower() not in ("", "0", "false")
def _uv_reaches_public_pypi() -> bool:
if _uv_is_offline() or _env_flag("UV_NO_INDEX"):
return False
extra_is_pypi = any(
_url_is_public_pypi(u)
for var in ("UV_INDEX", "UV_EXTRA_INDEX_URL")
for u in os.environ.get(var, "").split()
)
for var in ("UV_DEFAULT_INDEX", "UV_INDEX_URL"):
value = os.environ.get(var, "").strip()
if value:
return extra_is_pypi or _url_is_public_pypi(value)
policy = _uv_config_index_policy()
if policy["unreadable"] or policy["no_index"] is True:
return False
extra_is_pypi = extra_is_pypi or any(
_url_is_public_pypi(u) for u in policy["extra_indexes"] if isinstance(u, str)
)
default = policy["default_index"]
if isinstance(default, str) and not _url_is_public_pypi(default):
return extra_is_pypi
return True
def _pip_reaches_public_pypi() -> bool:
"""pip's policy: its environment first, then the configuration files it would read.
PIP_* outranks every file. Below that, `pip config list` reports the effective values from
the site, user and global files, an `[install]` key outranking its `[global]` twin for an
install. A `no-index` or an exclusive `index-url` set there replaces PyPI just as the
environment does. Doubt (a `pip config` that cannot be read) keeps the skip.
"""
if _env_flag("PIP_NO_INDEX"):
return False
extra_is_pypi = any(
_url_is_public_pypi(u) for u in os.environ.get("PIP_EXTRA_INDEX_URL", "").split()
)
value = os.environ.get("PIP_INDEX_URL", "").strip()
if value:
return extra_is_pypi or _url_is_public_pypi(value)
policy = _pip_config_index_policy()
if policy["unreadable"] or policy["no_index"] is True:
return False
extra_is_pypi = extra_is_pypi or any(_url_is_public_pypi(u) for u in policy["extra_index_urls"])
index_url = policy["index_url"]
if isinstance(index_url, str) and not _url_is_public_pypi(index_url):
return extra_is_pypi
return True
def _pip_config_index_policy() -> "dict[str, object]":
"""The index keys pip's configuration files set, read from `pip config list`.
Lines are `<section>.<key>='<value>'`; `:env:` entries mirror PIP_* variables the caller
already read, so they are skipped. `install.<key>` outranks `global.<key>`, as it does for
pip itself. A `pip config` that cannot run or be parsed is reported unreadable.
"""
policy: "dict[str, object]" = {
"no_index": None,
"index_url": None,
"extra_index_urls": [],
"unreadable": False,
}
try:
done = subprocess.run(
[sys.executable, "-m", "pip", "config", "list"],
capture_output = True,
text = True,
timeout = 60,
**_windows_hidden_subprocess_kwargs(),
)
except (OSError, subprocess.SubprocessError):
policy["unreadable"] = True
return policy
if done.returncode != 0:
policy["unreadable"] = True
return policy
found: "dict[str, dict[str, str]]" = {"global": {}, "install": {}}
for line in done.stdout.splitlines():
m = re.match(r"^(global|install)\.([a-z-]+)=(.*)$", line.strip())
if not m:
continue
section, key, raw = m.groups()
raw = raw.strip()
if len(raw) >= 2 and raw[0] == raw[-1] and raw[0] in "'\"":
raw = raw[1:-1]
found[section][key] = raw
for section in ("global", "install"):
keys = found[section]
if "no-index" in keys:
policy["no_index"] = keys["no-index"].strip().lower() in ("1", "true", "yes", "on")
if keys.get("index-url", "").strip():
policy["index_url"] = keys["index-url"].strip()
if "extra-index-url" in keys:
# pip prints the repr, so a multi-line value arrives with a literal backslash-n.
policy["extra_index_urls"] = [
u for u in re.split(r"\s+|\\n", keys["extra-index-url"]) if u
]
return policy
def _woa_pypi_provided_versions() -> "dict[str, set[str]]":
"""{canonical name: versions} install.ps1 found PyPI publishing for THIS interpreter.
UNSLOTH_WOA_PYPI_PROVIDED carries space-separated `name==version` entries: the wheelhouse
wheels install.ps1 discarded because PyPI serves the same version. The managed copy is
gone, so find-links no longer answers for them; this is the only record that they resolve.
"""
provided: "dict[str, set[str]]" = {}
for entry in os.environ.get("UNSLOTH_WOA_PYPI_PROVIDED", "").split():
name, sep, version = entry.partition("==")
if sep and name and version:
provided.setdefault(_canonical_dist_name(name), set()).add(version)
return provided
def _public_index_win_arm64_versions(canonical: str) -> "set[str]":
"""Versions the public index publishes a usable win_arm64 wheel of, for THIS build.
Empty off win_arm64, empty when the resolve cannot reach PyPI, and empty for an
interpreter the recorded wheel is not tagged for. The tag is judged by
_wheel_matches_interpreter rather than by comparing strings, so a free-threaded build does
not claim a wheel built for the GIL one. install.ps1's own probe is added on top.
"""
if not _is_win_arm64_interpreter() or not _public_pypi_is_reachable():
return set()
return {
version
for tag, version in WINDOWS_ARM64_PUBLIC_INDEX_WHEELS.get(canonical, {}).items()
if _wheel_matches_interpreter(f"{canonical}-{version}-{tag}-{tag}-win_arm64.whl")
} | _woa_pypi_provided_versions().get(_canonical_dist_name(canonical), set())
def _windows_arm64_skip_packages(req: "Path | None" = None) -> set[str]:
"""WINDOWS_ARM64_SKIP_PACKAGES minus whatever the wheelhouse already provides, so
hosting a wheel is all it takes to re-enable one of these features here.
``req`` is the requirements file about to be installed, when there is one. Its pins
decide whether a hosted wheel is actually usable: a name match is not enough, because
the resolver has to honour ``tiktoken==0.13.0`` and a staged 0.12.0 wheel leaves it
with the unbuildable sdist rather than the skip this list is here to keep.
"""
available = _find_links_wheel_versions()
if not available and not any(
_public_index_win_arm64_versions(name)
for name in set(WINDOWS_ARM64_PUBLIC_INDEX_WHEELS) | set(_woa_pypi_provided_versions())
):
return set(WINDOWS_ARM64_SKIP_PACKAGES)
pins = _requirement_pins(req)
def hosted(name: str) -> bool:
canonical = _canonical_dist_name(name)
versions = set(available.get(canonical) or ()) | _public_index_win_arm64_versions(canonical)
if not versions:
return False
clauses = [clause for clause in pins.get(canonical, []) if clause]
if not clauses:
floor = WINDOWS_ARM64_BLOCKER_FLOORS.get(canonical)
if floor is None:
return True
clauses = [floor[0]]
verdicts = [_version_satisfies(v, c) for v in versions for c in clauses]
if any(verdict is True for verdict in verdicts):
return True
return any(verdict is None for verdict in verdicts)
keep_skipping: set[str] = set()
for package in WINDOWS_ARM64_SKIP_PACKAGES:
canonical = _canonical_dist_name(package)
blockers = WINDOWS_ARM64_SKIP_UNBLOCKED_BY.get(canonical)
if blockers:
if all(hosted(b) for b in blockers):
continue
elif hosted(package):
continue
keep_skipping.add(package)
return keep_skipping
# Skipped without torch (Intel Mac GGUF-only), plus librosa, whose numba chain fails (#5046).
NO_TORCH_SKIP_PACKAGES = {
"torch-stoi",
@ -8285,10 +9074,39 @@ def pip_install(
if VERBOSE and result.stdout:
_safe_print(_redact_install_output(result.stdout))
return
if _woa_overrides_are_load_bearing():
_step("error", f"{label} failed and pip cannot stand in for it", _red)
_safe_print(
_red(
" The Windows on ARM stack resolves through UV_OVERRIDE, which pip has no "
"equivalent for: overrides REPLACE a requirement, and pip constraints can "
"only narrow one."
)
)
_safe_print(
_red(
" Falling back here would honour the released torch cap, which no "
"win_arm64 CUDA wheel satisfies, and pull back the packages that have no "
"win_arm64 build at all -- downgrading a working CUDA torch or failing "
"later, with nothing to say why."
)
)
_safe_print(_red(" Install uv and re-run, or re-run install.ps1."))
_report_failed_command(label, result)
_safe_print(_red(f" uv failed, falling back to pip..."))
if result.stdout:
_safe_print(_redact_install_output(result.stdout))
elif _woa_overrides_are_load_bearing():
_step("error", f"{label} needs uv on the Windows on ARM stack", _red)
_safe_print(
_red(
" The native ARM64 resolve depends on UV_OVERRIDE, which pip cannot express. "
"Install uv and re-run, or re-run install.ps1."
)
)
sys.exit(1)
pip_cmd, pip_env = _pinned_cmd_and_env(
_build_pip_cmd(args) + constraint_args_pip + req_args_pip
)
@ -8880,11 +9698,19 @@ def _effective_requirements(req: Path) -> "tuple[Path, list[Path]]":
if IS_WINDOWS and WINDOWS_SKIP_PACKAGES:
actual = _filter_requirements(actual, WINDOWS_SKIP_PACKAGES)
temps.append(actual)
if _is_win_arm64_interpreter():
# Judged against the ORIGINAL file: the pins decide whether a hosted wheel is usable,
# and a copy already filtered above no longer carries the rows they sit on.
arm64_skip = _windows_arm64_skip_packages(req)
if arm64_skip:
actual = _filter_requirements(actual, arm64_skip)
temps.append(actual)
if NO_TORCH and NO_TORCH_SKIP_PACKAGES:
actual = _filter_requirements(actual, NO_TORCH_SKIP_PACKAGES)
temps.append(actual)
if PLATFORM_LACKS_TORCHCODEC_WHEEL:
# Linux aarch64 / Windows ARM64 / Intel Mac have no torchcodec wheel.
if PLATFORM_LACKS_TORCHCODEC_WHEEL and not _wheelhouse_hosts("torchcodec"):
# Linux aarch64 / Windows ARM64 / Intel Mac have no torchcodec wheel, unless the
# win_arm64 wheelhouse hosts one -- step 13b installs that copy itself.
actual = _filter_requirements(actual, {"torchcodec"})
temps.append(actual)
return actual, temps
@ -10032,6 +10858,10 @@ def install_python_stack() -> int:
# beside a physical GPU.
torch_flavor_tag = _expected_torch_flavor_tag()
# 13x. Optional win_arm64 features whose released metadata excludes them outright. After the
# invariant above, so xformers is validated against the torch the install ends with.
_install_wheelhouse_optionals()
# 13b. torchcodec, pinned to the venv's torch minor (_select_torchcodec_spec), which
# extras-no-deps.txt cannot do because markers cannot see torch. Must run after the
# repair above: that can move torch onto another minor, staling an earlier choice.
@ -10039,14 +10869,38 @@ def install_python_stack() -> int:
# tolerate), so read the installed metadata before giving up: guessing here means
# downgrading a matching codec onto the default and recreating the mismatch.
_codec_torch_ver = None
if not NO_TORCH and not PLATFORM_LACKS_TORCHCODEC_WHEEL:
_codec_hosted = None
if not NO_TORCH and (not PLATFORM_LACKS_TORCHCODEC_WHEEL or _wheelhouse_hosts("torchcodec")):
_codec_torch_ver = _probe_installed_torch_version() or _installed_distribution_version(
"torch"
)
if not NO_TORCH and PLATFORM_LACKS_TORCHCODEC_WHEEL:
_codec_hosted = _wheelhouse_torchcodec_version(_codec_torch_ver)
if NO_TORCH:
_progress("torchcodec (skipped, no torch)")
elif _codec_hosted:
# The wheelhouse stands in for the wheel this platform lacks: --no-deps and pinned to
# the hosted version inside the resident torch's window, like the other hosted optionals.
_progress("torchcodec")
if pip_install_try(
f"Installing torchcodec=={_codec_hosted} from the Windows on ARM wheelhouse",
"--no-deps",
"--no-cache-dir",
f"torchcodec=={_codec_hosted}",
constrain = False,
):
_note(f"windows on arm: installed torchcodec=={_codec_hosted} from the wheelhouse")
else:
_note(
"windows on arm: could not install the wheelhouse torchcodec -- audio decoding stays disabled"
)
elif PLATFORM_LACKS_TORCHCODEC_WHEEL:
_progress("torchcodec (skipped, no wheel for this platform)")
if _wheelhouse_hosts("torchcodec"):
_note(
f"windows on arm: the wheelhouse torchcodec is outside the window torch "
f"{_codec_torch_ver or 'unknown'} selects -- leaving audio decoding disabled"
)
elif not _codec_torch_ver:
_progress("torchcodec (skipped, torch version unknown)")
_note("could not read the installed torch version -- leaving torchcodec alone")
@ -10240,6 +11094,7 @@ def install_python_stack() -> int:
expected_torch_tag = _recordable_torch_flavor_tag(torch_flavor_tag),
expected_torch_tag_pinned = bool(_recordable_torch_flavor_tag(torch_flavor_tag))
and _expected_torch_flavor_was_pinned(_recordable_torch_flavor_tag(torch_flavor_tag)),
woa_torch_index = os.environ.get("UNSLOTH_WOA_SELECTED_TORCH_INDEX"),
# What the NEXT run may skip. Digested from the CURRENT REQ_ROOT, which the core step
# may have replaced: a file that moved during this pass must not read as unchanged.
extra = {

View file

@ -932,6 +932,31 @@ def release_asset_map(release: dict[str, Any]) -> dict[str, str]:
}
def release_asset_digests(release: dict[str, Any]) -> dict[str, str]:
"""asset name -> bare sha256 hex, for the assets whose digest GitHub reports.
Separate from release_asset_map because the name -> url mapping is what nearly every
caller wants and widening its return type would touch all of them. GitHub returns
`digest` as an algorithm-prefixed string ("sha256:<hex>"); anything else is skipped
rather than guessed at, so a future algorithm cannot be read as a sha256.
"""
assets = release.get("assets")
if not isinstance(assets, list):
return {}
digests: dict[str, str] = {}
for asset in assets:
if not isinstance(asset, dict) or not isinstance(asset.get("name"), str):
continue
raw = asset.get("digest")
# The prefix is required, not stripped for convenience: normalize_sha256_digest accepts a bare 64-hex string, and an unprefixed digest is one whose algorithm GitHub did not state.
if not isinstance(raw, str) or not raw.lower().startswith("sha256:"):
continue
digest = normalize_sha256_digest(raw)
if digest:
digests[asset["name"]] = digest
return digests
def github_release(
ops: ModuleOps,
repo: str,

File diff suppressed because it is too large Load diff

View file

@ -1,3 +1,6 @@
// Building for aarch64-pc-windows-msvc needs clang on PATH: ring and aws-lc-sys assemble their
// aarch64 sources through cc-rs, which reaches for clang rather than cl.exe, and the failure
// names ring rather than the missing toolchain. No other leg builds for aarch64 Windows.
fn main() {
tauri_build::build()
}

View file

@ -704,17 +704,19 @@ mod tests {
let state = new_update_state();
state.lock().unwrap().child = Some(child);
for _ in 0..50 {
if child_pid_file.is_file() {
break;
// Wait for a pid that PARSES, not merely for the path to appear: the redirection above
// creates the file before anything is written, so is_file() can win that race and read "".
let mut descendant = None;
for _ in 0..100 {
if let Ok(text) = std::fs::read_to_string(&child_pid_file) {
if let Ok(pid) = text.trim().parse::<i32>() {
descendant = Some(pid);
break;
}
}
std::thread::sleep(std::time::Duration::from_millis(20));
}
let descendant = std::fs::read_to_string(&child_pid_file)
.unwrap()
.trim()
.parse::<i32>()
.unwrap();
let descendant = descendant.expect("the test child never wrote a usable descendant pid");
stop_update(&state).unwrap();

View file

@ -0,0 +1,391 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""The wheelhouse signer: what it signs, what it refuses, and what it must not break.
A .whl cannot be Authenticode signed, so the native binaries inside it are, and the wheel is
repacked. That makes RECORD the thing most likely to go quietly wrong -- signing changes every
signed member's bytes, and a stale RECORD ships a wheel that reads as corrupt to anything that
checks it. The signing call itself is stubbed here: this is about the packing, the refusals and
the RECORD, none of which need Azure.
"""
from __future__ import annotations
import base64
import hashlib
import importlib.util
import struct
import sys
import zipfile
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[2]
SIGNER_PY = REPO_ROOT / ".github" / "scripts" / "sign_wheel_binaries.py"
WORKFLOW = REPO_ROOT / ".github" / "workflows" / "woa-wheelhouse.yml"
@pytest.fixture(scope = "module")
def mod():
spec = importlib.util.spec_from_file_location("_sign_wheel_binaries", SIGNER_PY)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def _pe(
machine: int = 0xAA64,
*,
signed: bool = False,
plus: bool = True,
) -> bytes:
"""The smallest byte string the reader will accept as a PE image.
Built rather than vendored: a real .pyd in the tree would be a binary nobody can review,
and every field this tool reads is one of the four below.
"""
header_at = 0x80
blob = bytearray(b"\0" * 0x400)
blob[0:2] = b"MZ"
struct.pack_into("<I", blob, 0x3C, header_at)
blob[header_at : header_at + 4] = b"PE\0\0"
struct.pack_into("<H", blob, header_at + 4, machine)
magic_at = header_at + 24
struct.pack_into("<H", blob, magic_at, 0x20B if plus else 0x10B)
directory_at = magic_at + (112 if plus else 96) + 4 * 8
if signed:
struct.pack_into("<II", blob, directory_at, 0x300, 0x40)
return bytes(blob)
def _record(names_and_blobs) -> str:
lines = []
for name, blob in names_and_blobs:
digest = base64.urlsafe_b64encode(hashlib.sha256(blob).digest()).rstrip(b"=").decode()
lines.append(f"{name},sha256={digest},{len(blob)}")
return "\n".join(lines) + "\n"
def _wheel(path: Path, members: dict) -> Path:
"""A wheel whose RECORD matches its contents, the way a real one arrives."""
dist = "demo-1.0.dist-info"
record_name = f"{dist}/RECORD"
body = dict(members)
body.setdefault(f"{dist}/WHEEL", b"Wheel-Version: 1.0\nTag: cp313-cp313-win_arm64\n")
body.setdefault(f"{dist}/METADATA", b"Name: demo\nVersion: 1.0\n")
with zipfile.ZipFile(path, "w", zipfile.ZIP_DEFLATED) as z:
for name, blob in body.items():
z.writestr(name, blob)
z.writestr(record_name, _record(list(body.items())) + f"{record_name},,\n")
return path
@pytest.fixture
def fake_signer(tmp_path):
"""Stands in for trusted-signing-cli: appends a certificate table to the file it is given.
The real one talks to Azure. What this file's tests are about is everything around that
call, so the stub does the one thing the caller checks for -- the file comes back signed.
"""
script = tmp_path / "fake_signer.py"
script.write_text(
"import struct, sys\n"
"from pathlib import Path\n"
"p = Path(sys.argv[1]); d = bytearray(p.read_bytes())\n"
"h = struct.unpack_from('<I', d, 0x3C)[0]\n"
"magic_at = h + 24\n"
"magic = struct.unpack_from('<H', d, magic_at)[0]\n"
"at = magic_at + (112 if magic == 0x20b else 96) + 4 * 8\n"
"struct.pack_into('<II', d, at, len(d), 0x40)\n"
"d += b'CERTIFICATE'\n"
"p.write_bytes(bytes(d))\n",
encoding = "utf-8",
)
return [sys.executable, str(script)]
class TestWhatCountsAsAPeImage:
def test_a_pe_is_recognised(self, mod):
assert mod.pe_machine(_pe()) == 0xAA64
@pytest.mark.parametrize(
"blob, why",
[
(b"", "empty"),
(b"not a pe at all, just text in a wheel", "plain data"),
(b"MZ" + b"\0" * 8, "an MZ header too short to hold an offset"),
(b"MZ" + b"\0" * 0x200, "an MZ header whose PE offset points at zeros"),
],
)
def test_everything_else_is_not(self, mod, blob, why):
assert mod.pe_machine(blob) is None, why
def test_an_unsigned_image_reads_as_unsigned(self, mod):
assert mod.is_signed(_pe()) is False
@pytest.mark.parametrize("plus", [True, False], ids = ["PE32+", "PE32"])
def test_a_signed_image_reads_as_signed(self, mod, plus):
"""PE32 and PE32+ put the data directory at different offsets; a reader that assumed
one would call every 32-bit signed DLL unsigned and re-sign somebody else's binary."""
assert mod.is_signed(_pe(signed = True, plus = plus)) is True
class TestSigningAWheel:
def test_every_unsigned_binary_is_signed(self, mod, tmp_path, fake_signer):
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/__init__.py": b"x = 1\n",
"demo/_core.cp313-win_arm64.pyd": _pe(),
"demo/helper.dll": _pe(),
},
)
out, signed, skipped = mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
assert sorted(signed) == ["demo/_core.cp313-win_arm64.pyd", "demo/helper.dll"]
assert skipped == []
with zipfile.ZipFile(out) as z:
for name in signed:
assert mod.is_signed(z.read(name)), name
def test_an_already_signed_binary_is_left_alone(self, mod, tmp_path, fake_signer):
"""delvewheel vendors Microsoft's redistributables into pyarrow already signed by
Microsoft. Re-signing one replaces their signature with ours."""
original = _pe(signed = True)
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/_core.cp313-win_arm64.pyd": _pe(),
"demo.libs/msvcp140-abcdef.dll": original,
},
)
out, signed, skipped = mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
assert signed == ["demo/_core.cp313-win_arm64.pyd"]
assert skipped == ["demo.libs/msvcp140-abcdef.dll"]
with zipfile.ZipFile(out) as z:
assert z.read("demo.libs/msvcp140-abcdef.dll") == original, "byte-for-byte untouched"
def test_the_wheel_keeps_its_name(self, mod, tmp_path, fake_signer):
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/_core.cp313-win_arm64.pyd": _pe(),
},
)
out, _, _ = mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
assert out.name == wheel.name, "the index and every pin name the wheel by filename"
def test_non_binary_members_are_carried_through_unchanged(self, mod, tmp_path, fake_signer):
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/__init__.py": b"x = 1\n",
"demo/data.json": b'{"a": 1}',
"demo/_core.cp313-win_arm64.pyd": _pe(),
},
)
out, _, _ = mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
with zipfile.ZipFile(out) as z:
assert z.read("demo/__init__.py") == b"x = 1\n"
assert z.read("demo/data.json") == b'{"a": 1}'
def test_member_order_is_preserved(self, mod, tmp_path, fake_signer):
members = {
"demo/__init__.py": b"x = 1\n",
"demo/_core.cp313-win_arm64.pyd": _pe(),
"demo/zzz.py": b"pass\n",
}
wheel = _wheel(tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl", members)
with zipfile.ZipFile(wheel) as z:
before = z.namelist()
out, _, _ = mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
with zipfile.ZipFile(out) as z:
assert z.namelist() == before, "a diff should show signatures and nothing else"
class TestTheRecordIsRegenerated:
"""Signing appends a certificate table, so every signed member's digest and length move.
A wheel shipped with its original RECORD is one `pip check` away from reading as corrupt."""
@staticmethod
def _rows(wheel: Path) -> dict:
with zipfile.ZipFile(wheel) as z:
record = next(n for n in z.namelist() if n.endswith(".dist-info/RECORD"))
rows = {}
for line in z.read(record).decode().splitlines():
if line.strip():
parts = line.split(",")
rows[parts[0]] = (parts[1], parts[2])
return rows
def test_every_member_digest_matches_the_bytes_shipped(self, mod, tmp_path, fake_signer):
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/__init__.py": b"x = 1\n",
"demo/_core.cp313-win_arm64.pyd": _pe(),
},
)
out, _, _ = mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
rows = self._rows(out)
with zipfile.ZipFile(out) as z:
for name in z.namelist():
if name.endswith(".dist-info/RECORD"):
continue
blob = z.read(name)
expected = base64.urlsafe_b64encode(hashlib.sha256(blob).digest()).rstrip(b"=")
assert rows[name] == (f"sha256={expected.decode()}", str(len(blob))), name
def test_the_signed_binary_digest_actually_changed(self, mod, tmp_path, fake_signer):
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/_core.cp313-win_arm64.pyd": _pe(),
},
)
before = self._rows(wheel)["demo/_core.cp313-win_arm64.pyd"]
out, _, _ = mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
after = self._rows(out)["demo/_core.cp313-win_arm64.pyd"]
assert (
before != after
), "the fixture did not really change the bytes, so this proves nothing"
def test_record_names_itself_without_a_digest(self, mod, tmp_path, fake_signer):
"""PEP 376: RECORD cannot hash itself, and a tool that writes one anyway makes the
file self-contradictory."""
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/_core.cp313-win_arm64.pyd": _pe(),
},
)
out, _, _ = mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
assert self._rows(out)["demo-1.0.dist-info/RECORD"] == ("", "")
class TestItRefusesRatherThanShipSomethingWrong:
def test_a_foreign_architecture_is_refused(self, mod, tmp_path, fake_signer):
"""0x8664 is x64. A win_arm64 filename says nothing about what was linked in, and
signing one would publish a correctly signed wheel that cannot load."""
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/_core.cp313-win_arm64.pyd": _pe(machine = 0x8664),
},
)
with pytest.raises(SystemExit, match = "0x8664"):
mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
def test_a_foreign_architecture_is_allowed_when_asked(self, mod, tmp_path, fake_signer):
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_amd64.whl",
{
"demo/_core.cp313-win_amd64.pyd": _pe(machine = 0x8664),
},
)
_, signed, _ = mod.sign_wheel(wheel, tmp_path / "out", fake_signer, require_machine = None)
assert signed == ["demo/_core.cp313-win_amd64.pyd"]
def test_a_wheel_with_no_binaries_is_refused(self, mod, tmp_path, fake_signer):
"""Nothing to sign means the build produced a pure-Python wheel where a native one was
expected. Publishing it silently is how an unsigned wheelhouse comes back."""
wheel = _wheel(tmp_path / "demo-1.0-py3-none-win_arm64.whl", {"demo/__init__.py": b"x\n"})
with pytest.raises(SystemExit, match = "no PE images"):
mod.sign_wheel(wheel, tmp_path / "out", fake_signer)
def test_a_signer_that_fails_fails_the_run(self, mod, tmp_path):
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/_core.cp313-win_arm64.pyd": _pe(),
},
)
failing = [sys.executable, "-c", "import sys; sys.exit(3)"]
with pytest.raises(SystemExit, match = "signing failed"):
mod.sign_wheel(wheel, tmp_path / "out", failing)
def test_a_signer_that_silently_does_nothing_fails_the_run(self, mod, tmp_path):
"""Exit 0 with the file untouched is the dangerous case: it would publish an unsigned
wheel while every log line says it was signed."""
wheel = _wheel(
tmp_path / "demo-1.0-cp313-cp313-win_arm64.whl",
{
"demo/_core.cp313-win_arm64.pyd": _pe(),
},
)
noop = [sys.executable, "-c", "import sys"]
with pytest.raises(SystemExit, match = "no certificate after signing"):
mod.sign_wheel(wheel, tmp_path / "out", noop)
class TestTheWorkflowUsesTheProcedureTheDesktopReleaseUses:
"""One signing procedure, one definition. A second copy of the endpoint or the identity
drifts, and the drift is invisible until a release is signed by the wrong certificate."""
@pytest.fixture(scope = "class")
def text(self):
return WORKFLOW.read_text(encoding = "utf-8")
def test_it_delegates_to_the_shared_signing_helper(self, text):
assert "studio/src-tauri/windows/sign-with-trusted-signing.ps1" in text
assert "codesigning.azure.net" not in text, "the endpoint belongs to the helper only"
def test_the_signing_cli_is_digest_pinned(self, text):
assert "TRUSTED_SIGNING_CLI_SHA256" in text
desktop = (REPO_ROOT / ".github" / "workflows" / "release-desktop.yml").read_text(
encoding = "utf-8"
)
import re
pinned = re.findall(r'TRUSTED_SIGNING_CLI_SHA256:\s*"([0-9a-f]{64})"', text)
assert pinned, "no digest pin"
assert pinned[0] in desktop, "the two workflows pin different signing binaries"
def test_the_secrets_are_gated_by_an_environment(self, text):
assert "environment: release-signing" in text, (
"without an environment there is nowhere to hang a required reviewer, so anyone "
"who can dispatch this can sign under our identity"
)
def test_only_the_signing_job_reads_the_azure_secrets(self, text):
import yaml
data = yaml.safe_load(text)
for name, job in data["jobs"].items():
uses_azure = "AZURE_" in yaml.safe_dump(job)
if uses_azure:
assert name == "sign", f"{name} reads the signing secrets"
assert job.get("environment") == "release-signing"
def test_it_cannot_be_reached_from_a_pull_request(self, text):
import yaml
data = yaml.safe_load(text)
triggers = data[True] if True in data else data["on"]
assert set(triggers) == {
"workflow_dispatch"
}, "this job holds signing credentials and writes a release"
def test_publishing_is_opt_in(self, text):
import yaml
data = yaml.safe_load(text)
publish = data[True]["workflow_dispatch"]["inputs"]["publish"]
assert publish["default"] is False, "a dispatch must not publish unless asked"
assert "inputs.publish" in str(data["jobs"]["publish"]["if"])
def test_write_permission_is_scoped_to_the_publish_job(self, text):
import yaml
data = yaml.safe_load(text)
assert data["permissions"] == {"contents": "read"}, "the default must be read"
assert data["jobs"]["publish"]["permissions"] == {"contents": "write"}
for name, job in data["jobs"].items():
if name != "publish":
assert "permissions" not in job or job["permissions"].get("contents") != "write"
def test_the_published_set_is_checked_before_signing(self, text):
"""A short matrix leg would otherwise publish a wheelhouse missing an interpreter,
which reads downstream as 'no wheel for this Python' rather than as a failed build."""
assert "expected 5 wheels" in text

View file

@ -0,0 +1,344 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
"""Windows on ARM is pinned by splitting rows, so the split must be a true partition.
Every package that needs a different version on win_arm64 is written as two rows:
X==old ; sys_platform != "win32" or platform_machine != "ARM64"
X>=new ; sys_platform == "win32" and platform_machine == "ARM64"
The second marker is the exact complement of the first, so in every environment exactly one
row is live. Getting that wrong is silent either way: an OVERLAP makes pip intersect two
specifiers and can render the row unsatisfiable, a GAP drops the package on some platform
nobody tested. The compare is case-sensitive, which is what isolates Windows on ARM (macOS
reports ``arm64`` and Linux ``aarch64``), so that is asserted here too.
"""
from __future__ import annotations
import importlib.util
import itertools
from pathlib import Path
import pytest
from packaging.requirements import Requirement
from packaging.specifiers import SpecifierSet
REPO_ROOT = Path(__file__).resolve().parents[2]
REQ_FILES = [
REPO_ROOT / "studio/backend/requirements/extras.txt",
REPO_ROOT / "studio/backend/requirements/no-torch-runtime.txt",
REPO_ROOT / "studio/backend/requirements/single-env/constraints.txt",
REPO_ROOT / "studio/backend/requirements/studio.txt",
]
# (sys_platform, platform_system, platform_machine, os_name)
PLATFORMS = [
("linux", "Linux", "x86_64", "posix"),
("linux", "Linux", "aarch64", "posix"),
("linux", "Linux", "armv7l", "posix"),
("linux", "Linux", "ppc64le", "posix"),
("linux", "Linux", "s390x", "posix"),
("darwin", "Darwin", "arm64", "posix"),
("darwin", "Darwin", "x86_64", "posix"),
("win32", "Windows", "AMD64", "nt"),
("win32", "Windows", "x86", "nt"),
("win32", "Windows", "ARM64", "nt"),
]
PYTHONS = ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
WOA = ("win32", "Windows", "ARM64", "nt")
def _env(plat, py):
sys_platform, platform_system, platform_machine, os_name = plat
return {
"implementation_name": "cpython",
"implementation_version": f"{py}.0",
"os_name": os_name,
"platform_machine": platform_machine,
"platform_python_implementation": "CPython",
"platform_release": "",
"platform_system": platform_system,
"platform_version": "",
"python_full_version": f"{py}.0",
"python_version": py,
"sys_platform": sys_platform,
"extra": "",
}
#: Every environment the rows below are evaluated in, built once.
ENVS = [(plat, py, _env(plat, py)) for plat, py in itertools.product(PLATFORMS, PYTHONS)]
def _live(rows, env):
"""The rows pip would install in `env`."""
return [r for r in rows if r.marker is None or r.marker.evaluate(env)]
def _rows(path: Path) -> list[Requirement]:
out = []
for raw in path.read_text(encoding = "utf-8").splitlines():
line = raw.strip()
if not line or line.startswith(("#", "-")):
continue
line = line.split("#", 1)[0].strip()
if line:
out.append(Requirement(line))
return out
def _by_name(reqs: list[Requirement]) -> dict[str, list[Requirement]]:
grouped: dict[str, list[Requirement]] = {}
for req in reqs:
grouped.setdefault(req.name.lower().replace("_", "-"), []).append(req)
return grouped
def _multi_row_groups(reqs):
"""Packages stated more than once, minus the ones that are different targets.
unsloth[a] and unsloth[b] may legitimately co-exist, so a group whose rows differ in
their extras is not a platform split at all.
"""
for name, group in _by_name(reqs).items():
if len(group) < 2 or len({tuple(sorted(r.extras)) for r in group}) > 1:
continue
yield name, group
def _markers(group) -> list[str]:
return [str(r.marker).replace("'", '"') for r in group if r.marker is not None]
def _pyproject_extras() -> dict[str, list[Requirement]]:
try:
import tomllib
except ModuleNotFoundError: # pragma: no cover - 3.10 runs use tomli
import tomli as tomllib
data = tomllib.loads((REPO_ROOT / "pyproject.toml").read_text(encoding = "utf-8"))
out = {}
for extra, lines in data["project"]["optional-dependencies"].items():
out[extra] = [Requirement(l) for l in lines]
return out
ALL_SOURCES: list[tuple[str, list[Requirement]]] = [(p.name, _rows(p)) for p in REQ_FILES] + [
(f"pyproject[{k}]", v) for k, v in _pyproject_extras().items()
]
per_source = pytest.mark.parametrize("label,reqs", ALL_SOURCES, ids = [s[0] for s in ALL_SOURCES])
@per_source
def test_split_rows_never_overlap(label, reqs):
"""Two rows for one package must never both be live: pip would intersect them."""
for name, group in _multi_row_groups(reqs):
for plat, py, env in ENVS:
live = _live(group, env)
assert len(live) <= 1, (
f"{label}: {name} has {len(live)} live rows on "
f"{plat[0]}/{plat[2]}/py{py}: {[str(r) for r in live]}"
)
@per_source
def test_no_package_is_dropped_on_a_non_woa_platform(label, reqs):
"""A split may remove a package on Windows ARM64 only."""
if "constraints" in label:
return # a constraints file may legitimately have no cap in force
for name, group in _multi_row_groups(reqs):
# Only the complement-pair shape; triton-windows is two disjoint Windows-only rows.
markers = _markers(group)
if not (
any('platform_machine == "ARM64"' in m for m in markers)
and any('platform_machine != "ARM64"' in m for m in markers)
):
continue
for plat, py, env in ENVS:
if plat == WOA:
continue
assert _live(
group, env
), f"{label}: {name} has no live row on {plat[0]}/{plat[2]}/py{py}"
def test_arm64_marker_is_case_sensitive_and_windows_only():
"""``ARM64`` must not match macOS ``arm64`` or Linux ``aarch64``."""
woa = Requirement('x==1; sys_platform == "win32" and platform_machine == "ARM64"')
# The complement really is the complement.
other = Requirement('x==1; sys_platform != "win32" or platform_machine != "ARM64"')
for plat, py, env in ENVS:
live = woa.marker.evaluate(env)
assert live == (plat == WOA), f"win-ARM64 marker fired on {plat[0]}/{plat[2]}"
assert live != other.marker.evaluate(
env
), f"the two halves are not complementary on {plat[0]}/{plat[2]}"
@per_source
def test_no_row_is_dead_on_arrival(label, reqs):
"""Every row must be live in at least one real environment."""
for req in reqs:
if req.marker is None:
continue
assert any(req.marker.evaluate(env) for _, _, env in ENVS), (
f"{label}: `{req}` is live in none of the {len(ENVS)} environments tested, so it "
f'can never install. A lowercase "arm64" next to sys_platform == "win32" is the '
f'usual cause: Windows reports "ARM64".'
)
# Which packages carry a Windows-on-ARM row, PER SOURCE, and in which shape.
# "split" -- a positive `platform_machine == "ARM64"` row giving a different version
# "dropped" -- only the negative row, so the package is absent on Windows on ARM
# Checked per source and by shape: studio.txt and pyproject[studio] mirror each other, so a global
# check stays green when one loses a row.
WOA_ROWS_BY_SOURCE = {
"extras.txt": {"av": "split", "scikit-learn": "split"},
"no-torch-runtime.txt": {"pymupdf": "split", "hf-transfer": "dropped", "sqlite-vec": "dropped"},
"constraints.txt": {
"av": "split",
"cryptography": "split",
"pandas": "split",
"pyarrow": "split",
},
"studio.txt": {"cryptography": "split", "pandas": "split", "pymupdf": "split"},
"pyproject[studio]": {"cryptography": "split", "pandas": "split", "pymupdf": "split"},
"pyproject[triton]": {"triton-windows": "split"},
"pyproject[huggingfacenotorch]": {"hf-transfer": "dropped"},
"pyproject[windows]": {"xformers": "dropped"},
}
@pytest.mark.parametrize("label,expected", sorted(WOA_ROWS_BY_SOURCE.items()))
def test_the_woa_split_is_used_where_we_claim_it_is(label, expected):
"""Guard against a Windows-on-ARM row silently disappearing in a future edit."""
groups = _by_name(dict(ALL_SOURCES)[label])
for name, shape in sorted(expected.items()):
markers = _markers(groups.get(name, []))
positive = [m for m in markers if 'platform_machine == "ARM64"' in m]
negative = [m for m in markers if 'platform_machine != "ARM64"' in m]
if shape == "split":
assert positive, f"{label}: {name} lost its Windows-on-ARM row"
else:
assert negative and not positive, (
f"{label}: {name} should be excluded on Windows on ARM by a negative "
f"marker only; found positive={positive}"
)
# A package's OWN requires-python floor, for rows pinning into a range that does not exist for
# every interpreter. Only floors above our own 3.9 can make a row unsatisfiable.
PACKAGE_PYTHON_FLOORS = {
"pandas": [(SpecifierSet(">=3.0"), (3, 11))],
}
def _minor(py: str) -> tuple:
major, minor = py.split(".")
return (int(major), int(minor))
def _lowest_allowed(req) -> str:
"""The smallest concrete version the row's specifier admits, for floor comparison."""
lowers = [s.version for s in req.specifier if s.operator in (">=", "==", "~=", ">")]
return lowers[0] if lowers else "0"
@per_source
def test_a_selected_row_is_installable_on_the_python_it_was_selected_for(label, reqs):
"""Splitting on platform is not enough on its own: a row can be live for an interpreter
that no release in its range supports, which is not a resolution failure anyone reads
as a marker bug -- pip just reports that no version matches.
"""
for plat, py, env in ENVS:
for req in _live(reqs, env):
for spec, floor in PACKAGE_PYTHON_FLOORS.get(req.name.lower(), ()):
# Does this row admit ONLY versions that need a newer interpreter?
if not spec.contains(_lowest_allowed(req), prereleases = True):
continue
assert _minor(py) >= floor, (
f"{label}: `{req}` is live on Python {py} {plat[2]}, but every "
f"version it admits needs Python >= {floor[0]}.{floor[1]}. "
"The row is unsatisfiable there; the marker needs a "
"python_version bound as well as a platform one."
)
def test_the_woa_pandas_split_covers_every_supported_python():
"""The complement of the test above: having added a python_version bound, no ARM64
interpreter may be left with no pandas row at all.
"""
for label, reqs in ALL_SOURCES:
rows = [r for r in reqs if r.name.lower() == "pandas"]
if not rows:
continue
for py in PYTHONS:
live = _live(rows, _env(WOA, py))
assert len(live) == 1, (
f"{label}: Windows ARM64 on Python {py} has {len(live)} live pandas "
f"rows, expected exactly 1: {[str(r) for r in live]}"
)
if _minor(py) < (3, 11):
assert "3.0" not in str(
live[0].specifier
), f"{label}: Python {py} must not be handed the pandas 3 row"
# install_python_stack.py, loaded so the skip list is read rather than copied here.
_SPEC = importlib.util.spec_from_file_location(
"_ips_marker_skiplist", REPO_ROOT / "studio" / "install_python_stack.py"
)
IPS = importlib.util.module_from_spec(_SPEC)
_SPEC.loader.exec_module(IPS)
# Every name install_python_stack.py filters out of the requirements files on win_arm64.
WOA_SKIPPED = {IPS._canonical_dist_name(n) for n in IPS.WINDOWS_ARM64_SKIP_PACKAGES}
# Scoped to `studio` deliberately: it is the extra a Windows-on-ARM user installs. The other
# 190-odd are x64 recipes, so an ARM64 marker there would assert what they never promised.
WOA_INSTALLABLE_EXTRAS = ["studio"]
per_extra = pytest.mark.parametrize("extra", WOA_INSTALLABLE_EXTRAS, ids = WOA_INSTALLABLE_EXTRAS)
@per_extra
def test_a_skipped_package_is_not_left_live_in_an_extra(extra):
"""The runtime skip list cannot reach package METADATA, so the extra has to agree.
`pip install "unsloth[studio]"` never runs the installer's filtering: it resolves
pyproject's rows directly, so a row left live on win_arm64 for a package with no wheel
and no buildable sdist there fails the install outright. sqlite-vec was exactly this.
"""
live = [
str(req)
for req in _live(_pyproject_extras()[extra], _env(WOA, "3.13"))
if IPS._canonical_dist_name(req.name) in WOA_SKIPPED
]
assert not live, (
f"pyproject[{extra}] leaves these live on Windows ARM64 even though the installer "
f"treats them as unavailable there, so `pip install unsloth[{extra}]` cannot "
f"resolve: {live}"
)
@per_extra
def test_dropping_a_package_on_woa_drops_it_nowhere_else(extra):
"""A negative ARM64 marker is a scalpel: every other platform keeps the row.
Checked as an outcome rather than a spelling, over the same platform table the rest of
this file uses, so a marker that reads correctly but excludes (say) Windows x86 as well
is still caught.
"""
for req in _pyproject_extras()[extra]:
if IPS._canonical_dist_name(req.name) not in WOA_SKIPPED or req.marker is None:
continue
for plat, py, env in ENVS:
if plat == WOA:
continue
assert req.marker.evaluate(env), (
f"pyproject[{extra}] {req.name} is dropped on {plat[0]}/{plat[2]}/"
f"py{py} too, which is not what the ARM64 marker is for"
)

File diff suppressed because it is too large Load diff

View file

@ -1679,6 +1679,35 @@ def test_the_private_temp_removal_only_takes_what_it_created(tmp_path: Path):
assert (data / "studio.port").exists()
# Split-Path's -LiteralPath lives in its own parameter set in Windows PowerShell 5.1, so naming
# -Parent with it throws AmbiguousParameterSet at runtime, not at parse time, which is why eight of
# them reached a release. -LiteralPath alone already splits off the parent, and -Path globs.
# Static, because CI has no Windows PowerShell 5.1 to run the scripts under.
_SPLIT_PATH_LITERAL_PARENT = re.compile(
r"Split-Path\b[^\r\n|;]*?-LiteralPath\b[^\r\n|;]*?-Parent\b"
r"|Split-Path\b[^\r\n|;]*?-Parent\b[^\r\n|;]*?-LiteralPath\b"
)
# The two installers this change touches. scripts/uninstall.ps1 has its own eight of these and its
# own PR (#10471); listed here it failed on this tree unconditionally.
@pytest.mark.parametrize("name", ("install.ps1", "studio/setup.ps1"))
def test_split_path_never_pairs_literalpath_with_parent(name: str) -> None:
text = (REPO_ROOT / name).read_text(encoding = "utf-8")
offenders = [
f"{name}:{number}: {line.strip()}"
for number, line in enumerate(text.splitlines(), start = 1)
# Comments are prose about the rule, not a call the shell binds.
if not line.strip().startswith("#") and _SPLIT_PATH_LITERAL_PARENT.search(line)
]
assert not offenders, (
"Split-Path -LiteralPath cannot be combined with -Parent: Windows PowerShell 5.1 "
"resolves no parameter set for the pair and the call throws at runtime. Drop "
"-Parent -- -LiteralPath alone already returns the parent, and unlike -Path it does "
"not treat [ ] in an install root as a wildcard.\n " + "\n ".join(offenders)
)
SETUP_PS1 = REPO_ROOT / "studio" / "setup.ps1"

View file

@ -233,12 +233,23 @@ def test_installer_never_installs_an_unpinned_xformers():
necessarily built for the CUDA family the resident torch came from. The installer
picks the exact version for (torch, cuda) instead, so every spec must be pinned."""
source = _source()
# The Windows on ARM drop list names packages the installer REMOVES, so a bare name there is
# the opposite of an install spec and has to match the requirement line exactly.
drop_list = re.search(r"\$WoaDropCandidates = @\(.*?\n\s*\)\n", source, re.DOTALL)
drop_span = drop_list.span() if drop_list else (-1, -1)
# Same for the drop floors: the key is a package NAME being looked up, not an install spec.
floors = re.search(r"\$WoaDropFloors = @\{[^}]*\}", source, re.DOTALL)
floor_span = floors.span() if floors else (-1, -1)
for match in re.finditer(r'"xformers[^"]*"', source):
spec = match.group(0)
# A wheel FILENAME is not a spec: it names one exact file and cannot resolve to anything else, which is the
# whole point of preferring it.
if spec.endswith('.whl"'):
continue
if drop_span[0] <= match.start() < drop_span[1]:
continue
if floor_span[0] <= match.start() < floor_span[1]:
continue
assert spec == '"xformers==$_xfVersion"', f"unpinned xFormers spec in install.ps1: {spec}"

View file

@ -0,0 +1,167 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""The Windows ARM64 CUDA bundle was the one archive this installer would install unchecked.
Every other prebuilt goes through apply_approved_hashes, which drops an attempt our checksum
manifest does not cover. The ARM64 CUDA branch could not: we publish no windows-arm64-cuda
artifact, so nothing covers upstream's zip, and refusing outright would mean no CUDA
llama.cpp on this hardware at all. It logged that it had no hash and installed anyway.
GitHub's release API now reports a `digest` for every asset (ggml-org/llama.cpp b10853: 27 of
27, including llama-*-bin-win-cuda-13.4-arm64.zip and its paired cudart archive). That is
weaker than a manifest we compute ourselves rather than read from the host serving the bytes,
but it pins the download to what the API listed. An asset GitHub states no digest for is now
refused, so no path installs an unverified archive.
"""
from __future__ import annotations
import pathlib
import sys
import pytest
REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
sys.path.insert(0, str(REPO_ROOT / "studio"))
import install_llama_prebuilt as ip # noqa: E402
import prebuilt_core # noqa: E402
SHA_A = "a" * 64
SHA_B = "b" * 64
def _release(*assets: dict) -> dict:
return {"tag_name": "b10853", "assets": list(assets)}
def _asset(name: str, digest: str | None) -> dict:
out = {"name": name, "browser_download_url": f"https://example.invalid/{name}"}
if digest is not None:
out["digest"] = digest
return out
class TestTheDigestMapReadsOnlyWhatGitHubStated:
def test_a_sha256_digest_is_read_without_its_prefix(self):
release = _release(_asset("bundle.zip", f"sha256:{SHA_A}"))
assert prebuilt_core.release_asset_digests(release) == {"bundle.zip": SHA_A}
@pytest.mark.parametrize(
("digest", "why"),
[
(None, "no digest field at all"),
("", "empty"),
(SHA_A, "bare hex: GitHub did not say which algorithm produced it"),
(f"sha512:{'c' * 128}", "a different algorithm"),
("sha256:nothex" + "0" * 58, "not hex"),
("sha256:" + "a" * 63, "too short"),
("sha256:" + "a" * 65, "too long"),
],
ids = ["absent", "empty", "unprefixed", "sha512", "not-hex", "short", "long"],
)
def test_anything_else_is_left_out_rather_than_guessed_at(self, digest, why):
release = _release(_asset("bundle.zip", digest))
assert prebuilt_core.release_asset_digests(release) == {}, why
def test_an_upper_case_digest_still_matches_a_lower_case_file_hash(self):
"""The verifier compares hex strings, so the case has to be normalised here."""
release = _release(_asset("bundle.zip", "SHA256:" + SHA_A.upper()))
assert prebuilt_core.release_asset_digests(release) == {"bundle.zip": SHA_A}
def test_a_release_with_no_assets_is_empty_not_an_error(self):
assert prebuilt_core.release_asset_digests({"tag_name": "b1"}) == {}
assert prebuilt_core.release_asset_digests({"assets": "not a list"}) == {}
class TestAnAttemptWithNoDigestIsDropped:
"""The rule that makes this safe. [] is what the caller reads as "refuse"."""
def _choice(
self,
name = "bundle.zip",
runtime = None,
):
return ip.AssetChoice(
repo = "ggml-org/llama.cpp",
tag = "b10853",
name = name,
url = f"https://example.invalid/{name}",
source_label = "upstream",
install_kind = "windows-arm64-cuda",
runtime_name = runtime,
runtime_url = f"https://example.invalid/{runtime}" if runtime else None,
)
def test_a_covered_attempt_carries_the_digest_forward(self):
kept = ip._apply_release_digests([self._choice()], {"bundle.zip": SHA_A})
assert [c.expected_sha256 for c in kept] == [SHA_A]
def test_an_uncovered_attempt_is_dropped_not_installed(self):
assert ip._apply_release_digests([self._choice()], {"other.zip": SHA_A}) == []
def test_an_empty_digest_map_refuses_everything(self):
assert ip._apply_release_digests([self._choice()], {}) == []
def test_a_paired_runtime_gets_its_own_digest(self):
kept = ip._apply_release_digests(
[self._choice(runtime = "cudart.zip")],
{"bundle.zip": SHA_A, "cudart.zip": SHA_B},
)
assert kept[0].runtime_sha256 == SHA_B
def test_an_uncovered_runtime_unpairs_rather_than_riding_along(self):
"""Dropping the whole attempt would lose CUDA over a missing side archive, and
keeping the pair would install one unverified. Unpairing is neither."""
kept = ip._apply_release_digests(
[self._choice(runtime = "cudart.zip")], {"bundle.zip": SHA_A}
)
assert len(kept) == 1
assert kept[0].expected_sha256 == SHA_A
assert kept[0].runtime_name is None
assert kept[0].runtime_url is None
assert kept[0].runtime_sha256 is None
class TestTheFetcherFailsClosed:
def test_an_api_failure_is_an_empty_map_not_an_exception(self, monkeypatch):
"""A rate limit or an outage has to cost the CUDA bundle, not the whole install,
and it must not be mistaken for "no digest needed"."""
def _boom(repo, tag):
raise RuntimeError("429 rate limited")
monkeypatch.setattr(ip, "github_release", _boom)
assert ip.github_release_asset_digests("ggml-org/llama.cpp", "b10853") == {}
def test_it_reads_the_digests_off_the_release_it_fetched(self, monkeypatch):
monkeypatch.setattr(
ip, "github_release", lambda repo, tag: _release(_asset("b.zip", f"sha256:{SHA_A}"))
)
assert ip.github_release_asset_digests("r", "t") == {"b.zip": SHA_A}
def test_no_branch_returns_attempts_that_were_never_hash_gated():
"""The property this file exists for, asserted against the source.
Every `return` of asset choices in the release-plan resolver goes through
apply_approved_hashes or _apply_release_digests. A future branch that returns a bare
list would reintroduce exactly the hole this closes.
"""
source = (REPO_ROOT / "studio" / "install_llama_prebuilt.py").read_text(encoding = "utf-8")
start = source.index("def resolve_release_asset_choice(")
end = source.index("\ndef ", start + 1)
body = source[start:end]
returns = [
line.strip()
for line in body.splitlines()
if line.strip().startswith("return ") and "attempts" not in line
]
for line in returns:
assert (
"apply_approved_hashes" in line
or "_apply_release_digests" in line
or "verified" in line
or "published_choice" in line
), f"a return that is not hash gated: {line}"

View file

@ -836,6 +836,7 @@ def _run_flavor_invariant(
index_family = None,
index_url = None,
win_arm64 = False,
win_arm64_interpreter = None,
probe_cuda = None,
probe_hip = None,
):
@ -913,6 +914,13 @@ def _run_flavor_invariant(
patch.object(stack_mod, "_RECORDED_TORCH_TAG", recorded),
patch.object(stack_mod.platform, "machine", return_value = "AMD64"),
patch.object(stack_mod, "_is_windows_arm64", return_value = win_arm64),
# The interpreter's arch, which the CUDA-preservation shortcut reads.
# They separate on an ARM64 machine running an emulated x64 python.
patch.object(
stack_mod,
"_is_win_arm64_interpreter",
return_value = win_arm64 if win_arm64_interpreter is None else win_arm64_interpreter,
),
patch.object(stack_mod, "_has_usable_nvidia_gpu", return_value = nvidia),
patch.object(stack_mod.shutil, "which", side_effect = _which),
patch.object(stack_mod.os.path, "isfile", return_value = True),
@ -1397,6 +1405,84 @@ class TestAPinnedCpuIndexIsEnforcedToo:
mock_pip.assert_not_called()
class TestWindowsOnArmPreservesCudaOnlyForAnInferredExpectation:
"""The win_arm64 CUDA shortcut must not swallow an explicit CPU pin.
Preserving an installed cu* build is right for an expectation DERIVED from the driver:
download.pytorch.org publishes no win_arm64 CUDA wheel, so that "repair" resolves
nothing. A pin is not a derivation, and /cpu does publish win_arm64 torch and
torchvision, so that repair has somewhere to go.
"""
_PIN = "https://download.pytorch.org/whl/cpu"
def test_an_explicit_cpu_pin_still_repairs_a_cuda_venv(self):
ok, mock_pip = _run_flavor_invariant(
installed = "2.11.0+cu134",
repaired = "2.10.0+cpu",
expected_env = "cpu",
index_url = self._PIN,
backend = "cpu",
win_arm64 = True,
)
assert ok is True
assert mock_pip.call_count == 1
assert _index_url(mock_pip) == self._PIN
args = [str(a) for a in mock_pip.call_args.args]
# No win_arm64 torchaudio on /cpu either, so the existing drop still applies.
assert not any(a.startswith("torchaudio") for a in args)
def test_the_family_spelling_of_the_pin_is_honoured_too(self):
ok, mock_pip = _run_flavor_invariant(
installed = "2.11.0+cu134",
repaired = "2.10.0+cpu",
expected_env = "cpu",
index_family = "cpu",
backend = "cpu",
win_arm64 = True,
)
assert ok is True
assert mock_pip.call_count == 1
def test_a_driver_inferred_cuda_expectation_is_still_preserved(self):
ok, mock_pip = _run_flavor_invariant(
installed = "2.11.0+cu134",
expected_env = "cu130",
win_arm64 = True,
)
assert ok is True
mock_pip.assert_not_called()
def test_a_probed_cpu_tag_with_no_pin_does_not_downgrade_it(self):
# setup.ps1 also publishes "cpu" when its nvidia-smi probe comes back empty, and a probe
# result is not evidence, so the shortcut keeps the CUDA build.
ok, mock_pip = _run_flavor_invariant(
installed = "2.11.0+cu134",
expected_env = "cpu",
nvidia = False,
win_arm64 = True,
)
assert ok is True
mock_pip.assert_not_called()
def test_an_emulated_x64_interpreter_on_an_arm64_machine_still_repairs(self):
"""The machine and the interpreter are separate axes, and the shortcut reads the
interpreter. Every Windows on ARM install predating native support runs an
emulated x64 python against ordinary win_amd64 wheels from
download.pytorch.org, so the repair must reach them exactly as it always did.
"""
ok, mock_pip = _run_flavor_invariant(
installed = "2.9.1+cu118",
repaired = "2.10.0+cu128",
expected_env = "cu128",
win_arm64 = True,
win_arm64_interpreter = False,
)
assert ok is True
assert mock_pip.call_count == 1
assert _index_url(mock_pip).endswith("/cu128")
class TestWindowsOnArmKeepsTheNoTorchaudioException:
"""No win_arm64 torchaudio wheel is published on any index.
@ -2192,9 +2278,13 @@ class TestTheDelegatedRocmRepairKeepsTheArm64Exception:
def test_the_windows_rocm_install_drops_torchaudio_on_arm64(self):
source = inspect.getsource(stack_mod._ensure_rocm_torch)
block = source[source.index("_WINDOWS_ROCM_TORCH_PKG_SPECS.get") :][:1200]
# The interpreter's arch, not the machine's: an emulated x64 venv installs win_amd64.
assert (
"_is_windows_arm64()" in block
"_is_win_arm64_interpreter()" in block
), "the delegated ROCm repair needs the same exception as the flavor repair"
assert (
"_is_windows_arm64()" not in block
), "the machine predicate would drop torchaudio from x64 venvs"
assert "*_rocm_trio" in block, "the trio has to be built, not passed positionally"
def test_x64_windows_still_asks_for_all_three(self):

View file

@ -19,8 +19,11 @@ import ast
import io
import pathlib
import re
import subprocess
import tokenize
import pytest
REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
REQ_ROOT = REPO_ROOT / "studio" / "backend" / "requirements"
PIN_FILE = REQ_ROOT / "diffusers-pin.txt"
@ -181,3 +184,70 @@ def test_install_sh_still_delegates_the_core_package_skip():
"""The handoff flag skips core packages while allowing other base entries through."""
assert 'SKIP_STUDIO_BASE="$_SKIP_BASE"' in INSTALL_SH.read_text(encoding = "utf-8")
assert "_SKIP_BASE=1" in INSTALL_SH.read_text(encoding = "utf-8")
def test_no_generated_filter_snapshot_is_tracked():
"""They are a copy of a file already in the tree, and one got committed.
_filter_requirements writes beside the source so relative -r/-c includes resolve.
pip_install unlinks them in a finally, but a test calling the helper directly, or an
install killed mid-run, leaves them in the checkout, where `git add -A` picks them up.
A stale snapshot then reads as a second, silently divergent copy of the pins.
"""
done = subprocess.run(
["git", "ls-files", "-z", "--", "studio/backend/requirements/"],
cwd = REPO_ROOT,
capture_output = True,
text = True,
)
if done.returncode != 0:
pytest.skip("not a git checkout")
tracked = [p for p in done.stdout.split("\0") if p]
offenders = [p for p in tracked if _GENERATED_FILTER.fullmatch(pathlib.Path(p).name)]
assert not offenders, f"generated filter snapshots are tracked: {offenders}"
def test_gitignore_covers_the_generated_snapshots():
"""So the next `git add -A` cannot put one back."""
probe = REQ_ROOT / ".studio-filtered-abcd1234.txt"
assert _GENERATED_FILTER.fullmatch(probe.name), "the probe must match the generated shape"
done = subprocess.run(
["git", "check-ignore", "-q", "--no-index", str(probe)],
cwd = REPO_ROOT,
capture_output = True,
text = True,
)
if done.returncode == 128:
pytest.skip("not a git checkout")
assert done.returncode == 0, f"{probe.name} is not ignored; .gitignore needs the pattern"
# A win_arm64 floor set above the first release that actually publishes one costs the
# resolver every wheel in between, and for scikit-learn it cost the only one that exists on
# a free-threaded 3.13. Each floor below is the earliest release carrying a win_arm64 wheel,
# read off PyPI's own file list, so the pin can be checked against the index by hand.
WIN_ARM64_FLOORS = [
# scikit-learn 1.9.0 dropped cp313-cp313t; 1.8.0 is the only release with one, so a
# >=1.9.0 floor leaves a free-threaded 3.13 with no wheel and an sdist to compile.
("extras.txt", "scikit-learn", "1.8.0"),
# av publishes cp311-abi3 plus a cp314t from 17.0.0. No release has a 3.13t wheel.
("extras.txt", "av", "17.0.0"),
("single-env/constraints.txt", "av", "17.0.0"),
]
@pytest.mark.parametrize(
"relpath, dist, floor",
WIN_ARM64_FLOORS,
ids = [f"{r.split('/')[-1]}:{d}" for r, d, _ in WIN_ARM64_FLOORS],
)
def test_the_win_arm64_floor_is_the_first_release_that_has_a_wheel(relpath, dist, floor):
text = (REQ_ROOT / relpath).read_text(encoding = "utf-8")
wanted = f'{dist}>={floor}; sys_platform == "win32" and platform_machine == "ARM64"'
assert wanted in text, f"{relpath} no longer floors {dist} at {floor}"
# And nothing else floors the same dist higher on that marker.
for line in text.splitlines():
line = line.strip()
if not line.startswith(f"{dist}>=") or 'platform_machine == "ARM64"' not in line:
continue
assert line == wanted, f"{relpath}: a second ARM64 floor for {dist}: {line}"

View file

@ -132,6 +132,28 @@ def test_extras_cannot_shadow_a_field_verify_install_reads(tmp_path: pathlib.Pat
assert payload["package_version"] != "9.9.9"
def test_extras_cannot_name_the_windows_on_arm_torch_index(tmp_path: pathlib.Path) -> None:
"""setup.ps1 reads woa_torch_index back as an index to install torch FROM, and the
parameter that writes it accepts NVIDIA's own https channel only. Evidence reaching the
same key would hand a fresh-shell update any URL the caller liked."""
im.write_manifest(
root = tmp_path,
req_root = tmp_path,
package_name = "pytest",
extra = {"woa_torch_index": "https://evil.test/simple"},
)
assert "woa_torch_index" not in _payload(tmp_path)
im.write_manifest(
root = tmp_path,
req_root = tmp_path,
package_name = "pytest",
woa_torch_index = "https://pypi.nvidia.com/nvtorch_oot",
extra = {"woa_torch_index": "https://evil.test/simple"},
)
assert _payload(tmp_path)["woa_torch_index"] == "https://pypi.nvidia.com/nvtorch_oot"
def test_update_manifest_merges_without_touching_the_rest(tmp_path: pathlib.Path) -> None:
im.write_manifest(root = tmp_path, req_root = tmp_path, package_name = "pytest")
before = _payload(tmp_path)

View file

@ -0,0 +1,183 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""The llama.cpp mismatch gate and the llama.cpp selector have to agree, or updates churn.
Before reinstalling, setup.ps1 deletes the existing llama.cpp tree when its recorded
`install_kind` is not in the set the gate considers correct for this host; then
install_llama_prebuilt.py picks a bundle. Nothing makes those two agree, and a disagreement
is permanent: every update deletes the tree and refetches the identical bundle, forever, on
a working host. The only relationship that matters is "the kind the selector installs is one
the gate accepts", so that is asserted directly, over the product of what either side
branches on, rather than the text of either.
Both churning combinations this found are on Windows ARM64 and invisible from an x64 box:
* NVIDIA. The gate expected windows-cuda; on an ARM64 venv the selector installs
windows-arm64. That is every Windows ARM64 machine with an NVIDIA GPU.
* ROCm. No ROCm bundle exists for Windows ARM64 at all, so the selector falls through to
the ARM64 CPU bundle while the gate still expected windows-rocm or windows-hip.
Offline. direct_upstream_release_plan takes a release dict, and the gate's own block is lifted
out of setup.ps1 and run with only its inputs replaced.
"""
from __future__ import annotations
import itertools
import pathlib
import re
import shutil
import subprocess
import sys
import pytest
REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
SETUP_PS1 = REPO_ROOT / "studio" / "setup.ps1"
SETUP_SRC = SETUP_PS1.read_text(encoding = "utf-8")
sys.path.insert(0, str(REPO_ROOT / "studio"))
import install_llama_prebuilt as ip # noqa: E402
PWSH = shutil.which("pwsh") or shutil.which("powershell")
requires_pwsh = pytest.mark.skipif(PWSH is None, reason = "PowerShell is unavailable")
TAG = "b9334"
# Everything upstream publishes for Windows, so no row is starved of a candidate and a
# disagreement is a disagreement rather than a missing asset. Note what is NOT here and
# cannot be: there is no ROCm or CUDA bundle for Windows ARM64 upstream.
ASSET_NAMES = (
f"llama-{TAG}-bin-win-cpu-arm64.zip",
f"llama-{TAG}-bin-win-cpu-x64.zip",
f"llama-{TAG}-bin-win-cuda-12.4-x64.zip",
f"llama-{TAG}-bin-win-cuda-13.0-x64.zip",
f"llama-{TAG}-bin-win-vulkan-x64.zip",
f"llama-{TAG}-bin-win-hip-radeon-x64.zip",
)
RELEASE = {
"tag_name": TAG,
"assets": [
{"name": name, "browser_download_url": f"https://example.invalid/{name}"}
for name in ASSET_NAMES
],
}
def _host(*, arm64: bool, nvidia: bool, rocm: bool) -> ip.HostInfo:
"""The interpreter's own view of the machine, which is what the selector reads: an
emulated x64 Python on an ARM64 box reports AMD64, and is an x64 host here."""
return ip.HostInfo(
system = "Windows",
machine = "ARM64" if arm64 else "AMD64",
is_windows = True,
is_linux = False,
is_macos = False,
is_x86_64 = not arm64,
is_arm64 = arm64,
nvidia_smi = "nvidia-smi" if nvidia else None,
driver_cuda_version = (13, 0) if nvidia else None,
compute_caps = ["12.1"] if nvidia else [],
visible_cuda_devices = None,
has_physical_nvidia = nvidia,
has_usable_nvidia = nvidia,
has_rocm = rocm,
rocm_gfx_target = "gfx1201" if rocm else None,
)
def _gate_block() -> str:
"""setup.ps1's own expected-kinds computation, from the opt-out read to the delete."""
start = SETUP_SRC.index("$_arm64CudaOptOut = ")
end = SETUP_SRC.index("if ($existingKind -and", start)
return SETUP_SRC[start:end]
def _expected_kinds(*, arm64_venv: bool, nvidia: bool, rocm: bool, opt_out: bool) -> list[str]:
block = _gate_block()
# The parentheses stay: the source writes `if (Test-WinArm64Venv)`, and replacing the
# whole parenthesised form would leave `if $true {`, which does not parse.
block = block.replace("Test-WinArm64Venv", "$true" if arm64_venv else "$false")
# The evidence index is read through a chain of markers and a venv on disk. Its only use
# is Test-WoaPersistableIndex, so supply the answer and stub the predicate.
block = re.sub(
r"(?s)\$_woaEvidenceIndex = if.*?\n(\s*)\$_nvidiaEvidence",
"$_woaEvidenceIndex = '{}'\n\\1$_nvidiaEvidence".format(
"https://pypi.nvidia.com/nvtorch_oot" if nvidia else ""
),
block,
)
script = "\n".join(
[
"$ErrorActionPreference = 'Stop'",
"function Test-WoaPersistableIndex { param($i) return ($i -like '*nvidia*') }",
f"$env:UNSLOTH_LLAMA_ARM64_CUDA = '{'0' if opt_out else '1'}'",
f"$HasNvidiaSmi = ${str(nvidia).lower()}",
f"$HasROCm = ${str(rocm).lower()}",
"$script:ROCmGfxArch = " + ("'gfx1201'" if rocm else "$null"),
block,
"Write-Output ('<<<' + ($expectedKinds -join ',') + '>>>')",
]
)
done = subprocess.run(
[PWSH, "-NoProfile", "-NonInteractive", "-Command", script],
capture_output = True,
timeout = 120,
)
out = done.stdout.decode("utf-8", "replace")
assert "<<<" in out, done.stderr.decode("utf-8", "replace")
return out[out.index("<<<") + 3 : out.rindex(">>>")].split(",")
def _installed_kinds(host: ip.HostInfo) -> list[str]:
plan = ip.direct_upstream_release_plan(RELEASE, host, "ggml-org/llama.cpp", "latest")
return [attempt.install_kind for attempt in plan.attempts] if plan else []
# nvidia and rocm together is not a host we ship to: the ROCm arm wins outright on both sides.
COMBINATIONS = [
(arm64_venv, nvidia, rocm, opt_out)
for arm64_venv, nvidia, rocm, opt_out in itertools.product([True, False], repeat = 4)
if not (nvidia and rocm)
]
@requires_pwsh
@pytest.mark.parametrize(
("arm64_venv", "nvidia", "rocm", "opt_out"),
COMBINATIONS,
ids = [
"{}-{}{}".format(
"arm64" if a else "x64",
"nvidia" if n else ("rocm" if r else "cpu"),
"-optout" if o else "",
)
for a, n, r, o in COMBINATIONS
],
)
def test_the_gate_accepts_what_the_selector_installs(arm64_venv, nvidia, rocm, opt_out):
expected = _expected_kinds(arm64_venv = arm64_venv, nvidia = nvidia, rocm = rocm, opt_out = opt_out)
installed = _installed_kinds(_host(arm64 = arm64_venv, nvidia = nvidia, rocm = rocm))
assert installed, "no candidate at all, so this row proves nothing"
assert installed[0] in expected, (
f"the gate would delete what the selector just installed, on every update: "
f"selector picks {installed[0]}, gate accepts {expected}"
)
@requires_pwsh
def test_no_arm64_row_expects_a_kind_only_published_for_x64():
"""The shape behind both bugs: on an ARM64 venv a gate that expects an x64-only kind and
nothing else is a delete on every update whatever the selector does. windows-vulkan counts
as x64-only, since upstream's bundle is vulkan-x64."""
x64_only = {"windows-cuda", "windows-rocm", "windows-hip", "windows-cpu", "windows-vulkan"}
for nvidia, rocm, opt_out in itertools.product([True, False], repeat = 3):
if nvidia and rocm:
continue
expected = set(_expected_kinds(arm64_venv = True, nvidia = nvidia, rocm = rocm, opt_out = opt_out))
assert expected - x64_only, (
f"nvidia={nvidia} rocm={rocm} optout={opt_out}: the gate expects only x64 kinds "
f"on an ARM64 venv: {sorted(expected)}"
)

View file

@ -161,9 +161,13 @@ class TestUnsupportedNameLookup:
# ── The Windows WMI path end to end ──────────────────────────────────────────
def _wmi_detect(names):
def _wmi_detect(names, arm64 = False):
"""Drive _detect_windows_gfx_arch over `names` with no hipinfo and no amd-smi,
which is the reporter's host: Adrenalin driver only. Returns (arch, stdout)."""
which is the reporter's host: Adrenalin driver only. Returns (arch, stdout).
`arm64` is the host the advice is being written for. Pinned here rather than read
from the box running the tests, because the machine predicate is right to answer
True on an ARM64 dev box and the reporter's box is x64."""
ps_result = MagicMock()
ps_result.returncode = 0
amd = [n for n in names if re.search(r"AMD|Radeon", n, re.IGNORECASE)]
@ -190,7 +194,8 @@ def _wmi_detect(names):
with patch("shutil.which", return_value = None):
with patch("os.path.isfile", return_value = False):
with patch("subprocess.run", side_effect = _run):
result = stack_mod._detect_windows_gfx_arch()
with patch.object(stack_mod, "_is_windows_arm64", return_value = arm64):
result = stack_mod._detect_windows_gfx_arch()
return result, buf.getvalue()
@ -308,8 +313,7 @@ class TestPythonStackWindowsArm64:
ARM64 throw applies to it: setup.ps1 rejects the variable there."""
def test_arm64_gets_a_source_build_note_instead_of_the_setter(self):
with patch.object(stack_mod, "_is_windows_arm64", return_value = True):
_arch, out = _wmi_detect(["AMD Radeon RX 5700 XT"])
_arch, out = _wmi_detect(["AMD Radeon RX 5700 XT"], arm64 = True)
assert "gfx1010" in out, "the card is still named"
assert (
"UNSLOTH_LLAMA_CPP_BACKEND" not in out
@ -318,29 +322,43 @@ class TestPythonStackWindowsArm64:
def test_x64_still_gets_the_setter(self):
"""Positive control: the guard must not silence the advice everywhere."""
with patch.object(stack_mod, "_is_windows_arm64", return_value = False):
_arch, out = _wmi_detect(["AMD Radeon RX 5700 XT"])
_arch, out = _wmi_detect(["AMD Radeon RX 5700 XT"])
assert "UNSLOTH_LLAMA_CPP_BACKEND" in out
@pytest.mark.parametrize(
"env,expected",
"registry,env,expected",
[
({"PROCESSOR_ARCHITECTURE": "ARM64"}, True),
({"PROCESSOR_ARCHITECTURE": "AMD64", "PROCESSOR_ARCHITEW6432": "ARM64"}, True),
({"PROCESSOR_ARCHITECTURE": "AMD64", "PROCESSOR_ARCHITEW6432": ""}, False),
("ARM64", {"PROCESSOR_ARCHITECTURE": "ARM64"}, True),
# x64 emulation on an ARM64 box: the process copy says AMD64 and ARCHITEW6432 is
# unset, so only the machine-scope registry value tells the truth.
("ARM64", {"PROCESSOR_ARCHITECTURE": "AMD64"}, True),
# ARCHITEW6432 still counts on the builds that do set it.
("", {"PROCESSOR_ARCHITECTURE": "AMD64", "PROCESSOR_ARCHITEW6432": "ARM64"}, True),
("AMD64", {"PROCESSOR_ARCHITECTURE": "AMD64", "PROCESSOR_ARCHITEW6432": ""}, False),
# Unreadable registry falls back to the per-process signals, not to True.
("", {"PROCESSOR_ARCHITECTURE": "AMD64"}, False),
],
ids = [
"native-arm64",
"emulated-x64-on-arm64",
"architew6432-set",
"real-x64",
"no-registry-x64",
],
ids = ["native-arm64", "emulated-x64-on-arm64", "real-x64"],
)
def test_the_arch_probe_reads_the_machine_not_the_process(self, env, expected):
def test_the_arch_probe_reads_the_machine_not_the_process(self, registry, env, expected):
"""PROCESSOR_ARCHITECTURE describes the PROCESS, so an emulated x64 Python on an
ARM64 box reports AMD64; ARCHITEW6432 is ARM64 in exactly that case."""
ARM64 box reports AMD64 and so does platform.machine(). The machine-scope value in
the registry is the one signal emulation cannot misreport. Patched per case so the
answer is the same on an ARM64 host as on x64 CI."""
with patch.object(stack_mod, "IS_WINDOWS", True):
with patch.object(stack_mod.platform, "machine", return_value = "AMD64"):
with patch.dict(os.environ, env, clear = False):
for _k in ("PROCESSOR_ARCHITEW6432", "PROCESSOR_ARCHITECTURE"):
if _k not in env:
os.environ.pop(_k, None)
assert stack_mod._is_windows_arm64() is expected
with patch.object(stack_mod, "_machine_arch_from_registry", return_value = registry):
with patch.object(stack_mod.platform, "machine", return_value = "AMD64"):
with patch.dict(os.environ, env, clear = False):
for _k in ("PROCESSOR_ARCHITEW6432", "PROCESSOR_ARCHITECTURE"):
if _k not in env:
os.environ.pop(_k, None)
assert stack_mod._is_windows_arm64() is expected
def test_it_is_false_off_windows(self):
with patch.object(stack_mod, "IS_WINDOWS", False):

View file

@ -0,0 +1,54 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""The bundled seed plugin is installed with --no-deps, so its own pandas range was never
resolved. On Windows on ARM (Python 3.11+) constraints.txt installs pandas 3, which the
plugin declared out of range: a dependency-aware reinstall of the plugin would then look for
a pandas 2 that has no win_arm64 wheel. The plugin's range must admit every pandas the
constraints can install."""
import pathlib
import re
from packaging.specifiers import SpecifierSet
ROOT = pathlib.Path(__file__).resolve().parents[3]
PLUGIN = (
ROOT / "studio" / "backend" / "plugins" / "data-designer-unstructured-seed" / "pyproject.toml"
)
CONSTRAINTS = ROOT / "studio" / "backend" / "requirements" / "single-env" / "constraints.txt"
def _plugin_pandas_range():
text = PLUGIN.read_text(encoding = "utf-8")
match = re.search(r'"pandas([^"]*)"', text)
assert match, "the plugin declares pandas"
return SpecifierSet(match.group(1))
def _constraint_pandas_pins():
pins = []
for line in CONSTRAINTS.read_text(encoding = "utf-8").splitlines():
if line.startswith("pandas"):
spec = line.split(";", 1)[0][len("pandas") :].strip()
pins.append(SpecifierSet(spec))
assert pins, "constraints.txt pins pandas"
return pins
def test_every_constrained_pandas_is_inside_the_plugins_range():
plugin = _plugin_pandas_range()
for pin in _constraint_pandas_pins():
# The lowest version each constraint admits is the one that gets installed.
floor = next(s.version for s in pin if s.operator in ("==", ">="))
assert plugin.contains(
floor
), f"constraints install pandas {floor}, outside the plugin's {plugin}"
def test_the_arm64_row_installs_pandas_3():
"""The row this test exists for: without it the range check would pass on pandas 2 alone."""
rows = [
l for l in CONSTRAINTS.read_text(encoding = "utf-8").splitlines() if l.startswith("pandas>=3")
]
assert len(rows) == 1 and 'platform_machine == "ARM64"' in rows[0]

View file

@ -3019,6 +3019,137 @@ class TestPublishedWindowsCudaAttemptsDynamicMajor:
assert result[0].runtime_line == "cuda12"
class TestPublishedLegacyNamedArm64BundlesAreOrdered:
"""The legacy-minor ordering synthesised -x64.zip names and called windows_cuda_attempts
without the arch, so an approved llama-<tag>-bin-win-cuda-13.1-arm64.zip produced no
runtime line and was skipped for the unchecksummed upstream fallback."""
TAG = "b8508"
def _release(
self,
minors_lines,
ranks = None,
cudart = False,
):
artifacts = [
make_artifact(
f"llama-{self.TAG}-bin-win-cuda-{minor}-arm64.zip",
install_kind = "windows-arm64-cuda",
runtime_line = line,
supported_sms = ["75", "80", "86", "89", "90", "100", "120"],
max_sm = 120,
rank = (ranks or {}).get(minor, 100),
)
for minor, line in minors_lines
]
release = make_release(artifacts, upstream_tag = self.TAG)
if cudart:
for minor, _ in minors_lines:
name = f"cudart-llama-bin-win-cuda-{minor}-arm64.zip"
release.assets[name] = f"https://example.com/{name}"
return release
def _host(self, driver):
return make_host(
system = "Windows", machine = "ARM64", driver_cuda_version = driver, compute_caps = ["120"]
)
def test_the_arm64_legacy_bundle_is_selected(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
release = self._release([("13.1", "cuda13"), ("12.8", "cuda12")])
result = published_windows_cuda_attempts(self._host((13, 1)), release, None, arch = "arm64")
assert result, "no attempt at all: the arch was lost in the legacy ordering"
assert result[0].runtime_line == "cuda13"
assert result[0].name == f"llama-{self.TAG}-bin-win-cuda-13.1-arm64.zip"
def test_the_minor_gate_applies_to_arm64_too(self, monkeypatch):
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
release = self._release([("13.3", "cuda13"), ("12.8", "cuda12")])
result = published_windows_cuda_attempts(self._host((13, 1)), release, None, arch = "arm64")
assert result and result[0].runtime_line == "cuda12"
assert all(not a.name.endswith("13.3-arm64.zip") for a in result)
def test_the_artifact_gate_reads_the_arm64_name(self, monkeypatch):
"""Two legacy minors on one line, the higher one ranked first: the line survives on
the lower minor, and the per-artifact gate must parse the -arm64 name to drop the
higher one; an x64 spelling never matches, and the preferred 13.3 build slips through."""
mock_windows_runtime(monkeypatch, ["cuda13"])
release = self._release([("13.3", "cuda13"), ("13.1", "cuda13")], ranks = {"13.3": 10})
result = published_windows_cuda_attempts(self._host((13, 1)), release, None, arch = "arm64")
names = [a.name for a in result]
assert names == [f"llama-{self.TAG}-bin-win-cuda-13.1-arm64.zip"], names
def test_the_cudart_bundle_pairs_by_the_arm64_name(self, monkeypatch):
"""The same match feeds the cudart pairing: the arm64 runtime archive rides along."""
mock_windows_runtime(monkeypatch, ["cuda13"])
release = self._release([("13.1", "cuda13")], cudart = True)
result = published_windows_cuda_attempts(self._host((13, 1)), release, None, arch = "arm64")
assert result and result[0].runtime_name == f"cudart-llama-bin-win-cuda-13.1-arm64.zip"
def test_the_explicit_arch_wins_over_the_host(self, monkeypatch):
"""The arch argument is the contract: it must reach the legacy ordering rather than
being re-derived from the host, or an x64 host resolving arm64 bundles finds none."""
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
release = self._release([("13.1", "cuda13"), ("12.8", "cuda12")])
host = make_host(
system = "Windows", machine = "AMD64", driver_cuda_version = (13, 1), compute_caps = ["120"]
)
result = published_windows_cuda_attempts(host, release, None, arch = "arm64")
assert result and result[0].name == f"llama-{self.TAG}-bin-win-cuda-13.1-arm64.zip"
class TestAnUpstreamLookupFailureCostsOnlyCuda:
"""On a Windows ARM64 NVIDIA host with no approved CUDA bundle, the upstream asset list is
fetched from the release API. A rate limit or an outage there raised out of the planner,
which catches only PrebuiltFallback, so the whole install aborted although the published
ARM64 CPU bundle was there to fall through to. It now costs the CUDA bundle only, like the
digest fetch beside it."""
TAG = "b8508"
CPU = "app-b8508-windows-arm64-cpu.zip"
def _plan(self, monkeypatch, assets):
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT,
"detect_torch_cuda_runtime_preference",
lambda host: CudaRuntimePreference(runtime_line = None, selection_log = []),
)
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "github_release_assets", assets)
monkeypatch.setattr(
INSTALL_LLAMA_PREBUILT, "github_release_asset_digests", lambda repo, tag: {}
)
monkeypatch.delenv("UNSLOTH_LLAMA_ARM64_CUDA", raising = False)
host = make_host(
system = "Windows", machine = "ARM64", driver_cuda_version = (13, 4), compute_caps = ["121"]
)
release = make_release(
[
make_artifact(
self.CPU,
install_kind = "windows-arm64",
runtime_line = None,
bundle_profile = "windows-cpu-arm64",
)
],
upstream_tag = self.TAG,
)
return resolve_release_asset_choice(host, self.TAG, release, make_checksums([self.CPU]))
def test_an_api_failure_falls_through_to_the_cpu_bundle(self, monkeypatch):
def _boom(repo, tag):
raise RuntimeError("429 rate limited")
result = self._plan(monkeypatch, _boom)
assert [a.name for a in result] == [self.CPU]
assert result[0].install_kind == "windows-arm64"
def test_an_empty_listing_lands_in_the_same_place(self, monkeypatch):
result = self._plan(monkeypatch, lambda repo, tag: {})
assert [a.name for a in result] == [self.CPU]
# ===========================================================================
# N.1e. resolve_release_asset_choice -- pin on the published install path
# ===========================================================================
@ -4288,8 +4419,7 @@ class TestCudaDriverToolkitMismatchMessage:
"exceeds driver CUDA major 12 (12.9)."
) in output
assert (
"Update the NVIDIA GPU driver to run CUDA Toolkit 13.3, or install "
"a CUDA 12.x toolkit."
"Update the NVIDIA GPU driver to run CUDA Toolkit 13.3, or install a CUDA 12.x toolkit."
) in output
assert "prebuilt CUDA bundle" in output

View file

@ -661,3 +661,41 @@ def test_the_rocm_trio_is_reinstalled_when_the_architecture_index_moves():
# The record follows the install, never precedes it: a failed trio must not be recorded.
failed = text.index("AMD ROCm PyTorch install failed -- falling back to CPU")
assert failed < record
class TestSetupPs1WindowsOnArmCudaPreservation:
"""The win_arm64 CUDA shortcut is for an INFERRED expectation, not a stated one.
It runs ahead of the pin branch that raises $script:PinChangedForceReinstall, and that
flag is the only thing that clears $SkipPythonDeps. So without the exemption an
explicit pin skipped the dependency pass, install_python_stack.py and every
--force-reinstall at once, and `studio update` kept the old CUDA build while reporting
success. Exempting only /cpu was not enough: a user moving the venv to their own
cu129 mirror is stating an instruction just as much, and the index selection further
down is written to let a pin outrank the persisted NVIDIA channel.
"""
_GUARD = "if ((Test-WinArm64Venv) -and $installedTorchTag -and"
def _condition(self) -> str:
start = _SETUP_SRC.index(self._GUARD)
return _SETUP_SRC[start : _SETUP_SRC.index("{", start + len(self._GUARD))]
def test_any_explicit_pin_is_exempt(self):
condition = self._condition()
assert "-not $_pinnedIdx" in condition
assert (
"$_woaCpuPinned" not in condition
), "a cu129 mirror pin is as much an instruction as a /cpu one"
def test_the_exemption_reads_a_variable_that_is_always_assigned(self):
# Not $_pinLeaf: it is assigned only inside `if ($_pinnedIdx)`, so reading it here would
# be fatal under Set-StrictMode. $_pinnedIdx is assigned unconditionally above.
block = _SETUP_SRC[: _SETUP_SRC.index(self._GUARD)]
assert "$_pinnedIdx = Get-PinnedTorchIndexUrl" in block
assert "$_pinLeaf" not in self._condition()
def test_it_still_sits_ahead_of_the_pin_branch(self):
shortcut = _line_of(_SETUP_SRC, self._GUARD)
pin_branch = _line_of(_SETUP_SRC, "Torch-index pin changed ($installedTorchTag)")
assert shortcut < pin_branch

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,260 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""The PyPI-first wheelhouse guard, executed rather than read.
test_woa_wheelhouse_prefers_pypi.py pins the SHAPE of the guard against install.ps1's
source, which cannot catch a helper that is wired correctly and answers wrongly. The three
bugs covered here, all found reviewing the guard's first version:
1. A wheel the guard skipped stopped counting as available. $WoaWheelNames is rebuilt by
scanning the staging directory, and a $WoaDropCandidates name missing from it is emitted
as `name ; platform_machine == "AMD64"`, which EXCLUDES the package on ARM64. So the day
PyPI published a win_arm64 hf_transfer or brotli, the guard turned "installed from our
wheelhouse" into "not installed at all", firing on the event it exists for.
2. abi3 was treated as universally compatible. It is forward compatible from the version it
was built against, so a cp314-abi3 wheel does not import on cp313, and calling ours
redundant against one would leave the package uninstallable.
3. PyPI publishing a wheel is only availability if the resolve will look at PyPI. Offline,
or pointed at an exclusive mirror, dropping our copy leaves it obtainable from nowhere.
Hermetic: no network. The live-PyPI leg of the guard is exercised on hardware, not here.
"""
from __future__ import annotations
import pathlib
import re
import shutil
import subprocess
import textwrap
import pytest
REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
INSTALL_PS1 = REPO_ROOT / "install.ps1"
#: A corporate index that is not PyPI, and PyPI itself: the only distinction these make.
CORP = "https://corp.example/simple"
PYPI = "https://pypi.org/simple"
requires_pwsh = pytest.mark.skipif(shutil.which("pwsh") is None, reason = "PowerShell is unavailable")
HELPERS = (
"Test-WoaVersionAtLeast",
"Test-WoaWheelTags",
"Test-WoaWheelTagsUsable",
# Everything Test-WoaResolveReachesPyPI calls: a missing helper answers True and lets a
# lookalike index pass unnoticed.
"Test-WoaUrlIsPublicPyPI",
"Remove-WoaTomlComment",
"Split-WoaTomlKey",
"Read-WoaUvInlineIndexArray",
"Read-WoaUvTomlIndexKeys",
"Get-WoaUvConfigIndexPolicy",
"Test-WoaResolveReachesPyPI",
)
INSTALL_SRC = INSTALL_PS1.read_text(encoding = "utf-8-sig")
def _function(source: str, name: str) -> str:
match = re.search(r"(?ms)^ function " + re.escape(name) + r"\b.*?^ \}\r?$", source)
assert match, f"install.ps1 no longer defines {name}"
# Lifted out of Install-UnslothStudio, so the four-space body indent comes with it.
return textwrap.dedent(match.group(0))
def _drop_list_block(source: str) -> str:
"""The real block that builds $WoaWheelNames and emits the override drop lines."""
start = re.search(r"(?m)^ \$WoaWheelNames = @\{\}\s*$", source)
end = re.search(r"(?m)^ \$WoaOverrideLines \+= 'torch>=2\.4'\s*$", source)
assert start and end, "the drop-list block moved"
return textwrap.dedent(source[start.start() : end.start()].replace("\r\n", "\n"))
# PowerShell does not hoist, so every snippet below carries all of HELPERS in front of it.
PRELUDE = "\n".join(_function(INSTALL_SRC, name) for name in HELPERS)
def _run(script: str) -> str:
proc = subprocess.run(
["pwsh", "-NoProfile", "-NonInteractive", "-Command", PRELUDE + "\n" + script],
capture_output = True,
text = True,
encoding = "utf-8",
errors = "replace",
timeout = 120,
)
assert proc.returncode == 0, f"pwsh failed: {proc.stdout}\n{proc.stderr}"
return proc.stdout.strip()
def _dropped_names(provided: str) -> set[str]:
"""Which $WoaDropCandidates get excluded on ARM64, given what PyPI supplies.
The staging directory is empty on purpose: the only thing standing between a name and
the AMD64 drop line is the $WoaPyPIProvided bookkeeping this exercises.
"""
out = _run(f"""
$dir = Join-Path ([System.IO.Path]::GetTempPath()) ([guid]::NewGuid().ToString("N"))
New-Item -ItemType Directory -Path $dir -Force | Out-Null
$WoaWheelDir = $dir
$WoaVenvMinor = "3.13"
$WoaWheelTag = "cp313"; $WoaWheelAbi = "cp313"; $WoaWheelStable = $true; $WoaWheelMinor = 13
$script:WoaTorchAudio = $false
$script:WoaPyarrowWheelName = $null
$script:WoaPyPIProvided = {provided}
function substep {{ param($a, $b) }}
{_drop_list_block(INSTALL_SRC)}
Remove-Item -LiteralPath $dir -Recurse -Force
($WoaOverrideLines | Where-Object {{ $_ -like "*platform_machine*" }} |
ForEach-Object {{ ($_ -split ' ')[0] }} | Sort-Object -Unique) -join ","
""")
return {name for name in out.split(",") if name}
@requires_pwsh
@pytest.mark.parametrize(
("wheel", "py_tag", "abi_tag", "usable"),
[
# The two interpreter-agnostic shapes the wheelhouse actually ships.
("hf_transfer-0.1.9-cp38-abi3-win_arm64.whl", "cp313", "cp313", True),
("sqlite_vec-0.1.9-py3-none-win_arm64.whl", "cp313", "cp313", True),
# Bug 2: abi3 does not reach BACKWARDS to an older interpreter.
("hf_transfer-0.2.0-cp314-abi3-win_arm64.whl", "cp313", "cp313", False),
("foo-1.0-py314-none-win_arm64.whl", "cp313", "cp313", False),
# Free-threaded CPython has no stable ABI (CPython #111506).
("hf_transfer-0.1.9-cp38-abi3-win_arm64.whl", "cp313", "cp313t", False),
# An exact match needs no special case.
("regex-2026.9.3-cp313-cp313-win_arm64.whl", "cp313", "cp313", True),
# A wheel built for another interpreter is not this venv's business either way.
("regex-2026.9.3-cp312-cp312-win_arm64.whl", "cp313", "cp313", False),
],
)
def test_a_wheel_is_usable_only_where_it_actually_imports(wheel, py_tag, abi_tag, usable):
out = _run(f'Test-WoaWheelTagsUsable -Name "{wheel}" -PyTag "{py_tag}" -AbiTag "{abi_tag}"')
assert out == str(usable), f"{wheel} on {py_tag}/{abi_tag}: expected {usable}, got {out}"
@requires_pwsh
@pytest.mark.parametrize(
("env", "reaches"),
[
({}, True),
# Bug 3: these REPLACE the default index, so PyPI is not consulted at all.
({"UV_DEFAULT_INDEX": CORP}, False),
({"UV_INDEX_URL": CORP}, False),
# install.ps1 resolves with uv, which never reads pip's variables.
({"PIP_INDEX_URL": CORP}, True),
({"PIP_NO_INDEX": "1"}, True),
({"UV_INDEX_URL": CORP, "PIP_EXTRA_INDEX_URL": PYPI}, False),
# Pointed at PyPI explicitly is still PyPI.
({"UV_DEFAULT_INDEX": PYPI}, True),
({"UV_OFFLINE": "1"}, False),
({"UV_NO_INDEX": "1"}, False),
# An unset-looking value must not read as "offline".
({"UV_OFFLINE": "0"}, True),
({"UV_NO_INDEX": "false"}, True),
],
)
def test_pypi_counts_only_when_the_resolve_would_reach_it(env, reaches):
sets = "".join(f'$env:{key} = "{value}"; ' for key, value in env.items())
assert _run(sets + "Test-WoaResolveReachesPyPI") == str(reaches)
@requires_pwsh
def test_a_wheel_taken_from_pypi_is_not_then_excluded_on_arm64():
"""Bug 1, the one that matters: skipping our copy must not read as "no wheel exists"."""
nothing_anywhere = _dropped_names("@{}")
assert (
{"brotli", "hf-transfer", "hf_transfer"} <= nothing_anywhere
), f"a name with no wheel anywhere must be dropped on ARM64, got {nothing_anywhere}"
from_pypi = _dropped_names('@{ "brotli" = @("1.3.0"); "hf-transfer" = @("0.2.0") }')
assert not (
{"brotli", "hf-transfer", "hf_transfer"} & from_pypi
), f"PyPI supplies these, so they must not be excluded on ARM64; dropped {from_pypi}"
# What is genuinely unavailable is still dropped, so the fix did not blanket-allow.
assert {"xformers", "torchcodec", "brotlicffi"} <= from_pypi
@requires_pwsh
@pytest.mark.parametrize(
("version", "still_dropped"),
[("0.0.20", True), ("0.0.30", False)],
)
def test_a_pypi_version_below_a_floor_keeps_the_drop(version, still_dropped):
"""xformers carries a floor because the released metadata does, so a PyPI version below
it is not a usable answer and the drop has to stay."""
dropped = _dropped_names(f'@{{ "xformers" = @("{version}") }}')
assert ("xformers" in dropped) is still_dropped
# A uv config can take PyPI out of the resolve as decisively as UV_OFFLINE can. Every case below
# is one install.ps1's subset parser got WRONG until it learned to scan for quotes, each answering
# "PyPI is reachable" for a config that had disabled it, which deletes wheelhouse copies of wheels
# the resolve can never fetch. Expectations derived with tomllib, not written by hand.
UV_CONFIG_CASES = [
# A comment needs no whitespace in front of it. `(^|\s)#` missed this one entirely.
("comment_nospace", "uv.toml", "no-index = true# offline lab\n", False),
("value_hash_nospace", "uv.toml", f'index-url = "{CORP}"#corp\n', False),
# A quoted key is the same key as the bare spelling.
("quoted_key", "uv.toml", f'"index-url" = "{CORP}"\n', False),
# A dotted key is the same statement as writing the leaf under [pip].
("dotted_no_index", "uv.toml", "pip.no-index = true\n", False),
("dotted_index_url", "uv.toml", f'pip.index-url = "{CORP}"\n', False),
("pyproject_dotted", "pyproject.toml", "[tool.uv]\npip.no-index = true\n", False),
# The other side of the same scan: a `#` INSIDE a string is not a comment.
("fragment_kept", "uv.toml", f'index-url = "{CORP}#frag"\n', False),
("url_only_in_comment", "uv.toml", f'# index-url = "{CORP}"\n', True),
# Cases that must keep answering True, so the fix cannot be a blanket "not reachable".
("extra_index_only", "uv.toml", f'extra-index-url = "{CORP}"\n', True),
("index_default_false", "uv.toml", f'[[index]]\nurl = "{CORP}"\ndefault = false\n', True),
("explicit_pypi", "uv.toml", f'index-url = "{PYPI}"\n', True),
("no_index_false", "uv.toml", f'no-index = false\nindex-url = "{PYPI}"\n', True),
# [pip] outranks the top level, in both directions.
(
"pip_pypi_beats_top_corp",
"uv.toml",
f'index-url = "{CORP}"\n[pip]\nindex-url = "{PYPI}"\n',
True,
),
(
"pip_corp_beats_top_pypi",
"uv.toml",
f'index-url = "{PYPI}"\n[pip]\nindex-url = "{CORP}"\n',
False,
),
# A host that merely contains the name is not PyPI.
("lookalike_host", "uv.toml", 'index-url = "https://pypi.org.evil.com/simple"\n', False),
# An inline `index = [...]` is not modelled, and unreadable resolves to "not PyPI".
("inline_index_array", "uv.toml", f'index = [{{ url = "{CORP}", default = true }}]\n', False),
]
@requires_pwsh
@pytest.mark.parametrize(
("name", "filename", "text", "reaches"),
UV_CONFIG_CASES,
ids = [case[0] for case in UV_CONFIG_CASES],
)
def test_a_uv_config_decides_whether_pypi_is_in_the_resolve(
tmp_path, name, filename, text, reaches
):
work = tmp_path / name
work.mkdir()
(work / filename).write_text(text, encoding = "utf-8")
# APPDATA/ProgramData would let the developer's own uv.toml decide the answer.
out = _run(f"""
Remove-Item Env:UV_OFFLINE,Env:PIP_NO_INDEX,Env:UV_DEFAULT_INDEX,Env:UV_INDEX_URL,\
Env:PIP_INDEX_URL,Env:UV_CONFIG_FILE,Env:UV_NO_CONFIG -ErrorAction SilentlyContinue
$env:APPDATA = ''
$env:ProgramData = ''
Set-Location -LiteralPath '{work}'
[string](Test-WoaResolveReachesPyPI)
""")
assert out == str(reaches), f"{name}: expected reaches={reaches}, got {out}"

View file

@ -0,0 +1,180 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""The wheelhouse is for wheels PyPI does not build, and only for those.
The Windows on ARM wheelhouse exists because PyPI publishes no win_arm64 build of pyarrow,
tiktoken, grpcio, brotli, hf_transfer or sqlite-vec. regex is different: PyPI has shipped
win_arm64 regex since 2025.7.29, and the staging directory is first in UV_FIND_LINKS, so a
user asking for regex got our binary rather than the one the project released.
So staging asks PyPI first and skips the copy when PyPI publishes the same project at or
above the version that would have been staged. The version half matters as much as the
check: an upstream BEHIND the wheelhouse has to leave ours in place, or the guard turns
into a downgrade.
Text-level tests, like the rest of the installer suite: install.ps1 is PowerShell, so the
shape is asserted against its source rather than by running it.
"""
from __future__ import annotations
import pathlib
import re
import shutil
import subprocess
import pytest
REPO_ROOT = pathlib.Path(__file__).resolve().parents[3]
INSTALL_PS1 = REPO_ROOT / "install.ps1"
@pytest.fixture(scope = "module")
def source() -> str:
return INSTALL_PS1.read_text(encoding = "utf-8-sig")
def _function_body(source: str, name: str) -> str:
"""One nested ` function Name {` block, to its closing ` }`."""
match = re.search(r"(?ms)^ function " + re.escape(name) + r"\b.*?^ \}$", source)
assert match, f"install.ps1 no longer defines {name}"
return match.group(0)
def _staging_loop(source: str) -> str:
"""The generic wheelhouse mirror: the block that copies everything but pyarrow."""
match = re.search(
r"(?ms)^ \$WoaExtraStaged = 0$.*?^ if \(\$WoaExtraStaged -gt 0\)", source
)
assert match, "the wheelhouse staging loop moved"
return match.group(0)
def test_the_pypi_probe_is_pypi_only(source):
"""Test-WoaWheelAvailable falls back to the wheelhouse, where the wheel being staged
always is, so the guard would never fire. This probe has to ask PyPI and nothing else."""
body = _function_body(source, "Test-WoaPyPIWheel")
assert "pypi.org/simple" in body
for other in ("WoaWheelhouse", "index.txt", "Get-ChildItem"):
assert other not in body, f"Test-WoaPyPIWheel consults {other}; it must ask PyPI alone"
def test_both_staging_branches_skip_what_pypi_publishes(source):
"""Both the local-directory branch and the URL branch mirror the wheelhouse, so a guard
on one of them leaves the other shipping our copy."""
loop = _staging_loop(source)
calls = re.findall(r"Test-WoaWheelhouseWheelIsRedundant", loop)
assert (
len(calls) == 2
), f"expected the redundancy guard in both staging branches, found {len(calls)}"
def test_the_guard_is_version_aware(source):
"""Published is not enough: upstream behind the wheelhouse means dropping ours
downgrades the install, so the floor is the staged wheel's own version."""
body = _function_body(source, "Test-WoaWheelhouseWheelIsRedundant")
assert "-Floor $fields[1]" in body, "the guard must floor PyPI at the staged version"
probe = _function_body(source, "Test-WoaPyPIWheel")
assert "Test-WoaVersionAtLeast" in probe, "the probe must compare versions, not just presence"
def test_the_guard_only_judges_wheels_this_venv_could_use(source):
"""A cp312 wheel in the wheelhouse is not made redundant by a cp313 wheel on PyPI."""
body = _function_body(source, "Test-WoaWheelhouseWheelIsRedundant")
assert "Test-WoaWheelTagsUsable" in body
def test_interpreter_agnostic_wheels_still_count(source):
"""hf_transfer ships cp38-abi3 and sqlite_vec py3-none, both usable on cp313: an
exact-tag test would call them foreign and go on shipping ours forever."""
body = _function_body(source, "Test-WoaWheelTagsUsable")
assert '"abi3"' in body and '"none"' in body
# Free-threaded venvs are the exception the exact-tag helper exists for.
assert '$AbiTag -like "*t"' in body
def test_pyarrow_keeps_its_own_pypi_first_path(source):
"""Get-WoaPyarrowSource already asks PyPI first and the generic loop skips pyarrow for
that reason. Both halves have to stay, or pyarrow is staged twice or never probed."""
body = _function_body(source, "Get-WoaPyarrowSource")
assert 'return "pypi"' in body
loop = _staging_loop(source)
assert re.search(
r'-like "pyarrow-\*"\) \{ continue \}', loop
), "the generic loop must leave pyarrow to Get-WoaPyarrowSource"
# The default wheelhouse URL had no test at all: changing it to a working but wrong host left
# the whole suite green. It is the only source for the pyarrow that gates the native path, so a
# typo sends every Windows on ARM host back to the emulated x64 stack, and it is fetched over
# the network, so a wrong host is a wrong download.
DEFAULT_WHEELHOUSE = "https://huggingface.co/danielhanchen/unsloth-blackwell-docker/resolve/main/windows-arm64-wheels"
PWSH = shutil.which("pwsh")
requires_pwsh = pytest.mark.skipif(PWSH is None, reason = "pwsh not available")
def _wheelhouse_assignment(source: str) -> str:
"""The `$script:WoaWheelhouse = if (...) {...} else {...}` block, lifted verbatim."""
match = re.search(r"(?ms)^ \$script:WoaWheelhouse = if .*?^ \}$", source)
assert match, "install.ps1 no longer assigns $script:WoaWheelhouse in one block"
return match.group(0)
def test_the_default_wheelhouse_url_is_exactly_this(source):
assert DEFAULT_WHEELHOUSE in _wheelhouse_assignment(source)
def test_the_default_is_an_https_resolve_url_on_hugging_face(source):
"""`resolve/main` serves the file; a plain repo URL serves an HTML page, which the
staging code would happily save as a .whl. The path may continue past `resolve/main`:
the wheelhouse is a folder of a repo we own."""
assert DEFAULT_WHEELHOUSE.startswith("https://huggingface.co/danielhanchen/")
assert "/resolve/main" in DEFAULT_WHEELHOUSE
assert not DEFAULT_WHEELHOUSE.endswith("/"), "Join-UrlPath adds the slash"
@requires_pwsh
@pytest.mark.parametrize(
("configured", "expected", "why"),
[
(None, DEFAULT_WHEELHOUSE, "unset falls back to the published wheelhouse"),
("", DEFAULT_WHEELHOUSE, "empty is not a configuration"),
("https://example.test/wheels", "https://example.test/wheels", "a mirror is honoured"),
("https://example.test/wheels/", "https://example.test/wheels", "one trailing slash goes"),
("https://example.test/wheels///", "https://example.test/wheels", "so do several"),
(
" https://example.test/wheels ",
"https://example.test/wheels",
"surrounding space goes",
),
(r"C:\wheels", r"C:\wheels", "a local directory survives untouched"),
(
"C:" + chr(92) + "wheels" + chr(92),
"C:" + chr(92) + "wheels" + chr(92),
"TrimEnd takes '/' only, so a trailing backslash stays",
),
],
)
def test_the_wheelhouse_override_is_normalised(source, configured, expected, why):
setup = (
"Remove-Item Env:UNSLOTH_WOA_WHEELHOUSE -ErrorAction SilentlyContinue; "
if configured is None
else f"$env:UNSLOTH_WOA_WHEELHOUSE = '{configured}'; "
)
script = (
"$ErrorActionPreference = 'Stop'; "
+ setup
+ _wheelhouse_assignment(source).strip()
+ '; Write-Output "<<<$script:WoaWheelhouse>>>"'
)
done = subprocess.run(
[PWSH, "-NoProfile", "-NonInteractive", "-Command", script],
capture_output = True,
text = True,
timeout = 120,
)
assert done.returncode == 0, done.stderr
out = done.stdout
assert out[out.index("<<<") + 3 : out.rindex(">>>")] == expected, why

View file

@ -0,0 +1,314 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Shared plumbing for the Windows-on-ARM installer tests.
install.ps1 and studio/setup.ps1 cannot dot-source each other and cannot be imported, so a
behavioural test lifts a function body or a live block out of the script under test, pastes
it into a bare pwsh session with the few stubs that block needs, and asserts on what it
writes out. The sources, the lifting and the stub bundles are the same from one test to the
next, so they live here rather than being restated in every module that wants them.
"""
from __future__ import annotations
import pathlib
import re
import shutil
import subprocess
import pytest
PACKAGE_ROOT = pathlib.Path(__file__).resolve().parents[3]
MANIFEST_PY = PACKAGE_ROOT / "studio" / "install_manifest.py"
SETUP_PS1 = PACKAGE_ROOT / "studio" / "setup.ps1"
STACK_PY = PACKAGE_ROOT / "studio" / "install_python_stack.py"
STACK_LLAMA = PACKAGE_ROOT / "studio" / "install_llama_prebuilt.py"
INSTALL_PS1 = PACKAGE_ROOT / "install.ps1"
# Read once. Well over a hundred tests want one of these whole files, and none of them
# mutate what they read.
INSTALL_SRC = INSTALL_PS1.read_text(encoding = "utf-8")
SETUP_SRC = SETUP_PS1.read_text(encoding = "utf-8")
STACK_SRC = STACK_PY.read_text(encoding = "utf-8")
LLAMA_SRC = STACK_LLAMA.read_text(encoding = "utf-8")
CONSTRAINTS_SRC = (
PACKAGE_ROOT / "studio" / "backend" / "requirements" / "single-env" / "constraints.txt"
).read_text(encoding = "utf-8")
PWSH = shutil.which("pwsh")
requires_pwsh = pytest.mark.skipif(PWSH is None, reason = "pwsh not available")
# The channels, mirrors and indexes these tests name over and over. Spelled once so a
# parametrize row stays one readable line instead of six.
NV_GA = "https://pypi.nvidia.com/nvtorch_oot"
NV_NIGHTLY = "https://pypi.nvidia.com/nvtorch_oot_nightly"
PYPI = "https://pypi.org/simple"
CORP_INDEX = "https://pypi.corp.test/simple"
def _script(*lines: str) -> str:
"""The lines of a PowerShell snippet, joined the way pwsh -Command wants them."""
return "\n".join(lines)
def _ps(
script,
timeout = 120,
**kwargs,
):
"""Run a PowerShell snippet and hand back the completed process."""
return subprocess.run(
[PWSH, "-NoProfile", "-NonInteractive", "-Command", script],
capture_output = True,
text = True,
timeout = timeout,
**kwargs,
)
def _ps_ok(
script,
timeout = 120,
**kwargs,
):
"""Same, but fail the test with PowerShell's own stderr when it does not exit 0."""
done = _ps(script, timeout = timeout, **kwargs)
assert done.returncode == 0, done.stderr
return done
def _ps_last(script, **kwargs) -> str:
"""Run a snippet that must succeed and hand back its last line of output.
Every one of these scripts ends in a Write-Output the assertion is about, and a
PowerShell prelude can print before it, so the last line is the answer.
"""
return _ps_ok(script, **kwargs).stdout.strip().splitlines()[-1]
def _ps_kv(script, **kwargs) -> dict:
"""A snippet whose Write-Outputs are NAME=value lines, read back as a dict."""
return dict(l.split("=", 1) for l in _ps_ok(script, **kwargs).stdout.splitlines() if "=" in l)
def _function_source(text: str, name: str) -> str:
"""Extract a PowerShell function by matching balanced braces."""
match = re.search(rf"(?im)^[ \t]*function[ \t]+{re.escape(name)}\b", text)
assert match, f"{name} is not defined in setup.ps1"
start = text.index("{", match.start())
depth = 0
for index in range(start, len(text)):
if text[index] == "{":
depth += 1
elif text[index] == "}":
depth -= 1
if depth == 0:
return text[match.start() : index + 1]
raise AssertionError(f"unbalanced braces in {name}")
def functions(text: str, *names: str) -> str:
"""Several bodies in one lift, in the order given.
PowerShell does not hoist and a helper the prelude does not lift is a
command-not-found, not a false answer, so a body's helpers come with it.
"""
return "\n".join(_function_source(text, name) for name in names)
def _ps_function(path: pathlib.Path, name: str) -> str:
"""A function body by SCRIPT PATH, for the tests that parametrize over the two files."""
src = {INSTALL_PS1: INSTALL_SRC, SETUP_PS1: SETUP_SRC}.get(path)
return _function_source(src if src is not None else path.read_text(encoding = "utf-8"), name)
def _ps_copies(name: str) -> tuple:
"""install.ps1's and setup.ps1's copies of one function, bar comments and indentation.
Neither script can dot-source the other, so each is carried twice and the parity is
pinned instead. Returned as a pair rather than compared here so a mismatch fails with
pytest's own diff of the two bodies.
"""
def normalized(source: str) -> str:
lines = [
line.rstrip()
for line in _function_source(source, name).splitlines()
if line.strip() and not line.strip().startswith("#")
]
indent = min(len(line) - len(line.lstrip()) for line in lines)
return "\n".join(line[indent:] for line in lines)
return normalized(INSTALL_SRC), normalized(SETUP_SRC)
def slice_between(
src: str,
start_marker: str,
end_marker: str,
*,
include_end = False,
) -> str:
"""The live text from one marker to the next, sliced out of the script itself.
Restating a block in the test instead means a copy that passes forever after the
original stopped matching it, which is the failure these tests exist to catch.
"""
start = src.index(start_marker)
end = src.index(end_marker, start)
return src[start : end + len(end_marker)] if include_end else src[start:end]
# ── Stub bundles ──────────────────────────────────────────────────────────────────────────
SUBSTEP_NOOP = "function substep { param($m, $c) }"
def substep_collector(var: str = "Messages") -> str:
"""substep, collected into $script:<var> so a test can assert on what was said."""
return _script(
f"$script:{var} = @()",
f"function substep {{ param($m, $c) $script:{var} += $m }}",
)
# The real one shortens a path to its 8.3 form; a passthrough is what every test that is
# not about 8.3 wants, and it keeps the assertions readable.
UV_SAFE_PATH = "function Get-UvSafePath { param([string]$Path) return $Path }"
JOIN_URL_PATH = _script(
"function Join-UrlPath { param([string]$Base,[string]$Path)",
" return ($Base.TrimEnd('/') + '/' + $Path.TrimStart('/')) }",
)
JOIN_URL_RETURNS_BASE = "function Join-UrlPath { param($Base, $Path) return $Base }"
JOIN_URL_RETURNS_PATH = "function Join-UrlPath { param($Base, $Path) return $Path }"
def invoke_restmethod(body: str) -> str:
"""Invoke-RestMethod serving one synthetic PEP 503 page, so the probes are offline."""
return (
"function Invoke-RestMethod { param([Parameter(ValueFromRemainingArguments=$true)]$a) "
f"return @'\n{body}\n'@ }}"
)
INVOKE_RESTMETHOD_OFFLINE = (
"function Invoke-RestMethod { param([Parameter(ValueFromRemainingArguments=$true)]$a) "
"throw 'offline' }"
)
INVOKE_RESTMETHOD_NO_NETWORK = "function Invoke-RestMethod { throw 'no network in this test' }"
# The resolver settings a lifted block must not inherit from the session running the tests.
UV_INDEX_ENV = (
"UV_NO_INDEX",
"PIP_NO_INDEX",
"UV_DEFAULT_INDEX",
"UV_INDEX_URL",
"PIP_INDEX_URL",
"UV_INDEX",
"UV_EXTRA_INDEX_URL",
"PIP_EXTRA_INDEX_URL",
"UV_CONFIG_FILE",
)
UV_POLICY_ENV = ("UV_OFFLINE",) + UV_INDEX_ENV + ("UV_NO_CONFIG",)
UV_ONLY_INDEX_ENV = (
"UV_NO_INDEX",
"UV_DEFAULT_INDEX",
"UV_INDEX_URL",
"UV_INDEX",
"UV_EXTRA_INDEX_URL",
"UV_CONFIG_FILE",
)
def clear_env(names) -> str:
"""Remove every one of `names` from the environment the lifted block will read."""
listed = ",".join(f"'{name}'" for name in names)
return f'foreach ($n in {listed}) {{ Remove-Item "Env:$n" -ErrorAction SilentlyContinue }}'
# The four marker helpers travel together: the path builder, the write guard, the writer and
# the reader. Injecting one without the others is a command-not-found inside the body.
MARKER_FUNCS = functions(
SETUP_SRC,
"Get-WoaTorchIndexMarkerPath",
"Test-WoaPersistableIndex",
"Save-WoaTorchIndexMarker",
"Get-WoaTorchIndexMarker",
)
# Tag matching, the version order it needs, the floor and the pyarrow gate built on both.
PYARROW_FLOOR = '$script:WoaPyarrowFloor = "21.0.0"'
WHEEL_TAG_FUNCS = functions(INSTALL_SRC, "Test-WoaWheelTags", "Test-WoaWheelTagsUsable")
PYARROW_USABLE_FUNCS = _script(
WHEEL_TAG_FUNCS,
_function_source(INSTALL_SRC, "Test-WoaVersionAtLeast"),
PYARROW_FLOOR,
_function_source(INSTALL_SRC, "Test-WoaPyarrowWheelUsable"),
)
def pyarrow_source_script(
*,
wheelhouse: str,
local: str = "$false",
rest_method: str = INVOKE_RESTMETHOD_NO_NETWORK,
reaches_pypi: str | None = None,
lifts: tuple = (),
preamble: tuple = (),
tail: tuple = ("Write-Output ('[' + (Get-WoaPyarrowSource -PythonMinor '3.13') + ']')",),
) -> str:
"""Get-WoaPyarrowSource with its network branches stubbed out.
`wheelhouse` is the PowerShell expression for $script:WoaWheelhouse, `lifts` names any
extra install.ps1 helper the branch under test reaches for.
"""
return _script(
SUBSTEP_NOOP,
JOIN_URL_RETURNS_BASE,
f"function Test-WoaWheelhouseIsLocal {{ {local} }}",
rest_method,
*((f"function Test-WoaResolveReachesPyPI {{ {reaches_pypi} }}",) if reaches_pypi else ()),
f"$script:WoaWheelhouse = {wheelhouse}",
PYARROW_USABLE_FUNCS,
functions(INSTALL_SRC, "Test-ZipArchiveReadable", *lifts, "Get-WoaPyarrowSource"),
*preamble,
*tail,
)
def native_probe_script(
*,
driver: str,
driver_leaf: str = "$null",
stubs: tuple = (),
lifts: tuple = (),
outputs: tuple = (),
) -> str:
"""Initialize-WoaNativeCudaTorch run against a stubbed host.
`stubs` carries the index list and the wheel answers, which is all the callers differ
by; `outputs` names the extra script variables to read back between NATIVE and MSG.
"""
return _script(
"$SkipTorch = $false",
substep_collector(),
"function Get-HostMachineArch { 'arm64' }",
"function Get-WoaAbiTag { param($PythonMinor, $FreeThreaded) 'cp313' }",
"function Test-WoaNvidiaPresent { $true }",
"function Test-WoaResolverPathsUsable { $true }",
f"function Get-WoaDriverCudaLeaf {{ {driver_leaf} }}",
f"function Get-WoaDriverCudaVersion {{ {driver} }}",
*stubs,
"function Get-WoaPyarrowSource { param($PythonMinor, $AbiTag) 'pypi' }",
"function Test-WoaWheelAvailable { $true }",
functions(
INSTALL_SRC, *lifts, "Test-WoaAudioMatchesTorch", "Initialize-WoaNativeCudaTorch"
),
"Initialize-WoaNativeCudaTorch -PythonMinor '3.13'",
"Write-Output ('NATIVE=' + $script:WoaNativeCudaTorch)",
*outputs,
"Write-Output ('MSG=' + ($script:Messages -join ' | '))",
)

View file

@ -3,8 +3,8 @@
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
# Unit test for install.ps1's New-UnslothTorchOverridesFile, the Windows twin of install.sh's
# _build_unsloth_torch_overrides. It folds a caller's UV_OVERRIDE file into the frozen torch-trio
# pins without corrupting it: non-ASCII lines must survive, and the merged file must stay beside
# the caller's override, as uv resolves `-r nested.txt` relative to THAT file's own directory.
# pins without corrupting it: non-ASCII lines must survive, and every relative reference must come
# across REBASED, since uv resolves them against the file that contains them.
# Run: pwsh -NoProfile -File tests/studio/test_torch_overrides_merge.ps1
$ErrorActionPreference = "Stop"
@ -21,6 +21,14 @@ $fn = $ast.FindAll({ param($n)
$n.Name -eq "New-UnslothTorchOverridesFile"
}, $true)
if ($fn.Count -ne 1) { throw "expected exactly one New-UnslothTorchOverridesFile in install.ps1, found $($fn.Count)" }
# PowerShell does not hoist, so the helpers the merge calls have to be defined here too.
foreach ($helper in "Get-WoaRequirementEntries", "Resolve-WoaOverrideLine") {
$h = $ast.FindAll({ param($n)
$n -is [System.Management.Automation.Language.FunctionDefinitionAst] -and $n.Name -eq $helper
}, $true)
if ($h.Count -ne 1) { throw "expected exactly one $helper in install.ps1, found $($h.Count)" }
Invoke-Expression $h[0].Extent.Text
}
Invoke-Expression $fn[0].Extent.Text
$failures = 0
@ -77,14 +85,14 @@ try {
Check "non-ASCII caller requirement survives the merge (no '?' substitution)" `
($mergedText -match "caf${acute}pkg==1\.0" -and $mergedText -notmatch 'caf\?pkg')
Check "merged file sits where the caller's relative includes still resolve" `
(Test-Path -LiteralPath (Join-Path (Split-Path -Parent $merged) "nested.txt"))
Check "the caller's relative include is rebased, not carried as a relative line" `
($mergedText -notmatch '(?m)^\s*-r\s')
Check "and its contents come across instead" ($mergedText -match '(?m)^idna==3\.6$')
# Guards on the behaviour that already worked, so the fix cannot regress it.
Check "frozen torch trio is pinned first" ($mergedText -match '^torch==2\.11\.0\+cu130')
Check "caller's torch-trio lines are dropped" `
($mergedText -notmatch '(?m)^torch==1\.0$' -and $mergedText -notmatch '(?m)^torchvision==0\.1$')
Check "caller's relative include line is carried over" ($mergedText -match '(?m)^-r nested\.txt$')
Check "caller's ordinary requirements are carried over" ($mergedText -match '(?m)^plainpkg==2\.0$')
Check "merged file is newline terminated" ($mergedText.EndsWith("`n"))
@ -141,7 +149,7 @@ try {
# Three Windows-only hazards, so each is asserted behaviourally where it can be and
# structurally from the AST, which runs everywhere.
$env:UV_OVERRIDE = $ovPath
$env:UV_OVERRIDE = $callerOv
$script:TorchOverridesFile = $null
$merged = New-UnslothTorchOverridesFile -PythonExe $fakePy
$made += $merged
@ -156,8 +164,14 @@ try {
# Get-Content decodes a BOM-less file with the ANSI code page on PS 5.1, which is where the
# mojibake came from. pwsh on Linux defaults to UTF-8, so the round trip cannot fail here
# even unfixed; the assertion holding the fix in place is the one on the reader.
# The reading moved into Get-WoaRequirementEntries; the non-ASCII check above holds it.
$scanSrc = ($ast.FindAll({ param($n)
$n -is [System.Management.Automation.Language.FunctionDefinitionAst] -and
$n.Name -eq "Get-WoaRequirementEntries"
}, $true))[0].Extent.Text
Check "the caller's override is read as UTF-8, not by the shell's default codepage" (
($src -match '\[System\.IO\.File\]::ReadAllLines\(') -and ($src -notmatch 'Get-Content -LiteralPath \$ovFile')
($scanSrc -match '\[System\.IO\.File\]::ReadAllLines\(') -and
($src -notmatch 'Get-Content -LiteralPath \$ovFile')
)
Check "the inherited ACL is replaced rather than kept" (
($src -match 'SetAccessRuleProtection\(\$true, \$false\)') -and ($src -match 'Set-Acl -LiteralPath \$f')

View file

@ -97,7 +97,8 @@ Remove-Item -LiteralPath $leakFile -Force -ErrorAction SilentlyContinue
Write-Host "the inherited-override filter drops the torch trio in any casing"
# PowerShell's -notmatch is case-insensitive, so a caller override `Torch<2.11` is dropped.
$filterPattern = $null
if ($fnText -match '\$_ -notmatch ''([^'']+)''') { $filterPattern = $Matches[1] }
# The merge folds entry by entry, so the trio filter reads `-match ... { continue }` instead.
if ($fnText -match '\$ovEntry\.Line -match ''([^'']+)''') { $filterPattern = $Matches[1] }
Check "filter pattern extracted from the helper" ($null -ne $filterPattern)
$inherited = @(
"# comment survives",