* installer: stop condemning a warm uv cache for a directory uv never writes
* Cover every uv 0.12.1 cache bucket in the probe allowlist for PR #10899
* Match uv cache bucket kinds case-insensitively on Windows for PR #10899
* Resolve cased uv cache buckets by asking the filesystem for PR #10899
* Decide the uv cache filesystem's case folding with a probe for PR #10899
* Measure case folding on the Windows cache directory too for PR #10899
* Tighten the comments added for PR #10899
---------
Co-authored-by: danielhanchen <elliegouldingstuff@gmail.com>
* installer: the adaptive uv cache selection was unreachable on every writable install
Two halves of `install.sh`, each correct to read, wrong in the order they run.
The block below `# Keep uv's cache on the same filesystem as the venv it fills` exports
`UV_CACHE_DIR="$STUDIO_HOME/cache/uv"` whenever the caller left it unset. It has to run there
and not later: uv aborts on a cache it cannot create, and several install steps come before
`_configure_uv_cache`.
`_configure_uv_cache` runs much further down, and its first `case` matched any non-blank
`${UV_CACHE_DIR-}`, set `_UV_CACHE_MODE=custom` and returned. It cannot tell a caller's choice
from the installer's own default, because from inside the variable they are the same string.
So on every writable install the two together took the `custom` branch:
* the `env -u UV_CACHE_DIR uv cache dir` probe never ran;
* `shared` was never selected, so a machine with a populated uv cache re-downloaded
multi-gigabyte Torch and CUDA wheels into a second one;
* `_prepare_studio_uv_cache_for_launch`, which opens
`[ "${_UV_CACHE_MODE:-}" = shared ] || return 0`, was dead code;
* and `--isolated-uv-cache` never reached its own branch either. The directory it produced was
right by coincidence -- `custom` and `isolated` both leave `$STUDIO_HOME/cache/uv` -- but the
mode was wrong, so the "already-cached packages may download again" warning the flag exists to
print was never printed.
Fixed by recording whether the value is ours: `_UV_CACHE_DIR_INSTALLER_DEFAULT`, set true only
where the installer assigns the default and cleared again on the unwritable-cache fallback. A
caller's `UV_CACHE_DIR` keeps its priority; the installer's own default falls through to the
selection it was always meant to reach.
`tests/sh/test_uv_cache_adaptive_selection.sh` runs the two halves in the order the installer
runs them, which is the only way this is visible -- neither half is wrong on its own. Anchored
on text that predates the fix, so it exercises the old code rather than failing to find it:
against unmodified main it reports 5 failures, and 13/13 with the fix.
Nothing else moved. `test_uv_cache_colocation.sh` 17/17, `test_install_uv_cache_root.sh` all
passed, `test_uv_pinned_release.sh` 72/72, `test_uv_cache_discipline.py` and
`test_ci_shell_suite_coverage.py` 39/39. `sh -n` and `bash -n` clean.
* Probe the shared uv cache for writes and absolutize it before scanning
* Probe every populated uv cache bucket, not just the cache root
* Keep a warm bucket warm when the scan pipeline reports SIGPIPE
* Treat a cache bucket we cannot enter as one we cannot write
* Probe every existing cache bucket and expand the scan globs under set -f
* Cut the rationale comments this change added down to the load-bearing lines
* Refuse a cache bucket that is not a directory
* Keep CACHEDIR.TAG from failing the bucket probe, and stop the WSL reroute forwarding our own cache path
* Let a rerun keep the warm cache it already recorded
* Probe only the cache root and uv's own buckets
* Match uv bucket names exactly, and fail the probe when it cannot be cleaned up
* Apply the bucket-name rule to the warmth scan too, and stand down under UV_NO_CACHE
* Read UV_NO_CACHE the way uv reads it
* Require a uv artifact kind before counting a bucket as warm
* Select the uv cache by the same rules on Windows, and stop the POSIX side losing a cache it already has
Three things the selection could not see, all found by running it rather than reading it.
## install.ps1 never had the rules install.sh grew here
install.ps1 sets UV_CACHE_DIR in one place, inside the selector, so it never had the ordering
defect this branch fixes and its adaptive selection has always run. What it does not have is
everything the selection learned since. Measured by running its own selector: a rerun abandoned
a warm Studio cache the moment one unrelated wheel made uv's default read as warm, UV_NO_CACHE
still probed and recorded a cache uv was not using, `archive-v0.backup` counted as warmth, and a
populated-but-unwritable cache was selected, which uv aborts on -- that last one fails the
install outright. Two of those are the same bugs this branch fixed on POSIX, and the update path
(unsloth_cli/commands/studio.py) already behaved the new way on Windows, so install.ps1 was the
odd one out on its own platform.
So the marker precedence, the UV_NO_CACHE stand-down, the <kind>-v<N> bucket rule and the
create-and-delete write probe are now on both sides, along with the `studio`-not-`shared`
verdict when the chosen cache already is the Studio cache.
tests/python/test_windows_uv_cache_selection.py runs install.ps1's real selector under pwsh and
asserts the behaviour, the way tests/sh/test_uv_cache_adaptive_selection.sh does for the POSIX
half. Against install.ps1 as it was: 12 of its cases fail.
## An install from before the marker loses the cache it already has
The candidate list was [marker, uv's default]. An install from before b66d2a4c8 has a populated
$STUDIO_HOME/cache/uv and nothing recording it, so one unrelated wheel anywhere in uv's default
read as warm, the install switched to it, and the Torch and CUDA already on disk were
re-downloaded and then orphaned -- while the step line said it was avoiding duplicate downloads.
That is the whole installed base from before the marker, and on POSIX the case could not arise
before this branch, because the custom branch always fired. A warm Studio cache is self-evidently
ours, since nothing else writes there, so it is a candidate when there is no marker. With a
marker it is not: a marker naming somewhere else is a decision. Both installers.
## The launch could repoint at a cache uv would abort on
shared mode is reachable only when the early block failed its own write probe on
$STUDIO_HOME/cache/uv and unset UV_CACHE_DIR, which is what lets the selection run at all. The
launch then repointed at that same directory unconditionally, so an install that succeeded
handed the autostarted backend a cache uv aborts on. Both installers now probe before repointing
and keep the shared cache otherwise: that is the cache this install actually filled.
## Also
* The sh marker read was the only one of the three that did not tolerate a BOM or a CR.
install.ps1 writes the file with Set-Content -Encoding utf8, which under Windows PowerShell
5.1 means both, and a WSL install shares $STUDIO_HOME with the Windows one. A trailing CR
failed [ -d ] and the warm cache was abandoned in silence; a BOM made the value non-absolute,
so $PWD was prepended to it.
* A trailing separator now comes off in _absolutize_uv_cache_dir and Resolve-StudioUvCachePath.
It compared unequal to the Studio cache, so a marker written with one was announced as a
shared cache and got the launch repoint the studio branch exists to avoid.
* UV_NO_CACHE takes clap's literals, which is what uv binds it to (BoolishValueParser over
`y yes t true on 1`). `y` and `t` were missing from all three copies, and missing one leaves
us probing and recording a cache uv is not using. studio.py also stopped trimming, for the
same reason install.sh never did: uv rejects a padded value rather than reading it as true.
* An unreadable first candidate no longer costs us the second. Under ErrorActionPreference =
Stop a Test-Path inside an ACL-denied directory throws, and the throw reached the loop's own
catch, so a marker naming a cache the user can no longer stat took uv's default down with it.
install.sh's `[ -d ] && [ -r ]` simply reads false, which is the behaviour to match.
* $scanBlocked is per candidate on the ps1 side. The warmth scan can report blocked and still
return true, so a shared flag pinned the NEXT candidate's name into the warning and suppressed
the one message that named the cache actually refused.
* Neither "cached packages may download again" warning names the directory being fallen back TO.
The Studio cache is a candidate itself now, so it can be the one that was refused.
## Test
tests/sh/test_uv_cache_adaptive_selection.sh gains the cases above and two arrangements it never
used: install.sh is #!/bin/sh with set -e on line 5, and every case ran one shell without
errexit. The core verdicts now run under every /bin/sh on the box, with errexit both ways.
tests/sh, all 64 files pass
test_uv_cache_adaptive_selection.sh 81 pass, against main 18/81
test_windows_uv_cache_selection.py 36 pass, against install.ps1 as it was 24/36
test_windows_python_venv_hardening.py 51 pass
test_cross_platform_parity.py 83 pass
test_studio_update_uv_cache.py 76 pass
test_windows_python_venv_hardening.py's harness had to learn the new helper names: the selector
wraps its candidate loop in a catch, so a helper it cannot find does not raise, it quietly turns
every case in that file into "studio, could not be inspected".
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Hand PowerShell the path it was given, so the Windows cases assert on Windows
Found by running the new suite on real Windows runners rather than only under pwsh on Linux.
`json.dumps` produced a double-quoted PowerShell string, and PowerShell does not read `\` in a
double-quoted string as an escape. So `C:\Users\runneradmin\...` reached the selector as a path
with DOUBLED separators, and every Windows assertion in the file compared a doubled path against
a real one: 6 of the cases failed on windows-latest and windows-11-arm, and one of them
(`a trailing separator in the marker is the same directory`) failed on the mode rather than the
path, because the marker Python wrote held the real path while `$studioCache` held the doubled
one, so they compared unequal and the selection read `shared`.
It passed on Linux and macOS because a POSIX path has no backslashes to double, which is exactly
the shape of bug a cross-platform run exists to catch and a single-platform run cannot.
A single-quoted PowerShell string is literal and `''` is its only escape, so the paths arrive as
given. The last case in the file is the guard: it hands pwsh a Windows path, a UNC path, a path
with a space and a path with a quote, and asserts each comes back byte-identical.
* A fallback we cannot write is not a fallback
The write probe refuses a whole cache for one bucket uv may never touch, and the early block has
already given up on $STUDIO_HOME/cache/uv when it could not write there -- which is the only
reason the selection is running at all. So the certain failure and the merely suspect cache can
both be on the table at once, and the selection landed on the certain one.
Reproduced with real uv 0.10.7, not a stub: a warm shared cache carrying an unused read-only
bucket, beside a Studio cache whose root is 0555. Before this change the install selected the
Studio cache and `uv pip install` exited 2, unable to create CACHEDIR.TAG. It now keeps the
populated cache, and the warning below still names it as the one that failed the probe.
The rule is narrow on purpose: only when a warm candidate was actually refused, only when it is
not the Studio cache itself, and only when the Studio cache really cannot be created and written,
tested the same way everything else here is -- a real create and delete rather than a mode bit.
A writable Studio cache still wins over a refused candidate, which the second new case pins.
`_uv_cache_root_is_writable` / `Test-StudioUvCacheRootWritable` also replace the inline probe the
launch repoint grew in the previous commit, so there is one answer to "can we fill this" per
installer rather than two copies drifting apart.
test_uv_cache_adaptive_selection.sh 86 pass
test_install_uv_cache_root.sh 89 pass
test_windows_uv_cache_selection.py 43 pass
test_cross_platform_parity.py 83 pass
test_windows_python_venv_hardening.py 51 pass
test_studio_update_uv_cache.py 76 pass
Both extraction harnesses had to learn the new helper name. That is the same trap as last time
and it fails loudly in only one of them: test_install_uv_cache_root.sh rebuilds the selector from
a named list and a missing name is a `command not found` inside `|| return 0`, so the launch
repoint silently stopped happening in three unrelated cases rather than erroring.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Spawn pwsh through the shared runner in the installed-release printer test
Repo tests (CPU) went red on this branch with
Unhandled exception. System.IO.FileLoadException: The given assembly name was invalid.
from tests/studio/install/test_installed_release_backend_line.py, across 40-odd
parametrisations at once. That is not the printer. It is the failure
tests/_shared/unsloth_pwsh_runner.py was written for, and its module docstring names this
exact exception: every -NonInteractive startup reads and rewrites one ~83 KB
StartupProfileData file under $XDG_CACHE_HOME, all four xdist workers share one $HOME, and a
startup that deserialises a half-written one dies before it reaches the script. Measured there
at 7 deaths per 4000 startups on a shared cache dir and 0 on private ones.
This file spawned pwsh with a bare subprocess.run, so it never got the private cache or the
retry. It is also one of the densest pwsh users in the job, which is why it is where the race
surfaced once this branch added another 43 pwsh-spawning cases to the same run.
No behaviour of the test changes: run_pwsh is subprocess.run plus a private cache directory per
worker and a retry for an interpreter that died without producing a verdict. A pwsh that runs to
completion and gives the wrong answer still fails, first time, with its own message.
496 passed, 2 skipped locally.
* Reject an empty version in the uv bucket-name rule
`##*-v` strips through the LAST `-v`, so a name ending in one leaves an empty
suffix behind, and no `*[!0-9]*` matches an empty string. `archive-v1-v` read
as a bucket, the write probe covered it, and one read-only directory a user
happened to name that way condemned an otherwise usable warm cache, sending a
machine that already holds Torch and CUDA back to redownload them.
Every other numeric-suffix case in install.sh already spells this `''|*[!0-9]*`,
and Test-StudioUvBucketName has rejected it on IsNullOrEmpty since it was
written, so this was a real sh/ps1 split rather than a spelling difference. The
parity test now asserts both halves of the rule instead of just the function
names.
* Cut the 4688 compiler window back to the window it was given
The no-compiler lane failed on this branch reporting one compiler process. The
record it named is timestamped 17:51:57.107, and the step it was scored against
did not print its first line until 17:51:58.58, so nothing in install.ps1 can
have spawned it. It belongs to the positive control one step earlier, which
compiles a type on purpose.
That is the case $prior exists to subtract, and it did not: the Security log is
written asynchronously, so a record 1.5s old can be absent when $prior samples
it and present by the time the measurement reads back. Either way the record's
own TimeCreated is below the floor, so the floor is the thing to enforce. The
query now asks for a second more than the window at each end and cuts the result
back by TimeCreated, which is the instant the claim is actually about. The pad at
the far end also removes the opposite risk, of a bound that rounds the wrong way
dropping a real compile in the window's last fraction of a second.
$prior stays as it was. This narrows what can be counted, and nothing that a
compile inside the window did stops being counted.
* Fail the root write probe when the probe cannot be removed
uv renames distributions into these directories, so a root that grants create
and denies unlink fails the install later rather than not at all, and the probe
file is left behind in the user's cache. NTFS carries DELETE as its own ACE and
an append-only directory does the same on ext4, so this is expressible on both
platforms.
The bucket probes already decided this: install.sh:812 fails the candidate when
rm does, and the PowerShell bucket loop uses -ErrorAction Stop, with a comment
naming the NFSv4 and CIFS case. Only the three root probes disagreed with their
own sibling, by way of `|| true` and SilentlyContinue. They now agree.
The regression case simulates a failing rm rather than an ACL, because the mode
bits cannot express create-without-unlink and a suite running as root would hide
it either way. It fails against the old helper.
test_cross_platform_parity.py had one slice that ran from the populated scan to
the marker reader, two functions on, so it read "no -ErrorAction Stop here" over
a helper that uses Stop deliberately. Its comment already said the scan only;
the slice now ends where the scan does.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Believe the filesystem, not the delete, and treat a stale marker as stale
Three findings from re-auditing the branch, each reproduced before and after.
A probe that could not be deleted condemned the cache on the first failure. An
indexer or a scanner holding the handle open makes one delete fail and the next
succeed, and PowerShell's Remove-Item -ErrorAction Stop throws outright for a
probe something else already removed, where rm -f exits 0. So the two sides
disagreed, and a transient error could divert an install away from the Studio
cache and into a shared one already known to be worse: an offline install that
used to succeed then failed. Both helpers now retry and judge by whether the
probe is GONE.
install.sh's early block kept its own copy of that rule. It now calls the same
helper, so the two cannot drift, and tests/sh/test_uv_cache_colocation.sh
extracts the helper alongside the block: a missing function is `command not
found`, which exits 127, which `if !` reads as "not writable", so every case
would have quietly reported an unset UV_CACHE_DIR.
A marker naming a directory that no longer exists was treated as a decision and
suppressed the Studio cache, so a warm Studio cache holding Torch and CUDA lost
to uv's default. A stale pointer is not a decision. The other half of the rule
is kept as its own case: a recorded cache that still EXISTS and is merely cold
does still fall through to shared, which is the machine that really has moved on.
Test-StudioUvBucketName used ^[0-9]+$, and in .NET `$` also matches before a
final newline, so `archive-v1<LF>` passed there and failed in sh. \A and \z.
* Source the helpers before the early block, and prove the harness can
My own last commit put a call to _uv_cache_root_is_writable inside install.sh's
early block, and this harness sourced that block BEFORE the file defining the
helper. A missing command exits 127, `if !` reads 127 as "not writable", and the
block then unset the very default whose survival every ordering case depends on.
The suite went on reporting 93 passes while no longer exercising the writable
install ordering it was written to guard.
install.sh defines those helpers around line 622 and runs the early block at
line 1001, so definitions first is also the faithful order.
The order now lives in one file, $_BOOT, sourced by the runner and by a new
check alike. The first version of that check repeated the order instead of
sharing it, so reversing the runner left it green -- the same shape of mistake
one level up. Reversing $_BOOT now fails the suite before any case runs.
The check asserts only that the block kept a cache it could create, not that
_UV_CACHE_DIR_INSTALLER_DEFAULT is true: that flag is what this branch adds, and
the suite has to keep running against code that predates it. Against the merge
base and against main it is 27 passed, 66 failed; on this branch, 93 and 0.
* Tighten the comments added by this PR for #10626
* Honor UV_NO_CACHE before recording a cache for PR #10626
The custom and isolated branches both choose a directory without probing, and
both returned before the UV_NO_CACHE check, so they recorded a marker naming a
cache uv never touched. Checked against uv 0.10.7: with UV_NO_CACHE=1 a caller's
UV_CACHE_DIR is left completely empty, CACHEDIR.TAG included. A later repair
reads that marker and can prefer an unrelated cache that happens to hold bytes.
Both installers were the outliers here, the same way as the earlier fixes on
this branch: unsloth_cli/commands/studio.py has always asked
_uv_no_cache_requested() before choosing or recording anything.
The spelling test is now one helper per installer rather than a case and an
inline -in, so the three copies of clap's literal set cannot drift apart.
Each branch keeps its own directory: a caller's UV_CACHE_DIR is still exported
exactly as written, and isolation still points at the Studio cache. Only the
marker write is skipped, and the custom step line says so.
Tests cover both directions on both sides, since a guard that simply stopped
recording would also pass the first half. The new helper is added to every
extraction list; a name missing from one of those does not fail loudly.
* Give the CLI updater the two cache rules the installers gained for PR #10626
The installers stopped abandoning a warm Studio cache on a pre-marker or stale
marker install, and stopped counting `archive-v0.backup` as warmth. The CLI is
what an existing installation actually runs on `unsloth studio update`, and it
had neither, so the update could undo the fix on the same machine.
_uv_cache_has_packages matched bucket names by prefix, which also takes
`archive-v0.backup` and `archive-backup-v0`, whose bytes uv cannot reuse. Its
docstring already claimed the same rule as install.sh. It now validates the
whole `<kind>-v<N>` suffix through a helper that mirrors both installers.
The precedence is POSIX-only, and the asymmetry is historical rather than a
shortcut. install.sh always took its `custom` branch before the marker shipped,
so shared mode was unreachable and nothing else wrote to $STUDIO_HOME/cache/uv:
verified by running the selector from b66d2a4c8~1. A warm unmarked Studio cache
there is this install's own. install.ps1's selection already worked, so a
pre-marker Windows install could have used the shared cache and repointed the
backend at the Studio one, leaving on-demand wheels there while Torch and CUDA
sit in the shared cache. Windows therefore keeps uv's default ahead of it, which
is what the test being retargeted here was protecting.
A marker naming a cache that still EXISTS and has gone cold stays a decision on
both platforms, so the machine that really moved on still reaches uv's default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Ask about buckets, not just the root, where the answer decides for PR #10626
Two sites asked _uv_cache_root_is_writable about a cache whose buckets the
candidate probe had already rejected, so both could hand back the cache uv is
about to abort on. A `sudo` run leaves a root-owned bucket and a dangling link
is an existing path to mkdir(2), so the root can be perfectly writable while the
cache is not.
The fallback chose the Studio cache unconditionally when nothing was warm, since
a cold cache never reaches the warn path. Reproduced: a Studio cache holding a
dangling archive-v0 and a cold default selected the Studio cache, and uv fails
on the first bucket it touches. It now falls through to the populated cache that
only failed the probe, then to uv's default when that is usable, which at worst
costs the downloads this cache was never going to save.
The launch repoint had the same root-only question. Reproduced: with a warm
default the selection correctly chose shared, then the repoint sent the
autostarted backend into the rejected Studio cache anyway.
The rule now lives in one helper per installer, _uv_cache_is_writable and
Test-StudioUvCacheUsable, and the candidate probe calls it too, so the three
sites cannot drift. Both new names went into every extraction list in the same
commit: a missing one is `command not found`, which reads as a clean answer.
* Put uv's default ahead of an unmarked Studio cache for PR #10626
I had the history backwards. The marker arrived in b66d2a4c8 on 2026-09-06 and
the early UV_CACHE_DIR block only in e12963071 the day AFTER, so an install old
enough to have no marker is old enough that `shared` was reachable on POSIX, not
too old. In that mode the launch repoint leaves a few on-demand backend wheels in
$STUDIO_HOME/cache/uv while Torch and CUDA sit in uv's default.
Ordering the unmarked Studio cache ahead of the default therefore picked the
leftovers: reproduced as a real offline install that succeeds on main and fails
on this branch. The earlier check that "confirmed" the opposite hardcoded the
early block into its own harness, so it proved nothing about that revision.
The unmarked Studio cache is now the LAST candidate in both installers, which is
the order unsloth_cli/commands/studio.py already used, so the platform split I
added there is gone with it. A warm Studio cache still wins whenever uv's default
is cold, which is the install the rule was added for, and `unsloth studio update`
is now byte-identical to main across all five marker states an install reaches.
Separately, the fallback treated a Studio cache that is WARM but failed the probe
as unusable and moved to a cold default, losing the bytes. A warm cache that only
failed a speculative bucket probe beats a cold one we can write; a cold Studio
cache with an unusable bucket still loses to uv's default, which is the case that
prompted the probe.
* Keep the CLI bucket rule ASCII for PR #10626
str.isdigit() is true for Arabic-Indic and superscript digits, so the helper
added in c592585ab counted `archive-v<Arabic-Indic one>` as a bucket where the sh
`*[!0-9]*` case and the PowerShell \A[0-9]+\z both reject it. uv writes ASCII, so
Python was the only one of the three that would have read such a directory as
warmth. Exotic, but it is a three-way rule and the other two already agreed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
Co-authored-by: danielhanchen <elliegouldingstuff@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Daniel Han