mirror of
https://github.com/unslothai/unsloth.git
synced 2026-10-22 22:41:04 +00:00
Give unsloth studio update the same uv cache the backend already uses (#10386)
Some checks failed
Core / Core (HF=default + TRL=default) (push) Waiting to run
Core / Core (HF=4.57.6 + TRL<1) (push) Waiting to run
Core / Core (HF=latest + TRL=latest) (push) Waiting to run
Core / llama.cpp build + smoke (push) Waiting to run
Cross-platform parity / parity (ubuntu-latest) (push) Waiting to run
Cross-platform parity / parity (windows-latest) (push) Waiting to run
Kaggle T4 Notebook CI / gate (push) Waiting to run
Kaggle T4 Notebook CI / T4 smoke (push) Blocked by required conditions
Kaggle T4 Unsloth GPU CI / gate (push) Waiting to run
Kaggle T4 Unsloth GPU CI / Unsloth GPU smoke (push) Blocked by required conditions
Lint CI / Source lint (Python + shell + YAML + JSON + safety nets) (push) Waiting to run
Local Agent Guides CI / connection (claude, stable) (push) Waiting to run
Local Agent Guides CI / connection (codex, stable) (push) Waiting to run
Local Agent Guides CI / connection (dsh, stable) (push) Waiting to run
Local Agent Guides CI / connection (hermes, stable) (push) Waiting to run
Local Agent Guides CI / connection (openclaw, stable) (push) Waiting to run
Local Agent Guides CI / connection (opencode, stable) (push) Waiting to run
Local Agent Guides CI / connection (opencode, v2) (push) Waiting to run
Local Agent Guides CI / connection (pi, stable) (push) Waiting to run
Local Agent Guides CI / file-edit (claude) (push) Waiting to run
Local Agent Guides CI / file-edit (codex) (push) Waiting to run
Local Agent Guides CI / file-edit (dsh) (push) Waiting to run
Local Agent Guides CI / file-edit (hermes) (push) Waiting to run
Local Agent Guides CI / file-edit (openclaw) (push) Waiting to run
Local Agent Guides CI / file-edit (opencode) (push) Waiting to run
Local Agent Guides CI / file-edit (pi) (push) Waiting to run
Local Agent Guides CI / resume (claude) (push) Waiting to run
Local Agent Guides CI / resume (codex) (push) Waiting to run
Local Agent Guides CI / resume (opencode) (push) Waiting to run
Local Agent Guides CI / resume (pi) (push) Waiting to run
Local Agent Guides CI / prompt-cache (gemma-3-270m) (push) Waiting to run
Security audit / advisory audit (pip + npm + cargo) (push) Waiting to run
Security audit / pip scan-packages :: extras (push) Waiting to run
Security audit / pip scan-packages :: studio (push) Waiting to run
Security audit / pip scan-packages :: hf-stack (push) Waiting to run
Security audit / npm scan-packages (Unsloth frontend tarballs) (push) Waiting to run
Security audit / npm provenance + new install-script diff (push) Waiting to run
Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Backend CI / (Python 3.11) (push) Waiting to run
Backend CI / (Python 3.13) (push) Waiting to run
Backend CI / Repo tests (CPU) (push) Waiting to run
Unsloth export capability / capability (windows-latest) (push) Waiting to run
Frontend CI / Frontend build + bundle sanity (push) Waiting to run
Frontend CI / Frontend unit tests (Windows) (push) Waiting to run
Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Mac Unsloth Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Unsloth Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Unsloth Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Unsloth Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Unsloth UI + API + Update + Inference CI / Chat UI, API, Update and GGUF inference (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (chat) (push) Waiting to run
Unsloth UI CI / Loaded-models indicator (cross-browser) (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection + VC++ round-trip (VS 2026) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (windows) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (macos) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (extra) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (picker) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (banner) (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 units, no-VS resolve, VC++ round-trip (windows-latest) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
studiobench CI / Selftests, lint and the single-file build (push) Waiting to run
studiobench CI / Real-path session, every action ran (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Windows Application Control CI / installer survives a denied unsloth.exe (push) Waiting to run
Workflow trigger lint / workflow-trigger lint (pull_request_target / cache-poisoning) (push) Waiting to run
Workflow trigger lint / security suites (torch-dependent) (push) Waiting to run
Publish Blackwell Docker image / prepare (push) Has been cancelled
Publish Blackwell Docker image / build (linux/amd64, ubuntu-latest) (push) Has been cancelled
Publish Blackwell Docker image / build (linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Publish Blackwell Docker image / merge (push) Has been cancelled
Publish Blackwell Docker image / build-studio (linux/amd64, ubuntu-latest) (push) Has been cancelled
Publish Blackwell Docker image / build-studio (linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Publish Blackwell Docker image / merge-studio (push) Has been cancelled
Publish Blackwell Docker image / hub-readme (push) Has been cancelled
Publish Blackwell Docker image / cleanup (push) Has been cancelled
Publish Blackwell Docker image / smoke-test (push) Has been cancelled
Some checks failed
Core / Core (HF=default + TRL=default) (push) Waiting to run
Core / Core (HF=4.57.6 + TRL<1) (push) Waiting to run
Core / Core (HF=latest + TRL=latest) (push) Waiting to run
Core / llama.cpp build + smoke (push) Waiting to run
Cross-platform parity / parity (ubuntu-latest) (push) Waiting to run
Cross-platform parity / parity (windows-latest) (push) Waiting to run
Kaggle T4 Notebook CI / gate (push) Waiting to run
Kaggle T4 Notebook CI / T4 smoke (push) Blocked by required conditions
Kaggle T4 Unsloth GPU CI / gate (push) Waiting to run
Kaggle T4 Unsloth GPU CI / Unsloth GPU smoke (push) Blocked by required conditions
Lint CI / Source lint (Python + shell + YAML + JSON + safety nets) (push) Waiting to run
Local Agent Guides CI / connection (claude, stable) (push) Waiting to run
Local Agent Guides CI / connection (codex, stable) (push) Waiting to run
Local Agent Guides CI / connection (dsh, stable) (push) Waiting to run
Local Agent Guides CI / connection (hermes, stable) (push) Waiting to run
Local Agent Guides CI / connection (openclaw, stable) (push) Waiting to run
Local Agent Guides CI / connection (opencode, stable) (push) Waiting to run
Local Agent Guides CI / connection (opencode, v2) (push) Waiting to run
Local Agent Guides CI / connection (pi, stable) (push) Waiting to run
Local Agent Guides CI / file-edit (claude) (push) Waiting to run
Local Agent Guides CI / file-edit (codex) (push) Waiting to run
Local Agent Guides CI / file-edit (dsh) (push) Waiting to run
Local Agent Guides CI / file-edit (hermes) (push) Waiting to run
Local Agent Guides CI / file-edit (openclaw) (push) Waiting to run
Local Agent Guides CI / file-edit (opencode) (push) Waiting to run
Local Agent Guides CI / file-edit (pi) (push) Waiting to run
Local Agent Guides CI / resume (claude) (push) Waiting to run
Local Agent Guides CI / resume (codex) (push) Waiting to run
Local Agent Guides CI / resume (opencode) (push) Waiting to run
Local Agent Guides CI / resume (pi) (push) Waiting to run
Local Agent Guides CI / prompt-cache (gemma-3-270m) (push) Waiting to run
Security audit / advisory audit (pip + npm + cargo) (push) Waiting to run
Security audit / pip scan-packages :: extras (push) Waiting to run
Security audit / pip scan-packages :: studio (push) Waiting to run
Security audit / pip scan-packages :: hf-stack (push) Waiting to run
Security audit / npm scan-packages (Unsloth frontend tarballs) (push) Waiting to run
Security audit / npm provenance + new install-script diff (push) Waiting to run
Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Backend CI / (Python 3.11) (push) Waiting to run
Backend CI / (Python 3.13) (push) Waiting to run
Backend CI / Repo tests (CPU) (push) Waiting to run
Unsloth export capability / capability (windows-latest) (push) Waiting to run
Frontend CI / Frontend build + bundle sanity (push) Waiting to run
Frontend CI / Frontend unit tests (Windows) (push) Waiting to run
Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Mac Unsloth Install Matrix CI / Install + load (macos-15) (push) Waiting to run
Mac Unsloth Install Matrix CI / Install + load (macos-26) (push) Waiting to run
Mac Unsloth Install Matrix CI / Install + load (macos-15-intel) (push) Waiting to run
Mac Unsloth Install Matrix CI / Install + load (macos-26-intel) (push) Waiting to run
Mac Unsloth UI + API + Update + Inference CI / Chat UI, API, Update and GGUF inference (push) Waiting to run
Unsloth Tauri CI / Tauri Linux debug build (no codesign) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (chat) (push) Waiting to run
Unsloth UI CI / Loaded-models indicator (cross-browser) (push) Waiting to run
Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
Windows Unsloth API CI / Unsloth API & Auth Tests (push) Waiting to run
Windows Unsloth GGUF CI / GGUF inference smoke (API, tools, vision) (push) Waiting to run
Windows Unsloth GGUF CI / Unsloth install + inference without Visual Studio (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection (VS 2022) (push) Waiting to run
Windows Unsloth GGUF CI / real-VS detection + VC++ round-trip (VS 2026) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (windows) (push) Waiting to run
Unsloth Tauri CI / Rust unit tests (macos) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (extra) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (picker) (push) Waiting to run
Unsloth UI CI / Chat UI Tests (banner) (push) Waiting to run
Windows Unsloth GGUF CI / setup.ps1 units, no-VS resolve, VC++ round-trip (windows-latest) (push) Waiting to run
Windows Unsloth UI CI / Chat UI Tests (push) Waiting to run
Windows Unsloth Update CI / Unsloth Updating Tests (push) Waiting to run
studiobench CI / Selftests, lint and the single-file build (push) Waiting to run
studiobench CI / Real-path session, every action ran (push) Waiting to run
Wheel CI / Wheel build + content sanity + import smoke (push) Waiting to run
Windows Application Control CI / installer survives a denied unsloth.exe (push) Waiting to run
Workflow trigger lint / workflow-trigger lint (pull_request_target / cache-poisoning) (push) Waiting to run
Workflow trigger lint / security suites (torch-dependent) (push) Waiting to run
Publish Blackwell Docker image / prepare (push) Has been cancelled
Publish Blackwell Docker image / build (linux/amd64, ubuntu-latest) (push) Has been cancelled
Publish Blackwell Docker image / build (linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Publish Blackwell Docker image / merge (push) Has been cancelled
Publish Blackwell Docker image / build-studio (linux/amd64, ubuntu-latest) (push) Has been cancelled
Publish Blackwell Docker image / build-studio (linux/arm64, ubuntu-24.04-arm) (push) Has been cancelled
Publish Blackwell Docker image / merge-studio (push) Has been cancelled
Publish Blackwell Docker image / hub-readme (push) Has been cancelled
Publish Blackwell Docker image / cleanup (push) Has been cancelled
Publish Blackwell Docker image / smoke-test (push) Has been cancelled
* Give `unsloth studio update` the same uv cache the install used Three places pick a uv cache and only two of them agreed. install.sh and install.ps1 choose one for the install itself (#10204), and storage_roots._setup_cache_env seeds cache_root()/uv for the backend server. An update reaches neither: `unsloth studio update` runs setup.sh / setup.ps1 straight from the CLI process, nothing on that path sets UV_CACHE_DIR, and _setup_cache_env is only called from the backend, so uv fell back to its own user-wide default. On a fresh install the installer downloads into <StudioHome>/cache/uv and the first update downloads all of it again into ~/.cache/uv or %LOCALAPPDATA%\uv\cache, where uninstall cannot reclaim it. The rule is the installer's own rather than an unconditional redirect: prefer the Studio cache, and defer to uv's default only when the Studio one holds no package bytes and uv's does. _setup_cache_env mkdirs an empty Studio cache on every server start, so a shared-mode install would otherwise be sent to a cache holding nothing -- one duplicate download online, and with UV_OFFLINE, --offline or uv.toml offline = true, where uv may read only what is cached, an update that cannot resolve at all. Whichever cache wins is passed to the child as an explicit absolute path. uv reads UV_CACHE_DIR as --cache-dir, so a blank inherited value is a missing argument rather than an unset one (`UV_CACHE_DIR= uv cache dir` exits 2 with "a value is required"), and uv answers a relative cache-dir with the relative spelling, which setup.sh would resolve from a different directory. * Resolve uv's cache answer the way uv does Two ways the probe could name a directory uv never uses, both measured on the uv this repo pins, 0.10.7. A configured `cache-dir = "~/.myuv"` is printed verbatim and uv treats the tilde as an ordinary relative segment: it creates a literal "~" directory in its working directory, not one in $HOME. expanduser therefore probed a path that does not exist, so a warm configured cache read as cold, or an unrelated $HOME cache was picked up. A relative cache-dir resolves against uv's working directory, and --directory (env alias UV_WORKING_DIR) moves that out from under ours. Measured: with UV_WORKING_DIR set, `uv cache dir` prints "relcache" and the cache lands under UV_WORKING_DIR, not under the caller's cwd. * Fall back to uv's platform cache when the probe cannot answer uv discovers configuration from the CURRENT directory, so a malformed uv.toml beside whoever ran `unsloth studio update` makes `uv cache dir` exit nonzero even though setup.sh changes directory before it runs uv at all. Treating that as "there is no default cache" sent the update to a cold Studio cache: a duplicate download online, and an offline update that cannot resolve despite a populated platform cache. install.sh:646 already falls back to XDG_CACHE_HOME/uv or HOME/.cache/uv in exactly this situation. The probe now does the same, adding LOCALAPPDATA/uv/cache for Windows, and returns nothing only when there is no anchor to build a path from. * Trim the comments this branch added Delete first, collapse second. The rationale that survives names a mistake the next reader would otherwise make: metadata-only caches reading as warm, UnicodeDecodeError escaping an OSError handler, a blank UV_CACHE_DIR exiting 2, uv's literal tilde. The measured numbers and the alternatives considered are in the PR description, which is where they belong. Prose in the added block: 51 lines to 16. * Record which uv cache the install used, instead of inferring it later (#10410) * Record which uv cache the install used, instead of inferring it later Content cannot answer the question the update asks. In shared mode the installers point the running backend at the Studio cache (_prepare_studio_uv_cache_for_launch, Set-StudioUvCacheForLaunch), so one on-demand install from the server (wheel_utils.py:405) leaves package bytes there. Both caches then hold packages, and the update stops preferring the one the install filled: a duplicate download online, and under UV_OFFLINE or uv.toml offline = true an update that cannot resolve. The mirror case is just as wrong, a studio-mode install losing once the user warms their own default cache with an unrelated uv pip install. Both installers now write the cache they chose to <StudioHome>/cache/uv-cache-dir and the update reads it, falling back to inspecting both caches for installs that predate the marker. Every mode records, a caller's own UV_CACHE_DIR included, since that is where the install put its wheels and an earlier marker left in place would aim later updates at a cache this install never filled. A caller keeps control either way: a nonblank UV_CACHE_DIR outranks the marker at update time too. The marker is honoured only while the cache it names still holds packages, so `uv cache clean` does not leave a stale pointer outranking a warm cache. The marker describes the environment, so it travels with it. It is written absolute, since the update resolves it against its own working directory (on PowerShell that needs $PWD explicitly: GetFullPath resolves against the .NET process directory, which Set-Location does not move). The previous contents are saved before the first write and restored wherever a failed install rolls the venv back, and the snapshot resets per run because under `irm | iex` the script scope is the caller's session. An unwritable Studio root skips the marker rather than failing the install. For installs that predate the marker, the update records what the fallback worked out once setup has returned 0, and replaces a marker whose cache has since been emptied. A caller's UV_CACHE_DIR is never promoted: that belongs to one run, and the installers are what record an install. * Reset the marker snapshot before the selector, and restore it on any failure Four fixes, all on the marker's lifecycle. The per-run reset ran AFTER Set-StudioUvCacheEnvironment, which is what writes the marker, so it cleared the snapshot that call had just taken and made both Restore-StudioUvCacheMarker calls no-ops. A failed reinstall kept the new marker. Moved above the call, and the parity test now pins it to the lines directly above rather than anywhere in the file, since searching the file matches the copy inside the restore helper and passes either way. Restoring only from inside the venv rollback missed every failure that happens before a replacement is in flight: a first install has no previous venv, the ownership guard can refuse the directory, the backup move can fail. The restore now runs from the exit and signal handlers on the sh side and from Exit-InstallFailure and the commit guard's finally on the PowerShell side. Test-Path was unsuppressed under $ErrorActionPreference = "Stop". Probing a path inside a directory the ACL denies throws UnauthorizedAccessException rather than returning false, so an optional marker could abort the install. Measured: Test-Path THREW UnauthorizedAccessException on a chmod 000 parent. The reader stripped whole lines. The installers write UV_CACHE_DIR through verbatim and a directory name may legitimately begin or end with a space, so stripping probed a different path and read a warm cache as cold. splitlines already drops the terminator; blank lines are still skipped, since blank means unset. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Read the marker the way uv reads a path Same reason the probe stopped expanding: the installers write this path through as uv would read it, and uv treats a leading tilde as an ordinary segment rather than $HOME. Both installers already make the value absolute, so expanduser could only ever have been wrong here, never right. * Tie the marker's lifetime to the environment it describes Eight findings, all on when the marker may be written, reverted or kept. Committing the environment commits the marker. The unconditional restore I added reverted it on any nonzero exit, including after the venv was already committed, so a later failure left the installed environment pointing at the previous install's cache. Both shells now clear the snapshot at the commit. The PowerShell reset moves to the first lines of Install-UnslothStudio. It sat after the lock checks, and nothing clears the flag on success, so under `irm | iex` a second install failing early reverted the marker belonging to the first, which had succeeded. A marker that cannot be read is left alone. `cat` failing was recorded as an empty snapshot with existed=true, so a rollback wrote a blank marker over the previous install's; Get-Content failing left $null, and TrimEnd on it throws inside the rollback, before -ErrorAction can apply. Writes no longer follow symlinks. A redirection, Set-Content and write_text all truncate the target rather than replacing the entry, so a marker path linked elsewhere silently destroyed an unrelated file. Relative caches resolve against uv's working directory, which --directory and UV_WORKING_DIR move, rather than the installer's $PWD. The marker directory is created with the .NET API, since New-Item -Path treats brackets in a custom Studio root as a wildcard, as this installer already notes at two other call sites. The backfill is suppressed during a staged update: STUDIO_HOME names the live install there, and the stage can still be rejected. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Anchor a relative uv working directory, and clear the marker with the venv flag UV_WORKING_DIR may itself be relative, and uv resolves it against the directory the installer ran from before it resolves the cache under it. Recording "work/cache" would name somewhere else at update time. Both shells anchor the base first; on PowerShell that has to be $PWD.Path, since GetFullPath would use the .NET process directory instead. The commit cleared the marker flag after the venv flag and after the backup deletion. A signal in that window kept the newly committed environment, because venv rollback was already inactive, and reverted the marker it came with, because that flag was still set. Both are cleared together now, and a parity test asserts the order rather than mere presence. * Commit the marker on a first install, gate its write, and record paths as bytes Three items from the last review. The commit cleared _UV_MARKER_SAVED inside the rollback branch, and a first install has no previous environment, so it took no branch and left the marker revertible. Anything failing after that point, the shim-path guard or a signal, kept the newly installed environment and reverted the marker it came with. Both shells clear it before the branch now. The unlink that keeps a symlinked marker from truncating what it points at can fail silently, on a directory that denies deletion while the target stays writable. Both writes now happen only once the entry is confirmed gone; on PowerShell that probe is Get-Item -Force, since Test-Path follows the link and reports a broken one as absent. A POSIX path may hold bytes that are not UTF-8, and Python hands those over as surrogates. Encoding them raised UnicodeEncodeError, which is not an OSError and escaped the best-effort handler, failing an update whose setup had already succeeded and whose old marker was already unlinked. The path is written with os.fsencode and read back with surrogateescape. Two of the existing marker cases in the rollback suite were passing vacuously: _record_uv_cache_choice sits outside the spliced block, so the harness exited 127 before writing anything, which satisfied every expectation about a marker that must not change. The helper is spliced alongside the others and the cases now fail if the harness writes to stderr or dies on a missing command. * Tighten the comments added by this PR Comments and docstrings only, no code. Verified with comment_tools.py check and by rerunning the suites: 183 passed, the two sh suites under sh, dash and bash, and all 24 pwsh studio tests. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Publish a staged update's cache choice, and treat a malformed marker as cold Two items from the last review. A desktop update never runs the direct path, so suppressing the backfill under --stage left exactly the installs the marker exists for permanently on the content fallback. The staged child still cannot write the live marker, since STUDIO_HOME names the environment it is replacing and the stage may never activate, so it parks the choice in the stage root and _studio_stage.stage promotes it after the probes pass. A stage that fails verification publishes nothing, and a stage that parks nothing leaves the live marker alone. A marker holding an embedded NUL builds a Path fine and then raises ValueError, not OSError, out of scandir, so a corrupted or hand-edited file aborted every update before setup ran. That cache reads as cold now, which is what the rest of this code already does with a cache it cannot inspect. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Probe uv where setup will run it, and record the marker unambiguously Two items from the last review. Both setup scripts change into their own directory before the dependency pass (studio/setup.sh:1788), and uv discovers uv.toml and pyproject.toml from its working directory. The probe asked from the caller's, so running an update while standing in an unrelated project answered with that project's cache, and the answer is then forced on the child through UV_CACHE_DIR. It now runs with the setup script's directory as its working directory, and resolves a relative answer against that same base. A newline is legal in a POSIX path and uv reports one verbatim, but the marker reader split the record into lines and kept the last fragment, so such a cache read as absent and the update redirected uv elsewhere. The record is one path followed by one delimiter now: strip that delimiter, keep everything else. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make one install agree with itself about its cache, and commit in one step Six items from the last review. setup.ps1 never changes directory: it addresses everything through PSScriptRoot and hands install_python_stack.py the cwd it inherited, so probing from the script's directory answered for a configuration that child never sees. The probe now uses the directory setup will actually run uv from, which is the script's on POSIX and the inherited one on Windows. UV_WORKING_DIR may itself be relative, and uv resolves it after starting where the probe started, so it is anchored to the probe's directory rather than to wherever the update was launched. uv reports a cache path verbatim, and stripping the reported line changed a directory name that legitimately ends in a space. Blank lines are still dropped. A relative UV_CACHE_DIR named a different directory in each phase of one install, because setup.sh changes into its own directory before the dependency pass. It is resolved once, before anything reads it, so both phases and the marker agree. What is restored afterwards is still the caller's own spelling. Windows PowerShell 5.1 decodes a BOM-less file with the active ANSI code page, and the update writes this marker BOM-less UTF-8, so a non-ASCII path was read back as mojibake and restored that way. It is read as UTF-8 explicitly. The commit cleared two flags, and a signal between them put the previous environment back while keeping the marker of the attempt that replaced it. One flag decides both rollbacks now, set first and alone, and both restores consult it. * Resolve the Windows cache through one helper, and leave ambiguous installs alone Two items from the last review. The custom branch on Windows anchored a relative UV_CACHE_DIR to $PWD alone, ignoring UV_WORKING_DIR, and rooting it that way also stopped the marker writer from applying its own resolution. From C:\work with UV_WORKING_DIR=subdir and UV_CACHE_DIR=cache, uv uses C:\work\subdir\cache and the installer recorded C:\work\cache. Both call Resolve-StudioUvCachePath now, which mirrors _absolutize_uv_cache_dir. An install that predates the marker cannot say which cache it used, and content cannot tell either: the running backend is pointed at the Studio cache even in shared mode, so one on-demand wheel warms it. Where both caches are warm and no marker settles it, the update keeps uv's default, which is what such an install has been updating from all along. Preferring the Studio cache there was a new way for an offline update to fail on an install that cannot record its way out. A Studio cache that wins on content alone still wins when uv's default is cold, which is the case this PR exists for, and a recorded marker still settles both. * Tighten the comments this branch added Comments and docstrings only, no code. Verified with comment_tools.py check and by rerunning the suites: 238 passed, both sh suites under sh, dash and bash, and all 24 pwsh studio tests. * Stay out of the way when uv is told not to cache Confirmed against uv 0.10.7: UV_NO_CACHE=1 makes `uv cache dir` report a temporary directory that is discarded when the process exits. The probe inherited that, read the throwaway as cold, and the update then seeded a UV_CACHE_DIR that --no-cache outranks anyway and recorded it as the install's cache. On an install predating the marker that redirected every later update to a cache nothing had ever filled. Neither seeds nor records now, on uv's own boolish spelling of the variable: 0 and false leave caching on, and a value uv cannot parse is one it refuses to run on at all, which is not the same as running without a cache. * Tighten the comments the last change added Comments and docstrings only, no code. Verified with comment_tools.py check and by rerunning the suites: 246 passed, both sh suites under sh, dash and bash, and all 24 pwsh studio tests. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
parent
fb9bb4ea00
commit
b66d2a4c89
12 changed files with 2079 additions and 4 deletions
111
install.ps1
111
install.ps1
|
|
@ -16,6 +16,15 @@
|
|||
function Install-UnslothStudio {
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
# First: Exit-InstallFailure restores the marker long before the selection runs, and
|
||||
# under `irm | iex` the script scope is the caller's session.
|
||||
$script:StudioUvMarkerSaved = $false
|
||||
$script:StudioUvMarkerExisted = $false
|
||||
$script:StudioUvMarkerPrevious = $null
|
||||
# One flag for both rollbacks: clearing them separately leaves a window either way
|
||||
# round, where an interruption restores one half of a committed install.
|
||||
$script:StudioInstallCommitted = $false
|
||||
|
||||
# The user's PowerShell profile has already run by the time this does, and the documented
|
||||
# piped web entry point documented in the README has no script file to re-launch
|
||||
# with -NoProfile, so each way a profile can reach in here is cut individually below.
|
||||
|
|
@ -318,6 +327,10 @@ function Install-UnslothStudio {
|
|||
if (Get-Command Restore-StudioVenvRollback -CommandType Function -ErrorAction SilentlyContinue) {
|
||||
Restore-StudioVenvRollback
|
||||
}
|
||||
# Separate from the venv rollback: an install can fail before one is in flight.
|
||||
if (Get-Command Restore-StudioUvCacheMarker -CommandType Function -ErrorAction SilentlyContinue) {
|
||||
Restore-StudioUvCacheMarker -StudioRoot $StudioHome
|
||||
}
|
||||
# Most failures return before the lock try/finally, and under `irm | iex`
|
||||
# these variables are the caller's own. Defined later, so probed like above.
|
||||
if (Get-Command Restore-StudioTempEnvironment -CommandType Function -ErrorAction SilentlyContinue) {
|
||||
|
|
@ -1221,6 +1234,89 @@ public static class UnslothStudioFinalPathV2
|
|||
}
|
||||
$VenvDir = Join-Path $StudioHome "unsloth_studio"
|
||||
|
||||
# Records which cache this install used, so an update reuses it rather than guessing:
|
||||
# Set-StudioUvCacheForLaunch repoints the backend at the Studio cache even in shared
|
||||
# mode, so one on-demand install makes an empty Studio cache look full. Never fatal.
|
||||
# uv resolves a relative cache-dir against its working directory, which UV_WORKING_DIR
|
||||
# moves and which may itself be relative to where the installer was run. $PWD.Path
|
||||
# explicitly: GetFullPath resolves against the .NET process directory, which
|
||||
# Set-Location does not move. Mirrors _absolutize_uv_cache_dir in install.sh.
|
||||
function Resolve-StudioUvCachePath {
|
||||
param([Parameter(Mandatory = $true)][AllowEmptyString()][string]$Cache)
|
||||
if ([string]::IsNullOrEmpty($Cache) -or [System.IO.Path]::IsPathRooted($Cache)) {
|
||||
return $Cache
|
||||
}
|
||||
try {
|
||||
$base = if (-not [string]::IsNullOrWhiteSpace($env:UV_WORKING_DIR)) {
|
||||
if ([System.IO.Path]::IsPathRooted($env:UV_WORKING_DIR)) {
|
||||
$env:UV_WORKING_DIR
|
||||
} else { Join-Path $PWD.Path $env:UV_WORKING_DIR }
|
||||
} else { $PWD.Path }
|
||||
return [System.IO.Path]::GetFullPath((Join-Path $base $Cache))
|
||||
} catch { return $Cache }
|
||||
}
|
||||
|
||||
function Write-StudioUvCacheMarker {
|
||||
param(
|
||||
[Parameter(Mandatory = $true)][string]$StudioRoot,
|
||||
[Parameter(Mandatory = $true)][string]$Cache
|
||||
)
|
||||
$markerDir = Join-Path $StudioRoot "cache"
|
||||
$markerFile = Join-Path $markerDir "uv-cache-dir"
|
||||
# Absolute: the update resolves this against ITS working directory.
|
||||
$Cache = Resolve-StudioUvCachePath -Cache $Cache
|
||||
# Remembered so a rollback can put it back.
|
||||
if (-not $script:StudioUvMarkerSaved) {
|
||||
# Under "Stop", Test-Path inside an ACL-denied directory throws.
|
||||
$existing = Test-Path -LiteralPath $markerFile -ErrorAction SilentlyContinue
|
||||
if ($existing) {
|
||||
# UTF8, not the default: Windows PowerShell 5.1 decodes a BOM-less file
|
||||
# with the active ANSI code page, and the update writes this one BOM-less
|
||||
# UTF-8, so a non-ASCII path came back as mojibake and was restored that way.
|
||||
$previous = Get-Content -LiteralPath $markerFile -Raw -Encoding UTF8 `
|
||||
-ErrorAction SilentlyContinue
|
||||
# One we cannot read is one we cannot put back, so leave it alone.
|
||||
if ($null -eq $previous) { return }
|
||||
$script:StudioUvMarkerPrevious = $previous
|
||||
$script:StudioUvMarkerExisted = $true
|
||||
} else {
|
||||
$script:StudioUvMarkerPrevious = $null
|
||||
$script:StudioUvMarkerExisted = $false
|
||||
}
|
||||
$script:StudioUvMarkerSaved = $true
|
||||
}
|
||||
# CreateDirectory, not New-Item -Path: -Path treats [] in the root as a wildcard.
|
||||
if (-not (Test-Path -LiteralPath $markerDir -PathType Container -ErrorAction SilentlyContinue)) {
|
||||
try { [System.IO.Directory]::CreateDirectory($markerDir) | Out-Null } catch { }
|
||||
}
|
||||
# Removed first: Set-Content follows a symlink and would truncate its target.
|
||||
Remove-Item -LiteralPath $markerFile -Force -ErrorAction SilentlyContinue
|
||||
# And only once gone, since that removal fails non-terminatingly. Get-Item -Force
|
||||
# reports the link itself; Test-Path would follow it.
|
||||
if ($null -ne (Get-Item -LiteralPath $markerFile -Force -ErrorAction SilentlyContinue)) {
|
||||
return
|
||||
}
|
||||
Set-Content -LiteralPath $markerFile -Value $Cache `
|
||||
-Encoding utf8 -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
function Restore-StudioUvCacheMarker {
|
||||
# AllowEmptyString and the blank guard: a throw here would be reported as a
|
||||
# failed environment restore.
|
||||
param([Parameter(Mandatory = $true)][AllowEmptyString()][string]$StudioRoot)
|
||||
if ($script:StudioInstallCommitted) { return }
|
||||
if (-not $script:StudioUvMarkerSaved) { return }
|
||||
if ([string]::IsNullOrWhiteSpace($StudioRoot)) { return }
|
||||
$markerFile = Join-Path (Join-Path $StudioRoot "cache") "uv-cache-dir"
|
||||
Remove-Item -LiteralPath $markerFile -Force -ErrorAction SilentlyContinue
|
||||
$stillThere = $null -ne (Get-Item -LiteralPath $markerFile -Force -ErrorAction SilentlyContinue)
|
||||
if (-not $stillThere -and $script:StudioUvMarkerExisted -and $null -ne $script:StudioUvMarkerPrevious) {
|
||||
Set-Content -LiteralPath $markerFile -Value ([string]$script:StudioUvMarkerPrevious).TrimEnd("`r", "`n") `
|
||||
-Encoding utf8 -ErrorAction SilentlyContinue
|
||||
}
|
||||
$script:StudioUvMarkerSaved = $false
|
||||
}
|
||||
|
||||
function Set-StudioUvCacheEnvironment {
|
||||
param(
|
||||
[Parameter(Mandatory = $true)][string]$StudioRoot,
|
||||
|
|
@ -1230,6 +1326,11 @@ public static class UnslothStudioFinalPathV2
|
|||
$studioCache = Join-Path (Join-Path $StudioRoot "cache") "uv"
|
||||
if (-not [string]::IsNullOrWhiteSpace($env:UV_CACHE_DIR)) {
|
||||
$script:StudioUvCacheMode = "custom"
|
||||
# Absolute before anything uses it, so every phase of one install and the
|
||||
# marker name the same directory (see _absolutize_uv_cache_dir in install.sh).
|
||||
$env:UV_CACHE_DIR = Resolve-StudioUvCachePath -Cache $env:UV_CACHE_DIR
|
||||
# Recorded like any other choice; a caller still outranks the marker.
|
||||
Write-StudioUvCacheMarker -StudioRoot $StudioRoot -Cache $env:UV_CACHE_DIR
|
||||
step "uv cache" "preserving custom UV_CACHE_DIR ($env:UV_CACHE_DIR)"
|
||||
return
|
||||
}
|
||||
|
|
@ -1299,6 +1400,7 @@ public static class UnslothStudioFinalPathV2
|
|||
}
|
||||
}
|
||||
Set-Item -LiteralPath Env:UV_CACHE_DIR -Value $selectedCache
|
||||
Write-StudioUvCacheMarker -StudioRoot $StudioRoot -Cache $selectedCache
|
||||
|
||||
switch ($script:StudioUvCacheMode) {
|
||||
"shared" {
|
||||
|
|
@ -4213,6 +4315,9 @@ exit 0
|
|||
}
|
||||
|
||||
function Restore-StudioVenvRollback {
|
||||
# The same flag the marker restore consults, so an interruption mid-commit cannot
|
||||
# put one half of a committed install back and keep the other.
|
||||
if ($script:StudioInstallCommitted) { return }
|
||||
if (-not $script:StudioVenvRollbackActive) { return }
|
||||
$backup = $script:StudioVenvRollbackDir
|
||||
$target = $script:StudioVenvRollbackTarget
|
||||
|
|
@ -4264,6 +4369,11 @@ exit 0
|
|||
}
|
||||
|
||||
function Complete-StudioVenvRollback {
|
||||
# First and alone: this one assignment is what both restores consult, so an
|
||||
# interruption cannot find them disagreeing. A first install rolls nothing back
|
||||
# and still commits.
|
||||
$script:StudioInstallCommitted = $true
|
||||
$script:StudioUvMarkerSaved = $false
|
||||
if (-not $script:StudioVenvRollbackActive) { return }
|
||||
$backup = $script:StudioVenvRollbackDir
|
||||
# The replacement is committed. Disable restoration before deleting the
|
||||
|
|
@ -6573,6 +6683,7 @@ sys.exit(2 if conflict else (0 if installed else 1))
|
|||
} finally {
|
||||
if (-not $studioVenvReplacementCommitted) {
|
||||
Restore-StudioVenvRollback
|
||||
Restore-StudioUvCacheMarker -StudioRoot $StudioHome
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
81
install.sh
81
install.sh
|
|
@ -619,12 +619,75 @@ _resolve_studio_destinations() {
|
|||
_STUDIO_HOME_REDIRECT=default
|
||||
}
|
||||
|
||||
# Records which cache this install used, so an update reuses it rather than guessing: the
|
||||
# launch below repoints the backend at the Studio cache even in shared mode, so one
|
||||
# on-demand install makes an empty Studio cache look full. Never fatal.
|
||||
# A relative UV_CACHE_DIR names a different directory in each phase of one install: uv
|
||||
# resolves it against its working directory, and setup.sh changes into its own before the
|
||||
# dependency pass (studio/setup.sh:1788). Absolute once, here, so both phases and the
|
||||
# marker agree. The base is uv's working directory, which --directory / UV_WORKING_DIR
|
||||
# moves, and which may itself be relative to where the installer was run.
|
||||
_absolutize_uv_cache_dir() {
|
||||
case "$UV_CACHE_DIR" in
|
||||
/*) return 0 ;;
|
||||
esac
|
||||
_uv_cache_base="${UV_WORKING_DIR:-$PWD}"
|
||||
case "$_uv_cache_base" in
|
||||
/*) ;;
|
||||
*) _uv_cache_base="$PWD/$_uv_cache_base" ;;
|
||||
esac
|
||||
UV_CACHE_DIR="$_uv_cache_base/$UV_CACHE_DIR"
|
||||
}
|
||||
|
||||
_record_uv_cache_choice() {
|
||||
# In place, before anything reads it: every branch records, so this is the one point
|
||||
# every phase of the install and the marker are made to agree on one directory.
|
||||
_absolutize_uv_cache_dir
|
||||
_uv_marker_dir="$STUDIO_HOME/cache"
|
||||
_uv_marker_file="$_uv_marker_dir/uv-cache-dir"
|
||||
_uv_marker_value="$UV_CACHE_DIR"
|
||||
# Remembered so a failed install can put it back.
|
||||
if [ "$_UV_MARKER_SAVED" != true ]; then
|
||||
if [ -e "$_uv_marker_file" ] || [ -L "$_uv_marker_file" ]; then
|
||||
# One we cannot read is one we cannot put back, so leave it alone.
|
||||
_UV_MARKER_PREVIOUS=$(cat "$_uv_marker_file" 2>/dev/null) || return 0
|
||||
_UV_MARKER_EXISTED=true
|
||||
else
|
||||
_UV_MARKER_PREVIOUS=""
|
||||
_UV_MARKER_EXISTED=false
|
||||
fi
|
||||
_UV_MARKER_SAVED=true
|
||||
fi
|
||||
(
|
||||
mkdir -p "$_uv_marker_dir" 2>/dev/null &&
|
||||
# Unlinked first: a redirection follows a symlink and truncates its target.
|
||||
rm -f "$_uv_marker_file" 2>/dev/null &&
|
||||
# And only once gone: rm can fail on a link in an undeletable directory.
|
||||
! { [ -e "$_uv_marker_file" ] || [ -L "$_uv_marker_file" ]; } &&
|
||||
printf '%s\n' "$_uv_marker_value" > "$_uv_marker_file" 2>/dev/null
|
||||
) || true
|
||||
}
|
||||
|
||||
_restore_uv_cache_marker() {
|
||||
[ "${_STUDIO_INSTALL_COMMITTED:-false}" = true ] && return 0
|
||||
[ "$_UV_MARKER_SAVED" = true ] || return 0
|
||||
_uv_marker_file="$STUDIO_HOME/cache/uv-cache-dir"
|
||||
rm -f "$_uv_marker_file" 2>/dev/null || true
|
||||
if [ "$_UV_MARKER_EXISTED" = true ] \
|
||||
&& ! { [ -e "$_uv_marker_file" ] || [ -L "$_uv_marker_file" ]; }; then
|
||||
printf '%s\n' "$_UV_MARKER_PREVIOUS" > "$_uv_marker_file" 2>/dev/null || true
|
||||
fi
|
||||
_UV_MARKER_SAVED=false
|
||||
}
|
||||
|
||||
_configure_uv_cache() {
|
||||
_uv_studio_cache="$STUDIO_HOME/cache/uv"
|
||||
case "${UV_CACHE_DIR-}" in
|
||||
*[![:space:]]*)
|
||||
_UV_CACHE_MODE=custom
|
||||
export UV_CACHE_DIR
|
||||
# Recorded like any other choice; a caller still outranks the marker.
|
||||
_record_uv_cache_choice
|
||||
step "uv cache" "preserving custom UV_CACHE_DIR ($UV_CACHE_DIR)"
|
||||
return 0
|
||||
;;
|
||||
|
|
@ -634,6 +697,7 @@ _configure_uv_cache() {
|
|||
UV_CACHE_DIR="$_uv_studio_cache"
|
||||
_UV_CACHE_MODE=isolated
|
||||
export UV_CACHE_DIR
|
||||
_record_uv_cache_choice
|
||||
step "uv cache" "forced Studio cache isolation ($UV_CACHE_DIR); already-cached packages may download again" "$C_WARN"
|
||||
return 0
|
||||
fi
|
||||
|
|
@ -687,6 +751,7 @@ _configure_uv_cache() {
|
|||
_UV_CACHE_MODE=studio
|
||||
fi
|
||||
export UV_CACHE_DIR
|
||||
_record_uv_cache_choice
|
||||
|
||||
case "$_UV_CACHE_MODE" in
|
||||
shared)
|
||||
|
|
@ -718,6 +783,13 @@ VENV_DIR="$STUDIO_HOME/unsloth_studio"
|
|||
_VENV_ROLLBACK_DIR=""
|
||||
_VENV_ROLLBACK_TARGET="$VENV_DIR"
|
||||
_VENV_ROLLBACK_ACTIVE=false
|
||||
# The marker travels with the environment. See _record_uv_cache_choice.
|
||||
_UV_MARKER_SAVED=false
|
||||
_UV_MARKER_EXISTED=false
|
||||
_UV_MARKER_PREVIOUS=""
|
||||
# One flag for both rollbacks: two leave a window either way round, where a signal
|
||||
# restores half a committed install.
|
||||
_STUDIO_INSTALL_COMMITTED=false
|
||||
|
||||
_start_studio_venv_replacement() {
|
||||
_existing_dir="$1"
|
||||
|
|
@ -789,6 +861,8 @@ _discard_venv_for_recreate() { # venv dir
|
|||
}
|
||||
|
||||
_restore_studio_venv_replacement() {
|
||||
# The flag the marker restore consults too, so a signal mid-commit cannot split them.
|
||||
[ "${_STUDIO_INSTALL_COMMITTED:-false}" = true ] && return 0
|
||||
[ "$_VENV_ROLLBACK_ACTIVE" = true ] || return 0
|
||||
# -e/-L, not -d: a rollback holds whatever _dir_has_entries called occupied,
|
||||
# and -d would drop a file or a dangling link and strand the original.
|
||||
|
|
@ -850,6 +924,10 @@ _prune_stale_studio_venv_rollbacks() {
|
|||
}
|
||||
|
||||
_commit_studio_venv_replacement() {
|
||||
# First and alone, because a signal can land between any two statements. A first
|
||||
# install rolls nothing back and still commits.
|
||||
_STUDIO_INSTALL_COMMITTED=true
|
||||
_UV_MARKER_SAVED=false
|
||||
if [ "$_VENV_ROLLBACK_ACTIVE" = true ]; then
|
||||
_rollback_to_remove="$_VENV_ROLLBACK_DIR"
|
||||
# The new environment is already committed. Clear the restore state
|
||||
|
|
@ -886,6 +964,8 @@ _on_install_exit() {
|
|||
_status=$?
|
||||
if [ "$_status" -ne 0 ]; then
|
||||
_restore_studio_venv_replacement
|
||||
# Separate from the venv restore: an install can fail before one is in flight.
|
||||
_restore_uv_cache_marker
|
||||
fi
|
||||
_cleanup_install_temporaries
|
||||
exit "$_status"
|
||||
|
|
@ -898,6 +978,7 @@ _on_install_signal() {
|
|||
trap - EXIT
|
||||
trap '' HUP INT TERM
|
||||
_restore_studio_venv_replacement
|
||||
_restore_uv_cache_marker
|
||||
_cleanup_install_temporaries
|
||||
exit "$_signal_status"
|
||||
}
|
||||
|
|
|
|||
|
|
@ -36,6 +36,16 @@ def _fallback_range(lines):
|
|||
raise AssertionError("install.sh's GPU-detection fallback branch is never closed")
|
||||
|
||||
|
||||
def _all_indexes(haystack, needle):
|
||||
"""Every occurrence, so an assertion about "each write" cannot pass on the first one."""
|
||||
found, at = [], haystack.find(needle)
|
||||
while at != -1:
|
||||
found.append(at)
|
||||
at = haystack.find(needle, at + 1)
|
||||
assert found, needle
|
||||
return found
|
||||
|
||||
|
||||
class TestNoTorchBackendAutoInInstallSh:
|
||||
"""install.sh primary paths must not use --torch-backend=auto (only the fallback else-branch may)."""
|
||||
|
||||
|
|
@ -1129,6 +1139,155 @@ class TestInstallUvCacheRootParity:
|
|||
for metadata_suffix in (".msgpack", ".http", ".rev", ".lock"):
|
||||
assert metadata_suffix in source, metadata_suffix
|
||||
|
||||
def test_both_installers_record_the_cache_they_chose(self):
|
||||
"""`unsloth studio update` reads this to reuse the install's cache. Content
|
||||
cannot decide it: install.sh:705 points the backend at the Studio cache even in
|
||||
shared mode, so a runtime install leaves bytes in the losing one."""
|
||||
sh = INSTALL_SH.read_text(encoding = "utf-8")
|
||||
ps1 = INSTALL_PS1.read_text(encoding = "utf-8")
|
||||
cli = (REPO_ROOT / "unsloth_cli" / "commands" / "studio.py").read_text(encoding = "utf-8")
|
||||
|
||||
for source in (sh, ps1, cli):
|
||||
assert "uv-cache-dir" in source
|
||||
|
||||
# Written after the choice is made.
|
||||
assert "_record_uv_cache_choice() {" in sh
|
||||
# Every mode records, custom included, or a previous install's marker survives.
|
||||
call_sites = [
|
||||
match.start()
|
||||
for match in re.finditer(r"^[ \t]+_record_uv_cache_choice[ \t]*$", sh, re.MULTILINE)
|
||||
]
|
||||
assert len(call_sites) == 3, f"one call site per mode branch, found {len(call_sites)}"
|
||||
assert (
|
||||
sh.index("_UV_CACHE_MODE=custom")
|
||||
< min(call_sites)
|
||||
< sh.index("_UV_CACHE_MODE=isolated")
|
||||
), "the custom branch records before it returns"
|
||||
assert ps1.count("Write-StudioUvCacheMarker -StudioRoot") == 2
|
||||
# A marker is a preference, not a requirement, so neither installer may fail on it.
|
||||
marker_write = ps1[
|
||||
ps1.index("function Write-StudioUvCacheMarker") : ps1.index(
|
||||
"function Set-StudioUvCacheEnvironment"
|
||||
)
|
||||
]
|
||||
# Covers both marker functions, which sit together above the selector.
|
||||
assert "-ErrorAction Stop" not in marker_write
|
||||
assert marker_write.count("-ErrorAction SilentlyContinue") >= 2
|
||||
assert "|| true" in sh[sh.index("_record_uv_cache_choice() {") :]
|
||||
# Windows PowerShell 5.1 writes -Encoding utf8 WITH a BOM, so the reader allows one.
|
||||
assert "utf-8-sig" in cli
|
||||
|
||||
# Absolute on both sides: the update resolves it against its own directory.
|
||||
assert 'case "$UV_CACHE_DIR" in' in sh
|
||||
assert "IsPathRooted" in ps1
|
||||
|
||||
# The reset must precede both consumers, the selector that writes the marker and
|
||||
# every Exit-InstallFailure that restores it. Under `irm | iex` the script scope is
|
||||
# the caller's session, so only the entry point is ahead of both.
|
||||
entry = ps1.index("function Install-UnslothStudio {")
|
||||
# The call form, not the bare name, which also appears in prose above.
|
||||
first_consumer = min(
|
||||
ps1.index("Set-StudioUvCacheEnvironment -StudioRoot $StudioHome"),
|
||||
ps1.index('(Exit-InstallFailure "', entry),
|
||||
)
|
||||
for variable in (
|
||||
"$script:StudioUvMarkerSaved = $false",
|
||||
"$script:StudioUvMarkerExisted = $false",
|
||||
"$script:StudioUvMarkerPrevious = $null",
|
||||
"$script:StudioInstallCommitted = $false",
|
||||
):
|
||||
assert variable in ps1[entry:first_consumer], variable
|
||||
|
||||
# Committing the environment commits the marker that came with it.
|
||||
assert (
|
||||
"$script:StudioUvMarkerSaved = $false"
|
||||
in ps1[
|
||||
ps1.index("function Complete-StudioVenvRollback") : ps1.index(
|
||||
"function Complete-StudioVenvRollback"
|
||||
)
|
||||
+ 900
|
||||
]
|
||||
)
|
||||
commit_start = sh.index("_commit_studio_venv_replacement() {")
|
||||
commit_body = sh[commit_start : sh.index("\n}", commit_start)]
|
||||
# Before the venv flag: a signal between the two keeps the environment and
|
||||
# reverts its marker.
|
||||
assert commit_body.index("_UV_MARKER_SAVED=false") < commit_body.index(
|
||||
"_VENV_ROLLBACK_ACTIVE=false"
|
||||
)
|
||||
ps1_commit = ps1[ps1.index("function Complete-StudioVenvRollback") :][:900]
|
||||
assert ps1_commit.index("$script:StudioUvMarkerSaved = $false") < ps1_commit.index(
|
||||
"$script:StudioVenvRollbackActive = $false"
|
||||
)
|
||||
# And outside the rollback branch, which a first install skips entirely.
|
||||
assert commit_body.index("_UV_MARKER_SAVED=false") < commit_body.index(
|
||||
'if [ "$_VENV_ROLLBACK_ACTIVE" = true ]'
|
||||
)
|
||||
assert ps1_commit.index("$script:StudioUvMarkerSaved = $false") < ps1_commit.index(
|
||||
"if (-not $script:StudioVenvRollbackActive) { return }"
|
||||
)
|
||||
|
||||
# Both writes are gated on the old entry being gone: the unlink is what keeps a
|
||||
# symlinked marker from truncating its target, and it can fail silently.
|
||||
for body in (
|
||||
sh[sh.index("_record_uv_cache_choice() {") : sh.index("_restore_uv_cache_marker() {")],
|
||||
sh[sh.index("_restore_uv_cache_marker() {") :][:600],
|
||||
):
|
||||
unlink = body.index('rm -f "$_uv_marker_file"')
|
||||
write = body.index("printf '%s\\n'", unlink)
|
||||
assert '[ -L "$_uv_marker_file" ]' in body[unlink:write], body[unlink:write]
|
||||
ps1_marker = ps1[
|
||||
ps1.index("function Write-StudioUvCacheMarker") : ps1.index(
|
||||
"function Set-StudioUvCacheEnvironment"
|
||||
)
|
||||
]
|
||||
# One helper for both, or they disagree the moment UV_WORKING_DIR is set.
|
||||
assert ps1.count("Resolve-StudioUvCachePath -Cache") == 2, ps1.count(
|
||||
"Resolve-StudioUvCachePath -Cache"
|
||||
)
|
||||
assert "$env:UV_CACHE_DIR = Resolve-StudioUvCachePath -Cache $env:UV_CACHE_DIR" in ps1
|
||||
# And both sides consult UV_WORKING_DIR, itself resolvable against the run dir.
|
||||
assert "UV_WORKING_DIR" in sh[sh.index("_absolutize_uv_cache_dir() {") :][:600]
|
||||
assert "UV_WORKING_DIR" in ps1[ps1.index("function Resolve-StudioUvCachePath") :][:800]
|
||||
for start in _all_indexes(ps1_marker, "Remove-Item -LiteralPath $markerFile"):
|
||||
# The call form: the comment above the gate names the cmdlet too.
|
||||
window = ps1_marker[
|
||||
start : ps1_marker.index("Set-Content -LiteralPath $markerFile", start)
|
||||
]
|
||||
assert "Get-Item -LiteralPath $markerFile -Force" in window, window
|
||||
|
||||
# UTF-8, not the ANSI code page: the update writes this file BOM-less UTF-8 and
|
||||
# 5.1 would restore mojibake. Asserted on source, since pwsh 7 passes either way.
|
||||
read_back = ps1_marker.index("Get-Content -LiteralPath $markerFile")
|
||||
assert "-Encoding UTF8" in ps1_marker[read_back : read_back + 220], ps1_marker[read_back:]
|
||||
|
||||
# One flag decides both rollbacks. Clearing them separately leaves a window either
|
||||
# way round, where a signal restores one half of a committed install.
|
||||
assert commit_body.index("_STUDIO_INSTALL_COMMITTED=true") < commit_body.index(
|
||||
"_UV_MARKER_SAVED=false"
|
||||
)
|
||||
for body in (
|
||||
sh[sh.index("_restore_studio_venv_replacement() {") :][:400],
|
||||
sh[sh.index("_restore_uv_cache_marker() {") :][:400],
|
||||
):
|
||||
assert "_STUDIO_INSTALL_COMMITTED" in body, body
|
||||
assert ps1_commit.index("$script:StudioInstallCommitted = $true") < ps1_commit.index(
|
||||
"$script:StudioUvMarkerSaved = $false"
|
||||
)
|
||||
for name in ("Restore-StudioVenvRollback", "Restore-StudioUvCacheMarker"):
|
||||
body = ps1[ps1.index(f"function {name} {{") :][:800]
|
||||
assert "if ($script:StudioInstallCommitted) { return }" in body, name
|
||||
|
||||
# Restored whether or not a venv replacement was ever in flight.
|
||||
assert "_restore_uv_cache_marker" in sh[sh.index("_on_install_exit() {") :]
|
||||
assert "_restore_uv_cache_marker" in sh[sh.index("_on_install_signal() {") :]
|
||||
assert ps1.count("Restore-StudioUvCacheMarker -StudioRoot") == 2
|
||||
|
||||
# And the marker travels with the environment: a rolled-back install puts it back.
|
||||
assert "_restore_uv_cache_marker" in sh
|
||||
assert sh.count("_restore_uv_cache_marker") >= 2, "defined but never called"
|
||||
assert "function Restore-StudioUvCacheMarker" in ps1
|
||||
|
||||
assert "${XDG_CACHE_HOME}/uv" in sh
|
||||
assert "${HOME}/.cache/uv" in sh
|
||||
assert 'Join-Path (Join-Path $env:LOCALAPPDATA "uv") "cache"' in ps1
|
||||
|
|
|
|||
|
|
@ -67,6 +67,8 @@ def _uv_cache_functions(source: str) -> str:
|
|||
return "".join(
|
||||
_extract(rf" function {name} \{{.*?\n \}}\n", source)
|
||||
for name in (
|
||||
"Resolve-StudioUvCachePath",
|
||||
"Write-StudioUvCacheMarker",
|
||||
"Set-StudioUvCacheEnvironment",
|
||||
"Set-StudioUvCacheForLaunch",
|
||||
"Restore-StudioUvCacheEnvironment",
|
||||
|
|
@ -238,6 +240,7 @@ def test_restoring_a_split_move_never_deletes_the_half_left_behind(tmp_path: Pat
|
|||
"Remove-StudioVenvTreeWithRetry",
|
||||
"Merge-StudioVenvRollbackTree",
|
||||
"Restore-StudioVenvRollback",
|
||||
"Restore-StudioUvCacheMarker",
|
||||
)
|
||||
)
|
||||
target = tmp_path / "unsloth_studio"
|
||||
|
|
@ -293,6 +296,7 @@ def test_merging_a_split_move_keeps_every_sibling_at_its_own_path(tmp_path: Path
|
|||
"Remove-StudioVenvTreeWithRetry",
|
||||
"Merge-StudioVenvRollbackTree",
|
||||
"Restore-StudioVenvRollback",
|
||||
"Restore-StudioUvCacheMarker",
|
||||
)
|
||||
)
|
||||
target = tmp_path / "unsloth_studio"
|
||||
|
|
@ -360,6 +364,7 @@ def test_merging_a_split_move_never_walks_through_a_link(tmp_path: Path, shell:
|
|||
"Remove-StudioVenvTreeWithRetry",
|
||||
"Merge-StudioVenvRollbackTree",
|
||||
"Restore-StudioVenvRollback",
|
||||
"Restore-StudioUvCacheMarker",
|
||||
)
|
||||
)
|
||||
target = tmp_path / "unsloth_studio"
|
||||
|
|
@ -685,6 +690,15 @@ try {{
|
|||
) == "keep"
|
||||
assert (shared / ".gitignore").read_text(encoding = "utf-8") == "*\n"
|
||||
|
||||
# The marker `unsloth studio update` reads back, so it reuses the cache this install
|
||||
# used instead of re-deriving it from content that a runtime install can change.
|
||||
# utf-8-sig because Windows PowerShell 5.1 writes -Encoding utf8 with a BOM.
|
||||
# Every mode records, custom included: a marker left by a previous install would aim
|
||||
# later updates at a cache this one never filled.
|
||||
marker = studio_root / "cache" / "uv-cache-dir"
|
||||
recorded = marker.read_text(encoding = "utf-8-sig").strip()
|
||||
assert norm(recorded) == norm(str(expected_selected)), case
|
||||
|
||||
|
||||
@pytest.mark.skipif(not POWERSHELLS, reason = "PowerShell is unavailable")
|
||||
@pytest.mark.parametrize("shell", POWERSHELLS)
|
||||
|
|
@ -736,7 +750,13 @@ try {{
|
|||
env["LOCALAPPDATA"] = str(tmp_path / "local app data")
|
||||
result = json.loads(_run_powershell(shell, script, env).splitlines()[-1])
|
||||
|
||||
expected = initial_value if initial_value.strip() else str(tmp_path / "studio" / "cache" / "uv")
|
||||
# A caller's own value is kept, and resolved: setup runs uv from a directory of its
|
||||
# own choosing, so a relative one would name two caches across the one install. What
|
||||
# gets RESTORED afterwards is still the caller's spelling, asserted below.
|
||||
if initial_value.strip():
|
||||
expected = os.path.abspath(initial_value)
|
||||
else:
|
||||
expected = str(tmp_path / "studio" / "cache" / "uv")
|
||||
assert os.path.normcase(os.path.normpath(result["Active"])) == os.path.normcase(
|
||||
os.path.normpath(expected)
|
||||
)
|
||||
|
|
@ -793,3 +813,203 @@ foreach ($root in @($env:TEST_STUDIO_HOME_ONE, $env:TEST_STUDIO_HOME_TWO)) {{
|
|||
assert result["Modes"] == ["studio", "studio"]
|
||||
assert result["PresentAfter"] is False
|
||||
assert result["ProviderPresentAfter"] is False
|
||||
|
||||
|
||||
@pytest.mark.skipif(not POWERSHELLS, reason = "PowerShell is unavailable")
|
||||
@pytest.mark.parametrize("shell", POWERSHELLS)
|
||||
def test_a_non_ascii_marker_survives_the_rollback(tmp_path: Path, shell: str):
|
||||
"""The update writes this file BOM-less UTF-8, and Windows PowerShell 5.1 reads a
|
||||
BOM-less file with the active ANSI code page, so a non-ASCII path came back as
|
||||
mojibake and the rollback wrote that back over a marker that had been correct."""
|
||||
source = INSTALL_PS1.read_text(encoding = "utf-8")
|
||||
functions = "".join(
|
||||
_extract(rf" function {name} \{{.*?\n \}}\n", source)
|
||||
for name in (
|
||||
"Resolve-StudioUvCachePath",
|
||||
"Write-StudioUvCacheMarker",
|
||||
"Restore-StudioUvCacheMarker",
|
||||
)
|
||||
)
|
||||
studio_root = tmp_path / "studio root"
|
||||
marker = studio_root / "cache" / "uv-cache-dir"
|
||||
previous = tmp_path / "kaffee cache"
|
||||
# As the Python backfill writes it: UTF-8, no BOM.
|
||||
marker.parent.mkdir(parents = True)
|
||||
marker.write_bytes(f"{previous}\n".replace("kaffee", "k\u00e4ffee").encode("utf-8"))
|
||||
expected = str(previous).replace("kaffee", "k\u00e4ffee")
|
||||
|
||||
script = f"""
|
||||
$ErrorActionPreference = "Stop"
|
||||
{functions}
|
||||
$script:StudioUvMarkerSaved = $false
|
||||
$script:StudioUvMarkerExisted = $false
|
||||
$script:StudioUvMarkerPrevious = $null
|
||||
Write-StudioUvCacheMarker -StudioRoot $env:TEST_STUDIO_HOME -Cache $env:TEST_CHOSEN
|
||||
Restore-StudioUvCacheMarker -StudioRoot $env:TEST_STUDIO_HOME
|
||||
[pscustomobject]@{{
|
||||
After = (Get-Content -LiteralPath $env:TEST_MARKER -Raw -Encoding UTF8).Trim()
|
||||
}} | ConvertTo-Json -Compress
|
||||
"""
|
||||
env = os.environ.copy()
|
||||
env["TEST_STUDIO_HOME"] = str(studio_root)
|
||||
env["TEST_CHOSEN"] = str(tmp_path / "chosen cache")
|
||||
env["TEST_MARKER"] = str(marker)
|
||||
result = json.loads(_run_powershell(shell, script, env).splitlines()[-1])
|
||||
|
||||
assert result["After"] == expected, result
|
||||
|
||||
|
||||
@pytest.mark.skipif(not POWERSHELLS, reason = "PowerShell is unavailable")
|
||||
@pytest.mark.parametrize("shell", POWERSHELLS)
|
||||
@pytest.mark.parametrize("preexisting", [True, False])
|
||||
def test_a_rolled_back_install_restores_the_previous_uv_cache_marker(
|
||||
tmp_path: Path, shell: str, preexisting: bool
|
||||
):
|
||||
"""The marker describes the environment, so it goes back when the environment does:
|
||||
install.ps1 restores the previous venv on failure, and a marker for an install that
|
||||
never happened would outlive it."""
|
||||
source = INSTALL_PS1.read_text(encoding = "utf-8")
|
||||
functions = "".join(
|
||||
_extract(rf" function {name} \{{.*?\n \}}\n", source)
|
||||
for name in (
|
||||
"Resolve-StudioUvCachePath",
|
||||
"Write-StudioUvCacheMarker",
|
||||
"Restore-StudioUvCacheMarker",
|
||||
)
|
||||
)
|
||||
studio_root = tmp_path / "studio root"
|
||||
marker = studio_root / "cache" / "uv-cache-dir"
|
||||
previous = tmp_path / "previous cache"
|
||||
chosen = tmp_path / "chosen cache"
|
||||
if preexisting:
|
||||
marker.parent.mkdir(parents = True)
|
||||
marker.write_text(f"{previous}\n", encoding = "utf-8")
|
||||
|
||||
script = f"""
|
||||
$ErrorActionPreference = "Stop"
|
||||
{functions}
|
||||
$script:StudioUvMarkerSaved = $false
|
||||
$script:StudioUvMarkerExisted = $false
|
||||
$script:StudioUvMarkerPrevious = $null
|
||||
Write-StudioUvCacheMarker -StudioRoot $env:TEST_STUDIO_HOME -Cache $env:TEST_CHOSEN
|
||||
$during = Get-Content -LiteralPath $env:TEST_MARKER -Raw
|
||||
Restore-StudioUvCacheMarker -StudioRoot $env:TEST_STUDIO_HOME
|
||||
$after = if (Test-Path -LiteralPath $env:TEST_MARKER) {{
|
||||
(Get-Content -LiteralPath $env:TEST_MARKER -Raw).Trim()
|
||||
}} else {{ "<gone>" }}
|
||||
[pscustomobject]@{{ During = $during.Trim(); After = $after }} | ConvertTo-Json -Compress
|
||||
"""
|
||||
env = os.environ.copy()
|
||||
env["TEST_STUDIO_HOME"] = str(studio_root)
|
||||
env["TEST_CHOSEN"] = str(chosen)
|
||||
env["TEST_MARKER"] = str(marker)
|
||||
result = json.loads(_run_powershell(shell, script, env).splitlines()[-1])
|
||||
|
||||
norm = lambda value: os.path.normcase(os.path.normpath(value))
|
||||
assert norm(result["During"]) == norm(str(chosen))
|
||||
assert result["After"] == ("<gone>" if not preexisting else result["After"])
|
||||
if preexisting:
|
||||
assert norm(result["After"]) == norm(str(previous))
|
||||
|
||||
|
||||
@pytest.mark.skipif(not POWERSHELLS, reason = "PowerShell is unavailable")
|
||||
@pytest.mark.parametrize("shell", POWERSHELLS)
|
||||
def test_a_relative_cache_is_recorded_absolute(tmp_path: Path, shell: str):
|
||||
"""`uv cache dir` answers a relative cache-dir with the relative spelling, and the
|
||||
update resolves the marker against ITS working directory, not the installer's."""
|
||||
source = INSTALL_PS1.read_text(encoding = "utf-8")
|
||||
functions = "".join(
|
||||
_extract(rf" function {name} \{{.*?\n \}}\n", source)
|
||||
for name in ("Resolve-StudioUvCachePath", "Write-StudioUvCacheMarker")
|
||||
)
|
||||
studio_root = tmp_path / "studio root"
|
||||
marker = studio_root / "cache" / "uv-cache-dir"
|
||||
|
||||
script = f"""
|
||||
$ErrorActionPreference = "Stop"
|
||||
{functions}
|
||||
$script:StudioUvMarkerSaved = $false
|
||||
Set-Location -LiteralPath $env:TEST_CWD
|
||||
Write-StudioUvCacheMarker -StudioRoot $env:TEST_STUDIO_HOME -Cache "relcache"
|
||||
(Get-Content -LiteralPath $env:TEST_MARKER -Raw).Trim()
|
||||
"""
|
||||
env = os.environ.copy()
|
||||
env["TEST_STUDIO_HOME"] = str(studio_root)
|
||||
env["TEST_MARKER"] = str(marker)
|
||||
env["TEST_CWD"] = str(tmp_path)
|
||||
recorded = _run_powershell(shell, script, env).splitlines()[-1]
|
||||
|
||||
assert os.path.normcase(os.path.normpath(recorded)) == os.path.normcase(
|
||||
os.path.normpath(str(tmp_path / "relcache"))
|
||||
), recorded
|
||||
|
||||
|
||||
@pytest.mark.skipif(not POWERSHELLS, reason = "PowerShell is unavailable")
|
||||
@pytest.mark.parametrize("shell", POWERSHELLS)
|
||||
def test_a_relative_working_directory_moves_the_relative_cache(tmp_path: Path, shell: str):
|
||||
"""UV_WORKING_DIR is where uv starts, so a relative cache hangs off it and not off the
|
||||
installer's own directory. It may itself be relative, against the installer's."""
|
||||
source = INSTALL_PS1.read_text(encoding = "utf-8")
|
||||
functions = "".join(
|
||||
_extract(rf" function {name} \{{.*?\n \}}\n", source)
|
||||
for name in ("Resolve-StudioUvCachePath", "Write-StudioUvCacheMarker")
|
||||
)
|
||||
studio_root = tmp_path / "studio root"
|
||||
marker = studio_root / "cache" / "uv-cache-dir"
|
||||
|
||||
script = f"""
|
||||
$ErrorActionPreference = "Stop"
|
||||
{functions}
|
||||
$script:StudioUvMarkerSaved = $false
|
||||
Set-Location -LiteralPath $env:TEST_CWD
|
||||
$env:UV_WORKING_DIR = "subdir"
|
||||
Write-StudioUvCacheMarker -StudioRoot $env:TEST_STUDIO_HOME -Cache "relcache"
|
||||
(Get-Content -LiteralPath $env:TEST_MARKER -Raw).Trim()
|
||||
"""
|
||||
env = os.environ.copy()
|
||||
env.pop("UV_WORKING_DIR", None)
|
||||
env["TEST_STUDIO_HOME"] = str(studio_root)
|
||||
env["TEST_MARKER"] = str(marker)
|
||||
env["TEST_CWD"] = str(tmp_path)
|
||||
recorded = _run_powershell(shell, script, env).splitlines()[-1]
|
||||
|
||||
assert os.path.normcase(os.path.normpath(recorded)) == os.path.normcase(
|
||||
os.path.normpath(str(tmp_path / "subdir" / "relcache"))
|
||||
), recorded
|
||||
|
||||
|
||||
@pytest.mark.skipif(not POWERSHELLS, reason = "PowerShell is unavailable")
|
||||
@pytest.mark.parametrize("shell", POWERSHELLS)
|
||||
def test_an_unreadable_cache_directory_does_not_abort_the_install(tmp_path: Path, shell: str):
|
||||
"""The marker is optional, so probing for it must not fail the install: under
|
||||
$ErrorActionPreference = "Stop", Test-Path inside an ACL-denied directory throws
|
||||
UnauthorizedAccessException rather than returning $false."""
|
||||
if os.name == "nt" or os.geteuid() == 0:
|
||||
pytest.skip("POSIX mode bits do not deny this caller")
|
||||
source = INSTALL_PS1.read_text(encoding = "utf-8")
|
||||
functions = "".join(
|
||||
_extract(rf" function {name} \{{.*?\n \}}\n", source)
|
||||
for name in ("Resolve-StudioUvCachePath", "Write-StudioUvCacheMarker")
|
||||
)
|
||||
studio_root = tmp_path / "studio root"
|
||||
(studio_root / "cache").mkdir(parents = True)
|
||||
(studio_root / "cache").chmod(0o000)
|
||||
|
||||
script = f"""
|
||||
$ErrorActionPreference = "Stop"
|
||||
{functions}
|
||||
$script:StudioUvMarkerSaved = $false
|
||||
try {{
|
||||
Write-StudioUvCacheMarker -StudioRoot $env:TEST_STUDIO_HOME -Cache $env:TEST_CACHE
|
||||
"survived"
|
||||
}} catch {{
|
||||
"THREW: " + $_.Exception.GetType().Name
|
||||
}}
|
||||
"""
|
||||
env = os.environ.copy()
|
||||
env["TEST_STUDIO_HOME"] = str(studio_root)
|
||||
env["TEST_CACHE"] = str(tmp_path / "chosen")
|
||||
try:
|
||||
assert _run_powershell(shell, script, env).splitlines()[-1] == "survived"
|
||||
finally:
|
||||
(studio_root / "cache").chmod(0o755)
|
||||
|
|
|
|||
|
|
@ -14,6 +14,22 @@ ok() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
|||
bad() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
ROLLBACK_BLOCK=$(sed -n '/^_VENV_ROLLBACK_DIR=""/,/^trap '\''_on_install_signal 143'\'' TERM$/p' "$INSTALL_SH")
|
||||
# Both are defined above the block, with the rest of the cache selector, and both are
|
||||
# called from it. Splicing without them makes the cases exit 127 on a command the real
|
||||
# installer has, which satisfies any expectation about a marker that must not change.
|
||||
# printf, not $'\n': the workflow runs this file with `sh`, whatever the shebang says.
|
||||
MARKER_HELPER=$(awk '
|
||||
/^(_restore_uv_cache_marker|_record_uv_cache_choice|_absolutize_uv_cache_dir)\(\) \{/ { grab = 1 }
|
||||
grab { print }
|
||||
grab && /^}/ { grab = 0 }
|
||||
' "$INSTALL_SH")
|
||||
if ! printf '%s\n' "$MARKER_HELPER" | grep -q '^_restore_uv_cache_marker() {' \
|
||||
|| ! printf '%s\n' "$MARKER_HELPER" | grep -q '^_record_uv_cache_choice() {' \
|
||||
|| ! printf '%s\n' "$MARKER_HELPER" | grep -q '^_absolutize_uv_cache_dir() {'; then
|
||||
echo " FAIL: could not extract the uv cache marker helpers from install.sh"
|
||||
exit 1
|
||||
fi
|
||||
ROLLBACK_BLOCK=$(printf '%s\n%s\n' "$MARKER_HELPER" "$ROLLBACK_BLOCK")
|
||||
if ! printf '%s\n' "$ROLLBACK_BLOCK" | grep -q '^_on_install_signal() {'; then
|
||||
echo " FAIL: could not extract rollback lifecycle block from install.sh"
|
||||
exit 1
|
||||
|
|
@ -393,6 +409,94 @@ else
|
|||
bad "Windows rollback deletion still hides failures"
|
||||
fi
|
||||
|
||||
# The marker must not outlive the attempt that wrote it, even when no venv replacement
|
||||
# was ever in flight: a first install has none, and the ownership guard can refuse early.
|
||||
marker_case() { # label, pre-existing marker value or empty, expect, [commit]
|
||||
_label="$1"; _pre="$2"; _expect="$3"; _commit="${4:-}"
|
||||
_dir="$WORK/marker-$_label"
|
||||
mkdir -p "$_dir/cache"
|
||||
[ -n "$_pre" ] && printf '%s\n' "$_pre" > "$_dir/cache/uv-cache-dir"
|
||||
_h="$_dir/harness.sh"
|
||||
{
|
||||
printf '%s\n' 'set -e'
|
||||
printf '%s\n' 'substep() { :; }'
|
||||
printf '%s\n' 'rollback_substep() { substep "$@"; }'
|
||||
printf '%s\n' 'C_WARN=""'
|
||||
printf "STUDIO_HOME='%s'\n" "$_dir"
|
||||
printf "VENV_DIR='%s/unsloth_studio'\n" "$_dir"
|
||||
printf '%s\n' "$ROLLBACK_BLOCK"
|
||||
printf '%s\n' 'UV_CACHE_DIR="/tmp/this-attempt-cache"'
|
||||
printf '%s\n' '_record_uv_cache_choice'
|
||||
[ -n "$_commit" ] && printf '%s\n' '_commit_studio_venv_replacement'
|
||||
printf '%s\n' 'exit 1'
|
||||
} > "$_h"
|
||||
( cd "$_dir" && sh "$_h" >/dev/null 2>"$_dir/err" ) || _rc=$?
|
||||
if [ "${_rc:-0}" = 127 ] || [ -s "$_dir/err" ]; then
|
||||
bad "$_label (harness did not run: $(cat "$_dir/err"))"
|
||||
return 0
|
||||
fi
|
||||
if [ -f "$_dir/cache/uv-cache-dir" ]; then _got=$(cat "$_dir/cache/uv-cache-dir"); else _got="<gone>"; fi
|
||||
if [ "$_got" = "$_expect" ]; then
|
||||
ok "$_label"
|
||||
else
|
||||
bad "$_label (expected [$_expect], got [$_got])"
|
||||
fi
|
||||
}
|
||||
|
||||
# A signal can land between any two statements, so the commit sets one flag that both
|
||||
# restores consult. With two flags, a signal mid-commit put the previous environment back
|
||||
# and kept the marker of the attempt that replaced it, or the reverse.
|
||||
commit_flag_case() {
|
||||
_dir="$WORK/marker-commit-window"
|
||||
mkdir -p "$_dir/cache"
|
||||
printf '%s\n' "/previous/install/cache" > "$_dir/cache/uv-cache-dir"
|
||||
_h="$_dir/harness.sh"
|
||||
{
|
||||
printf '%s\n' 'set -e'
|
||||
printf '%s\n' 'substep() { :; }'
|
||||
printf '%s\n' 'rollback_substep() { substep "$@"; }'
|
||||
printf '%s\n' 'C_WARN=""'
|
||||
printf "STUDIO_HOME='%s'\n" "$_dir"
|
||||
printf "VENV_DIR='%s/unsloth_studio'\n" "$_dir"
|
||||
printf '%s\n' "$ROLLBACK_BLOCK"
|
||||
printf '%s\n' 'UV_CACHE_DIR="/tmp/this-attempt-cache"'
|
||||
printf '%s\n' '_record_uv_cache_choice'
|
||||
# The state a signal would find between the two flag assignments: committed, but
|
||||
# with the venv rollback still armed.
|
||||
printf '%s\n' 'mkdir -p "$VENV_DIR" "$STUDIO_HOME/backup"'
|
||||
printf '%s\n' 'printf "new\n" > "$VENV_DIR/generation"'
|
||||
printf '%s\n' 'printf "old\n" > "$STUDIO_HOME/backup/generation"'
|
||||
printf '%s\n' '_STUDIO_INSTALL_COMMITTED=true'
|
||||
printf '%s\n' '_VENV_ROLLBACK_ACTIVE=true'
|
||||
printf '%s\n' '_VENV_ROLLBACK_DIR="$STUDIO_HOME/backup"'
|
||||
printf '%s\n' '_VENV_ROLLBACK_TARGET="$VENV_DIR"'
|
||||
printf '%s\n' '_restore_studio_venv_replacement'
|
||||
printf '%s\n' '_restore_uv_cache_marker'
|
||||
} > "$_h"
|
||||
( cd "$_dir" && sh "$_h" >/dev/null 2>"$_dir/err" ) || _rc=$?
|
||||
if [ "${_rc:-0}" = 127 ] || [ -s "$_dir/err" ]; then
|
||||
bad "a committed install is not half rolled back (harness: $(cat "$_dir/err"))"
|
||||
return 0
|
||||
fi
|
||||
_got=$(cat "$_dir/cache/uv-cache-dir" 2>/dev/null)
|
||||
_gen=$(cat "$_dir/unsloth_studio/generation" 2>/dev/null)
|
||||
if [ "$_got" = "/tmp/this-attempt-cache" ] && [ "$_gen" = new ]; then
|
||||
ok "a committed install is not half rolled back by a signal mid-commit"
|
||||
else
|
||||
bad "a signal mid-commit undid half the commit (marker [$_got], venv [$_gen])"
|
||||
fi
|
||||
}
|
||||
|
||||
echo "=== uv cache marker survives only a successful install ==="
|
||||
marker_case "a failed install with no venv replacement restores the previous marker" \
|
||||
"/previous/install/cache" "/previous/install/cache"
|
||||
marker_case "a failed first install leaves no marker behind" "" "<gone>"
|
||||
# A first install takes no rollback branch and must still commit the marker: what follows
|
||||
# can fail, and the environment it installed stays.
|
||||
marker_case "a committed first install keeps its marker when a later step fails" \
|
||||
"/previous/install/cache" "/tmp/this-attempt-cache" commit
|
||||
commit_flag_case
|
||||
|
||||
echo ""
|
||||
echo " PASS: $PASS"
|
||||
echo " FAIL: $FAIL"
|
||||
|
|
|
|||
|
|
@ -16,10 +16,14 @@ bad() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
|||
HELPERS=$(awk '
|
||||
/^_configure_uv_cache\(\) \{/ { grab = 1 }
|
||||
/^_prepare_studio_uv_cache_for_launch\(\) \{/ { grab = 1 }
|
||||
/^_record_uv_cache_choice\(\) \{/ { grab = 1 }
|
||||
/^_absolutize_uv_cache_dir\(\) \{/ { grab = 1 }
|
||||
/^_restore_uv_cache_marker\(\) \{/ { grab = 1 }
|
||||
grab { print }
|
||||
grab && /^}/ { grab = 0 }
|
||||
' "$INSTALL_SH")
|
||||
for _helper in _configure_uv_cache _prepare_studio_uv_cache_for_launch; do
|
||||
for _helper in _configure_uv_cache _prepare_studio_uv_cache_for_launch _record_uv_cache_choice \
|
||||
_restore_uv_cache_marker _absolutize_uv_cache_dir; do
|
||||
if ! printf '%s\n' "$HELPERS" | grep -q "^${_helper}() {"; then
|
||||
echo " FAIL: could not extract $_helper from install.sh"
|
||||
exit 1
|
||||
|
|
@ -245,6 +249,215 @@ for shell in sh bash; do
|
|||
"$STUDIO_CACHE"
|
||||
chmod 755 "$DEEP_CACHE/archive-v0/aaa hidden" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# The marker `unsloth studio update` reads: content cannot tell a Studio cache the
|
||||
# installer filled from one a runtime install dropped a single wheel into.
|
||||
MARKER="$ROOT/cache/uv-cache-dir"
|
||||
check_marker() { # label, expected
|
||||
if [ "$(cat "$MARKER" 2>/dev/null)" = "$2" ]; then
|
||||
ok "$shell: $1"
|
||||
else
|
||||
bad "$shell: $1 (expected [$2], got [$(cat "$MARKER" 2>/dev/null)])"
|
||||
fi
|
||||
}
|
||||
|
||||
rm -f "$MARKER"
|
||||
EMPTY_CACHE="$CASE/unrecorded/uv"
|
||||
run_case "$shell" "studio mode still selects the Studio cache" unset "" false \
|
||||
"$HOME_DIR" unset "" "$ROOT" "$EMPTY_CACHE" "$STUDIO_CACHE" studio \
|
||||
"using new Studio-owned cache ($STUDIO_CACHE)" "$STUDIO_CACHE"
|
||||
check_marker "studio mode records the Studio cache" "$STUDIO_CACHE"
|
||||
|
||||
rm -f "$MARKER"
|
||||
run_case "$shell" "shared mode still selects the shared cache" unset "" false \
|
||||
"$HOME_DIR" unset "" "$ROOT" "$BUILDS_CACHE" "$BUILDS_CACHE" shared \
|
||||
"reusing existing shared cache ($BUILDS_CACHE) to avoid duplicate Torch/CUDA downloads; use --isolated-uv-cache to isolate" \
|
||||
"$STUDIO_CACHE"
|
||||
check_marker "shared mode records the shared cache, not the Studio one" "$BUILDS_CACHE"
|
||||
|
||||
rm -f "$MARKER"
|
||||
run_case "$shell" "isolation still forces the Studio cache" unset "" true \
|
||||
"$HOME_DIR" unset "" "$ROOT" "$BUILDS_CACHE" "$STUDIO_CACHE" isolated \
|
||||
"forced Studio cache isolation ($STUDIO_CACHE); already-cached packages may download again" \
|
||||
"$STUDIO_CACHE"
|
||||
check_marker "isolated mode records the Studio cache" "$STUDIO_CACHE"
|
||||
|
||||
# A custom cache holds this install's wheels too, so it is recorded; the caller still
|
||||
# wins on any run that sets the variable.
|
||||
printf '%s\n' "$STUDIO_CACHE" > "$MARKER"
|
||||
run_case "$shell" "custom mode is still preserved" value "$OVERRIDE" false \
|
||||
"$HOME_DIR" unset "" "$ROOT" "$BUILDS_CACHE" "$OVERRIDE" custom \
|
||||
"preserving custom UV_CACHE_DIR ($OVERRIDE)" "$OVERRIDE"
|
||||
check_marker "custom mode replaces an earlier install's marker" "$OVERRIDE"
|
||||
|
||||
# A symlinked marker is replaced, not followed: a redirection truncates its target.
|
||||
rm -rf "$ROOT/cache"; mkdir -p "$ROOT/cache"
|
||||
VICTIM="$CASE/someone elses file"
|
||||
printf 'do not clobber\n' > "$VICTIM"
|
||||
ln -s "$VICTIM" "$ROOT/cache/uv-cache-dir" 2>/dev/null && {
|
||||
run_case "$shell" "a symlinked marker is replaced, not followed" unset "" false \
|
||||
"$HOME_DIR" unset "" "$ROOT" "$EMPTY_CACHE" "$STUDIO_CACHE" studio \
|
||||
"using new Studio-owned cache ($STUDIO_CACHE)" "$STUDIO_CACHE"
|
||||
if [ "$(cat "$VICTIM")" = "do not clobber" ] && [ ! -L "$ROOT/cache/uv-cache-dir" ]; then
|
||||
ok "$shell: the symlink target was left alone"
|
||||
else
|
||||
bad "$shell: writing the marker followed a symlink"
|
||||
fi
|
||||
}
|
||||
|
||||
# One we cannot read is one we cannot put back, so leave it alone.
|
||||
rm -rf "$ROOT/cache"; mkdir -p "$ROOT/cache"
|
||||
printf '%s\n' "/previous/install/cache" > "$MARKER"
|
||||
if [ "$(id -u 2>/dev/null || echo 0)" != 0 ] && chmod 200 "$MARKER" 2>/dev/null; then
|
||||
run_case "$shell" "an unreadable marker does not fail the install" unset "" false \
|
||||
"$HOME_DIR" unset "" "$ROOT" "$EMPTY_CACHE" "$STUDIO_CACHE" studio \
|
||||
"using new Studio-owned cache ($STUDIO_CACHE)" "$STUDIO_CACHE"
|
||||
chmod 600 "$MARKER" 2>/dev/null || true
|
||||
if [ "$(cat "$MARKER")" = "/previous/install/cache" ]; then
|
||||
ok "$shell: an unreadable marker is left as it was"
|
||||
else
|
||||
bad "$shell: an unreadable marker was overwritten ([$(cat "$MARKER")])"
|
||||
fi
|
||||
fi
|
||||
|
||||
# uv resolves a relative cache against its own working directory, which --directory
|
||||
# and UV_WORKING_DIR move.
|
||||
rm -rf "$ROOT/cache"
|
||||
WORKDIR_PROBE="$WORK/$shell workdir.sh"
|
||||
{
|
||||
printf '%s\n' "$HELPERS"
|
||||
cat <<WORKDIR
|
||||
step() { :; }
|
||||
STUDIO_HOME='$ROOT'
|
||||
_UV_MARKER_SAVED=false
|
||||
cd '$CASE'
|
||||
UV_WORKING_DIR='$CASE/uvdir'; export UV_WORKING_DIR
|
||||
UV_CACHE_DIR='relcache'
|
||||
_record_uv_cache_choice
|
||||
cat '$MARKER'
|
||||
WORKDIR
|
||||
} > "$WORKDIR_PROBE"
|
||||
_wd=$($shell "$WORKDIR_PROBE")
|
||||
if [ "$_wd" = "$CASE/uvdir/relcache" ]; then
|
||||
ok "$shell: a relative cache is recorded against uv's working directory"
|
||||
else
|
||||
bad "$shell: recorded [$_wd], wanted [$CASE/uvdir/relcache]"
|
||||
fi
|
||||
|
||||
# Which may itself be relative, against the directory the installer ran from.
|
||||
rm -rf "$ROOT/cache"
|
||||
REL_PROBE="$WORK/$shell relworkdir.sh"
|
||||
{
|
||||
printf '%s\n' "$HELPERS"
|
||||
cat <<RELWD
|
||||
step() { :; }
|
||||
STUDIO_HOME='$ROOT'
|
||||
_UV_MARKER_SAVED=false
|
||||
cd '$CASE'
|
||||
UV_WORKING_DIR='uvdir'; export UV_WORKING_DIR
|
||||
UV_CACHE_DIR='relcache'
|
||||
_record_uv_cache_choice
|
||||
cat '$MARKER'
|
||||
RELWD
|
||||
} > "$REL_PROBE"
|
||||
_rw=$($shell "$REL_PROBE")
|
||||
if [ "$_rw" = "$CASE/uvdir/relcache" ]; then
|
||||
ok "$shell: a relative UV_WORKING_DIR is anchored before the cache is appended"
|
||||
else
|
||||
bad "$shell: recorded [$_rw], wanted [$CASE/uvdir/relcache]"
|
||||
fi
|
||||
|
||||
|
||||
# The marker describes the environment, so a rolled-back install puts it back.
|
||||
ROLLBACK_PROBE="$WORK/$shell rollback.sh"
|
||||
{
|
||||
printf '%s\n' "$HELPERS"
|
||||
cat <<ROLLBACK
|
||||
step() { :; }
|
||||
STUDIO_HOME='$ROOT'
|
||||
_UV_MARKER_SAVED=false
|
||||
_UV_MARKER_EXISTED=false
|
||||
_UV_MARKER_PREVIOUS=""
|
||||
UV_CACHE_DIR='$BUILDS_CACHE'
|
||||
_record_uv_cache_choice
|
||||
printf 'during=%s\n' "\$(cat '$MARKER')"
|
||||
_restore_uv_cache_marker
|
||||
if [ -f '$MARKER' ]; then printf 'after=%s\n' "\$(cat '$MARKER')"; else printf 'after=<gone>\n'; fi
|
||||
ROLLBACK
|
||||
} > "$ROLLBACK_PROBE"
|
||||
|
||||
printf '%s\n' "$STUDIO_CACHE" > "$MARKER"
|
||||
_rb=$($shell "$ROLLBACK_PROBE")
|
||||
if [ "$_rb" = "$(printf 'during=%s\nafter=%s' "$BUILDS_CACHE" "$STUDIO_CACHE")" ]; then
|
||||
ok "$shell: a rolled-back install restores the previous marker"
|
||||
else
|
||||
bad "$shell: rollback did not restore the marker (got [$_rb])"
|
||||
fi
|
||||
|
||||
rm -f "$MARKER"
|
||||
_rb=$($shell "$ROLLBACK_PROBE")
|
||||
if [ "$_rb" = "$(printf 'during=%s\nafter=<gone>' "$BUILDS_CACHE")" ]; then
|
||||
ok "$shell: a rolled-back first install leaves no marker behind"
|
||||
else
|
||||
bad "$shell: rollback left a marker on a first install (got [$_rb])"
|
||||
fi
|
||||
|
||||
# Absolute, because the update resolves the marker against its own directory.
|
||||
RELATIVE_PROBE="$WORK/$shell relative.sh"
|
||||
{
|
||||
printf '%s\n' "$HELPERS"
|
||||
cat <<RELATIVE
|
||||
step() { :; }
|
||||
STUDIO_HOME='$ROOT'
|
||||
_UV_MARKER_SAVED=false
|
||||
cd '$CASE'
|
||||
UV_CACHE_DIR='relcache'
|
||||
_record_uv_cache_choice
|
||||
cat '$MARKER'
|
||||
RELATIVE
|
||||
} > "$RELATIVE_PROBE"
|
||||
rm -f "$MARKER"
|
||||
_rel=$($shell "$RELATIVE_PROBE")
|
||||
if [ "$_rel" = "$CASE/relcache" ]; then
|
||||
ok "$shell: a relative cache is recorded absolute"
|
||||
else
|
||||
bad "$shell: relative cache recorded as [$_rel], wanted [$CASE/relcache]"
|
||||
fi
|
||||
|
||||
# And the exported variable is absolute too, not only the marker: setup.sh changes
|
||||
# into its own directory before the dependency pass, so a relative value would put
|
||||
# that phase's artifacts somewhere neither the install phase nor the marker names.
|
||||
EXPORT_PROBE="$WORK/$shell custom export.sh"
|
||||
{
|
||||
printf '%s\n' "$HELPERS"
|
||||
cat <<EXPORTED
|
||||
step() { :; }
|
||||
substep() { :; }
|
||||
STUDIO_HOME='$ROOT'
|
||||
_UV_MARKER_SAVED=false
|
||||
cd '$CASE'
|
||||
UV_CACHE_DIR='relcache'
|
||||
_configure_uv_cache
|
||||
printf '%s\n' "\$UV_CACHE_DIR"
|
||||
EXPORTED
|
||||
} > "$EXPORT_PROBE"
|
||||
rm -f "$MARKER"
|
||||
_exp=$($shell "$EXPORT_PROBE")
|
||||
if [ "$_exp" = "$CASE/relcache" ]; then
|
||||
ok "$shell: a relative custom cache is exported absolute"
|
||||
else
|
||||
bad "$shell: custom cache exported as [$_exp], wanted [$CASE/relcache]"
|
||||
fi
|
||||
|
||||
# An unwritable STUDIO_HOME is a reason to skip the marker, never to fail the install.
|
||||
rm -rf "$ROOT/cache"
|
||||
if [ "$(id -u 2>/dev/null || echo 0)" != 0 ] && mkdir -p "$ROOT" && chmod 500 "$ROOT" 2>/dev/null; then
|
||||
run_case "$shell" "an unwritable Studio root still installs" unset "" false \
|
||||
"$HOME_DIR" unset "" "$ROOT" "$BUILDS_CACHE" "$BUILDS_CACHE" shared \
|
||||
"reusing existing shared cache ($BUILDS_CACHE) to avoid duplicate Torch/CUDA downloads; use --isolated-uv-cache to isolate" \
|
||||
"$STUDIO_CACHE"
|
||||
chmod 755 "$ROOT" 2>/dev/null || true
|
||||
fi
|
||||
done
|
||||
|
||||
_resolve_line=$(grep -n '^_resolve_studio_destinations$' "$INSTALL_SH" | head -n1 | cut -d: -f1)
|
||||
|
|
|
|||
|
|
@ -23,7 +23,8 @@ $subjectNames = @(
|
|||
"Test-StudioVenvRollbackMustBePreserved",
|
||||
"Remove-StaleStudioVenvRollbacks",
|
||||
"Restore-StudioVenvRollback",
|
||||
"Complete-StudioVenvRollback"
|
||||
"Complete-StudioVenvRollback",
|
||||
"Restore-StudioUvCacheMarker"
|
||||
)
|
||||
|
||||
$definitions = @{}
|
||||
|
|
@ -109,6 +110,9 @@ function Reset-RollbackState($target) {
|
|||
$script:StudioVenvRollbackDir = $null
|
||||
$script:StudioVenvRollbackTarget = $target
|
||||
$script:StudioVenvRollbackActive = $false
|
||||
# As Install-UnslothStudio does at entry: the commit flag is per install, and the
|
||||
# restore consults it, so a previous section's commit would suppress this one.
|
||||
$script:StudioInstallCommitted = $false
|
||||
}
|
||||
|
||||
$StudioHome = Join-Path ([System.IO.Path]::GetTempPath()) "unsloth-rollback-$([guid]::NewGuid().ToString('N'))"
|
||||
|
|
|
|||
|
|
@ -14,6 +14,10 @@ from typing import Callable, Optional
|
|||
|
||||
STAGE_DIR_NAME = ".update-stage"
|
||||
STAGE_ROOT_ENV = "UNSLOTH_STUDIO_STAGE_ROOT"
|
||||
# Where a staged update parks the uv cache it used. The live marker is written only once
|
||||
# the stage is accepted, so an update that never activates cannot redirect the environment
|
||||
# it did not replace.
|
||||
UV_CACHE_MARKER = "uv-cache-dir"
|
||||
SHELL_VERSION_ENV = "UNSLOTH_TAURI_SHELL_VERSION"
|
||||
READY_MARKER = "READY.json"
|
||||
VENV_NAME = "unsloth_studio"
|
||||
|
|
@ -227,6 +231,30 @@ def run_staged_update(root: Path, args: list[str]) -> int:
|
|||
return subprocess.call(command, cwd = str(root), env = child_environment(root))
|
||||
|
||||
|
||||
def promote_uv_cache_marker(root: Path, studio_home: Path) -> Optional[Path]:
|
||||
"""Publish the staged update's cache choice to the live install.
|
||||
|
||||
Best effort in both directions: a stage that parked nothing leaves the live marker
|
||||
alone, and a live root that cannot be written costs the next update a preference.
|
||||
"""
|
||||
source = root / UV_CACHE_MARKER
|
||||
try:
|
||||
payload = source.read_bytes()
|
||||
except OSError:
|
||||
return None
|
||||
if not payload.strip():
|
||||
return None
|
||||
marker = studio_home / "cache" / UV_CACHE_MARKER
|
||||
try:
|
||||
marker.parent.mkdir(parents = True, exist_ok = True)
|
||||
# Unlinked first: a write follows a symlink and truncates its target.
|
||||
marker.unlink(missing_ok = True)
|
||||
marker.write_bytes(payload)
|
||||
except OSError:
|
||||
return None
|
||||
return marker
|
||||
|
||||
|
||||
def write_ready_marker(root: Path, backend_version: str, shell_version: Optional[str]) -> Path:
|
||||
marker = root / READY_MARKER
|
||||
payload = {
|
||||
|
|
@ -273,6 +301,8 @@ def stage(
|
|||
version = installed_version(root / VENV_NAME, env)
|
||||
shell_version = (os.environ.get(SHELL_VERSION_ENV) or "").strip() or None
|
||||
write_ready_marker(root, version, shell_version)
|
||||
# After verification: until the probes pass there is no accepted stage.
|
||||
promote_uv_cache_marker(root, studio_home)
|
||||
except BaseException:
|
||||
discard(root)
|
||||
raise
|
||||
|
|
|
|||
|
|
@ -3650,6 +3650,195 @@ _PS_PROXY_DEFAULTS_PRELUDE = (
|
|||
)
|
||||
|
||||
|
||||
_UV_CACHE_BUCKETS = ("archive-", "builds-", "built-wheels-", "wheels-", "sdists-")
|
||||
_UV_CACHE_METADATA_SUFFIXES = (".lock", ".msgpack", ".http", ".rev")
|
||||
|
||||
|
||||
def _uv_cache_has_packages(cache_dir: Path) -> bool:
|
||||
"""wheels-* is metadata only on uv 0.10, so counting any file reads a cache that was
|
||||
merely resolved against as warm. Same rule as install.sh:_configure_uv_cache."""
|
||||
try:
|
||||
buckets = [
|
||||
entry
|
||||
for entry in cache_dir.iterdir()
|
||||
if entry.name.startswith(_UV_CACHE_BUCKETS) and entry.is_dir()
|
||||
]
|
||||
except (OSError, ValueError):
|
||||
# ValueError too: an embedded NUL builds a Path fine and then raises out of
|
||||
# scandir, aborting an update over a file that is only advisory.
|
||||
return False
|
||||
for bucket in buckets:
|
||||
for _root, _dirs, files in os.walk(bucket):
|
||||
for name in files:
|
||||
if name in ("CACHEDIR.TAG", ".git", ".gitignore"):
|
||||
continue
|
||||
if name.endswith(_UV_CACHE_METADATA_SUFFIXES):
|
||||
continue
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _uv_platform_cache_dir() -> Optional[Path]:
|
||||
"""install.sh:646's fallback, for when uv cannot be asked: that is not "no cache"."""
|
||||
if platform.system() == "Windows":
|
||||
local_app_data = (os.environ.get("LOCALAPPDATA") or "").strip()
|
||||
return Path(local_app_data) / "uv" / "cache" if local_app_data else None
|
||||
xdg = (os.environ.get("XDG_CACHE_HOME") or "").strip()
|
||||
if xdg:
|
||||
return Path(xdg) / "uv"
|
||||
home = (os.environ.get("HOME") or "").strip()
|
||||
return Path(home) / ".cache" / "uv" if home else None
|
||||
|
||||
|
||||
# uv's boolish spelling. Anything outside it is a value uv refuses to run on, which is
|
||||
# not "no cache" either.
|
||||
_UV_TRUE = ("1", "true", "yes", "on")
|
||||
|
||||
|
||||
def _uv_no_cache_requested() -> bool:
|
||||
"""uv --no-cache caches in a temporary directory and discards it on exit, so the
|
||||
probe would report that throwaway, --no-cache outranks --cache-dir anyway, and
|
||||
recording it would aim later updates at a cache that never existed."""
|
||||
return (os.environ.get("UV_NO_CACHE") or "").strip().lower() in _UV_TRUE
|
||||
|
||||
|
||||
def _uv_default_cache_dir(cwd: Optional[Path] = None) -> Optional[Path]:
|
||||
"""Asked of uv, not reconstructed, so uv.toml and UV_CONFIG_FILE count.
|
||||
|
||||
Asked from where setup will ask it, too. Both setup scripts change into their own
|
||||
directory before the dependency pass (studio/setup.sh:1788), and uv discovers
|
||||
uv.toml and pyproject.toml from its working directory, so probing in the caller's
|
||||
would answer for whatever project the user happens to be standing in.
|
||||
"""
|
||||
uv = shutil.which("uv")
|
||||
if not uv:
|
||||
return _uv_platform_cache_dir()
|
||||
child_env = {key: value for key, value in os.environ.items() if key != "UV_CACHE_DIR"}
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[uv, "cache", "dir"],
|
||||
cwd = str(cwd) if cwd is not None else None,
|
||||
capture_output = True,
|
||||
text = True,
|
||||
# UnicodeDecodeError is a ValueError, so the handler below would not catch it.
|
||||
encoding = "utf-8",
|
||||
errors = "replace",
|
||||
env = child_env,
|
||||
timeout = 30,
|
||||
# Creation flags are not inherited from a hidden desktop update.
|
||||
**_windows_hidden_subprocess_kwargs(),
|
||||
)
|
||||
except Exception:
|
||||
# Best effort: not knowing costs a preference, never the update.
|
||||
return _uv_platform_cache_dir()
|
||||
if result.returncode != 0:
|
||||
# A malformed uv.toml beside the CALLER fails this, though setup.sh runs uv elsewhere.
|
||||
return _uv_platform_cache_dir()
|
||||
# Not stripped: a name may end in a space, and uv reports it verbatim.
|
||||
lines = [line for line in (result.stdout or "").splitlines() if line.strip()]
|
||||
if not lines:
|
||||
return _uv_platform_cache_dir()
|
||||
# uv answers a relative cache-dir with the relative spelling, against its own working
|
||||
# directory, and resolves a relative UV_WORKING_DIR after starting where this probe
|
||||
# did. No expanduser: uv makes a literal "~" directory, not one in $HOME.
|
||||
probe_cwd = str(cwd) if cwd is not None else os.getcwd()
|
||||
working = os.environ.get("UV_WORKING_DIR")
|
||||
base = os.path.join(probe_cwd, working) if working else probe_cwd
|
||||
return Path(os.path.abspath(os.path.join(base, lines[-1])))
|
||||
|
||||
|
||||
def _recorded_install_uv_cache() -> Optional[Path]:
|
||||
"""The cache the installer used, as it recorded it.
|
||||
|
||||
Content cannot tell a Studio cache the installer filled from one holding a single
|
||||
wheel the running backend dropped there (wheel_utils.py:405), since install.sh:705
|
||||
points it there even in shared mode. Absent, the caller falls back to content.
|
||||
"""
|
||||
try:
|
||||
# utf-8-sig: Windows PowerShell 5.1 writes `-Encoding utf8` WITH a BOM.
|
||||
# surrogateescape: a POSIX path may not be UTF-8, and U+FFFD would name nothing.
|
||||
recorded = (STUDIO_HOME / "cache" / "uv-cache-dir").read_text(
|
||||
encoding = "utf-8-sig", errors = "surrogateescape"
|
||||
)
|
||||
except OSError:
|
||||
return None
|
||||
# One record, one trailing delimiter, everything before it the path: splitting on
|
||||
# lines would take a POSIX path containing a newline for several. Not otherwise
|
||||
# stripped, since a path may end or begin with a space; blank means unset.
|
||||
if recorded.endswith("\n"):
|
||||
recorded = recorded[:-1]
|
||||
if recorded.endswith("\r"):
|
||||
recorded = recorded[:-1]
|
||||
if not recorded.strip():
|
||||
return None
|
||||
# No expanduser, as in the probe: uv treats a tilde as an ordinary path segment.
|
||||
return Path(os.path.abspath(recorded))
|
||||
|
||||
|
||||
def _backfill_uv_cache_marker(env: Optional[dict]) -> None:
|
||||
"""Record the cache this update used, for installs whose installer never did.
|
||||
|
||||
Those reach the content fallback, which goes stale the moment the backend drops one
|
||||
wheel into the Studio cache and both caches look warm. Writing the choice down once
|
||||
setup has succeeded closes that, and likewise for a marker whose cache was emptied.
|
||||
"""
|
||||
if (os.environ.get("UV_CACHE_DIR") or "").strip():
|
||||
# One run's value. Only an installer's own choice becomes a marker.
|
||||
return
|
||||
if _uv_no_cache_requested():
|
||||
# Setup cached nothing that outlived it, so there is no choice to record.
|
||||
return
|
||||
chosen = (env or {}).get("UV_CACHE_DIR")
|
||||
if not chosen:
|
||||
return
|
||||
live = _recorded_install_uv_cache()
|
||||
if live is not None and _uv_cache_has_packages(live):
|
||||
return
|
||||
stage_root = (os.environ.get(_studio_stage.STAGE_ROOT_ENV) or "").strip()
|
||||
if stage_root:
|
||||
# STUDIO_HOME names the LIVE install here and the stage can still be rejected,
|
||||
# so the choice is parked and _studio_stage.stage promotes it on acceptance.
|
||||
# Dropping it instead left desktop-only installs on the content fallback.
|
||||
marker = Path(stage_root) / _studio_stage.UV_CACHE_MARKER
|
||||
else:
|
||||
marker = STUDIO_HOME / "cache" / "uv-cache-dir"
|
||||
try:
|
||||
marker.parent.mkdir(parents = True, exist_ok = True)
|
||||
# Unlinked first: a write follows a symlink and truncates its target.
|
||||
marker.unlink(missing_ok = True)
|
||||
# fsencode: an undecodable path arrives as surrogates, and encoding those
|
||||
# raises UnicodeEncodeError, which is not an OSError.
|
||||
marker.write_bytes(os.fsencode(f"{chosen}\n"))
|
||||
except (OSError, ValueError):
|
||||
# ValueError covers UnicodeError, for a path fsencode still cannot render.
|
||||
pass
|
||||
|
||||
|
||||
def _with_studio_uv_cache(env: Optional[dict], cwd: Optional[Path] = None) -> Optional[dict]:
|
||||
"""An update reached neither installer nor _setup_cache_env, so uv re-downloaded
|
||||
what the install had just fetched."""
|
||||
if (os.environ.get("UV_CACHE_DIR") or "").strip():
|
||||
return env
|
||||
if _uv_no_cache_requested():
|
||||
# --no-cache outranks --cache-dir, so naming one changes nothing.
|
||||
return env
|
||||
studio_cache = STUDIO_HOME / "cache" / "uv"
|
||||
recorded = _recorded_install_uv_cache()
|
||||
if recorded is not None and _uv_cache_has_packages(recorded):
|
||||
# Only while it holds something: a marker for an emptied cache loses to a warm one.
|
||||
return {**(env or os.environ), "UV_CACHE_DIR": str(recorded)}
|
||||
# No marker, so this install predates it and content is all there is, and content
|
||||
# cannot settle it: install.sh:705 points the running backend at the Studio cache even
|
||||
# in shared mode, so one on-demand wheel warms it. uv's default is what such an install
|
||||
# has been updating from all along, and it cannot record its way out of a wrong guess.
|
||||
default_cache = _uv_default_cache_dir(cwd)
|
||||
if default_cache is not None and _uv_cache_has_packages(default_cache):
|
||||
# Named, not re-resolved: a blank inherited value reaches uv as
|
||||
# `--cache-dir ''` and exits 2.
|
||||
return {**(env or os.environ), "UV_CACHE_DIR": str(default_cache)}
|
||||
return {**(env or os.environ), "UV_CACHE_DIR": str(studio_cache)}
|
||||
|
||||
|
||||
def _run_setup_script(*, verbose: bool = False, repo_root: Optional[Path] = None) -> None:
|
||||
"""Find and run the studio setup/update script."""
|
||||
script = _find_setup_script(repo_root)
|
||||
|
|
@ -3664,6 +3853,11 @@ def _run_setup_script(*, verbose: bool = False, repo_root: Optional[Path] = None
|
|||
raise typer.Exit(1)
|
||||
|
||||
env = {**os.environ, "UNSLOTH_VERBOSE": "1"} if verbose else None
|
||||
# Where setup will run uv from, which differs by platform: setup.sh changes into its
|
||||
# own directory (setup.sh:1788), setup.ps1 never does and hands install_python_stack.py
|
||||
# the cwd it inherited from here (setup.ps1:5191).
|
||||
setup_cwd = None if platform.system() == "Windows" else script.parent
|
||||
env = _with_studio_uv_cache(env, cwd = setup_cwd)
|
||||
|
||||
if platform.system() == "Windows":
|
||||
# Resolved, not bare: the gate that runs immediately before this in setup() and update()
|
||||
|
|
@ -3719,6 +3913,8 @@ def _run_setup_script(*, verbose: bool = False, repo_root: Optional[Path] = None
|
|||
|
||||
if returncode != 0:
|
||||
raise typer.Exit(returncode)
|
||||
# Only now: a cache that did not get through setup is not one to record.
|
||||
_backfill_uv_cache_marker(env)
|
||||
|
||||
|
||||
# The refresh re-runs the installer with --shortcuts-only, fetched rather than shipped
|
||||
|
|
|
|||
|
|
@ -88,7 +88,12 @@ def test_the_setup_handoff_spawns_the_resolved_interpreter(monkeypatch, tmp_path
|
|||
|
||||
studio._run_setup_script(repo_root = repo_root)
|
||||
|
||||
assert spawned and spawned[0][0] == _RESOLVED, spawned
|
||||
# The handoff, not merely the first spawn: _run_setup_script asks uv where its cache
|
||||
# is before it hands over, and subprocess.run is built on Popen, so that probe lands
|
||||
# here too. What this test is about is which interpreter the handoff itself uses.
|
||||
handoffs = [argv for argv in spawned if argv and argv[-1].endswith("*>&1")]
|
||||
assert len(handoffs) == 1, spawned
|
||||
assert handoffs[0][0] == _RESOLVED, spawned
|
||||
|
||||
|
||||
def test_the_profile_probe_falls_back_to_the_resolved_interpreter(monkeypatch, tmp_path):
|
||||
|
|
|
|||
|
|
@ -155,6 +155,71 @@ def test_stage_builds_a_ready_marker_from_a_successful_update(monkeypatch, tmp_p
|
|||
assert echoed == ["[TAURI:STEP] clone", "[TAURI:STEP] update", "[TAURI:STEP] verify"]
|
||||
|
||||
|
||||
def test_an_accepted_stage_publishes_the_uv_cache_it_used(monkeypatch, tmp_path):
|
||||
"""The staged child cannot write the live marker: STUDIO_HOME names the install it is
|
||||
replacing, and the stage may never activate. It parks the choice, and acceptance is
|
||||
what publishes it, or a desktop-only install never records one at all."""
|
||||
home = tmp_path / "studio"
|
||||
_make_venv(home)
|
||||
monkeypatch.delenv(_studio_stage.SHELL_VERSION_ENV, raising = False)
|
||||
|
||||
def fake_update(root: Path, args: list[str]) -> int:
|
||||
(root / _studio_stage.UV_CACHE_MARKER).write_text("/warm/uv\n", encoding = "utf-8")
|
||||
return 0
|
||||
|
||||
monkeypatch.setattr(_studio_stage, "installed_version", lambda venv, env: "2026.9.1")
|
||||
monkeypatch.setattr(_studio_stage, "probe_cli", lambda venv, env: None)
|
||||
monkeypatch.setattr(_studio_stage, "probe_console_script", lambda venv, env: None)
|
||||
|
||||
_studio_stage.stage(home, update_args = [], echo = lambda _: None, run_update = fake_update)
|
||||
|
||||
live = home / "cache" / _studio_stage.UV_CACHE_MARKER
|
||||
assert live.read_text(encoding = "utf-8").strip() == "/warm/uv"
|
||||
|
||||
|
||||
def test_a_rejected_stage_publishes_nothing(monkeypatch, tmp_path):
|
||||
"""Verification is the acceptance point, so a stage that parks a choice and then fails
|
||||
its probes must leave the live install pointing where it already pointed."""
|
||||
home = tmp_path / "studio"
|
||||
_make_venv(home)
|
||||
(home / "cache").mkdir(parents = True)
|
||||
(home / "cache" / _studio_stage.UV_CACHE_MARKER).write_text("/live/uv\n", encoding = "utf-8")
|
||||
monkeypatch.delenv(_studio_stage.SHELL_VERSION_ENV, raising = False)
|
||||
|
||||
def fake_update(root: Path, args: list[str]) -> int:
|
||||
(root / _studio_stage.UV_CACHE_MARKER).write_text("/staged/uv\n", encoding = "utf-8")
|
||||
return 0
|
||||
|
||||
def refuse(venv, env):
|
||||
raise _studio_stage.StageError("staged environment failed to start")
|
||||
|
||||
monkeypatch.setattr(_studio_stage, "probe_cli", refuse)
|
||||
|
||||
with pytest.raises(_studio_stage.StageError):
|
||||
_studio_stage.stage(home, update_args = [], echo = lambda _: None, run_update = fake_update)
|
||||
|
||||
live = home / "cache" / _studio_stage.UV_CACHE_MARKER
|
||||
assert live.read_text(encoding = "utf-8").strip() == "/live/uv"
|
||||
|
||||
|
||||
def test_a_stage_that_parks_nothing_leaves_the_live_marker_alone(monkeypatch, tmp_path):
|
||||
"""Nothing to promote is the normal case for a caller-pinned UV_CACHE_DIR, and it must
|
||||
not blank the marker an installer wrote."""
|
||||
home = tmp_path / "studio"
|
||||
_make_venv(home)
|
||||
(home / "cache").mkdir(parents = True)
|
||||
(home / "cache" / _studio_stage.UV_CACHE_MARKER).write_text("/live/uv\n", encoding = "utf-8")
|
||||
monkeypatch.delenv(_studio_stage.SHELL_VERSION_ENV, raising = False)
|
||||
monkeypatch.setattr(_studio_stage, "installed_version", lambda venv, env: "2026.9.1")
|
||||
monkeypatch.setattr(_studio_stage, "probe_cli", lambda venv, env: None)
|
||||
monkeypatch.setattr(_studio_stage, "probe_console_script", lambda venv, env: None)
|
||||
|
||||
_studio_stage.stage(home, update_args = [], echo = lambda _: None, run_update = lambda root, args: 0)
|
||||
|
||||
live = home / "cache" / _studio_stage.UV_CACHE_MARKER
|
||||
assert live.read_text(encoding = "utf-8").strip() == "/live/uv"
|
||||
|
||||
|
||||
def test_stage_discards_the_tree_when_the_update_fails(monkeypatch, tmp_path):
|
||||
home = tmp_path / "studio"
|
||||
_make_venv(home)
|
||||
|
|
|
|||
887
unsloth_cli/tests/test_studio_update_uv_cache.py
Normal file
887
unsloth_cli/tests/test_studio_update_uv_cache.py
Normal file
|
|
@ -0,0 +1,887 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""`unsloth studio update` must reuse the cache the install filled, not uv's default.
|
||||
|
||||
The installers set UV_CACHE_DIR (#10204) and storage_roots._setup_cache_env sets it
|
||||
for the server; an update ran setup.sh/setup.ps1 from the CLI process and reached
|
||||
neither, so it re-downloaded into uv's own default what the install had just fetched.
|
||||
|
||||
It must not overcorrect either: a shared-mode install leaves the wheels in uv's own
|
||||
cache, and pointing the update at the empty Studio one costs a download online and
|
||||
fails outright when uv may read only what is cached.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
if str(_REPO_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(_REPO_ROOT))
|
||||
|
||||
|
||||
def _studio():
|
||||
from unsloth_cli.commands import studio as _studio_mod
|
||||
return _studio_mod
|
||||
|
||||
|
||||
def _setup_tree(tmp_path: Path) -> Path:
|
||||
repo_root = tmp_path / "repo"
|
||||
(repo_root / "studio").mkdir(parents = True, exist_ok = True)
|
||||
(repo_root / "studio" / "setup.sh").write_text("")
|
||||
(repo_root / "studio" / "setup.ps1").write_text("")
|
||||
return repo_root
|
||||
|
||||
|
||||
def _fill(
|
||||
cache_dir: Path,
|
||||
*,
|
||||
bucket: str = "archive-v0",
|
||||
name: str = "payload.so",
|
||||
) -> Path:
|
||||
"""Give a cache the shape uv gives it: a bucket directory holding package bytes."""
|
||||
leaf = cache_dir / bucket / "pkg"
|
||||
leaf.mkdir(parents = True, exist_ok = True)
|
||||
(leaf / name).write_bytes(b"\0" * 16)
|
||||
return cache_dir
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def caches(monkeypatch, tmp_path):
|
||||
"""Both caches, cold, and uv's default answered without spawning uv.
|
||||
|
||||
Every test states the state it needs; leaving the real machine's cache in play would
|
||||
make the outcome depend on whoever ran the suite.
|
||||
"""
|
||||
studio = _studio()
|
||||
monkeypatch.delenv("UV_CACHE_DIR", raising = False)
|
||||
studio_home = tmp_path / "StudioHome"
|
||||
default_cache = tmp_path / "default-uv"
|
||||
monkeypatch.setattr(studio, "STUDIO_HOME", studio_home)
|
||||
monkeypatch.setattr(studio, "_uv_default_cache_dir", lambda cwd = None: default_cache)
|
||||
return studio_home / "cache" / "uv", default_cache
|
||||
|
||||
|
||||
class _Result:
|
||||
returncode = 0
|
||||
|
||||
|
||||
def _run_posix(monkeypatch, tmp_path: Path) -> dict:
|
||||
studio = _studio()
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Linux")
|
||||
seen: dict = {}
|
||||
|
||||
def _fake_run(
|
||||
argv,
|
||||
env = None,
|
||||
**kwargs,
|
||||
):
|
||||
seen["argv"] = list(argv)
|
||||
seen["env"] = env
|
||||
return _Result()
|
||||
|
||||
monkeypatch.setattr(studio.subprocess, "run", _fake_run)
|
||||
studio._run_setup_script(repo_root = _setup_tree(tmp_path))
|
||||
return seen
|
||||
|
||||
|
||||
def test_the_update_uses_the_studio_cache_when_the_caller_set_none(monkeypatch, tmp_path, caches):
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"] is not None, "env must be materialised, not left as inherit-everything"
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("blank", ["", " ", "\t"])
|
||||
def test_a_blank_uv_cache_dir_counts_as_unset(monkeypatch, tmp_path, caches, blank):
|
||||
"""storage_roots.py:373 treats blank as unset; the update path must agree, or an
|
||||
inherited UV_CACHE_DIR= pins uv's cache to the empty string."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
monkeypatch.setenv("UV_CACHE_DIR", blank)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_an_explicit_uv_cache_dir_still_wins(monkeypatch, tmp_path, caches):
|
||||
"""Same precedence the installers use: a nonblank caller value is preserved
|
||||
(install.sh:626, install.ps1:1232), so CI images that pin a cache keep it."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
monkeypatch.setenv("UV_CACHE_DIR", str(tmp_path / "caller cache"))
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"] is None or seen["env"]["UV_CACHE_DIR"] == str(
|
||||
tmp_path / "caller cache"
|
||||
), seen["env"]
|
||||
|
||||
|
||||
def test_verbose_keeps_its_own_flag_alongside_the_cache(monkeypatch, tmp_path, caches):
|
||||
"""The verbose branch builds env first; the cache seeding must extend it, not
|
||||
replace it."""
|
||||
studio = _studio()
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Linux")
|
||||
seen: dict = {}
|
||||
|
||||
def _fake_run(
|
||||
argv,
|
||||
env = None,
|
||||
**kwargs,
|
||||
):
|
||||
seen["env"] = env
|
||||
return _Result()
|
||||
|
||||
monkeypatch.setattr(studio.subprocess, "run", _fake_run)
|
||||
studio._run_setup_script(verbose = True, repo_root = _setup_tree(tmp_path))
|
||||
|
||||
assert seen["env"]["UNSLOTH_VERBOSE"] == "1", seen["env"].get("UNSLOTH_VERBOSE")
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache)
|
||||
|
||||
|
||||
def test_the_windows_branch_gets_the_same_cache(monkeypatch, tmp_path, caches):
|
||||
"""setup.ps1 runs the same uv pip installs, so the PowerShell spawn needs it too."""
|
||||
studio = _studio()
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Windows")
|
||||
monkeypatch.setattr(
|
||||
studio._studio_runtime_gate, "resolve_windows_powershell", lambda: "powershell.exe"
|
||||
)
|
||||
monkeypatch.setattr(studio, "_probe_profile_proxy_defaults", lambda hosts: None)
|
||||
monkeypatch.setattr(studio, "_wait_for_windows_setup_process", lambda process: 0)
|
||||
seen: dict = {}
|
||||
|
||||
class _Process:
|
||||
pass
|
||||
|
||||
def _fake_popen(
|
||||
argv,
|
||||
env = None,
|
||||
**kwargs,
|
||||
):
|
||||
seen["env"] = env
|
||||
return _Process()
|
||||
|
||||
monkeypatch.setattr(studio.subprocess, "Popen", _fake_popen)
|
||||
studio._run_setup_script(repo_root = _setup_tree(tmp_path))
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache)
|
||||
|
||||
|
||||
def test_the_seeding_does_not_leak_into_this_process(monkeypatch, tmp_path, caches):
|
||||
"""_ensure_studio_env_exported mutates os.environ on purpose; this must not, or a
|
||||
later `unsloth studio` in the same process would look like a caller override to
|
||||
storage_roots._setup_cache_env."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
_run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert "UV_CACHE_DIR" not in os.environ, os.environ.get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
# --- Do not move the update off a warm cache onto a cold one -------------------------
|
||||
|
||||
|
||||
def test_a_shared_mode_install_keeps_the_cache_that_actually_has_the_wheels(
|
||||
monkeypatch, tmp_path, caches
|
||||
):
|
||||
"""install.sh picks uv's own cache when it is already populated, and
|
||||
_setup_cache_env mkdirs an empty Studio cache on every server start. Redirecting
|
||||
here would re-download online and, under UV_OFFLINE / `offline = true`, fail:
|
||||
uv reads only what is cached, and nothing is."""
|
||||
_studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(default_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("blank", ["", " ", "\t"])
|
||||
def test_a_blank_value_is_replaced_even_when_the_default_cache_wins(
|
||||
monkeypatch, tmp_path, caches, blank
|
||||
):
|
||||
"""Leaving the environment alone here would hand uv the blank value it inherited.
|
||||
uv reads UV_CACHE_DIR as --cache-dir, and `UV_CACHE_DIR= uv cache dir` exits 2 with
|
||||
"a value is required for '--cache-dir'", so every uv call in setup.sh would die."""
|
||||
_studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
monkeypatch.setenv("UV_CACHE_DIR", blank)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(default_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_the_chosen_cache_is_named_rather_than_left_to_the_child(monkeypatch, tmp_path, caches):
|
||||
"""setup.sh changes directory before running uv, so a cache uv resolved from a
|
||||
uv.toml beside the caller is not one the child would find again. Whichever cache
|
||||
wins, it reaches the child as an explicit path."""
|
||||
studio_cache, default_cache = caches
|
||||
for warm, expected in ((default_cache, default_cache), (studio_cache, studio_cache)):
|
||||
# Each iteration states its own starting point: the other cache goes cold, and the
|
||||
# previous run's backfilled marker goes away, or either would decide this one.
|
||||
for cache in (studio_cache, default_cache):
|
||||
shutil.rmtree(cache, ignore_errors = True)
|
||||
_fill(warm)
|
||||
(tmp_path / "StudioHome" / "cache" / "uv-cache-dir").unlink(missing_ok = True)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(expected), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_a_studio_mode_install_wins_over_a_cold_default(monkeypatch, tmp_path, caches):
|
||||
"""The case the PR exists for: the installer filled the Studio cache and the update
|
||||
was downloading all of it again into a cache holding nothing."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_two_warm_caches_and_no_marker_keep_uv_s_default(monkeypatch, tmp_path, caches):
|
||||
"""An install that predates the marker cannot say which cache it used, and content
|
||||
cannot tell either: one on-demand wheel warms the Studio cache (install.sh:705). uv's
|
||||
default is what it has been updating from, and it cannot record its way out."""
|
||||
studio_cache, default_cache = caches
|
||||
_fill(studio_cache)
|
||||
_fill(default_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(default_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_a_marker_still_settles_two_warm_caches(monkeypatch, tmp_path, caches):
|
||||
"""And that ambiguity is exactly what the marker removes: an install that recorded the
|
||||
Studio cache keeps it, warm default or not."""
|
||||
studio_cache, default_cache = caches
|
||||
_fill(studio_cache)
|
||||
_fill(default_cache)
|
||||
_record(tmp_path / "StudioHome", studio_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_two_cold_caches_still_go_to_the_studio_one(monkeypatch, tmp_path, caches):
|
||||
"""Nothing to preserve, so keep the bytes under the Studio root where uninstall can
|
||||
reclaim them (#10193)."""
|
||||
studio_cache, _default = caches
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("metadata", ["resolve.msgpack", "wheel.http", "pkg.lock", "x.rev"])
|
||||
def test_a_metadata_only_default_cache_is_not_warm(monkeypatch, tmp_path, caches, metadata):
|
||||
"""wheels-v6 holds only metadata on uv 0.10, so a single `uv pip install --dry-run`
|
||||
would otherwise pin every later update to uv's default cache (#10204)."""
|
||||
studio_cache, default_cache = caches
|
||||
_fill(default_cache, bucket = "wheels-v6", name = metadata)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_a_default_cache_uv_cannot_name_does_not_block_the_redirect(monkeypatch, tmp_path, caches):
|
||||
"""No uv on PATH, or `uv cache dir` failing, must not strand the update on a cache
|
||||
nobody can identify."""
|
||||
studio = _studio()
|
||||
studio_cache, _default = caches
|
||||
monkeypatch.setattr(studio, "_uv_default_cache_dir", lambda cwd = None: None)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["1", "true", "YES", "on"])
|
||||
def test_no_cache_mode_neither_seeds_nor_records(monkeypatch, tmp_path, caches, value):
|
||||
"""uv --no-cache caches in a temporary directory and discards it at exit, so seeding
|
||||
a cache would be ignored and recording one would name something already gone."""
|
||||
studio = _studio()
|
||||
studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
monkeypatch.setenv("UV_NO_CACHE", value)
|
||||
monkeypatch.setattr(
|
||||
studio,
|
||||
"_uv_default_cache_dir",
|
||||
lambda cwd = None: pytest.fail("the probe ran under --no-cache"),
|
||||
)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert "UV_CACHE_DIR" not in (seen["env"] or {}), seen["env"]
|
||||
assert not _marker(tmp_path).exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", ["0", "false", "", "maybe"])
|
||||
def test_a_non_true_no_cache_value_changes_nothing(monkeypatch, tmp_path, caches, value):
|
||||
"""0 and false leave caching on, and an unparseable value is one uv refuses to run
|
||||
on at all."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
monkeypatch.setenv("UV_NO_CACHE", value)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
# --- The warmth test itself ----------------------------------------------------------
|
||||
|
||||
|
||||
def test_package_bytes_beside_metadata_count_as_warm(tmp_path):
|
||||
"""A real cache has both; the metadata filter must not hide the payload."""
|
||||
studio = _studio()
|
||||
cache = tmp_path / "uv"
|
||||
_fill(cache, bucket = "wheels-v6", name = "resolve.msgpack")
|
||||
_fill(cache, bucket = "wheels-v6", name = "torch.whl")
|
||||
|
||||
assert studio._uv_cache_has_packages(cache) is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bucket", ["archive-v0", "builds-v0", "built-wheels-v3", "sdists-v9"])
|
||||
def test_every_bucket_uv_has_used_counts(tmp_path, bucket):
|
||||
"""uv renames these across versions, and an install.sh that scans a bucket this does
|
||||
not would disagree with itself about which cache is warm."""
|
||||
studio = _studio()
|
||||
cache = tmp_path / bucket.split("-")[0]
|
||||
_fill(cache, bucket = bucket)
|
||||
|
||||
assert studio._uv_cache_has_packages(cache) is True
|
||||
|
||||
|
||||
def test_an_absent_cache_is_not_warm(tmp_path):
|
||||
studio = _studio()
|
||||
|
||||
assert studio._uv_cache_has_packages(tmp_path / "nope") is False
|
||||
|
||||
|
||||
# --- The cache the installer recorded --------------------------------------------------
|
||||
|
||||
|
||||
def _record(studio_home: Path, value) -> None:
|
||||
marker = studio_home / "cache"
|
||||
marker.mkdir(parents = True, exist_ok = True)
|
||||
(marker / "uv-cache-dir").write_text(f"{value}\n", encoding = "utf-8")
|
||||
|
||||
|
||||
def test_the_recorded_install_cache_beats_both_guesses(monkeypatch, tmp_path, caches):
|
||||
"""The case content cannot decide: a shared install whose backend has since dropped
|
||||
one wheel into the Studio cache (install.sh:705), so both caches hold packages."""
|
||||
studio_cache, default_cache = caches
|
||||
_fill(studio_cache, name = "some_runtime_wheel.whl")
|
||||
_fill(default_cache)
|
||||
_record(tmp_path / "StudioHome", default_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(default_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_a_recorded_studio_cache_survives_the_user_warming_their_own(monkeypatch, tmp_path, caches):
|
||||
"""The mirror: a studio install, and the user has since warmed uv's own cache."""
|
||||
studio_cache, default_cache = caches
|
||||
_fill(studio_cache)
|
||||
_fill(default_cache)
|
||||
_record(tmp_path / "StudioHome", studio_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_an_emptied_recorded_cache_does_not_outrank_a_warm_one(monkeypatch, tmp_path, caches):
|
||||
"""A marker pointing at an emptied cache is stale, not authoritative."""
|
||||
studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
_record(tmp_path / "StudioHome", studio_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(default_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_installs_older_than_the_marker_still_work(monkeypatch, tmp_path, caches):
|
||||
"""The normal state for everyone installed before this change."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_a_reinstall_into_a_custom_cache_is_not_shadowed_by_the_old_marker(
|
||||
monkeypatch, tmp_path, caches
|
||||
):
|
||||
"""A reinstall with a nonblank UV_CACHE_DIR fills that cache, so it is recorded too:
|
||||
a stale marker would aim later updates at a cache this install never filled, and both
|
||||
hold packages, so nothing downstream could notice."""
|
||||
studio_cache, _default = caches
|
||||
custom = tmp_path / "caller cache"
|
||||
_fill(studio_cache)
|
||||
_fill(custom)
|
||||
_record(tmp_path / "StudioHome", custom)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(custom), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("spelling", ["trailing ", " leading", " both "])
|
||||
def test_a_recorded_path_keeps_its_whitespace(monkeypatch, tmp_path, caches, spelling):
|
||||
"""A recorded name may start or end with a space, and stripping it would probe a
|
||||
different path and read a warm cache as cold."""
|
||||
_studio_cache, _default = caches
|
||||
odd = tmp_path / spelling
|
||||
_fill(odd)
|
||||
_record(tmp_path / "StudioHome", odd)
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(odd), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_a_marker_written_by_windows_powershell_is_read_back(monkeypatch, tmp_path, caches):
|
||||
"""PowerShell 5.1 writes `-Encoding utf8` with a BOM, which utf-8 would decode into
|
||||
the first character of the path."""
|
||||
studio_cache, default_cache = caches
|
||||
_fill(studio_cache)
|
||||
_fill(default_cache)
|
||||
marker = tmp_path / "StudioHome" / "cache"
|
||||
marker.mkdir(parents = True, exist_ok = True)
|
||||
(marker / "uv-cache-dir").write_bytes(b"\xef\xbb\xbf" + f"{studio_cache}\r\n".encode("utf-8"))
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_a_relative_marker_is_resolved_before_it_is_handed_over(monkeypatch, tmp_path, caches):
|
||||
"""setup.sh changes directory, so a relative path would name somewhere else there."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
monkeypatch.chdir(tmp_path)
|
||||
_record(tmp_path / "StudioHome", "StudioHome/cache/uv")
|
||||
seen = _run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert seen["env"]["UV_CACHE_DIR"] == str(studio_cache), seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_a_recorded_path_containing_a_newline_survives_the_round_trip(
|
||||
monkeypatch, tmp_path, caches
|
||||
):
|
||||
"""A newline is legal in a POSIX path. Read line by line the record looked like
|
||||
several, only the last fragment survived, and a warm cache read as absent."""
|
||||
studio = _studio()
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
weird = tmp_path / "two\nline cache"
|
||||
_fill(weird)
|
||||
_record(tmp_path / "StudioHome", weird)
|
||||
|
||||
env = studio._with_studio_uv_cache({})
|
||||
|
||||
assert env["UV_CACHE_DIR"] == str(weird)
|
||||
|
||||
|
||||
# --- Backfilling the marker for installs that predate it --------------------------------
|
||||
|
||||
|
||||
def _marker(tmp_path: Path) -> Path:
|
||||
return tmp_path / "StudioHome" / "cache" / "uv-cache-dir"
|
||||
|
||||
|
||||
def test_a_legacy_install_records_what_the_update_worked_out(monkeypatch, tmp_path, caches):
|
||||
"""Otherwise the fallback is re-derived every time, and goes stale the moment the
|
||||
backend drops one wheel into the Studio cache."""
|
||||
_studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
_run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert _marker(tmp_path).read_text(encoding = "utf-8").strip() == str(default_cache)
|
||||
|
||||
|
||||
def test_a_failed_update_records_nothing(monkeypatch, tmp_path, caches):
|
||||
"""A cache that did not get through setup is not one to aim later updates at."""
|
||||
studio = _studio()
|
||||
_studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Linux")
|
||||
|
||||
class _Failed:
|
||||
returncode = 1
|
||||
|
||||
monkeypatch.setattr(studio.subprocess, "run", lambda argv, **kw: _Failed())
|
||||
with pytest.raises(studio.typer.Exit):
|
||||
studio._run_setup_script(repo_root = _setup_tree(tmp_path))
|
||||
|
||||
assert not _marker(tmp_path).exists(), _marker(tmp_path).read_text(encoding = "utf-8")
|
||||
|
||||
|
||||
def test_a_live_marker_is_not_overwritten_by_the_update(monkeypatch, tmp_path, caches):
|
||||
"""The installer's statement outranks the update's inference while it still holds."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
_record(tmp_path / "StudioHome", studio_cache)
|
||||
_run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert _marker(tmp_path).read_text(encoding = "utf-8").strip() == str(studio_cache)
|
||||
|
||||
|
||||
def test_a_stale_marker_is_replaced_once_the_fallback_works(monkeypatch, tmp_path, caches):
|
||||
"""A marker whose cache was emptied is already ignored; stop re-deriving that."""
|
||||
studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
_record(tmp_path / "StudioHome", studio_cache)
|
||||
_run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert _marker(tmp_path).read_text(encoding = "utf-8").strip() == str(default_cache)
|
||||
|
||||
|
||||
def test_a_caller_supplied_cache_is_never_promoted_to_the_marker(monkeypatch, tmp_path, caches):
|
||||
"""One run's environment variable must not become every later update's default."""
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
monkeypatch.setenv("UV_CACHE_DIR", str(tmp_path / "caller cache"))
|
||||
_run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert not _marker(tmp_path).exists()
|
||||
|
||||
|
||||
def test_a_staged_update_parks_its_choice_in_the_stage(monkeypatch, tmp_path, caches):
|
||||
"""STUDIO_HOME names the LIVE install even in a staged child and the stage can still
|
||||
be rejected, so the choice waits there for stage() to promote it. Dropping it left
|
||||
desktop-only installs on the content fallback forever."""
|
||||
studio = _studio()
|
||||
_studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
stage = tmp_path / "stage"
|
||||
stage.mkdir()
|
||||
monkeypatch.setenv(studio._studio_stage.STAGE_ROOT_ENV, str(stage))
|
||||
_run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert not _marker(tmp_path).exists()
|
||||
assert (stage / "uv-cache-dir").read_text(encoding = "utf-8").strip() == str(default_cache)
|
||||
|
||||
|
||||
def test_a_marker_holding_a_null_byte_reads_as_a_cold_cache(monkeypatch, tmp_path, caches):
|
||||
"""Path takes an embedded NUL and scandir then raises ValueError, not OSError, so a
|
||||
corrupted or hand-edited marker aborted every update before setup ran."""
|
||||
studio = _studio()
|
||||
studio_cache, _default = caches
|
||||
_fill(studio_cache)
|
||||
_record(tmp_path / "StudioHome", "/nul\x00cache")
|
||||
|
||||
env = studio._with_studio_uv_cache({})
|
||||
|
||||
assert env["UV_CACHE_DIR"] == str(studio_cache)
|
||||
|
||||
|
||||
def test_the_backfill_replaces_a_symlink_rather_than_its_target(monkeypatch, tmp_path, caches):
|
||||
"""write_text follows a symlink, so a marker path linked elsewhere would truncate an
|
||||
unrelated file."""
|
||||
_studio_cache, default_cache = caches
|
||||
_fill(default_cache)
|
||||
victim = tmp_path / "someone elses file"
|
||||
victim.write_text("do not clobber", encoding = "utf-8")
|
||||
marker = _marker(tmp_path)
|
||||
marker.parent.mkdir(parents = True, exist_ok = True)
|
||||
marker.symlink_to(victim)
|
||||
_run_posix(monkeypatch, tmp_path)
|
||||
|
||||
assert victim.read_text(encoding = "utf-8") == "do not clobber"
|
||||
assert not marker.is_symlink()
|
||||
assert marker.read_text(encoding = "utf-8").strip() == str(default_cache)
|
||||
|
||||
|
||||
@pytest.mark.skipif(os.name != "posix", reason = "POSIX filesystem byte semantics")
|
||||
def test_a_cache_path_that_is_not_utf_8_is_recorded_and_read_back(monkeypatch, tmp_path):
|
||||
"""An undecodable POSIX path arrives as surrogates, and encoding those raises
|
||||
UnicodeEncodeError, not an OSError, so it escaped the best-effort handler and failed
|
||||
an update whose setup had succeeded, with its marker already gone."""
|
||||
studio = _studio()
|
||||
weird = (tmp_path / os.fsdecode(b"caf\xe9-cache")).resolve()
|
||||
_fill(weird)
|
||||
monkeypatch.setattr(studio, "STUDIO_HOME", tmp_path / "StudioHome")
|
||||
monkeypatch.setattr(studio, "_uv_default_cache_dir", lambda cwd = None: weird)
|
||||
monkeypatch.delenv("UV_CACHE_DIR", raising = False)
|
||||
|
||||
studio._backfill_uv_cache_marker({"UV_CACHE_DIR": str(weird)})
|
||||
|
||||
assert _marker(tmp_path).read_bytes().strip() == os.fsencode(str(weird))
|
||||
# And the reader gives back the path the filesystem uses, not one with U+FFFD in it.
|
||||
assert studio._recorded_install_uv_cache() == weird
|
||||
# The tmp_path reaper cannot always delete a name it cannot decode.
|
||||
shutil.rmtree(weird, ignore_errors = True)
|
||||
|
||||
|
||||
# --- The uv probe ---------------------------------------------------------------------
|
||||
|
||||
|
||||
def _probe_kwargs(
|
||||
monkeypatch,
|
||||
stdout: str = "/cache/uv\n",
|
||||
cwd = None,
|
||||
) -> dict:
|
||||
studio = _studio()
|
||||
monkeypatch.setattr(studio.shutil, "which", lambda name: "/usr/bin/uv")
|
||||
seen: dict = {}
|
||||
|
||||
class _Completed:
|
||||
returncode = 0
|
||||
|
||||
def _fake_run(argv, **kwargs):
|
||||
seen.update(kwargs)
|
||||
seen["argv"] = list(argv)
|
||||
completed = _Completed()
|
||||
completed.stdout = stdout
|
||||
return completed
|
||||
|
||||
monkeypatch.setattr(studio.subprocess, "run", _fake_run)
|
||||
seen["result"] = studio._uv_default_cache_dir(cwd)
|
||||
return seen
|
||||
|
||||
|
||||
def test_the_probe_asks_from_the_directory_setup_will_ask_from(monkeypatch, tmp_path):
|
||||
"""uv discovers uv.toml and pyproject.toml from its working directory, and both setup
|
||||
scripts change into their own before the dependency pass (studio/setup.sh:1788). Asked
|
||||
in the caller's directory instead, the probe answers for whatever project the user
|
||||
happens to be standing in, and that answer is then forced on the child."""
|
||||
seen = _probe_kwargs(monkeypatch, stdout = "relcache\n", cwd = tmp_path / "studio")
|
||||
|
||||
assert seen["cwd"] == str(tmp_path / "studio")
|
||||
# And the relative answer resolves against that directory, not against the caller's.
|
||||
assert seen["result"] == tmp_path / "studio" / "relcache", seen["result"]
|
||||
|
||||
|
||||
def test_the_windows_handoff_probes_from_the_directory_it_hands_the_child(monkeypatch, tmp_path):
|
||||
"""setup.ps1 never changes directory: it hands install_python_stack.py the cwd it
|
||||
inherited (setup.ps1:5191), so probing the script's directory would answer for a
|
||||
configuration the child never sees, and that answer is forced on it."""
|
||||
studio = _studio()
|
||||
repo_root = tmp_path / "repo"
|
||||
(repo_root / "studio").mkdir(parents = True)
|
||||
(repo_root / "studio" / "setup.ps1").write_text("", encoding = "utf-8")
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Windows")
|
||||
monkeypatch.setattr(
|
||||
studio._studio_runtime_gate, "resolve_windows_powershell", lambda: "powershell.exe"
|
||||
)
|
||||
monkeypatch.setattr(studio, "_probe_profile_proxy_defaults", lambda hosts: None)
|
||||
monkeypatch.setattr(studio, "_backfill_uv_cache_marker", lambda env: None)
|
||||
monkeypatch.delenv("UV_CACHE_DIR", raising = False)
|
||||
seen: dict = {}
|
||||
monkeypatch.setattr(
|
||||
studio, "_with_studio_uv_cache", lambda env, cwd = None: seen.update(cwd = cwd) or env
|
||||
)
|
||||
|
||||
class _Process:
|
||||
def wait(self):
|
||||
return 0
|
||||
|
||||
monkeypatch.setattr(studio.subprocess, "Popen", lambda argv, **kw: _Process())
|
||||
|
||||
studio._run_setup_script(repo_root = repo_root)
|
||||
|
||||
assert seen["cwd"] is None, seen
|
||||
|
||||
|
||||
def test_the_posix_handoff_probes_from_the_setup_script_directory(monkeypatch, tmp_path):
|
||||
"""setup.sh does change into its own directory before the dependency pass
|
||||
(studio/setup.sh:1788), so there is where the probe has to ask."""
|
||||
studio = _studio()
|
||||
repo_root = tmp_path / "repo"
|
||||
(repo_root / "studio").mkdir(parents = True)
|
||||
(repo_root / "studio" / "setup.sh").write_text("", encoding = "utf-8")
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Linux")
|
||||
monkeypatch.setattr(studio, "_backfill_uv_cache_marker", lambda env: None)
|
||||
monkeypatch.delenv("UV_CACHE_DIR", raising = False)
|
||||
seen: dict = {}
|
||||
monkeypatch.setattr(
|
||||
studio, "_with_studio_uv_cache", lambda env, cwd = None: seen.update(cwd = cwd) or env
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
studio.subprocess, "Popen", lambda *a, **kw: (_ for _ in ()).throw(RuntimeError("stop"))
|
||||
)
|
||||
|
||||
with pytest.raises(RuntimeError, match = "stop"):
|
||||
studio._run_setup_script(repo_root = repo_root)
|
||||
|
||||
assert seen["cwd"] == repo_root / "studio", seen
|
||||
|
||||
|
||||
def test_the_probe_keeps_whitespace_uv_reported(monkeypatch):
|
||||
"""A directory name may begin or end with a space and uv reports it verbatim, so
|
||||
stripping the line probes a path that does not exist and reads a warm cache as cold."""
|
||||
seen = _probe_kwargs(monkeypatch, stdout = "/spaced cache/uv \n")
|
||||
|
||||
assert seen["result"] == Path("/spaced cache/uv "), seen["result"]
|
||||
|
||||
|
||||
def test_a_relative_uv_working_dir_anchors_to_the_probe_directory(monkeypatch, tmp_path):
|
||||
"""uv resolves --directory after starting where it was started, so a relative
|
||||
UV_WORKING_DIR belongs to the probe's cwd, not to wherever the update was launched."""
|
||||
monkeypatch.setenv("UV_WORKING_DIR", "uvdir")
|
||||
seen = _probe_kwargs(monkeypatch, stdout = "relcache\n", cwd = tmp_path / "studio")
|
||||
|
||||
assert seen["result"] == tmp_path / "studio" / "uvdir" / "relcache", seen["result"]
|
||||
|
||||
|
||||
def test_the_probe_decodes_utf8_whatever_the_console_codec_is(monkeypatch):
|
||||
"""text=True alone decodes with the locale codec and strict errors, so a non-ASCII
|
||||
cache path raises UnicodeDecodeError. That is a ValueError, so the OSError /
|
||||
SubprocessError handler does not catch it and the update dies. Same reason the
|
||||
profile probe above already pins the codec."""
|
||||
seen = _probe_kwargs(monkeypatch)
|
||||
|
||||
assert seen["encoding"] == "utf-8", seen.get("encoding")
|
||||
assert seen["errors"] == "replace", seen.get("errors")
|
||||
|
||||
|
||||
def test_the_probe_is_hidden_like_every_other_spawn(monkeypatch):
|
||||
"""A desktop update runs the CLI with CREATE_NO_WINDOW, and creation flags are not
|
||||
inherited, so each process it starts has to ask for them itself."""
|
||||
studio = _studio()
|
||||
monkeypatch.setattr(studio, "_should_hide_windows_subprocesses", lambda: True)
|
||||
monkeypatch.setattr(studio.subprocess, "CREATE_NO_WINDOW", 0x08000000, raising = False)
|
||||
seen = _probe_kwargs(monkeypatch)
|
||||
|
||||
expected = studio._windows_hidden_subprocess_kwargs()
|
||||
for key, value in expected.items():
|
||||
assert key in seen, f"{key} missing from the probe call"
|
||||
assert seen.get("creationflags") == expected.get("creationflags")
|
||||
|
||||
|
||||
def test_the_probe_absolutises_a_relative_answer(monkeypatch, tmp_path):
|
||||
"""uv answers `cache-dir = "relcache"` with "relcache" verbatim, and setup.sh runs uv
|
||||
from a different directory, so a relative answer names a different, cold cache there."""
|
||||
monkeypatch.delenv("UV_WORKING_DIR", raising = False)
|
||||
monkeypatch.chdir(tmp_path)
|
||||
seen = _probe_kwargs(monkeypatch, stdout = "relcache\n")
|
||||
|
||||
assert seen["result"] == tmp_path / "relcache", seen["result"]
|
||||
|
||||
|
||||
def test_the_probe_resolves_against_uvs_working_directory(monkeypatch, tmp_path):
|
||||
"""--directory, whose env alias is UV_WORKING_DIR, moves uv out from under us before
|
||||
it resolves a relative cache-dir. Measured on uv 0.10.7: with UV_WORKING_DIR set, the
|
||||
cache lands under that directory and not under ours."""
|
||||
elsewhere = tmp_path / "elsewhere"
|
||||
elsewhere.mkdir()
|
||||
monkeypatch.setenv("UV_WORKING_DIR", str(elsewhere))
|
||||
monkeypatch.chdir(tmp_path)
|
||||
seen = _probe_kwargs(monkeypatch, stdout = "relcache\n")
|
||||
|
||||
assert seen["result"] == elsewhere / "relcache", seen["result"]
|
||||
|
||||
|
||||
def test_the_probe_leaves_uvs_tilde_alone(monkeypatch, tmp_path):
|
||||
"""uv prints `cache-dir = "~/.myuv"` verbatim and treats the tilde as an ordinary
|
||||
relative segment: measured on uv 0.10.7 it creates a literal "~" directory in its
|
||||
working directory. Expanding it here would probe a path uv never writes to."""
|
||||
monkeypatch.delenv("UV_WORKING_DIR", raising = False)
|
||||
monkeypatch.chdir(tmp_path)
|
||||
seen = _probe_kwargs(monkeypatch, stdout = "~/.myuv\n")
|
||||
|
||||
assert seen["result"] == tmp_path / "~" / ".myuv", seen["result"]
|
||||
|
||||
|
||||
def test_a_probe_that_blows_up_costs_a_preference_not_the_update(monkeypatch, tmp_path):
|
||||
"""subprocess.run is implemented with Popen, so any caller that fakes Popen reaches
|
||||
this probe too. Raising through an update is not acceptable, and neither is reporting
|
||||
no cache at all: it falls back to the platform default install.sh:646 uses."""
|
||||
studio = _studio()
|
||||
monkeypatch.setattr(studio.shutil, "which", lambda name: "/usr/bin/uv")
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Linux")
|
||||
monkeypatch.setenv("XDG_CACHE_HOME", str(tmp_path))
|
||||
|
||||
def _boom(argv, **kwargs):
|
||||
raise TypeError("object does not support the context manager protocol")
|
||||
|
||||
monkeypatch.setattr(studio.subprocess, "run", _boom)
|
||||
|
||||
assert studio._uv_default_cache_dir() == tmp_path / "uv"
|
||||
|
||||
|
||||
def test_a_malformed_uv_toml_beside_the_caller_does_not_hide_the_cache(monkeypatch, tmp_path):
|
||||
"""uv discovers config from the CURRENT directory, so a broken uv.toml where the user
|
||||
happens to be makes `uv cache dir` exit nonzero even though setup.sh changes directory
|
||||
before it runs uv. install.sh falls back to the platform default rather than calling
|
||||
the cache cold, and so must this."""
|
||||
studio = _studio()
|
||||
monkeypatch.setattr(studio.shutil, "which", lambda name: "/usr/bin/uv")
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Linux")
|
||||
monkeypatch.setenv("XDG_CACHE_HOME", str(tmp_path))
|
||||
|
||||
class _Failed:
|
||||
returncode = 2
|
||||
stdout = ""
|
||||
|
||||
monkeypatch.setattr(studio.subprocess, "run", lambda argv, **kw: _Failed())
|
||||
|
||||
assert studio._uv_default_cache_dir() == tmp_path / "uv"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("system", "env", "expected"),
|
||||
[
|
||||
("Linux", {"XDG_CACHE_HOME": "/xdg"}, Path("/xdg/uv")),
|
||||
("Linux", {"HOME": "/home/someone"}, Path("/home/someone/.cache/uv")),
|
||||
("Darwin", {"HOME": "/Users/someone"}, Path("/Users/someone/.cache/uv")),
|
||||
("Windows", {"LOCALAPPDATA": "/local"}, Path("/local/uv/cache")),
|
||||
],
|
||||
)
|
||||
def test_the_platform_default_matches_the_installers(monkeypatch, system, env, expected):
|
||||
"""Same locations install.sh:646 and install.ps1 use. macOS is XDG/HOME, not
|
||||
~/Library/Caches, which is what uv documents for Unix generally."""
|
||||
studio = _studio()
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: system)
|
||||
for key in ("XDG_CACHE_HOME", "HOME", "LOCALAPPDATA"):
|
||||
monkeypatch.delenv(key, raising = False)
|
||||
for key, value in env.items():
|
||||
monkeypatch.setenv(key, value)
|
||||
|
||||
assert studio._uv_platform_cache_dir() == expected
|
||||
|
||||
|
||||
def test_no_platform_default_is_better_than_a_guess(monkeypatch):
|
||||
"""Nothing to anchor to means nothing to report; the caller then keeps the Studio
|
||||
cache rather than probing a path assembled out of nothing."""
|
||||
studio = _studio()
|
||||
monkeypatch.setattr(studio.platform, "system", lambda: "Linux")
|
||||
for key in ("XDG_CACHE_HOME", "HOME"):
|
||||
monkeypatch.delenv(key, raising = False)
|
||||
|
||||
assert studio._uv_platform_cache_dir() is None
|
||||
|
||||
|
||||
def test_the_probe_reads_the_last_nonblank_line(monkeypatch):
|
||||
"""A wrapper or a future uv may print a notice ahead of the path, and a two-line
|
||||
value is not a directory."""
|
||||
seen = _probe_kwargs(monkeypatch, stdout = "warning: something\n/real/cache/uv\n\n")
|
||||
|
||||
assert seen["result"] == Path("/real/cache/uv"), seen["result"]
|
||||
|
||||
|
||||
def test_the_probe_hides_uv_cache_dir_from_uv(monkeypatch):
|
||||
"""Asking uv for its default while UV_CACHE_DIR is set would just echo that value
|
||||
back, and a blank one makes uv exit 2 instead of answering."""
|
||||
monkeypatch.setenv("UV_CACHE_DIR", "")
|
||||
seen = _probe_kwargs(monkeypatch)
|
||||
|
||||
assert "UV_CACHE_DIR" not in seen["env"], seen["env"].get("UV_CACHE_DIR")
|
||||
|
||||
|
||||
def test_files_outside_a_bucket_are_not_warm(tmp_path):
|
||||
"""uv writes CACHEDIR.TAG and .gitignore at the cache root; a cache holding only
|
||||
those has never fetched anything."""
|
||||
studio = _studio()
|
||||
cache = tmp_path / "uv"
|
||||
cache.mkdir()
|
||||
(cache / "CACHEDIR.TAG").write_text("Signature: 8a477f597d28d172789f06886806bc55")
|
||||
(cache / ".gitignore").write_text("*")
|
||||
(cache / "simple-v20").mkdir()
|
||||
(cache / "simple-v20" / "index.msgpack").write_bytes(b"\0")
|
||||
|
||||
assert studio._uv_cache_has_packages(cache) is False
|
||||
Loading…
Add table
Add a link
Reference in a new issue