vrr/unsloth
2
0
Fork 0
mirror of https://github.com/unslothai/unsloth.git synced 2026-10-22 22:41:04 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 34

Studio installer: torch 2.11 on Linux CPU, Vulkan for AMD without ROCm, gfx1033 gate, and three install fixes (#8412)

Browse source 
* Studio installer: torch 2.11 on Linux CPU, Vulkan for AMD without ROCm, gfx1033 gate, and three install fixes

Split out of #8343, which also rewrites the Linux AppImage packaging. These six
installer changes are independent of that work and verified on their own, so they
land here while the AppImage side finishes.

Original work by Thomas Eric (@ThomasEricB), who found all six on a Steam Deck.

Co-authored-by: Thomas Eric <thombelcar@gmail.com>

* Find the venv creation call by what it runs, not by its label

#8479 moved venv creation behind _run_uv_venv, so the label this test
grepped for no longer exists and the ordering assertion read as a
too-late cache rather than a stale grep. Also drop a .studio-test-root
artifact that a local test run left behind in the commit.

* Three install.sh fixes: gfx1033 token match, override handoff, uv cache fallback

Match gfx1033 as a TOKEN in the multiline ROCm probe. _probe_amd_gfx_arch keeps
every `grep -oE` hit, and rocminfo names each GPU agent twice: once as the
agent's own "Name: gfx1033" and once in its ISA Info block as
"amdgcn-amd-amdhsa--gfx1033". A single-GPU Steam Deck therefore already probes
as "gfx1033\ngfx1033", which the exact-string case did not match, so the host
fell through to the version-keyed ROCm index and installed the very wheels the
gate exists to avoid. Flatten to one space-delimited, lowercased,
suffix-stripped line; the surrounding spaces keep gfx10330 out.

Stop forwarding a rejected gfx1033 override to llama.cpp. Clearing the local
_linux_inferred_gfx is not enough: setup.sh copies UNSLOTH_ROCM_GFX_ARCH
straight into --rocm-gfx, and _apply_host_overrides reads any forwarded gfx as
proof of ROCm (has_rocm = True). On a Van Gogh host with no ROCm that skips the
AMD-without-ROCm Vulkan branch this release adds and asks for a ROCm prebuilt
or a HIP source build the host cannot run. Unset the rejected override so
setup.sh re-probes; a host that really does have ROCm still finds gfx1033
through its own rocminfo call, and a UNSLOTH_TORCH_INDEX_URL pin returns long
before this block.

Fall back to uv's default cache when the colocated one cannot be created. uv
aborts with "Failed to initialize cache at ..." on a cache path it cannot
create, so ignoring the mkdir status while keeping the export turned a disk
optimisation into a hard install failure on a host where uv's own default would
have worked. Measured: with STUDIO_HOME writable and its "cache" entry a file,
`uv venv` exits 2 with the export and 0 without it.

Tests: the gate test fed the gate a hand-built one-token probe, which is not
what the probe produces, so it passed throughout. Drive the real
get_torch_index_url against a real rocminfo shape instead, and assert first
that the probe really is multi-line so a future single-hit probe cannot turn
that into a vacuous pass.

* Demote an already-installed ROCm torch on a miscomputing AMD arch

Returning None from _amd_arch_index_url only stops a gfx1033 host from being
GIVEN ROCm wheels. A venv that already holds them -- installed before the gate
existed, and carried through the legacy-venv migration untouched because its
forward-only validation passes on this arch -- was never demoted: install.sh
resolves UNSLOTH_TORCH_BACKEND=cpu, which returns _ensure_rocm_torch at its
first line; _ensure_cpu_torch fired only for an EXPLICIT pin; and the base
update does not reinstall an already-satisfied torch. Upgrading therefore left
in place exactly the build the gate exists to remove, on the machines that most
need it removed.

Treat the arch itself as CPU authority so _ensure_cpu_torch does the demotion.
Scoped tightly: every AMD arch on the host must be a measured-bad one (a
healthy dGPU beside the APU keeps ROCm), an explicit UNSLOTH_TORCH_INDEX_URL /
_FAMILY still wins, and the disk label is read before any probe so hosts with
no ROCm torch pay nothing. The arch probe ignores HSA_OVERRIDE_GFX_VERSION,
which is the usual Van Gogh workaround and would otherwise make rocminfo answer
gfx1030 and hide the arch being judged.

* Honour an AMD visibility mask before auto-selecting Vulkan on Linux

ROCR_VISIBLE_DEVICES filters the HSA runtime's agent list, so it removes GPUs
from rocminfo's output too. A masked-out AMD device on a ROCm host therefore
leaves the ROCm probe empty and reads as a driver-only box, and the new DRM
sysfs pass set has_amd_gpu_without_rocm anyway. Vulkan honours none of the HIP
masks -- it selects through GGML_VK_VISIBLE_DEVICES -- so the install would
hand llama.cpp the exact GPU the caller hid, which
_should_auto_vulkan_for_amd_windows already refuses on Windows and the
has_physical_nvidia gate already refuses for CUDA_VISIBLE_DEVICES.

Conjoin the mask check with a ROCm probe actually being installed rather than
reusing _hip_visible_device_mask_set alone: the driver-only AMD host this
branch exists for ships no rocminfo or amd-smi, and measuring it showed a bare
exported CUDA_VISIBLE_DEVICES would otherwise cost a Steam Deck the Vulkan
bundle. Intel is not addressed by HIP masks and keeps its plain detection.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Judge the whole AMD inventory, and prove the uv cache is writable

* Restore the read-only rc warning main's tests assert

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Decline the ROCm route on a miscomputing target, and honour ROCR isolation on its own

* Read the physical arch through an HSA spoof and a visibility mask

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Say why ARM64 keeps the CPU archive

* Gate the CPU torch 2.11 floor on glibc 2.28

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Judge the gfx1033 gate on the visible AMD targets

* Set the colocated uv cache on the standalone update path too

* Resolve HIP-layer ordinals before classifying the visible gfx set

* Resolve both visibility masks in the gate, not in the probe environment

* Create the uv cache write probe with mktemp

* Decline ROCm when a visibility mask or declared arch cannot be trusted

* Gate gfx1033 on presence, not on which device the runtime picks

* Ask the kernel before falling back to a spoofable probe

* Disqualify the shared wheel family when gfx1033 is in the inventory

* Carry the physical gfx1033 veto into the reroute and refuse an unverifiable spoof

* Refuse the ROCm route before the inferred install, not after it

* Demote ROCm torch when the selected target is a miscomputing arch

The gfx1033 gate declined to INSTALL ROCm when the selected runtime target is a
miscomputing arch, but it did not remove a ROCm build the venv already held. On
a mixed host, a gfx1033 APU beside a healthy dGPU, _rocm_miscomputing_host() is
False by design (it requires every physical AMD arch to be miscomputing), so
_ensure_cpu_torch() had no authority to demote either. A device mask selecting
the APU therefore printed "keeping CPU torch" while ROCm torch stayed installed,
and the selected APU kept producing incorrect gradients under it. The message
was wrong at the same time as the state.

The two gates answer different questions and only the first could demote:
_rocm_miscomputing_host() reads the inventory, while this branch reads the
selected target, which is the one that decides what actually runs. Recorded the
selected-target verdict and taught _ensure_cpu_torch to treat it as CPU
authority, which is the path the inventory gate already used.

Scoped to the case: with no such verdict a mixed host is untouched, so a machine
whose selected target is the healthy dGPU keeps ROCm.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert the selected-target ROCm demotion

I accepted a review item asking for this and was wrong: it contradicts a
deliberate invariant this file already documents and tests.

test_a_mask_cannot_shrink_the_host_to_its_bad_gpu states it directly. "Every
arch is bad" is a question about the HOST, so it is asked of the whole host:
ROCR_VISIBLE_DEVICES hiding a healthy dGPU would otherwise leave gfx1033 as the
only arch and demote a working ROCm install to CPU. The demotion path calls
_detect_amd_gfx_codes(ignore_hsa_override = True, ignore_visible_masks = True)
for that reason, and install.sh's _probe_amd_gfx_arch unsets both masks too.

Keying demotion off the selected runtime target is exactly the bypass that
guards against, one door over from the HSA_OVERRIDE spoof the same tests cover.
It went in green only because the demotion tests call _ensure_cpu_torch()
directly, so the flag set inside _ensure_rocm_torch() was never exercised.

What remains true is narrower and not worth this risk: on a masked mixed host
the branch prints "keeping CPU torch" while a ROCm build stays installed, so the
wording is inaccurate even though keeping it is the intended outcome.

* Say what the selected-target branch actually does

It printed "keeping CPU torch", which it cannot promise: whether an
already-installed ROCm build is removed is _ensure_cpu_torch's call, and that
asks about the whole host with the visible-device masks stripped, so a mixed
host with a healthy dGPU keeps ROCm on purpose. The branch only declines to
install ROCm for this target, so it now says that. The whole-host message above
is unchanged, since there the demotion really does follow.

* Tighten the comments added by this PR

Compress the explanatory blocks to their load-bearing content: 584 added
comment lines down to 313. No behaviour change; every test suite this PR
touches still passes.

* Make the arch-gate scenarios about the simulated host, not the test machine

* Shorten the comments added by this PR

Same content, fewer lines. Comments and docstrings only; no code changed.

---------

Co-authored-by: Thomas Eric <thombelcar@gmail.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-09-07 03:02:04 -07:00 • committed by GitHub
parent b66d2a4c89
commit e129630712
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 2010 additions and 41 deletions
Download patch file Download diff file Expand all files Collapse all files

151
install.sh
View file

@ -780,6 +780,28 @@ _resolve_studio_destinations
# for us; the pinned path does not.
_UNSLOTH_LOGIN_PATH="$PATH"
VENV_DIR="$STUDIO_HOME/unsloth_studio"
# Keep uv's cache on the same filesystem as the venv it fills.
# uv hardlinks wheels within one filesystem and copies across a boundary, so a moved
# STUDIO_HOME paid double the disk and stranded the cache. An explicit UV_CACHE_DIR wins.
# The fallback is required, since uv aborts on a cache it cannot create. mkdir -p exits 0 for
# an existing unwritable directory and -w reads the mode rather than the filesystem, so probe
# with a real create.
if [ -z "${UV_CACHE_DIR:-}" ]; then
UV_CACHE_DIR="$STUDIO_HOME/cache/uv"
export UV_CACHE_DIR
# mktemp, not a $$ name: a predictable path in another account's directory can be
# pre-created as a symlink for `: >` to follow and truncate as root.
_uv_cache_probe=""
if ! mkdir -p "$UV_CACHE_DIR" 2>/dev/null \
|| ! _uv_cache_probe=$(mktemp "$UV_CACHE_DIR/.unsloth-write-probe.XXXXXX" 2>/dev/null); then
echo "[WARN] Cannot write to $UV_CACHE_DIR -- using uv's default cache." >&2
echo "[WARN] Wheels will be copied into the venv rather than hardlinked, costing extra disk." >&2
unset UV_CACHE_DIR
fi
[ -z "$_uv_cache_probe" ] || rm -f "$_uv_cache_probe" 2>/dev/null || true
unset _uv_cache_probe
fi
_VENV_ROLLBACK_DIR=""
_VENV_ROLLBACK_TARGET="$VENV_DIR"
_VENV_ROLLBACK_ACTIVE=false
@ -3648,16 +3670,37 @@ _hsa_spoofed_physical_gfx() {
# as the last command would trip set -e in callers' assignments). Shared by
# get_torch_index_url's gfx gate and the runtime-less reroute gate so the two
# can never disagree on what "readable" means.
# $1: unset strip the visible-device masks only (#7314).
# physical also strip HSA_OVERRIDE_GFX_VERSION, which ROCr applies in userland so
# rocminfo reports the SPOOFED ISA while it is set (unslothai#7331).
# Mirrors _detect_amd_gfx_codes(ignore_hsa_override = True).
#
# Neither mode answers which device the runtime SELECTS, and no probe here applies both mask
# families. _runtime_gfx_target() in install_python_stack.py answers that.
#
# shellcheck disable=SC2086 # $_pg_strip is a LIST of names for unset; quoting it would
# unset one variable whose name contains spaces.
_probe_amd_gfx_arch() {
_ensure_rocm_probe_env
_pg=$(printf '%s' "${UNSLOTH_ROCM_GFX_ARCH:-}" | tr '[:upper:]' '[:lower:]')
case "${1:-}" in
physical) _pg_strip="ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES HSA_OVERRIDE_GFX_VERSION" ;;
*) _pg_strip="ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES" ;;
esac
# "physical" ignores the declared arch: a stale UNSLOTH_ROCM_GFX_ARCH=gfx1030 on a real Van
# Gogh would answer the miscomputing gate with a healthy arch. It stays authoritative for
# ordinary routing.
if [ "${1:-}" = "physical" ]; then
_pg=""
else
_pg=$(printf '%s' "${UNSLOTH_ROCM_GFX_ARCH:-}" | tr '[:upper:]' '[:lower:]')
fi
if [ -z "$_pg" ] && command -v rocminfo >/dev/null 2>&1; then
_pg=$( (unset ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES; rocminfo 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)
_pg=$( (unset $_pg_strip; rocminfo 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)
fi
if [ -z "$_pg" ] && command -v amd-smi >/dev/null 2>&1; then
_pg=$( (unset ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES; amd-smi list 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)
_pg=$( (unset $_pg_strip; amd-smi list 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)
if [ -z "$_pg" ]; then
_pg=$( (unset ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES; amd-smi static --asic 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)
_pg=$( (unset $_pg_strip; amd-smi static --asic 2>/dev/null) | grep -oE 'gfx[1-9][0-9a-z]{2,3}' || true)
fi
fi
printf '%s\n' "$_pg"
@ -3953,7 +3996,39 @@ get_torch_index_url() {
echo "[WARN] For GPU PyTorch, install or repair rocminfo/amd-smi (e.g. sudo pacman -S rocm-hip-sdk) and re-run this installer." >&2
echo "$_base/cpu"; return
fi
# AMD GPU confirmed -- detect ROCm version
# Archs measured to compute INCORRECTLY under ROCm route to CPU instead. Not
# "everything AMD does not list": unsloth serves gfx906 and gfx1031-gfx1036 on purpose
# (#7277). gfx1033 (Van Gogh) installs ROCm wheels and then computes wrong answers
# (studio/ROCM_RDNA2_APU.md). PRESENCE, not selection: picking the runtime's GPU needs
# the mask layering _runtime_gfx_target() implements, so a mixed host takes the cpu
# index and keeps the UNSLOTH_TORCH_INDEX_URL escape hatch. Inline, not a helper:
# harnesses extract get_torch_index_url alone. "physical" mode so no override hides the
# silicon; KFD first, since amdkfd writes gfx_target_version from the kernel.
_amd_gfx_gate_probe=$(_probe_amd_gfx_arch physical 2>/dev/null || true)
[ -n "$_amd_gfx_gate_probe" ] || _amd_gfx_gate_probe=$(_kfd_gfx_targets 2>/dev/null || true)
if [ -z "$_amd_gfx_gate_probe" ] && [ -n "${HSA_OVERRIDE_GFX_VERSION:-}" ]; then
# Under a spoof nothing override-independent named the silicon, so the only value
# left is the spoofed name: absence of evidence, not evidence of a healthy arch.
# HSA_OVERRIDE_GFX_VERSION only, NOT UNSLOTH_ROCM_GFX_ARCH, which is a DECLARED
# arch for the runtime-less #7301 hosts.
echo "[WARN] HSA_OVERRIDE_GFX_VERSION is set and this host cannot confirm its real arch (no unspoofed rocminfo, amd-smi or KFD topology)." >&2
echo "[WARN] Installing CPU-only PyTorch rather than trusting the spoofed name: gfx1033 (Van Gogh) computes incorrect results under ROCm (studio/ROCM_RDNA2_APU.md)." >&2
echo "[WARN] Unset HSA_OVERRIDE_GFX_VERSION so the arch can be read, or pin UNSLOTH_TORCH_INDEX_URL to choose deliberately." >&2
echo "$_base/cpu"; return
fi
[ -n "$_amd_gfx_gate_probe" ] || _amd_gfx_gate_probe="$_amd_gfx_probe"
_amd_gfx_tokens=" $(printf '%s\n' "$_amd_gfx_gate_probe" | sed 's/:.*$//' \
| tr '[:upper:]' '[:lower:]' | tr '\n' ' ')"
case "$_amd_gfx_tokens" in
*" gfx1033 "*)
echo "[WARN] AMD gfx1033 (Van Gogh) computes incorrect results under ROCm -- installing CPU-only PyTorch." >&2
echo "[WARN] ROCm wheels install on it but training diverges to NaN and gradcheck fails; forward math is fine." >&2
echo "[WARN] Details: studio/ROCM_RDNA2_APU.md. Override with UNSLOTH_TORCH_INDEX_URL if you want ROCm anyway." >&2
echo "$_base/cpu"; return ;;
esac
# end of the miscomputing-arch gate -- tests/sh/test_rocm_bad_arch_gate.sh lifts
# the block between the header comment above and this line, so keep both exact.
# detect ROCm version
_rocm_tag=""
_rocm_tag=$(_detect_rocm_version_tag) || _rocm_tag=""
# ^ || guard: belt and braces on the set -e contract the helpers hold, so a
@ -4577,6 +4652,17 @@ case "$TORCH_INDEX_URL" in
|| _amd_probed_family=""
_amd_probed_gfx_first=$(_amd_sole_index_arch "$_amd_probe_out") \
|| _amd_probed_gfx_first=""
# A measured-bad arch anywhere in the inventory disqualifies the whole family, at
# the source not at each consumer: gfx1033 shares gfx103X-all with gfx1030-gfx1036,
# so the shared-family arm below would otherwise rewrite the cpu index the gate
# just chose back to ROCm wheels.
case " $(printf '%s\n' "$_amd_probe_out" | sed 's/:.*$//' \
| tr '[:upper:]' '[:lower:]' | tr '\n' ' ')" in
*" gfx1033 "*)
echo "[WARN] AMD gfx1033 (Van Gogh) is in the probed inventory -- not routing torch to the shared $_amd_probed_family index (studio/ROCM_RDNA2_APU.md)." >&2
_amd_probed_family=""
_amd_probed_gfx_first="" ;;
esac
if [ -n "${_amd_probed_family:-}" ] && \
[ -z "$(_detect_rocm_version_tag 2>/dev/null)" ]; then
_amd_no_rocm_version_reroute=true
@ -4601,17 +4687,43 @@ if [ "$_torch_index_pinned" = false ] && [ "$SKIP_TORCH" = false ] && \
# for, so the suffix silently cost the reroute. Normalise as the caller did.
_linux_inferred_gfx=$(_amd_sole_index_arch "$_linux_inferred_gfx") \
|| _linux_inferred_gfx=""
# Route on the arch the probe READ, not on lspci marketing-name inference:
# Route on the arch the probe read, not on lspci marketing-name inference:
# the two disagree on a mixed APU + discrete host, and inference's answer
# would install wheels for a GPU the reroute decision never looked at.
if [ "${_amd_no_rocm_version_reroute:-false}" = true ]; then
_linux_inferred_gfx="${_amd_probed_gfx_first:-}"
fi
# The gfx1033 gate in get_torch_index_url is not enough alone: this reroute can take
# an explicit override or a no-version probe family straight back to gfx103X-all.
# UNSLOTH_ROCM_GFX_ARCH is unset too, not just the local: setup.sh forwards it as
# --rocm-gfx and _apply_host_overrides reads any forwarded gfx as proof of ROCm,
# skipping the Vulkan branch (112.8 vs 49.8 tok/s) and asking for a HIP build the host
# cannot run. Keyed on the PHYSICAL inventory, since a stale gfx1030 override makes the
# inferred value look healthy. The gate's own verdict ran in a command substitution and
# cannot be reused, hence the re-probe.
_amd_reroute_physical=$(_probe_amd_gfx_arch physical 2>/dev/null || true)
[ -n "$_amd_reroute_physical" ] || _amd_reroute_physical=$(_kfd_gfx_targets 2>/dev/null || true)
case " $(printf '%s\n' "$_amd_reroute_physical" | sed 's/:.*$//' \
| tr '[:upper:]' '[:lower:]' | tr '\n' ' ')" in
*" gfx1033 "*) _linux_inferred_gfx="gfx1033" ;;
esac
case "${_linux_inferred_gfx%%:*}" in
gfx1033)
echo "[WARN] AMD gfx1033 (Van Gogh) computes incorrect results under ROCm -- keeping CPU-only PyTorch (studio/ROCM_RDNA2_APU.md)." >&2
_linux_inferred_gfx=""
unset UNSLOTH_ROCM_GFX_ARCH ;;
esac
_amd_family=""
if [ "${_amd_no_rocm_version_reroute:-false}" = true ]; then
if [ "${_amd_no_rocm_version_reroute:-false}" = true ] && \
[ -n "$_linux_inferred_gfx" ]; then
_amd_family="${_amd_probed_family:-}"
elif [ -n "$_linux_inferred_gfx" ]; then
_amd_family=$(_amd_arch_index_family_for_gfx "$_linux_inferred_gfx") || _amd_family=""
elif [ "${_amd_no_rocm_version_reroute:-false}" = true ] && \
[ -z "${_amd_probed_gfx_first:-}" ]; then
# Multiple healthy arches may share one wheel family. No single arch is
# forwarded to llama.cpp, but torch can still use the agreed family.
_amd_family="${_amd_probed_family:-}"
fi
if [ -n "$_amd_family" ]; then
_amd_mirror="${UNSLOTH_AMD_ROCM_MIRROR:-https://repo.amd.com/rocm/whl}"
@ -5194,7 +5306,13 @@ elif case "$TORCH_INDEX_URL" in */rocm*|*/gfx*) true ;; *) false ;; esac; then
step "gpu" "AMD ROCm"
fi
_rocm_root="${ROCM_PATH:-${HIP_PATH:-/opt/rocm}}"
substep "ROCm: $_rocm_root"
# Only claim a path that is really there: /opt/rocm is a FALLBACK, not a detection, and
# a runtime-only ROCm (no SDK tree) reaches here with nothing at that path.
if [ -d "$_rocm_root" ]; then
substep "ROCm: $_rocm_root"
else
substep "ROCm: runtime detected (no SDK tree at $_rocm_root)"
fi
[ -n "$_gpu_rocm_ver" ] && substep "hipconfig: $_gpu_rocm_ver"
[ -n "$_gpu_disp_mkt" ] && [ -n "$_gpu_disp_gfx" ] && substep "GPU: $_gpu_disp_mkt"
elif [ "$OS" = "macos" ] && [ "$_ARCH" = "arm64" ]; then
@ -5957,6 +6075,9 @@ _persist_fish_path_dir() {
# The exact line we would write, not any occurrence of the directory: /opt/uv-old must not
# pass for /opt/uv, and fish reads none of the POSIX files that would otherwise cover it.
if ! grep -v '^[[:space:]]*#' "$_pfp_file" 2>/dev/null | grep -qxF "fish_add_path '$_pfp_quoted'"; then
# Same single-redirect, warning-not-failure contract as the POSIX arm. 2>/dev/null
# comes FIRST: redirections apply left to right, so the other order prints the
# shell's own "Permission denied" before the redirect can silence it.
if {
echo "# Added by Unsloth installer"
echo "fish_add_path '$_pfp_quoted'"
@ -5964,6 +6085,9 @@ _persist_fish_path_dir() {
step "path" "added $_pfp_label to PATH in $_pfp_file"
else
step "path" "could not write $_pfp_file; add $_pfp_label to PATH yourself" "$C_WARN"
substep "Unsloth is installed and works; only the PATH line is missing."
substep "Add this to your fish config to get 'unsloth' in new shells:"
substep " fish_add_path '$_pfp_quoted'"
fi
fi
}
@ -6008,8 +6132,12 @@ _persist_login_path_dir() {
# shell with no uv at all.
if ! grep -v '^[[:space:]]*#' "$_SHELL_PROFILE" 2>/dev/null \
| grep -E "$_PATH_LINE_RE" | grep -qE "$_plp_pattern"; then
# One redirect, so an unwritable profile leaves no half-written entry; a warning rather
# than a failure, because under set -e an unguarded append would end the install.
# One redirect, not three: a write dying midway would leave a dangling
# "# Added by Unsloth installer" with no export under it. A failure is a WARNING
# carrying the manual line, never a failed install: set -e would abort after the venv,
# llama.cpp and the shim are in place, and a read-only rc (NixOS, home-manager,
# chezmoi) is a supported setup.
# 2>/dev/null first, as in the fish arm.
if {
echo ''
echo '# Added by Unsloth installer'
@ -6018,6 +6146,9 @@ _persist_login_path_dir() {
step "path" "added $_plp_label to PATH in $_SHELL_PROFILE"
else
step "path" "could not write $_SHELL_PROFILE; add $_plp_label to PATH yourself" "$C_WARN"
substep "Unsloth is installed and works; only the PATH line is missing."
substep "Add this to your shell config to get 'unsloth' in new shells:"
substep " export PATH=\"$_plp_literal:\$PATH\""
fi
fi
}

107
studio/ROCM_RDNA2_APU.md Normal file
View file

@ -0,0 +1,107 @@
# ROCm on RDNA2 APUs (Steam Deck / gfx1033)
Measured findings for AMD RDNA2 integrated GPUs, recorded so the routing decisions in
`install_llama_prebuilt.py` (`has_amd_gpu_without_rocm`, the Linux Vulkan branches) are not
"fixed" back into ROCm by someone who reasonably assumes ROCm must be the fast path.
Hardware: Valve Steam Deck LCD, AMD Van Gogh, `gfx1033`, 8 CU RDNA2, 16 GB LPDDR5
shared, 1 GB default UMA carveout. SteamOS 3 (immutable, no system ROCm).
## Summary
| | verdict |
|---|---|
| GGUF inference via **Vulkan** | **works, fastest** — the shipped default |
| GGUF inference via ROCm | works, but slower than CPU |
| **PyTorch training** via ROCm | **numerically wrong — do not enable** |
| PyTorch training via CPU | correct |
## 1. Inference: Vulkan wins by a wide margin
`llama-bench`, Qwen2.5-0.5B-Instruct Q4_0, `-p 128 -n 64 -ngl 99 -r 2`, one session:
| backend | prompt t/s | generation t/s |
|---|---|---|
| **Vulkan (RADV)** | **1444.80 ± 8.03** | **112.81 ± 18.32** |
| CPU (`-ngl 0`) | 753.08 ± 43.65 | 49.76 ± 2.15 |
| ROCm `gfx103X` prebuilt, via `HSA_OVERRIDE_GFX_VERSION=10.3.0` | 802.06 ± 431.34 | 17.51 ± 1.89 |
ROCm is ~6x slower than Vulkan and ~3x slower than plain CPU, with very unstable prompt
throughput.
**Building llama.cpp from source for the real arch does not help.** Built from the same
source commit with `-DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1033` against AMD's native
gfx1033 `rocm-sdk` wheels (device reports `gfx1033 (0x1033)`, no `HSA_OVERRIDE`), all
three measured in one session:
| backend | prompt t/s | generation t/s |
|---|---|---|
| native `gfx1033` (self-built) | 864.60 ± 163.37 | 12.46 ± 0.34 |
| Vulkan (RADV) | 1268.85 ± 1.12 | 62.16 ± 9.99 |
| CPU | 686.43 ± 9.76 | 32.16 ± 0.42 |
Root cause: **108 of 142 gfx1033 rocBLAS Tensile files are named `fallback`** (76%).
AMD ships the arch but never tuned GEMM kernels for it, so "native but untuned" loses to
"foreign but tuned" (`gfx1030`), and both lose to Vulkan. Recompiling cannot synthesise a
tuning library; that needs Tensile's tuning pipeline, which wants far more RAM than the
device has.
Memory is *not* the limitation, so raising the BIOS UMA carveout does not change this:
llama.cpp's ROCm backend reports 8192 MiB (4478 free) via UMA detection and Vulkan
reports 9216 MiB (7361 free), both well past the 1 GB carveout.
## 2. Training: ROCm produces wrong results
Identical seed, init and data; 60 SGD steps on a 64→64→1 MLP with MSE loss:
| stack | CPU final loss | GPU final loss |
|---|---|---|
| rocm7.2 / torch 2.11 (`HSA_OVERRIDE` gfx1030) | 0.1148 | 17.4674 |
| rocm7.1 / torch 2.10 (`HSA_OVERRIDE` gfx1030) | 0.1148 | **-0.7765 → nan** |
| native gfx1033, TheRock torch 2.13 + ROCm 7.15 | 0.1148 | **nan** |
A negative MSE loss is not attainable, and `torch.autograd.gradcheck` fails in float64
on all three ("backward is not reentrant" on the native build — the same inputs and
`grad_output` produce different gradients between runs). The runtime also raises
`HSA_STATUS_ERROR_EXCEPTION` (code `0x1016`) and hangs the GPU queue under load.
Forward math is fine — full-module forward matched CPU to `4.2e-07` and loss to seven
significant figures — so this is specifically the backward pass. Three independent ROCm
versions fail identically, including AMD's own native gfx1033 build, so it is the
hardware/driver rather than a packaging or version problem.
Forward-only inference through PyTorch also disagreed with CPU on 1 of 40 fp32 runs, so
it is not dependable either. CPU is the correct default for the training stack here.
## 3. Two traps when debugging this hardware
* **Without `HSA_OVERRIDE_GFX_VERSION` set, GPU ops segfault** rather than degrading —
`torch.randn(256, 256, device="cuda") @ itself` dumps core. The device still
enumerates and `torch.cuda.is_available()` still returns `True`, so nothing warns
first.
* On the `HSA_OVERRIDE` stacks, **`.item()` can return the previous reduction's value**:
after `x.sum().item()`, a following `x.max().item()` on the same tensor returns the
sum. `.cpu().item()` and `float()` are safe. This silently fabricates plausible-looking
numbers and is easy to mistake for a compute bug. AMD's native gfx1033 build fixes this
one; the training failures above survive it.
## 4. Why the Vulkan routing covers AMD generally
The Linux Vulkan branch accepts any AMD GPU with no usable ROCm, not just this part.
ROCm supports a short list of AMD hardware — APUs least of all — and many distros ship
Mesa/RADV without ROCm ever being installed, so that population is large and was getting
a CPU-only binary on a GPU host.
Widening it is safe because **the Vulkan bundle is a superset of the CPU bundle**: it
ships the same `libggml-cpu-*.so` variants alongside `libggml-vulkan.so`. With no usable
Vulkan device it enumerates zero devices and runs on the CPU backend — verified by
running `--list-devices` with the Vulkan ICD removed, which lists nothing and exits 0. So
a host that turns out to have no working Vulkan lands exactly where it does today, and
one that does gets its GPU. The numbers above are what justify preferring Vulkan when it
is available.
The probe only runs when there is no usable NVIDIA and no ROCm, so a host where ROCm
works keeps the ROCm branches; the flag cannot divert it. Such a host also resolves to the
`cpu` torch index — `get_torch_index_url` falls back to CPU when an AMD GPU is present
with no ROCm — so it ends up with the CPU training stack plus Vulkan inference, which is
the combination verified end to end on this hardware.

71
studio/install_llama_prebuilt.py
View file

@ -373,6 +373,9 @@ class HostInfo:
has_usable_nvidia: bool
has_rocm: bool = False
has_intel_gpu: bool = False
# AMD GPU present but ROCm unusable. Linux only, probed only when there is no usable
# NVIDIA and no ROCm, so the ROCm branches still own every host where ROCm works.
has_amd_gpu_without_rocm: bool = False
rocm_gfx_target: str | None = None
rocm_gfx_targets: list[str] = field(default_factory = list)
# (major, minor) from platform.mac_ver(); None off macOS or if unparseable.
@ -1127,10 +1130,11 @@ def direct_upstream_release_plan(
# ROCm hosts are excluded: this ggml-org path ships no per-gfx ROCm
# asset, so they fall through to the empty-attempts raise (HIP source
# build) rather than silently getting a CPU binary on a GPU host.
# Intel (or other non-NVIDIA/non-AMD) GPU: use the Vulkan prebuilt. The
# elif already excludes usable NVIDIA and ROCm; also require no PHYSICAL
# NVIDIA so a CUDA-hidden card isn't reached through Vulkan (CPU below).
if host.has_intel_gpu and not host.has_physical_nvidia:
# Intel or AMD GPU with no usable ROCm: use the Vulkan prebuilt. No PHYSICAL NVIDIA,
# so a CUDA-hidden card is not reached through Vulkan. Widening from Intel-only is
# safe: the Vulkan bundle is a superset of the CPU one and falls back to CPU when no
# Vulkan device enumerates. Steam Deck (gfx1033, Qwen2.5-0.5B Q4_0): 112.8 vs 49.8 tok/s.
if (host.has_intel_gpu or host.has_amd_gpu_without_rocm) and not host.has_physical_nvidia:
vulkan_asset = f"llama-{release_tag}-bin-ubuntu-vulkan-x64.tar.gz"
vulkan_url = assets.get(vulkan_asset)
if vulkan_url:
@ -1163,9 +1167,9 @@ def direct_upstream_release_plan(
# selector returned 0 attempts and the installer fell back to a
# source build on every Linux ARM64 host (DGX Spark, Ampere
# Altra, GitHub-hosted ubuntu-24.04-arm runners, etc.).
# Intel (or other non-NVIDIA/non-AMD) GPU: prefer the Vulkan prebuilt,
# mirroring the x86_64 branch. Upstream ships bin-ubuntu-vulkan-arm64.
# No physical NVIDIA: don't reach a CUDA-hidden card through Vulkan.
# Intel GPU: prefer the Vulkan prebuilt (bin-ubuntu-vulkan-arm64). No physical NVIDIA,
# so a CUDA-hidden card is not reached through Vulkan. Deliberately NOT
# has_amd_gpu_without_rocm as x86_64 does: that widening was only measured there.
if host.has_intel_gpu and not host.has_physical_nvidia and not host.has_rocm:
vulkan_asset = f"llama-{release_tag}-bin-ubuntu-vulkan-arm64.tar.gz"
vulkan_url = assets.get(vulkan_asset)
@ -2739,22 +2743,34 @@ def detect_host(*, probe_rocm_with_nvidia: bool = False) -> HostInfo:
# Note: amdhip64.dll presence alone is NOT treated as GPU evidence
# since the HIP SDK can be installed without an AMD GPU.
# Detect an Intel GPU; gates the Vulkan prebuilt. Linux reads the DRM sysfs
# vendor id (0x8086); Windows reads the display-adapter registry class,
# then falls back to the WMI video controller list. Only probed with no
# usable NVIDIA and no ROCm (matching the Vulkan branches), keeping the
# probe (notably the Windows powershell call) off that path.
# Detect an Intel or AMD GPU; gates the Vulkan prebuilt. Linux reads DRM sysfs vendor ids
# (0x8086 Intel, 0x1002 AMD); Windows reads the display-adapter registry, then WMI. Only
# probed with no usable NVIDIA and no ROCm, matching the Vulkan branches, so
# "without_rocm" holds by construction.
has_intel_gpu = False
has_amd_gpu_without_rocm = False
if not has_usable_nvidia and not has_rocm:
# ROCR_VISIBLE_DEVICES set means the caller asked for GPU isolation, and Vulkan honours
# no HIP mask (it selects via GGML_VK_VISIBLE_DEVICES), so auto-routing would hand
# llama.cpp the GPU that request hid. ROCR alone, not the three
# _hip_visible_device_mask_set() reads: it filters the HSA agent list, while
# HIP_VISIBLE_DEVICES and its CUDA_VISIBLE_DEVICES alias filter HIP's device list (AMD
# GPU-isolation docs), so counting those would cost Vulkan to any host merely exporting
# CUDA_VISIBLE_DEVICES. Windows reads all three: its probe is hipinfo, a HIP
# application. Intel is unaffected by HIP masks.
_amd_hidden_by_mask = os.environ.get("ROCR_VISIBLE_DEVICES") is not None
if is_linux:
# No early break: a laptop can pair an Intel iGPU with an AMD dGPU.
for _vendor_file in glob.glob("/sys/class/drm/card*/device/vendor"):
try:
with open(_vendor_file, encoding = "utf-8") as _vf:
if _vf.read().strip().lower() == "0x8086":
has_intel_gpu = True
break
_vendor_id = _vf.read().strip().lower()
except (OSError, UnicodeDecodeError):
continue
if _vendor_id == "0x8086":
has_intel_gpu = True
elif _vendor_id == "0x1002" and not _amd_hidden_by_mask:
has_amd_gpu_without_rocm = True
elif is_windows:
# Registry first (in-process; see windows_intel_gpu_in_registry).
# The CIM query stays as the fallback when the registry shows no
@ -2795,6 +2811,7 @@ def detect_host(*, probe_rocm_with_nvidia: bool = False) -> HostInfo:
has_usable_nvidia = has_usable_nvidia,
has_rocm = has_rocm,
has_intel_gpu = has_intel_gpu,
has_amd_gpu_without_rocm = has_amd_gpu_without_rocm,
rocm_gfx_target = rocm_gfx_target,
rocm_gfx_targets = rocm_gfx_targets,
macos_version = macos_version,
@ -2834,6 +2851,7 @@ def _apply_host_overrides(
rocm_gfx_target = None,
rocm_gfx_targets = [],
has_intel_gpu = False,
has_amd_gpu_without_rocm = False,
)
gfx = _normalize_forwarded_gfx(override_rocm_gfx)
if gfx:
@ -3551,13 +3569,19 @@ def resolve_upstream_asset_choice(host: HostInfo, llama_tag: str) -> AssetChoice
"falling back to source build with HIP support"
)
# Intel (or other non-NVIDIA/non-AMD) GPU: use the Vulkan prebuilt. No
# physical NVIDIA (not just no usable one): a CUDA-hidden card must not
# be reached through Vulkan, which ignores CUDA_VISIBLE_DEVICES.
if host.has_intel_gpu and not host.has_physical_nvidia and not host.has_rocm:
# Intel or AMD GPU with no usable ROCm: the Vulkan prebuilt. No PHYSICAL NVIDIA, since
# Vulkan ignores CUDA_VISIBLE_DEVICES. The ROCm branch above still wins wherever ROCm
# runs; this only rescues hosts falling through to CPU.
if (
(host.has_intel_gpu or host.has_amd_gpu_without_rocm)
and not host.has_physical_nvidia
and not host.has_rocm
):
# Name the vendor actually found: this branch now also covers AMD.
vendor = "Intel GPU" if host.has_intel_gpu else "AMD GPU without usable ROCm"
vulkan_name = f"llama-{llama_tag}-bin-ubuntu-vulkan-x64.tar.gz"
if vulkan_name in upstream_assets:
log(f"Intel GPU detected -- using upstream Vulkan prebuilt {vulkan_name}")
log(f"{vendor} detected -- using upstream Vulkan prebuilt {vulkan_name}")
return AssetChoice(
repo = UPSTREAM_REPO,
tag = llama_tag,
@ -3566,7 +3590,7 @@ def resolve_upstream_asset_choice(host: HostInfo, llama_tag: str) -> AssetChoice
source_label = "upstream",
install_kind = "linux-vulkan",
)
log("Intel GPU detected but no Vulkan prebuilt found -- falling back to CPU")
log(f"{vendor} detected but no Vulkan prebuilt found -- falling back to CPU")
upstream_name = f"llama-{llama_tag}-bin-ubuntu-x64.tar.gz"
if upstream_name not in upstream_assets:
@ -6393,7 +6417,10 @@ def _linux_published_attempts(host: HostInfo, bundle: PublishedReleaseBundle) ->
if published_rocm is not None:
attempts.append(published_rocm)
else:
if host.has_intel_gpu and not host.has_physical_nvidia:
# Same condition as direct_upstream_release_plan and resolve_upstream_asset_choice,
# which all three must state; this is the normal published-bundle branch. Reaching here
# already means not has_rocm (it is the else).
if (host.has_intel_gpu or host.has_amd_gpu_without_rocm) and not host.has_physical_nvidia:
vulkan_choice = published_asset_choice_for_kind(bundle, "linux-vulkan", host = host)
if vulkan_choice is not None:
attempts.append(vulkan_choice)

130
studio/install_python_stack.py
View file

@ -565,6 +565,10 @@ _GFX_TO_AMD_INDEX_ARCH: dict[str, str] = {
"gfx908": "gfx908", # MI200/MI100
}
# Archs that install ROCm wheels fine and then compute WRONG answers, so they stay on CPU
# torch rather than route to their family index. See install.sh's matching gate.
_ROCM_MISCOMPUTING_GFX: "frozenset[str]" = frozenset({"gfx1033"}) # Van Gogh (Steam Deck)
# bitsandbytes continuous-release_main wheels with the ROCm 4-bit GEMV fix
# (bnb #1887, post-0.49.2). bnb <= 0.49.2 NaNs at decode shape on every AMD GPU;
# PyPI 0.50.0 is the first release with the fix, so the fallback below is safe.
@ -1462,6 +1466,11 @@ def _amd_arch_index_url(gfx_arch: str | None) -> str | None:
"""
if IS_WINDOWS:
return _windows_rocm_index_url(gfx_arch)
# gfx1033 (Van Gogh) miscomputes under ROCm (studio/ROCM_RDNA2_APU.md). Without this,
# the inferred-gfx repair below would force-reinstall the very ROCm wheels install.sh's
# gate exists to avoid. Linux only, matching where it was measured.
if (gfx_arch or "").lower() in _ROCM_MISCOMPUTING_GFX:
return None
arch_family = _GFX_TO_AMD_INDEX_ARCH.get(gfx_arch or "")
if arch_family is None:
return None
@ -1469,6 +1478,85 @@ def _amd_arch_index_url(gfx_arch: str | None) -> str | None:
return _index_url_join(base, arch_family)
def _physical_amd_gfx_archs() -> "list[str]":
"""The AMD arches on this Linux host, read from sources an override cannot move.
Strongest first: ROCm userland probes with HSA_OVERRIDE_GFX_VERSION and the visible-device
masks stripped, then KFD topology sysfs, then product-name inference, then the declared
UNSLOTH_ROCM_GFX_ARCH. Declared is LAST because it is a routing hint for a host whose
probes cannot answer, not a statement about silicon, and taking it first let a stale
gfx1030 on a real Van Gogh hide the arch. KFD precedes the inference for the same reason:
_infer_linux_amd_gfx_arch() returns the declared value first, so behind it the kernel
never answers.
"""
_archs = [
_code.strip().lower().split(":")[0]
for _code in _detect_amd_gfx_codes(ignore_hsa_override = True, ignore_visible_masks = True)
]
if not _archs:
_archs = [_code.strip().lower().split(":")[0] for _code in _kfd_gfx_targets()]
if not _archs:
_inferred = (_infer_linux_amd_gfx_arch() or "").strip().lower().split(":")[0]
_archs = [_inferred] if _inferred else []
if not _archs:
_env_gfx = (os.environ.get("UNSLOTH_ROCM_GFX_ARCH") or "").strip().lower().split(":")[0]
_archs = [_env_gfx] if _env_gfx else []
return _archs
def _miscomputing_arch_host() -> bool:
"""True when EVERY AMD arch this host physically has computes incorrectly under ROCm.
Every, not any: gfx1033 is one of the integrated parts _SHADOWING_INTEGRATED_GFX lists, so
it can lead the enumeration on a box whose real accelerator is a discrete Radeon (#7776),
and declining ROCm on presence alone would strand that card. install.sh's gate is a
presence test only because it cannot resolve which device the runtime picks; here the arch
list IS the host.
Shared with _rocm_miscomputing_host(), which adds only "and ROCm torch is already
installed": same question about the hardware, withholding wheels rather than replacing them.
"""
if IS_WINDOWS or IS_MACOS:
return False
_archs = _physical_amd_gfx_archs()
return bool(_archs) and all(_arch in _ROCM_MISCOMPUTING_GFX for _arch in _archs)
def _rocm_miscomputing_host() -> bool:
"""True when every AMD GPU on this Linux host is an arch measured to compute
incorrectly under ROCm, ROCm torch is already installed, and no explicit index pin
overrides that finding.
Returning None from _amd_arch_index_url() only stops such a host from being GIVEN ROCm
wheels. A venv that already HOLDS them was never demoted: install.sh resolves
UNSLOTH_TORCH_BACKEND=cpu, so _ensure_rocm_torch() returns at its first line,
_ensure_cpu_torch() fires only for an EXPLICIT pin, and the base update does not reinstall
an already-satisfied torch. Upgrading therefore left exactly the build the gate exists to
remove. Treat the arch itself as CPU authority and let _ensure_cpu_torch() demote.
EVERY arch, not any: a healthy dGPU beside a miscomputing APU is still served by ROCm. The
disk label is read first so the ROCm probes cost nothing on the vast majority of hosts. An
explicit UNSLOTH_TORCH_INDEX_URL / _FAMILY stays the escape hatch and wins.
KFD topology sysfs comes after the runtime probes but BEFORE the product-name inference and
the declared arch: a Van Gogh host can reach here with neither answering, since
_detect_amd_gfx_codes() needs rocminfo or amd-smi (absent once ROCm is uninstalled, or with
the user outside the render group) and _infer_linux_amd_gfx_arch() maps no Van Gogh product
name. _archs then came back empty and the host kept the NaN-producing wheels. amdkfd is in
the kernel driver, and is the source _hsa_probe_correction() already trusts over the runtime.
"""
if IS_WINDOWS or IS_MACOS:
return False
if _explicit_torch_index_url() is not None:
return False
if "+rocm" not in _installed_torch_label_on_disk():
return False
# Declared arch LAST: this asks what silicon is PRESENT, and a stale
# UNSLOTH_ROCM_GFX_ARCH=gfx1030 on a real Van Gogh answered with a healthy arch. It still
# answers when no probe can. install.sh's "physical" mode agrees.
return _miscomputing_arch_host()
def _windows_rocm_index_url(gfx_arch: str | None) -> str | None:
"""Return the AMD pip index URL for the given GPU arch, or None if unsupported."""
arch_family = _GFX_TO_AMD_INDEX_ARCH.get(gfx_arch or "")
@ -3378,15 +3466,20 @@ def _is_gpu_torch_label(label: str) -> bool:
def _ensure_cpu_torch() -> None:
"""Reinstall CPU torch when an explicit CPU pin is set but the venv has a GPU build.
"""Reinstall CPU torch when CPU is authoritative but the venv has a GPU build.
Counterpart to _ensure_cuda/rocm_torch for the explicit-CPU case (those treat a CPU
backend as a skip, so a standalone `studio update` would ignore the authoritative CPU
pin). Only fires for an EXPLICIT pin.
Counterpart to _ensure_cuda/rocm_torch for the CPU case: those treat a CPU backend as a
skip, so a standalone `studio update` would ignore the authoritative CPU choice. Authority
is an EXPLICIT pin, or an AMD arch measured to compute incorrectly under ROCm (see
_rocm_miscomputing_host for why that one must demote rather than just decline).
"""
if NO_TORCH:
return
pin = _explicit_cpu_torch_index_url()
_reason = "an explicit CPU index is pinned"
if pin is None and _rocm_miscomputing_host():
pin = f"{_PYTORCH_WHL_BASE}/cpu"
_reason = "this AMD arch computes incorrectly under ROCm (studio/ROCM_RDNA2_APU.md)"
if pin is None:
return
@ -3405,7 +3498,7 @@ def _ensure_cpu_torch() -> None:
# reinstall from the pin (self-resolving, no loop).
_torch_pkg, _vision_pkg, _audio_pkg = _CPU_TORCH_PKG_SPEC
_safe_print(
f" torch cannot import but an explicit CPU index is pinned -- reinstalling "
f" torch cannot import and {_reason} -- reinstalling "
f"CPU torch from {_strip_index_url_credentials(pin)}"
)
pip_install(
@ -3441,7 +3534,7 @@ def _ensure_cpu_torch() -> None:
return # already a CPU build
_safe_print(
" torch is a GPU build but an explicit CPU index is pinned -- reinstalling "
f" torch is a GPU build but {_reason} -- reinstalling "
f"CPU torch from {_strip_index_url_credentials(pin)}"
)
_torch_pkg, _vision_pkg, _audio_pkg = _CPU_TORCH_PKG_SPEC
@ -4382,6 +4475,17 @@ def _ensure_rocm_torch() -> None:
return
# An explicit ROCm pin commits to ROCm wheels whatever the visible GPU (headless / CI).
_rocm_pin = _explicit_rocm_torch_index_url()
# Before ANY install path, including the inferred-arch one below: that takes a declared
# UNSLOTH_ROCM_GFX_ARCH first, so a stale gfx1030 on a real Van Gogh force-installed the
# multi-GB gfx103X-all stack, skipped the runtime-target check, then had
# _ensure_cpu_torch() undo it: a ROCm-to-CPU cycle on every update. An explicit index
# pin still wins.
if _rocm_pin is None and not IS_WINDOWS and _miscomputing_arch_host():
_safe_print(
" This host has an AMD arch measured to compute incorrectly under ROCm "
"(studio/ROCM_RDNA2_APU.md) -- keeping CPU torch.\n"
)
return
_inferred_linux_gfx = (
_infer_linux_amd_gfx_arch() if (_rocm_pin is None and not IS_WINDOWS) else None
)
@ -4518,6 +4622,20 @@ def _ensure_rocm_torch() -> None:
_runtime_gfx, gfx_codes, _physical_gfx, _host_codes = _runtime_gfx_target(
_inferred_linux_gfx
)
# A miscomputing target has no ROCm route, so the rest of this function is skipped:
# _amd_arch_index_url returns None for such an arch and the reroute below would either
# raise on it or fall through to generic pytorch.org wheels. A standalone
# `studio update` reads no backend and reaches here with nothing else to stop it.
# Keyed on the SELECTED target, which install.sh's presence gate cannot see.
if _runtime_gfx in _ROCM_MISCOMPUTING_GFX:
# Declines to INSTALL ROCm for this target only; removing an existing ROCm build
# is _ensure_cpu_torch's call, and that asks about the whole host with masks
# stripped (test_a_mask_cannot_shrink_the_host_to_its_bad_gpu).
_safe_print(
f" {_runtime_gfx} computes incorrect results under ROCm "
f"(studio/ROCM_RDNA2_APU.md) -- not installing ROCm torch for it.\n"
)
return
_strix_gfx = {"gfx1151", "gfx1150", "gfx1152"}
# Only the Strix reroute has a ROCm-version floor.
_detected_strix = (

42
studio/setup.sh
View file

@ -941,6 +941,37 @@ fi
STAGE_ROOT="${UNSLOTH_STUDIO_STAGE_ROOT:-}"
RUNTIME_ROOT="${STAGE_ROOT:-$STUDIO_HOME}"
VENV_DIR="$RUNTIME_ROOT/unsloth_studio"
# Same uv cache install.sh chose, for the same reasons -- kept byte-identical to the
# block there, including the write probe and the unwind on failure.
#
# This script is also the standalone entry point: `unsloth studio update` runs it
# directly, without install.sh, so an export made only there covers the first install and
# nothing after it. A redirected STUDIO_HOME would then download a SECOND cache to
# $HOME/.cache/uv on the very first update and copy every wheel across the filesystem
# boundary, which is exactly the disk cost the co-location exists to avoid -- deferred by
# one run rather than fixed. Pointing at the same path also means the update reuses the
# cache the install filled instead of refetching it.
#
# STUDIO_HOME, not RUNTIME_ROOT: the cache has to be the one install.sh created, and the
# two agree whenever UNSLOTH_STUDIO_STAGE_ROOT is unset, which is every non-staged run.
if [ -z "${UV_CACHE_DIR:-}" ]; then
UV_CACHE_DIR="$STUDIO_HOME/cache/uv"
export UV_CACHE_DIR
# mktemp, not a $$-derived name: this branch exists for a cache directory another
# account can write, and there a predictable path can be pre-created as a symlink,
# which `: >` would follow and truncate -- as root, any file on the box. mktemp
# creates O_EXCL with an unpredictable suffix, so it cannot follow one, and failing
# to create IS the writability answer this probe wanted.
_uv_cache_probe=""
if ! mkdir -p "$UV_CACHE_DIR" 2>/dev/null \
|| ! _uv_cache_probe=$(mktemp "$UV_CACHE_DIR/.unsloth-write-probe.XXXXXX" 2>/dev/null); then
echo "[WARN] Cannot write to $UV_CACHE_DIR -- using uv's default cache." >&2
unset UV_CACHE_DIR
fi
[ -z "$_uv_cache_probe" ] || rm -f "$_uv_cache_probe" 2>/dev/null || true
unset _uv_cache_probe
fi
VENV_T5_530_DIR="$RUNTIME_ROOT/.venv_t5_530"
VENV_T5_550_DIR="$RUNTIME_ROOT/.venv_t5_550"
VENV_T5_510_DIR="$RUNTIME_ROOT/.venv_t5_510"
@ -2457,8 +2488,17 @@ EOF
else
step "gpu" "AMD ROCm"
fi
# Only claim a path that is really there, matching install.sh. /opt/rocm is a
# FALLBACK, not a detection, so a runtime-only ROCm host reaches here with nothing at
# it -- and this script runs after install.sh on every install and alone on every
# `unsloth studio update`, so leaving it unconditional here meant the run still ended
# by advertising the directory install.sh had just stopped claiming.
_setup_rocm_root="${ROCM_PATH:-${HIP_PATH:-/opt/rocm}}"
substep "ROCm: $_setup_rocm_root"
if [ -d "$_setup_rocm_root" ]; then
substep "ROCm: $_setup_rocm_root"
else
substep "ROCm: runtime detected (no SDK tree at $_setup_rocm_root)"
fi
[ -n "$_setup_rocm_ver" ] && substep "hipconfig: $_setup_rocm_ver"
[ -n "$_setup_mkt" ] && [ -n "$_setup_gfx" ] && substep "GPU: $_setup_mkt"
elif [ "$_setup_xpu_ready" = true ]; then

498
tests/sh/test_rocm_bad_arch_gate.sh Executable file
View file

@ -0,0 +1,498 @@
#!/bin/bash
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
#
# Guards the "arch computes incorrectly under ROCm" gate in get_torch_index_url:
# gfx1033 routes to the cpu index, every other arch is untouched. Keying off the ROCm
# VERSION alone routed a Deck to wheels that install and compute wrong answers
# (studio/ROCM_RDNA2_APU.md). Not AMD's support table: unsloth serves gfx906 and
# gfx1031-gfx1036 deliberately (#7277).
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
INSTALL_SH="$SCRIPT_DIR/../../install.sh"
PASS=0
FAIL=0
assert_eq() {
_label="$1"; _expected="$2"; _actual="$3"
if [ "$_actual" = "$_expected" ]; then
echo " PASS: $_label"
PASS=$((PASS + 1))
else
echo " FAIL: $_label (expected '$_expected', got '$_actual')"
FAIL=$((FAIL + 1))
fi
}
# The gate is inline in get_torch_index_url on purpose: harnesses that extract the function
# alone would turn a missed helper into an undefined command sending every ROCm case to cpu.
# So exercise the real case block, lifted from the function.
_FN_FILE=$(mktemp)
trap 'rm -f "$_FN_FILE"' EXIT
# An explicit end marker, not the closing `fi`: a second `if` at the same indent would
# truncate a structural extract.
awk '/# Archs measured to compute INCORRECTLY under ROCm/,/^ # end of the miscomputing-arch gate/' \
"$INSTALL_SH" > "$_FN_FILE"
if ! grep -q 'gfx1033' "$_FN_FILE"; then
echo "FAIL: could not extract the gfx gate from get_torch_index_url in install.sh"
exit 1
fi
_SH="${BASH:-/bin/bash}"
# A real function around the extracted block, not a dot-script: `return` in a sourced file
# unwinds the source rather than the caller, so sourcing would print both lines.
_GATE_FILE=$(mktemp)
{
echo '_gate() {'
cat "$_FN_FILE"
echo ' echo rocm'
echo '}'
} > "$_GATE_FILE"
trap 'rm -f "$_FN_FILE" "$_GATE_FILE"' EXIT
_SH="${BASH:-/bin/bash}"
_route() { # gfx -> the cpu index url when the gate intercepts, else "rocm"
"$_SH" -c "
_base=https://download.pytorch.org/whl
_amd_gfx_probe='$1'
. '$_GATE_FILE'
_gate
" 2>/dev/null | tail -1
}
echo "=== The measured-bad arch routes to CPU ==="
assert_eq "gfx1033 -> cpu" "$(printf 'https://download.pytorch.org/whl/cpu')" "$(_route gfx1033)"
assert_eq "gfx1033 with feature suffix" "$(printf 'https://download.pytorch.org/whl/cpu')" "$(_route 'gfx1033:xnack-')"
assert_eq "GFX1033 uppercase" "$(printf 'https://download.pytorch.org/whl/cpu')" "$(_route GFX1033)"
echo "=== Everything else falls through to the ROCm path, unchanged ==="
# gfx906 and gfx1031-1036 are supported beyond AMD's table; swallowing them here would
# silently drop verified support.
for _gfx in gfx906 gfx1030 gfx1031 gfx1032 gfx1034 gfx1035 gfx1036 \
gfx908 gfx90a gfx942 gfx950 gfx1100 gfx1101 gfx1102 gfx1103 \
gfx1150 gfx1151 gfx1152 gfx1153 gfx1200 gfx1201; do
assert_eq "$_gfx not intercepted" "rocm" "$(_route "$_gfx")"
done
assert_eq "empty probe not intercepted" "rocm" "$(_route '')"
assert_eq "garbage not intercepted" "rocm" "$(_route 'not-a-gfx')"
assert_eq "gfx10330 is not gfx1033" "rocm" "$(_route gfx10330)"
echo "=== A mixed host takes the cpu index too: presence, not selection ==="
# PRESENCE only: a healthy dGPU beside the APU no longer keeps the host on ROCm. Resolving
# which device the runtime selects needs the per-device identity and mask layering that
# _runtime_gfx_target() has and a token list does not.
assert_eq "gfx1033 + gfx1100 -> cpu" "https://download.pytorch.org/whl/cpu" "$(_route 'gfx1033
gfx1100')"
assert_eq "gfx1100 + gfx1033 -> cpu" "https://download.pytorch.org/whl/cpu" "$(_route 'gfx1100
gfx1033')"
# rocminfo names each agent twice, so the real single-GPU Deck probe repeats the token.
assert_eq "repeated gfx1033 -> cpu" \
"https://download.pytorch.org/whl/cpu" "$(_route 'gfx1033
gfx1033')"
assert_eq "mixed case gfx1033 -> cpu" \
"https://download.pytorch.org/whl/cpu" "$(_route 'gfx1033:xnack-
GFX1033')"
echo "=== The runtime-less reroute honours the same gate ==="
# The gate alone is not enough: the reroute below it reads UNSLOTH_ROCM_GFX_ARCH as the
# inferred arch and rewrites */cpu to the family index, undoing it. The escape hatch is
# UNSLOTH_TORCH_INDEX_URL, which returns long before either.
_REROUTE_FILE=$(mktemp)
trap 'rm -f "$_FN_FILE" "$_GATE_FILE" "$_REROUTE_FILE"' EXIT
{
awk '/^_amd_arch_index_family_for_gfx\(\)/,/^}/' "$INSTALL_SH"
awk '/^_amd_probe_arches\(\)/,/^}/' "$INSTALL_SH"
awk '/^_amd_sole_index_arch\(\)/,/^}/' "$INSTALL_SH"
awk '/^_infer_linux_amd_gfx_arch\(\)/,/^}/' "$INSTALL_SH"
echo '_reroute() {'
awk '/_linux_inferred_gfx=\$\(_infer_linux_amd_gfx_arch/,/^ if \[ -n "\$_amd_family" \]; then$/' "$INSTALL_SH"
echo ' echo "$_amd_family"'
echo ' else echo cpu' # gated out: no arch survives, so no reroute happens
echo ' fi'
echo '}'
} > "$_REROUTE_FILE"
_reroute_family() { # UNSLOTH_ROCM_GFX_ARCH -> family index, or cpu when gated out
"$_SH" -c "
UNSLOTH_ROCM_GFX_ARCH='$1'; export UNSLOTH_ROCM_GFX_ARCH
. '$_REROUTE_FILE'
_reroute
" 2>/dev/null | tail -1
}
assert_eq "gfx1033 override does not reroute" "cpu" "$(_reroute_family gfx1033)"
assert_eq "GFX1033 override does not reroute" "cpu" "$(_reroute_family GFX1033)"
assert_eq "gfx1032 override still reroutes" "gfx103X-all" "$(_reroute_family gfx1032)"
assert_eq "gfx1030 override still reroutes" "gfx103X-all" "$(_reroute_family gfx1030)"
assert_eq "gfx1151 override still reroutes" "gfx1151" "$(_reroute_family gfx1151)"
echo "=== The rejected override is not forwarded to llama.cpp either ==="
# Leaving UNSLOTH_ROCM_GFX_ARCH exported splits the host in two: setup.sh forwards it as
# --rocm-gfx and _apply_host_overrides reads any forwarded gfx as proof of ROCm, skipping the
# Vulkan branch (112.8 vs 49.8 tok/s). A rejected arch leaves nothing behind; the rest carry on.
_forwarded_gfx() { # UNSLOTH_ROCM_GFX_ARCH -> what survives for setup.sh, or <unset>
"$_SH" -c "
UNSLOTH_ROCM_GFX_ARCH='$1'; export UNSLOTH_ROCM_GFX_ARCH
. '$_REROUTE_FILE'
_reroute >/dev/null 2>&1
printf '%s' \"\${UNSLOTH_ROCM_GFX_ARCH:-<unset>}\"
" 2>/dev/null | tail -1
}
assert_eq "gfx1033 override not forwarded" "<unset>" "$(_forwarded_gfx gfx1033)"
assert_eq "GFX1033 override not forwarded" "<unset>" "$(_forwarded_gfx GFX1033)"
assert_eq "gfx1030 override still forwarded" "gfx1030" "$(_forwarded_gfx gfx1030)"
assert_eq "gfx1151 override still forwarded" "gfx1151" "$(_forwarded_gfx gfx1151)"
echo "=== End to end: the REAL get_torch_index_url against a REAL rocminfo shape ==="
# Everything above feeds a hand-built one-token probe. The real one names each GPU agent
# TWICE ("Name:" and ISA Info), so a single-GPU Deck yields "gfx1033\ngfx1033" and a
# whole-string compare matches neither. Drive the real function to catch that.
_E2E_DIR=$(mktemp -d)
_E2E_FUNCS="$_E2E_DIR/funcs.sh"
_FAKE_SMI_DIR=$(mktemp -d)
_FAKE_ROCM_DIR=$(mktemp -d)
_TOOLS_DIR=$(mktemp -d)
trap 'rm -rf "$_FN_FILE" "$_GATE_FILE" "$_REROUTE_FILE" "$_E2E_DIR" "$_FAKE_SMI_DIR" "$_FAKE_ROCM_DIR" "$_TOOLS_DIR"' EXIT
# Same extraction contract as tests/sh/test_get_torch_index_url.sh: a missed helper makes the
# ROCm branch answer cpu and these pass for the wrong reason. The ROCm assertion below guards it.
{
for _fn in _run_bounded _cvd_hides_nvidia _has_amd_rocm_gpu _has_usable_nvidia_gpu \
_ensure_rocm_probe_env _probe_amd_gfx_arch _amd_gfx_select_ordinals \
_amd_gpu_present_via_pci \
_infer_amd_gfx_arch_from_gpu_name _infer_linux_amd_gfx_arch \
_amd_arch_index_family_for_gfx _trim_index_path_slashes \
_nvidia_cu126_verdict _cap_cuda_family_for_pre_turing \
_rocm_tag_from_amd_smi _rocm_tag_from_version_file _rocm_tag_from_hipconfig \
_rocm_tag_from_dpkg _rocm_tag_from_rpm _highest_rocm_tag \
_detect_rocm_version_tag _kfd_gfx_targets get_torch_index_url; do
sed -n "/^$_fn()/,/^}/p" "$INSTALL_SH"
echo ""
done
} | sed -e "s|/usr/bin/nvidia-smi|$_FAKE_SMI_DIR/nvidia-smi-absent|g" \
-e "s|/opt/rocm|$_FAKE_ROCM_DIR|g" > "$_E2E_FUNCS"
for _cmd in uname grep sed head sh bash cat awk printf tr cut sort timeout; do
_real=$(command -v "$_cmd" 2>/dev/null || true)
[ -n "$_real" ] && ln -sf "$_real" "$_TOOLS_DIR/$_cmd"
done
_make_rocminfo_host() { # $1 = gfx arch -> a dir holding rocminfo + hipconfig mocks
_mk_dir=$(mktemp -d)
# Real single-GPU APU shape: a CPU agent with no ISA, then a GPU agent whose arch appears
# in both its Name and its ISA Info Name.
cat > "$_mk_dir/rocminfo" <<ROCMINFO
#!/bin/sh
cat <<'OUT'
=====================
HSA System Attributes
=====================
Runtime Version: 1.1
==========
HSA Agents
==========
*******
Agent 1
*******
Name: AMD Custom APU 0405
Uuid: CPU-XX
Marketing Name: AMD Custom APU 0405
Device Type: CPU
ISA Info:
N/A
*******
Agent 2
*******
Name: $1
Uuid: GPU-XX
Marketing Name: AMD Custom GPU 0405
Device Type: GPU
ISA Info:
ISA 1
Name: amdgcn-amd-amdhsa--$1
Machine Models: HSA_MACHINE_MODEL_LARGE
*** Done ***
OUT
ROCMINFO
# A readable ROCm 7.2 userspace, so a host clearing the gate reaches the version-keyed
# index rather than the "no ROCm version" cpu fallback.
printf '#!/bin/sh\necho 7.2.0\n' > "$_mk_dir/hipconfig"
chmod +x "$_mk_dir/rocminfo" "$_mk_dir/hipconfig"
printf '%s' "$_mk_dir"
}
_index_for_rocminfo_host() { # $1 = gfx arch -> the index get_torch_index_url picks
_ifh_dir=$(_make_rocminfo_host "$1")
PATH="$_ifh_dir:$_TOOLS_DIR" "$_SH" -c "
unset CUDA_VISIBLE_DEVICES UNSLOTH_ROCM_GFX_ARCH UNSLOTH_TORCH_INDEX_URL
unset UNSLOTH_TORCH_INDEX_FAMILY
_ARCH=x86_64
. '$_E2E_FUNCS'
get_torch_index_url
" 2>/dev/null | tail -1
rm -rf "$_ifh_dir"
}
# Assert the probe really is multi-line first, or a future single-hit probe makes the two
# assertions below vacuous.
_probe_lines=$( _pl_dir=$(_make_rocminfo_host gfx1033)
PATH="$_pl_dir:$_TOOLS_DIR" "$_SH" -c "
unset UNSLOTH_ROCM_GFX_ARCH; . '$_E2E_FUNCS'; _probe_amd_gfx_arch" 2>/dev/null \
| grep -c gfx1033
rm -rf "$_pl_dir" )
assert_eq "rocminfo yields more than one gfx token" "yes" \
"$([ "${_probe_lines:-0}" -gt 1 ] && echo yes || echo no)"
assert_eq "gfx1033 rocminfo host -> cpu index" \
"https://download.pytorch.org/whl/cpu" "$(_index_for_rocminfo_host gfx1033)"
# Negative arm: only the measured-bad arch is intercepted. gfx1030 is the family neighbour
# install.sh serves through gfx103X-all/ROCm.
assert_eq "gfx1030 rocminfo host -> rocm index" \
"https://download.pytorch.org/whl/rocm7.2" "$(_index_for_rocminfo_host gfx1030)"
# A REAL two-agent rocminfo shape: presence decides, and no mask can move it either way,
# because the gate never asks about selection.
_make_two_agent_host() {
_ta_apu=$(_make_rocminfo_host gfx1033)
_ta_dgpu=$(_make_rocminfo_host gfx1100)
_ta_dir=$(mktemp -d)
cp "$_ta_apu/hipconfig" "$_ta_dir/hipconfig"
printf '#!/bin/sh\n"%s/rocminfo"\n"%s/rocminfo"\n' "$_ta_apu" "$_ta_dgpu" > "$_ta_dir/rocminfo"
chmod +x "$_ta_dir/rocminfo"
printf '%s' "$_ta_dir"
}
_index_for_two_agent_host() { # $1 = extra "VAR=value" env, or empty
_ith_dir=$(_make_two_agent_host)
PATH="$_ith_dir:$_TOOLS_DIR" "$_SH" -c "
unset CUDA_VISIBLE_DEVICES UNSLOTH_ROCM_GFX_ARCH UNSLOTH_TORCH_INDEX_URL
unset UNSLOTH_TORCH_INDEX_FAMILY HSA_OVERRIDE_GFX_VERSION ROCR_VISIBLE_DEVICES
unset HIP_VISIBLE_DEVICES
[ -z '$1' ] || export $1
_ARCH=x86_64
. '$_E2E_FUNCS'
get_torch_index_url
" 2>/dev/null | tail -1
rm -rf "$_ith_dir"
}
assert_eq "two-agent host (APU + dGPU) -> cpu" \
"https://download.pytorch.org/whl/cpu" "$(_index_for_two_agent_host '')"
# Mask-independent by construction, which is what ended the ordinal/UUID/probe-precedence
# chase: nothing is left for a mask to distort.
assert_eq "two-agent host + ROCR=1 -> cpu (unchanged)" \
"https://download.pytorch.org/whl/cpu" "$(_index_for_two_agent_host ROCR_VISIBLE_DEVICES=1)"
assert_eq "two-agent host + HIP=1 -> cpu (unchanged)" \
"https://download.pytorch.org/whl/cpu" "$(_index_for_two_agent_host HIP_VISIBLE_DEVICES=1)"
assert_eq "two-agent host + UUID mask -> cpu (unchanged)" \
"https://download.pytorch.org/whl/cpu" \
"$(_index_for_two_agent_host ROCR_VISIBLE_DEVICES=GPU-DEADBEEFDEADBEEF)"
echo "=== HSA_OVERRIDE_GFX_VERSION=10.3.0, the circulated Van Gogh workaround ==="
# ROCr applies the override in USERLAND while building agent names, so with it set a real
# Deck's rocminfo answers gfx1030 and the gate saw no bad token on the one host it exists
# for. This mock is that: one host, two answers.
_make_spoofing_host() { # -> a dir whose rocminfo honours HSA_OVERRIDE_GFX_VERSION
_sp_real=$(_make_rocminfo_host gfx1033)
_sp_spoofed=$(_make_rocminfo_host gfx1030)
_sp_dir=$(mktemp -d)
cp "$_sp_real/hipconfig" "$_sp_dir/hipconfig"
cat > "$_sp_dir/rocminfo" <<SPOOF
#!/bin/sh
if [ -n "\${HSA_OVERRIDE_GFX_VERSION:-}" ]; then
exec "$_sp_spoofed/rocminfo"
fi
exec "$_sp_real/rocminfo"
SPOOF
chmod +x "$_sp_dir/rocminfo"
printf '%s' "$_sp_dir"
}
_index_for_spoofed_host() { # $1 = HSA_OVERRIDE_GFX_VERSION ("" to leave it unset)
_ish_dir=$(_make_spoofing_host)
PATH="$_ish_dir:$_TOOLS_DIR" "$_SH" -c "
unset CUDA_VISIBLE_DEVICES UNSLOTH_ROCM_GFX_ARCH UNSLOTH_TORCH_INDEX_URL
unset UNSLOTH_TORCH_INDEX_FAMILY
if [ -n '$1' ]; then HSA_OVERRIDE_GFX_VERSION='$1'; export HSA_OVERRIDE_GFX_VERSION
else unset HSA_OVERRIDE_GFX_VERSION; fi
_ARCH=x86_64
. '$_E2E_FUNCS'
get_torch_index_url
" 2>/dev/null | tail -1
rm -rf "$_ish_dir"
}
# Assert the mock really does spoof, or the gate assertion below passes with nothing hidden.
_spoofed_probe=$( _sp_check=$(_make_spoofing_host)
PATH="$_sp_check:$_TOOLS_DIR" "$_SH" -c "
unset UNSLOTH_ROCM_GFX_ARCH
HSA_OVERRIDE_GFX_VERSION=10.3.0; export HSA_OVERRIDE_GFX_VERSION
. '$_E2E_FUNCS'; _probe_amd_gfx_arch | head -1" 2>/dev/null
rm -rf "$_sp_check" )
assert_eq "the mock hides gfx1033 behind the override" "gfx1030" "$_spoofed_probe"
assert_eq "unspoofed Deck -> cpu index" \
"https://download.pytorch.org/whl/cpu" "$(_index_for_spoofed_host '')"
assert_eq "spoofed Deck -> cpu index anyway" \
"https://download.pytorch.org/whl/cpu" "$(_index_for_spoofed_host 10.3.0)"
# The re-probe is scoped to the gate, so a host that really is the reported arch keeps its
# ROCm index: only gfx1033 silicon answers gfx1033 unspoofed.
assert_eq "spoofed gfx1030 host keeps rocm" \
"https://download.pytorch.org/whl/rocm7.2" \
"$(export HSA_OVERRIDE_GFX_VERSION=10.3.0; _index_for_rocminfo_host gfx1030)"
echo "=== KFD answers when the probe cannot, so a spoof cannot fill the gap ==="
# The fallback chain ends at _amd_gfx_probe, collected WITH the override in force, so where
# ROCr cannot re-enumerate without it the fallback reports the spoofed arch. amdkfd is the
# kernel's own table and no runtime variable reaches it.
_make_kfd_only_host() { # rocminfo that answers ONLY while the override is set
_ko_dir=$(mktemp -d)
_ko_spoof=$(_make_rocminfo_host gfx1030)
cp "$_ko_spoof/hipconfig" "$_ko_dir/hipconfig"
cat > "$_ko_dir/rocminfo" <<KFDONLY
#!/bin/sh
if [ -n "\${HSA_OVERRIDE_GFX_VERSION:-}" ]; then
exec "$_ko_spoof/rocminfo"
fi
exit 1
KFDONLY
chmod +x "$_ko_dir/rocminfo"
printf '%s' "$_ko_dir"
}
_index_for_kfd_host() { # $1 = gfx the kernel reports ("" for a KFD that says nothing)
_ifk_dir=$(_make_kfd_only_host)
_ifk_stub=$(mktemp -d)
cat > "$_ifk_stub/kfd.sh" <<KSTUB
_kfd_gfx_targets() { [ -z '$1' ] || printf '%s\n' '$1'; }
KSTUB
PATH="$_ifk_dir:$_TOOLS_DIR" "$_SH" -c "
unset CUDA_VISIBLE_DEVICES UNSLOTH_ROCM_GFX_ARCH UNSLOTH_TORCH_INDEX_URL
unset UNSLOTH_TORCH_INDEX_FAMILY ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES
HSA_OVERRIDE_GFX_VERSION=10.3.0; export HSA_OVERRIDE_GFX_VERSION
_ARCH=x86_64
. '$_E2E_FUNCS'
. '$_ifk_stub/kfd.sh'
get_torch_index_url
" 2>/dev/null | tail -1
rm -rf "$_ifk_dir" "$_ifk_stub"
}
assert_eq "KFD names gfx1033 behind the spoof -> cpu" \
"https://download.pytorch.org/whl/cpu" "$(_index_for_kfd_host gfx1033)"
# KFD naming a healthy card is believed too: this is a source, not a veto.
assert_eq "KFD names gfx1030 -> rocm" \
"https://download.pytorch.org/whl/rocm7.2" "$(_index_for_kfd_host gfx1030)"
# Makes the SIMULATED host the subject, not the machine running the test. _has_amd_rocm_gpu
# and _kfd_gfx_targets read /dev/kfd and the KFD topology by absolute path with no seam, so on
# a real AMD box the runner's own silicon answered. Asserting presence and silencing KFD makes
# these read the same on a Deck, a gfx1151 runner and a laptop with no GPU.
_amd_host_no_kfd_stub() { # -> a file to source AFTER funcs.sh
_ahs_dir=$(mktemp -d)
{
printf '_has_amd_rocm_gpu() { return 0; }\n'
printf '_kfd_gfx_targets() { :; }\n'
} > "$_ahs_dir/host.sh"
printf '%s' "$_ahs_dir/host.sh"
}
echo "=== An override nothing can verify is not evidence of a healthy arch ==="
# Older ROCr answering only under the override, no amd-smi, no KFD: the chain used to end at
# the spoofed probe. Absence of evidence is not evidence of absence.
_index_unverifiable_override() { # $1 = the env assignment to apply
_iuo_dir=$(_make_kfd_only_host)
_iuo_stub=$(_amd_host_no_kfd_stub)
PATH="$_iuo_dir:$_TOOLS_DIR" "$_SH" -c "
unset CUDA_VISIBLE_DEVICES UNSLOTH_ROCM_GFX_ARCH UNSLOTH_TORCH_INDEX_URL
unset UNSLOTH_TORCH_INDEX_FAMILY ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES
unset HSA_OVERRIDE_GFX_VERSION
export $1
_ARCH=x86_64
. '$_E2E_FUNCS'
. '$_iuo_stub'
get_torch_index_url
" 2>/dev/null | tail -1
rm -rf "$_iuo_dir" "$(dirname "$_iuo_stub")"
}
assert_eq "HSA override with no verifiable source -> cpu" \
"https://download.pytorch.org/whl/cpu" \
"$(_index_unverifiable_override HSA_OVERRIDE_GFX_VERSION=10.3.0)"
# UNSLOTH_ROCM_GFX_ARCH is a DECLARED arch, not a spoof: it renames nothing, and a tool-blind
# host is the runtime-less #7301 population the reroute serves. Both reach the cpu index, so
# assert on which branch spoke rather than on the index.
_stderr_unverifiable_override() { # $1 = env assignment -> stderr only
_suo_dir=$(_make_kfd_only_host)
_suo_stub=$(_amd_host_no_kfd_stub)
PATH="$_suo_dir:$_TOOLS_DIR" "$_SH" -c "
unset CUDA_VISIBLE_DEVICES UNSLOTH_ROCM_GFX_ARCH UNSLOTH_TORCH_INDEX_URL
unset UNSLOTH_TORCH_INDEX_FAMILY ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES
unset HSA_OVERRIDE_GFX_VERSION
export $1
_ARCH=x86_64
. '$_E2E_FUNCS'
. '$_suo_stub'
get_torch_index_url >/dev/null
" 2>&1
rm -rf "$_suo_dir" "$(dirname "$_suo_stub")"
}
assert_eq "the spoof refusal names HSA_OVERRIDE_GFX_VERSION" "yes" \
"$(_stderr_unverifiable_override HSA_OVERRIDE_GFX_VERSION=10.3.0 \
| grep -qF 'HSA_OVERRIDE_GFX_VERSION is set and this host cannot confirm' && echo yes || echo no)"
assert_eq "a declared arch on a tool-blind host is not a spoof" "yes" \
"$(_stderr_unverifiable_override UNSLOTH_ROCM_GFX_ARCH=gfx1151 \
| grep -qF 'cannot confirm its real arch' && echo no || echo yes)"
# ROCm, not cpu: nothing is spoofed, so the declared gfx1151 is simply the arch. This asserted
# cpu until the presence stub landed, and only ever got there by short-circuiting on a host
# with no AMD GPU at all.
assert_eq "a declared arch on a real AMD host keeps its rocm index" \
"https://download.pytorch.org/whl/rocm7.2" \
"$(_index_unverifiable_override UNSLOTH_ROCM_GFX_ARCH=gfx1151)"
# With no override there is nothing to spoof, so an empty physical read is just an unreadable
# host and the pre-existing routing stands.
assert_eq "no override and no probe is not treated as a spoof" \
"rocm" "$(_route '')"
echo "=== A declared arch must not answer for the silicon ==="
# UNSLOTH_ROCM_GFX_ARCH short-circuits _probe_amd_gfx_arch, so a stale gfx1030 on a real Van
# Gogh answered the gate with a healthy arch. "physical" mode skips it.
_index_for_declared_arch() { # $1 = real silicon, $2 = UNSLOTH_ROCM_GFX_ARCH
_ida_dir=$(_make_rocminfo_host "$1")
PATH="$_ida_dir:$_TOOLS_DIR" "$_SH" -c "
unset CUDA_VISIBLE_DEVICES UNSLOTH_TORCH_INDEX_URL UNSLOTH_TORCH_INDEX_FAMILY
unset HSA_OVERRIDE_GFX_VERSION ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES
UNSLOTH_ROCM_GFX_ARCH='$2'; export UNSLOTH_ROCM_GFX_ARCH
_ARCH=x86_64
. '$_E2E_FUNCS'
get_torch_index_url
" 2>/dev/null | tail -1
rm -rf "$_ida_dir"
}
assert_eq "stale UNSLOTH_ROCM_GFX_ARCH=gfx1030 on a Deck -> cpu" \
"https://download.pytorch.org/whl/cpu" "$(_index_for_declared_arch gfx1033 gfx1030)"
# The override still routes an ordinary host: a real gfx1030 keeps ROCm either way.
assert_eq "declared gfx1030 on a real gfx1030 host keeps rocm" \
"https://download.pytorch.org/whl/rocm7.2" "$(_index_for_declared_arch gfx1030 gfx1030)"
# And it is still honoured where nothing else can answer: no rocminfo, no amd-smi.
_no_probe_index=$( _np=$(mktemp -d)
_np_stub=$(_amd_host_no_kfd_stub)
printf '#!/bin/sh\necho 7.2.0\n' > "$_np/hipconfig"; chmod +x "$_np/hipconfig"
PATH="$_np:$_TOOLS_DIR" "$_SH" -c "
unset UNSLOTH_TORCH_INDEX_URL UNSLOTH_TORCH_INDEX_FAMILY HSA_OVERRIDE_GFX_VERSION
unset ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES CUDA_VISIBLE_DEVICES
UNSLOTH_ROCM_GFX_ARCH=gfx1033; export UNSLOTH_ROCM_GFX_ARCH
_ARCH=x86_64
. '$_E2E_FUNCS'
. '$_np_stub'
get_torch_index_url" 2>/dev/null | tail -1
rm -rf "$_np" "$(dirname "$_np_stub")" )
assert_eq "declared gfx1033 with no probe tool -> cpu" \
"https://download.pytorch.org/whl/cpu" "$_no_probe_index"
echo "=== Structural: the gate precedes the version-keyed index selection ==="
_gate_line=$(grep -n 'Archs measured to compute INCORRECTLY under ROCm' "$INSTALL_SH" | head -1 | cut -d: -f1)
_idx_line=$(grep -n 'rocm7.2|rocm7.2.\*) echo "\$_base/rocm7.2"' "$INSTALL_SH" | head -1 | cut -d: -f1)
assert_eq "gate is before the rocm index case" "yes" \
"$([ -n "$_gate_line" ] && [ -n "$_idx_line" ] && [ "$_gate_line" -lt "$_idx_line" ] && echo yes || echo no)"
echo ""
echo "PASS=$PASS FAIL=$FAIL"
[ "$FAIL" -eq 0 ]

60
tests/sh/test_rocm_no_version_arch_route_e2e.sh
View file

@ -289,6 +289,17 @@ EOF
printf 'INDEX=<installer exited %s>\n' "$?"
}
# run_installer unsets UNSLOTH_ROCM_GFX_ARCH inside the generated script, so this variant
# re-applies it AFTER that unset, for the stale-declared-arch cases.
run_installer_env() { # $1 = "VAR=value", $2 = _ARCH
run_installer "${2:-x86_64}" >/dev/null 2>&1 || true
sed -i "s|^_ARCH=|export $1\n_ARCH=|" "$_ROOT/run.sh"
PATH="$_MOCK:$_TOOLS" "$_RUN_SHELL" "$_ROOT/run.sh" 2>"$_ROOT/stderr.txt" || \
printf 'INDEX=<installer exited %s>\n' "$?"
}
run_index_env() { run_installer_env "$1" "${2:-x86_64}" | sed -n 's/^INDEX=//p'; }
run_gfx_env() { run_installer_env "$1" "${2:-x86_64}" | sed -n 's/^GFX=//p'; }
run_index() { run_installer "${1:-x86_64}" | sed -n 's/^INDEX=//p'; }
run_gfx() { run_installer "${1:-x86_64}" | sed -n 's/^GFX=//p'; }
run_radeon() { run_installer "${1:-x86_64}" | sed -n 's/^RADEON=//p'; }
@ -463,6 +474,55 @@ assert_eq "a real gfx1100 corroborated by the kernel keeps its own family" \
"$_AMD/gfx110X-all/" "$(HSA_OVERRIDE_GFX_VERSION=11.0.0 run_index)"
# gfx1033 (Van Gogh) shares gfx103X-all with gfx1030-gfx1036, so a mixed 103X host is the one
# shape where _amd_agreed_index_family AGREES while _amd_sole_index_arch declines, and the
# shared-family arm then rewrote the cpu index the gate had just chosen back to ROCm wheels.
fedora_no_version_host gfx1033 gfx1030
assert_eq "a mixed gfx1033 + gfx1030 host does not reroute to the shared family" \
"$_BASE/cpu" "$(run_index)"
fedora_no_version_host gfx1033 gfx1030
assert_eq "and names no arch for setup.sh to build" "" "$(run_gfx)"
# The same family without the bad arch still reroutes: this is a gfx1033 rule, not a
# "distrust shared families" rule.
fedora_no_version_host gfx1030 gfx1032
assert_eq "gfx1030 + gfx1032 still take the shared gfx103X-all index" \
"$_AMD/gfx103X-all/" "$(run_index)"
# A lone Deck was already covered by the gate itself; assert it here too so the reroute
# cannot regrow its own path back to ROCm.
fedora_no_version_host gfx1033
assert_eq "a lone gfx1033 does not reroute" "$_BASE/cpu" "$(run_index)"
# A stale UNSLOTH_ROCM_GFX_ARCH=gfx1030 on a real Deck: _infer_linux_amd_gfx_arch returns the
# override, so an arm testing only the inferred value walked the gate's cpu index back to
# gfx103X-all. A READABLE ROCm version takes the plain inferred-gfx arm, which never consults
# the probe.
readable_version_host() { # $@ = gfx arches
reset_host
mock_rocminfo "$@"
mock_amdsmi "N/A" "$@"
mock_hipconfig_offpath "7.2.0"
mock_rpm
mock_kfd
mock_pci_amd_display
}
readable_version_host gfx1033
assert_eq "a stale gfx1030 override on a real Deck does not reroute" \
"$_BASE/cpu" "$(run_index_env UNSLOTH_ROCM_GFX_ARCH=gfx1030)"
readable_version_host gfx1033
assert_eq "and the rejected override is not forwarded to setup.sh" \
"" "$(run_gfx_env UNSLOTH_ROCM_GFX_ARCH=gfx1030)"
# A declared arch on hardware that really is that arch still routes normally, so this
# stays a gfx1033 veto rather than a blanket distrust of the override.
readable_version_host gfx1030
# A readable ROCm version means get_torch_index_url returns the version-keyed index and never
# reaches the */cpu reroute, so the override is simply not consulted here.
assert_eq "a gfx1030 override on real gfx1030 keeps the version index" \
"$_BASE/rocm7.2" "$(run_index_env UNSLOTH_ROCM_GFX_ARCH=gfx1030)"
# The no-version path stays covered too, since it reaches the family arm instead.
fedora_no_version_host gfx1033
assert_eq "a stale override on a no-version Deck does not reroute either" \
"$_BASE/cpu" "$(run_index_env UNSLOTH_ROCM_GFX_ARCH=gfx1030)"
mock_kfd_two() { # $1 $2 = gfx_target_version per KFD node
: > "$_F_KFD"
for _i in 1 2; do

176
tests/sh/test_shell_rc_path_append.sh Executable file
View file

@ -0,0 +1,176 @@
#!/bin/bash
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
#
# Guards the shell-rc PATH append in install.sh (_persist_login_path_dir and the fish drop-in
# arm _persist_fish_path_dir). Under `set -e` an unwritable rc used to abort the installer at
# its last cosmetic step, which immutable homes (NixOS/home-manager, chezmoi) hit every time.
# Contract: a writable rc gets the export; an rc already mentioning .local/bin is untouched;
# no rc is a no-op; an unwritable rc warns, prints the manual line and returns 0; the lines go
# in as ONE redirect so no partial write is possible. The fish arm shares all of it.
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
INSTALL_SH="$SCRIPT_DIR/../../install.sh"
PASS=0
FAIL=0
assert_contains() {
_label="$1"; _haystack="$2"; _needle="$3"
if echo "$_haystack" | grep -qF "$_needle"; then
echo " PASS: $_label"
PASS=$((PASS + 1))
else
echo " FAIL: $_label (expected to find '$_needle')"
echo " ---- output ----"; echo "$_haystack" | sed 's/^/ | /'
FAIL=$((FAIL + 1))
fi
}
assert_not_contains() {
_label="$1"; _haystack="$2"; _needle="$3"
if echo "$_haystack" | grep -qF "$_needle"; then
echo " FAIL: $_label (found '$_needle' but should not)"
echo " ---- output ----"; echo "$_haystack" | sed 's/^/ | /'
FAIL=$((FAIL + 1))
else
echo " PASS: $_label"
PASS=$((PASS + 1))
fi
}
# Extract the functions under test. The POSIX arm delegates to the fish arm and reads
# _PATH_LINE_RE, so all three come along.
_FN_FILE=$(mktemp)
{
grep '^_PATH_LINE_RE=' "$INSTALL_SH"
sed -n '/^_persist_fish_path_dir()/,/^}/p' "$INSTALL_SH"
sed -n '/^_persist_login_path_dir()/,/^}/p' "$INSTALL_SH"
} > "$_FN_FILE"
for _fn in _persist_fish_path_dir _persist_login_path_dir; do
if ! grep -q "^$_fn()" "$_FN_FILE"; then
echo "FAIL: could not extract $_fn from install.sh"
echo " (it must stay a top-level function so this test can reach it)"
exit 1
fi
done
if ! grep -q '^_PATH_LINE_RE=' "$_FN_FILE"; then
echo "FAIL: could not extract _PATH_LINE_RE from install.sh"
exit 1
fi
_HARNESS=$(mktemp)
cat > "$_HARNESS" <<'HARNESS'
C_WARN=''; C_ERR=''; C_OK=''; C_DIM=''; C_RST=''
step() { echo "STEP $1 $2"; }
substep() { echo "SUBSTEP $1"; }
HARNESS
_SH="${BASH:-/bin/bash}"
# `set -e` mirrors install.sh: a non-zero return kills the runner before it echoes RC, which
# is the regression being guarded. $5 pins the profile file so the rc-selection cascade stays
# out of it; SHELL is forced non-fish so the POSIX arm runs.
_run() { # $1 = rc path
# LC_ALL=C so the raw-diagnostic assertions below can match the shell's own wording.
( LC_ALL=C SHELL=/bin/bash "$_SH" -c "set -e; . '$_HARNESS'; . '$_FN_FILE'; _persist_login_path_dir \"\$HOME/.local/bin\" '\$HOME/.local/bin' '~/.local/bin' '\\.local/bin' '$1'; echo \"RC=\$?\"" 2>&1 )
}
# The fish arm picks its own file under $HOME, so steer it with HOME, not an argument.
_run_fish() { # $1 = HOME
( LC_ALL=C HOME="$1" SHELL=/usr/bin/fish "$_SH" -c "set -e; . '$_HARNESS'; . '$_FN_FILE'; _persist_login_path_dir \"\$HOME/.local/bin\" '\$HOME/.local/bin' '~/.local/bin' '\\.local/bin'; echo \"RC=\$?\"" 2>&1 )
}
_TMP=$(mktemp -d)
trap 'rm -rf "$_TMP" "$_FN_FILE" "$_HARNESS"' EXIT
echo "=== writable rc: appends the export ==="
_rc="$_TMP/.bashrc"; : > "$_rc"
_out=$(_run "$_rc")
assert_contains "reports the append" "$_out" "STEP path added ~/.local/bin to PATH"
assert_contains "returns 0" "$_out" "RC=0"
assert_contains "export written" "$(cat "$_rc")" 'export PATH="$HOME/.local/bin:$PATH"'
assert_contains "marker written" "$(cat "$_rc")" '# Added by Unsloth installer'
echo "=== already mentions .local/bin: left untouched (idempotent) ==="
_rc2="$_TMP/.bashrc_existing"
printf 'export PATH="$HOME/.local/bin:$PATH"\n' > "$_rc2"
_before=$(cat "$_rc2")
_out=$(_run "$_rc2")
assert_contains "returns 0" "$_out" "RC=0"
assert_not_contains "no second append" "$_out" "added ~/.local/bin"
if [ "$(cat "$_rc2")" = "$_before" ]; then
echo " PASS: file unchanged"; PASS=$((PASS + 1))
else
echo " FAIL: file changed on a re-run"; FAIL=$((FAIL + 1))
fi
echo "=== no home at all (nothing to write to): no-op ==="
# The current shape derives the file, so "nowhere to persist to" means an unset HOME.
_out=$( env -u HOME "$_SH" -c "set -e; . '$_HARNESS'; . '$_FN_FILE'; _persist_login_path_dir '/x/.local/bin' '\$HOME/.local/bin' '~/.local/bin' '\\.local/bin'; echo \"RC=\$?\"" 2>&1 )
assert_contains "returns 0" "$_out" "RC=0"
assert_not_contains "no step" "$_out" "STEP path"
echo "=== READ-ONLY rc: warns, does not fail the install ==="
# The NixOS / home-manager / chezmoi shape. Root ignores the write bit, so skip there.
_rc3="$_TMP/.bashrc_ro"; : > "$_rc3"; chmod 0444 "$_rc3"
if [ "$(id -u)" = "0" ] || { echo x >> "$_rc3"; } 2>/dev/null; then
echo " SKIP: cannot make a file unwritable as this user (running as root?)"
: > "$_rc3"
else
_out=$(_run "$_rc3")
assert_contains "returns 0 (set -e survives)" "$_out" "RC=0"
assert_contains "warns about the rc" "$_out" "could not write"
assert_contains "prints the manual line" "$_out" 'export PATH="$HOME/.local/bin:$PATH"'
assert_contains "reassures install is fine" "$_out" "only the PATH line is missing"
assert_not_contains "does not claim success" "$_out" "added ~/.local/bin to PATH in"
# `>> file 2>/dev/null` silences nothing: redirections apply left to right, so stderr is
# still the terminal when the append fails. Only 2>/dev/null placed first is quiet, and a
# raw diagnostic is what this branch exists to replace.
assert_not_contains "no raw shell diagnostic" "$_out" "Permission denied"
if [ ! -s "$_rc3" ]; then
echo " PASS: read-only file left empty (no partial write)"; PASS=$((PASS + 1))
else
echo " FAIL: read-only file was modified"; FAIL=$((FAIL + 1))
fi
fi
chmod 0644 "$_rc3" 2>/dev/null || true
echo "=== fish: writable conf.d gets the drop-in ==="
_fhome="$_TMP/fishhome"; mkdir -p "$_fhome"
_out=$(_run_fish "$_fhome")
assert_contains "returns 0" "$_out" "RC=0"
assert_contains "reports the add" "$_out" "STEP path added ~/.local/bin to PATH"
assert_contains "fish_add_path written" "$(cat "$_fhome/.config/fish/conf.d/unsloth.fish" 2>&1)" "fish_add_path '$_fhome/.local/bin'"
echo "=== fish: re-run is idempotent ==="
_before=$(cat "$_fhome/.config/fish/conf.d/unsloth.fish")
_out=$(_run_fish "$_fhome")
assert_contains "returns 0" "$_out" "RC=0"
if [ "$(cat "$_fhome/.config/fish/conf.d/unsloth.fish")" = "$_before" ]; then
echo " PASS: fish file unchanged"; PASS=$((PASS + 1))
else
echo " FAIL: fish file changed on a re-run"; FAIL=$((FAIL + 1))
fi
echo "=== fish: READ-ONLY drop-in warns, does not fail the install ==="
# Same immutable-home shape as the POSIX case: home-manager manages conf.d too.
_fro="$_TMP/fishro"; mkdir -p "$_fro/.config/fish/conf.d"
_frofile="$_fro/.config/fish/conf.d/unsloth.fish"; : > "$_frofile"; chmod 0444 "$_frofile"
if [ "$(id -u)" = "0" ] || { echo x >> "$_frofile"; } 2>/dev/null; then
echo " SKIP: cannot make a file unwritable as this user (running as root?)"
else
_out=$(_run_fish "$_fro")
assert_contains "returns 0 (set -e survives)" "$_out" "RC=0"
assert_contains "warns about the drop-in" "$_out" "could not write"
assert_contains "prints the manual line" "$_out" "fish_add_path '$_fro/.local/bin'"
assert_contains "reassures install is fine" "$_out" "only the PATH line is missing"
assert_not_contains "does not claim success" "$_out" "added ~/.local/bin to PATH in"
assert_not_contains "no raw shell diagnostic" "$_out" "Permission denied"
fi
chmod 0644 "$_frofile" 2>/dev/null || true
echo ""
echo "PASS=$PASS FAIL=$FAIL"
[ "$FAIL" -eq 0 ]

167
tests/sh/test_uv_cache_colocation.sh Executable file
View file

@ -0,0 +1,167 @@
#!/bin/bash
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
#
# Guards the uv cache co-location in install.sh.
#
# uv's cache defaulted to $HOME/.cache/uv while STUDIO_HOME can be pointed anywhere with
# UNSLOTH_STUDIO_HOME. uv hardlinks wheels out of its cache into the venv when both are on
# one filesystem and COPIES when they are not, so every redirected install paid twice the
# disk and left several GB on the drive the user had deliberately moved off (an SD card or
# second disk being the usual reason to redirect at all).
#
# Measured with torch 2.11.0+cpu: co-located, a 749 MB cache and a 748 MB venv occupy 755 MB
# between them (st_nlink 2 on the shared objects); across a boundary they are duplicated.
#
# The contract:
# * unset UV_CACHE_DIR -> $STUDIO_HOME/cache/uv, exported, directory created
# * caller-set -> left exactly as-is
# * follows STUDIO_HOME wherever UNSLOTH_STUDIO_HOME puts it
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
INSTALL_SH="$SCRIPT_DIR/../../install.sh"
PASS=0
FAIL=0
assert_eq() {
_label="$1"; _expected="$2"; _actual="$3"
if [ "$_actual" = "$_expected" ]; then
echo " PASS: $_label"
PASS=$((PASS + 1))
else
echo " FAIL: $_label (expected '$_expected', got '$_actual')"
FAIL=$((FAIL + 1))
fi
}
# Lift the block out of install.sh so the real code is what runs here.
_FN_FILE=$(mktemp)
_TMP=$(mktemp -d)
trap 'rm -rf "$_FN_FILE" "$_TMP"' EXIT
awk '/^# Keep uv.s cache on the same filesystem as the venv it fills\.$/,/^fi$/' \
"$INSTALL_SH" > "$_FN_FILE"
if ! grep -q 'UV_CACHE_DIR="\$STUDIO_HOME/cache/uv"' "$_FN_FILE"; then
echo "FAIL: could not extract the UV_CACHE_DIR block from install.sh"
exit 1
fi
_SH="${BASH:-/bin/bash}"
_run() { # $1 = STUDIO_HOME, $2 = preset UV_CACHE_DIR ("" for unset)
"$_SH" -c "
STUDIO_HOME='$1'
if [ -n '$2' ]; then UV_CACHE_DIR='$2'; export UV_CACHE_DIR; else unset UV_CACHE_DIR; fi
. '$_FN_FILE'
printf '%s' \"\${UV_CACHE_DIR:-<unset>}\"
"
}
echo "=== unset: defaults under STUDIO_HOME ==="
assert_eq "default home" "$_TMP/studio/cache/uv" "$(_run "$_TMP/studio" '')"
# The whole point: a redirected STUDIO_HOME takes the cache with it.
assert_eq "redirected home" "$_TMP/sdcard/unsloth/cache/uv" "$(_run "$_TMP/sdcard/unsloth" '')"
echo "=== the directory is actually created (uv would too, but not before we log) ==="
_run "$_TMP/mk" '' >/dev/null
assert_eq "cache dir created" "yes" "$([ -d "$_TMP/mk/cache/uv" ] && echo yes || echo no)"
echo "=== an uncreatable cache falls back to uv's default, it does not stay exported ==="
# uv aborts with "Failed to initialize cache at ..." on a cache path it cannot create, so
# keeping the export after a failed mkdir turns a disk optimisation into a hard install
# failure on a host where uv's own default would have worked.
: > "$_TMP/blocked" # STUDIO_HOME is a FILE -> $STUDIO_HOME/cache/uv cannot exist
assert_eq "uncreatable cache is dropped" "<unset>" "$(_run "$_TMP/blocked" '')"
mkdir -p "$_TMP/rofile" && : > "$_TMP/rofile/cache" # writable home, "cache" is a file
assert_eq "cache-as-file is dropped" "<unset>" "$(_run "$_TMP/rofile" '')"
# mkdir -p exits 0 for a directory that ALREADY exists, so an unwritable leftover cache from
# an earlier install under another account passed the mkdir and then failed uv exactly as an
# uncreatable path does. Writability has to be probed, not inferred.
mkdir -p "$_TMP/leftover/cache/uv" && chmod 500 "$_TMP/leftover/cache/uv"
if [ "$(id -u)" = "0" ]; then
echo " SKIP: unwritable-cache case (root writes through the mode bits)"
else
assert_eq "unwritable existing cache is dropped" "<unset>" "$(_run "$_TMP/leftover" '')"
fi
chmod 700 "$_TMP/leftover/cache/uv"
# ...and the probe file it writes does not survive into the cache uv then fills.
_run "$_TMP/probe" '' >/dev/null
assert_eq "write probe cleaned up" "" "$(ls -A "$_TMP/probe/cache/uv")"
# ... and the fallback really is usable, unlike the path we just refused.
if command -v uv >/dev/null 2>&1; then
_uv_rc=$("$_SH" -c "
STUDIO_HOME='$_TMP/rofile'
unset UV_CACHE_DIR
. '$_FN_FILE'
uv venv '$_TMP/rofile/venv' >/dev/null 2>&1
printf '%s' \"\$?\"
")
assert_eq "uv still runs after the fallback" "0" "$_uv_rc"
fi
echo "=== a caller-set UV_CACHE_DIR is never overridden ==="
assert_eq "explicit value kept" "/custom/uvcache" "$(_run "$_TMP/studio" '/custom/uvcache')"
echo "=== it is exported, not just assigned (uv runs in child processes) ==="
_exported=$("$_SH" -c "
STUDIO_HOME='$_TMP/studio'
unset UV_CACHE_DIR
. '$_FN_FILE'
sh -c 'printf %s \"\${UV_CACHE_DIR:-<unset>}\"'
")
assert_eq "visible to child processes" "$_TMP/studio/cache/uv" "$_exported"
echo "=== structural: set before uv is first invoked ==="
_set_line=$(grep -n 'UV_CACHE_DIR="\$STUDIO_HOME/cache/uv"' "$INSTALL_SH" | head -1 | cut -d: -f1)
_uv_line=$(grep -n 'installing uv package manager' "$INSTALL_SH" | head -1 | cut -d: -f1)
assert_eq "precedes uv bootstrap" "yes" \
"$([ -n "$_set_line" ] && [ -n "$_uv_line" ] && [ "$_set_line" -lt "$_uv_line" ] && echo yes || echo no)"
# Match the call that creates the venv, not the label it carries: the literal
# 'run_install_cmd "create venv" uv venv' stopped existing when #8479 moved venv
# creation behind _run_uv_venv, and a label this file cannot find reads as "the cache
# is set too late" rather than "the grep is stale". Comment lines are dropped so the
# prose above the helper does not answer first.
_venv_line=$(grep -nE '(^|[^[:alnum:]_"`])uv venv([[:space:]]|$)' "$INSTALL_SH" \
| grep -vE '^[0-9]+:[[:space:]]*#' | head -1 | cut -d: -f1)
assert_eq "found the venv creation call" "yes" "$([ -n "$_venv_line" ] && echo yes || echo no)"
assert_eq "precedes venv creation" "yes" \
"$([ -n "$_set_line" ] && [ -n "$_venv_line" ] && [ "$_set_line" -lt "$_venv_line" ] && echo yes || echo no)"
echo "=== studio/setup.sh sets the SAME cache (it is the standalone update entry point) ==="
# install.sh exports UV_CACHE_DIR for its own process only, and `unsloth studio update` runs
# studio/setup.sh directly, so without the same block there a redirected STUDIO_HOME
# downloads a second cache to $HOME/.cache/uv on the first update and copies every wheel
# across the boundary. Same path, so the update also REUSES what the install fetched.
SETUP_SH="$SCRIPT_DIR/../../studio/setup.sh"
_SETUP_FN=$(mktemp)
trap 'rm -rf "$_FN_FILE" "$_TMP" "$_SETUP_FN"' EXIT
awk '/^# Same uv cache install\.sh chose/,/^fi$/' "$SETUP_SH" > "$_SETUP_FN"
if ! grep -q 'UV_CACHE_DIR="\$STUDIO_HOME/cache/uv"' "$_SETUP_FN"; then
echo " FAIL: could not extract the UV_CACHE_DIR block from studio/setup.sh"
FAIL=$((FAIL + 1))
else
_run_setup() { # $1 = STUDIO_HOME, $2 = preset UV_CACHE_DIR ("" for unset)
"$_SH" -c "
STUDIO_HOME='$1'
if [ -n '$2' ]; then UV_CACHE_DIR='$2'; export UV_CACHE_DIR; else unset UV_CACHE_DIR; fi
. '$_SETUP_FN'
printf '%s' \"\${UV_CACHE_DIR:-<unset>}\"
"
}
assert_eq "setup.sh defaults under STUDIO_HOME" \
"$_TMP/upd/cache/uv" "$(_run_setup "$_TMP/upd" '')"
# The whole point: the update lands on the SAME path install.sh chose, so a redirected
# home does not grow a second cache on the original filesystem.
assert_eq "setup.sh matches install.sh for a redirected home" \
"$_TMP/sdcard/unsloth/cache/uv" "$(_run_setup "$_TMP/sdcard/unsloth" '')"
assert_eq "setup.sh keeps a caller-set value" \
"/custom/uvcache" "$(_run_setup "$_TMP/upd" '/custom/uvcache')"
# Same unwritable-path contract as install.sh, so an update cannot fail where the install
# succeeded.
mkdir -p "$_TMP/updro" && : > "$_TMP/updro/cache"
assert_eq "setup.sh drops an unusable cache" "<unset>" "$(_run_setup "$_TMP/updro" '')"
fi
echo ""
echo "PASS=$PASS FAIL=$FAIL"
[ "$FAIL" -eq 0 ]

298
tests/studio/install/test_rocm_support.py
View file

@ -2872,10 +2872,13 @@ class TestInstallShStructure:
"amd-smi list 2>/dev/null) | grep -oE 'gfx" in helper
), "probe must read gfx from amd-smi"
# The probe clears ROCR/HIP_VISIBLE_DEVICES so a container mask
# (ROCR_VISIBLE_DEVICES=-1) can't blind the env-independent KFD detection.
# (ROCR_VISIBLE_DEVICES=-1) can't blind the env-independent KFD detection. Assert
# both halves: the names in the strip list, and the list reaching unset. The
# literal `unset ROCR... HIP...` went away when "physical" mode was added.
assert (
"unset ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES" in helper
'_pg_strip="ROCR_VISIBLE_DEVICES HIP_VISIBLE_DEVICES"' in helper
), "the gfx probe must clear the visibility masks so a mask can't force CPU"
assert "unset $_pg_strip" in helper, "the strip list must reach unset"
cpu_guard = body.find('if [ -z "$_amd_gfx_probe" ]')
assert cpu_guard >= 0, "unreadable gfx must fall back to CPU"
assert cpu_guard < body.find(
@ -3402,6 +3405,11 @@ class TestInstallShStructure:
assert (
"URL:https://repo.amd.com/rocm/whl/gfx1151/ GFX:gfx1151" in r.stdout
), f"the probed arch must win over lspci inference here: {r.stdout!r}"
r_bad = run("true", "gfx1033", probed_family = "gfx103X-all")
assert r_bad.returncode == 0, f"gfx1033 no-version reroute aborted: {r_bad.stderr}"
assert (
"URL:https://download.pytorch.org/whl/cpu GFX:\n" in r_bad.stdout
), f"the no-version reroute must not bypass the gfx1033 gate: {r_bad.stdout!r}"
# A same-family pair must export no arch: setup.sh prefers
# UNSLOTH_ROCM_GFX_ARCH and would build llama.cpp for a guessed card.
r_pair = run("true", "", probed_family = "gfx120X-all")
@ -3409,6 +3417,11 @@ class TestInstallShStructure:
assert (
"URL:https://repo.amd.com/rocm/whl/gfx120X-all/ GFX:\n" in r_pair.stdout
), f"a same-family pair routes but must not export a card: {r_pair.stdout!r}"
r_mixed = run("true", "", probed_family = "gfx103X-all")
assert r_mixed.returncode == 0, f"mixed gfx103X pair aborted: {r_mixed.stderr}"
assert (
"URL:https://repo.amd.com/rocm/whl/gfx103X-all/ GFX:\n" in r_mixed.stdout
), f"a healthy same-family GPU beside gfx1033 must keep ROCm: {r_mixed.stdout!r}"
r2 = run("false", "", probe_stub = "printf '\\n'")
assert r2.returncode == 0, f"inferred-arch case aborted: {r2.stderr}"
assert (
@ -6154,6 +6167,22 @@ class TestStrixRocm71Override:
with patch.object(m, "IS_WINDOWS", True):
assert m._amd_arch_index_url("gfx1151") == m._windows_rocm_index_url("gfx1151")
def test_amd_arch_index_url_refuses_miscomputing_arch(self):
"""gfx1033 (Van Gogh) installs ROCm wheels fine and then computes wrong answers
(studio/ROCM_RDNA2_APU.md), so install.sh routes it to the cpu index. The
inferred-gfx repair here is exempt from the runtime gate when
UNSLOTH_ROCM_GFX_ARCH is set, so without this it would force-reinstall the exact
ROCm wheels that gate avoids. Its family neighbours must keep working (#7277)."""
m = stack_mod
with (
patch.object(m, "IS_WINDOWS", False),
patch.dict(os.environ, {"UNSLOTH_AMD_ROCM_MIRROR": ""}),
):
assert m._amd_arch_index_url("gfx1033") is None
assert m._amd_arch_index_url("GFX1033") is None
for _gfx in ("gfx1030", "gfx1031", "gfx1032", "gfx1034", "gfx1035", "gfx1036"):
assert m._amd_arch_index_url(_gfx) == "https://repo.amd.com/rocm/whl/gfx103X-all/"
def test_a_windows_mirror_token_ending_in_a_slash_survives_the_join(self):
"""_index_url_join trims the path head and leaves the query alone, which is the whole
point of it: "/" is in the base64 alphabet, so a token may legitimately end with one.
@ -7245,5 +7274,270 @@ class TestRocmWslSupplyChainPins:
assert "_co_failed=1" in helper
class TestRocmMiscomputingArchDemotion:
"""A gfx1033 venv that ALREADY holds ROCm torch must be demoted, not just declined.
_amd_arch_index_url() returning None only blocks a fresh ROCm install. On an upgrade the
existing build survived every gate: install.sh resolves UNSLOTH_TORCH_BACKEND=cpu, which
returns _ensure_rocm_torch at its first line; _ensure_cpu_torch fired only for an explicit
pin; and the base update does not reinstall an already-satisfied torch.
"""
def _demotion_calls(
self,
monkeypatch,
label,
archs,
env = None,
codes_fn = None,
kfd = (),
):
calls = []
monkeypatch.setattr(stack_mod, "_kfd_gfx_targets", lambda: list(kfd))
monkeypatch.setattr(stack_mod, "NO_TORCH", False)
monkeypatch.setattr(stack_mod, "IS_WINDOWS", False)
monkeypatch.setattr(stack_mod, "IS_MACOS", False)
monkeypatch.setattr(stack_mod, "_installed_torch_label_on_disk", lambda: label)
monkeypatch.setattr(
stack_mod, "_detect_amd_gfx_codes", codes_fn or (lambda **_kw: list(archs))
)
monkeypatch.setattr(stack_mod, "_infer_linux_amd_gfx_arch", lambda: None)
monkeypatch.setattr(
stack_mod,
"_probe_torch_runtime",
lambda: (True, True, label, "7.1" if "+rocm" in label else "", ""),
)
monkeypatch.setattr(stack_mod, "pip_install", lambda *a, **k: calls.append((a, k)))
for _var in (
"UNSLOTH_TORCH_INDEX_URL",
"UNSLOTH_TORCH_INDEX_FAMILY",
"UNSLOTH_ROCM_GFX_ARCH",
):
monkeypatch.delenv(_var, raising = False)
for _var, _val in (env or {}).items():
monkeypatch.setenv(_var, _val)
stack_mod._ensure_cpu_torch()
return calls
def test_installed_rocm_torch_on_gfx1033_is_demoted(self, monkeypatch):
calls = self._demotion_calls(monkeypatch, "2.10.0+rocm7.1", ["gfx1033"])
assert len(calls) == 1, "gfx1033 kept its ROCm torch across the upgrade"
assert "--force-reinstall" in calls[0][0]
assert "download.pytorch.org/whl/cpu" in str(calls[0])
def test_hsa_spoofed_gfx1033_is_still_demoted(self, monkeypatch):
"""HSA_OVERRIDE_GFX_VERSION=10.3.0 is the usual Van Gogh workaround and makes
rocminfo answer gfx1030, so the probe must ignore it or the arch hides itself."""
seen = {}
def _codes(**kw):
seen.update(kw)
return ["gfx1030"] if not kw.get("ignore_hsa_override") else ["gfx1033"]
calls = self._demotion_calls(monkeypatch, "2.10.0+rocm7.1", [], codes_fn = _codes)
assert seen.get("ignore_hsa_override") is True
assert len(calls) == 1
def test_a_mask_cannot_shrink_the_host_to_its_bad_gpu(self, monkeypatch):
""" "Every arch is bad" is a question about the host, so it must be asked of the
whole host. ROCR_VISIBLE_DEVICES hiding the healthy dGPU would otherwise leave
gfx1033 as the only arch and demote a working ROCm install to CPU. install.sh's
_probe_amd_gfx_arch unsets both masks for exactly this reason."""
seen = {}
def _codes(**kw):
seen.update(kw)
return ["gfx1033"] if not kw.get("ignore_visible_masks") else ["gfx1033", "gfx1100"]
calls = self._demotion_calls(
monkeypatch,
"2.10.0+rocm7.1",
[],
env = {"ROCR_VISIBLE_DEVICES": "0"},
codes_fn = _codes,
)
assert seen.get("ignore_visible_masks") is True
assert calls == [], "a mask shrank the host to its miscomputing GPU"
def test_kfd_topology_names_the_arch_when_no_runtime_does(self, monkeypatch):
"""A Deck with ROCm torch but no working rocminfo most needs the demotion, and is where
both runtime sources answer nothing: _detect_amd_gfx_codes() has no probe to run and no
Van Gogh product name maps through _infer_linux_amd_gfx_arch(). amdkfd is in the kernel
and still answers."""
calls = self._demotion_calls(monkeypatch, "2.10.0+rocm7.1", [], kfd = ["gfx1033"])
assert len(calls) == 1, "gfx1033 kept its ROCm torch with only KFD to name it"
assert "--force-reinstall" in calls[0][0]
assert "download.pytorch.org/whl/cpu" in str(calls[0])
def test_kfd_reports_every_node_so_a_mixed_host_keeps_rocm(self, monkeypatch):
"""_kfd_gfx_targets() returns one entry per GPU node, so the all-archs rule reads
the same mixed host through this source as through the runtime probes."""
assert (
self._demotion_calls(monkeypatch, "2.10.0+rocm7.1", [], kfd = ["gfx1033", "gfx1100"])
== []
)
def test_the_runtime_probe_still_wins_over_kfd(self, monkeypatch):
"""KFD is the last resort, not an override: a runtime that answered keeps its
answer, so HSA_OVERRIDE handling and the mask semantics above are untouched."""
assert (
self._demotion_calls(monkeypatch, "2.10.0+rocm7.1", ["gfx1100"], kfd = ["gfx1033"]) == []
)
def test_a_declared_arch_does_not_outrank_the_silicon(self, monkeypatch):
"""UNSLOTH_ROCM_GFX_ARCH is a routing hint, not an answer about the hardware.
A stale gfx1030 on a real Van Gogh, which HSA_OVERRIDE_GFX_VERSION=10.3.0 also spoofs
to, used to be taken first and skip the demotion outright, so a standalone
`studio update` left the miscomputing wheels in place. The probes are asked first now,
and the declared arch only answers when none of them can.
"""
calls = self._demotion_calls(
monkeypatch,
"2.10.0+rocm7.1",
["gfx1033"],
env = {"UNSLOTH_ROCM_GFX_ARCH": "gfx1030"},
)
assert len(calls) == 1, "a declared arch hid the silicon from the demotion"
def test_kfd_outranks_a_declared_arch(self, monkeypatch):
"""_infer_linux_amd_gfx_arch() returns the declared arch first, so KFD has to be
consulted before it or the kernel never gets to answer."""
calls = self._demotion_calls(
monkeypatch,
"2.10.0+rocm7.1",
[],
kfd = ["gfx1033"],
env = {"UNSLOTH_ROCM_GFX_ARCH": "gfx1030"},
)
assert len(calls) == 1, "the declared arch outranked the kernel"
def test_env_override_names_the_arch(self, monkeypatch):
calls = self._demotion_calls(
monkeypatch, "2.10.0+rocm7.1", [], env = {"UNSLOTH_ROCM_GFX_ARCH": "GFX1033"}
)
assert len(calls) == 1
@pytest.mark.parametrize(
"label, archs, env, why",
[
# A supported arch keeps its ROCm torch: the gate is scoped to what was measured.
("2.10.0+rocm7.1", ["gfx1030"], None, "gfx1030 demoted"),
("2.10.0+rocm7.1", ["gfx906"], None, "gfx906 demoted"),
# Mixed host: a healthy dGPU beside the APU is still served by ROCm.
("2.10.0+rocm7.1", ["gfx1033", "gfx1100"], None, "mixed host demoted"),
# Nothing to demote -- and no ROCm probe should be paid for on these hosts.
("2.10.0+cu128", ["gfx1033"], None, "CUDA torch demoted"),
("2.10.0", ["gfx1033"], None, "CPU torch reinstalled for nothing"),
# The documented escape hatch stays authoritative.
(
"2.10.0+rocm7.1",
["gfx1033"],
{"UNSLOTH_TORCH_INDEX_URL": "https://download.pytorch.org/whl/rocm7.2"},
"explicit ROCm pin overridden",
),
(
"2.10.0+rocm7.1",
["gfx1033"],
{"UNSLOTH_TORCH_INDEX_FAMILY": "rocm7.2"},
"explicit family pin overridden",
),
],
)
def test_no_demotion_outside_the_measured_case(self, monkeypatch, label, archs, env, why):
assert self._demotion_calls(monkeypatch, label, archs, env) == [], why
@pytest.mark.parametrize(
"codes, expect_install",
[
(["gfx1033"], False),
# A mixed host whose selected target is the healthy card keeps its ROCm route.
(["gfx1100"], True),
],
)
def test_ensure_rocm_torch_declines_a_miscomputing_target(
self, monkeypatch, codes, expect_install
):
"""`studio update` run standalone has no UNSLOTH_TORCH_BACKEND to read, so the
early `cpu` return in _ensure_rocm_torch does not fire and a gfx1033 host walks into
the missing-kernel reroute. gfx1033 is not in _GENERIC_ROCM_WHEEL_GFX but does map to
gfx103X-all, so `_leaf` is set while _amd_arch_index_url() returns None by design: the
reroute printed that None through _strip_index_url_credentials (AttributeError), and
guarding the print alone would drop through to the generic pytorch.org wheels.
"""
calls = []
monkeypatch.setattr(stack_mod, "IS_WINDOWS", False)
monkeypatch.setattr(stack_mod, "IS_MACOS", False)
monkeypatch.setattr(stack_mod, "_TORCH_BACKEND", "")
monkeypatch.setattr(stack_mod, "_has_usable_nvidia_gpu", lambda: False)
monkeypatch.setattr(stack_mod, "_has_rocm_gpu", lambda: True)
monkeypatch.setattr(stack_mod, "_detect_amd_gfx_codes", lambda **_kw: list(codes))
monkeypatch.setattr(stack_mod, "_kfd_gfx_targets", lambda: [])
monkeypatch.setattr(stack_mod, "_infer_linux_amd_gfx_arch", lambda: None)
monkeypatch.setattr(stack_mod, "_detect_rocm_version", lambda: (7, 1))
monkeypatch.setattr(stack_mod, "pip_install_try", lambda *a, **k: True)
monkeypatch.setattr(stack_mod, "pip_install", lambda *a, **k: calls.append((a, k)))
for _var in (
"UNSLOTH_TORCH_INDEX_URL",
"UNSLOTH_TORCH_INDEX_FAMILY",
"UNSLOTH_ROCM_GFX_ARCH",
"HIP_VISIBLE_DEVICES",
"ROCR_VISIBLE_DEVICES",
"CUDA_VISIBLE_DEVICES",
):
monkeypatch.delenv(_var, raising = False)
_probe = MagicMock(returncode = 0, stdout = _MARK + "2.10.0+cpu||\n")
with patch("os.path.isdir", return_value = True):
with patch("subprocess.run", return_value = _probe):
stack_mod._ensure_rocm_torch() # must not raise
assert bool(calls) is expect_install, calls
def test_a_stale_override_does_not_start_a_rocm_to_cpu_cycle(self, monkeypatch):
"""The thrash this ordering caused: a real Deck exporting a stale
UNSLOTH_ROCM_GFX_ARCH=gfx1030 took the inferred-arch path, which resolves its index
from _infer_linux_amd_gfx_arch() and so believed the override, force-installed the
multi-GB gfx103X-all stack, and set _inferred_arch_installed, which skips the
runtime-target check below it. _ensure_cpu_torch() then saw the real gfx1033 and
force-reinstalled CPU torch, so every `studio update` paid for both installs instead
of settling.
"""
calls = []
monkeypatch.setattr(stack_mod, "IS_WINDOWS", False)
monkeypatch.setattr(stack_mod, "IS_MACOS", False)
monkeypatch.setattr(stack_mod, "NO_TORCH", False)
monkeypatch.setattr(stack_mod, "_TORCH_BACKEND", "")
monkeypatch.setattr(stack_mod, "_has_usable_nvidia_gpu", lambda: False)
monkeypatch.setattr(stack_mod, "_has_rocm_gpu", lambda: False)
# The real silicon, from a source the override cannot move.
monkeypatch.setattr(stack_mod, "_detect_amd_gfx_codes", lambda **_kw: ["gfx1033"])
monkeypatch.setattr(stack_mod, "_kfd_gfx_targets", lambda: [])
monkeypatch.setattr(stack_mod, "pip_install", lambda *a, **k: calls.append(a))
monkeypatch.setattr(stack_mod, "pip_install_try", lambda *a, **k: True)
monkeypatch.setenv("UNSLOTH_ROCM_GFX_ARCH", "gfx1030")
for _var in ("UNSLOTH_TORCH_INDEX_URL", "UNSLOTH_TORCH_INDEX_FAMILY"):
monkeypatch.delenv(_var, raising = False)
_probe = MagicMock(returncode = 0, stdout = _MARK + "2.10.0+cpu||\n")
with patch("os.path.isdir", return_value = True):
with patch("subprocess.run", return_value = _probe):
stack_mod._ensure_rocm_torch()
assert calls == [], f"a stale override still bought the ROCm stack: {calls}"
def test_probe_is_skipped_when_no_rocm_torch_is_installed(self, monkeypatch):
"""The arch probes run subprocesses; they must not run on every ordinary install."""
monkeypatch.setattr(stack_mod, "IS_WINDOWS", False)
monkeypatch.setattr(stack_mod, "IS_MACOS", False)
monkeypatch.setattr(stack_mod, "_installed_torch_label_on_disk", lambda: "2.10.0+cu128")
def _boom(**_kw):
raise AssertionError("probed the AMD arch on a host with no ROCm torch")
monkeypatch.setattr(stack_mod, "_detect_amd_gfx_codes", _boom)
monkeypatch.setattr(stack_mod, "_infer_linux_amd_gfx_arch", _boom)
monkeypatch.setattr(stack_mod, "_kfd_gfx_targets", _boom)
for _var in ("UNSLOTH_TORCH_INDEX_URL", "UNSLOTH_TORCH_INDEX_FAMILY"):
monkeypatch.delenv(_var, raising = False)
assert stack_mod._rocm_miscomputing_host() is False
if __name__ == "__main__":
pytest.main([__file__, "-v"])

89
tests/studio/install/test_selection_logic.py
View file

@ -2668,6 +2668,95 @@ class TestLinuxPublishedAttemptsNvidiaCpuGate:
attempts = INSTALL_LLAMA_PREBUILT._linux_published_attempts(host, self._cpu_only_bundle())
assert [a.install_kind for a in attempts] == ["linux-cpu"]
def _vulkan_and_cpu_bundle(self):
"""What unslothai/llama.cpp actually publishes: both an x64 Vulkan bundle and
an x64 CPU one (app-<tag>-linux-x64-vulkan.tar.gz / -cpu.tar.gz)."""
return make_release(
[
make_artifact(
"app-b8508-linux-x64-vulkan.tar.gz",
install_kind = "linux-vulkan",
runtime_line = None,
coverage_class = None,
supported_sms = [],
min_sm = None,
max_sm = None,
bundle_profile = "linux-vulkan-x64",
rank = 500,
),
make_artifact(
"app-b8508-linux-x64-cpu.tar.gz",
install_kind = "linux-cpu",
runtime_line = None,
coverage_class = None,
supported_sms = [],
min_sm = None,
max_sm = None,
bundle_profile = None,
rank = 1000,
),
]
)
def _gpu_host(self, **overrides):
base = dict(
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = False,
)
base.update(overrides)
return make_host(**base)
def test_amd_without_rocm_gets_the_published_vulkan_bundle(self):
"""The regression this guards: an AMD host with no usable ROCm silently took the
CPU bundle.
The Vulkan preference was stated in direct_upstream_release_plan and in
resolve_upstream_asset_choice, but NOT here, and this is the branch that runs whenever a
published bundle exists. So a Steam Deck installed the CPU llama.cpp while both upstream
paths said Vulkan, reachable only by setting UNSLOTH_LLAMA_CPP_BACKEND by hand.
"""
host = self._gpu_host(has_amd_gpu_without_rocm = True)
attempts = INSTALL_LLAMA_PREBUILT._linux_published_attempts(
host, self._vulkan_and_cpu_bundle()
)
# Vulkan first, CPU kept behind it: if the Vulkan bundle fails validation the
# host still gets a working engine rather than nothing.
assert [a.install_kind for a in attempts] == ["linux-vulkan", "linux-cpu"]
def test_intel_routing_is_unchanged(self):
host = self._gpu_host(has_intel_gpu = True)
attempts = INSTALL_LLAMA_PREBUILT._linux_published_attempts(
host, self._vulkan_and_cpu_bundle()
)
assert [a.install_kind for a in attempts] == ["linux-vulkan", "linux-cpu"]
def test_plain_cpu_host_never_reaches_vulkan(self):
host = self._gpu_host()
attempts = INSTALL_LLAMA_PREBUILT._linux_published_attempts(
host, self._vulkan_and_cpu_bundle()
)
assert [a.install_kind for a in attempts] == ["linux-cpu"]
def test_amd_next_to_a_physical_nvidia_does_not_reach_vulkan(self):
"""Vulkan ignores CUDA_VISIBLE_DEVICES, so a hidden NVIDIA card must not be
enumerated through it -- the same reason the other two paths test physical
rather than usable NVIDIA."""
host = self._gpu_host(has_amd_gpu_without_rocm = True, has_physical_nvidia = True)
attempts = INSTALL_LLAMA_PREBUILT._linux_published_attempts(
host, self._vulkan_and_cpu_bundle()
)
assert "linux-vulkan" not in [a.install_kind for a in attempts]
def test_amd_without_rocm_falls_back_to_cpu_when_no_vulkan_bundle_exists(self):
"""An older release that shipped no Vulkan bundle must still install something."""
host = self._gpu_host(has_amd_gpu_without_rocm = True)
attempts = INSTALL_LLAMA_PREBUILT._linux_published_attempts(host, self._cpu_only_bundle())
assert [a.install_kind for a in attempts] == ["linux-cpu"]
# ===========================================================================
# N.1d. published_windows_cuda_attempts -- version-dynamic ordering seed

262
tests/studio/test_amd_no_rocm_vulkan_routing.py Normal file
View file

@ -0,0 +1,262 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Guards Vulkan routing for Linux AMD GPUs with no usable ROCm.
Every AMD branch in the selector was gated on has_rocm and the Vulkan branch required
has_intel_gpu, so an AMD GPU without working ROCm looked like a headless CPU-only box and
fell through to the CPU llama.cpp bundle. ROCm does not support most AMD parts, APUs least
of all, and many distros ship Mesa/RADV without it, so that population is large.
Widening is safe because the Vulkan bundle is a SUPERSET of the CPU one: it ships the same
libggml-cpu-*.so variants alongside libggml-vulkan.so, and with no usable Vulkan device it
enumerates zero devices and runs on the CPU backend (verified with `--list-devices` and the
Vulkan ICD removed: nothing listed, exit 0).
Measured on a Steam Deck (Van Gogh gfx1033, Qwen2.5-0.5B Q4_0, llama-bench, pp128/tg64):
Vulkan (RADV) 1444.80 pp / 112.81 tg
CPU 753.08 pp / 49.76 tg
ROCm (gfx103X prebuilt via HSA_OVERRIDE_GFX_VERSION=10.3.0)
802.06 pp / 17.51 tg
The contract:
* AMD GPU + no usable ROCm -> Vulkan bundle (was: CPU)
* AMD GPU + working ROCm -> unchanged; detect_host() only probes DRM vendor ids
when ROCm is absent, so the flag stays False there
and the ROCm branches keep the host
* Any host + PHYSICAL NVIDIA -> never Vulkan (it ignores CUDA_VISIBLE_DEVICES and
would enumerate a deliberately hidden card)
* No GPU at all -> unchanged (CPU bundle)
* force_cpu -> clears the flag, so CPU still wins
"""
from __future__ import annotations
import dataclasses
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from studio import install_llama_prebuilt as ilp # noqa: E402
def _host(**overrides):
"""A Linux x86_64 host with no NVIDIA and no ROCm, tweaked per test."""
base = dict(
system = "Linux",
machine = "x86_64",
is_windows = False,
is_linux = True,
is_macos = False,
is_x86_64 = True,
is_arm64 = False,
nvidia_smi = None,
driver_cuda_version = None,
compute_caps = [],
visible_cuda_devices = None,
has_physical_nvidia = False,
has_usable_nvidia = False,
has_rocm = False,
has_intel_gpu = False,
has_amd_gpu_without_rocm = False,
)
base.update(overrides)
fields = {f.name for f in dataclasses.fields(ilp.HostInfo)}
return ilp.HostInfo(**{k: v for k, v in base.items() if k in fields})
def _patch_drm(monkeypatch, tmp_path, vendor_ids):
"""Fake /sys/class/drm/card*/device/vendor for the given vendor ids."""
files = []
for index, vendor in enumerate(vendor_ids):
card = tmp_path / f"card{index}" / "device"
card.mkdir(parents = True)
(card / "vendor").write_text(vendor + "\n")
files.append(str(card / "vendor"))
monkeypatch.setattr(ilp.glob, "glob", lambda _pat: files)
_HOST_GPU_TOOLS = frozenset(
{"nvidia-smi", "rocminfo", "amd-smi", "rocm_agent_enumerator", "hipconfig"}
)
def _patch_no_nvidia_no_rocm(monkeypatch):
"""Hide the REAL host's NVIDIA/ROCm from detect_host().
The DRM vendor pass is gated on `not has_usable_nvidia and not has_rocm`, so on any GPU
machine the fake sysfs is never read and every case collapses to False. Stubs the probes
that see through to the host: nvidia-smi/rocminfo on PATH, the /proc/driver/nvidia/gpus
fallback, and os.access, which detect_host() uses for /opt/rocm/bin/rocminfo when
rocminfo is not on PATH.
"""
_which = ilp.shutil.which
monkeypatch.setattr(
ilp.shutil,
"which",
lambda name, *a, **k: None if name in _HOST_GPU_TOOLS else _which(name, *a, **k),
)
_isdir = ilp.os.path.isdir
monkeypatch.setattr(
ilp.os.path,
"isdir",
lambda p: False if "nvidia" in str(p) else _isdir(p),
)
_exists = ilp.os.path.exists
monkeypatch.setattr(
ilp.os.path,
"exists",
lambda p: False if ("rocm" in str(p) or "kfd" in str(p)) else _exists(p),
)
_access = ilp.os.access
monkeypatch.setattr(
ilp.os,
"access",
lambda p, mode, *a, **k: False if "rocm" in str(p) else _access(p, mode, *a, **k),
)
def test_hostinfo_exposes_the_flag():
assert "has_amd_gpu_without_rocm" in {f.name for f in dataclasses.fields(ilp.HostInfo)}
assert _host().has_amd_gpu_without_rocm is False
@pytest.mark.parametrize(
"vendor_ids, expect_amd, expect_intel",
[
(["0x1002"], True, False), # AMD only
(["0x8086"], False, True), # Intel only
(["0x8086", "0x1002"], True, True), # Intel iGPU + AMD dGPU: both seen
(["0x1002", "0x8086"], True, True), # order must not matter (no early break)
(["0x10de"], False, False), # NVIDIA vendor id is not ours to claim
([], False, False), # headless
],
)
def test_drm_vendor_detection(monkeypatch, tmp_path, vendor_ids, expect_amd, expect_intel):
"""Both vendors come from one sysfs pass; an AMD dGPU next to an Intel iGPU must not
be lost to an early break."""
_patch_drm(monkeypatch, tmp_path, vendor_ids)
_patch_no_nvidia_no_rocm(monkeypatch)
monkeypatch.setattr(ilp.platform, "system", lambda: "Linux")
host = ilp.detect_host()
assert host.has_amd_gpu_without_rocm is expect_amd
assert host.has_intel_gpu is expect_intel
@pytest.mark.parametrize(
"env, expect_amd",
[
({}, True), # the target host: driver-only AMD, no mask
# ROCR_VISIBLE_DEVICES is a GPU isolation request and Vulkan honours no HIP mask, so
# auto-routing would hand llama.cpp the card it hid. Honoured on its own terms: no
# ROCm inventory tool need be installed for it to be real.
({"ROCR_VISIBLE_DEVICES": ""}, False),
({"ROCR_VISIBLE_DEVICES": "-1"}, False),
({"ROCR_VISIBLE_DEVICES": "0"}, False),
# Only ROCR_VISIBLE_DEVICES empties an HSA agent list. HIP_VISIBLE_DEVICES and its
# CUDA_VISIBLE_DEVICES alias filter the HIP runtime, which rocminfo is no client of;
# counting them denied Vulkan to any host merely exporting CUDA_VISIBLE_DEVICES.
({"HIP_VISIBLE_DEVICES": "-1"}, True),
({"CUDA_VISIBLE_DEVICES": ""}, True),
({"CUDA_VISIBLE_DEVICES": "0"}, True),
# Both set: ROCR still decides, so the isolation request is honoured.
({"CUDA_VISIBLE_DEVICES": "0", "ROCR_VISIBLE_DEVICES": ""}, False),
],
)
def test_amd_visibility_mask_suppresses_auto_vulkan(monkeypatch, tmp_path, env, expect_amd):
_patch_drm(monkeypatch, tmp_path, ["0x1002"])
_patch_no_nvidia_no_rocm(monkeypatch)
monkeypatch.setattr(ilp.platform, "system", lambda: "Linux")
for _var in ("HIP_VISIBLE_DEVICES", "ROCR_VISIBLE_DEVICES", "CUDA_VISIBLE_DEVICES"):
monkeypatch.delenv(_var, raising = False)
for _var, _val in env.items():
monkeypatch.setenv(_var, _val)
assert ilp.detect_host().has_amd_gpu_without_rocm is expect_amd
def test_the_host_stub_also_hides_an_unexported_opt_rocm(monkeypatch, tmp_path):
"""rocminfo off PATH but present under /opt/rocm must not reach these cases.
detect_host() falls back to os.access("/opt/rocm/bin/rocminfo", os.X_OK) when
shutil.which finds nothing, which os.path.exists and os.path.isdir do not cover. The stub
has to answer for that machine, or the cases above take has_rocm from the real host and
stop testing the AMD-without-ROCm path.
"""
_patch_drm(monkeypatch, tmp_path, ["0x1002"])
monkeypatch.setattr(ilp.platform, "system", lambda: "Linux")
for _var in ("HIP_VISIBLE_DEVICES", "ROCR_VISIBLE_DEVICES", "CUDA_VISIBLE_DEVICES"):
monkeypatch.delenv(_var, raising = False)
_access = ilp.os.access
monkeypatch.setattr(
ilp.os,
"access",
lambda p, mode, *a, **k: True if "rocm" in str(p) else _access(p, mode, *a, **k),
)
assert ilp.os.access("/opt/rocm/bin/rocminfo", ilp.os.X_OK) is True
_patch_no_nvidia_no_rocm(monkeypatch)
# Asserted directly, not just via the outcome: with the fallback reachable the probe below
# only fails to execute a binary this runner lacks, so it would be green for the wrong reason.
assert ilp.os.access("/opt/rocm/bin/rocminfo", ilp.os.X_OK) is False
assert ilp.detect_host().has_amd_gpu_without_rocm is True
def test_intel_detection_survives_an_amd_mask(monkeypatch, tmp_path):
"""The masks address HIP devices; an Intel iGPU keeps its own Vulkan route."""
_patch_drm(monkeypatch, tmp_path, ["0x8086", "0x1002"])
_patch_no_nvidia_no_rocm(monkeypatch)
monkeypatch.setattr(ilp.platform, "system", lambda: "Linux")
monkeypatch.setenv("ROCR_VISIBLE_DEVICES", "")
host = ilp.detect_host()
assert host.has_intel_gpu is True
assert host.has_amd_gpu_without_rocm is False
def test_force_cpu_clears_the_flag():
"""--force-cpu must not leak out through the new Vulkan arm."""
forced = ilp._apply_host_overrides(_host(has_amd_gpu_without_rocm = True), force_cpu = True)
assert (
forced.has_amd_gpu_without_rocm is False
), "force_cpu left the flag set; CPU would be bypassed"
assert forced.has_intel_gpu is False
assert forced.has_rocm is False
@pytest.mark.parametrize(
"host_kwargs, expect_vulkan_eligible",
[
(dict(has_amd_gpu_without_rocm = True), True), # the widening
(dict(has_intel_gpu = True), True), # unchanged
(dict(has_amd_gpu_without_rocm = True, has_physical_nvidia = True), False), # hidden CUDA card
(dict(), False), # headless / CPU-only
],
)
def test_vulkan_eligibility_gate(host_kwargs, expect_vulkan_eligible):
"""Mirrors the guard used by the Linux x86_64 dispatch branches."""
host = _host(**host_kwargs)
eligible = (
host.has_intel_gpu or host.has_amd_gpu_without_rocm
) and not host.has_physical_nvidia
assert eligible is expect_vulkan_eligible
def test_rocm_host_keeps_the_rocm_branch():
"""detect_host() only probes DRM vendor ids when ROCm is absent, so a working-ROCm
host keeps the flag False and stays on the ROCm branches."""
assert _host(has_rocm = True, rocm_gfx_target = "gfx1100").has_amd_gpu_without_rocm is False
def test_vendor_probe_is_gated_off_rocm_in_source():
"""Belt and braces on the above, in the source rather than a monkeypatch that could pass
vacuously: the DRM vendor scan must sit inside the `not has_usable_nvidia and not has_rocm`
guard, or a working-ROCm host starts setting the flag and can leave the ROCm branches."""
src = Path(ilp.__file__).read_text(encoding = "utf-8")
guard = "if not has_usable_nvidia and not has_rocm:"
assert guard in src
after_guard = src.index(guard)
scan = src.index('_vendor_id == "0x1002"')
assert scan > after_guard, "AMD vendor scan is no longer inside the no-ROCm guard"
# ...and that nothing re-enables it later outside the guard.
assert src.count('_vendor_id == "0x1002"') == 1