vrr/unsloth
2
0
Fork 0
mirror of https://github.com/unslothai/unsloth.git synced 2026-10-19 08:54:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 39

Keep the Docker notebooks and the models they save after docker rm (#13660)

Browse source 
* Keep the Docker notebooks and the models they save after docker rm

* Give run.sh --rocm its own workspace volume

* Mount the workspace volume in the core Jupyter example

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Run the ROCm smoke test without the workspace volume so a rebuilt image tests its own copy

* Mount the workspace volume in the CPU-only and ROCm quick starts

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
This commit is contained in:
Nilay 2026-10-14 22:43:52 +05:30 • committed by GitHub
parent ae842f7635
commit 38ca9963fd
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 95 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -191,6 +191,7 @@ For Windows instructions and more information, read our [Unsloth Docker Guide](h
docker run -d --name unsloth --gpus all --ipc=host \
-p 8000:8000 -p 8888:8888 \
-v "$PWD":/workspace/host \
-v unsloth-workspace:/workspace \
-v "$HOME/.cache/huggingface":/workspace/.cache/huggingface \
-v unsloth-studio:/opt/unsloth-studio \
unsloth/unsloth && docker logs -f unsloth

5
docker/DOCKERHUB-ROCM.md
View file

@ -25,6 +25,7 @@ done
docker run --rm -it \
$GPU_FLAGS \
--ipc=host \
-v unsloth-workspace-rocm:/workspace \
-v "$HOME/.cache/huggingface":/workspace/.cache/huggingface \
unsloth/unsloth-rocm
```
@ -41,6 +42,7 @@ GPU_FLAGS="--device /dev/dxg -e HSA_ENABLE_DXG_DETECTION=1 \
docker run --rm -it \
$GPU_FLAGS \
--ipc=host \
-v unsloth-workspace-rocm:/workspace \
-v "$HOME/.cache/huggingface":/workspace/.cache/huggingface \
unsloth-rocm:latest
```
@ -111,11 +113,12 @@ Included: PyTorch with ROCm, Unsloth, unsloth-zoo, transformers, TRL, PEFT, acce
Not included in `latest`, unlike `unsloth/unsloth`: Unsloth Studio and its web UI, JupyterLab, prebuilt llama.cpp and whisper.cpp, vLLM and xformers. `latest` is a training image.
The `studio` tag adds Unsloth Studio, JupyterLab with the Unsloth notebooks (the `AMD-*` set first) and a CPU llama.cpp for GGUF chat, still without whisper.cpp, vLLM or xformers. Training, the notebooks and the UI use the GPU; GGUF chat runs on the CPU, since a ROCm llama.cpp bundle is per-architecture and would pin the image to one card. Studio's data (accounts, chats, project folders, outputs) lives at `/opt/unsloth-studio`, so mount a volume there to keep it (projects created by an image from before project folders moved to the volume keep their folder under `/root/Documents` inside the container, so copy their files out before `docker rm`), with `GPU_FLAGS` set as in the quick start:
The `studio` tag adds Unsloth Studio, JupyterLab with the Unsloth notebooks (the `AMD-*` set first) and a CPU llama.cpp for GGUF chat, still without whisper.cpp, vLLM or xformers. Training, the notebooks and the UI use the GPU; GGUF chat runs on the CPU, since a ROCm llama.cpp bundle is per-architecture and would pin the image to one card. Studio's data (accounts, chats, project folders, outputs) lives at `/opt/unsloth-studio`, so mount a volume there to keep it, and one on `/workspace` for the notebooks, your edits and what they save (projects created by an image from before project folders moved to the volume keep their folder under `/root/Documents` inside the container, so copy their files out before `docker rm`), with `GPU_FLAGS` set as in the quick start:
```bash
docker run --rm $GPU_FLAGS --ipc=host -p 127.0.0.1:8000:8000 -p 127.0.0.1:8888:8888 \
-v unsloth-studio-rocm:/opt/unsloth-studio \
-v unsloth-workspace-rocm:/workspace \
-v "$HOME/.cache/huggingface":/workspace/.cache/huggingface \
unsloth/unsloth-rocm:studio
```

7
docker/DOCKERHUB.md
View file

@ -12,6 +12,7 @@ docker run -d --name unsloth --gpus all --ipc=host \
--ulimit memlock=-1 --ulimit stack=67108864 \
-p 8000:8000 -p 8888:8888 \
-v "$PWD":/workspace/host \
-v unsloth-workspace:/workspace \
-v "$HOME/.cache/huggingface":/workspace/.cache/huggingface \
-v unsloth-studio:/opt/unsloth-studio \
unsloth/unsloth && docker logs -f unsloth
@ -62,7 +63,7 @@ docker rm -f unsloth # stop and delete the container
docker ps -a # find it again, running or not
```
`docker rm -f` deletes the container, not your work: models stay in the Hugging Face cache, your files in the directory you mounted, and Unsloth Studio's accounts, chats and project folders on the `unsloth-studio` volume, so the next container with the same `-v unsloth-studio:/opt/unsloth-studio` resumes where this one left off, password included. Only what was written inside the container is lost, which includes the folders of projects created by an image from before project folders moved to the volume: those keep their old place under `/root/Documents`, so copy their files out before removing the container. `docker volume rm unsloth-studio` discards the Unsloth Studio data too, and cannot be undone.
`docker rm -f` deletes the container, not your work: models stay in the Hugging Face cache, your files in the directory you mounted, the notebooks with your edits and what they save on the `unsloth-workspace` volume, and Unsloth Studio's accounts, chats and project folders on the `unsloth-studio` volume, so the next container with the same `-v unsloth-studio:/opt/unsloth-studio` resumes where this one left off, password included. Only what was written inside the container is lost, which includes the folders of projects created by an image from before project folders moved to the volume: those keep their old place under `/root/Documents`, so copy their files out before removing the container. `docker volume rm unsloth-studio` discards the Unsloth Studio data too, and cannot be undone.
These take a container, not an image, so `docker stop unsloth/unsloth` fails with "No such container". Use the `NAMES` or `CONTAINER ID` column of `docker ps -a`; without `--name`, Docker assigns a random one.
@ -73,6 +74,7 @@ The `core` image has no service manager. Start JupyterLab on the command line:
```bash
docker run -d --gpus all --ipc=host -p 8888:8888 \
-v "$PWD":/workspace/host \
-v unsloth-workspace:/workspace \
unsloth/unsloth:core \
jupyter lab --ip 0.0.0.0 --port 8888 --allow-root
```
@ -91,7 +93,7 @@ docker run --rm --gpus all --ipc=host -v "$PWD":/workspace/host -w /workspace/ho
`latest` starts without a GPU on its own. Unsloth Studio chat with GGUF models, JupyterLab and the GGUF tooling work; training does not. `core` refuses to start without a GPU unless you opt in:
```bash
docker run -d -p 8000:8000 -p 8888:8888 unsloth/unsloth
docker run -d -p 8000:8000 -p 8888:8888 -v unsloth-workspace:/workspace unsloth/unsloth
docker run --rm -e UNSLOTH_ALLOW_CPU=1 unsloth/unsloth:core python -c "import unsloth"
```
@ -147,6 +149,7 @@ The working directory is `/workspace`. Mount what you want to keep:
| Container path | What it holds |
|---|---|
| `/workspace` | The notebooks, your edits to them and what they save (`lora_model/`, `outputs/`). Use a named volume: without one, `docker rm` loses them. Files the image puts there, such as `smoke_test.py`, keep the first image's copy. |
| `/workspace/host` | Your files. Mount your project directory here. |
| `/workspace/.cache/huggingface` | Model downloads. Mount your host HF cache to reuse it. |
| `/opt/unsloth-studio` | Unsloth Studio's accounts, chats, project folders, outputs, exports and runs (`latest`). Use a named volume: without one, `docker rm` loses them. |

2
docker/build.sh
View file

@ -136,7 +136,7 @@ if [[ $ROCM -eq 1 ]]; then
echo "Built ${IMAGE_NAME}:${TAG}"
echo
echo "Smoke test on an AMD host (run.sh defaults to the published image, so name this one):"
echo " UNSLOTH_IMAGE=${IMAGE_NAME}:${TAG} bash run.sh --rocm python /workspace/smoke_test_rocm.py"
echo " UNSLOTH_IMAGE=${IMAGE_NAME}:${TAG} UNSLOTH_WORKSPACE_VOLUME= bash run.sh --rocm python /workspace/smoke_test_rocm.py"
exit 0
fi

14
docker/run.sh
View file

@ -26,7 +26,8 @@
#
# AMD hosts: a leading --rocm (or UNSLOTH_ROCM=1) runs the ROCm image and passes
# the AMD device nodes instead of --gpus, which is NVIDIA-only.
# bash docker/run.sh --rocm python /workspace/smoke_test_rocm.py
# UNSLOTH_WORKSPACE_VOLUME= bash docker/run.sh --rocm python /workspace/smoke_test_rocm.py
# (no /workspace volume: it would keep the first image's copy of the smoke test)
#
# Overridable env:
# UNSLOTH_IMAGE=unsloth/unsloth:latest image and tag to pull/run
@ -43,6 +44,8 @@
# UNSLOTH_STUDIO_VOLUME=unsloth-studio named volume for Studio's data (accounts,
# chats, outputs) at /opt/unsloth-studio;
# set it empty to run without one
# UNSLOTH_WORKSPACE_VOLUME=unsloth-workspace named volume for /workspace (unsloth-workspace-rocm
# under --rocm); empty to skip it
# UNSLOTH_STUDIO_SHUTDOWN_STOP_TIMEOUT_S=120 how long a training run gets to save a
# checkpoint on docker stop; --stop-timeout is
# set 30s above it
@ -186,6 +189,14 @@ STUDIO_MOUNT=()
if [ -n "$STUDIO_VOLUME" ]; then
STUDIO_MOUNT=(-v "$STUDIO_VOLUME":/opt/unsloth-studio)
fi
# per image family: a new volume keeps the files of the first image that mounts it
WORKSPACE_DEFAULT=unsloth-workspace
[[ $ROCM -eq 1 ]] && WORKSPACE_DEFAULT=unsloth-workspace-rocm
WORKSPACE_VOLUME="${UNSLOTH_WORKSPACE_VOLUME-$WORKSPACE_DEFAULT}"
WORKSPACE_MOUNT=()
if [ -n "$WORKSPACE_VOLUME" ]; then
WORKSPACE_MOUNT=(-v "$WORKSPACE_VOLUME":/workspace)
fi
mkdir -p "$HF_CACHE" "$TRITON_CACHE"
@ -540,6 +551,7 @@ exec docker run --rm ${TTY_FLAG[@]+"${TTY_FLAG[@]}"} \
--stop-timeout "$STOP_TIMEOUT" \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
${WORKSPACE_MOUNT[@]+"${WORKSPACE_MOUNT[@]}"} \
-v "$HF_CACHE":/workspace/.cache/huggingface \
-v "$TRITON_CACHE":/workspace/.cache/triton \
-v "$WORK_DIR":/workspace/host \

2
docker/smoke_test_rocm.py
View file

@ -12,7 +12,7 @@ What this checks (in order, fail-fast):
5. A 5-step LoRA train on a tiny model actually runs forward + backward.
Run inside the container:
bash docker/run.sh --rocm python /workspace/smoke_test_rocm.py
UNSLOTH_WORKSPACE_VOLUME= bash docker/run.sh --rocm python /workspace/smoke_test_rocm.py
Skip step 5 (faster, no model download):
... python /workspace/smoke_test_rocm.py --skip-train

31
tests/python/test_docker_cpu_fallback.py
View file

@ -107,6 +107,7 @@ def _invoke_run_sh(
"UNSLOTH_GPUS",
"UNSLOTH_ALLOW_CPU",
"UNSLOTH_STUDIO_VOLUME",
"UNSLOTH_WORKSPACE_VOLUME",
):
env.pop(leak, None)
env.pop("UNSLOTH_IMAGE", None)
@ -242,6 +243,36 @@ class TestRunShMountsTheStudioVolume:
assert "UNSLOTH_STUDIO_VOLUME=unsloth-studio" in body
def _workspace_mounts(argv):
return [
argv[i + 1] for i, a in enumerate(argv) if a == "-v" and argv[i + 1].endswith(":/workspace")
]
@_posix_shell
class TestRunShKeepsTheWorkspace:
@pytest.mark.parametrize("nvidia, amd", [(True, False), (False, True), (False, False)])
def test_notebooks_and_their_saves_survive_the_container(self, tmp_path, nvidia, amd):
argv, _ = _invoke_run_sh(tmp_path, nvidia = nvidia, amd = amd)
assert _workspace_mounts(argv) == ["unsloth-workspace:/workspace"], argv
def test_an_empty_value_disables_the_mount(self, tmp_path):
argv, _ = _invoke_run_sh(
tmp_path, nvidia = True, amd = False, extra_env = {"UNSLOTH_WORKSPACE_VOLUME": ""}
)
assert _workspace_mounts(argv) == [], argv
def test_the_nested_mounts_are_still_passed(self, tmp_path):
argv, _ = _invoke_run_sh(tmp_path, nvidia = True, amd = False)
targets = [argv[i + 1].rsplit(":", 1)[1] for i, a in enumerate(argv) if a == "-v"]
for nested in (
"/workspace/host",
"/workspace/.cache/huggingface",
"/workspace/.cache/triton",
):
assert nested in targets, argv
class TestStudioImageAllowsCpu:
def test_studio_image_opts_in_through_its_own_variable(self):
""":latest is the Studio image: without an opt-in every CPU-only, AMD and Docker-Desktop user gets an exit 1."""

18
tests/python/test_docker_hub_readme.py
View file

@ -92,6 +92,24 @@ def test_the_hub_readme_explains_the_studio_volume():
assert "https://unsloth.ai/docs/get-started/install/docker" in quick_start
def test_the_quick_starts_keep_the_workspace():
hub = HUB_README.read_text(encoding = "utf-8")
hub_quick_start = hub[hub.index("## Quick start") : hub.index("### Before that")]
running = [
s for s in _docker_sections(REPO_README.read_text(encoding = "utf-8")) if "docker run" in s
]
core = hub[hub.index("### Notebooks only") : hub.index("### Scripts")]
for quick_start in (hub_quick_start, core, *running):
assert "-v unsloth-workspace:/workspace " in quick_start, quick_start
cpu = hub[hub.index("### CPU-only hosts") : hub.index("## Tags")]
assert "-v unsloth-workspace:/workspace unsloth/unsloth\n" in cpu, cpu
rocm = (HUB_README.parent / "DOCKERHUB-ROCM.md").read_text(encoding = "utf-8")
shells = [b for b in rocm.split("```bash")[1:] if "docker run --rm -it" in b]
assert len(shells) == 2 and all(
"-v unsloth-workspace-rocm:/workspace " in b for b in shells
), shells
def _docker_sections(text: str) -> list[str]:
"""Every `#### Docker` section in the README, not just the first one.

21
tests/python/test_docker_rocm.py
View file

@ -109,6 +109,7 @@ def _run_sh(
"UNSLOTH_GPUS",
"UNSLOTH_ALLOW_CPU",
"UNSLOTH_STUDIO_VOLUME",
"UNSLOTH_WORKSPACE_VOLUME",
"UNSLOTH_IMAGE",
"UNSLOTH_ROCM",
"HSA_OVERRIDE_GFX_VERSION",
@ -274,6 +275,26 @@ class TestRunShRocm:
assert "unsloth-studio:/opt/unsloth-studio" in mounts, mounts
assert any(m.endswith(":/workspace/.cache/huggingface") for m in mounts), mounts
def test_the_workspace_volume_is_not_shared_with_the_cuda_image(self, tmp_path):
rocm, _ = _run_sh(tmp_path, ["--rocm", "true"])
cuda, _ = _run_sh(tmp_path, ["true"], nvidia = True)
ws = lambda argv: [
argv[i + 1]
for i, a in enumerate(argv)
if a == "-v" and argv[i + 1].endswith(":/workspace")
]
assert ws(rocm) == ["unsloth-workspace-rocm:/workspace"], rocm
assert ws(cuda) == ["unsloth-workspace:/workspace"], cuda
def test_the_smoke_test_commands_skip_the_workspace_volume(self):
# a kept volume serves the first image's smoke test, not the one just built
for name in ("run.sh", "build.sh", "smoke_test_rocm.py"):
with open(os.path.join(_DOCKER, name), encoding = "utf-8") as f:
lines = [l for l in f if "run.sh --rocm python /workspace/smoke_test_rocm.py" in l]
assert lines, name
for line in lines:
assert "UNSLOTH_WORKSPACE_VOLUME= " in line, (name, line)
def test_a_custom_image_is_kept(self, tmp_path):
argv, _ = _run_sh(tmp_path, ["--rocm", "true"], extra_env = {"UNSLOTH_IMAGE": "me/rocm:dev"})
image, _ = _image_and_cmd(argv)