mirror of
https://github.com/unslothai/unsloth.git
synced 2026-10-19 17:06:43 +00:00
db0e78c150
Fixes #13926. ## Problem #13899 taught the quant regex `NVFP4`, so every file in the reporter's custom folder (`Qwen3.8-27B-NVFP4-MTP-HIGHEST.gguf`, `-LOW`, `-MEDIUM`, `-MID-HIGH`, `-MID-HIGH_enru`) now gets the key `NVFP4`. The lister keeps one file per key, so the picker shows a single `NVFP4` row that loads `-HIGHEST`, and the other four cannot be selected. Before #13899 each file was its own row. Differently named builds at standard quants in custom folders already collapsed the same way, for example `Llama-3-8B-Instruct-Q4_K_M.gguf` beside `Llama-3-8B-abliterated-Q4_K_M.gguf`. ## Fix Outside the HF cache, `list_local_gguf_variants` now keys files with `gguf_variant_keys_by_checkpoint`, which deletion already uses to tell builds that share a quant apart. - The first build keeps the bare quant key, so existing selections still open the same file. Each other build is keyed by its file stem, which the loader already accepts. - Each build is labelled with its file name, for example `NVFP4 · Qwen3.8-27B-NVFP4-MTP-HIGHEST`. - `/gguf-variants` now returns the chosen row's own key as `default_variant`, instead of deriving it again from the filename. Without this, a folder with a torn `a-Q4_K_M-00001-of-00002.gguf` and a whole `z-Q4_K_M.gguf` would default to `Q4_K_M`, the torn build. HF cache snapshots still list one build per quant (#13327), because their keys must match the remote listing and are stored in download manifests and pins. `gguf_variant_key` is unchanged. ## Verification Studio before and after the change, with a custom folder holding the issue's five filenames (copies of a small real GGUF): | | main | this PR | |---|---|---| | Picker, custom folder row | one `NVFP4` row | five `NVFP4 · <build>` rows, all "downloaded" | | Loading each listed key | only `-HIGHEST` reachable | each key started llama-server on its own file | The new tests fail on main and pass with the change. Five existing tests for HF snapshots now build their snapshot under an HF cache path instead of a plain temp directory; their assertions are unchanged. Across 108 GGUF-related backend test files, 8877 passed. One timing test unrelated to GGUF listing failed (`test_tunnel_safe_long_post.py`), and it passes when its file runs alone. ## Before / after A custom LM Studio-style folder holding five copies of a small GGUF under the issue's filenames (`Qwen3.8-27B-NVFP4-MTP-HIGHEST/LOW/MEDIUM/MID-HIGH/MID-HIGH_enru.gguf`), Studio built at the merge base and at this branch. The quant menu goes from one `NVFP4` row to five `NVFP4 · Qwen3.8-27B-NVFP4-MTP-<build>` rows (the menu truncates the build name; `/api/models/gguf-variants` returns the full labels). Loading each listed key started llama-server on that key's own file; before, only `-HIGHEST` was reachable.  --------- Co-authored-by: mahiatlinux <mahiatlinux@users.noreply.github.com> Co-authored-by: carlitogwapito <20962552+carlitogwapito@users.noreply.github.com> Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| routes | ||
| __init__.py | ||
| schemas.py | ||
| service.py | ||
