* Trim comments in tests, scripts and CI helpers
Comment-only pass over 453 changed files under tests/ (excluding the
studiobench harness), scripts/ and .github/scripts/.
17,749 -> 10,635 comment lines (-40.1%)
1,183,079 -> 757,190 comment characters (-36.0%)
Distinctive-fact retention against the base commit: 93.2%. A sample of 100
restoration and deletion decisions was judged by hand: 98 correct.
No comment line exceeds 120 physical characters. Nothing outside comments
changes: no code, docstrings, string literals, licence headers or pragmas.
Verified with an AST code-only signature of every file against the base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Put the moved comments back on the code they describe
The signal-handler rationale in launch.py had been reattached to the
handler registration loop, which has no try/finally, and its except-branch
note had been split into a fragment starting mid-sentence with the noqa
marker duplicated into the prose. The PORT_FILE launcher comments had been
reattached to the _studio_root_id caching test, which does not touch
PORT_FILE.
* Put the section banners back, and use plain dashes in comments
The trim removed the rule-line comments that index these files. Tests locate a
region by finding its banner and end it on the next one, so a missing banner is
a broken anchor rather than lost decoration. Each is restored above the code
line it preceded at 1d2679f; where that anchor is not unique the banner is left
out rather than guessed at.
Em and en dashes are rewritten as plain dashes inside comment tokens only.
String literals are code and are untouched, so the diff stays comment-only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Put reattached comments back on the code they describe
The comment pass moved a number of explanatory comments off the code they
were written for and onto unrelated statements nearby, so they now read as
false or as truncated fragments. This restores each one at its correct site:
- lint_duplicate_definitions: the import-key rationale goes back on the
emitted Finding key instead of being half-pasted onto `implicit =`.
- lockfile_supply_chain_audit: "Empty by default" belongs on the empty
CARGO_IOC_STRINGS tuple, not on the non-empty source allowlist; the
missing-lockfile and traceback notes go back on their own branches.
- scan_packages: "archives landed in dest" describes the successful pip
download, not `if not downloaded:`; the backslash-continuation note goes
back on the continuation check in _scan_line_end.
- interrupted_install_probe: the start_new_session/getpgid note belongs on
the posix branch that calls getpgid, not on the win32 branch.
- virustotal_scan: the missing-secret rationale goes back on the skip path
and the :⚠️: note back on the detection print.
- scan_npm_packages: the rmtree cleanup note goes back on the rmtree.
- check_frontend_dep_removal: one per-key rationale per key again, instead
of five concatenated onto browserslist.
- profile_startup: failed-launch and nothing-measured notes swapped back.
- enforce_kwargs_spacing: fold-rejection reasons back on the guards that
reject for them, and the f-string fold comment back in order.
- tests/conftest: the bitsandbytes broken-wheel note back on the
bitsandbytes handler.
Comments only; no code, string literal, docstring or pragma changes.
* Repair the banner restore: drop duplicate headings, keep the blank line
The scripted restore re-added each section banner without removing the
de-bannered plain heading the trim had left behind, so a heading appeared
twice, once ruled and once bare. It also inserted the banner flush against the
next line of code where the base had a blank line after it, which turns a
section separator into a doc comment for whichever def follows.
Both are the same fault the trim itself was being screened for, introduced by
the remediation rather than by the trim.
* Reattach 12 review comments to the code they describe
Each of these was a rationale that ended up on a neighbouring test or
assertion, or a fragment cut mid-sentence, so it described something
other than the line under it.
- test_gguf_disk_preflight: the 3GB-embeddings arithmetic belongs to the
untied case, not the tied one (2GB there).
- lockfile_supply_chain_audit: reassemble the workflow-command escaping
rationale on the skip branch.
- studio_smoke/multi_turn_chat: restore the retry-policy explanation in
main().
- test_virustotal_scan: restore the advisory-posture rationale on the
scan step.
- test_amd_fastpath_probe: return each ROCm rationale to its parameter
row.
- test_cuda_repair: the Blackwell span note goes back on the Blackwell
test; complete the Kepler sentence.
- test_windows_rocm_bnb_version: realign the sitecustomize / opt-out
rationales with their cases.
- test_dataset_num_proc: the Pool(1) note belongs to the low-memory auto
path.
- test_install_whisper_prebuilt_logic: decoy-DLL, libomp, build-bump and
same-ggml notes back on their own tests; the link copy fallback needs
none.
- playwright_loaded_models_indicator: stale-eject notes back on their
sections and the preference banner restored.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Restore the review comments that were cut mid-sentence or reattached to the wrong code
* Restore the comment tails the 120-column pass cut off
The trim enforced the column limit by chopping the physical line rather than
rewrapping it, so a block's last sentence stopped mid-thought and the clause
carrying the reason was gone:
-# ...fails in a second rather than
+# ...fails in a second rather than after the quantise and the multi-gigabyte save.
-# fp8 / mxfp8 need bf16 weights, so skip non-bf16 Linears;
+# fp8 / mxfp8 need bf16 weights, so skip non-bf16 Linears; nvfp4 handles fp32. Mirrors the runtime gate.
883 blocks across 266 files, restored from the merge base and rewrapped at word
boundaries within 120 columns. Blocks that end on proper punctuation followed by
a new sentence are left alone: those were shortened on purpose, and only a base
that continues the same sentence where the head stops counts as a cut. No word
is invented; every restored word comes from the base. Pragmas, shebangs, coding
lines, licence text and rule-line banners are skipped.
Dangling truncations drop from 336 to 77. Full suite is 65 failed / 18683
passed, matching main exactly, so no test changes behaviour.
* Restore comment tails cut inside a parenthetical or abbreviation
The first repair pass keyed on terminal punctuation, so a block cut after
"(e.g." or mid-code-span looked finished and was left alone. Balance the
brackets and backticks instead, and treat a trailing abbreviation as
mid-sentence:
-# ...skip the whole module up front on runners without it (e.g.
+# ...skip the whole module up front on runners without it (e.g. CPU-only) before importing unsloth, otherwise
+# collection errors instead of producing a skip.
22 further blocks across 20 files. _ts_deps.py is fixed by hand: it had also
lost a sentence from the MIDDLE of the block, so it was not a prefix of the
base text and no prefix rule could reach it.
* Reassemble the review comments that were left scrambled or cut short
Restore the full rationale for the comments the trim pass left as reversed
or truncated fragments, and put the openenv, reasoning-effort and recovery
window notes back on the code they describe.
* Put the scrambled comment blocks back over the code they describe
* Put the inline labels back on the entries they label
* Restore an inline label lost in the merge
* Restore the facts the reworded comments dropped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move the displaced notes back onto the tests they describe
* Re-pair the fixture rationales with their fixtures
* Put the relocated notes back and finish the sentences the trim cut
* Restore the dropped facts and correct the ones that were never true
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Unfuse the merged notes and drop the reformatting the trim smuggled in
* Restore the clauses the trim cut off and unfuse the notes it merged
Thirty sites where a comment was shortened past the point it still said
anything. Most ended on a colon or a dangling word, so the reason the
branch exists was gone while the sentence introducing it stayed. The
rest had absorbed a neighbouring test's note, which left one row
documented as another row's behaviour.
Each restored clause was checked against the code under it rather than
copied back on trust. Three were not restored: a pointer to a
measurement that is not in the function it names, a claim that a missing
import fails as NameError when it fails at collection, and a redundant
restatement in the autoscroll check.
tests/kaggle/t4_smoke/training_evidence.py ends up identical to main,
which is the right outcome for a file whose only trimmed sentence was
load-bearing.
* Give back the halves of the comments the trim cut off
Two hundred and twenty comments across a hundred and twenty files had
been shortened to the sentence that introduces a fact and none of the
fact. A note ending on a colon or on "so" or "carried over" still reads
as a comment and still parses, which is why every structural check has
been passing over them: a truncated comment is a well-formed comment.
Most of these were only findable by asking what a block ENDS on rather
than what it starts with. The seven passes before this one all keyed on
the first line, so they were blind to the commonest damage in the whole
trim. The supply-chain audit had seven "Mini Shai-Hulud May-12 wave:"
headers with the package names deleted, which was the entire content of
each one.
A handful were worse than truncated. A "can't fit on one line, so leave
it" note had moved from the test asserting changed is False onto the one
asserting changed is True; the sm_61 and sm_70 rationales in the CUDA
repair tests were stacked on a test that exercises neither, leaving
their own tests bare; an unpinned-latest note sat on a test that stubs a
working install on disk.
Restored clauses were checked against the code under them, not copied
back on trust. Around a dozen were left out or rewritten because the
base was wrong about the code as it stands: a pip stub credited with an
append it does not do, five exports said to be filled by a loop that
fills one, a __dict__ rationale that argues for the opposite of what the
loop needs, and UI strings quoted as current that the sources no longer
use.
* Put the fused row notes back on the rows they are true of
Ten sites where a comment described a sibling. The torchao skew table
had three rows' notes concatenated onto the 0.18.0+cu130 row; the
revision-forwarding table had the adapter-tokenizer exception one row
below the only row it holds for; the vision LoRA pair had the tied and
untied cases on a single anchor.
Four of these were mine, not the trim's. The 120-column rewrap left an
orphaned continuation line sitting on a class, one repair copied an
untagged-build note onto a fixture that pins a URL and has no untagged
build in it, another duplicated a namespace-package note and pushed the
interrupted-install label onto the wrong row, and a fourth moved a
collision note onto a fixture with a single host, where it is false. A
pass that rewrites 220 comments will get some of them wrong, and these
are the ones it got wrong.
Two reported sites were left alone. The masking note in the VirusTotal
client already carries its full clause, reflowed; the IOC scan comment
reads complete, and its missing "4." went with the numbering on every
other step in that function.
* Move the row and branch notes back onto what they describe
Nineteen sites. The token-count parity table had every row comment one
param too low, so each seeded reasoning style was documented as the next
one; the font-scale contract had two rationales slid down a continue
each; the validator fixture banner for R-INST-001 sat under R-INST-005,
leaving three tests unlabelled and two double-labelled.
Three of these I caused. The parity table was misplaced by the previous
commit, which moved fused notes onto rows without checking the seeds
underneath them; a timing bound landed in a test that has no timing
assertion; a restore note was reinserted between the two commands it
claims to cover instead of above them.
The pacing benchmark had lost its measurement table entirely, including
the #7892 result that is the reason there are two budgets rather than
one headline number. Restored with the columns intact.
* Undo the damage the last four repair passes did
Twenty-five sites, and almost none of them were the trim's. Round nine
was called "move the row and branch notes back onto what they describe",
and it moved a good many onto things they do not describe: the xpu swap
tests had the pinned-mirror rationale on the test that asserts the index
is dropped, the cuda repair file had a whole run of rationales slid one
test upward, and several notes were duplicated onto a neighbour rather
than moved. Where a clause was already correct on its real anchor, the
copy is deleted rather than parked somewhere new.
Separately, this PR had been changing code, and the gate could not see
it. Five files: three where a repair split a tuple across lines to make
room for a restored trailing comment, two where the trim deleted a
trailing comment and then joined up the wrapping that existed only to
carry it. Parentheses and line breaks leave the parse tree untouched, so
an AST comparison calls all of it comments. Every one is back to base
byte for byte, with the comments that justified the wrapping restored.
scripts/code_reflow.py is the check for it: same tokens in a different
arrangement is a code change, whatever the AST says.
* Let ruff rejoin the two lines the shorter comments freed
Both were split only to make room for a trailing comment. Round ten cut
those comments back to what is true of the row, the expressions now fit,
and the formatter joins them. Running it is not optional: pre-commit.ci
was red on exactly these two.
* Delete the stray copies rather than move them again
Five sites. Three were duplicates: the correct sentence still sits on
the test it describes, and a repair pass left a second copy on a
neighbour. Moving those would have been the fourth round of shuffling
one clause between two tests, so the copies are simply gone.
The mislabelling was the point in each case. An empty handover tag and
an absent one are different precedence rules, and the PowerShell 7.5
parsing note had been copied from the empty-string test onto the one
whose variable is unset. A non-zero-exit rationale sat on a parser test
that asserts a successful exit.
The other two are the trim's own: "#5854)," with its subject cut away,
and a regex description that stopped before naming the self. prefix it
matches, which is what explains the narrower filter below it.
* Put the nine slid notes back on what they assert
Seven had moved one branch or one call down. The autocast one had landed
on the case that forces float32 and asserts dtype float16, directly
under a note saying there is no dtype at all; the JAX rationale sat on
the USE_FLAX env-var case, whose module map has no jax in it; the
kill-and-sentinel note sat on a test that only checks a GPU row regex.
Each is back on the assert it describes, with a tenth found in the same
file as the probe timeouts.
Two are the trim's own. A mid-block deletion in the settings-tabs driver
spliced two half sentences into "blocking a / error-handling regression
and was a stale constant"; the macOS slot table lost the only statement
of why macos-14 is absent from a set named for images GitHub still
schedules. The retirement dates are restored, verified against all three
workflows that carry them.
The settings-tabs note is shorter than base on purpose. It now says the
invariant, that blocking a panel must not collapse the dialog, and drops
the tab count, which nothing asserts.
* Move twelve more notes back onto the branch they are about
All twelve had slid onto an adjacent test or branch, and in each the
displaced note contradicted what it now sat on. The CPU-backend note
about a wheel being left alone had landed on the test asserting a repair
runs; the XPU one about a dispatched shard sat on the single-device
case; the redirect-body note sat in a test that never binds served; the
draft-state refusal sat on the test where the target is present and the
run returns 0.
One lost text on the way as well: the missing-target rationale had been
cut to its first two sentences, dropping the reason the draft and
prerelease refusal above it would otherwise degrade into no check.
Also a rewrap that duplicated its own tail instead of replacing it, so
the merge-budget note read "else a sharded own device, not GPU0, else a
sharded model OOMs".
* Return six notes to the test whose behaviour they describe
Eight reports, six defects: two pairs described the same misplacement
from either side.
One was worse than misplaced. "A comment naming a probe is not a probe"
had moved onto the metadata scan, whose predicate matches the raw line
with no comment handling at all, so it claimed an exemption that scan
does not implement. It belongs on the Test-Path scan, the only one that
splits on # before matching. Moved, not reworded.
The rest argued against the test they sat on. The lazy-trl note was on
a stub with no __getattr__ to raise; the bogus-variant note was above a
request sending a valid variant; the imported-TensorFlow note was on
the case that supplies no tensorflow and asserts the flag is cleared;
the 62-token rationale was on the test that asserts 12.
* Move ten more notes onto the statement they explain
Seven had slid off their anchor, and in most the stray was also cut
short, so the fix was to restore the whole sentence where it is true and
delete the fragment. The torchao dependency reason sat on a test about
re-verification rather than on the --no-deps assertion it justifies; the
bitsandbytes heading sat on a test that fetches quantizers_utils.py; the
no-selection latch note sat on the delete block, which asserts nothing
reaches the action.
Two are the trim's own truncations. The stand-aside rationale in the
shortcut driver stopped at a colon, losing why Enter on a focused Deny
button would otherwise be defaulted away into approve. The kwargs-union
case stopped at a semicolon, and the trim had eaten the leading ** from
**kwargs as well.
One was about the driver rather than a test: the JSON note sat beside
the final print, where it described the document, instead of on the
stderr redirection that is what actually keeps stdout clean.
* Restore what the reflow cut, including three markers it mangled
Nine sites. Most are the trim's own truncations: the dill fixture had
lost the property that makes lookup by reference fail, so an otherwise
unexplained __module__ = "builtins" sat there with no reason; the rope
drift test had two clauses spliced into "every buffer keeps scaling in a
buffer"; the playwright guard had lost the three command shapes it
matches and why the regex is not anchored on an engine list.
Three are a shape I had not seen. The reflow rewrote the `#:` variable
documentation markers in the fast-copy constructs to `# :`, and dropped
stray colons into the wrapped prose, leaving "text : node". Those
markers are syntax, not punctuation to be tidied.
One was misplacement rather than truncation. The dotted-import rationale
had moved onto the second copy of its case and lost its tail, leaving a
dangling "and" above a capitalised "Still caught" and the first row bare.
Restored where base has it, stray deleted.
lint_duplicate_definitions self-test still passes, 63 cases.
* Put five drifted comments back on the code they describe
The trim slid several notes onto neighbouring tests, where they describe
something they no longer sit on:
- test_qat.py: the TODO moved off _test_full_model_fake_quantize, whose
leading underscore is what keeps pytest from collecting it, onto a test
that is collected and was never disabled.
- playwright_chat_ime_i18n.py: 6f carried a stray copy of 6e's
compositionstart note above an Enter KeyboardEvent, and had lost its own
rationale for why Enter reports isComposing=false while composingRef is
pinned.
- test_install_node_prebuilt_logic.py: the short-circuit note landed on the
transient-download-failure test, leaving the SHA-enforcement test bare and
dropping the carve-out that a different usable version is still kept for
offline resilience.
- test_chat_autoload_failure_gate.py: the default/namespace note moved onto
the named-import loop, and the mixed-form note explaining that regex's
optional prefix was dropped.
- test_frontend_dep_removal.py: the unquoted CSS url() note moved onto the
quoted fixture, and the HTML package-name-boundary note moved off the
script-src case onto the link-href case.
Comments only; the gate reports code unchanged on all five files.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Keep `import unsloth` working when bitsandbytes is absent
device_type.py already prints "bitsandbytes is not installed - 4bit QLoRA
unallowed, but 16bit and full finetuning works" and clears
ALLOW_BITSANDBYTES / ALLOW_PREQUANTIZED_MODELS, but the import chain then
hard-required the module anyway, so `import unsloth` raised instead.
#7354 made this reachable: the gfx906 install path uninstalls the generic
bitsandbytes wheel (no gfx906 kernels in it), which leaves an MI50 / Radeon VII
host unable to import unsloth at all, not on the 16bit path the message
promises.
- kernels/utils.py: guard the bnb import; bind get_ptr and the five 4bit ctypes
handles to a stub that raises a clear message if a 4bit path is entered.
HAS_CUDA_STREAM stays False, which is the correct route.
- save.py, models/granite.py: guard Bnb_Linear4bit and peft's Linear4bit
(peft exports it only when bnb imported cleanly) with placeholder classes.
Both names only feed isinstance checks, so nothing matching is exact.
- _gpu_init.py: same degradation on the xpu branch as the cuda branch above.
Verified on a Strix Halo (gfx1151, DEVICE_TYPE=hip, torch 2.11.0+rocm7.13.0)
by blocking bitsandbytes with sys.modules["bitsandbytes"] = None, so
find_spec returns None and the import raises exactly as when the package is
absent. Before: ModuleNotFoundError at kernels/utils.py:136. After: import
succeeds, FastLanguageModel/FastModel import, ALLOW_BITSANDBYTES=False,
ALLOW_PREQUANTIZED=False, and the 4bit stub raises with the real cause. With
bitsandbytes present, every binding is unchanged.
New test walks the `import unsloth` module graph with ast and fails on any
unguarded bitsandbytes (or peft Linear4bit) import; verified it catches the
old code. Targeted suites: 702 passed, 18 skipped.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address the review on #7502: zoo coupling, non-hip flags, py3.9 collection
Three findings, each reproduced first and negative-controlled after.
1. The fix still needed an unreleased unsloth_zoo (P1). save.py imported
unsloth_zoo.saving_utils at module scope, and any zoo without the companion
#953 fix imports bitsandbytes there, so `import unsloth` kept failing for a
dependency set pyproject.toml allows. Raising the floor was not an option:
PyPI's newest zoo is 2026.7.6 and #953 is merged but unreleased, so a bump
would break every install today. Both names it pulled in are used only inside
functions, so the import is now lazy at those two call sites, matching what
determine_base_model_source in the same file already does. Verified against a
real pre-#953 zoo checkout with bitsandbytes blocked: import succeeds, and
restoring the eager import reproduces the failure at saving_utils.py:70.
This PR no longer depends on a zoo release.
2. Capability flags were only cleared on hip (P2). device_type.py probed
bitsandbytes inside its DEVICE_TYPE == "hip" branch, so a cuda or xpu host
without bnb imported fine but still reported ALLOW_BITSANDBYTES=True, and the
default load_in_4bit=True path in models/loader.py would select a 4bit
checkpoint before failing. Clear both flags whenever the module is absent, on
every backend, via find_spec so a working install pays nothing. A cuda host
with bnb blocked now reports False/False; with bnb present nothing changes.
3. The new test could not be collected on Python 3.9 (P2). `Path | None` is a
PEP 604 union and requires-python still allows 3.9, so pytest raised
TypeError at import. Added `from __future__ import annotations`. Checked in
real uv venvs on 3.9, 3.10 and 3.13: 2 passed each; removing the future
import reproduces "unsupported operand type(s) for |" on 3.9 only.
The xpu branch in _gpu_init.py needs no separate flag handling now that the
probe is backend-independent.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address the second review on #7502: guarded probe, and 8bit in the same guard
1. The capability probe used find_spec while the fallbacks in kernels/utils.py
and _gpu_init.py treat any import failure as unavailable, so an installed but
unusable wheel would leave ALLOW_BITSANDBYTES true while the kernels had
already bound the stub. Probe with the same guarded import instead, so all
three agree by construction. No new cost on any path: _gpu_init.py already
imports bnb before device_type is reached on cuda, and device_type's own hip
block imports it a few lines later.
Worth recording that the state this prevents is currently unreachable for an
unrelated reason: a broken wheel takes `import unsloth` down earlier, in
transformers/integrations/bitsandbytes.py:20 via
unsloth_zoo/patching_utils.py:680, whichever exception it raises (OSError also
escapes the zoo moe_utils `except ImportError`). So this is correctness for
when those imports get guarded, not an observable fix today.
2. Both loader guards printed for load_in_4bit or load_in_8bit but only cleared
load_in_4bit, so an explicit load_in_8bit=True survived and reached
Transformers, which builds the bnb quantizer and fails there. Clear both. The
message no longer says AMD either: the flag now goes false whenever bnb is
unusable on any backend.
Tests: the probe must not use find_spec, and an ast walk requires every
ALLOW_BITSANDBYTES guard in loader.py to clear both flags, so a third guard
cannot be added with the same omission. Dropping either fix reddens them (1 and
2 failures respectively). 4 passed on 3.9, 3.13 and the ROCm venv; absent and
healthy bnb both stay consistent across hip and cuda.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop the importlib import left over from the find_spec probe on #7502
* Address the third review on #7502: exact-name bypass and a forwarded bnb config
Both findings hold up, so both are fixed.
1. use_exact_model_name=True skipped the guard entirely. load_in_4bit defaults
to True, so on a host without bitsandbytes
FastLanguageModel.from_pretrained(name, use_exact_model_name=True) kept 4bit
set and failed downstream. That option suppresses repo-name remapping and
cannot make bitsandbytes available, so it has no business gating a capability
check. Ungated at both sites.
2. A user-supplied quantization_config survived the fallback. It sets
load_in_4bit/8bit at the top of from_pretrained and stays in kwargs, so
clearing the local flags still let Transformers rebuild the bnb quantizer.
Now dropped as part of the fallback.
One correction to the second suggestion: it cannot be dropped whenever the
fallback runs. quantization_config also carries GPTQ, AWQ, fp8 and torchao
configs, which have nothing to do with bitsandbytes and must reach the loader
untouched. The pop is gated on the config actually requesting load_in_4bit or
load_in_8bit, reusing the same dict/attr probe from the top of the function.
Behaviour, exercising the real guard block against synthetic inputs with
use_exact_model_name=True and bnb unusable:
default 4bit, no cfg 4bit=False 8bit=False
explicit 8bit, no cfg 4bit=False 8bit=False
BitsAndBytesConfig(4bit/8bit) 4bit=False 8bit=False config dropped
dict bnb config 4bit=False 8bit=False config dropped
GPTQ config 4bit=False 8bit=False config SURVIVES
fp8 dict 4bit=False 8bit=False config SURVIVES
Nothing changes when bitsandbytes works: the whole block is inside
`if not ALLOW_BITSANDBYTES`.
Tests: an ast walk requires neither guard to reference use_exact_model_name in
its test, and requires each to pop quantization_config behind a _wants_bnb
check, so an unconditional pop fails too. Re-gating one guard or removing one
pop reddens a test each. 6 passed on 3.9, 3.13 and the ROCm venv.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address the fourth review on #7502: FastModel never reached the 16bit path
Both findings are real, and the second one meant this PR did not actually
deliver what it advertises for FastModel or vision loads. Reproduced first.
1. patch_compiling_bitsandbytes() ran unguarded at the top of
FastModel.from_pretrained, and unsloth_zoo's copy imports bitsandbytes
unconditionally (patching_utils.py:40). So every FastModel call on a
bnb-less host died there, whatever the arguments:
FastModel(load_in_16bit=True) -> ModuleNotFoundError at patching_utils.py:40
FastModel(full_finetuning=True) -> ModuleNotFoundError at patching_utils.py:40
The FastLanguageModel path already wraps this call in try/except with a
warning, and its comment even says "Mirror FastModel" - FastModel was the
unwrapped one. Wrapped it the same way, so behaviour is unchanged wherever
bitsandbytes imports.
2. The mode-exclusivity check ran before the capability fallback. load_in_4bit
defaults to True, so load_in_16bit=True made
int(load_in_4bit) + int(load_in_16bit) == 2 and raised "Can only load in 4bit
or 8bit or 16bit" before the fallback could clear the unavailable 4bit
request. Moved the fallback ahead of that check.
After both, the same three calls get past every bitsandbytes gate and reach
model resolution, failing only on the deliberately fake repo name used by the
probe. Nothing changes when bitsandbytes works: the fallback is still inside
`if not ALLOW_BITSANDBYTES`, and the wrapper only swallows an import that
previously crashed the load.
Tests: the mode check must be preceded by an ALLOW_BITSANDBYTES fallback in the
same function, and no call to patch_compiling_bitsandbytes may sit outside a
try. The ordering assertion is scoped to the enclosing function on purpose - my
first version compared line numbers file-wide, so the other loader's guard
satisfied it and the negative control passed when it should have failed. With
the scoping fixed, moving the fallback back after the mode check reddens it, as
does unwrapping the patch call. 8 passed on 3.9, 3.13 and the ROCm venv.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Daniel Han