mirror of
https://github.com/unslothai/unsloth.git
synced 2026-10-21 14:00:32 +00:00
edd81b4f8d
* Fix silent dropping of embed_tokens/lm_head when listed in target_modules [pre-commit.ci skip] * fix: preserve embedding target scoping * fix: retain lone lm head LoRA target * fix: preserve fast inference lm head targets * fix: preserve FastBase embedding targets * fix: normalize embedding target names * test: skip embedding redirect integration without GPU * fix: reject embedding saves with fast inference * Fix embed_tokens/lm_head redirect for PR #9331 LoRA on either module never trains in Unsloth, so redirect both to modules_to_save unconditionally instead of preserving a dead LoRA target, and raise a clear error when that leaves target_modules empty. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep tied embeddings tied and fix Studio CPT for PR #9331 Redirecting embed_tokens and lm_head into modules_to_save gives PEFT two independent copies, so on a tied model they diverge during training and the merge keeps only one (measured: the trained lm_head was dropped). Default ensure_weight_tying to True when we redirect a tied pair, guarded for older PEFT releases that lack the argument. Studio's CPT path put lm_head in target_modules, which the redirect would empty out; route both embedding modules to modules_to_save instead, and let the VRAM estimator count them as full matrices rather than LoRA pairs. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Tie embeddings whenever both land in modules_to_save, and allow target_parameters Key ensure_weight_tying on the resolved modules_to_save rather than on what the redirect moved, so a caller that routes both names itself (Studio's CPT path) also gets the tie instead of two divergent copies whose merge drops one. The empty-target guard now accepts target_parameters, which PEFT treats as a LoRA target in its own right and accepts with no target_modules at all. * Tighten the comments this PR adds * Count a tied embedding pair once in the VRAM estimate ensure_weight_tying collapses a tied embed_tokens/lm_head pair to one trainable matrix, so charging two overestimated VRAM and could reject a viable GPU. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Keep get_peft_model idempotent when lm_head is tied for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cover the explicit modules_to_save form and CPT projection fallback for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Shorten comments for PR #9331 * Leave the tied output module for PEFT to retie for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Only split a real tied pair and normalize all-linear estimates for PR #9331 * Shorten comments for PR #9331 * Keep fast inference working when lm_head is a target for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate the tied-head drop on the model, not the flag, for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Compare a repeat get_peft_model call against the current request for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fold only the embedding alias in the adapter equality check for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Recognise qualified embedding target names for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Shorten comments for PR #9331 * Forward modules_to_save through the audio adapter branches for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Recognise qualified embedding names in Studio CPT and VRAM estimates for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Compare the requested weight-tying mode on a repeat call for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Mirror PEFT's classifier modules on a repeat call for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Normalize the tied embedding name and recognise single-sided tying for PR #9331 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Gate weight tying on the input embedding being saved for PR #9331 * Keep qualified embedding scope on composite models for PR #9331 * Shorten comments for PR #9331 --------- Co-authored-by: imagineer99 <samleejackson0@gmail.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
7 lines
364 B
Python
7 lines
364 B
Python
"""Shared pytest configuration for tests/python/."""
|
|
|
|
|
|
def pytest_configure(config):
|
|
config.addinivalue_line("markers", "server: heavyweight tests requiring studio venv")
|
|
config.addinivalue_line("markers", "e2e: end-to-end tests requiring network and venv creation")
|
|
config.addinivalue_line("markers", "slow: tests needing a GPU and a model download")
|