mirror of
https://github.com/unslothai/unsloth.git
synced 2026-10-14 21:31:02 +00:00
6df2e235a3
* Studio: fine-tune Laya decision models and serve them * Check decision checkpoints at their subfolder, refuse DoRA/LoftQ, steady the toy test * Turn gradient checkpointing back on in FastDecisionModel.for_training * Merge decision LoRA on a CPU copy, check decision subfolders before the repo root * Log decision runs to TensorBoard when it is enabled * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Block a decision run whose model config failed to load, gate the GPU memory test on a real CUDA device * Use the AGPL-3.0 SPDX header on the new decision model files * Record the vendored laya import in the risky loader baseline * Train Cloudflare Clef decision models with FastDecisionModel * Install the Qwen3.5 fast-path kernels for Clef decision runs, test they engage * Studio: train and serve Cloudflare Clef decision models Clef and Clef-flash repos (config.json plus joint_head.*) are detected as decision models next to Laya. Training goes through FastDecisionModel with QLoRA allowed and the Qwen3.5 fast-path kernel setup the worker skips for Laya. The Decision API lists clef and clef-flash and Clef fine-tunes, and serves them from a spawn child that loads the backbone through Unsloth, so unloading returns the GPU memory. Clef waits for the GPU while a run trains instead of loading on CPU. * Keep Clef off fp16 autocast and fp16 weights * Vendor Cloudflare's Clef joint_schema_model.py unmodified * Drop the decision trainer's backward rescale: accelerate never scales the loss here * Run the decision toy task on CPU, where its dropout stream is fixed * Fold Clef calibration into the saved head; add label smoothing and Brier loss options * Build Clef's encoding and head on the vendored joint_schema_model.py * Check Clef reloads against the folded temperature; skip Clef forwards on CPU when causal_conv1d is installed * Batch the Clef head, pool its fp32 norm in chunks, checkpoint and compile it * Record QLoRA in Clef run metadata, stub decision_layout in the export route test * Decision runs: hold out from training when eval files are training files, honour a cancel during calibration * Serve Clef with the trainer's temperature rule, incl. option-count buckets * Fetch a decision checkpoint by its layout so Unsloth's download check accepts it * Save decision models crash-safely and fold Clef's calibrated temperature into its head * Load Clef in Unsloth's dynamic 4-bit and train it in float16 and bfloat16 * Add Cloudflare's Clef recipe options and record accuracy, drop the no-op accumulation wrapper * Run the toy learning test on CPU, where its step budget was tuned * Format the decision model changes * Run the Clef head under torch's checkpoint and autocast's matmul dtype; add a train bench * Refuse Clef on machines without an NVIDIA or AMD GPU, and mark it unavailable in Settings * Read Clef availability in Settings from finished device detection only * Keep a Clef save's tokenizer files and config version as the source has them * Score the Clef bench's head parity against an fp32 per-record oracle * State the AGPL header and Cloudflare derivation notice on clef.py * Keep the Clef head's norm op off Unsloth's patched layer_norm; compile test after a model load * Keep Clef's logit scales float32 on save and fit its temperature with a line search * Skip the bfloat16 Clef training test on GPUs without bfloat16 * Read the Qwen3.5 kernel a layer picked on transformers before 5.6 too * Apply the gradient accumulation fix for Laya training on transformers 5.0 to 5.5 * Review the vendored Clef config read and skip the Clef head compile test on CPU * Name the Clef norm-pool fake kernels instead of reusing _ * Serve Clef with the calibration autocast rule, calibrate a one-row holdout, and give Clef full fine-tunes the full fine-tuning rate * Send Clef's recipe optimizer instead of forcing torch AdamW on every decision run * Stop a Clef worker that times out loading or that training overtakes * Count every Clef question in the held-out decision count * Never reuse a Clef worker after a timeout or a dead pipe * Detect a local decision model's layout in the selected subfolder * Refuse KL regularization for a fully fine-tuned Clef at construction * Plan decision runs on the one GPU they use and keep an env-pinned local Clef's layout * Keep full Clef fine-tunes unquantized, save a resized Laya vocabulary, and stop flagging exact-fit Clef prompts as truncated * Stand in for unsloth in the Clef truncation test on Studio's backend CI * Load Clef backbones on one device when no device_map is given, and make decision tests portable to multi-GPU, Windows and Apple Silicon hosts * Score Clef metric passes in the run's batch size and drop the parent run's training record when loading a Clef checkpoint * Refuse Clef LoRA options it cannot apply and stop a loading Clef worker as soon as training starts --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
51 lines
2.3 KiB
YAML
51 lines
2.3 KiB
YAML
# pre-commit.ci reads its settings from here and only from here. These lived in a
|
|
# root .pre-commit-ci.yaml for ten months, which is not a file pre-commit.ci looks
|
|
# at, so none of them applied: every autoupdate commit on main used the default
|
|
# message and arrived weekly, not monthly.
|
|
#
|
|
# autofix_prs is the default, but it is stated because the sync-allow-scripts-pins
|
|
# hook below depends on it by name. skip and submodules are left out: they are
|
|
# defaults that guard nothing, and unused settings are what produced this bug.
|
|
ci:
|
|
autofix_prs: true
|
|
autoupdate_schedule: monthly
|
|
autoupdate_commit_msg: "chore: pre-commit autoupdate"
|
|
|
|
repos:
|
|
- repo: https://github.com/astral-sh/ruff-pre-commit
|
|
rev: v0.16.10
|
|
hooks:
|
|
- id: ruff
|
|
args:
|
|
- --fix
|
|
- --exit-non-zero-on-fix
|
|
exclude: '\.ipynb$'
|
|
- repo: local
|
|
hooks:
|
|
- id: ruff-format-with-kwargs
|
|
name: Ruff format with kwarg spacing
|
|
# `python <script>` not a direct exec, for the reason spelled out on the
|
|
# hook below: an autofix commit can drop the executable bit, and a
|
|
# shebang-style entry then dies with "Executable not found". That is
|
|
# twice now (#10519, #10523); this makes the mode bit irrelevant.
|
|
entry: python scripts/run_ruff_format.py
|
|
language: python
|
|
types: [python]
|
|
# Mirror ruff's [tool.ruff] extend-exclude so this hook does not
|
|
# half-process files ruff itself skips (which produced churn). vendor/ is
|
|
# repeated here because this hook does not read ruff's config.
|
|
exclude: '(chat_templates|ollama_template_mappers|_auto_install|mapper)\.py$|^studio/backend/vendor/|^unsloth/_vendor/'
|
|
additional_dependencies:
|
|
- ruff==0.6.9
|
|
# Re-pins allowScripts entries after dependency bumps. pre-commit.ci
|
|
# pushes the fix to PR branches, Dependabot's included, so stale pins
|
|
# heal without a human in the loop.
|
|
- id: sync-allow-scripts-pins
|
|
name: Sync allowScripts pins with the frontend lockfile
|
|
# `python <script>` not a direct exec: autofix commits can drop the
|
|
# executable bit, which kills shebang-style entries.
|
|
entry: python scripts/sync_allow_scripts_pins.py
|
|
args: [--fix]
|
|
language: python
|
|
files: ^studio/frontend/(package\.json|package-lock\.json)$
|
|
pass_filenames: false
|