* Studio: install and update audio.cpp from the newest unslothai/audio.cpp release
audio.cpp was pinned in source (v0.9.0-unsloth.1 plus committed sha256
pins), so an install never moved past it and the in-app update never
offered v0.9.1-unsloth.1. It now follows llama.cpp and whisper.cpp:
- The installer resolves the newest unslothai/audio.cpp release through
prebuilt_core (download host first, then the GitHub API) and checks the
bundle and the Windows CUDA runtime zip against that release's own
audio-cpp-prebuilt-sha256.json, cross-checked with its manifest. An
archive the index does not cover is refused. UNSLOTH_AUDIO_CPP_TAG pins
a release; UNSLOTH_AUDIO_CPP_REPO still picks another repo. The
0xShug0/audio.cpp fallback, the pins file and --write-pins are gone.
- A current install costs one HEAD to github.com and no download; an
older "latest" never replaces a newer install; offline keeps the tree.
The glibc floor comes from the manifest's min_os.
- utils/audio_cpp_freshness.py finds the newest release (24h cache, fails
open) with a version key that tolerates hotfix tags, so the combined
update item offers audio.cpp updates. The update installs exactly the
offered tag (--published-release-tag) and clears the cache afterwards.
The Audio page notice reads the cache only.
- The model family list covers v0.9.0-unsloth.1 and v0.9.1-unsloth.1
(identical model_specs), and the notice now says "<Y> is available".
* Studio: let the in-app audio.cpp update use a user-picked repo, and size it from the offered release
- The in-app update runs the installer with --published-release-tag, which turned off the
GitHub-digest fallback even when UNSLOTH_AUDIO_CPP_REPO / UNSLOTH_AUDIO_CPP_TAG picked the
release. For a repo without a sha256 index (0xShug0/audio.cpp) the offer could never apply
and came back after every attempt. Trust now follows the user's pick for a pinned tag too;
the default unslothai/audio.cpp release always publishes its index.
- The newest tag can come from the 24h disk cache while the asset list is fetched fresh, so
after a newer release the plan could check host coverage and download size against the
wrong release. Only the offered release's assets are used now.
* Studio: match the offered audio.cpp release's assets by their full name prefix
A tag can extend the offered one (upstream published v0.8.2 and later
v0.8.2-audio8-perf-hotfix), so "audio-<tag>-" also matched the newer release's
assets. Match "audio-<tag>-bin-" and "audio-<tag>-cudart-" instead.
* Studio: report a same-tag audio.cpp replacement from another repo as updated
Switching UNSLOTH_AUDIO_CPP_REPO to a repo whose newest release has the installed
tag is offered and does replace the runtime, but the in-app update compared tags
only and said the existing install was kept. Compare the install record's repo
and tag before and after the installer run.
* Studio: do not call an audio.cpp runtime already at the offered release "kept"
When a setup run installs the offered release between the plan and the in-app
install, the record is unchanged across the installer run, but the runtime is
current. Report it as updated.
* Studio: drop the README notes on the audio.cpp release install
* Studio: drop the Audio page audio.cpp update notice; the update card offers it
* Studio: show the installed audio.cpp release in Settings > About
* Studio: fail an in-app audio.cpp update whose release lookup cannot answer instead of keeping the old tree
* Studio: an UNSLOTH_AUDIO_CPP_TAG pin fails instead of keeping another audio.cpp release when its lookup cannot answer
* Studio: cache a failed audio.cpp asset lookup for 60s like the tag lookup
* Studio: refresh the Audio page after the update card unloads audio.cpp models
* Studio: ask for a reload when the audio.cpp update stops a dictation model
* Studio: keep the audio.cpp phase's reload flag to the models it unloads
A stopped dictation sidecar is re-read within 30s on the Transcribe page, while a reload flag makes
the update card clear the chat checkpoint and its RAG setting even though chat stayed loaded.
This reverts e15d1bf57.
* Studio: re-read dictation status whenever an update job settles
* Studio: never downgrade an audio.cpp runtime a setup run updated after the offer
* Studio: re-check the audio.cpp downgrade guard under the installer lock
* Studio: size the audio.cpp CUDA runtime zip by what the installer will download now
* Studio: re-read Audio status once per finished update job on every path
* Studio: size another repo's audio.cpp CUDA bundle with the installer's driver and torch limits
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* fix(studio): preserve chat tool selections across model unloads
* fix(studio): disable research on incompatible local models
* fix(studio): preserve tool preferences across capability changes
* fix(studio): keep unsupported tool pills off without losing the preference
A pill now follows the live store unless the selected model clamps it
off or a model is still loading. In those cases the chat's stored value
stands in, so a clamp never overwrites the preference and the composer
never shows a tool the model cannot run.
- Kimi: a non-persisted Search-off is no longer re-armed from storage,
so opening or leaving a chat keeps Thinking on.
- A settings write during a model load no longer stores the previous
model's clamp over the chat's Search, Code or Research.
- Unloading re-derives the clamped pills from the preference, so a
provider default does not outlive its model.
- Restoring defaults after a held edit, and applying a chat's stored
pills, both go through the capability clamp.
- Revert the out-of-scope composer gating port; keep only the Research
menu gate for local models without tools.
- Re-pin the deep research contract test to the new setCheckpoint rule.
* fix(studio): keep recent pill changes across quick model switches
A pill's recorded value now follows the live store while the chat's
settings write is still waiting on its debounce, the same way that
write reads it, unless the model clamps the pill off or is loading.
It used to keep the value from the moment of the click, so a change
that does not persist, such as Kimi turning Search off when Thinking
is enabled, could be undone by switching models before the write ran.
- Kimi: Search, then Thinking, then a quick switch to another
provider no longer turns Search back on or saves it as on.
- The external model patch reapplied after a cancelled local load no
longer carries a cached Research value, so a Research click made in
between is kept. setCheckpoint and the chat page's provider effect
already set Research for the new model.
- Tests cover both Kimi orderings and the incognito checks on the
Research paths, and the reload matrix only checks the pills a local
load controls.
- The deep research contract test drops two assertions that also
matched upstream and pins the Research menu clause instead.
- Remove comment text that no longer matches the code and restore the
still accurate part of an upstream doc comment.
* fix(studio): align research, legacy chat and reload pills with upstream
Three places where the branch had moved away from upstream behaviour.
- Research support for an external model is read from the providers
store with the same rule the composer menu uses. The previous lookup
parsed the provider list from localStorage on every snapshot and
hydration pass. When that read failed it refused Research while the
menu item stayed enabled, so the click did nothing.
- A chat's own pill override no longer falls back to the default
captured when the chat was opened. Chats saved before the pill keys
existed reach the stored global value and the provider default
again, so picking Anthropic or OpenAI in one turns Search on as
upstream does.
- A same-model reload keeps the Search and Code pills the model was
running with, as upstream does, instead of reading them back from
storage, which can hold a stale value after a failed write.
- Tests cover each case.
* Raise the startup bundle budget by the tool pill growth
* Keep main's routes/inference.py as is
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* feat(studio): re-embed RAG documents after an embedding model switch
Dense search skips documents indexed under a previous embedding model, so after a switch they only match by keyword and the only remedy was deleting and re-uploading them. Add a per-account job that re-encodes each stale document's stored chunk text with the current model and swaps its vectors in place, keeping document and chunk ids so chat and research citations keep resolving. No source file, OCR or captioning is needed.
- `GET /api/rag/reembed` reports stale uploaded documents and archived chat turns plus the job; `POST /api/rag/reembed` starts the job or returns the running one; `POST /api/rag/reembed/cancel` stops it.
- The job follows the current model, so a switch mid-job is absorbed; a failing document is counted and skipped, and three consecutive failures end the job.
- A chunk longer than the new embedder accepts embeds its longest fitting prefix (llama-server rejects inputs past its batch size); stored text, keyword search and highlights are unchanged.
- The swap runs under the write lock and writes only if the document, its chunks and its stored identity are unchanged and the model is still the configured one, read uncached so a switch made by another backend process is seen before a width change could drop the new model's vectors.
- Account retirement stops the job.
* feat(studio): offer Re-embed after an embedding model switch
Saving a new embedding model only warned that existing documents would not use it, with no way to fix that short of deleting and re-uploading them. Surface the re-embed job in the UI.
- After a save that leaves documents stale, the success toast says how many and offers Re-embed; it stays a plain success when nothing is stale or a running job already follows the model, and falls back to the previous warning if status cannot be read.
- Settings > Documents & RAG shows the stale count with a Re-embed button, or progress with Stop while a job runs; accounts without that section see the same row in Settings > Data.
- One shared store drives both surfaces and polls while a job runs, so a job started from the toast shows progress in Settings; its end is announced once.
- A reply sent before the last applied reply arrived, or before a failed start or cancel, may describe an older server state, so it is dropped and replaced by a fresh status read; failed reads keep retrying until one applies, so a lost start reply or a failed Stop cannot leave a running job untracked.
- Strings for every locale.
* fix(studio): keep RAG documents whose vectors a width change dropped re-embeddable
A width change drops the whole vector table, but documents kept their embedding identity, so after cancelling a re-embed or switching back to the previous model they looked current while dense search could no longer find them. `ensure_vec` now stamps every document but the one being written with an identity no embedder matches, so they stay stale and Re-embed rebuilds them; the untagged-document warning ignores the stamp.
* fix(studio): read the re-embed job before the stale counts
A job that ends between the two reads is then still reported running, so the client polls again and gets counts from after its last document instead of keeping stale ones beside a finished job.
* fix(studio): give a document its identity back once its vectors land after a width change
With several ingestion workers, one writer's width change marked another in-flight document that had recorded its identity but not yet written its vectors, so that document completed with valid vectors yet stayed marked and dense search skipped it. The mark is now a prefix that keeps the previous identity, `add_chunks` strips it in the same transaction as its vector inserts, and a second width change does not prefix an already marked document again.
* fix(studio): re-embed archived turns and documents rewritten while they encode
Archived conversation turns never set num_chunks, so the stale scan now checks for chunks instead. A swap refused because another writer changed the row mid-encode is retried once the row has changed, while a document whose encode failed is still not retried under the same model.
* fix(studio): reset the re-embed store on sign-out
Signing out keeps the page loaded, so a running job kept polling on the login page and each 401 cleared the session again. The store now drops its job, polling and status on AUTH_SESSION_CLEARED_EVENT, like the other per-account stores.
* fix(studio): keep a re-embed job registered until retirement has stopped it
A retirement that timed out used to unregister the job anyway, so the retried account deletion found nothing to wait for and could move the account's storage under a live thread. The job now stays registered until its thread has stopped, as ingestion workers do.
* fix(studio): keep the last known embedding model for a fresh settings read that fails
A fresh read discarded the cached value before reading, which disabled the store's fallback to the last known model on a read failure and answered with the default instead. It now skips the cache's TTL but keeps the value for that fallback.
* fix(studio): ignore re-embed requests that settle after sign-out
A reply or failure for a request sent before signing out could re-show the previous account's job, restart polling or show its error toast. Such requests now settle without effect.
* fix(studio): fail a fresh embedding model read instead of falling back
The re-embed swap guard reads the configured model fresh to catch a switch in another process. Answering a failed read with the cached or default model could let a swap pass for a model that is no longer selected. A fresh read now raises, so the swap rolls back and the document is counted as failed and retried by the next run; ordinary reads keep their fallback.
* fix(studio): offer Re-embed in Settings only once the model is on disk
Settings offered the Re-embed action after any save, including a model still waiting for its download, where every document then fails. It now follows the RAG menu's rule and offers it only when the resolve shows the model on disk, and the Settings row disables Re-embed while the download is needed.
* fix(studio): stop a re-embed job at once when the model is not downloaded
A selected model whose files are not on disk makes every document fail the same way, so the job used to fail documents until its failure streak ended it. It now ends on the loader's EmbeddingModelDownloadRequiredError with that message, without counting or touching any document.
* fix(studio): refresh stale counts after every embedding model save
A save whose model still needs its download skipped the status read, so the Settings row kept the previous model's counts. The save toast now always reads them and only withholds the Re-embed action. The Settings row no longer tries to predict model readiness from the picker's state, which follows the candidate rather than the saved model; a job started before the download now stops with the loader's explanation.
* fix(studio): report a new re-embed job as started even if it ends at once
The start reply was snapshotted after the worker started, so a job that ended immediately, such as one stopped because the model is not downloaded, came back already finished and the client never announced it. The snapshot is now taken before the worker runs.
* fix(studio): start the re-embed worker before retirement can see its job
start() registered the job and released the lock before starting its thread, so an account retirement in between joined an unstarted thread and failed the account deletion. The thread now starts under the same lock.
* Test retirement racing a re-embed start
* Raise the startup bundle budget by the re-embed UI's growth
* Tighten re-embed comments
* Read dense search's KNN and identity filter from one snapshot
---------
Co-authored-by: Nilay Yadav <nilayyadav10@gmail.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: deleting a GGUF row no longer deletes other builds at the same quant
A repo publishing several builds at one bare quant (huihui-ai's
...-GSQ-RCO-IQ3_S-mtp beside ...-Swift-1.5-GSQ-RCO-IQ3_S-mtp and
...-UD-DW-IQ3_S) is not supported (#13327): the listers show only the first.
The others can still be in the HF cache, fetched with hf download, and every
destructive path matched files on the bare quant, so deleting the listed
IQ3_S row deleted all of them, and the stale reclaim after updating it read
their blobs as old copies and deleted those.
Destructive paths now key a file of a different checkpoint by its own family,
so the listed row deletes only its own build. Copies of that build (QwQ's -BF16
and .BF16, whole and split) still go with it. The listing is unchanged. Covers
the cache delete, the local-folder delete, the delete preview and the stale
reclaim.
* Fix GGUF deletion checkpoint selection
* Trim comments in PR scope
* Correct GGUF delete preview for shared blobs
* Take the row's build from the newest snapshot when keying deletions
Keyed per snapshot alone, another build that is the only one at its quant in
an older snapshot (an hf download made before the repo moved on) became that
snapshot's representative, so deleting the listed row deleted it too, and a
stale reclaim after updating the row deleted its blob. The row's build now
comes from the newest snapshot holding the key, the one the lister shows, and
every snapshot deletes that build only: its older copies still go, other
builds stay.
* Keep GGUF deletion aligned with complete snapshots
* Trim comments in expanded PR scope
* Keep builds under different quant-named directories apart when keying deletions
Checkpoint identity dropped every quant-named directory, so
alpha-Q4_K_M/model.gguf and beta-Q4_K_M/model.gguf read as copies of one build
and deleting the listed alpha row deleted beta too. A quant-named directory now
keeps its other words, unless they only repeat the file's own name, so
DeepSeek-R1-Q4_K_M/DeepSeek-R1-Q4_K_M-* and Q6_K/ beside model-Q6_K/ still fold.
* Align GGUF deletion with per-quant cache rows
* Trim comments after GGUF cache fix
* Honor GGUF cache readiness when deleting rows
* Clarify GGUF deletion selection comments
* Align GGUF scoped deletion and package previews
* Trim comments in PR scope
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Preserve GGUF sibling partial state
* Authenticate GGUF delete previews
* Preserve unmanifested sibling partials
* Protect quant-only sibling state
* Normalize GGUF row keys consistently
* Align case-folded GGUF completeness
* Trim comments in PR scope
* Trim restating comments in PR scope
---------
Co-authored-by: mahiatlinux <mahiatlinux@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: render numbers written as $3$ in a reply as maths
* Studio: keep $1$HOME as text, not maths
* Preserve braced shell parameters after numeric variables
* Trim comments in PR scope
* Protect all adjacent shell parameter forms
* Keep adjacent shell parameters literal in Markdown
* Protect shell substitutions and raw text
* Disambiguate streaming numeric math from shell text
* Tighten malformed substitution performance regression
* Disambiguate Markdown references and shell context
* Preserve quoted Markdown references
* Disambiguate legacy arithmetic and shell context
* Handle Markdown and streaming shell ambiguities
* Contain shell context within Markdown structure
* Bound generic command context in long prose
* Handle Markdown boundaries around numeric math
* Exclude complete Markdown table rows from shell context
* Disambiguate shell concatenation from Markdown prose
* Cover shell wrappers and enclosing emphasis
* Handle prompt and emphasis shell boundaries
* Generalize shell and emphasis context scanning
* Handle remaining inline math shell boundaries
* Respect shell and Markdown continuation boundaries
* Keep line-start spaced inline math rendering and relax streaming timing bound
* Stop the emphasis opener scan at offset zero and accept !/? after spaced line-start math
* Format latex.ts with prettier
* Recognize spaced line-start math closers before Markdown suffixes
* Look for spaced line-start math closers through the paragraph
* Raise the Studio startup budget by the inline math growth
* Re-measure the inline math startup budget raise against current main
---------
Co-authored-by: mahiatlinux <mahiatlinux@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: keep the start of an attached file that is larger than the context window
* Keep only the path of an earlier attachment too big for the window
* Cut inside the attachment wrapper, keep the turn's identity and decide per attachment
* Cut a long paste inside its own wrapper too
* Cut only what exceeds the real budget, and inside any wrapper tag
* Fall back to the path alone when only that fits the budget
* Keep blank lines at the end of an excerpt and bound the size search
* Trim comments
* Studio: seed the LoRA init with the Random Seed setting
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Seed the text-diffusion LoRA init too
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio frontend: raise the startup bundle budget by the measured growth since the last raise
* Bundle budget: include #12821 and #13630, which merged while this was open
* Studio: ask before a desktop Update stops downloads in progress
* Studio: check downloads after the training prompt is answered
* Studio: recheck training after the downloads update prompt
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: let browser panel pages call their own site's API
* Keep page headers and redirected writes on the page's origin
* Filter cross-origin response headers and honor fetch's redirect mode
* Fix test types for the page request relay
* Send Origin on cross-origin redirects and combine repeated page headers
* Bracket an IPv6 host in the page's Origin
* Drop body headers when a redirect turns a write into a GET
* Serialize an XHR's Document body before relaying it
* Relabel an XHR text body's charset as UTF-8
* Report a relayed fetch's redirect and CORS state from the backend
* Keep CORS-safelisted page headers across cross-origin redirects
* CORS-check redirects from other sites before following them
* Leave own-site requests native when the site answers any origin
* Share the page request cap across panel pages
* Encode the relayed Referer for non-ASCII page addresses
* Take turns between pages for the shared request slots
* Try the browser first for own-site GETs it can already read
* Send an origin-only Referer on cross-origin redirect hops
* Check a relayed fetch's integrity metadata
* Drop aborted requests from the page queue
* Relay own-site GETs on pages whose site answers any origin
* Bound queued page requests and keep oversized writes native
A page that aborts and resends requests could pile bodies up behind a stalled
site, since a disconnect does not end the worker; /request now holds one of 64
slots until its worker ends and answers 503 past that. Writes past the relay's
1 MiB body limit go to the browser as they did before relaying, instead of
failing.
* Measure URLSearchParams and Document XHR bodies before relaying
* Read relayed fetch uploads only up to the limit; count multipart overhead
* Refuse https-to-http redirects; abort capped upload reads with the request
---------
Co-authored-by: Nilay Yadav <nilayyadav10@gmail.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: remember where a threadless conversation was cut so later requests reuse the prompt cache
* Replay a remembered cut only while the request overflows, check the whole dropped prefix, and look it up off the event loop
* Count tool schemas in the replay check, cover every removed turn, and scope passthrough cuts to the loaded model
* Scope passthrough cuts to the tool catalogue and leave images out of the overflow estimate
* Trim comments
* fix: align Python installer warnings with amber palette
* Test installer warnings use the shell installer warning color
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: report prompt cache hits to Anthropic and Responses clients, and ask vLLM and SGLang for them
* Name the APIs in the cached token test docstring
* Trim comments
* Studio: forget a deleted audio model only after its Hub cache listing lands
The Audio page spent a delete marker on the first settled listing whose
available set was a new object. After a delete the undebounced local
listing lands first beside the old Hub cache rows, so the marker was
spent while the deleted repo was still listed and the remembered model
was never forgotten. Markers now keep the delete time and retire only
once both Hub cache fetches behind the listing were sent after it.
The page's inventory also asks for audio.cpp folder rows, so deleting
one folder while others stay downloaded forgets that folder.
* Studio: spend an audio delete marker only on a cache fetch sent after the delete
A Hub cache fetch sent in the same millisecond as the delete could still
have read the old cache, and its late response stamped a start equal to
the delete time, spending the marker on stale rows. Require a strictly
later fetch start.
* Studio: tighten the audio delete marker doc comment
* Studio: trim audio delete marker comments
---------
Co-authored-by: Etherl <mrmrmidoessam@gmail.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: complete the OpenAI-compatible audio API
A script with an API key can now do what the Audio pages do, against /v1:
- /v1/audio/speech takes a saved voice (id or {"id"}), a built-in speaker
of the model, or a reference clip to clone, and returns wav, mp3, flac,
opus, aac or pcm. History keeps the WAV.
- /v1/audio/transcriptions gives OpenAI segments and words on audio.cpp
models, plus diarized_json for models that tell speakers apart.
- /v1/audio/run takes a model and loads it, refusing one that cannot run
the workflow before the resident model is evicted. API runs show in the
API monitor.
- Audio inputs, saved voices and history clip files answer on /v1, with
the upload streamed past the /v1 body cap. URLs returned on /v1 point
at /v1.
* Studio: refuse long timed transcriptions and send OpenAI's pcm
- A timed /v1/audio/transcriptions upload past 30 minutes is refused with
413, like the plain one, instead of being transcribed cut short.
- pcm speech is 24 kHz mono, the format OpenAI's clients play.
- The input sweep reaps a prepared upload a crash left behind.
* Studio: keep a prepared transcription upload for the input TTL
A timed transcription can wait an hour on the aligner download before
audio.cpp reads its copy, so the sweep reaps a leftover copy after the
inputs' 24 h TTL, not the hour a temp file gets.
* Studio: add an Audio API card to Settings → API keys
Settings → API keys only showed chat examples. The new Audio API card
shows copy-ready curl, Python and JavaScript calls for Speak, Clone,
Transcribe and the /v1/audio/run workflows (Separate, Convert, Music,
Edit), filled in with this server's URL, the revealed key and a
downloaded model that can run each one.
- Which model clones, converts, edits or makes music comes from the
Audio pages' audio.cpp catalog, since /v1/models lists only a task.
- Transcribe asks for segment timestamps only on models that return
them, and the Edit body is built by the same adapter the Edit page
uses for that model family.
- The card says when no downloaded model fits, and when Model
auto-switch is off, since a named model does not load then.
- The Secure HTTPS switch moves up to the tab so both cards share it.
* Studio: open the Audio API card from the Audio pages
Each Audio page gets a "Use via API" button in its header, left of the
Library link. It opens Settings → API keys at the Audio API card, on
that page's workflow and with the page's model filled in. The model is
used only when it can run the workflow, so a separator still loaded
while Speak is open falls back to a downloaded speech model.
The button shows its label only where the header has room beside the
Create/Train pills, and is icon-only with a tooltip otherwise.
* Studio: make each Audio API example match what its model accepts
- Convert on RVC sends no target recording and picks a built-in voice
instead, since RVC refuses a target.
- Music on Stable Audio's SFX build and ControlFoley asks for a sound
effect, since they make no songs.
- An Audio page's model is used only when /v1/models lists it and it
can run the workflow, so a resident chat model never ends up in a
speech example.
- The JavaScript workflow examples wait for runs longer than 5 minutes
instead of failing on fetch's default response timeout.
- With keyless access set to everything else, the examples use the
keyless placeholder key, and the card no longer says a key is always
needed.
- Picking a language in the Audio card keeps the chat card's stored
variant of that language (Python + tools stays).
* Add Audio API strings to he and sv, drop Studio branding from the Speak example
* Trim comments in the Audio API card
* Format audio-api-request test
* Audio API card: trust the workflows /v1/models advertises, and warn about loading only once it answers
* Audio API card: treat listed separators like every other workflow
* Audio API card: follow auto-switch changes, and skip its warning for transcription
* Correct the Audio API card's model-matching comment
* Audio API curl examples: stop on an HTTP error instead of saving it as audio
* Upload a sized file in the JavaScript transcription example
The OpenAI SDK sends fs.createReadStream as a chunked multipart body, which Studio refuses with 411 Content-Length required. toFile reads the clip so the request carries a length.
* Keep polling the Audio API card's model list
A model handed over while it was still downloading, or loaded and unloaded later, never reached the card, which read /v1/models once per mount. Relist on the chat examples' pace: 15 s until something is resident, 60 s after.
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: keep Anthropic system messages sent mid-conversation in place
Claude Code sends its per-session environment (working directory, platform) as a
system message after the first user turn. Studio merged every such message into the
top system block, which most chat templates render before the tool schemas, so each
new session re-read system + tools. Later system messages now stay in the user turn
they follow as a <system-reminder> note; leading ones still join the system block.
* Keep a system message that follows a tool result inside that result
As its own text block it turned the tool turn into a new user query, so the converter
and the templates dropped the current turn's reasoning.
* Leave malformed turns and prefills as before, keep MCP image envelopes last
A turn whose content is not a string or list is left for validation to reject, a
system message after a trailing assistant turn still joins the system block so the
prefill is continued, and a note inside a tool result goes before its text.
* Trim comments
* Studio: keep Start Export off for a Hub push until the repo is named
* Studio export: trim Username / Org and Model Name when building the Hub repo id
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Keep the Docker notebooks and the models they save after docker rm
* Give run.sh --rocm its own workspace volume
* Mount the workspace volume in the core Jupyter example
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Run the ROCm smoke test without the workspace volume so a rebuilt image tests its own copy
* Mount the workspace volume in the CPU-only and ROCm quick starts
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
fast_rms_layernorm_inference_gemma starts with X.to(torch.float32),
which returns X itself when X is already float32, so the norm then
scales X in place. GemmaModel_fast_forward_inference and
Gemma2Model_fast_forward_inference keep X as the residual, so in
float32 every decode step added the normed values back instead of the
layer input. Copy X first; bfloat16 and float16 already got a copy.
* Studio: open the Images and Audio workflows from the collapsed sidebar rail
* Studio: return focus to the rail icon when its keyboard-opened flyout closes
* Studio: move focus into an open rail flyout on ArrowRight
* Studio: tighten rail flyout comments
* Studio: search for llama.cpp and other file names in the address bar
* Keep host:port addresses as hosts in the address bar
* Only search bare file names, keep hosts with a path as hosts
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: keep a Claude session working after an image on text-only GGUFs
* Name the image block on the auto-switch refusal and keep the client's message index
* Name the image block on the auto-download vision refusal
* Decode small 4bit batches with the NF4 GEMV instead of dequantizing every token
Batched decode (q_len 1, 2+ rows) of a bitsandbytes 4bit model fell off the
GEMV path and dequantized every weight to 16bit on every token, so batch 2
decoded slower in total than batch 1 (Llama-3.3-70B on RTX PRO 6000: 8.4 tok/s
at batch 2 vs 19.8 at batch 1).
The NF4 GEMV kernels now take M activation rows, one program per (row, weight
tile), each row bit-identical to the batch-1 kernel. fast_linear_forward sends
2 to 4 decode rows to it where it times at least 10% faster than dequantize +
matmul on the GPU, decided once per weight shape and row count, with a planned
relaunch afterwards.
Fixes#2798
* Add UNSLOTH_GEMV_MAX_ROWS to cap or disable the batched NF4 GEMV
* Trim comments in the batched NF4 GEMV path
* Pick the batched NF4 GEMV by GPU architecture instead of timing it, so decode is reproducible
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Check the GEMV row cap before anything else in _gemv_rows
Rows above the cap (A100 batch 4) still unpacked the quant state and could
copy a strided X before the cap sent them to dequantize, costing A100 batch-4
decode about 10% against main. The cap is now the first check, so those rows
pay one dict lookup.
* Reuse a planned rows GEMV only for an out with unit element stride
* Take the NF4 GEMV row count from X, and reuse rows plans when no out is given
* Keep 0-row batches and outs the rows GEMV cannot fill on the dequantize path
* Keep a foreign-device out or a mismatched activation width off the rows GEMV
* Require a (M, 1, N) out for the rows GEMV; say where the row cap override applies
* Keep weights or quant state on another device off the rows GEMV
* Trim comments in the batched NF4 GEMV change
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: say why Sandbox High is unavailable when there is nothing to install
When the OS sandbox cannot be set up from the High popup (Windows with MXC
allowed and prepared but failing its live check, an ARM64 PC, or a build older
than 24H2), the popup showed only its generic title with no reason, and Learn
more scrolled back to Permissions, where High had just been picked.
Show the check's reason in that case, and point Learn more at the OS sandbox
status and setup section instead.
* Studio: no unavailable reason in the High popup once the sandbox passes
* Studio: trim sandbox popup comments
* Studio: scroll Learn more to the OS sandbox once its status has loaded
* Compute RMSNorm weight gradient when the norm weight is trainable
* Chunk the eager RMSNorm weight-gradient reduction to bound fp32 temporaries
* Compute the RMSNorm weight gradient inside the Triton backward kernels
A TRAIN_W constexpr switches both backward kernels (one-row and multi-row)
to persistent programs that accumulate dL/dW in fp32 registers over their
rows and store one partial each, like flash-linear-attention's
layer_norm_bwd_kernel_row. A small Triton kernel sums the partials in a
fixed order, so eager and torch.compile give the same bytes. With a frozen
weight TRAIN_W is False and the kernels compile to the previous code.
Replaces the torch-side reduction from the earlier commits.
* Launch one RMSNorm dW program for an empty input instead of dividing by zero
* Sum RMSNorm dW partials in 64x64 tiles instead of one row at a time
The serial row loop cost about 73 us per 148 partials on a B200; tiles cut
the sum 4-8x and keep a fixed order, so eager and compiled bytes still match.
* Trim comments in the RMSNorm weight-gradient change
* Allow one bf16 ulp between trainable and frozen RMSNorm dX, as on Blackwell
* Raise RMSNorm dW backward occupancy and pipeline the partials sum
- Accumulate the per-row dW term before the row sum, so the fp32 dY row is
not held across it (fewer registers).
- On 64-warp SMs (A100 / H100 / B200, CUDA only), rows of at most 16 elements
per lane launch 32 warps of programs per SM with maxnreg = 64.
- Pipeline the 64x64-tile dW partials reduction (num_stages = 3).
- Cache the row-path launch choice per (device, n_cols): at small shapes the
backward is host bound, and recomputing it per call cost about 12 us of
Python per forward + backward on a Colab A100 host.
* Use the forward's rounded normed for the non-Gemma RMSNorm dW
* Trim comments in the RMSNorm dW occupancy and rounding changes
* Studio: run native image generation on Vulkan when torch is CPU-only on an AMD or Arc GPU
Fixes#13666
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Read the Vulkan load policy off the build this load runs, not a fresh host probe
* Vulkan sd.cpp route: honour HIP/CUDA/ZE masks, keep a GGML_VK pin, skip UHD-first and unmounted render nodes, walk the whole fallback ladder
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Vulkan sd.cpp route: GPU arbiter claim, placement after a deferred install, Vulkan failure markers, reject GGML_VK selectors naming no card
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Vulkan sd.cpp route: hardware ICD only, sysfs NVIDIA check, no GGML_VK bypass on mixed hosts; place for the running build after every fallback; arbiter follows the preferred build
* Place an unowned sd.cpp build by its own device listing
* Vulkan sd.cpp route: the loadable ICD must be a driver for the inventoried card's vendor
* Vulkan sd.cpp route: render node of the card's vendor, Windows driver filters, arbiter for a retained Vulkan build
* Video sd.cpp ladder: the CPU rung stays on its own branch, which also releases the GPU placement
* Vulkan sd.cpp route: Windows NVIDIA from the adapter registry, node and driver of one vendor, server failure drops a cached sd-cli of the failed build
* sd.cpp load keeps the selected card across a deferred upgrade; another vendor's render node keeps the CPU build
* Vulkan sd.cpp route: radv_icd manifests count as an AMD driver
* Vulkan sd.cpp route: masks opt out only their own vendor, unreachable cards are not candidates, arbiter preflight reads the CLI behind an unowned server
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Image load: a GPU torch running the managed CPU rung takes no GPU claim
* Image load: drop the GPU claim only when every runnable sd.cpp binary is a managed CPU install
* Vulkan sd.cpp route: an openable render node of unknown vendor keeps CPU, claim re-settled on the activated engine, neutral fallback hint
* Vulkan sd.cpp route: only an openable foreign render node blocks it; the claim follows the binaries selection installed
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Pick the attention backend per device on mixed-generation multi-GPU hosts
Flash Attention, the xformers sm_120 probe and the xformers fp32 limit were read from device 0
and applied to layers a device_map places on other cards. On a mixed host (e.g. sm_86 + sm_75)
flash then raised on the older card. run_attention now re-checks the backend against Q's device
from a per-device table that only exists when cards differ, and the HF path falls back from
flash_attention_2 when the loaded device map reaches a pre-Ampere card.
* Format
* Flash fallback: only attention placements trigger it, and known-wrong SDPA models fall back to eager
* Flash fallback before the text core is extracted, attention ancestors count, softcapped models stay on eager, no probe on same-class cards
* Flash fallback: one answer for the whole config tree, read before any write
* Probe a secondary sm_120 card for xformers on first dispatch there, not at import
* Flash fallback: bare cuda reads the current device, most specific device-map entry wins, eager-only towers keep eager
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(mlx): decision rows and predict carry audio for a Clef that hears it
FastDecisionModel.build_dataset reads a row's "audio" (data URLs, an undecoded audio column's bytes or file, or decoded 16 kHz mono clips) and predict takes audio = the same way, passing both to unsloth-zoo's MLX decision engine. An unsloth-zoo that takes no audio is reported instead of training without those rows, as for images.
* feat(studio): the Decision API takes audio for a Clef that hears, through MLX
POST /v1/systemone accepts `audio`, an array of base64 data URLs, beside `images`. Only unsloth-zoo's MLX engine hears, so a request with clips is answered by a Clef that has an audio tower on Apple Silicon or refused: an entry that does not hear, a PyTorch or llama.cpp runtime setting, and a Decision API connection all say so instead of answering without the clips. The route checks the number of clips, their shape and their total size; the engine decodes them. Such a Clef lists `audio` among its input modalities. A local Clef folder that keeps its towers under `thinker_config` is now recognized as reading images too.
* feat(studio): Clef Omni in the Decision API catalog, served by MLX on Apple Silicon
* feat(studio): the training page lists Clef Omni on a Mac and starts a decision run on rows with audio
* feat(studio): an LLM given a decision head starts on rows with audio on a Mac
* fix(studio): an offline decision run on an LLM no longer fails waiting for a model pin it never uses
* fix(studio): a Phi-4-multimodal or Nemotron Omni Clef reports the images and audio it reads
Whether a local Clef reads images or audio on the MLX engine was read from a `vision_config` or `audio_config` in its config. Phi-4-multimodal names its towers `embd_layer.image_embd_layer` and `embd_layer.audio_embd_layer`, and Nemotron Omni its audio tower `sound_config`, so a Clef trained on either was served as text only and its images and audio refused.
* fix(studio): decision rows from S3 keep their audio after the download is cleaned up
An S3 download fetches the audio files a manifest names and rewrites the rows to the local copies, but the decision trainer cleaned the download up before training read them, so every such row was skipped. Rows now keep the bytes of the clips they name while the download is still there.
* fix(studio): an S3 decision run holds the audio of only the rows its slice keeps
Rows from S3 kept the bytes of every clip before the dataset slice was applied, so a small slice of a large audio dataset held all of it in memory. The clips are now read for the sliced rows only, and the slice is applied once.
* fix(mlx): an audio clip the datasets decoder cannot read skips its decision row
* fix(studio): an S3 dataset row listing several audio clips has each rewritten to its download
* fix(studio): Clef Omni needs no PyTorch, an audio-only Clef reports audio, S3 path-string clips survive cleanup
* Trim comments in the Clef Omni audio changes
* Trim narrating comments in the Clef Omni audio tests
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: default export load to offload_layers=auto
Export checkpoint loading never passed Unsloth's offload_layers, so
low-VRAM Output exports hit bitsandbytes CPU-spill while Training already
supported Auto (#13696). Plumb the same kwarg through the export API and
default it to auto so models that fit with layer offload can export.
* Studio export: offload layers only when the load spills to the CPU
Retry a single-GPU 4-bit export load that bitsandbytes refuses with offload_layers='auto'
and a 1 GiB merge reserve instead of a training step's, so loads that fit stay unchanged.
Match Unsloth's rewritten CPU-spill error via __cause__, carry base_model and
offload_layers into the load retries, and keep offload_layers keyword-last.
* Tighten export offload test comment
* Trim export offload comments
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Studio: search and load large MCP catalogs on demand instead of listing every schema
* Keep image-mapped and forced MCP tools listed, bound relisted and searched schemas by the window, rank without ids
* Name each deferred MCP tool with a one-line summary and preload up to five ranked tools
* Bound MCP tools a turn loads, pin image tools only with an attachment, detect calls to loadable tools
* Trim MCP tool search comments
* Keep image-mapped MCP tools listed for mcp_image, fit the search declaration by tokens, admit loadable names in the bare-JSON gate
* Price each preloaded MCP schema, fall back to a tool count, let a repeated search run, remember MCP calls before deferral
* Do not count a turn that only searched MCP tools against the tool budget
* Trim MCP tool search comments
* Match write verbs to differently named MCP tools when ranking the preload
"Add a comment" ranked update-comment and fetch-comment above create-comments
when no tool text said "add", so the model called the listed sibling. The name
leg now also matches the other verbs of the same write (create/add/new/reply,
delete/remove, update/edit/change).
* Studio (macOS): read the Metal working set from torch when MLX is missing
The llama.cpp Metal budget, wired ceiling and --fit-target floor read
recommendedMaxWorkingSetSize only through MLX. macOS 13 has no MLX wheel
(and a broken MLX import looks the same), so the budget fell back to total
RAM and the wired ceiling and fit floor to 0. Read it through one helper that
tries MLX, then torch.mps.recommended_max_memory().
* Check torch.mps and MLX agree on the Metal working set on a real Mac
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio installer (AMD, Linux): Vulkan llama.cpp for RDNA 1 and Polaris when ROCm is installed
With ROCm userspace present, rocminfo sets has_rocm and the Linux ROCm branch finds no
linux-rocm bundle for gfx1010/1011/1012 or gfx803, so setup source-builds HIP (no rocBLAS
kernels for these archs) or, without hipcc, a CPU build. Route these hosts to the Vulkan
prebuilt, as Windows already does (#7357), behind the same guards as the integrated-GPU
preference: no physical NVIDIA, no HIP/ROCR/CUDA or GGML_VK mask, no
HSA_OVERRIDE_GFX_VERSION, every physical gfx uncovered, and an AMD Vulkan ICD present.
An explicit backend still wins.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Add 2.10.0 to win_amd64's extra_torches (Python 3.11-3.13, flash-attn, causal-conv1d,
mamba-ssm) with triton-windows 3.6.0.post26 for the smoke test, since torch 2.10 pins
triton 3.6.0 like 2.11. Upstream's torch2.10 wheels have no Windows build. A default
dispatch still builds only 2.13.0 and 2.14.0.
* Studio: stop Expand thinking from restyling the whole trace
* Test the tail flex box transitions directly
* Trim test comments that restate the asserted traces
---------
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Lint CI: scan committed files, not the tree the background lanes write into
* Lint CI: decode git's tracked paths with os.fsdecode
* Run the Lint CI scan guard in the unfiltered workflow-trigger-lint job
* Studio: keep the composer draft across chats, pages and tabs
* Studio: carry old per-chat drafts over, sync open tabs, keep temporary chats unsaved, share the draft with Compare
* Studio: keep Compare and the chat composer in step, drop temporary pastes on toggle, roll back partial draft adoption
* Studio: keep unsaved text over old per-chat drafts, restore the draft when Compare leaves a temporary chat, ignore late paste restores
* Studio: restore only the saved pastes missing from the composer, clear Compare's draft at once on send
* Trim comments
