mirror of
https://github.com/unslothai/unsloth.git
synced 2026-10-18 09:07:17 +00:00
201790e691
* WSL installer: stop running the C# compiler on the user's Windows machine `install.sh` generates a PowerShell script and runs it on the Windows side to create the WSL shortcut. Two constructs in that path were removed from `install.ps1` in earlier passes and left live here, where nothing looked: - `install.sh:2249` declared `SHChangeNotify` with `Add-Type -MemberDefinition`. On Windows PowerShell 5.1 that writes C# to `%TEMP%` and runs `csc.exe`, producing a DLL on the user's machine. #10540 traced exactly that chain to Bitdefender `Gen:Variant.MSILHeracles.272113`: `unsloth-stud... -> powershell.exe -> csc.exe -> %TEMP%\<random>.dll`. It replaced the declaration in `install.ps1` with reflection emit and missed this copy. - `install.sh:2273` launched the generated script with `-ExecutionPolicy Bypass`, the flag #7822 removed everywhere else. Neither was caught because `test_the_installer_never_runs_the_c_sharp_compiler` parametrised over `("install.ps1", "studio/setup.ps1")` only, and `windows-no-compiler-ci.yml` did not list `install.sh` in its path triggers. A shell script that emits PowerShell can put a compiler on a Windows machine without touching a `.ps1`. ## The compile `SHChangeNotify` is now emitted, mirroring `install.ps1:3485-3499`: a `DefineDynamicAssembly` / `DefineDynamicModule` / `DefineType` / `DefinePInvokeMethod` / `CreateType` block inlined, since the generated script is standalone and cannot call `New-StudioEmittedNativeType`. The enclosing `try {} catch {}` stays, so a failure still costs only a stale icon, and both notifications stay: per-item `SHCNE_UPDATEITEM` + `SHCNF_PATHW` per `.lnk`, then the global `SHCNE_ASSOCCHANGED`. The per-item call is not redundant, and the `ALLOWED_PINVOKES` entry records why: `ie4uinit.exe -show` does not recover a stale same-name `.lnk`. I ran the emitted type rather than assuming it: under pwsh it produces `void SHChangeNotify(Int32, UInt32, String, IntPtr)`, identical to the `Add-Type` declaration, `PreserveSig` set, and `Assembly.Location` empty, which is the point. Nothing reaches disk. ## The execution policy `Bypass` was load-bearing, for one reason: `wslpath -w /tmp/unsloth-shortcut-XXXXXX.ps1` yields a `\\wsl.localhost\<distro>\...` UNC path, PowerShell treats a script there as remote, and `RemoteSigned` refuses an unsigned remote script. So rather than relax the policy, the script now goes to the Windows `%TEMP%` (`cmd.exe /c echo %TEMP%`, CR stripped, `wslpath -u`), which is a local volume and therefore `MyComputer` zone, where `RemoteSigned` loads it unsigned. Same generated script, same launch, same result. When `%TEMP%` cannot be resolved there is deliberately **no fallback to WSL's `/tmp`**, because that is the case that needed the relaxed policy in the first place. Nothing is allocated and the caller takes the notice it already prints when interop is unavailable. That population is very nearly the population where interop is already broken. ## Tripwires, with messages that say what is wrong `tests/sh/test_wsl_shortcut_script_location.sh` evals the real resolution block out of `install.sh` with `cmd.exe` and `wslpath` stubbed, covering resolved, no-interop, unexpanded `%TEMP%`, and a `%TEMP%` that is not a directory, then asserts statically that the launch uses `RemoteSigned`, that the file relaxes the policy nowhere, that it compiles no C#, and that both refreshes survive. A failed extraction is reported as a failure rather than exiting early, so the output names every problem at once, and it still fails closed. `clean-machine-install-ci.yml`'s `wsl` job is the only lane that runs this branch, so it gains the behavioural half: read the created `.lnk` back and refuse a script-host target, require the generated script to have been cleaned up, and fail on any `.cmdline`/`.rsp` under the Windows `%TEMP%`, which is csc.exe output and nothing else. That check plants two intermediates and requires the search to find both before trusting a negative result, because a detector that sees nothing looks exactly like a clean run and every reason it might see nothing is silent. If the control does not fire the job fails as a broken detector instead of reporting a pass. Shortcut absence is a warning, not an error, since the installer itself treats shortcut creation as best-effort. `install.sh` is added to the compiler test's parametrise list and to `windows-no-compiler-ci.yml`'s paths. The `DefinePInvokeMethod` assertion in that test becomes conditional on the file declaring a native import at all, so it does not fail for being satisfied on `scripts/uninstall.ps1` or `studio/setup.sh`. ## Verification - The new shell suite: 16 passed on this branch, and against `main` in a worktree it exits 1 with 6 named failures. Proved to bite rather than assumed to. - The generated WSL script rendered and parsed with `[System.Management.Automation.Language.Parser]::ParseFile`: clean over 765 tokens. No existing gate parses that here-string, and this change edits it. - `bash -n` and `sh -n` on `install.sh`; all of `tests/sh` passing; the parse gate clean over `install.ps1`, `studio/setup.ps1` and `scripts/uninstall.ps1`. - `tests/studio/test_installer_av_shapes.py`, `tests/studio/install/`, `tests/security/`: green. - The new CI PowerShell parsed standalone, and its positive control plus detector exercised for real against a planted `csc.exe`-style intermediate: control 2 of 2, simulated compile caught, clean directory reports zero. I have no Windows or WSL here, so the runtime behaviour of the shortcut path rests on the `clean-machine-install-ci.yml` WSL job, not on a local run. This is a shape fix with a confirmed detection behind the shape; it is not a claim that any particular user report is resolved. * Fix a test of mine that passed for the wrong reason, and could only fail on a real CI machine `test_wsl_shortcut_script_location.sh` asserted that the generated shortcut script's path does not start with `/tmp`, as a proxy for "not in WSL's own temp, which wslpath turns into a \\wsl.localhost UNC path that RemoteSigned refuses". The proxy is broken: the harness builds its *fake Windows temp* with `mktemp -d`, so on any machine with `TMPDIR=/tmp` -- which is every normal CI runner -- the stub's own directory matched the pattern and the test failed while install.sh was behaving exactly as intended. It passed locally only because this workspace sets `TMPDIR` elsewhere. That is precisely the class of accidental pass the rest of this work has been built to catch, so it is worth naming rather than quietly correcting. The replacement asserts the property the proxy was standing in for, structurally: the block contains exactly one `mktemp`, so there is no second path to fall back to, and that `mktemp` allocates inside `$_css_win_temp`. Both are true or false regardless of where anyone's temp directory happens to live. Verified under the exact condition that broke it: `TMPDIR=/tmp` gives 17 passed, 0 failed on this branch, and 6 failed on `main` in a clean worktree -- so the test still bites. * WSL installer: read %TEMP% with cmd /d, so an AutoRun command cannot cost the user their shortcut Reading the Windows %TEMP% through interop is new in this branch, and it reads it with `cmd.exe /c echo %TEMP%`. Without /d, cmd first runs the AutoRun command from HKLM or HKCU \Software\Microsoft\Command Processor, and `cmd /c` is not exempt from that. AutoRun writes to the same stdout, so its output arrives ahead of the path and `tr -d '\r\n'` glues the two into one string. wslpath then rejects it, the directory check fails, and the branch takes its "couldn't create the Windows shortcut" notice. Clink sets an AutoRun, so Cmder does, and so do plenty of corporate images. Every one of those users got a working shortcut before this branch and would silently stop getting one, while being told WSL interop is disabled, which is not what happened. So: /d, which is the cheap fix rather than out-parsing whatever prints. Then the last line rather than the whole stream, for anything that still reaches stdout by another route, and trailing blanks trimmed, because Win32 strips them from a path and [ -d ] does not, so a %TEMP% set with one resolved to a directory that appeared not to exist. The || stays: this file runs under set -e, where a failed substitution ends the install. Tests ----- tests/sh/test_wsl_shortcut_script_location.sh grows the AutoRun and trailing-blank cases and a static check for /d, and its wslpath stub is now input-sensitive. It was not, and that mattered: a stub that returns the fake Windows temp whatever it is handed cannot tell "install.sh read the path" from "install.sh read a banner glued to a path", so both new cases passed against the unfixed file until the stub compared its argument. Against the unfixed install.sh the four new assertions fail; with it they pass. tests/studio/test_wsl_shortcut_script_is_valid_powershell.py is new and covers a gap nothing else did: that what install.sh generates is PowerShell at all. bash -n sees the here-string as a here-string, the contents only run on Windows with both streams sent to /dev/null, and the icon-refresh block runs inside try/catch, so a parse error there is not a crash and not a message, just no shortcut and a misleading notice. The block was just rewritten into a twenty-line reflection-emit sequence, which is when that matters most. The file renders the here-string with a real shell rather than keeping a copy, parses it, and then executes the emit sequence retargeted at a symbol this host exports, so a wrong call order or argument count fails on Linux. Five mutations of the shipped file were tried; four fail it, and the docstring names the fifth that does not and why. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Watch the WSL install live, instead of searching for what CodeDom already deleted The lane looked for .cmdline and .rsp files in the Windows temp after the install had finished. CodeDom removes its whole intermediate directory once the assembly is loaded, which Watch-ForCompiler.ps1 records as a before-and-after diff seeing nothing at all while 4688 recorded csc.exe, so that search could not fail for the case this lane exists to catch. Its positive control planted persistent files, which proves the search can traverse a directory and nothing about whether ephemeral compiler output is observable. The install now runs inside Invoke-WithCompilerWatch, the same detector windows-no-compiler-ci.yml uses, which unions 4688 with a live FileSystemWatcher and treats an unreadable Security log as void rather than clean. The dropped-library half is checked too, since that is what the Bitdefender report in #10540 keyed on. The post-hoc listing stays for what genuinely is left behind, which is the other half of the union, and its message now says that is what it means. Watch-ForCompiler.ps1 is in both path filters now, because this job's detection depends on it, and the new guard is registered in the unfiltered lint job since it reads a workflow file. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Arm the 4688 half of the compiler watch in the WSL lane Watch-ForCompiler.ps1 requires the caller to enable process creation auditing, and this job never did, so its Security log query returned an empty compiler list whatever ran while the step still printed that no compiler process was seen. Enable and verify the audit policy before the install, and require a deliberate compile to be detected first, so a clean verdict is distinguishable from a detector that never attached. * Close the AV shape-guard holes, and load the installers under RemoteSigned (#10982) * AV shape guards: watch the two shipped scripts nothing watched, and ratchet the policy sites Stacked on the WSL compiler fix, which shares this file. Two problems, both found while investigating #10805, and both about the guard rather than the code it guards. ## Two shipped scripts were in no list `PS_SCRIPTS` and `SH_SCRIPTS` between them missed `studio/setup.bat` and `scripts/uninstall.sh`. A shipped `.bat` is scanned like any other file, and `setup.bat` launches PowerShell, so every rule here applies to it. That is how its `-ExecutionPolicy Bypass` survived the passes in #7822 and #8586: the guards were reading a set of files that did not include it. Both are added to `ALL_SCRIPTS`. ## The hidden-window pair check only ever looked at one line `test_a_hidden_window_never_pairs_with_a_bypassed_policy` iterated `splitlines()` and compared the two flags within a single physical line. `install.ps1` assigns `$shortcutArgs` a hidden window at :3419 and overwrites it with a relaxed policy at :3433, fourteen lines apart in the same function, and that passed for as long as it existed. A scanner reads the file, not the line. Three layers now: 1. Same line, unchanged, because it gives the clearest message when it is the real shape. 2. A ratchet on how many relaxed policies the file contains at all, against `KNOWN_BYPASS_SITES`, which records each remaining site with the reason and what would remove it. It fails **both** ways: too many is a new site, too few is a stale entry that has quietly stopped guarding anything. This is the layer that catches a policy arriving somewhere the other two do not model. 3. Same variable, any distance: the union of everything assigned to one name must not contain both flags. This is the layer that sees the 3419/3433 shape. `install.ps1`'s `$shortcutArgs` is recorded in `KNOWN_SPLIT_PAIR_VARIABLES` rather than failed on. Removing it needs `launch-studio.ps1` written to a guaranteed-local directory first, because `%LOCALAPPDATA%` can be folder-redirected to a UNC path where RemoteSigned refuses an unsigned script, and a shortcut that silently does nothing is worse than the token. Recording it lets the guard land now and lock in "no new ones", which is the part that matters going forward. ## Verification 70 passed in the suite, and each new layer was confirmed to bite rather than assumed to: - A new relaxed policy injected into `studio/setup.sh` fails layer 2, naming the file, line number, the offending line, and what to do about it. - Removing the recorded site from `studio/setup.bat` without lowering its count fails layer 2 the other way, telling the reader to lower the count in the same commit. - Layer 3 proved in isolation by renaming `$shortcutArgs` to `$launchArgs` in `install.ps1`: the relaxed-policy count is unchanged so layer 2 passes, and layer 3 fires on its own. The tree was restored after each and re-verified clean. `tests/studio/install/` and the shape suite are green; `test_pr10648_offline_and_status.py::test_the_pr_only_added_new_arms_to_the_setup_status_blocks` fails identically on `main` in a separate worktree and is untouched by this change. * Load the installer scripts under RemoteSigned instead of relaxing the policy Removes three of the four relaxed execution policies the ratchet in the previous commit recorded. RemoteSigned refuses an unsigned script only in the Internet or Untrusted zone, so for a local, unmarked script it behaves identically to Bypass while dropping a token endpoint products score. ## studio/setup.bat `powershell -ExecutionPolicy Bypass -File "%~dp0setup.ps1"` becomes RemoteSigned. setup.ps1 ships beside the batch file inside an installed package, so it is MyComputer-zone and loads unsigned either way. One population could regress: a package unzipped from a download, where setup.ps1 carries a mark of the web that RemoteSigned honours and Bypass ignored. So the mark is cleared first. Execution policy governs script *files* and not `-Command`, so that call runs under any machine policy, and clearing an absent stream writes nothing. The path travels in an environment variable rather than being interpolated into the command string, because an apostrophe in an install path is a real Windows user name and would otherwise break the quoting. `-NoProfile` is on the unblock call and deliberately **not** on the launch. The launch has always run setup.ps1 with profiles loaded, and `tests/studio/test_amd_venv_repair_loop.ps1` drives a profile that sets `Set-StrictMode -Version Latest` against it. Adding the flag would have retired that coverage with nothing failing, so the new test asserts its absence and says why. ## scripts/uninstall.ps1 and README.md Both print `Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass` as the documented way to run a cloned script. Changed to RemoteSigned rather than dropped: the Windows client default is `Restricted`, which blocks every script file including a local unsigned one, so the `Set-ExecutionPolicy` itself is load-bearing and removing it would break the documented path. RemoteSigned is sufficient because a git clone carries no mark of the web, and both now say so and point a zip user at `Unblock-File`, which is the case where the two policies genuinely differ. The uninstaller line is never executed -- it lives in the printed `_Usage` here-string -- but AMSI scans the file whole and a classifier cannot tell a string from a statement. ## Where that leaves the ratchet Four recorded sites down to one. The survivor is `install.ps1:3433`, the roaming-profile fallback for a launcher on a share, and it stays because `%LOCALAPPDATA%` can be folder-redirected to a UNC path where RemoteSigned would refuse the launcher, and a desktop shortcut that silently does nothing is worse than the token. Removing it needs `launch-studio.ps1` written to a guaranteed-local directory first, which is a larger change than this one. ## Verification - 751 passed, 6 skipped across the shape suite, `tests/test_installer_interactive_prompts.py` and `tests/security/`. All of `tests/sh` passing. The parse gate clean over all three `.ps1` files. - Both new `setup.bat` assertions confirmed to bite: adding `-NoProfile` to the launch fails with the reason and names the test that would lose coverage, and reverting the policy to Bypass fails naming the offending line. The tree was restored after each. - Relaxed-policy sites counted directly across all seven shipped scripts afterwards: one, in install.ps1, as recorded. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Step setup.bat down to Bypass for a script on a share, the way install.ps1 already does Tightening setup.bat from Bypass to RemoteSigned had one case that is a genuine regression rather than a theoretical one, and it needed handling rather than accepting. Execution policy is judged on the script file's ZONE. A plain \\server\share and a mapped drive are Intranet and load fine under RemoteSigned -- but a dotted-FQDN UNC (\\corp.example.com\...), a DFS path or an IP-literal share is the Internet zone, where RemoteSigned refuses an unsigned script. And Unblock-File cannot save that case: with no Zone.Identifier stream present the PATH decides, so clearing a stream that was never there changes nothing and exits silently. Bypass worked; RemoteSigned does not. install.ps1:3405-3421 already solves exactly this when it writes the Start Menu shortcut -- UNC prefix, then DriveInfo.DriveType -eq 'Network' -- and the new test's own comment calls that site "the only one that is genuinely load-bearing" for the same reason. setup.bat got no such exemption. Now it reuses the identical logic rather than inventing a second answer. The probe does the Unblock-File too, so this costs one process rather than two, and it runs under -Command, which execution policy does not govern at all. Two details that matter more than they look: - The batch variable DEFAULTS to RemoteSigned. A probe that fails to run therefore leaves the tightened policy in place rather than silently restoring the relaxed one. - -NoProfile is on the probe and still deliberately absent from the launch. tests/studio/test_amd_venv_repair_loop.ps1 drives a profile that sets Set-StrictMode against setup.ps1, and adding it would retire that coverage with nothing failing. Tests: the embedded probe is extracted from the batch file and PARSED, because it is one long line inside a `for /f` backquote block and a syntax error there fails quietly -- the `for /f` captures nothing, the RemoteSigned default stands, and the mark is never cleared, so an unzipped download gets a refusal with no hint that the probe was what broke. The pre-existing setup.bat test located the launch by literal filename, which is now an environment variable, so it matches on the flags instead. 5713 passed, 39 skipped across tests/studio/test_installer_av_shapes.py, tests/studio/install/ and tests/security/. The single failure, test_pr10648_offline_and_status, is pre-existing on main. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Group the hidden/bypass pair by casefolded variable name PowerShell variable names are not case-sensitive, so $shortcutArgs and $ShortcutArgs are one variable. The same-variable layer keyed its map on the captured spelling, so writing the two assignments with different capitalisation filed them as two unrelated names, each holding only one of the two flags, and the pair went through. The file-wide ratchet did not catch it either, since install.ps1 already records one permitted relaxed policy. Verified by constructing that exact evasion: renaming the two install.ps1 assignments to $launchArgs and $LaunchArgs passes under the old grouping and fails under this one, naming both spellings. * Stop the installers from explaining, in prose a classifier reads, which antivirus flagged them (#10990) * Stop the installers from explaining, in prose a classifier reads, which antivirus flagged them PowerShell hands the entire top-level script block to AMSI at compile time, before the first statement runs, so every byte of install.ps1 is classifier input -- comments included. VirusTotal's analysis of the file quoted one of our own comments back as grounds for suspicion: "The presence of comments that suggest the script is designed to evade detection by security tools (e.g. "AMSI scans this file in full before a line of it runs") further adds to the suspicion." Six hardening passes wrote careful explanations of which product flagged which shape directly into the files being scanned. The explanations are worth keeping. Their location is not. - A comment in a shipped script says **what the code does and what breaks if you change it**. - `AV_SHAPES_RECORD`, in `tests/studio/test_installer_av_shapes.py`, says **which scanner flagged which shape, and when**. The record lives in that test rather than a doc because the file already owns the guards that enforce the split, it ships to nobody, it is not packaged, and nothing scans it. Each relocated comment keeps its operational content verbatim and gains a pointer. The diff over the five shipped scripts contains no non-comment line, which is checked mechanically rather than asserted. The record is not a dumping ground. It holds the measured detections (Skyhigh `BehavesLike.PS.Suspicious.gr` on install.ps1, Bitdefender `Gen:Variant.MSILHeracles.272113` on the csc.exe output in #10540, Kaspersky `HEUR:Trojan.VBS.Agent.gen` on the .vbs launcher shape, the two fixture archives that made Panda flag the repo zip), the measured before-and-after from `virustotal_delta.py` including the part that did NOT move, why each decision is a robustness fix and not an evasion, the five shapes we are **keeping** on purpose with the reason each is load-bearing, and how to report a new detection. Three things a future pass will otherwise rediscover are written down there, because each one already cost a wrong proposal: execution policy is evaluated against the script file's zone and not against who launched PowerShell, so a `.cmd` shim cannot launder a remote-zone script; UNC paths are a remote zone, which is the whole reason install.sh resolves the Windows `%TEMP%`; and the Windows client default is `Restricted`, so documentation telling a user to set a process-scoped policy is load-bearing and only the word `Bypass` needed to go. A banned-token test alone is satisfiable by deleting the knowledge rather than moving it, so there are three: - `test_no_shipped_script_names_a_detection` over every shipped script including `studio/setup.bat` and `scripts/uninstall.sh`, which no shape test read before. - `test_the_record_survives_and_keeps_its_evidence`, which requires the sections AND the specific detection strings -- a record with headings and no evidence is the same loss with extra steps. - `test_every_script_that_dropped_its_explanation_points_at_the_record`, because a comment reduced to "security software blocks this" with no forward reference is worse than the prose it replaced: the next maintainer cannot tell whether it is still true. Deliberately not banned: `antivirus`, `quarantine`, `scanner`, `blocked`. Those describe a real runtime hazard the code handles and one of them reaches the user. Also not banned, for a more interesting reason: **`false positive`**. It was on the list, and the test immediately found it in install.ps1 and studio/setup.ps1 -- where it means a *statistical* false positive in a registry probe and has nothing to do with a scanner. A token list is only as good as its words having one meaning, and that one does not. - 14 of the new assertions fail against `main` and none against this branch, so they are proved to bite rather than to pass vacuously. - No non-comment line changed in any of the five scripts (`git diff -U0` filtered for lines that are not comments returns empty). - `bash -n` clean on both shell scripts; `Parser::ParseFile` clean on all three `.ps1` files. - Existing tests and CI that reference the old detection strings are unaffected: they live in `tests/` and `.github/`, which ship to nobody and are scanned by nothing. - One self-inflicted bug caught while writing this: a blanket rename of the old doc path rewrote the "must not still point at the doc" assertion into asserting the opposite. The stale path is now assembled from fragments so a future rename cannot do it again. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match banned vendor tokens on word boundaries, and drop the one that cannot work The guard searched for each token as a raw substring, so it rejected ordinary installer prose: 'panda' is inside 'pandas', a real dependency name, and 'rising' is inside 'surprising', 'arising' and 'comprising'. The failure message then accuses a normal sentence of naming an antivirus vendor, and the fix the next reader reaches for is to delete the guard. Boundaries are conditional on the token's own edges, because 'heur:' and 'gen:variant' sit on a colon where \b asserts the opposite of what is wanted. 'rising' is removed rather than bounded. It is a real engine and one of the two that flagged the fixture archives, but the word is also ordinary English -- 'rising memory use' is a sentence someone will write -- and no amount of boundary matching separates the two. A guard that fails on valid prose is worth less than nothing. 'tencent', the other engine that flagged those archives, has no such problem and stays. Verified both directions against studio/setup.sh: a comment naming Bitdefender, Gen:Variant and Panda still produces three failures, while 'rising memory use' plus 'installs pandas' now passes and produced two misleading failures before. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Carry the AppDomain fallback into the WSL emit, and quote TEMP inside cmd The generated script had only the static AssemblyBuilder spelling under a bare catch, so on any host lacking that overload both SHChangeNotify refreshes would stop with nothing printed. install.ps1 already tries both for this reason; mirror it. cmd expands a variable before it parses metacharacters, so an unquoted echo %TEMP% on a profile holding an ampersand, which is legal in a Windows account name, splits into two commands and the captured line is not the temp path. Quote it inside cmd and strip the quotes back off. The test stub now reproduces cmd's expand-then-parse order so the ordinary path still passes and only the ampersand case moves. * Keep the backend prewarm files as they are on main An all-files formatter pass reflowed two studio/backend files that have nothing to do with the WSL installer change and pulled them into this PR's diff. Restore them so the blast radius is only the installer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Strip cmd's quotes before trimming trailing blanks Trimming first found nothing to trim, because the closing quote was still the last character, so a Windows TEMP ending in spaces came back untrimmed and the directory check then cleared it. Win32 strips those spaces from a path and [ -d ] does not. The test stub was also wrong in a way that hid this: it consumed the quotes itself, but cmd's echo prints its argument verbatim, quotes included. With the stub corrected the new case fails against the previous commit and the rest still pass. * Fall through to a local Windows temp when TEMP is on a share A script written under a redirected %TEMP% is a remote script, and RemoteSigned refuses an unsigned one, so roaming-profile users would silently stop getting the shortcut they used to get. Rather than relax the policy back to Bypass, try %LOCALAPPDATA%\Temp and then %SystemRoot%\Temp, skipping any candidate that is UNC or still unexpanded. install.ps1 solves the same problem for the launcher by relaxing the policy because the launcher cannot move; this script can. * Reject mapped network drives as temporary locations too Z:\Temp is the same share and the same remote zone as its UNC spelling, and install.ps1 treats DriveType.Network as remote for that reason, but the filter only rejected paths beginning with two backslashes. A mapped drive was therefore accepted and then refused by RemoteSigned at launch. Read the mapped letters from net use and skip candidates on them; if that cannot be read the set is empty and only the UNC check applies, which is the previous behaviour rather than a new failure. * Keep shortcuts working when no Windows drive is mounted With [automount] enabled=false, interop still works but no Windows directory is reachable as a Linux path, so no candidate can be allocated and the file-based launch cannot run at all. Those users got a shortcut before the move off /tmp, so losing it silently is a regression rather than a pre-existing gap. The generated script is now captured once and either written to the Windows temp and run with -File, or handed to powershell on stdin. The stdin path needs no policy relaxation, because execution policy applies to -File and not to -Command, and nothing has to be mounted. It is fed from its own pipe so it cannot drink the installer's stdin under curl | sh. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Validate both halves of the compiler watch in the WSL control The control checked only the process half, so a FileSystemWatcher that never attached, or stopped reporting creations, looked exactly like a run that dropped no library. TempLibraries is the half the Bitdefender report in #10540 keyed on, and the install below trusts it. Require both, as windows-no-compiler-ci.yml already does. * Check the shell suite's failure counter after the last assertion The only $FAIL check sat before the static half of the file. `bad` records a failure and returns success, so an assertion that ran after it printed FAIL and left the script exiting 0, which made the launch-path regression checks ineffective in CI. Moved the summary and the check to the end of the file. * Do not render install.sh with a Windows bash The new here-string renderer drives install.sh through bash, and on a Windows runner bash is Git Bash or the WSL stub in System32, neither of which install.sh is ever run by: its WSL arm is gated on /proc/version naming microsoft, so the shell reading this here-string is always a POSIX one inside the distro. The rendered text is a pure function of install.sh's bytes, so the Linux and macOS legs cover the subject and the Windows leg only contributed three failures. Skipped there, the way the repository already skips its other bash-driven installer tests. Also report the shell's own diagnostic rather than letting CalledProcessError replace it with a 6 KB repr of the script. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
42 lines
3 KiB
Batchfile
42 lines
3 KiB
Batchfile
@echo off
|
|
REM SPDX-License-Identifier: AGPL-3.0-only
|
|
REM Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
rem RemoteSigned, not Bypass. RemoteSigned refuses an unsigned script only in the Internet or
|
|
rem Untrusted zone, and setup.ps1 normally sits beside this file inside an installed package, so it
|
|
rem is MyComputer-zone and loads unsigned either way. Pairing a relaxed policy with a PowerShell
|
|
rem launch is a token endpoint products score, and it was buying nothing in that case. Which product
|
|
rem scored what: tests/studio/test_installer_av_shapes.py (AV_SHAPES_RECORD).
|
|
rem
|
|
rem Two populations could regress, and both are handled below rather than assumed away.
|
|
rem
|
|
rem 1. A package unzipped from a download, where setup.ps1 carries a mark of the web that
|
|
rem RemoteSigned honours and Bypass ignored. Unblock-File clears it. Execution policy governs
|
|
rem script FILES and not -Command, so that call runs whatever the machine policy is, and clearing
|
|
rem an absent stream is a no-op that writes nothing.
|
|
rem
|
|
rem 2. setup.ps1 living on a share. Execution policy is judged on the script file's ZONE, and a
|
|
rem dotted-FQDN UNC (\\corp.example.com\...), a DFS path or an IP-literal share lands in the
|
|
rem Internet zone -- where RemoteSigned refuses an unsigned script and Unblock-File cannot save
|
|
rem it, because with no Zone.Identifier stream present the PATH decides and there is nothing to
|
|
rem clear. install.ps1:3405-3421 already steps down to Bypass for exactly this case when it
|
|
rem writes the Start Menu shortcut; the probe below reuses that logic rather than inventing a
|
|
rem second answer. A plain \\server\share and a mapped drive are Intranet and stay RemoteSigned.
|
|
rem
|
|
rem The probe also does the Unblock-File, so this costs one process rather than two.
|
|
rem
|
|
rem The path travels in an environment variable rather than being interpolated into the command
|
|
rem string, so an apostrophe in the install path cannot break the quoting.
|
|
rem
|
|
rem -NoProfile on the probe only, and deliberately NOT on the launch below: setup.ps1 is run with
|
|
rem profiles loaded on purpose, and tests/studio/test_amd_venv_repair_loop.ps1 drives a profile that
|
|
rem sets Set-StrictMode against it.
|
|
setlocal
|
|
set "UNSLOTH_SETUP_SCRIPT=%~dp0setup.ps1"
|
|
set "UNSLOTH_SETUP_POLICY=RemoteSigned"
|
|
for /f "usebackq delims=" %%P in (`powershell -NoProfile -Command "$p = $env:UNSLOTH_SETUP_SCRIPT; Unblock-File -LiteralPath $p -Confirm:$false -ErrorAction SilentlyContinue; $remote = $p -like '\\*'; if (-not $remote) { try { $remote = ([System.IO.DriveInfo]::new([System.IO.Path]::GetPathRoot($p))).DriveType -eq 'Network' } catch {} }; if ($remote) { 'Bypass' } else { 'RemoteSigned' }"`) do set "UNSLOTH_SETUP_POLICY=%%P"
|
|
|
|
rem Defaulting to RemoteSigned above means a probe that fails to run leaves the tightened policy in
|
|
rem place rather than silently restoring the relaxed one.
|
|
powershell -ExecutionPolicy %UNSLOTH_SETUP_POLICY% -File "%UNSLOTH_SETUP_SCRIPT%" %*
|
|
exit /b %ERRORLEVEL%
|