vrr/unsloth
2
0
Fork 0
mirror of https://github.com/unslothai/unsloth.git synced 2026-10-29 18:25:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 35

Locate the MTP drafter-drop fallback by its shape, not one spelling (#13922)

Browse source 
`(Python 3.13, rest)` is red on main since #13806 (1be7795180),
including on #13909's own run:

```
FAILED studio/backend/tests/test_tensor_parallel.py::test_mtp_decode_probe_wired_under_tensor_parallel
AssertionError: the probe must precede the drafter-drop fallback
```

#13806 is right. It made an EAGLE-3 drafter droppable on a failed load
like the others, so the fallback in `LlamaCppBackend.load_model` became

```python
if (
    not healthy
    and (
        _spec_requested_mtp
        or _spec_requested_dspark
        or _spec_requested_dflash
        or _spec_requested_eagle3
    )
    and not _load_cancelled()
):
```

The test located that block with `src.find("and (_spec_requested_mtp or
_spec_requested_dspark or _spec_requested_dflash)")`, the old one-line
spelling. After the change `find` returns -1, so `0 <= probe < fallback`
fails even though the probe still comes first. The ordering it guards is
intact.

The test now finds the fallback by its shape: `not healthy and (`
followed by the `_spec_requested_mtp or _spec_requested_dspark ...
_spec_requested_dflash` alternatives, whitespace and line breaks
allowed, so adding another drafter kind or rewrapping does not break it.
If no such block exists it fails with its own message (`load_model no
longer has a drafter-drop fallback for MTP`) rather than reporting a
misleading order failure.

Verification:
- `studio/backend/tests/test_tensor_parallel.py`: fails on main as
above, 111 passed with this change.
- Sabotage, `_spec_requested_dflash` removed from the fallback in
`llama_cpp.py`: fails with `load_model no longer has a drafter-drop
fallback for MTP`.
- Sabotage, the fallback block moved ahead of the probe in the source
the test reads: fails with `the probe must precede the drafter-drop
fallback`.
- `tests/test_formatter_fixed_point.py`: passes.
This commit is contained in:
Daniel Han 2026-10-15 12:36:23 -07:00 • committed by GitHub
parent a2003f1a05
commit bd063767f4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 10 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

13
studio/backend/tests/test_tensor_parallel.py
View file

@ -19,6 +19,7 @@ from __future__ import annotations
import asyncio
import inspect
import re
import socket
import sys
import threading
@ -329,10 +330,16 @@ def test_mtp_decode_probe_wired_under_tensor_parallel():
# shared drafter-drop fallback fires.
after = src[probe : probe + 900]
assert "_with_flash_attn_off" in after and "healthy = False" in after
fallback = src.find(
"and (_spec_requested_mtp or _spec_requested_dspark or _spec_requested_dflash)"
# The drafter-drop fallback is `if not healthy and (<any drafter requested>) ...`. Matched on
# its shape, not its exact spelling: #13806 added eagle3 to that list and wrapped it over
# several lines, and a one-line literal then found nothing.
match = re.search(
r"not healthy\s+and\s*\(\s*_spec_requested_mtp\s+or\s+_spec_requested_dspark\b"
r"[^)]*?_spec_requested_dflash\b",
src,
)
assert 0 <= probe < fallback, "the probe must precede the drafter-drop fallback"
assert match is not None, "load_model no longer has a drafter-drop fallback for MTP"
assert probe < match.start(), "the probe must precede the drafter-drop fallback"
def test_probe_mtp_decode_returns_false_on_crash(monkeypatch):