vrr/BigMoeOnEdge
2
0
Fork 0
mirror of https://github.com/Helldez/BigMoeOnEdge.git synced 2026-10-19 07:23:46 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

refactor(moe): drop the llada-moe recipe (diffusion, out of scope)

Browse source
This commit is contained in:
Helldez 2026-07-13 16:02:32 +02:00
parent 3e1cee676a
commit 19d3d5eb8f
4 changed files with 10 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

5
docs/limitations.md
View file

@ -19,9 +19,8 @@ serial path, and only a single ~25-line hook (with an explicit sunset) for the o
## Limitations
- **n=1 only.** The expert sparsity exists only for single-token decode, so streaming is
incompatible with speculative decoding, batching, or a diffusion canvas. Prefill streams
the union of the prompt's routed experts (still far below the full bank, but larger than
one token's).
incompatible with speculative decoding or batching. Prefill streams the union of the
prompt's routed experts (still far below the full bank, but larger than one token's).
- **CPU experts.** Streamed experts are computed on CPU; the rebind targets host memory.
GPU offload of the streamed experts is not supported (the dense parts can still use the
GPU). Decode is flash-I/O-bound anyway, so this is rarely the bottleneck.