docs: document the fused expert layout and gemma4 support

adding-a-model.md: the merged ffn_gate_up_exps layout is now a single recipe
row (fused suffix first, nullptr tail); refresh the gate instructions for the
per-arch fixtures. roadmap.md: fused and shared-expert layouts are supported.
limitations.md: note that shared experts stay resident, lowering the streamed
fraction. seam.md: the recipe-driven suffixes and the naming-convention
fragility that the gates guard on each submodule bump.
This commit is contained in:
Helldez 2026-07-11 09:52:17 +02:00
parent 6260fa2997
commit 4fc46e3ac3
4 changed files with 44 additions and 21 deletions

View file

@ -23,8 +23,10 @@ expert-selective streaming that stays lossless and requires no fork.
- **CPU experts.** Streamed experts are computed on CPU; the rebind targets host memory.
GPU offload of the streamed experts is not supported (the dense parts can still use the
GPU). Decode is flash-I/O-bound anyway, so this is rarely the bottleneck.
- **Three separate projections.** Models that pack gate+up into one expert tensor are not
yet supported (see [adding-a-model.md](adding-a-model.md)).
- **Shared experts stay resident.** Architectures with an always-on shared expert (e.g.
`gemma4`) stream the routed experts but keep the shared expert — and any dense layers —
mmap-resident, so the streamed fraction (and the memory saving) is smaller than for a
purely routed model like `qwen3moe`.
- **Repack must stay off.** Loading uses `use_extra_bufts=false`; you cannot combine
streaming with weight repacking.
- **Windows throughput.** The cache's reserve-then-commit-per-slice path is heavier on