Raffaele 674e7dc136 feat(cli): say which mode a run used, and give --moe-stream a cache by default (#187) ... A first command with nothing but -m, -p and -t ran plain llama.cpp on mmap: streaming off, cache off, and a dense policy that only applies once streaming is on. Nothing in the report said so, because the moe-stream: block only prints when streaming is enabled, so a baseline run read as a measurement of this engine and got reported as one (#186). A mode: line is now printed on every run. With streaming off on a MoE architecture the build has a recipe for it says the run is a baseline and names the flag; on any other model it says the architecture is not one this build streams. RunSummary carries the model's arch so the CLI can tell those apart. --cache-mb defaults to auto whenever --moe-stream is on. The previous default of 0 meant the cache was off, which re-reads every routed expert from flash every token. On a 16 GB host streaming Qwen3.6-35B-A3B Q4_K_M, 63 tokens, -t 8 --overlap: 1.164 to 2.351 tok/s and 585 to 238 MiB per token, same output. The default is resolved in the CLI, not in the library, so an embedder passing 0 still means no cache; an explicit --cache-mb or BMOE_CACHE_MB still wins, including an explicit 0. Reported by @eiffel31.		2026-08-29 20:42:17 +02:00
..
CMakeLists.txt	chore: sweep the retired governor's residue from core and CLI	2026-07-17 10:04:27 +02:00
main.cpp	feat(cli): say which mode a run used, and give --moe-stream a cache by default (#187)	2026-08-29 20:42:17 +02:00