mirror of
https://github.com/LostRuins/koboldcpp.git
synced 2026-10-26 08:09:57 +00:00
d454a204ef
* llama: create new fused op for QSA scoring * enroll glm5.3-next too * tests : initialize the remaining tensors in test_kpool_mask The harness sentinels were left uninitialized, so leftover memory could read as NaN and fail the NaN check. * ggml: generalize the k-pool mask op into ggml_top_k_mask The k-pool mask op only covered the pooled QSA selection of qwen4exp and glm5-next, while the DSA models and the DeepSeek V4 CSA build the same top-k mask by hand with fill, set_rows and add. ggml_top_k_mask(mask, top_k, score, blocks, extra) keeps the mask cells of the live picks and the extra cells, and sets the rest to -inf. Every row tensor is 4D in the layout of top_k, and the mask is broadcast to those rows, so a selection can also be made per head. score, blocks and extra are optional: without score every pick is live, without blocks a pick is a cell, and extra holds the cells a row keeps without a pick. llm_graph_context::build_fused_top_k_mask views the row tensors in the layout of kq_mask and registers the fused node. The k-pool models, the DSA attention of build_attn and the DeepSeek V4 CSA mask use it when the probe enables the op, and keep their current path on backends without it. --------- Co-authored-by: Pascal <admin@serveurperso.com> |
||
|---|---|---|
| .. | ||
| ggml-alloc.h | ||
| ggml-backend.h | ||
| ggml-blas.h | ||
| ggml-cann.h | ||
| ggml-cpp.h | ||
| ggml-cpu.h | ||
| ggml-cuda.h | ||
| ggml-et.h | ||
| ggml-hexagon.h | ||
| ggml-metal.h | ||
| ggml-opencl.h | ||
| ggml-openvino.h | ||
| ggml-opt.h | ||
| ggml-rpc.h | ||
| ggml-sycl.h | ||
| ggml-virtgpu.h | ||
| ggml-vulkan.h | ||
| ggml-webgpu.h | ||
| ggml-zdnn.h | ||
| ggml-zendnn.h | ||
| ggml.h | ||
| gguf.h | ||