Concedo
cce035abcc
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cuda.Dockerfile
# .github/actions/windows-setup-cuda/action.yml
# .github/workflows/build-cpu.yml
# .github/workflows/build-cuda-windows.yml
# .github/workflows/ci-self-hosted-cpu.yml
# .github/workflows/docker.yml
# .github/workflows/models-check.yml
# .github/workflows/release-publish.yml
# .github/workflows/release.yml
# CODEOWNERS
# ci/run.sh
# cmake/win-app.cmake
# docs/backend/snapdragon/README.md
# docs/ops.md
# docs/ops/OpenCL.csv
# docs/ops/WebGPU.csv
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-common.h
# ggml/src/ggml-hexagon/htp-fusion.cpp
# ggml/src/ggml-hexagon/htp-fusion.h
# ggml/src/ggml-hexagon/htp-kparams.cpp
# ggml/src/ggml-hexagon/htp-kparams.h
# ggml/src/ggml-hexagon/htp-repack.h
# ggml/src/ggml-hexagon/htp/conv2d-ops.c
# ggml/src/ggml-hexagon/htp/conv2d-ops.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-norm.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.h
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q6_k_f32_tiled.cl
# ggml/src/ggml-opencl/kernels/repeat.cl
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# scripts/sync_vendor.py
# skills/code-review/SKILL.md
# src/CMakeLists.txt
# tests/fusion/MTL.csv
# tests/gguf-model-data.cpp
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tests/test-llama-context.cpp
# tests/test-quantize-fns.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/mtmd/CMakeLists.txt
# tools/server/README.md
2026-11-08 11:45:08 +08:00
Aman Gupta
e95426f169
llama_moe_cache: run missed experts on CPU, add tuning script ( #30966 )
...
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* moe-cache : compute part of the misses on the CPU
--moe-cache-cpu-share F computes a share F of the MoE cache misses on the CPU
from the host experts instead of uploading them. The graph builds the expert
FFN twice: on the device from the cache slots, where the misses left to the CPU
read zero slots, and on the CPU from the host experts, where the experts
computed on the device have id -1. The two results are added before the down
bias.
- ggml-cpu: MUL_MAT_ID skips negative ids and writes zero rows
- sched: the copy callback is called for all the inputs of a split, and before
the host sync when there are no events
- the slot lookup callback reads the inputs of the CPU part together with the
selected experts, so the CPU part does not wait for the device, and its
result is copied back asynchronously
- each token uses distinct zero slots, mul_mat_id does not support an expert
selected twice by a token
- LLAMA_MOE_CACHE_TRACE writes the selected experts of each lookup, for a
simulator of the cache
* scripts : add moe-cache-tune.py to pick the MoE cache arguments
The script wraps a llama-server command: it starts the server at two cache
sizes and three CPU shares, fits the upload and CPU speeds of the machine, and
replays the routing of the runs through a copy of the cache logic to predict
other sizes and shares. It prints the recommended --moe-cache-mib and
--moe-cache-cpu-share.
docs/moe-cache.md describes the MoE cache arguments and the script.
* add am17an as codeowner for MoE cache
* moe-cache : upload all the misses until the cache is full
With a high --moe-cache-cpu-share, few misses are uploaded, so an empty cache
fills slowly and more misses than needed go to the CPU. The CPU share now
starts once all the slots of a cache group hold an expert. moe-cache-tune.py
replays the same rule.
* add backend sync for async transfer queue
* moe-cache : let the scheduler copy the results of the CPU part
* Fix MSVC compilation
* remove sync change
* llama-graph : extract the CPU share condition into can_cpu_share
* llama-graph : make llm_graph_params::moe_cache const
The graph construction no longer updates the MoE cache directly: the CPU
parts of the layers are accumulated in the graph result and applied to
the cache in llama_context::process_ubatch, after the graph inputs are
set and before the graph is computed.
Assisted-by: dsh:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (Mac Studio, M5 Ultra, 256GB)
* scripts : replay the ClassLRU policy in moe-cache-tune.py
* moe-cache : pick CPU misses by CRF instead of random (#31853 )
* moe-cache : remove the backend logic of the CPU part from the graph
The CPU part reads the host experts, so the scheduler places it on the
CPU without set_tensor_backend. The cache gives no CPU map for batches
that a device would offload. Graphs with a CPU part look up their own
slot map, so the graph does not register the part with the cache.
ggml-backend : the inputs of the next split that are already computed
are copied to host memory before the current split runs, so a CPU split
does not wait for the device split before it. This replaces the stash
of the cache, and the copy callback is again called for host weights
only.
* llama-graph : return the MoE cache slots and CPU ids in a struct
The scheduler prefetch no longer checks that the next split runs on
another backend: the inputs of a split are never on its own backend,
so the input loop already skips that case.
---------
Co-authored-by: Gaurav Garg <gaugarg@nvidia.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: Alexey Kopytko <alexey@kopytko.com>
2026-11-08 00:20:58 +08:00
Concedo
4d72f598ab
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/intel.Dockerfile
# .github/workflows/build-cmake-pkg.yml
# .github/workflows/build-sanitize.yml
# .github/workflows/ci-self-hosted-cpu.yml
# AUTHORS
# CMakeLists.txt
# ci/run.sh
# docs/ops.md
# docs/ops/WebGPU.csv
# ggml/CMakeLists.txt
# ggml/src/ggml-cann/aclnn_ops.cpp
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-cpu/arch/riscv/repack.cpp
# ggml/src/ggml-cuda/mmq.cuh
# ggml/src/ggml-et/ggml-et.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-common.h
# ggml/src/ggml-hexagon/htp-kparams.cpp
# ggml/src/ggml-hexagon/htp-kparams.h
# ggml/src/ggml-hexagon/htp-opnode.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# ggml/src/ggml-hexagon/htp/glu-ops.c
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-sigmoid.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q2_0_f32.cl
# ggml/src/ggml-opencl/kernels/mul_mv_f32_f32.cl
# ggml/src/ggml-opencl/kernels/sigmoid.cl
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/rope.cpp
# ggml/src/ggml-virtgpu/ggml-backend-device.cpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/memset.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/sum_rows.wgsl
# ggml/src/ggml-zdnn/fattn.cpp
# scripts/snapdragon/run.py
# scripts/sync-ggml.last
# tests/fusion/MTL.csv
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tests/test-model-resolution.cpp
# tests/test-mtmd-impl.cpp
# tests/test-rope.cpp
# tools/export-lora/export-lora.cpp
# tools/mtmd/CMakeLists.txt
2026-11-06 19:47:23 +08:00
Adrien Gallouët
bbc2e4b657
common : restrict HF token to huggingface.co hosts ( #31730 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-11-06 08:43:33 +01:00
Adrien Gallouët
52e53a76df
common : delete common_get_path_from_env() ( #31444 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-11-04 14:21:38 +01:00
Concedo
75ae20d53f
Merge commit ' 00ef136649' into concedo_experimental
...
# Conflicts:
# README.md
# docs/ops.md
# docs/ops/OpenCL.csv
# ggml/src/ggml-hexagon/htp/dma-queue.c
# ggml/src/ggml-hexagon/htp/dma-queue.h
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/cpy.hpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# requirements/requirements-gguf_editor_gui.txt
# requirements/requirements-server-bench.txt
# requirements/requirements-tool_bench.txt
# scripts/sync_vendor.py
# tests/CMakeLists.txt
# tests/fusion/MTL.csv
# tests/gguf-model-data.cpp
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tests/test-llama-archs.cpp
# tools/server/README.md
2026-11-03 22:30:09 +08:00
Adrien Gallouët
a34de8618b
common : delete fs_path_to_utf8() ( #31423 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-10-30 16:20:04 +01:00
Concedo
4f78d791c5
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/behavior-check.yml
# .github/workflows/build-apple.yml
# CMakeLists.txt
# docs/backend/SYCL.md
# docs/ops.md
# docs/ops/OpenCL.csv
# docs/speculative.md
# ggml/CMakeLists.txt
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-cpu/amx/amx.cpp
# ggml/src/ggml-cuda/common.cuh
# ggml/src/ggml-cuda/fattn.cu
# ggml/src/ggml-et/ggml-et.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-kparams.cpp
# ggml/src/ggml-hexagon/htp-kparams.h
# ggml/src/ggml-hexagon/htp-opnode.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/concat-ops.c
# ggml/src/ggml-hexagon/htp/concat-ops.h
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/im2col-ops.c
# ggml/src/ggml-hexagon/htp/im2col-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# ggml/src/ggml-hexagon/htp/ssm-conv.h
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/cvt.cl
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-openvino/openvino/op/scale.cpp
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/fattn-mkl.cpp
# ggml/src/ggml-sycl/fattn-onednn.cpp
# ggml/src/ggml-sycl/fattn.cpp
# ggml/src/ggml-sycl/getrows.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmq.cpp
# ggml/src/ggml-virtgpu/ggml-backend-buffer.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-zdnn/ggml-zdnn.cpp
# scripts/hip/gcn-cdna-vgpr-check.py
# scripts/sync_vendor.py
# skills/code-review/SKILL.md
# tests/CMakeLists.txt
# tests/fusion/MTL.csv
# tests/llama-context/cases-kv.cpp
# tests/llama-context/cases-rs.cpp
# tests/llama-context/test-util.cpp
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tests/test-quantize-fns.cpp
# tests/test-reasoning-budget.cpp
2026-10-30 23:09:16 +08:00
Adrien Gallouët
aa588a3463
common : drop the Windows argv reconstruction ( #31417 )
...
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / python type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
The manifest from #31329 already makes argv UTF-8.
Fixes #26416
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-10-30 14:02:00 +01:00
Concedo
9792f6b4cb
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/ci-self-hosted-webgpu.yml
# .github/workflows/release.yml
# CMakeLists.txt
# common/fit.h
# scripts/snapdragon/qdc/run_qdc_jobs.py
# tests/fusion/MTL.csv
# tests/llama-context/cases-rs.cpp
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tests/test-llama-archs.cpp
# tests/test-llama-context.cpp
# tests/test-server-component.cpp
# tools/mtmd/CMakeLists.txt
# tools/server/CMakeLists.txt
# tools/server/README.md
2026-10-28 22:25:38 +08:00
Tarek Dakhran
19ff817fb2
model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B ( #31257 )
...
* model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Assisted-by: Claude Opus 5.5
* convert : read the LFM3-Audio config.json as released
A transformers that knows lfm3_audio rewrites the config into another layout (no lfm/encoder keys).
Assisted-by: Claude Opus 5.5
* mtmd : disable flash attention for the LFM3-Audio depthformer
Assisted-by: Claude Opus 5.5
* tts : add a voice option for LFM3-Audio and LFM3-Omni
The model card offers a female and a male voice; --tts-voice for llama-tts, "voice" for the server /tts.
Assisted-by: Claude Opus 5.5
* clean up template handling
* convert : detect LFM3-Audio and LFM3-Omni by model_type
Assisted-by: Claude Opus 5.5
* move banned tokens --> logits bias --> mtmd_helper
* leave mtmd-cli unchanged
* no more template_asr_system
* left over
* llama : keep the LFM3-Audio encoder convs and codec unquantized
Assisted-by: Claude Opus 5.5
* common, mtmd : shorten comments
Assisted-by: Claude Opus 5.5
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-10-27 18:47:14 +01:00
Ruben Ortlam
85cc432e2c
server: add router device memory margin parameter for dynamic unloading ( #21231 )
...
* server: add --models-memory-max parameter to allow dynamically unloading models when they exceed a memory size threshold
estimate with to-be-loaded model size included
use no_alloc to get memory requirements for model load
only set model memory_mb if not previously calculated
use memory margin instead of total size limit, apply to each device separately
add server memory debug logging
move llama_context_device_memory function to llama-ext.h
fix model count exceeded check
improve memory_per_device map naming
improve variable naming, fix style
also strip models memory margin from child processes
cont : clean-up
replace device memory map with buft memory map. Use llama_get_memory_breakdown
extract duplicated check into helper function
move model memory estimation to subprocess
precompute name->buft map, map GPU host types to CPU buft
cleanup unused variable
remove duplicated init calls
* adapt to upstream changes
* wip
* common_memory_reports internface
* SERVER_ROLE_MEASURE
* server-models-sched
* fix build
* server : fix memory accounting of measure children and claim spin
A measure child runs no_alloc, so the free memory it reports does not
include its estimate: subtracting self there erased the memory held by
other processes, down to zero with --fit on, and the next load could
overshoot the margin. Subtract self only for instance reports.
A queued load only checked --models-max before claiming, so when the
memory margin blocked it behind a busy model, load() threw and the
waiter retried in a tight loop. Refuse the claim while the limits are
exceeded and no idle model can be unloaded, so the waiter sleeps on
the condition variable until one goes idle.
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Pascal <admin@serveurperso.com>
2026-10-27 12:14:42 +01:00
Julien Chaumond
74a1fd3d03
common : --model or -hf is required ( #31242 )
...
Assisted-by: Codex
2026-10-27 11:55:48 +01:00
Concedo
329f6a5b31
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-cpu.yml
# ci/run.sh
# common/CMakeLists.txt
# docs/backend/SYCL.md
# docs/ops.md
# docs/ops/Hexagon.csv
# docs/ops/SYCL.csv
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-cpu/kleidiai/kernels.cpp
# ggml/src/ggml-cpu/kleidiai/kernels.h
# ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
# ggml/src/ggml-hexagon/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-opnode.h
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/allreduce-ops.h
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# ggml/src/ggml-hexagon/htp/argsort-ops.h
# ggml/src/ggml-hexagon/htp/binary-ops.c
# ggml/src/ggml-hexagon/htp/binary-ops.h
# ggml/src/ggml-hexagon/htp/concat-ops.c
# ggml/src/ggml-hexagon/htp/concat-ops.h
# ggml/src/ggml-hexagon/htp/cpy-ops.c
# ggml/src/ggml-hexagon/htp/cpy-ops.h
# ggml/src/ggml-hexagon/htp/cumsum-ops.c
# ggml/src/ggml-hexagon/htp/diag-ops.c
# ggml/src/ggml-hexagon/htp/dma-queue.h
# ggml/src/ggml-hexagon/htp/fill-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.h
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.h
# ggml/src/ggml-hexagon/htp/gather-rows.c
# ggml/src/ggml-hexagon/htp/gather-rows.h
# ggml/src/ggml-hexagon/htp/get-rows-ops.c
# ggml/src/ggml-hexagon/htp/get-rows-ops.h
# ggml/src/ggml-hexagon/htp/glu-ops.c
# ggml/src/ggml-hexagon/htp/hex-common.h
# ggml/src/ggml-hexagon/htp/hex-profile.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-arith.h
# ggml/src/ggml-hexagon/htp/hvx-base.h
# ggml/src/ggml-hexagon/htp/hvx-floor.h
# ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/hvx-norm.h
# ggml/src/ggml-hexagon/htp/hvx-sigmoid.h
# ggml/src/ggml-hexagon/htp/im2col-ops.c
# ggml/src/ggml-hexagon/htp/im2col-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/matmul-ops.h
# ggml/src/ggml-hexagon/htp/pad-ops.c
# ggml/src/ggml-hexagon/htp/pool-ops.c
# ggml/src/ggml-hexagon/htp/pool-ops.h
# ggml/src/ggml-hexagon/htp/repeat-ops.c
# ggml/src/ggml-hexagon/htp/roll-ops.c
# ggml/src/ggml-hexagon/htp/rope-ops.c
# ggml/src/ggml-hexagon/htp/rope-ops.h
# ggml/src/ggml-hexagon/htp/set-rows-ops.c
# ggml/src/ggml-hexagon/htp/set-rows-ops.h
# ggml/src/ggml-hexagon/htp/softmax-ops.c
# ggml/src/ggml-hexagon/htp/softmax-ops.h
# ggml/src/ggml-hexagon/htp/solve-tri-ops.c
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# ggml/src/ggml-hexagon/htp/ssm-conv.h
# ggml/src/ggml-hexagon/htp/sum-rows-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.h
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/conv2d.cpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/esimd.hpp
# ggml/src/ggml-sycl/fattn-mkl.cpp
# ggml/src/ggml-sycl/fattn-sparse.cpp
# ggml/src/ggml-sycl/fattn.cpp
# ggml/src/ggml-sycl/fattn.hpp
# ggml/src/ggml-sycl/fusion.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/mmvq.hpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-mtmd-impl.cpp
# tests/test-server-component.cpp
# tests/test-tokenizer-0.cpp
# tools/mtmd/CMakeLists.txt
# tools/server/README.md
2026-10-24 12:10:10 +08:00
Concedo
4224e52453
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .flake8
# .github/workflows/build-and-test-snapdragon.yml
# .github/workflows/build-webgpu.yml
# .github/workflows/release.yml
# CMakeLists.txt
# cmake/common.cmake
# docs/backend/snapdragon/README.md
# docs/backend/snapdragon/linux.md
# docs/backend/snapdragon/windows.md
# docs/backend/zDNN.md
# docs/build.md
# ggml/cmake/common.cmake
# ggml/src/ggml-hexagon/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-repack.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/libggml-htp.inf
# ggml/src/ggml-opencl/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/gemm_noshuffle_q4_0_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_noshuffle_q4_k_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_noshuffle_q5_k_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_noshuffle_q6_k_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemm_noshuffle_q8_0_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q1_0_f32.cl
# ggml/src/ggml-opencl/kernels/mul_mm_f16_f32_l4_lm.cl
# ggml/src/ggml-opencl/kernels/mul_mv_mxfp4_f32_flat.cl
# ggml/src/ggml-opencl/kernels/rope.cl
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/concat.cpp
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/count-equal.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/dpct/helper.hpp
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-sycl/fattn-common.hpp
# ggml/src/ggml-sycl/fattn-sparse.cpp
# ggml/src/ggml-sycl/fattn-tile.cpp
# ggml/src/ggml-sycl/fattn.cpp
# ggml/src/ggml-sycl/fused-gemm.cpp
# ggml/src/ggml-sycl/fused-gemm.hpp
# ggml/src/ggml-sycl/getrows.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/gla.cpp
# ggml/src/ggml-sycl/mem.cpp
# ggml/src/ggml-sycl/mmq.cpp
# ggml/src/ggml-sycl/topk-moe.cpp
# ggml/src/ggml-sycl/upscale.cpp
# ggml/src/ggml-sycl/vecdotq.hpp
# ggml/src/ggml-zdnn/common.hpp
# ggml/src/ggml-zdnn/ggml-zdnn.cpp
# ggml/src/ggml-zdnn/mmq.cpp
# scripts/check-apiabi-compat.sh
# scripts/snapdragon/build.py
# scripts/snapdragon/ggml-hexagon-inspect.py
# scripts/snapdragon/run.py
# scripts/snapdragon/sdk.py
# skills/slice-model/slice_model.py
# src/llama-vocab.cpp
# tests/CMakeLists.txt
# tests/fusion/MTL.csv
# tests/test-backend-ops.cpp
# tests/test-jinja.cpp
# tests/test-llama-archs.cpp
# tests/test-mtmd-impl.cpp
# tools/CMakeLists.txt
# tools/cli/README.md
# tools/completion/README.md
# tools/mtmd/CMakeLists.txt
# tools/server/CMakeLists.txt
# tools/server/README.md
# ty.toml
2026-10-24 12:02:00 +08:00
Adrien Gallouët
21ec307146
common : allow concurrent downloads across processes ( #29312 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-10-23 23:52:56 +02:00
Xuan-Son Nguyen
7adf160c41
server: add --connect, allow p2p access to server outside LAN ( #28277 )
...
* init
* wip
* wire it up
* make it clear that it's experimental
* add props.connect_code
* nits
* fix windows build
* server: do not advertise a connect code without the tunnel
--connect-code on its own had no effect other than being served in
/props, so the Web UI showed a remote access code that no tunnel was
serving. Clear it when --connect is off, and say so once at startup.
The teardown comment claimed that terminate() closes the child stdout.
It does not: the log thread leaves fgets() only once every writer of
that pipe is gone, so a descendant of llama-connect would block the
join forever. Describe the real contract instead.
Also restore the --reasoning-preserve default in the generated docs.
* Update tools/connect/CMakeLists.txt
Co-authored-by: Adrien Gallouët <angt@huggingface.co>
* server: dial llama-connect on the bound port and a reachable host
Pass ctx_http.port so --port 0 and the GCP override work, pick the
dial host from --host (wildcards map to loopback, IPv6 is bracketed)
and refuse --connect when only unix sockets are bound.
---------
Co-authored-by: Pascal <admin@serveurperso.com>
Co-authored-by: Adrien Gallouët <angt@huggingface.co>
2026-10-20 00:23:14 +02:00
Concedo
8826b9c532
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/rocm.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# .github/workflows/build-sycl.yml
# .github/workflows/python-type-check.yml
# .github/workflows/release.yml
# .pi/gg/SYSTEM.md
# CMakeLists.txt
# docs/backend/snapdragon/developer.md
# docs/backend/zDNN.md
# examples/sycl/build.sh
# examples/sycl/win-build-sycl.bat
# ggml/CMakeLists.txt
# ggml/src/ggml-hexagon/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-opnode.h
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/get-rows-ops.c
# ggml/src/ggml-hexagon/htp/get-rows-ops.h
# ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-dcache.c
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/htp-tensor.h
# ggml/src/ggml-hexagon/htp/hvx-mm-kernels-float.h
# ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/hvx-quant.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/matmul-ops.h
# ggml/src/ggml-metal/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/clamp.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f16.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_q4_0.cl
# ggml/src/ggml-opencl/kernels/flash_attn_f32_q8_0.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_iq4_nl_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q1_0_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_0_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_0_f32_spec.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_1_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_k_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_k_f32_o4.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q5_0_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q5_1_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q5_k_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q6_k_f32.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q6_k_f32_o4.cl
# ggml/src/ggml-opencl/kernels/gemv_noshuffle_q8_0_f32.cl
# ggml/src/ggml-openvino/openvino/op/glu_swiglu.cpp
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/convert.cpp
# ggml/src/ggml-sycl/dequantize.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/quants.hpp
# ggml/src/ggml-sycl/vecdotq.hpp
# scripts/server-bench.py
# scripts/snapdragon/run.py
# scripts/sync-ggml.last
# src/llama-vocab.cpp
# tests/llama-context/cases-kv.cpp
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tests/test-llama-archs.cpp
# tests/test-mtmd-impl.cpp
# tests/test-thread-safety.cpp
# tests/test-unicode.cpp
# tools/cli/README.md
# tools/mtmd/CMakeLists.txt
# tools/server/README.md
2026-10-16 13:43:20 +08:00
Concedo
c1ecdb2a78
Merge commit ' 5cdadfc326' into concedo_experimental
...
# Conflicts:
# docs/backend/SYCL.md
# docs/backend/snapdragon/README.md
# docs/ops.md
# docs/ops/zDNN.csv
# docs/speculative.md
# examples/diffusion/diffusion-cli.cpp
# examples/sycl/build.sh
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-cpu/kleidiai/kernels.cpp
# ggml/src/ggml-cpu/kleidiai/kernels.h
# ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/htp_iface.idl
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/gemm_moe_q8_1_dp4a.cl
# ggml/src/ggml-opencl/kernels/moe_sort_by_expert.cl
# ggml/src/ggml-rpc/CMakeLists.txt
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_id.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_id_gather.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_id_vec.wgsl
# scripts/gen-unicode-data.py
# scripts/snapdragon/run.py
# scripts/sync_vendor.py
# skills/code-review/SKILL.md
# tests/CMakeLists.txt
# tests/test-arg-parser.cpp
# tests/test-backend-ops.cpp
# tests/test-chat-auto-parser.cpp
# tests/test-chat-template.cpp
# tests/test-chat.cpp
# tests/test-jinja.cpp
# tests/test-server-component.cpp
# tests/test-unicode.cpp
# tools/cli/README.md
# tools/server/README.md
2026-10-16 13:26:05 +08:00
Concedo
b776b7a136
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-apple.yml
# .github/workflows/build-cache.yml
# .github/workflows/build-openvino.yml
# .github/workflows/ci-self-hosted-openvino.yml
# .github/workflows/release-publish.yml
# .github/workflows/release.yml
# CODEOWNERS
# ci/run.sh
# docs/backend/zDNN.md
# examples/speculative/speculative.cpp
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cuda/ggml-cuda.cu
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/binary-ops.c
# ggml/src/ggml-hexagon/htp/binary-ops.h
# ggml/src/ggml-hexagon/htp/concat-ops.c
# ggml/src/ggml-hexagon/htp/concat-ops.h
# ggml/src/ggml-hexagon/htp/cpy-ops.c
# ggml/src/ggml-hexagon/htp/cpy-ops.h
# ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/im2col-ops.c
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/matmul-ops.h
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-opencl/kernels/flash_attn_f32_f16.cl
# ggml/src/ggml-sycl/dmmv.cpp
# ggml/src/ggml-vulkan/ggml-vulkan-types.h
# ggml/src/ggml-zdnn/common.hpp
# ggml/src/ggml-zdnn/ggml-zdnn.cpp
# ggml/src/ggml-zdnn/mmf.cpp
# ggml/src/ggml-zdnn/utils.cpp
# ggml/src/ggml-zdnn/utils.hpp
# scripts/snapdragon/build.py
# skills/code-review/SKILL.md
# src/llama-context.cpp
# tests/.gitignore
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tests/testing.h
# tools/rpc/README.md
# tools/server/README.md
2026-10-16 13:13:44 +08:00
Concedo
9be8c136ae
Merge commit ' 23b0202a18' into concedo_experimental
...
# Conflicts:
# ggml/src/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# tests/test-batch-alloc.cpp
# tools/server/README.md
2026-10-16 00:15:23 +08:00
Pascal
2639a3b4e0
server: save only restorable slot states to the prompt cache ( #30498 )
...
* server: save only restorable slot states to the prompt cache
The idle-slot save loop stores every idle slot whenever a task starts,
while only completion tasks restore from the prompt cache. Rerank,
embedding, decision and infill states fill the cache up to its limit
without ever being read back, and slots already cached are offered
again at every task start.
Each slot now tracks with cache_pending whether its state is restorable
and missing from the cache: a completion release or an explicit slot
restore sets it, prompt_save consumes it, prompt_clear resets it. Both
save paths share this rule, and the LRU path restores only for
completion tasks.
* server: test that decision slots skip the prompt cache
Decision tasks never mark their slot as cacheable, so starting new
tasks must not save their idle slots to the RAM prompt cache, with or
without a unified KV cache.
* server: restore any reusing task from the prompt cache
Embedding and rerank tasks never reuse their state, so their slots are
cleared on release like child slots and the cache only holds states
that a later task can restore. Infill and decision tasks now restore
from the cache too, which replaces the cache_pending flag.
Decision models disable the prompt cache unless --cache-ram is given,
since their states seldom repeat.
2026-10-15 06:30:17 +02:00
Georgi Gerganov
3fb47677ea
common : add --spec-draft-n-ubatch ( #30459 )
...
* common : add --spec-draft-n-ubatch
Allow configuring the physical batch size of the draft context. The
drafters feed it at most the drafted tokens per sequence and chunk larger
batches by its own n_ubatch, so it does not need the ubatch of the target.
A smaller value shrinks the draft compute buffer and leaves more VRAM for
the target KV cache (the draft is fitted together with the target).
The default 0 keeps the current behavior of following --n-ubatch.
Assisted-by: pi:llama.cpp/Qwen3.8-Flash-Next
* cont : remove comment
[no ci]
2026-10-14 13:58:05 +03:00
Daniel Bevenius
77757a3467
examples : fix draft sampling and rejection handling ( #30095 )
...
* examples : fix draft sampling and rejection handling
This commit fixes two issues in the speculative example which are
described in detail in the linked issue.
Resolves: https://github.com/ggml-org/llama.cpp/issues/29975
* examples : add kv-unified requirement for parallel > 1
2026-10-12 12:02:24 +02:00
Mikolaj Kucharski
aa94f20861
args: add LLAMA_ARG_SLOT_SAVE_PATH env for --slot-save-path ( #30272 )
...
Allow configuring --slot-save-path via LLAMA_ARG_SLOT_SAVE_PATH so
it can be set from an EnvironmentFile in a systemd unit file.
2026-10-10 15:34:40 +02:00
Concedo
b3284ea444
Merge commit ' b42b7e6d30' into concedo_experimental
...
# Conflicts:
# .devops/cann.Dockerfile
# .devops/cpu.Dockerfile
# .devops/cuda.Dockerfile
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .devops/openvino.Dockerfile
# .devops/rocm.Dockerfile
# .devops/s390x.Dockerfile
# .devops/vulkan.Dockerfile
# .devops/zendnn.Dockerfile
# docs/backend/ZenDNN.md
# docs/function-calling.md
# examples/json_schema_pydantic_example.py
# examples/model-conversion/scripts/causal/modelcard.template
# examples/model-conversion/scripts/embedding/modelcard.template
# examples/model-conversion/scripts/utils/curl-embedding-server.sh
# examples/pydantic_models_to_grammar_examples.py
# ggml/src/ggml-hexagon/htp/im2col-ops.c
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/mmvq.hpp
# ggml/src/ggml-sycl/vecdotq.hpp
# scripts/compare-logprobs.py
# scripts/hip/gcn-cdna-vgpr-check.py
# scripts/server-bench.py
# scripts/server-test-function-call.py
# scripts/server-test-model.py
# scripts/server-test-parallel-tc.py
# scripts/server-test-structured.py
# tests/test-backend-ops.cpp
# tests/test-llama-archs.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/server/README.md
# tools/server/bench/README.md
2026-10-10 13:37:53 +08:00
Concedo
0890a5ee77
Merge commit ' 033df86b69' into concedo_experimental
...
# Conflicts:
# docs/backend/SYCL.md
# docs/development/parsing.md
# examples/speculative-simple/speculative-simple.cpp
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/get-rows-ops.c
# ggml/src/ggml-hexagon/htp/get-rows-ops.h
# ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-erf.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.h
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/element_wise.cpp
# ggml/src/ggml-sycl/element_wise.hpp
# ggml/src/ggml-sycl/fusion.cpp
# ggml/src/ggml-sycl/fwht.cpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/im2col.hpp
# ggml/src/ggml-sycl/pad.hpp
# ggml/src/ggml-sycl/rope.hpp
# ggml/src/ggml-sycl/upscale.hpp
# scripts/snapdragon/run.py
# scripts/sync_vendor.py
# src/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-backend-sampler.cpp
# tests/test-chat.cpp
# tests/test-llama-archs.cpp
# tools/imatrix/imatrix.cpp
# tools/server/README.md
2026-10-10 13:19:02 +08:00
Xuan-Son Nguyen
b42b7e6d30
server: use port 9931 by default ( #30159 )
...
* server: use port 9931 by default
* revert unrelated changes
2026-10-09 10:40:03 +02:00
Concedo
14cf69fa7a
Merge commit ' 63bef2728d' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-apple.yml
# .github/workflows/docker.yml
# .github/workflows/make-release.yml
# .github/workflows/models-check.yml
# .github/workflows/winget.yml
# CMakeLists.txt
# docs/backend/OPENVINO.md
# docs/build-riscv64-spacemit.md
# ggml/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/CMakeLists.txt
# ggml/src/ggml-hexagon/htp/dma-queue.c
# ggml/src/ggml-hexagon/htp/dma-queue.h
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.h
# ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/hmx-utils.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/matmul-ops.h
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# ggml/src/ggml-hexagon/htp/ssm-conv.h
# ggml/src/ggml-openvino/ggml-decoder.cpp
# ggml/src/ggml-openvino/ggml-decoder.h
# ggml/src/ggml-openvino/ggml-openvino-extra.cpp
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-openvino/openvino/op/fill.cpp
# ggml/src/ggml-openvino/openvino/op_table.cpp
# ggml/src/ggml-openvino/openvino/pass/fuse_moe_compressed.cpp
# ggml/src/ggml-openvino/openvino/translate_session.cpp
# ggml/src/ggml-openvino/utils.cpp
# ggml/src/ggml-openvino/utils.h
# scripts/make-release-checks.sh
# scripts/make-release-summary.txt
# scripts/snapdragon/run.py
# scripts/sync-ggml.last
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-recurrent-state-rollback.cpp
# tests/test-save-load-state.cpp
# tests/test-state-restore-fragmented.cpp
# tools/server/README.md
2026-10-08 23:43:45 +08:00
Aman Gupta
c811cb8f0a
llama: support MoE cache over multiple GPUs ( #30112 )
2026-10-08 15:50:49 +05:30
Aman Gupta
d6cf9acb25
llama : add a GPU cache for MoE experts kept in host memory ( #29887 )
...
* llama : add a GPU cache for MoE experts kept in host memory
Assisted-by: Claude
* use llama_moe_cache_ptr
2026-10-07 21:07:45 +03:00
Concedo
acef3c226f
Merge commit ' ebe18bee5a' into concedo_experimental
...
# Conflicts:
# .devops/openvino.Dockerfile
# .github/workflows/ai-issues.yml
# .github/workflows/build-3rd-party.yml
# .github/workflows/build-and-test-snapdragon.yml
# .github/workflows/build-android.yml
# .github/workflows/build-apple.yml
# .github/workflows/build-cache.yml
# .github/workflows/build-cann.yml
# .github/workflows/build-cmake-pkg.yml
# .github/workflows/build-cpu.yml
# .github/workflows/build-cross.yml
# .github/workflows/build-cuda-ubuntu.yml
# .github/workflows/build-cuda-windows.yml
# .github/workflows/build-ibm.yml
# .github/workflows/build-msys.yml
# .github/workflows/build-opencl.yml
# .github/workflows/build-openvino.yml
# .github/workflows/build-riscv.yml
# .github/workflows/build-sanitize.yml
# .github/workflows/build-sycl.yml
# .github/workflows/build-virtgpu.yml
# .github/workflows/build-vulkan.yml
# .github/workflows/build-wasm.yml
# .github/workflows/build-webgpu.yml
# .github/workflows/check-vendor.yml
# .github/workflows/ci-self-hosted-cpu.yml
# .github/workflows/ci-self-hosted-cuda.yml
# .github/workflows/ci-self-hosted-kleidiai.yml
# .github/workflows/ci-self-hosted-metal.yml
# .github/workflows/ci-self-hosted-openvino.yml
# .github/workflows/ci-self-hosted-vulkan.yml
# .github/workflows/ci-self-hosted-webgpu.yml
# .github/workflows/close-issue.yml
# .github/workflows/code-style.yml
# .github/workflows/docker.yml
# .github/workflows/editorconfig.yml
# .github/workflows/gguf-publish.yml
# .github/workflows/hip-quality-check.yml
# .github/workflows/labeler.yml
# .github/workflows/make-release.yml
# .github/workflows/models-check.yml
# .github/workflows/pr-draft-label.yml
# .github/workflows/pre-tokenizer-hashes.yml
# .github/workflows/python-check-requirements.yml
# .github/workflows/python-lint.yml
# .github/workflows/python-type-check.yml
# .github/workflows/release.yml
# .github/workflows/server-sanitize.yml
# .github/workflows/server-self-hosted.yml
# .github/workflows/server.yml
# .github/workflows/ui-build-self-hosted.yml
# .github/workflows/ui-build.yml
# .github/workflows/ui-publish.yml
# .github/workflows/ui-self-hosted.yml
# .github/workflows/ui.yml
# .github/workflows/update-ops-docs.yml
# .github/workflows/winget.yml
# AGENTS.md
# docs/backend/OPENVINO.md
# ggml/src/ggml-metal/CMakeLists.txt
# ggml/src/ggml-openvino/CMakeLists.txt
# ggml/src/ggml-openvino/ggml-decoder.cpp
# ggml/src/ggml-openvino/ggml-decoder.h
# ggml/src/ggml-openvino/ggml-openvino-extra.cpp
# ggml/src/ggml-openvino/ggml-openvino-extra.h
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-openvino/model-cache.cpp
# ggml/src/ggml-openvino/model-cache.h
# ggml/src/ggml-openvino/openvino/node_context.h
# ggml/src/ggml-openvino/openvino/op/add.cpp
# ggml/src/ggml-openvino/openvino/op/argsort.cpp
# ggml/src/ggml-openvino/openvino/op/cpy.cpp
# ggml/src/ggml-openvino/openvino/op/flash_attn_ext.cpp
# ggml/src/ggml-openvino/openvino/op/gated_delta_net.cpp
# ggml/src/ggml-openvino/openvino/op/get_rows.cpp
# ggml/src/ggml-openvino/openvino/op/im2col.cpp
# ggml/src/ggml-openvino/openvino/op/l2_norm.cpp
# ggml/src/ggml-openvino/openvino/op/mul_mat_id.cpp
# ggml/src/ggml-openvino/openvino/op/reshape.cpp
# ggml/src/ggml-openvino/openvino/op/rms_norm.cpp
# ggml/src/ggml-openvino/openvino/op/rope.cpp
# ggml/src/ggml-openvino/openvino/op/scale.cpp
# ggml/src/ggml-openvino/openvino/op/view.cpp
# ggml/src/ggml-openvino/openvino/op_table.cpp
# ggml/src/ggml-openvino/openvino/op_table.h
# ggml/src/ggml-openvino/openvino/pass/fuse_moe_compressed.cpp
# ggml/src/ggml-openvino/openvino/pass/fuse_moe_compressed.h
# ggml/src/ggml-openvino/openvino/pass/fuse_to_conv.cpp
# ggml/src/ggml-openvino/openvino/translate_session.cpp
# ggml/src/ggml-openvino/openvino/utils.cpp
# ggml/src/ggml-openvino/openvino/utils.h
# ggml/src/ggml-openvino/utils.cpp
# ggml/src/ggml-openvino/utils.h
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_q_acc.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/quantize_q8.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/set_rows.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/set_rows_quant.wgsl
# scripts/sync_vendor.py
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-batch-alloc.cpp
# tests/test-chat.cpp
# tests/test-llama-archs.cpp
# tools/imatrix/README.md
# tools/imatrix/imatrix.cpp
# tools/server/README.md
2026-10-07 23:18:17 +08:00
Adrien Gallouët
4d60b4d087
common, server : report model input/output modalities in GET /models ( #29987 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-10-05 17:08:29 +02:00
Ed Addario
0504396140
imatrix: calculate activation-based statistics for new format (GGUF) imatrices ( #14891 )
...
* Use activations to calculate the stats
* Determine calculation mode
* Compute entropy for activations
* Compute cosine similarity based on activations
* Compute l2 norm
* Add compute_layer_statistics() function
* Update aggregated statistic report layout
* Fix printing l2 norm when calc_mode = 1
* Refactor variable name
* Compute aggregated (per layer) l2 norm
* Update aggregated sum of squared activations per layer
* Make ZD Score two-tailed
* Update report layout
* Reverse conditional logic to match convention
* Rename report heading
* Add --activation-statistics parameter
* Add Euclidean–Cosine Score (ECS)
* Add --activation-statistics logic to avoid doubling the imatrix size by default
* Update stats output sort based on imatrix type
* Process external NextN draft files (-md / --model-draft)
* Refactor to use new llama_batch_ext
Co-authored-by: compilade <git@compilade.net>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-10-04 11:24:53 +02:00
Concedo
4d27749c93
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-cuda-ubuntu.yml
# .github/workflows/build-cuda-windows.yml
# .github/workflows/build-ibm.yml
# .github/workflows/ci-self-hosted-cuda.yml
# .github/workflows/release.yml
# .github/workflows/server-self-hosted.yml
# AGENTS.md
# README.md
# ci/run.sh
# docs/autoparser.md
# docs/backend/SYCL.md
# docs/build.md
# docs/development/HOWTO-add-model.md
# docs/function-calling.md
# docs/ops.md
# docs/ops/CPU.csv
# ggml/CMakeLists.txt
# ggml/src/ggml-blas/CMakeLists.txt
# ggml/src/ggml-cann/ggml-cann.cpp
# ggml/src/ggml-cpu/amx/amx.cpp
# ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
# ggml/src/ggml-cuda/CMakeLists.txt
# ggml/src/ggml-et/ggml-et.cpp
# ggml/src/ggml-hexagon/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/allreduce-ops.c
# ggml/src/ggml-hexagon/htp/allreduce-ops.h
# ggml/src/ggml-hexagon/htp/concat-ops.c
# ggml/src/ggml-hexagon/htp/cpy-ops.c
# ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/matmul-ops.h
# ggml/src/ggml-hexagon/htp/work-queue.c
# ggml/src/ggml-metal/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/dmmv.cpp
# ggml/src/ggml-sycl/fattn-common.hpp
# ggml/src/ggml-sycl/fattn-onednn.cpp
# ggml/src/ggml-sycl/fattn-vec.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mmvq.cpp
# ggml/src/ggml-sycl/vecdotq.hpp
# ggml/src/ggml-virtgpu/ggml-backend-buffer-type.cpp
# ggml/src/ggml-vulkan/ggml-vulkan.cpp
# ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
# ggml/src/ggml-webgpu/wgsl-shaders/ssm_scan.wgsl
# ggml/src/ggml-zdnn/ggml-zdnn.cpp
# pyproject.toml
# scripts/snapdragon/run.py
# skills/code-review/SKILL.md
# tests/fusion/MTL.csv
# tests/test-alloc.cpp
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-llama-archs.cpp
# tests/test-model-resolution.cpp
# tests/test-quantize-fns.cpp
# tests/test-recurrent-state-rollback.cpp
# tools/llama-bench/README.md
# tools/llama-bench/llama-bench.cpp
# tools/server/CMakeLists.txt
# tools/server/README.md
2026-10-03 17:59:14 +08:00
Concedo
2374751a07
Merge commit ' 19e28a2770' into concedo_experimental
...
# Conflicts:
# .devops/intel.Dockerfile
# .devops/musa.Dockerfile
# .github/workflows/build-cuda-ubuntu.yml
# .github/workflows/build-ibm.yml
# .github/workflows/build-sycl.yml
# .github/workflows/release.yml
# ci/README-MUSA.md
# ci/run.sh
# docs/backend/SYCL.md
# examples/simple-chat/simple-chat.cpp
# examples/simple/simple.cpp
# examples/speculative-simple/speculative-simple.cpp
# ggml/include/ggml-zdnn.h
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/act-ops.c
# ggml/src/ggml-hexagon/htp/concat-ops.c
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-arith.h
# ggml/src/ggml-hexagon/htp/hvx-base.h
# ggml/src/ggml-hexagon/htp/hvx-sigmoid.h
# ggml/src/ggml-hexagon/htp/hvx-utils.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.h
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-openvino/ggml-openvino.cpp
# ggml/src/ggml-zdnn/ggml-zdnn.cpp
# scripts/hip/gcn-cdna-vgpr-check.py
# scripts/snapdragon/ggml-hexagon-trace.py
# src/llama-context.cpp
# tests/gguf-model-data.cpp
# tests/test-chat.cpp
# tests/test-fusion.cpp
# tests/test-gguf.cpp
# tools/cli/cli.cpp
# tools/server/README.md
# vendor/stb/CMakeLists.txt
2026-10-03 00:31:12 +08:00
Pranesh Gonegandla
1fb7ef3e33
spec : add probabilistic sampling for simple draft and MTP ( #27694 )
...
* Make the drafter probabilistic and the target verify by rejection sampling
* Drop stale spec_draft_q before drafting
* Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling. Default flag value is greedy.
* Support grammar-constrained requests in rejection sampling
* Fix - renormalize distribution after masking
* copy rng on sampler copy and re-accept drafted tokens on replay
* Fix draft sampler sharing the target's rng stream
* Simplify the rejection sampler's inputs and move replay to the server
* Truncate the draft candidates along with the draft
---------
Co-authored-by: praneshgo <227579474+praneshgo@users.noreply.github.com>
Co-authored-by: Pranesh Gonegandla <pgonegandla@nvidia.com>
2026-10-02 17:52:54 +03:00
pr3pony
feb9a3d6de
args: fix cli download mmproj arg ( #28977 )
...
* tests: add tests for cli download arg parsing
* args: fix cli download mmproj arg
2026-09-30 20:45:39 +02:00
Concedo
3cee526a68
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .devops/musa.Dockerfile
# .github/workflows/build-apple.yml
# .github/workflows/build-cpu.yml
# .github/workflows/build-vulkan.yml
# .github/workflows/python-type-check.yml
# CODEOWNERS
# docs/backend/snapdragon/CMakeUserPresets.json
# docs/build.md
# docs/docker.md
# docs/ops.md
# docs/ops/Hexagon.csv
# ggml/cmake/common.cmake
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-cpu/ggml-cpu.c
# ggml/src/ggml-cuda/CMakeLists.txt
# ggml/src/ggml-hexagon/CMakeLists.txt
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-opnode.h
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# ggml/src/ggml-hexagon/htp/binary-ops.c
# ggml/src/ggml-hexagon/htp/binary-ops.h
# ggml/src/ggml-hexagon/htp/cpy-ops.c
# ggml/src/ggml-hexagon/htp/dma-queue.h
# ggml/src/ggml-hexagon/htp/get-rows-ops.c
# ggml/src/ggml-hexagon/htp/get-rows-ops.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/hvx-arith.h
# ggml/src/ggml-hexagon/htp/hvx-base.h
# ggml/src/ggml-hexagon/htp/hvx-reduce.h
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/sum-rows-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.h
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-sycl/fwht.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# scripts/hip/gcn-cdna-vgpr-check.py
# scripts/snapdragon/ggml-hexagon-inspect.py
# tests/CMakeLists.txt
# tests/test-backend-ops.cpp
# tests/test-jinja.cpp
# tests/test-recurrent-state-rollback.cpp
# tests/test-save-load-state.cpp
# tests/test-state-restore-fragmented.cpp
# tools/llama-bench/README.md
# tools/llama-bench/llama-bench.cpp
# tools/mtmd/CMakeLists.txt
2026-09-29 23:24:34 +08:00
Adrien Gallouët
00af63567a
common : use fs::path for config dir ( #29649 )
...
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-09-29 17:22:49 +02:00
Adrien Gallouët
76a5bc86d1
common : use fs::path for cache dirs ( #29595 )
...
- Avoid useless string conversions on Windows.
- No need for BSD or emscripten special cases.
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-09-29 07:24:03 +02:00
Georgi Gerganov
a97cce86a8
common : avoid side effects around params parsing ( #29537 )
...
- register --rpc unconditionally and call llama_supports_rpc() only from its handler
- print server "initialization ..." log after args are parsed
Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL
2026-09-27 20:18:56 +03:00
Concedo
084d797f2d
Merge commit ' 4ceb171910' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-sycl.yml
# .github/workflows/docker.yml
# .github/workflows/release.yml
# cmake/llama-config.cmake.in
# docs/backend/snapdragon/README.md
# docs/backend/snapdragon/developer.md
# examples/simple-cmake-pkg/CMakeLists.txt
# ggml/include/ggml-sycl.h
# ggml/src/ggml-cpu/repack.cpp
# ggml/src/ggml-cpu/repack.h
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp-opnode.h
# ggml/src/ggml-hexagon/htp/act-ops.c
# ggml/src/ggml-hexagon/htp/allreduce-ops.c
# ggml/src/ggml-hexagon/htp/argsort-ops.c
# ggml/src/ggml-hexagon/htp/binary-ops.c
# ggml/src/ggml-hexagon/htp/concat-ops.c
# ggml/src/ggml-hexagon/htp/cpy-ops.c
# ggml/src/ggml-hexagon/htp/cumsum-ops.c
# ggml/src/ggml-hexagon/htp/diag-ops.c
# ggml/src/ggml-hexagon/htp/dma-queue.c
# ggml/src/ggml-hexagon/htp/dma-queue.h
# ggml/src/ggml-hexagon/htp/fill-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.c
# ggml/src/ggml-hexagon/htp/flash-attn-ops.h
# ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
# ggml/src/ggml-hexagon/htp/get-rows-ops.c
# ggml/src/ggml-hexagon/htp/hmx-fa-kernels.h
# ggml/src/ggml-hexagon/htp/htp-ctx.h
# ggml/src/ggml-hexagon/htp/htp-ops.h
# ggml/src/ggml-hexagon/htp/htp-tensor.c
# ggml/src/ggml-hexagon/htp/htp-tensor.h
# ggml/src/ggml-hexagon/htp/htp_iface.idl
# ggml/src/ggml-hexagon/htp/hvx-exp.h
# ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
# ggml/src/ggml-hexagon/htp/im2col-ops.c
# ggml/src/ggml-hexagon/htp/main.c
# ggml/src/ggml-hexagon/htp/matmul-ops.c
# ggml/src/ggml-hexagon/htp/matmul-ops.h
# ggml/src/ggml-hexagon/htp/pad-ops.c
# ggml/src/ggml-hexagon/htp/repeat-ops.c
# ggml/src/ggml-hexagon/htp/roll-ops.c
# ggml/src/ggml-hexagon/htp/rope-ops.c
# ggml/src/ggml-hexagon/htp/rope-ops.h
# ggml/src/ggml-hexagon/htp/set-rows-ops.c
# ggml/src/ggml-hexagon/htp/softmax-ops.c
# ggml/src/ggml-hexagon/htp/solve-tri-ops.c
# ggml/src/ggml-hexagon/htp/ssm-conv.c
# ggml/src/ggml-hexagon/htp/sum-rows-ops.c
# ggml/src/ggml-hexagon/htp/unary-ops.c
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-sycl/dsv4-hc.cpp
# ggml/src/ggml-sycl/fattn-mkl.cpp
# ggml/src/ggml-sycl/fattn-tile.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-webgpu/ggml-webgpu.cpp
# scripts/snapdragon/ggml-hexagon-profile.py
# scripts/snapdragon/ggml-hexagon-trace.py
# scripts/snapdragon/run.py
# scripts/sync_vendor.py
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-llama-archs.cpp
# tests/test-recurrent-state-rollback.cpp
# tests/test-save-load-state.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/server/README.md
2026-09-24 16:37:57 +08:00
Emanuil Rusev
217f81c266
server: Add support for binding to multiple addresses ( #28690 )
...
* Add support for binding llama-server to multiple addresses
Assisted-by: Codex
* remove redundant thread handler
* make it clear about overlapping addr
* reject --port 0 with multiple tcp addr
* improve arg handler
* nits
* fix test
* nits 2
* nits
* nits 2
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2026-09-22 15:16:40 +02:00
Mikolaj Kucharski
e0dff58475
args: add env vars for temperature, top-p, min-p and penalties ( #27380 )
...
Allow configuring --temp, --top-p, --min-p, --repeat-penalty,
--presence-penalty and --frequency-penalty via LLAMA_ARG_* so
llama-server can be fully controlled from an EnvironmentFile
(e.g. systemd on Debian).
Use `llama-gen-docs` to regenerate the readme files.
2026-09-21 12:47:38 +02:00
Concedo
c43b36bcb5
Merge branch 'upstream' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-ibm.yml
# .github/workflows/build-self-hosted.yml
# .github/workflows/code-style.yml
# .github/workflows/editorconfig.yml
# .github/workflows/fusion.yml
# .github/workflows/release.yml
# .github/workflows/server-sanitize.yml
# .pi/gg/SYSTEM.md
# CMakeLists.txt
# ci/run.sh
# common/CMakeLists.txt
# docs/backend/SYCL.md
# ggml/CMakeLists.txt
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-opencl/ggml-opencl.cpp
# ggml/src/ggml-sycl/CMakeLists.txt
# ggml/src/ggml-sycl/backend.hpp
# ggml/src/ggml-sycl/base.hpp
# ggml/src/ggml-sycl/common.hpp
# ggml/src/ggml-sycl/gemm.hpp
# ggml/src/ggml-sycl/ggml-sycl.cpp
# ggml/src/ggml-sycl/mem.cpp
# scripts/sync-ggml.last
# src/CMakeLists.txt
# tests/CMakeLists.txt
# tests/snapshots/nvidia-nemotron-3-nano-30b-a3b.schema
# tests/test-backend-ops.cpp
# tests/test-chat.cpp
# tests/test-llama-archs.cpp
# tests/test-quant-type-selection.cpp
# tools/mtmd/CMakeLists.txt
# tools/server/CMakeLists.txt
2026-09-14 22:44:04 +08:00
Concedo
a4f7e70b9f
Merge commit ' acecd56032' into concedo_experimental
...
# Conflicts:
# .github/workflows/build-cpu.yml
# common/CMakeLists.txt
# common/chat-auto-parser-generator.cpp
# docs/development/parsing.md
# grammars/README.md
# json_to_gbnf.py
# tests/CMakeLists.txt
# tests/test-chat-peg-parser.cpp
# tests/test-chat.cpp
# tests/test-grammar-integration.cpp
# tests/test-json-schema-to-grammar.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/server/README.md
2026-09-14 22:35:27 +08:00
Concedo
483a456fd0
Merge commit ' 52d4268656' into concedo_experimental
...
# Conflicts:
# .github/actions/windows-setup-cuda/action.yml
# .github/workflows/build-riscv.yml
# .github/workflows/build-sanitize.yml
# .github/workflows/build-self-hosted.yml
# .github/workflows/release.yml
# .github/workflows/server-sanitize.yml
# docs/build-s390x.md
# docs/build.md
# ggml/CMakeLists.txt
# ggml/cmake/common.cmake
# ggml/src/ggml-cpu/CMakeLists.txt
# ggml/src/ggml-cpu/repack.cpp
# ggml/src/ggml-cuda/CMakeLists.txt
# ggml/src/ggml-cuda/fattn.cu
# ggml/src/ggml-hexagon/ggml-hexagon.cpp
# ggml/src/ggml-hexagon/htp/hvx-sin-cos.h
# ggml/src/ggml-hexagon/htp/rope-ops.c
# ggml/src/ggml-hip/CMakeLists.txt
# ggml/src/ggml-musa/CMakeLists.txt
# ggml/src/ggml-vulkan/ggml-vulkan.cpp
# pyproject.toml
# requirements/requirements-convert_legacy_llama.txt
# requirements/requirements-gguf_editor_gui.txt
# requirements/requirements-server-bench.txt
# requirements/requirements-tool_bench.txt
# scripts/make-release-checks.sh
# tests/test-backend-ops.cpp
# tests/test-jinja.cpp
# tools/cli/README.md
# tools/completion/README.md
# tools/llama-bench/llama-bench.cpp
# tools/server/README.md
2026-09-13 23:12:41 +08:00
Xuan-Son Nguyen
8e330954ad
common: add LOG_JSON macro to log structured data ( #28586 )
...
* add LOG_JSON macro
* fit: add demo LOG_JSON
2026-09-13 01:36:34 +02:00
Aldehir Rojas
acecd56032
common : implement common_schema internal representation for JSON schemas ( #28736 )
...
* common : implement common_schema types
* common : implement a json schema optimizer
* common : reduce optimizations
* common : refactor json-schema-to-grammar to use common_schema
* common : use common_trie
* common/schema : implement type/kind resolution
* cont : cleanup
* cont : remove common_chat_tool_parameters
* cont : simplify schema resolution
* cont : pass common_schema through the json-schema-to-grammar builder
* cont : cleanup
* cont : move enums under common_schema and add type enum
* cont : reduce test cases
* cont : clean up
* cont : clean up
* refactor : rename common_schema_parse to common_schema_from_json
* tests : fix gcc dangling-reference warning in test-json-schema
* tests : take the schema label as const char * to satisfy gcc dangling-reference
* refactor : rename common_schema_builder parse_* methods to build_*
* cont : fix may_be_string
* cont : properly handle empty tool parameters
* cont : add tests for empty $ref
* cont : remove dead code
* cont : update docs
* cont : make "{}" mean any object for json_object as well
* cont : restore (min|max)Length to imply string type
* cont : rename common_schema to common_chat_schema
2026-09-12 16:14:50 -05:00
