vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-10-21 22:00:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
16337 commits 3 branches 258 tags 1,017 MiB
Commit graph

603 commits

Author SHA1 Message Date
Concedo
0890a5ee77 Merge commit '033df86b69' into concedo_experimental 
# Conflicts:
#	docs/backend/SYCL.md
#	docs/development/parsing.md
#	examples/speculative-simple/speculative-simple.cpp
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/get-rows-ops.c
#	ggml/src/ggml-hexagon/htp/get-rows-ops.h
#	ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-erf.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.h
#	ggml/src/ggml-sycl/CMakeLists.txt
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/element_wise.cpp
#	ggml/src/ggml-sycl/element_wise.hpp
#	ggml/src/ggml-sycl/fusion.cpp
#	ggml/src/ggml-sycl/fwht.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/im2col.hpp
#	ggml/src/ggml-sycl/pad.hpp
#	ggml/src/ggml-sycl/rope.hpp
#	ggml/src/ggml-sycl/upscale.hpp
#	scripts/snapdragon/run.py
#	scripts/sync_vendor.py
#	src/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-backend-sampler.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tools/imatrix/imatrix.cpp
#	tools/server/README.md
 2026-10-10 13:19:02 +08:00
Concedo
14cf69fa7a Merge commit '63bef2728d' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-apple.yml
#	.github/workflows/docker.yml
#	.github/workflows/make-release.yml
#	.github/workflows/models-check.yml
#	.github/workflows/winget.yml
#	CMakeLists.txt
#	docs/backend/OPENVINO.md
#	docs/build-riscv64-spacemit.md
#	ggml/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/dma-queue.c
#	ggml/src/ggml-hexagon/htp/dma-queue.h
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.h
#	ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/hmx-utils.h
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.h
#	ggml/src/ggml-hexagon/htp/ssm-conv.c
#	ggml/src/ggml-hexagon/htp/ssm-conv.h
#	ggml/src/ggml-openvino/ggml-decoder.cpp
#	ggml/src/ggml-openvino/ggml-decoder.h
#	ggml/src/ggml-openvino/ggml-openvino-extra.cpp
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-openvino/openvino/op/fill.cpp
#	ggml/src/ggml-openvino/openvino/op_table.cpp
#	ggml/src/ggml-openvino/openvino/pass/fuse_moe_compressed.cpp
#	ggml/src/ggml-openvino/openvino/translate_session.cpp
#	ggml/src/ggml-openvino/utils.cpp
#	ggml/src/ggml-openvino/utils.h
#	scripts/make-release-checks.sh
#	scripts/make-release-summary.txt
#	scripts/snapdragon/run.py
#	scripts/sync-ggml.last
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tests/test-save-load-state.cpp
#	tests/test-state-restore-fragmented.cpp
#	tools/server/README.md
 2026-10-08 23:43:45 +08:00
Aman Gupta
d6cf9acb25
llama : add a GPU cache for MoE experts kept in host memory (#29887) 
* llama : add a GPU cache for MoE experts kept in host memory

Assisted-by: Claude

* use llama_moe_cache_ptr
 2026-10-07 21:07:45 +03:00
Concedo
acef3c226f Merge commit 'ebe18bee5a' into concedo_experimental 
# Conflicts:
#	.devops/openvino.Dockerfile
#	.github/workflows/ai-issues.yml
#	.github/workflows/build-3rd-party.yml
#	.github/workflows/build-and-test-snapdragon.yml
#	.github/workflows/build-android.yml
#	.github/workflows/build-apple.yml
#	.github/workflows/build-cache.yml
#	.github/workflows/build-cann.yml
#	.github/workflows/build-cmake-pkg.yml
#	.github/workflows/build-cpu.yml
#	.github/workflows/build-cross.yml
#	.github/workflows/build-cuda-ubuntu.yml
#	.github/workflows/build-cuda-windows.yml
#	.github/workflows/build-ibm.yml
#	.github/workflows/build-msys.yml
#	.github/workflows/build-opencl.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-riscv.yml
#	.github/workflows/build-sanitize.yml
#	.github/workflows/build-sycl.yml
#	.github/workflows/build-virtgpu.yml
#	.github/workflows/build-vulkan.yml
#	.github/workflows/build-wasm.yml
#	.github/workflows/build-webgpu.yml
#	.github/workflows/check-vendor.yml
#	.github/workflows/ci-self-hosted-cpu.yml
#	.github/workflows/ci-self-hosted-cuda.yml
#	.github/workflows/ci-self-hosted-kleidiai.yml
#	.github/workflows/ci-self-hosted-metal.yml
#	.github/workflows/ci-self-hosted-openvino.yml
#	.github/workflows/ci-self-hosted-vulkan.yml
#	.github/workflows/ci-self-hosted-webgpu.yml
#	.github/workflows/close-issue.yml
#	.github/workflows/code-style.yml
#	.github/workflows/docker.yml
#	.github/workflows/editorconfig.yml
#	.github/workflows/gguf-publish.yml
#	.github/workflows/hip-quality-check.yml
#	.github/workflows/labeler.yml
#	.github/workflows/make-release.yml
#	.github/workflows/models-check.yml
#	.github/workflows/pr-draft-label.yml
#	.github/workflows/pre-tokenizer-hashes.yml
#	.github/workflows/python-check-requirements.yml
#	.github/workflows/python-lint.yml
#	.github/workflows/python-type-check.yml
#	.github/workflows/release.yml
#	.github/workflows/server-sanitize.yml
#	.github/workflows/server-self-hosted.yml
#	.github/workflows/server.yml
#	.github/workflows/ui-build-self-hosted.yml
#	.github/workflows/ui-build.yml
#	.github/workflows/ui-publish.yml
#	.github/workflows/ui-self-hosted.yml
#	.github/workflows/ui.yml
#	.github/workflows/update-ops-docs.yml
#	.github/workflows/winget.yml
#	AGENTS.md
#	docs/backend/OPENVINO.md
#	ggml/src/ggml-metal/CMakeLists.txt
#	ggml/src/ggml-openvino/CMakeLists.txt
#	ggml/src/ggml-openvino/ggml-decoder.cpp
#	ggml/src/ggml-openvino/ggml-decoder.h
#	ggml/src/ggml-openvino/ggml-openvino-extra.cpp
#	ggml/src/ggml-openvino/ggml-openvino-extra.h
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-openvino/model-cache.cpp
#	ggml/src/ggml-openvino/model-cache.h
#	ggml/src/ggml-openvino/openvino/node_context.h
#	ggml/src/ggml-openvino/openvino/op/add.cpp
#	ggml/src/ggml-openvino/openvino/op/argsort.cpp
#	ggml/src/ggml-openvino/openvino/op/cpy.cpp
#	ggml/src/ggml-openvino/openvino/op/flash_attn_ext.cpp
#	ggml/src/ggml-openvino/openvino/op/gated_delta_net.cpp
#	ggml/src/ggml-openvino/openvino/op/get_rows.cpp
#	ggml/src/ggml-openvino/openvino/op/im2col.cpp
#	ggml/src/ggml-openvino/openvino/op/l2_norm.cpp
#	ggml/src/ggml-openvino/openvino/op/mul_mat_id.cpp
#	ggml/src/ggml-openvino/openvino/op/reshape.cpp
#	ggml/src/ggml-openvino/openvino/op/rms_norm.cpp
#	ggml/src/ggml-openvino/openvino/op/rope.cpp
#	ggml/src/ggml-openvino/openvino/op/scale.cpp
#	ggml/src/ggml-openvino/openvino/op/view.cpp
#	ggml/src/ggml-openvino/openvino/op_table.cpp
#	ggml/src/ggml-openvino/openvino/op_table.h
#	ggml/src/ggml-openvino/openvino/pass/fuse_moe_compressed.cpp
#	ggml/src/ggml-openvino/openvino/pass/fuse_moe_compressed.h
#	ggml/src/ggml-openvino/openvino/pass/fuse_to_conv.cpp
#	ggml/src/ggml-openvino/openvino/translate_session.cpp
#	ggml/src/ggml-openvino/openvino/utils.cpp
#	ggml/src/ggml-openvino/openvino/utils.h
#	ggml/src/ggml-openvino/utils.cpp
#	ggml/src/ggml-openvino/utils.h
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_q_acc.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/quantize_q8.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/set_rows.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/set_rows_quant.wgsl
#	scripts/sync_vendor.py
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-batch-alloc.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tools/imatrix/README.md
#	tools/imatrix/imatrix.cpp
#	tools/server/README.md
 2026-10-07 23:18:17 +08:00
Adrien Gallouët
4d60b4d087
common, server : report model input/output modalities in GET /models (#29987) 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-10-05 17:08:29 +02:00
Ed Addario
0504396140
imatrix: calculate activation-based statistics for new format (GGUF) imatrices (#14891) 
* Use activations to calculate the stats
* Determine calculation mode
* Compute entropy for activations
* Compute cosine similarity based on activations
* Compute l2 norm
* Add compute_layer_statistics() function
* Update aggregated statistic report layout
* Fix printing l2 norm when calc_mode = 1
* Refactor variable name
* Compute aggregated (per layer) l2 norm
* Update aggregated sum of squared activations per layer
* Make ZD Score two-tailed
* Update report layout
* Reverse conditional logic to match convention
* Rename report heading
* Add --activation-statistics parameter
* Add Euclidean–Cosine Score (ECS)
* Add --activation-statistics logic to avoid doubling the imatrix size by default
* Update stats output sort based on imatrix type
* Process external NextN draft files (-md / --model-draft)
* Refactor to use new llama_batch_ext

Co-authored-by: compilade <git@compilade.net>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
 2026-10-04 11:24:53 +02:00
Concedo
4d27749c93 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-cuda-ubuntu.yml
#	.github/workflows/build-cuda-windows.yml
#	.github/workflows/build-ibm.yml
#	.github/workflows/ci-self-hosted-cuda.yml
#	.github/workflows/release.yml
#	.github/workflows/server-self-hosted.yml
#	AGENTS.md
#	README.md
#	ci/run.sh
#	docs/autoparser.md
#	docs/backend/SYCL.md
#	docs/build.md
#	docs/development/HOWTO-add-model.md
#	docs/function-calling.md
#	docs/ops.md
#	docs/ops/CPU.csv
#	ggml/CMakeLists.txt
#	ggml/src/ggml-blas/CMakeLists.txt
#	ggml/src/ggml-cann/ggml-cann.cpp
#	ggml/src/ggml-cpu/amx/amx.cpp
#	ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
#	ggml/src/ggml-cuda/CMakeLists.txt
#	ggml/src/ggml-et/ggml-et.cpp
#	ggml/src/ggml-hexagon/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/allreduce-ops.c
#	ggml/src/ggml-hexagon/htp/allreduce-ops.h
#	ggml/src/ggml-hexagon/htp/concat-ops.c
#	ggml/src/ggml-hexagon/htp/cpy-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.h
#	ggml/src/ggml-hexagon/htp/work-queue.c
#	ggml/src/ggml-metal/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/fattn-common.hpp
#	ggml/src/ggml-sycl/fattn-onednn.cpp
#	ggml/src/ggml-sycl/fattn-vec.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	ggml/src/ggml-virtgpu/ggml-backend-buffer-type.cpp
#	ggml/src/ggml-vulkan/ggml-vulkan.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/ssm_scan.wgsl
#	ggml/src/ggml-zdnn/ggml-zdnn.cpp
#	pyproject.toml
#	scripts/snapdragon/run.py
#	skills/code-review/SKILL.md
#	tests/fusion/MTL.csv
#	tests/test-alloc.cpp
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tests/test-model-resolution.cpp
#	tests/test-quantize-fns.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tools/llama-bench/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/server/CMakeLists.txt
#	tools/server/README.md
 2026-10-03 17:59:14 +08:00
Concedo
2374751a07 Merge commit '19e28a2770' into concedo_experimental 
# Conflicts:
#	.devops/intel.Dockerfile
#	.devops/musa.Dockerfile
#	.github/workflows/build-cuda-ubuntu.yml
#	.github/workflows/build-ibm.yml
#	.github/workflows/build-sycl.yml
#	.github/workflows/release.yml
#	ci/README-MUSA.md
#	ci/run.sh
#	docs/backend/SYCL.md
#	examples/simple-chat/simple-chat.cpp
#	examples/simple/simple.cpp
#	examples/speculative-simple/speculative-simple.cpp
#	ggml/include/ggml-zdnn.h
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/act-ops.c
#	ggml/src/ggml-hexagon/htp/concat-ops.c
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-arith.h
#	ggml/src/ggml-hexagon/htp/hvx-base.h
#	ggml/src/ggml-hexagon/htp/hvx-sigmoid.h
#	ggml/src/ggml-hexagon/htp/hvx-utils.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.h
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-zdnn/ggml-zdnn.cpp
#	scripts/hip/gcn-cdna-vgpr-check.py
#	scripts/snapdragon/ggml-hexagon-trace.py
#	src/llama-context.cpp
#	tests/gguf-model-data.cpp
#	tests/test-chat.cpp
#	tests/test-fusion.cpp
#	tests/test-gguf.cpp
#	tools/cli/cli.cpp
#	tools/server/README.md
#	vendor/stb/CMakeLists.txt
 2026-10-03 00:31:12 +08:00
Pranesh Gonegandla
1fb7ef3e33
spec : add probabilistic sampling for simple draft and MTP (#27694) 
* Make the drafter probabilistic and the target verify by rejection sampling

* Drop stale spec_draft_q before drafting

* Fallback to argmax sampling for grammar-constrained requests and adding flag for enabling probabilistic draft sampling. Default flag value is greedy.

* Support grammar-constrained requests in rejection sampling

* Fix - renormalize distribution after masking

* copy rng on sampler copy and re-accept drafted tokens on replay

* Fix draft sampler sharing the target's rng stream

* Simplify the rejection sampler's inputs and move replay to the server

* Truncate the draft candidates along with the draft

---------

Co-authored-by: praneshgo <227579474+praneshgo@users.noreply.github.com>
Co-authored-by: Pranesh Gonegandla <pgonegandla@nvidia.com>
 2026-10-02 17:52:54 +03:00
pr3pony
feb9a3d6de
args: fix cli download mmproj arg (#28977) 
* tests: add tests for cli download arg parsing

* args: fix cli download mmproj arg
 2026-09-30 20:45:39 +02:00
Concedo
3cee526a68 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/musa.Dockerfile
#	.github/workflows/build-apple.yml
#	.github/workflows/build-cpu.yml
#	.github/workflows/build-vulkan.yml
#	.github/workflows/python-type-check.yml
#	CODEOWNERS
#	docs/backend/snapdragon/CMakeUserPresets.json
#	docs/build.md
#	docs/docker.md
#	docs/ops.md
#	docs/ops/Hexagon.csv
#	ggml/cmake/common.cmake
#	ggml/src/ggml-cpu/CMakeLists.txt
#	ggml/src/ggml-cpu/ggml-cpu.c
#	ggml/src/ggml-cuda/CMakeLists.txt
#	ggml/src/ggml-hexagon/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp-opnode.h
#	ggml/src/ggml-hexagon/htp/argsort-ops.c
#	ggml/src/ggml-hexagon/htp/binary-ops.c
#	ggml/src/ggml-hexagon/htp/binary-ops.h
#	ggml/src/ggml-hexagon/htp/cpy-ops.c
#	ggml/src/ggml-hexagon/htp/dma-queue.h
#	ggml/src/ggml-hexagon/htp/get-rows-ops.c
#	ggml/src/ggml-hexagon/htp/get-rows-ops.h
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-arith.h
#	ggml/src/ggml-hexagon/htp/hvx-base.h
#	ggml/src/ggml-hexagon/htp/hvx-reduce.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/sum-rows-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.h
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-sycl/fwht.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	scripts/hip/gcn-cdna-vgpr-check.py
#	scripts/snapdragon/ggml-hexagon-inspect.py
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-jinja.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tests/test-save-load-state.cpp
#	tests/test-state-restore-fragmented.cpp
#	tools/llama-bench/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/mtmd/CMakeLists.txt
 2026-09-29 23:24:34 +08:00
Adrien Gallouët
00af63567a
common : use fs::path for config dir (#29649) 
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-09-29 17:22:49 +02:00
Adrien Gallouët
76a5bc86d1
common : use fs::path for cache dirs (#29595) 
- Avoid useless string conversions on Windows.
- No need for BSD or emscripten special cases.

Signed-off-by: Adrien Gallouët <angt@huggingface.co>
 2026-09-29 07:24:03 +02:00
Georgi Gerganov
a97cce86a8
common : avoid side effects around params parsing (#29537) 
- register --rpc unconditionally and call llama_supports_rpc() only from its handler
- print server "initialization ..." log after args are parsed

Assisted-by: pi:llama.cpp/MiMo-V2.6-Flash-RL
 2026-09-27 20:18:56 +03:00
Concedo
084d797f2d Merge commit '4ceb171910' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-sycl.yml
#	.github/workflows/docker.yml
#	.github/workflows/release.yml
#	cmake/llama-config.cmake.in
#	docs/backend/snapdragon/README.md
#	docs/backend/snapdragon/developer.md
#	examples/simple-cmake-pkg/CMakeLists.txt
#	ggml/include/ggml-sycl.h
#	ggml/src/ggml-cpu/repack.cpp
#	ggml/src/ggml-cpu/repack.h
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp-opnode.h
#	ggml/src/ggml-hexagon/htp/act-ops.c
#	ggml/src/ggml-hexagon/htp/allreduce-ops.c
#	ggml/src/ggml-hexagon/htp/argsort-ops.c
#	ggml/src/ggml-hexagon/htp/binary-ops.c
#	ggml/src/ggml-hexagon/htp/concat-ops.c
#	ggml/src/ggml-hexagon/htp/cpy-ops.c
#	ggml/src/ggml-hexagon/htp/cumsum-ops.c
#	ggml/src/ggml-hexagon/htp/diag-ops.c
#	ggml/src/ggml-hexagon/htp/dma-queue.c
#	ggml/src/ggml-hexagon/htp/dma-queue.h
#	ggml/src/ggml-hexagon/htp/fill-ops.c
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.h
#	ggml/src/ggml-hexagon/htp/gated-delta-net-ops.c
#	ggml/src/ggml-hexagon/htp/get-rows-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-fa-kernels.h
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/htp-tensor.c
#	ggml/src/ggml-hexagon/htp/htp-tensor.h
#	ggml/src/ggml-hexagon/htp/htp_iface.idl
#	ggml/src/ggml-hexagon/htp/hvx-exp.h
#	ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/im2col-ops.c
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.h
#	ggml/src/ggml-hexagon/htp/pad-ops.c
#	ggml/src/ggml-hexagon/htp/repeat-ops.c
#	ggml/src/ggml-hexagon/htp/roll-ops.c
#	ggml/src/ggml-hexagon/htp/rope-ops.c
#	ggml/src/ggml-hexagon/htp/rope-ops.h
#	ggml/src/ggml-hexagon/htp/set-rows-ops.c
#	ggml/src/ggml-hexagon/htp/softmax-ops.c
#	ggml/src/ggml-hexagon/htp/solve-tri-ops.c
#	ggml/src/ggml-hexagon/htp/ssm-conv.c
#	ggml/src/ggml-hexagon/htp/sum-rows-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-sycl/dsv4-hc.cpp
#	ggml/src/ggml-sycl/fattn-mkl.cpp
#	ggml/src/ggml-sycl/fattn-tile.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	scripts/snapdragon/ggml-hexagon-profile.py
#	scripts/snapdragon/ggml-hexagon-trace.py
#	scripts/snapdragon/run.py
#	scripts/sync_vendor.py
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tests/test-save-load-state.cpp
#	tools/cli/README.md
#	tools/completion/README.md
#	tools/server/README.md
 2026-09-24 16:37:57 +08:00
Emanuil Rusev
217f81c266
server: Add support for binding to multiple addresses (#28690) 
* Add support for binding llama-server to multiple addresses

Assisted-by: Codex

* remove redundant thread handler

* make it clear about overlapping addr

* reject --port 0 with multiple tcp addr

* improve arg handler

* nits

* fix test

* nits 2

* nits

* nits 2

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2026-09-22 15:16:40 +02:00
Mikolaj Kucharski
e0dff58475
args: add env vars for temperature, top-p, min-p and penalties (#27380) 
Allow configuring --temp, --top-p, --min-p, --repeat-penalty,
--presence-penalty and --frequency-penalty via LLAMA_ARG_* so
llama-server can be fully controlled from an EnvironmentFile
(e.g. systemd on Debian).

Use `llama-gen-docs` to regenerate the readme files.
 2026-09-21 12:47:38 +02:00
Concedo
c43b36bcb5 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-ibm.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/code-style.yml
#	.github/workflows/editorconfig.yml
#	.github/workflows/fusion.yml
#	.github/workflows/release.yml
#	.github/workflows/server-sanitize.yml
#	.pi/gg/SYSTEM.md
#	CMakeLists.txt
#	ci/run.sh
#	common/CMakeLists.txt
#	docs/backend/SYCL.md
#	ggml/CMakeLists.txt
#	ggml/src/ggml-cpu/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-sycl/CMakeLists.txt
#	ggml/src/ggml-sycl/backend.hpp
#	ggml/src/ggml-sycl/base.hpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/gemm.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mem.cpp
#	scripts/sync-ggml.last
#	src/CMakeLists.txt
#	tests/CMakeLists.txt
#	tests/snapshots/nvidia-nemotron-3-nano-30b-a3b.schema
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tests/test-quant-type-selection.cpp
#	tools/mtmd/CMakeLists.txt
#	tools/server/CMakeLists.txt
 2026-09-14 22:44:04 +08:00
Concedo
a4f7e70b9f Merge commit 'acecd56032' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-cpu.yml
#	common/CMakeLists.txt
#	common/chat-auto-parser-generator.cpp
#	docs/development/parsing.md
#	grammars/README.md
#	json_to_gbnf.py
#	tests/CMakeLists.txt
#	tests/test-chat-peg-parser.cpp
#	tests/test-chat.cpp
#	tests/test-grammar-integration.cpp
#	tests/test-json-schema-to-grammar.cpp
#	tools/cli/README.md
#	tools/completion/README.md
#	tools/server/README.md
 2026-09-14 22:35:27 +08:00
Concedo
483a456fd0 Merge commit '52d4268656' into concedo_experimental 
# Conflicts:
#	.github/actions/windows-setup-cuda/action.yml
#	.github/workflows/build-riscv.yml
#	.github/workflows/build-sanitize.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/release.yml
#	.github/workflows/server-sanitize.yml
#	docs/build-s390x.md
#	docs/build.md
#	ggml/CMakeLists.txt
#	ggml/cmake/common.cmake
#	ggml/src/ggml-cpu/CMakeLists.txt
#	ggml/src/ggml-cpu/repack.cpp
#	ggml/src/ggml-cuda/CMakeLists.txt
#	ggml/src/ggml-cuda/fattn.cu
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/hvx-sin-cos.h
#	ggml/src/ggml-hexagon/htp/rope-ops.c
#	ggml/src/ggml-hip/CMakeLists.txt
#	ggml/src/ggml-musa/CMakeLists.txt
#	ggml/src/ggml-vulkan/ggml-vulkan.cpp
#	pyproject.toml
#	requirements/requirements-convert_legacy_llama.txt
#	requirements/requirements-gguf_editor_gui.txt
#	requirements/requirements-server-bench.txt
#	requirements/requirements-tool_bench.txt
#	scripts/make-release-checks.sh
#	tests/test-backend-ops.cpp
#	tests/test-jinja.cpp
#	tools/cli/README.md
#	tools/completion/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/server/README.md
 2026-09-13 23:12:41 +08:00
Xuan-Son Nguyen
8e330954ad
common: add LOG_JSON macro to log structured data (#28586) 
* add LOG_JSON macro

* fit: add demo LOG_JSON
 2026-09-13 01:36:34 +02:00
Aldehir Rojas
acecd56032
common : implement common_schema internal representation for JSON schemas (#28736) 
* common : implement common_schema types

* common : implement a json schema optimizer

* common : reduce optimizations

* common : refactor json-schema-to-grammar to use common_schema

* common : use common_trie

* common/schema : implement type/kind resolution

* cont : cleanup

* cont : remove common_chat_tool_parameters

* cont : simplify schema resolution

* cont : pass common_schema through the json-schema-to-grammar builder

* cont : cleanup

* cont : move enums under common_schema and add type enum

* cont : reduce test cases

* cont : clean up

* cont : clean up

* refactor : rename common_schema_parse to common_schema_from_json

* tests : fix gcc dangling-reference warning in test-json-schema

* tests : take the schema label as const char * to satisfy gcc dangling-reference

* refactor : rename common_schema_builder parse_* methods to build_*

* cont : fix may_be_string

* cont : properly handle empty tool parameters

* cont : add tests for empty $ref

* cont : remove dead code

* cont : update docs

* cont : make "{}" mean any object for json_object as well

* cont : restore (min|max)Length to imply string type

* cont : rename common_schema to common_chat_schema
 2026-09-12 16:14:50 -05:00
Aaron Teo
14a9d09f75
args: officially deprecate --mmap|mlock|dio (#28334) 
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
 2026-09-09 18:36:27 +08:00
Concedo
ad1eee0c5f Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/server-self-hosted.yml
#	.github/workflows/ui-build-self-hosted.yml
#	.github/workflows/ui-build.yml
#	.github/workflows/ui.yml
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	tests/test-backend-ops.cpp
#	tests/test-sampling.cpp
#	tools/cli/README.md
#	tools/server/README.md
 2026-09-09 11:47:23 +08:00
Aman Gupta
415e909d84
spec: single device drafter should create meta backend wrapper (#28390) 2026-09-08 20:44:33 +08:00
Concedo
2d357d8359 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/nix/package.nix
#	.github/ISSUE_TEMPLATE/config.yml
#	.github/workflows/make-release.yml
#	docs/autoparser.md
#	flake.nix
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/norm.cpp
#	ggml/src/ggml-sycl/norm.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	models/templates/README.md
#	scripts/make-release-checks.sh
#	scripts/ui-assets.cmake
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tools/cli/README.md
#	tools/completion/README.md
#	tools/server/CMakeLists.txt
#	tools/server/README.md
 2026-09-08 11:59:02 +08:00
Xuan-Son Nguyen
7620399f58
common: add --log-jsonl (#28437) 
* common: add --log-jsonl

* rename unknown to none
 2026-09-06 08:21:22 +02:00
Concedo
a3e6006530 Merge commit 'd646c9d155' into concedo_experimental 
# Conflicts:
#	.github/actions/windows-setup-rocm/action.yml
#	.github/workflows/build-apple.yml
#	.github/workflows/release.yml
#	.github/workflows/server-self-hosted.yml
#	examples/training/README.md
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/hvx-arith.h
#	ggml/src/ggml-hexagon/htp/hvx-log.h
#	ggml/src/ggml-hexagon/htp/hvx-norm.h
#	ggml/src/ggml-hexagon/htp/hvx-scale.h
#	ggml/src/ggml-hexagon/htp/hvx-sqrt.h
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.h
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	tests/test-backend-ops.cpp
#	tests/test-mtmd-c-api.c
#	tests/test-mtmd-impl.cpp
 2026-09-04 16:05:40 +08:00
Georgi Gerganov
e750b887a8
common, server : enable preserve_reasoning kwarg by default, log its effective state (#28174) 
* common, server : enable preserve_reasoning kwarg by default, log its effective state

If the preserve_reasoning chat template kwarg is not specified explicitly
via --reasoning-preserve / --no-reasoning-preserve, it is enabled by
default after argument processing. The server logs the effective state of
the kwarg, warns that it is enabled by default when the template supports
it, and only warns "has no effect" when it was enabled explicitly on a
template that does not support it. Setting the kwarg via
--chat-template-kwargs is deprecated.

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* cont : update comment

Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>

---------

Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>
 2026-09-02 19:19:54 +03:00
Concedo
c22d6e5c35 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	ggml/src/ggml-cann/aclnn_ops.cpp
#	ggml/src/ggml-cann/aclnn_ops.h
#	ggml/src/ggml-cann/ggml-cann.cpp
#	ggml/src/ggml-et/et-kernels/src/glu_f32.c
#	ggml/src/ggml-et/ggml-et-cpu-compare.cpp
#	ggml/src/ggml-et/ggml-et-ops.cpp
#	ggml/src/ggml-et/ggml-et.cpp
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/act-ops.c
#	ggml/src/ggml-hexagon/htp/cpy-ops.c
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/glu.cl
#	ggml/src/ggml-openvino/openvino/op/glu_swiglu.cpp
#	ggml/src/ggml-openvino/openvino/op_table.cpp
#	ggml/src/ggml-openvino/openvino/op_table.h
#	ggml/src/ggml-rpc/CMakeLists.txt
#	ggml/src/ggml-sycl/element_wise.cpp
#	ggml/src/ggml-sycl/element_wise.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/glu.wgsl
#	scripts/sync_vendor.py
#	tests/test-backend-ops.cpp
#	tools/cli/README.md
#	tools/completion/README.md
#	tools/llama-bench/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/rpc/CMakeLists.txt
#	tools/server/README.md
 2026-08-31 18:39:16 +08:00
Concedo
4e3a25f3bc Merge commit '9e54e687cb' into concedo_experimental 
# Conflicts:
#	docs/backend/snapdragon/windows.md
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp-drv.cpp
#	ggml/src/ggml-hexagon/htp-drv.h
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-sycl/fattn-common.hpp
#	ggml/src/ggml-sycl/fattn-onednn.cpp
#	ggml/src/ggml-sycl/fattn-onednn.hpp
#	ggml/src/ggml-sycl/fattn.cpp
#	ggml/src/ggml-sycl/fattn.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-virtgpu/ggml-backend.cpp
#	scripts/snapdragon/build.py
#	tests/test-alloc.cpp
#	tests/test-backend-ops.cpp
#	tools/llama-bench/README.md
#	tools/llama-bench/llama-bench.cpp
 2026-08-31 18:05:39 +08:00
Georgi Gerganov
bebc9350ec
common: rename --tensor-read-lazy to --lazy-mode, add -lzm shorthand (#27969) 
Rename the --tensor-read-lazy CLI argument to --lazy-mode, to match the
internal lazy_mode parameter, and add a -lzm shorthand. Sync the READMEs.

Assisted-by: pi:llama.cpp/Qwen3.8-27B
 2026-08-30 09:18:10 +03:00
Xuan-Son Nguyen
50f068ffff
bench: add --tensor-read-lazy (#27881) 
* bench: add --tensor-read-lazy

* rm the alias

* rename to LLAMA_LAZY_MODE_*
 2026-08-28 20:51:05 +02:00
Concedo
8d223ab855 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/openvino.Dockerfile
#	.github/workflows/build-cache.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/release.yml
#	ci/run.sh
#	docs/backend/OPENVINO.md
#	docs/speculative.md
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-arith.h
#	ggml/src/ggml-hexagon/htp/hvx-log.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.h
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-openvino/CMakeLists.txt
#	ggml/src/ggml-openvino/ggml-decoder.cpp
#	ggml/src/ggml-openvino/ggml-decoder.h
#	ggml/src/ggml-openvino/ggml-openvino-extra.cpp
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-openvino/openvino/op/cpy.cpp
#	ggml/src/ggml-openvino/openvino/op/flash_attn_ext.cpp
#	ggml/src/ggml-openvino/openvino/op/gated_delta_net.cpp
#	ggml/src/ggml-openvino/openvino/op/view.cpp
#	ggml/src/ggml-openvino/openvino/op_table.cpp
#	ggml/src/ggml-openvino/openvino/op_table.h
#	ggml/src/ggml-openvino/openvino/translate_session.cpp
#	ggml/src/ggml-openvino/openvino/utils.cpp
#	ggml/src/ggml-openvino/utils.cpp
#	ggml/src/ggml-openvino/utils.h
#	ggml/src/ggml-sycl/fattn-onednn.cpp
#	ggml/src/ggml-sycl/fattn.cpp
#	scripts/pr2wt.sh
#	src/CMakeLists.txt
#	src/llama-mmap.cpp
#	src/llama-quant.cpp
#	tests/CMakeLists.txt
#	tests/test-arg-parser.cpp
#	tests/test-backend-ops.cpp
#	tests/test-llama-archs.cpp
#	tests/test-save-load-state.cpp
#	tools/cli/README.md
#	tools/completion/README.md
#	tools/server/README.md
 2026-08-28 22:29:03 +08:00
Concedo
ae60e1d6e0 Merge commit 'deae5ee133' into concedo_experimental 
# Conflicts:
#	.devops/openvino.Dockerfile
#	.github/workflows/build-apple.yml
#	.github/workflows/build-cuda-ubuntu.yml
#	.github/workflows/docker.yml
#	.github/workflows/release.yml
#	.github/workflows/server-sanitize.yml
#	.github/workflows/ui-build-self-hosted.yml
#	.github/workflows/ui-build.yml
#	.github/workflows/ui-publish.yml
#	.github/workflows/ui-self-hosted.yml
#	.github/workflows/ui.yml
#	CMakeLists.txt
#	docs/backend/snapdragon/CMakeUserPresets.json
#	docs/backend/snapdragon/README.md
#	docs/backend/snapdragon/developer.md
#	docs/backend/snapdragon/linux.md
#	docs/backend/snapdragon/windows.md
#	docs/ops.md
#	docs/ops/Vulkan.csv
#	ggml/cmake/ggml-config.cmake.in
#	ggml/src/ggml-cpu/CMakeLists.txt
#	ggml/src/ggml-cpu/kleidiai/kernels.cpp
#	ggml/src/ggml-cpu/kleidiai/kernels.h
#	ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp-opnode.h
#	ggml/src/ggml-hexagon/htp/CMakeLists.txt
#	ggml/src/ggml-hexagon/htp/act-ops.c
#	ggml/src/ggml-hexagon/htp/cpy-ops.c
#	ggml/src/ggml-hexagon/htp/dma-queue.h
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/get-rows-ops.c
#	ggml/src/ggml-hexagon/htp/hex-utils.h
#	ggml/src/ggml-hexagon/htp/htp-ctx.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/htp-tensor.c
#	ggml/src/ggml-hexagon/htp/htp-tensor.h
#	ggml/src/ggml-hexagon/htp/hvx-arith.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.h
#	ggml/src/ggml-hexagon/htp/set-rows-ops.c
#	ggml/src/ggml-rpc/CMakeLists.txt
#	scripts/snapdragon/ggml-hexagon-profile.py
#	scripts/snapdragon/ggml-hexagon-trace.py
#	tests/test-backend-ops.cpp
#	tools/cli/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/rpc/README.md
#	tools/server/README.md
#	tools/ui/tests/stories/a11y/ChatScreenForm.a11y.stories.svelte
 2026-08-28 19:31:29 +08:00
Bartowski
18443257a3
server: add ctx-per-slot (--kv-unified-per-slot) (#24124) 
* Add ctx-per-slot argument for unifid KV cache

* Swap out ctx fractions for ctx pool slots

* Formatting cleanup

* Remove ctx-pool-slots, make ctx-per-slot an int

* refactor it

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2026-08-27 22:39:14 +02:00
Xuan-Son Nguyen
fac889fb38
llama: model_loader: add TENSOR_READ_LAZY (#27794) 
* llama: model_loader: add TENSOR_GET_ROW_LAZY

* add --tensor-read-lazy

* rename to TENSOR_READ_LAZY

* gen docs

* address comments
 2026-08-27 15:14:34 +02:00
Gaurav Garg
2bb9bddafa
spec: Add benchmark-only synthetic speculative acceptance options (#27711) 
* Add benchmark-only synthetic speculative acceptance to llama-server and llama-cli

* Address review comments

* Address review comments

* Add some comments in the code
 2026-08-27 13:53:42 +03:00
Xuan-Son Nguyen
f29551215b
args: add --video-* CLI arguments (#24318) 
* args: add --video-* CLI arguments

* gen docs

* nits

* add mtmd_helper_init_opt
 2026-08-27 12:11:12 +02:00
Jonas J
c5fc7e3488
llama : add --n-cpu-ffn option (#26622) 
* common : dedupe --n-cpu-moe / --spec-draft-n-cpu-moe override loops

* common : add --n-cpu-ffn to CPU-offload dense FFN weights of first N layers

* common : generalize llm_ffn_block_regex over the FFN regex, drop TODO
 2026-08-27 11:26:42 +02:00
Concedo
4447017602 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/actions/ccache-clear/action.yml
#	.github/workflows/build-apple.yml
#	.github/workflows/build-cpu.yml
#	.github/workflows/build-cuda-ubuntu.yml
#	.github/workflows/build-opencl.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-sycl.yml
#	.github/workflows/build-vulkan.yml
#	.github/workflows/build-wasm.yml
#	.github/workflows/build-webgpu.yml
#	.github/workflows/hip-quality-check.yml
#	.github/workflows/server.yml
#	CONTRIBUTING.md
#	README.md
#	ci/run.sh
#	common/CMakeLists.txt
#	common/chat.cpp
#	docs/autoparser.md
#	ggml/src/ggml-webgpu/wgsl-shaders/flash_attn.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/flash_attn_tile.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/flash_attn_vec_split.wgsl
#	scripts/sync_vendor.py
#	tests/CMakeLists.txt
#	tests/peg-parser/test-json-serialization.cpp
#	tests/peg-parser/tests.h
#	tests/test-chat-auto-parser.cpp
#	tests/test-chat-peg-parser.cpp
#	tests/test-chat-template.cpp
#	tests/test-chat.cpp
#	tests/test-grammar-integration.cpp
#	tests/test-jinja.cpp
#	tests/test-json-schema-to-grammar.cpp
#	tests/test-llama-archs.cpp
#	tests/test-model-resolution.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tools/CMakeLists.txt
 2026-08-25 20:44:34 +08:00
Concedo
3265983620 Merge commit '2fb989b9e7' into concedo_experimental 
# Conflicts:
#	.github/actions/ccache-clear/action.yml
#	.github/workflows/build-cpu.yml
#	.github/workflows/make-release.yml
#	.github/workflows/release.yml
#	.pi/gg/SYSTEM.md
#	AGENTS.md
#	README.md
#	docs/build.md
#	ggml/src/ggml-opencl/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/moe_combine.cl
#	ggml/src/ggml-sycl/convert.cpp
#	ggml/src/ggml-sycl/dequantize.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/esimd.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/quants.hpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	src/CMakeLists.txt
#	tests/test-llama-archs.cpp
#	tools/llama-bench/llama-bench.cpp
#	tools/mtmd/CMakeLists.txt
 2026-08-25 20:30:15 +08:00
Concedo
bfd6500450 Merge commit '873e5d8e39' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-cmake-pkg.yml
#	.github/workflows/build-cpu.yml
#	.github/workflows/make-release.yml
#	.github/workflows/release.yml
#	.pi/gg/SYSTEM.md
#	CMakeLists.txt
#	cmake/arm64-windows-llvm.cmake
#	docs/backend/ET.md
#	docs/build.md
#	docs/development/HOWTO-add-model.md
#	ggml/CMakeLists.txt
#	ggml/src/CMakeLists.txt
#	ggml/src/ggml-cpu/CMakeLists.txt
#	ggml/src/ggml-cpu/kleidiai/kernels.cpp
#	ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/rope-ops.c
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/mul_mv_q6_k_f32_flat.cl
#	ggml/src/ggml-opencl/kernels/rope.cl
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/dpct/helper.hpp
#	ggml/src/ggml-sycl/element_wise.cpp
#	ggml/src/ggml-sycl/esimd.hpp
#	ggml/src/ggml-sycl/fattn-mkl.cpp
#	ggml/src/ggml-sycl/fattn-onednn.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/im2col.cpp
#	ggml/src/ggml-sycl/norm.cpp
#	ggml/src/ggml-sycl/rope.cpp
#	ggml/src/ggml-sycl/set_rows.cpp
#	ggml/src/ggml-vulkan/CMakeLists.txt
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/rope.wgsl
#	ggml/src/ggml-zendnn/CMakeLists.txt
#	scripts/make-release-desc.sh
#	scripts/sync-ggml.last
#	tests/test-backend-ops.cpp
#	tests/test-json-schema-to-grammar.cpp
#	tools/cli/README.md
#	tools/server/README.md
#	tools/ui/src/lib/constants/settings.constants.ts
#	tools/ui/src/lib/services/chat.service.ts
 2026-08-24 21:15:18 +08:00
Xuan-Son Nguyen
d9f918d2d0
common: add json.h abstraction (#27511) 
* add common/json

* migrate common

* adapt jinja

* migrate server

* big wip

* migrate tests

* wip

* revert some excessive changes

* wip

* wip 2

* revert redundant changes

* fix server crash

* various fixes

* fix ci

* harden a bit

* clean up

* rm json-shim

* add some comments

* rm redundant decl
 2026-08-22 16:28:28 +02:00
Xuan-Son Nguyen
9fee29e943
arg: remove -no-cnv from cli [no ci] (#27542) 
* arg: remove -no-cnv from cli

* clarify about not adding exccesive test cases
 2026-08-22 15:53:56 +02:00
Concedo
8c732ca2cc Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.devops/openvino.Dockerfile
#	.github/actions/windows-setup-cuda/action.yml
#	.github/workflows/build-cache.yml
#	.github/workflows/build-cpu.yml
#	.github/workflows/build-cuda-windows.yml
#	.github/workflows/build-openvino.yml
#	.github/workflows/build-self-hosted.yml
#	.github/workflows/build-vulkan.yml
#	.github/workflows/docker.yml
#	.github/workflows/make-release.yml
#	.github/workflows/release.yml
#	AUTHORS
#	CMakeLists.txt
#	README.md
#	build-xcframework.sh
#	ci/run.sh
#	common/CMakeLists.txt
#	docs/backend/OPENVINO.md
#	examples/gguf-hash/CMakeLists.txt
#	examples/gguf-hash/gguf-hash.cpp
#	ggml/CMakeLists.txt
#	ggml/src/ggml-cann/ggml-cann.cpp
#	ggml/src/ggml-et/ggml-et.cpp
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.c
#	ggml/src/ggml-hexagon/htp/flash-attn-ops.h
#	ggml/src/ggml-opencl/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/flash_attn_f16.cl
#	ggml/src/ggml-opencl/kernels/flash_attn_f32.cl
#	ggml/src/ggml-opencl/kernels/moe_sort_by_expert.cl
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_reg_tile.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_subgroup_matrix.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
#	scripts/sync-ggml.last
#	tests/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-llama-archs.cpp
#	tools/mtmd/CMakeLists.txt
#	tools/mtmd/mtmd-helper.cpp
#	tools/perplexity/perplexity.cpp
#	tools/server/README.md
#	tools/ui/src/lib/hooks/use-tools-panel.svelte.ts
#	vendor/hash/CMakeLists.txt
 2026-08-21 23:27:27 +08:00
John-Henry Lim
681c29d36a
mtmd: add --mmproj-device argument (#23255) 
* feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var

* feat: load mmproj device backend immediately, add -mmdev shortflag

* fix: its a pointer now get the name

* clean up

* gen docs

* nits

---------

Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
 2026-08-20 18:45:37 +02:00
Concedo
f0cd0225aa Merge commit '34af94cd9a' into concedo_experimental 
# Conflicts:
#	.github/workflows/docker.yml
#	.github/workflows/make-release.yml
#	.github/workflows/release.yml
#	.pi/gg/SYSTEM.md
#	CMakeLists.txt
#	build-xcframework.sh
#	docs/development/HOWTO-add-model.md
#	docs/ops.md
#	docs/ops/SYCL.csv
#	docs/speculative.md
#	examples/sycl/update-ops-doc.sh
#	ggml/CMakeLists.txt
#	ggml/src/ggml-sycl/cpy.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	scripts/make-release-checks.sh
#	scripts/sync-ggml.last
#	tests/test-chat-auto-parser.cpp
#	tests/test-chat.cpp
#	tests/test-jinja.cpp
#	tests/test-llama-archs.cpp
#	tests/testing.h
#	tools/llama-bench/llama-bench.cpp
#	tools/server/README.md
 2026-08-20 17:08:46 +08:00
Xuan-Son Nguyen
ee4c505a4f
server: add dedup-cache-models preset option (#27346) 2026-08-19 11:04:26 +02:00
Xuan-Son Nguyen
666f8898a2
ui: move get_datetime tool to frontend (#27255) 
* ui: move get_datetime tool to frontend

* clarify docs

* server: drop the now unused ctime include

strftime() and gmtime_r() were the only users, both went away with the
get_datetime tool. Also make the renderer's catch inert: the browser
executor always emits JSON, so a non-JSON result is no longer a date to
display.

---------

Co-authored-by: Pascal <admin@serveurperso.com>
 2026-08-17 14:33:55 +02:00