vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-10-22 06:09:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
16632 commits 3 branches 258 tags 1,022 MiB
Commit graph

37 commits

Author SHA1 Message Date
Concedo
cbd7e272f2 allow reading vocab from gemma (ltx) and umt5 (wan) gguf, no longer bundle them 2026-10-21 22:08:03 +08:00
Concedo
01e3b5032c allow using tokenizers baked into the gguf llm as a last resort 2026-10-21 17:57:28 +08:00
Concedo
5c13e63c66 paralleltotalcontext flag 2026-10-20 23:09:06 +08:00
Wagner Bruna
aa034e1004
sd: sync with master-975-95b3090 (#2542) 
* sd: sync with master-968-8cd731e

* feat: add TAEQI2.1 support for Qwen Image 2.1
* feat: add MoSR upscaler support
* feat: add Qwen Image 2.1 Turbo sigma preset
* fix: reclaim before accepting wrapped Vulkan iGPU memory estimates
* feat: add per-module automatic flash attention selection
* feat: add automatic direct convolution selection
* chore: move parameter and tokenizer diagnostics to debug
* fix: prevent core dump in CPU backend if using FP8 (e4m3) tensors
* fix: benchmark Vulkan direct convolution in auto mode
* feat: add FLUX.3 Image Klein and Klein Base support

* sd: update Makefile, fix flash attention workaround

* sd: add support for Flux 3

* sd: sync with master-973-f0aac52

* fix: open model files with UTF-8 paths on Windows
* fix: handle UTF-8 file paths on Windows
* chore: sync ggml and remove obsolete conv1d workarounds
* feat: add moss-image support

* sd: add support for MOSS-Image

* sd: sync with master-975-95b3090

* feat: support Diffusers-format MiniMax-H3 LoRAs
* feat: support MOSS split QKV weights and batch host loading

* sd: set conv_direct through the parameter list string
 2026-10-20 17:35:33 +08:00
Concedo
7f5372aed5 improve kv allocation when using parallelrequests, allocates more kv but in a diminishing way 2026-10-20 00:20:25 +08:00
Concedo
d3c1b40be1 use 2.0 swa warmup multiplier 2026-10-19 15:25:16 +08:00
Concedo
8a4a15c854 improvement to swa warmup 2026-10-19 15:25:01 +08:00
Concedo
f8e027021f remove forced 512 swa padding 2026-10-19 14:33:51 +08:00
Concedo
7190bf6e62 account for guidance 2026-10-18 11:58:40 +08:00
askmyteapot
43e511ef6e
Expose lazy tensor loading and the upstream MoE GPU cache (#2544) 
* model: add native lazy-loading and MoE cache inputs

* model: propagate lazy mode and target-only MoE cache

* launcher: expose lazy tensors and MoE cache in GUI, CLI and configs

* tests: cover memory option defaults, validation and config round trips

* docs: explain lazy tensor loading and experimental expert cache

* docs: trim trailing blank line

* Delete tests/test_model_memory_options.py

* docs: clarify temporary validation checks were removed

* logging: keep scheduler reallocation traces in verbose mode

* model: append new memory fields to native input struct

* launcher: append memory fields in matching ctypes order

* docs: remove guide already covered upstream
 2026-10-18 11:51:26 +08:00
Concedo
9cb9f5edc1 a better parallelrequests solution to allow cache reuse 2026-10-18 00:44:23 +08:00
Concedo
f2b6b1e14a improve swa handling for gemma4 heuristically 2026-10-17 17:33:33 +08:00
Concedo
990de011ae additional sanitizations for legacy model loaders 2026-10-16 17:15:31 +08:00
Wagner Bruna
2698d1dca0
sd: model loading refactors (#2530) 
* sd: rename internal t5xxl parameter

* sd: perform all model path changes inside kcpp_set_model_paths

* sd: simplify TAE file selection
 2026-10-12 18:42:43 +08:00
Concedo
4fd0ceee30 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	common/chat.cpp
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-opencl/kernels/cpy.cl
#	ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_0_f32.cl
#	ggml/src/ggml-opencl/kernels/gemv_noshuffle_q4_k_f32.cl
#	ggml/src/ggml-sycl/convert.cpp
#	ggml/src/ggml-sycl/dequantize.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/quants.hpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	scripts/sync_vendor.py
#	tests/CMakeLists.txt
#	tests/peg-parser/tests.h
#	tests/test-backend-ops.cpp
#	tests/test-chat-auto-parser.cpp
#	tests/test-chat.cpp
#	tests/test-peg-parser.cpp
#	tools/llama-bench/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/server/README.md
 2026-10-10 23:40:34 +08:00
Concedo
9e258d9059 remove deprecated field for stream_layers 2026-10-10 18:58:54 +08:00
Concedo
0890a5ee77 Merge commit '033df86b69' into concedo_experimental 
# Conflicts:
#	docs/backend/SYCL.md
#	docs/development/parsing.md
#	examples/speculative-simple/speculative-simple.cpp
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/get-rows-ops.c
#	ggml/src/ggml-hexagon/htp/get-rows-ops.h
#	ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-erf.h
#	ggml/src/ggml-hexagon/htp/main.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.c
#	ggml/src/ggml-hexagon/htp/unary-ops.h
#	ggml/src/ggml-sycl/CMakeLists.txt
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/element_wise.cpp
#	ggml/src/ggml-sycl/element_wise.hpp
#	ggml/src/ggml-sycl/fusion.cpp
#	ggml/src/ggml-sycl/fwht.cpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/im2col.hpp
#	ggml/src/ggml-sycl/pad.hpp
#	ggml/src/ggml-sycl/rope.hpp
#	ggml/src/ggml-sycl/upscale.hpp
#	scripts/snapdragon/run.py
#	scripts/sync_vendor.py
#	src/CMakeLists.txt
#	tests/test-backend-ops.cpp
#	tests/test-backend-sampler.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tools/imatrix/imatrix.cpp
#	tools/server/README.md
 2026-10-10 13:19:02 +08:00
Concedo
29b78bc0d3 allow batch handling of images 2026-10-10 12:29:07 +08:00
Concedo
fdbfae1d99 fixed grammar bug with token id 0 2026-10-10 11:39:26 +08:00
Concedo
e4532f051d coerce dry values 2026-10-08 17:15:04 +08:00
Concedo
a7c4632260 custom sampler order for batched requests 2026-10-08 15:44:54 +08:00
Concedo
b25d51cf0c Add remaining unbatched samplers 2026-10-08 15:44:36 +08:00
Concedo
ea6477ccaf reasoning budget for batching 2026-10-07 23:01:16 +08:00
Concedo
d2cd79bb3f fix tool calling batching 2026-10-07 21:18:20 +08:00
Concedo
062b30f36d allow grammar and dry in batched requests 2026-10-07 16:58:14 +08:00
Concedo
15fa0165b8 allow more samplers in batched mode 2026-10-07 16:37:57 +08:00
Concedo
c361c54b70 better mmproj fit 2026-10-04 23:51:01 +08:00
Concedo
77b990243f improve lifeboat behavior 2026-10-04 16:43:30 +08:00
Concedo
f334e5c0e9 fix rnn lifeboat regression 2026-10-04 11:37:27 +08:00
Wagner Bruna
bbb221144c
sd: support for custom sigma lists (#2514) 2026-10-04 11:09:55 +08:00
Concedo
4d7241d34c smartcache save/load improvements 2026-10-04 11:04:01 +08:00
Concedo
4d27749c93 Merge branch 'upstream' into concedo_experimental 
# Conflicts:
#	.github/workflows/build-cuda-ubuntu.yml
#	.github/workflows/build-cuda-windows.yml
#	.github/workflows/build-ibm.yml
#	.github/workflows/ci-self-hosted-cuda.yml
#	.github/workflows/release.yml
#	.github/workflows/server-self-hosted.yml
#	AGENTS.md
#	README.md
#	ci/run.sh
#	docs/autoparser.md
#	docs/backend/SYCL.md
#	docs/build.md
#	docs/development/HOWTO-add-model.md
#	docs/function-calling.md
#	docs/ops.md
#	docs/ops/CPU.csv
#	ggml/CMakeLists.txt
#	ggml/src/ggml-blas/CMakeLists.txt
#	ggml/src/ggml-cann/ggml-cann.cpp
#	ggml/src/ggml-cpu/amx/amx.cpp
#	ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
#	ggml/src/ggml-cuda/CMakeLists.txt
#	ggml/src/ggml-et/ggml-et.cpp
#	ggml/src/ggml-hexagon/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/allreduce-ops.c
#	ggml/src/ggml-hexagon/htp/allreduce-ops.h
#	ggml/src/ggml-hexagon/htp/concat-ops.c
#	ggml/src/ggml-hexagon/htp/cpy-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.h
#	ggml/src/ggml-hexagon/htp/work-queue.c
#	ggml/src/ggml-metal/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/fattn-common.hpp
#	ggml/src/ggml-sycl/fattn-onednn.cpp
#	ggml/src/ggml-sycl/fattn-vec.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	ggml/src/ggml-virtgpu/ggml-backend-buffer-type.cpp
#	ggml/src/ggml-vulkan/ggml-vulkan.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/ssm_scan.wgsl
#	ggml/src/ggml-zdnn/ggml-zdnn.cpp
#	pyproject.toml
#	scripts/snapdragon/run.py
#	skills/code-review/SKILL.md
#	tests/fusion/MTL.csv
#	tests/test-alloc.cpp
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tests/test-model-resolution.cpp
#	tests/test-quantize-fns.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tools/llama-bench/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/server/CMakeLists.txt
#	tools/server/README.md
 2026-10-03 17:59:14 +08:00
Concedo
945df48dd3 Merge commit '60e9cf7a7b' into concedo_experimental 
# Conflicts:
#	examples/batched/batched.cpp
#	examples/debug/debug.cpp
#	examples/eval-callback/eval-callback.cpp
#	examples/idle/idle.cpp
#	examples/llama.android/lib/src/main/cpp/ai_chat.cpp
#	examples/lookahead/lookahead.cpp
#	examples/lookup/lookup.cpp
#	examples/parallel/parallel.cpp
#	examples/passkey/passkey.cpp
#	examples/retrieval/retrieval.cpp
#	examples/simple-chat/simple-chat.cpp
#	examples/simple/simple.cpp
#	examples/speculative-simple/speculative-simple.cpp
#	examples/speculative/speculative.cpp
#	tests/test-backend-sampler.cpp
#	tests/test-fusion.cpp
#	tests/test-llama-archs.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tests/test-save-load-state.cpp
#	tests/test-state-restore-fragmented.cpp
#	tests/test-thread-safety.cpp
#	tools/batched-bench/batched-bench.cpp
#	tools/cvector-generator/cvector-generator.cpp
#	tools/imatrix/imatrix.cpp
#	tools/llama-bench/llama-bench.cpp
#	tools/perplexity/perplexity.cpp
#	tools/results/results.cpp
 2026-10-03 01:14:19 +08:00
Concedo
e64e62d86d relocate environment files for conda 2026-09-29 00:13:46 +08:00
Concedo
b11e4b0827 refactor: cleanup project root 3 2026-09-29 00:08:43 +08:00
Concedo
a202cb618d refactor: cleanup project root 2 2026-09-28 23:53:23 +08:00
Concedo
7ac9b99564 refactor: cleanup project root 1 2026-09-28 23:53:06 +08:00