Commit graph

667 commits

Author SHA1 Message Date
Concedo
76e77f3405 updated readme 2026-10-18 17:34:17 +08:00
Concedo
63634ec66a updated readme 2026-10-18 17:07:10 +08:00
Concedo
f41f621e09 updated readme 2026-10-17 18:34:27 +08:00
Concedo
5f70d40c87 link to koboldai lite beta site 2026-10-14 12:00:18 +08:00
Concedo
c3b9e61a55 updated readme 2026-10-12 14:44:25 +08:00
Concedo
d5797118dd add koboldcpp.net link 2026-10-08 21:49:39 +08:00
Concedo
ea6477ccaf reasoning budget for batching 2026-10-07 23:01:16 +08:00
Concedo
deedc8c361 jinja now always enables jinja tools 2026-10-05 21:11:18 +08:00
Concedo
8c8030ad7d change image model links 2026-10-04 22:19:25 +08:00
Concedo
807791a809 updated readme 2026-10-04 16:00:32 +08:00
Concedo
cd9db0dc5b updated readme, handle unsupported OAI tools 2026-10-04 11:52:50 +08:00
Concedo
4d27749c93 Merge branch 'upstream' into concedo_experimental
# Conflicts:
#	.github/workflows/build-cuda-ubuntu.yml
#	.github/workflows/build-cuda-windows.yml
#	.github/workflows/build-ibm.yml
#	.github/workflows/ci-self-hosted-cuda.yml
#	.github/workflows/release.yml
#	.github/workflows/server-self-hosted.yml
#	AGENTS.md
#	README.md
#	ci/run.sh
#	docs/autoparser.md
#	docs/backend/SYCL.md
#	docs/build.md
#	docs/development/HOWTO-add-model.md
#	docs/function-calling.md
#	docs/ops.md
#	docs/ops/CPU.csv
#	ggml/CMakeLists.txt
#	ggml/src/ggml-blas/CMakeLists.txt
#	ggml/src/ggml-cann/ggml-cann.cpp
#	ggml/src/ggml-cpu/amx/amx.cpp
#	ggml/src/ggml-cpu/kleidiai/kleidiai.cpp
#	ggml/src/ggml-cuda/CMakeLists.txt
#	ggml/src/ggml-et/ggml-et.cpp
#	ggml/src/ggml-hexagon/CMakeLists.txt
#	ggml/src/ggml-hexagon/ggml-hexagon.cpp
#	ggml/src/ggml-hexagon/htp/allreduce-ops.c
#	ggml/src/ggml-hexagon/htp/allreduce-ops.h
#	ggml/src/ggml-hexagon/htp/concat-ops.c
#	ggml/src/ggml-hexagon/htp/cpy-ops.c
#	ggml/src/ggml-hexagon/htp/hmx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/htp-ops.h
#	ggml/src/ggml-hexagon/htp/hvx-mm-kernels-tiled.h
#	ggml/src/ggml-hexagon/htp/matmul-ops.c
#	ggml/src/ggml-hexagon/htp/matmul-ops.h
#	ggml/src/ggml-hexagon/htp/work-queue.c
#	ggml/src/ggml-metal/CMakeLists.txt
#	ggml/src/ggml-opencl/ggml-opencl.cpp
#	ggml/src/ggml-openvino/ggml-openvino.cpp
#	ggml/src/ggml-sycl/common.hpp
#	ggml/src/ggml-sycl/dmmv.cpp
#	ggml/src/ggml-sycl/fattn-common.hpp
#	ggml/src/ggml-sycl/fattn-onednn.cpp
#	ggml/src/ggml-sycl/fattn-vec.hpp
#	ggml/src/ggml-sycl/ggml-sycl.cpp
#	ggml/src/ggml-sycl/mmvq.cpp
#	ggml/src/ggml-sycl/vecdotq.hpp
#	ggml/src/ggml-virtgpu/ggml-backend-buffer-type.cpp
#	ggml/src/ggml-vulkan/ggml-vulkan.cpp
#	ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp
#	ggml/src/ggml-webgpu/ggml-webgpu.cpp
#	ggml/src/ggml-webgpu/wgsl-shaders/common_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/get_rows.wgsl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_decls.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/mul_mat_vec_acc.tmpl
#	ggml/src/ggml-webgpu/wgsl-shaders/ssm_scan.wgsl
#	ggml/src/ggml-zdnn/ggml-zdnn.cpp
#	pyproject.toml
#	scripts/snapdragon/run.py
#	skills/code-review/SKILL.md
#	tests/fusion/MTL.csv
#	tests/test-alloc.cpp
#	tests/test-backend-ops.cpp
#	tests/test-chat.cpp
#	tests/test-llama-archs.cpp
#	tests/test-model-resolution.cpp
#	tests/test-quantize-fns.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tools/llama-bench/README.md
#	tools/llama-bench/llama-bench.cpp
#	tools/server/CMakeLists.txt
#	tools/server/README.md
2026-10-03 17:59:14 +08:00
Concedo
befafbc410 rewrote readme 2026-10-02 21:45:25 +08:00
Georgi Gerganov
d8fbd2583a
readme : add cmd install commands (#29850) 2026-10-02 15:39:11 +03:00
Concedo
49af14ccdb update readme 2026-10-01 00:08:38 +08:00
Concedo
bc96f29ad3 adjust readme 2026-09-30 21:13:54 +08:00
Concedo
7ac9b99564 refactor: cleanup project root 1 2026-09-28 23:53:06 +08:00
Concedo
36688de9c0 updated readme 2026-09-26 17:11:40 +08:00
Concedo
f44706a190 updated readme (+1 squashed commits)
Squashed commits:

[90fdd39c7] updated readme
2026-09-20 18:37:35 +08:00
henk717
6c8df269ea
Optimize koboldcpp.sh for local users (#2444)
* koboldcpp.sh is now optimized for local

* Restore 12.1 behavior when build system has no GPU

* Fix PORTABLE_SO text replace fail

* Restore accidentally deleted file

* More koboldcpp.sh fixes

* Move vulkan noavx2 to portable section

* Fix syntax on portable env

* Fix rocm env variables in CI
2026-09-18 10:49:52 +08:00
Concedo
fc1046959e added section for lite.koboldai.net 2026-09-17 23:20:02 +08:00
Concedo
9fd4fae644 update readme and sdui 2026-09-05 09:29:08 +08:00
Georgi Gerganov
a529af96e2
docs : update maintainer PRs link and regenerate AUTHORS (#28365)
Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp
2026-09-04 10:20:49 +03:00
Todor Boinovski
e107984bcf
ops: add Hexagon to ops.md and update main README.md (#28263) 2026-09-03 07:14:36 -07:00
Georgi Gerganov
ccc8fd2baa
readme : update links (#27617)
* readme : update links

* readme : update maintainer PRs list

Add the new members of the `ggml-org` `maintainers` team to the
author filter of the maintainer PRs link (nikwen, marty1885,
Titaniumtown), keeping the canonical team ordering. The list now
matches the team exactly (35 members).

Assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-08-23 20:55:56 +03:00
Sigbjørn Skjæret
3aeb924628
readme : fix server badge alt (#27533) 2026-08-22 10:08:07 +03:00
Georgi Gerganov
2100e59260
readme : update badges (#27531) 2026-08-22 08:25:00 +03:00
Georgi Gerganov
169e4a7ff2
readme : update status badges + regen AUTHORS (#27317)
* readme : update status badges

* authors : regen
2026-08-18 14:35:04 +03:00
Georgi Gerganov
27e345b574
build : fix xcframework + cmake clean-up (#27304)
* xcframework : fix build

* mtmd : remove unused include path

* vendor : use vendor::hash alias target in cmake

CMake reserves "::" in target names for imported/alias targets, so the real
target keeps the name vendor-hash and a vendor::hash ALIAS target is added.
Consumers (mtmd, llama-gguf-hash) now link against the namespaced alias.

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* vendor : add cmake targets for all vendored libs with vendor:: aliases

Add INTERFACE targets for the header-only vendor libs (miniaudio, nlohmann,
sheredom, stb) and ALIAS targets named vendor::<lib> for all of them,
including cpp-httplib and hash. Each exposes the vendor/ root so includes
are namespaced, e.g. <nlohmann/json.hpp>.

Consolidate the per-lib add_subdirectory calls into a single
add_subdirectory(vendor), keeping the cpp-httplib gate on LLAMA_BUILD_COMMON.
Consumers (llama-common, mtmd) now link the aliases instead of relying on
raw vendor/ include paths.

hash: consumers now include via "hash/hash.h"; the vendor/hash dir is kept
as a PRIVATE include so the synced upstream sources compile unmodified.

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* readme : use foo/bar names in acknowledgements

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* ocd : fix valign
2026-08-18 11:16:51 +03:00
Daniel Bevenius
680a9ae63d
cmake : introduce semantic versioning (#26839)
* cmake : introduce semantic versioning (wip)

This commit introduces semantic versioning to llama.cpp.

* squash! cmake : introduce semantic versioning (wip)

* cmake : update test-cmake README notes [no ci]

* include libmtmd in output so show its semversioned

* ci : add make-release workflow

* ci : fix build number check in build-cmake-pkg.yml

* examples : remove trailing whitespace

* ci : abort if upstream ggml version does not exist

* ci : extract step contents into scripts

* ci : add GGML_NATIVE=OFF to ubuntu job

* examples : remove CI build information from test-cmake [no ci]

This commit removes the nightly/release information that I added
previously to keep this focused only on using building and installing
llama.cpp with cmake and being able to quickly verify changes or
troubleshoot issues.

* ci : merge scripts into single script

* remove -dev-build_number support

This commit removes the incremental build number (versioning) support
that I added. This was incorrect and we should only use the semver for
the version. Releases will be tag a nightly build and package
maintainers/managers that build from source can use the tag and it is
therefor important that the correct version is reported. So a
nightly-build will report the semver without the build number. The build
number and commit as availble via cmake and test-cmake has been updated
to include an example of using them:
```console
$ ./build.sh
[test-cmake] version: 0.1.0, build: 10360 (08c69e381)
...
```

Refs: https://github.com/ggml-org/llama.cpp/pull/26839#discussion_r3755836969

* docs: add initial release.md documentation

* cmake : clean-up and add LLAMA_BUILD_IS_DEV option

* ci : remove version input from make-release job

* ci : add LLAMA_BUILD_IS_DEV=OFF to build-cmake-pkg.yml

Refs: https://github.com/danbev/llama.cpp/actions/runs/31576801921/job/94050639145

* docs : update release notes with LLAMA_BUILD_IS_DEV info [no ci]

* ci : add TODO to winget workflow [no ci]

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-08-12 14:15:03 +02:00
Georgi Gerganov
6ad4ab0ea0
readme : remove dev branches (#26832) 2026-08-10 09:53:26 +03:00
Recoordinate
650a4f2eb8
docs: fix --blasbatchssize typo in README (#2373)
The flag is defined as --blasbatchsize in koboldcpp.py; the README had a
doubled 's' (--blasbatchssize) which argparse would reject.
2026-08-01 09:02:56 +08:00
Georgi Gerganov
6b36c23056
readme : refresh (#26280)
* docs : center badges and links, remove Hot topics

- Use <div align="center"> for GitHub-compatible centering
- Add dev branches and compile times links
- Remove Hot topics section

Assisted-by: llama.cpp:Qwen3.6-27B

* readme : remove sections

* docs : center badges, remove Hot topics, extract sections, remove tools

- Use <div align="center"> for GitHub-compatible centering
- Add dev branches and compile times links
- Add lib llama API and llama-server REST API links
- Remove Hot topics section
- Remove Recent API changes section
- Extract XCFramework section into docs/xcframework.md
- Extract Completions section into docs/completions.md
- Extract Obtaining and quantizing models into docs/models.md
- Remove tools usage sections (llama-cli, llama-server, etc.)
- Move Contributing section to the end

Assisted-by: llama.cpp:Qwen3.6-27B

* cont : arrange links

* cont : fix ws

* cont : remove seminal papers

* cont : change sample model

* cont : trim-down contributing section

* cont : sort backends alphabetically

* cont : words

* cont : add fig captions

* docs : models words

* readme : shorter caption

* cont : fix typo

* cont : add window frame to screenshot
2026-07-30 16:14:37 +03:00
Concedo
9fa97f4ca5 made the scam warning more prominent 2026-07-18 11:05:17 +08:00
Georgi Gerganov
4193ea697f
readme : add link to maintainer PRs (#25621) 2026-07-13 16:07:58 +03:00
Tarek Dakhran
88636e178f
model : Add LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M (#24913)
* model : Add LFM2.5-ColBERT-350M and LFM2.5-Embedding-350M

* Restore LFM2 models in README.md
2026-06-24 09:49:46 +03:00
Julien Jerphanion
a1eb756c0b
docs: Add instructions to install llama.cpp from conda-forge (#22219)
* docs: Add instructions to install `llama.cpp` from conda-forge

Signed-off-by: Julien Jerphanion <git@jjerphan.xyz>

* Rewording of instructions

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Signed-off-by: Julien Jerphanion <git@jjerphan.xyz>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-06-15 17:12:25 +02:00
Aleksander Grygier
f7ca93d12c
ui: PWA support (#23871)
* feat: Add basic PWA support and service worker for offline caching

* feat: Vite PWA implementation WIP

* feat: Improve PWA icons generation

* feat: Add PWA workbox to server routes

* feat: Include `version.json` in static assets

* feat: Add HTTP cache headers for PWA static assets

* feat: Update app name for `apple-mobile-web-app-title`

* feat: Implement PWA versioning and automatic update detection

* chore: Update `.gitignore` files

* feat: Splash Screens

* feat: Add dark mode favicon support

* refactor: Cleanup

* fix: Use dark logo for dark splash screens

* refactor: Simplify favicons SVG code

* fix: Adjust caching and polling for reliable service worker updates

* fix: Add missing favicon entry

* fix: Align PWA service worker configuration with SvelteKit build structure

* fix: Replace hashed bundle paths with versioned static paths

* test: Add PWA tests

* ci: Add build output for unit tests

* refactor: Cleanup

* fix: Server build & release versioning

* chore: Update package-lock.json

* chore: Increase PWA cache size

* chore: Update packages

* feat: Update favicons

* refactor: Post-merge fix

* feat: support explicit build version for PWA cache busting

* fix: CI

* feat: Improve PWA Refresh Alert UI

* feat: Add toggleable build version display

* refactor: Cleanup

* feat: Add version mismatch detection and manual app reload

* refactor: replace dynamic imports with static

* refactor: Cleanup

* feat: Add safe space for `pwa-<size>.png` rendered icons

* fix: use relative paths for PWA assets to support base path deployment

* feat: add PWA mode detection via URL query parameter

* feat: Use ?cache=true for SW-cached PWA assets

* refactor: Build process cleanup

* refactor: Decouple PWA versioning and remove ?cache=true workaround

* chore: Update README logo

* feat: Include PWA Assets generation in build script

* refactor: `usePwa` hook for core layout

* fix: Relativize base vite plugin

* fix: remove unnecessary backslash escapes in test regexes

* test: update static asset paths for API Key test

* refactor: Move SvelteKit PWA Options config to constants

* ui: fix update notification never appearing

Keep the PWA hook object intact instead of destructuring needRefreshByStorage,
which freezes the reactive getter. Also exclude loading.html from PWA
precache to prevent 404 errors and broken SW installation.
2026-06-12 15:53:26 +02:00
Georgi Gerganov
6ddc9430b1
readme : add status badges (#24104) 2026-06-04 10:58:13 +03:00
Mikhail Podvitskii
4fb16eccce
model: add Mellum architecture (#23966)
* model: support for Mellum architecture

* model: improve mellum.py formatting

* model: improve mellum.py formatting once again

* deps: downgrade transformers to 4.57.6 (to fix CI)

* deps: remove huggingface_hub dependency

* deps: remove huggingface_hub from test requirements

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
2026-06-02 22:11:12 +03:00
Reese Levine
ee7c30578a
Update WebGPU support and add link to blog/demo (#23483) 2026-05-21 11:00:27 -07:00
Neo Zhang
aabee047d8
[SCYL] add chapter for performance reference in SYCL.md (#23315)
* add chapter for performance reference

* rm unsupported GPU
2026-05-19 09:44:51 +03:00
KITAITI Makoto
7155a49771
readme : update bindings (#23063) 2026-05-15 08:41:24 +03:00
Gaurav Garg
b9afc19cb4
Write a readme on Multi-GPU usage in llama.cpp (#22729)
* Write a readme on Multi-GPU usage in llama.cpp

* Apply suggestions from code review

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

* Address review comments

* Apply suggestions from code review

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>

---------

Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
2026-05-07 17:48:40 +02:00
Concedo
15efea7bf3 add phishing warning 2026-04-29 22:27:45 +08:00
Concedo
755037dacc add docker reference source files 2026-04-27 20:38:26 +08:00
Concedo
9b38d83377 updated the readme for more docker information to make it clearer what to expect. please don't use the docker on a M series macOS 2026-04-27 18:21:22 +08:00
Concedo
fa3f86ee70 added simplepod cloud template to readme 2026-04-17 10:58:44 +08:00
Adrien Gallouët
c9dc43333f
readme : clarify MODEL_ENDPOINT usage (#20941)
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-03-24 10:35:07 +01:00
Adrien Gallouët
2d2d9c2062
common : add a WARNING for HF cache migration (#20935)
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
2026-03-24 09:24:39 +01:00