vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-10-20 11:00:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

ggml-cpu : size set_rows conversion buffer for all threads (#30454)

Browse source 
* ggml-cpu : size set_rows conversion buffer for all threads

SET_ROWS is planned with n_tasks = 1, but every thread runs it and uses its own row of the F16 -> F32 conversion buffer, so threads > 0 wrote past the work buffer when the graph had another multi-threaded op.

* move set_rows to n_task = n_threads group
This commit is contained in:
Aman Gupta 2026-10-14 21:01:53 +05:30 • committed by GitHub
parent 62df1815bc
commit 5cdadfc326
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
ggml/src/ggml-cpu/ggml-cpu.c
View file

@ -2360,11 +2360,11 @@ static int ggml_get_n_tasks(struct ggml_tensor * node, int n_threads) {
case GGML_OP_MUL_MAT:
case GGML_OP_MUL_MAT_ID:
case GGML_OP_OUT_PROD:
case GGML_OP_SET_ROWS:
{
n_tasks = n_threads;
} break;
case GGML_OP_GET_ROWS:
case GGML_OP_SET_ROWS:
{
// FIXME: get_rows can use additional threads, but the cost of launching additional threads
// decreases performance with GPU offloading