Xuan-Son Nguyen
781dbc5ac9
spec: properly handle mtmd input for mtp ( #30257 )
...
* spec: properly handle mtmd input for mtp
* nits
2026-10-10 11:22:38 +02:00
Xuan-Son Nguyen
0bb496dbd3
llama: support both embd + raw tokens in batch ( #29622 )
...
* llama: support both embd + raw tokens in batch
* add to test-llama-archs
* also check case llm_arch_supports_mixed_batch = false
* constant graph topology
* have dedicated input for mixed case
* rm set_tensor_backend
* is_embd --> type
* consolidate m-rope pos handling into one place
* nits
2026-10-05 01:35:49 +02:00
Xuan-Son Nguyen
fc343a84bb
llama: add llama_batch_ext ( #24669 )
...
* (wip) add llama_batch_ext
* wip
* updated design
* updated impl
* change signature
* unused var
* demo common_prompt_batch_decode
* fix pos
* tmp disable test-batch-alloc
* fix compat
* nits: add const
* no more pos_max
* add comment about llama_batch_ext_set_embd_state
* handle n_embd_out properly
* rename api --> embd_token
* llama_embd
* stub llama_batch_ext_set_embd_state
* support both token + embd + state in batch
* llama_batch_ext_add_embd
* upstream some changes
* nits
* fix test-batch-alloc
* add test for compat
2026-09-24 16:25:07 +02:00
Aman Gupta
961e4b26a7
llama-batch: add unit test ( #25471 )
...
* llama-batch: add unit test
* fix win32 builds
* add not implemented assertion in unused methods
* remove unreachable code
2026-07-10 11:04:31 +08:00
