vrr/koboldcpp
2
0
Fork 0
mirror of https://github.com/LostRuins/koboldcpp.git synced 2026-10-21 22:00:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
koboldcpp/examples/diffusion
History
Concedo 945df48dd3 Merge commit '60e9cf7a7b' into concedo_experimental 
# Conflicts:
#	examples/batched/batched.cpp
#	examples/debug/debug.cpp
#	examples/eval-callback/eval-callback.cpp
#	examples/idle/idle.cpp
#	examples/llama.android/lib/src/main/cpp/ai_chat.cpp
#	examples/lookahead/lookahead.cpp
#	examples/lookup/lookup.cpp
#	examples/parallel/parallel.cpp
#	examples/passkey/passkey.cpp
#	examples/retrieval/retrieval.cpp
#	examples/simple-chat/simple-chat.cpp
#	examples/simple/simple.cpp
#	examples/speculative-simple/speculative-simple.cpp
#	examples/speculative/speculative.cpp
#	tests/test-backend-sampler.cpp
#	tests/test-fusion.cpp
#	tests/test-llama-archs.cpp
#	tests/test-recurrent-state-rollback.cpp
#	tests/test-save-load-state.cpp
#	tests/test-state-restore-fragmented.cpp
#	tests/test-thread-safety.cpp
#	tools/batched-bench/batched-bench.cpp
#	tools/cvector-generator/cvector-generator.cpp
#	tools/imatrix/imatrix.cpp
#	tools/llama-bench/llama-bench.cpp
#	tools/perplexity/perplexity.cpp
#	tools/results/results.cpp
2026-10-03 01:14:19 +08:00
..
diffusion-cli.cpp args: refactor mlock/mmap/directio into load-mode (#20834) 2026-07-23 20:32:56 +08:00
diffusion.cpp batch: migrate the rest of examples to llama_batch_ext (#29601) 2026-09-30 18:08:43 +02:00
diffusion.h examples: refactor diffusion generation (#22590) 2026-05-04 20:19:30 +08:00
README.md examples: refactor diffusion generation (#22590) 2026-05-04 20:19:30 +08:00

README.md

Diffusion Text Generation

This directory contains implementations for Diffusion LLMs (DLLMs)

More Info:

Parameters

The diffusion CLI supports various parameters to control the generation process:

Core Diffusion Parameters

  • --diffusion-steps: Number of diffusion steps (default: 256)
  • --diffusion-algorithm: Algorithm for token selection
    • 0: DIFFUSION_ALGORITHM_ORIGIN - Token will be generated in a purely random order from https://arxiv.org/abs/2107.03006.
    • 1: DIFFUSION_ALGORITHM_ENTROPY_BASED - Entropy-based selection
    • 2: DIFFUSION_ALGORITHM_MARGIN_BASED - Margin-based selection
    • 3: DIFFUSION_ALGORITHM_RANDOM - Random selection
    • 4: DIFFUSION_ALGORITHM_CONFIDENCE_BASED - Confidence-based selection (default)
    • More documentation here https://github.com/DreamLM/Dream
  • --diffusion-visual: Enable live visualization during generation

Scheduling Parameters

Choose one of the following scheduling methods:

Timestep-based scheduling:

  • --diffusion-eps: Epsilon value for timestep scheduling (e.g., 0.001)

Block-based scheduling:

  • --diffusion-block-length: Block size for block-based scheduling (e.g., 32)

Sampling Parameters

  • --temp: Temperature for sampling (0.0 = greedy/deterministic, higher = more random)
  • --top-k: Top-k filtering for sampling
  • --top-p: Top-p (nucleus) filtering for sampling
  • --seed: Random seed for reproducibility

Model Parameters

  • -m: Path to the GGUF model file
  • -p: Input prompt text
  • -ub: Maximum sequence length (ubatch size)
  • -c: Context size
  • -b: Batch size

Examples

Dream architecture:

llama-diffusion-cli -m dream7b.gguf -p "write code to train MNIST in pytorch" -ub 512 --diffusion-eps 0.001 --diffusion-algorithm 3 --diffusion-steps 256 --diffusion-visual

LLaDA architecture:

llama-diffusion-cli -m llada-8b.gguf -p "write code to train MNIST in pytorch" -ub 512 --diffusion-block-length 32 --diffusion-steps 256 --diffusion-visual

RND1 architecture:

llama-diffusion-cli -m RND1-Base-0910.gguf -p "write code to train MNIST in pytorch" -ub 512 --diffusion-algorithm 1 --diffusion-steps 256 --diffusion-visual --temp 0.5 --diffusion-eps 0.001