WFGY/ProblemMap/GlobalFixMap/LocalDeploy_Inference/vllm.md

vLLM: Guardrails and Fix Patterns

Field guide for stabilizing vLLM-based local inference pipelines. Use these checks when models serve correctly on API providers but fail under high-throughput GPU serving with vLLM.

---

Open these first

• Visual recovery: RAG Architecture & Recovery
• Retrieval tuning knobs: Retrieval Playbook
• Snippet traceability: retrieval-traceability.md
• Ordering & race fixes: bootstrap-ordering.md, deployment-deadlock.md, predeploy-collapse.md
• Embedding issues: embedding-vs-semantic.md

---

Core acceptance

• ΔS(question, retrieved) ≤ 0.45
• Coverage ≥ 0.70 for target section
• λ remains convergent across 3 paraphrases and 2 seeds
• Throughput scaling does not shift retrieved citations

---

Typical vLLM breakpoints and fix

Symptom	Likely cause	Fix
Works at batch=1 but fails at scale	Context window fragmentation / GPU memory swap	context-drift.md, entropy-collapse.md
Citations disappear at high load	Async batch merge drops offsets	retrieval-traceability.md, data-contracts.md
Different answers run-to-run	λ flips with batch ordering	logic-collapse.md, rerankers.md
Index correct but retrieval unstable	Embedding vs metric mismatch in store	embedding-vs-semantic.md, vectorstore-fragmentation.md
GPU OOM / crash at warm-up	Preload sequence too large, missing fence	bootstrap-ordering.md

---

Fix in 60 seconds

1. Measure ΔS at batch=1 and batch=32. If ΔS rises >0.60 only at scale → async batching issue.
2. Probe λ across 3 paraphrases. If flips, apply BBAM.
3. Enforce contracts: citations must include snippet_id, offsets.
4. GPU warm-up: preload with a dummy batch before first live call.
5. Verify throughput stability with replay test (2 seeds, same dataset).

---

Copy-paste test prompt

I am running vLLM locally.  
Models served with async batching.  
Question: "{user_question}"  

Please return:
1. ΔS at batch=1 and batch=32  
2. λ across 3 paraphrases  
3. Whether citations preserved (snippet_id, offsets)  
4. Minimal structural fix if ΔS ≥ 0.60  

---

🔗 Quick-Start Downloads (60 sec)

Tool	Link	3-Step Setup
WFGY 1.0 PDF	Engine Paper	1️⃣ Download · 2️⃣ Upload to your LLM · 3️⃣ Ask “Answer using WFGY + <your question>”
TXT OS (plain-text OS)	TXTOS.txt	1️⃣ Download · 2️⃣ Paste into any LLM chat · 3️⃣ Type “hello world” — OS boots instantly

---

🧭 Explore More

Module	Description	Link
WFGY Core	WFGY 2.0 engine is live: full symbolic reasoning architecture and math stack	View →
Problem Map 1.0	Initial 16-mode diagnostic and symbolic fix framework	View →
Problem Map 2.0	RAG-focused failure tree, modular fixes, and pipelines	View →
Semantic Clinic Index	Expanded failure catalog: prompt injection, memory bugs, logic drift	View →
Semantic Blueprint	Layer-based symbolic reasoning & semantic modulations	View →
Benchmark vs GPT-5	Stress test GPT-5 with full WFGY reasoning suite	View →
🧙‍♂️ Starter Village 🏡	New here? Lost in symbols? Click here and let the wizard guide you through	Start →

---

👑 Early Stargazers: See the Hall of Fame ⭐ WFGY Engine 2.0 is already unlocked. ⭐ Star the repo to help others discover it and unlock more on the Unlock Board.