Problemsby GlyphTrouble with vLLM draft
3 posts from 3 people3d activeRising, +2 this week
Posts per day
The posts behind it
[Bug]: With `HF_HUB_OFFLINE=1`, a Hub-ID speculative draft model logs `ERROR ... Error retrieving safetensors: ... offline mode is enabled` twice (plus a 2 s retry sleep), although startup succeeds
### Your current environment Environment (summary; collect_env.py output not attached) The relevant code path is unchanged on `main` (checked at 86fdabec0c82d0470c160aa4df7602ac22c207e6), so this should also reproduce on the latest version. ### 🐛 Describe the bug When the...
[RFC]: Decouple Physical PCP Topology from Per-Forward Attention Layout and Draft Execution Policy
### Motivation. #### Problem Summary When running speculative decoding with a replicated Multi-head Latent Attention (MLA) draft model (e.g., `DSparkSpeculator` / `DFlashSpeculator`) under a valid target Prefill Context Parallelism (PCP) + Decode Context Parallelism (DCP)...
[Bug]: Draft model construction resets data_parallel_index to 0 (load_draft_model / create_draft_parallel_config) — residual site not covered by #62062
> **Correction (2026-10-18):** After review, the residual site described below is real but was **mis-scoped in this issue**: > - It does **not** affect DFlash/DFlash2/DSpark. DFlash overrides `load_draft_model` and builds via `load_dflash_model(target_model, self.vllm_config)` -...
Companies and products named
About this problem
Evidence
3 posts from 3 people in 1 place, about 1 a week over 19 days. Mostly on vllm-project/vllm. Tools named alongside: GitHub, PyTorch, NVIDIA.
Frustration Frustration 0 of 3· Seen on GitHub issues
How it was grouped
Posts about vLLM with no known issue, sharing the word "draft". First post Oct 18, 2026, latest Nov 5, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
Other vLLM problems
Running vLLM locally hits memory and speed limits
21 posts from 20 people on GitHub issues, Reddit.
vLLM users keep filing bugs
Top post: "Qwen3.8-flash-next 0% MTP acceptance rate in disaggregated PD serving", one of 34 posts from 31 people on GitHub issues, Reddit.
vLLM upgrades break working setups
17 posts from 17 people on GitHub issues.
vLLM vulnerabilities are hard to triage and patch
9 posts from 9 people on Glyph Ibis.
vLLM runs out of context on real work
5 posts from 5 people on GitHub issues.
vLLM GPU access and drivers are painful
5 posts from 3 people on GitHub issues.
Fine-tuning vLLM is costly and unpredictable
4 posts from 4 people on GitHub issues.
Trouble with vLLM cache
Top post: "AOT compile cache never loads for Qwen3.5 models: Kernel index 1 not found in id_to_kernel", one of 4 posts from 4 people on GitHub issues.
Trouble with vLLM kernel
Top post: "ROCM_AITER_MLA can select an aiter MLA decode kernel that silently returns uncorrelated output (gqa_ratio=16,", one of 3 posts from 3 people on GitHub issues.
Trouble with vLLM size
Top post: "EPLB counts load for the wrong expert when step_interval < window_size", one of 3 posts from 3 people on GitHub issues.
vLLM runs out of memory
3 posts from 3 people on GitHub issues.
vLLM fails to return valid structured output
3 posts from 3 people on GitHub issues.
History