Problemsby GlyphvLLM upgrades break working setups
17 posts from 17 people15d activeFading, +0 this week
Posts per day
The posts behind it
[Docs] Case 4 equations render as literal LaTeX on published Hybrid KV Cache Manager page
## Summary The equations in **Case 4: different `kv_hidden_size`** of the published Hybrid KV Cache Manager documentation display literal `$` delimiters and LaTeX commands instead of rendered math. ## Reproduction Open either published page and scroll to Case 4: -...
[Bug]: int8_per_token_head scale-view fill overwrites Gated DeltaNet state in hybrid models
...to run when the pool is allocated, before any layer writes to it. We run an out-of-tree workaround that overrides `_ensure_scale_caches` to build the `torch.as_strided` views without writing.
[Bug]: Hermes tool parser streaming silently drops all content after a stray <tool_call> marker in prose
### Your current environment The output of python collect_env.py ### 🐛 Describe the bug When serving with `--tool-call-parser hermes` and `stream=true`, all text after a literal ` ` in ordinary prose is silently dropped - e.g. when the model quotes the tag while explaining or...
[Bug]: --attention-backend b12x fails at startup without --block-size (impl checks block size before the platform applies the preferred one)
...pass `block_size=64/128` explicitly, so they never hit this. Workaround: pass `--block-size 64` or `--block-size 128`. I have a fix with a regression test; PR follows. ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues, and asked the...
[Bug]: Mamba2 hybrid models crash once the prompt is longer than block_size (regression from #57329)
...a block boundary inside the prompt, the checkpoint path added in #57329 kicks in, and it doesn't work on CPU: - `mamba_chunk_scan_combined_varlen` calls `_mamba_chunk_scan_combined_fwd_cpu` on CPU. That function accepts `return_intermediate_states` and `last_chunk_indices`, but...
[Bug]: mamba_ssm_cache_dtype="float16" crashes mamba-family models at startup — bare kernel assertion with an empty message (selective_scan_fwd.cu:791)
### Your current environment ### 🐛 Describe the bug ## Description Setting `mamba_ssm_cache_dtype="float16"` on mamba-family models (FalconMamba / Jamba / Mamba) passes `EngineArgs` validation - `float16` is a documented, user-settable value of this option - and then kills...
[Bug][KV Connector] Load-failure report arriving after the owner was preempted: blocks stay cached, `fail` is not applied, stale output can be delivered
### Summary Follow-up to #61768 / #61769 (owner *aborted* before its failure report). The same orphaning happens when the owner is **preempted** between `schedule()` and `update_from_output()`: `_handle_invalid_blocks` scans `running` and the `WAITING_FOR_REMOTE_KVS` entries of...
[Bug]: NIXL failed KV transfers are silently treated as loaded for HMA models (invalid blocks never reported)
...ids land in `_invalid_block_ids`), but for HMA nothing is queued, so the scheduler has no way to detect the failed load. ## Root cause `vllm/distributed/kv_transfer/kv_connector/v1/nixl/base_worker.py`, `get_transfer_results()`: The scheduler only learns about failed loads...
[RFC]:
...to know it runs on Kubernetes. In the rest of this RFC, Snapshot is the orchestrator. It can't make a running server safe to capture. Only vLLM can drain its requests, let go of its GPU-to-GPU links, and decide what stays in GPU memory. After the restore, only vLLM can rebuild...
[Bug]: RankLLMRerank cannot be imported without vllm installed, regardless of reranking backend
...appears to already be known internally. The regression test added in #21789 includes a workaround for the issue. Instead of importing `rank_llm` for real, the test mocks its module tree. The test includes a comment explaining why: > rank_llm is mocked here. Importing the real...
[RFC]: Fast-Track Merging for Model Optimization PRs
...that feature which they themselves have the most expertise with. In such cases it can be difficult to find a dedicated second reviewer who can confidently approve the changes. To resolve this I propose we permit core committers to merge their own PRs, provided they follow the...
[Performance]: FlashAttention backend rebuilds KV-cache views on every call (+~21 µs CPU/layer since #44455)
### Proposal to improve performance Cache the (canonicalized) views on the impl and reuse them while the same `kv_cache` tensor is passed in. On v0.30, a local patch doing this cuts TTFT p50 by 12-16%, and the outputs match an fp32 reference with no numerical change: Notes: -...
[Bug]: FlashMLA sparse decode allocates the split-KV workspace again on SM90 (regression of #53413)
### Your current environment H200 NVL, vLLM nightly `0bfc7a15d` (FlashMLA pin `0eee43b`), torch 2.13.0+cu130. ### 🐛 Describe the bug #53413 (runtime OOM in `flash_mla_cuda.sparse_decode_fwd` on H200, fp8 mixed-batch path) was closed by #53755, which pulled in...
[Feature]: Add lifecycle management hooks for the worker extension class (--worker-extension-cls)
...and teardown, not to an ad-hoc RPC from the client side; * The extension itself has no way to guarantee its initialization runs before any other injected method is used. **Proposal**: define a set of extension lifecycle interfaces (e.g., `init_worker_extension`, and...
[RFC]: Share KV transfer planning primitives across KV connectors
### Problem KV transfer connectors answer the same planning questions in different places: 1. What kind of state is each cache group: attention, SSM, or other? 2. What concrete spec does each layer use, and is it MLA? 3. What byte layout does a registered cache tensor expose? 4....
[Performance]: P2P tier control path stalls behind busy source model steps
## Proposal to improve performance A cold P2P KV pull from a saturated source spent 13.6-14.3 seconds in async lookup/control-plane delay even though the measured data movement took less than 0.5 seconds. The P2P work is CPU-side, but in the current implementation it is only...
[RFC] Qwen3.8-Flash-Next: fp8_e4m3 KV cache on the QSA path — working patch, one machine, looking for corroboration
## TL;DR The QSA path of `Qwen3.8-Flash-Next` refuses any KV dtype other than bf16. Most of the machinery for fp8 is already in vLLM - only the **read** side of the QSA Triton kernels was missing. Wiring it up roughly doubles the KV pool. Measured on **one** GB10 (DGX Spark,...
Companies and products named
About this problem
Evidence
17 posts from 17 people in 2 places, about 2 a week over 60 days. Mostly on vllm-project/vllm, run-llama/llama_index. Tools named alongside: NVIDIA, GitHub, PyPI, LlamaIndex.
Frustration Frustration 1 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "breaking-changes" rule about vLLM. First post Aug 30, 2026, latest Oct 28, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
GitHub upgrades break working setups
87 posts from 75 people on GitHub issues, Hacker News, Stack Overflow.
Upgrades and new releases break working setups
76 posts from 62 people on GitHub issues, Reddit, App Store reviews.
Claude Code upgrades break working setups
67 posts from 60 people on GitHub issues, Hacker News, Reddit.
Codex upgrades break working setups
65 posts from 58 people on GitHub issues, Reddit.
DuckDB upgrades break working setups
37 posts from 28 people on GitHub issues.
Claude upgrades break working setups
35 posts from 34 people on GitHub issues, Reddit, Hacker News.
ClickHouse upgrades break working setups
31 posts from 20 people on GitHub issues.
Anthropic upgrades break working setups
27 posts from 22 people on GitHub issues.
PyTorch upgrades break working setups
26 posts from 21 people on GitHub issues, Reddit.
ChatGPT upgrades break working setups
26 posts from 26 people on GitHub issues, Community forums, App Store reviews.
OpenAI upgrades break working setups
25 posts from 25 people on GitHub issues, Reddit, Community forums.
Open WebUI upgrades break working setups
25 posts from 2 people on GitHub issues.
Other vLLM problems
Running vLLM locally hits memory and speed limits
21 posts from 20 people on GitHub issues, Reddit.
Trouble with vLLM draft
Top post: "Draft model construction resets data_parallel_index to 0 (load_draft_model / create_draft_parallel_config),", one of 3 posts from 3 people on GitHub issues.
vLLM users keep filing bugs
Top post: "Qwen3.8-flash-next 0% MTP acceptance rate in disaggregated PD serving", one of 34 posts from 31 people on GitHub issues, Reddit.
vLLM vulnerabilities are hard to triage and patch
9 posts from 9 people on Glyph Ibis.
vLLM runs out of context on real work
5 posts from 5 people on GitHub issues.
vLLM GPU access and drivers are painful
5 posts from 3 people on GitHub issues.
Fine-tuning vLLM is costly and unpredictable
4 posts from 4 people on GitHub issues.
Trouble with vLLM cache
Top post: "AOT compile cache never loads for Qwen3.5 models: Kernel index 1 not found in id_to_kernel", one of 4 posts from 4 people on GitHub issues.
Trouble with vLLM kernel
Top post: "ROCM_AITER_MLA can select an aiter MLA decode kernel that silently returns uncorrelated output (gqa_ratio=16,", one of 3 posts from 3 people on GitHub issues.
Trouble with vLLM size
Top post: "EPLB counts load for the wrong expert when step_interval < window_size", one of 3 posts from 3 people on GitHub issues.
vLLM runs out of memory
3 posts from 3 people on GitHub issues.
vLLM fails to return valid structured output
3 posts from 3 people on GitHub issues.
History