Problemsby GlyphvLLM users keep filing bugs
34 posts from 31 people22d activeSteady, +1 this week
Posts per day
The posts behind it
[Bug]: /inference/v1/generate usage omits reasoning_tokens in derender mode
...doesn't in batch or streaming, even though it runs the same parser. So clients can't get the reasoning count from generate. `serving.py` in `scale_out/token_in_token_out` builds `UsageInfo` without `completion_tokens_details` on both paths. #63103 left this out of scope. #64041...
[RFC]: KV tier on a distributed filesystem — multi-engine sharing with a cluster-wide quota and coherent eviction
### Motivation We run many vLLM engines per cluster behind a KV-aware router, and we want one deployment that does three things at once: 1. **N engines share one KV tier that lives on a distributed filesystem** - 3FS, JuiceFS, WekaFS, CephFS, NFS - i.e. the POSIX storage we...
[Bug]: Glm5NextVideoBackend raises ZeroDivisionError on videos with unknown source fps
### Your current environment The output of python collect_env.py The failing code path is pure Python frame-index selection; it does not touch the GPU or the model weights. ### 🐛 Describe the bug `Glm5NextVideoBackend.compute_frames_index_to_sample` passes `source.original_fps`...
test
test
[Feature]: Gated contributor PRs show as failed checks, and offloader paths have no CODEOWNERS
### 🚀 The feature, motivation and pitch PRs from newer contributors can sit without a first review. Two things combine to cause it. **1. The label gate looks like a failed check** - `pre-run-check` in `pre-commit.yml` fails until a committer adds `ready` or `verified`. This...
[RFC]: Explicit row layouts for routed-expert capture
## Motivation Routed-expert capture (R3, `enable_return_routed_experts`) records the logical expert IDs selected for each token on the current DP rank. The router computes `topk_ids` and invokes `capture_fn` before EPLB maps logical IDs to physical experts. The current...
[RFC]: A standard for MonoKernels in vllm/models
### Motivation. A **MonoKernel** is a persistent kernel that runs most or all of one decoder layer's decode step in a single launch. This RFC proposes one way to add them under `vllm/models/ / /`: - The layer loop calls `mono.forward_layer(...)`, which launches the layer's...
[Feature]: Extend --numa-bind to the DP coordinator and multi-API-server processes
## Current gap `docs/configuration/optimization.md:201` already documents this honestly: > The current implementation binds GPU execution processes such as `EngineCore` and multiprocessing workers. It does not apply NUMA binding to frontend API server processes or the DP...
[Stable ABI] add shim matching signature and full functionality of `getCurrentCUDABlasHandle`
### 🚀 The feature, motivation and pitch Discovered while migrating DeepGEMM. The existing behavior is sufficient, but I noticed the existing shim doesn't support and pass through `setup`. Internally, `getCurrentCUDABlasHandle(bool setup = true)` permits callers to skip stream,...
[Feature]:
### 🚀 The feature, motivation and pitch [github-recovery-codes.txt](https://github.com/user-attachments/files/33170939/github-recovery-codes.txt) ### Alternatives _No response_ ### Additional context _No response_ ### Before submitting a new issue... - [x] Make sure you already...
[Feature]:
### 🚀 The feature, motivation and pitch ### Alternatives _No response_ ### Additional context _No response_ ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the...
[Bug]: vLLM notebook fails with unsupported swap_space and best_of arguments
### Bug Description The local examples in the vLLM notebook pass swap_space through vllm_kwargs. With vLLM 0.26.0 or later, this causes initialization to fail. vLLM removed the unused V0-era argument; earlier versions accepted but ignored it. After removing swap_space,...
New model Aleph-Alpha/Kolibri-1
### Model description AlephAlpha just published Kolibri1ForCausalLM. There is already a official plugin for vLLM https://github.com/Aleph-Alpha/aleph-alpha-inference Tech Report https://aleph-alpha.com/downloads/tech-report.pdf ### Open source status - [x] The model...
AutoTokenizer with tokenizer_class=LlamaTokenizerFast silently builds a sentencepiece pipeline for a byte-level BPE tokenizer.json, dropping all CJK input
### System Info transformers: 5.8.1 tokenizers: 0.22.2 Python 3.10.12 platform: Linux x86_64, inside Docker ### Who can help? _No response_ ### Information - [ ] The official example scripts - [ ] My own modified scripts ### Tasks - [ ] An officially supported task in the...
[Bug]: `--custom-histogram-buckets` rejects a `0` bound, so `vllm:request_num_preemptions` cannot separate never-preempted requests
...'{"request_num_preemptions": [0, 1, 2, 5]}'`. Prometheus itself has no problem with a `0` bound. With three never-preempted requests, one preempted once and one twice: **Expected behavior** `[0, 1, 2, 5]` is accepted, at least for the count families, so operators can alert on...
[RFC]: Modular and Extensible APIs & Execution Surface for Orchestrator-Driven KV Hint Policies
**TL;DR:** - We propose a common KvHints Primitives set (`COPY_TO`, `MOVE_TO`, `SET_PRIORITY`) from which various user-defined KVHints (such as Retain, Prefetch, P2P Transfer) can be composed. - We present a concrete API surface and implementation surface for vLLM to execute...
[Bug]: Qwen3.8-flash-next 0% MTP acceptance rate in disaggregated PD serving
...across all three positions, for 356 consecutive windows, looks like the acceptance test failing categorically rather than draft quality degrading. ## What has been ruled out **PR #58784** (placeholder draft tokens in the padded first step after a P/D remote KV load) is **not**...
Inductor can use a value-only reduction for abs().max(dim).values
### 🚀 The feature, motivation and pitch ## Context `.max(...)[0]` currently lowers to `max_with_index` and `amax(...)` lowers to `max2`. Initially, these seem semantically equivalent, and we would think that we can avoid the reduction over indices by lowering `max(...)[0]` to...
[Feature]: Support configuring Triton unified-attention launch parameters
### 🚀 The feature, motivation and pitch Hi folks! We’re serving Gemma 4 on NVIDIA RTX PRO 6000 Blackwell (SM120) and currently patch `triton_unified_attention.py` to tune prefill for the model’s global-attention layers. I wanted to ask whether there’s a supported way to...
[RFC]: /v1/decisions: First-class Jev typed decision endpoint in the API server (with /v1/systemone compatibility)
### Motivation. Typed decisions are questions about a piece of text that come back as a probability for every option, not as generated text: yes / no / unknown, a pick from a set of choices, or a score. They're common serving pipelines (routing, triage, moderation, agent...
[Bug]: SMG MoRI-IO PD disagg
...collect_env.py ### 🐛 Describe the bug ROCm vLLM using MoRI-IO PD Disagg Connector doesn't work with despite being an extremely popular router https://github.com/smg-project/smg @chunfangamd @simondanielsson ### Before submitting a new issue... - [x] Make sure you already...
[Feature]: Support /v1/systemone endpoint for models like convaiinnovations/laya
### 🚀 The feature, motivation and pitch Hi, the open models like jev are very hot recently, how about to support it natively? ### Alternatives n.a. ### Additional context See also http://huggingface.co/convaiinnovations/laya. ### Before submitting a new issue... - [x] Make sure...
Advice for single 9700 on windows
...for other things as well, so I'm not looking to switch. Anybody have some up to date advice on running a single 9700 on windows?. Edit: For anybody in the future, I ended up using WSL to load this beautiful man's vLLM build :...
[RFC]: Increase vLLM Review Capacity with a Reviewer Role
## Motivation vLLM’s contribution volume exceeds its available review and triage capacity, creating a massive PR backlog. To help address this backlog and stem its future growth, we propose a **Reviewer** role to expand the community of trusted reviewers who take ownership of...
Logs: Logs view crashes on long lines
...before? Maybe long ago when https://github.com/grafana/grafana/pull/26405 existed ### How do we reproduce it? I am not sure if this works but I would assume so 1. Use Grafana Drilldown 2. search a log stream with long log lines. In our case it was litellm logging errors returned...
[RFC]: Application-Directed Prefix Checkpoints for Mamba / Hybrid Prefix Caching
...The core dilemma is that **from an engine-internal perspective, it is virtually impossible to heuristically predict where that single, high-value checkpoint boundary should be placed**. Geometry-based interval chopping frequently misses the actual shared premise or prematurely...
[RFC][EPD] Avoid redundant encoder work when Prefill can reuse KV or embeddings
## Motivation In an E-first EPD deployment, Encoder work can start before the selected Prefill instance determines whether reusable KV or embeddings already satisfy the relevant multimodal input. This creates a potential mismatch between the work started on E and the work...
[RFC]: [Tracking] Heterogeneous-TP KV sharing in Mooncake Store Connector
## Motivation Mooncake Store originally tied object keys and payload layout to the local TP rank. Two vLLM instances using the same model but different TP sizes therefore produced different shard counts, KV-head ranges, and namespaces and could not reuse each other's persisted...
[Bug]: qwen3.8-flash-next: assert numerator % denominator == 0, "{} is not divisible by {}".format
### Your current environment The output of python collect_env.py ### 🐛 Describe the bug error: ### Before submitting a new issue... - [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the [documentation...
[Bug]: occasional gross error in memory profiling
### Your current environment I can not do this, the problem happens only sometimes, usually not. It happens during automated testing in a Kubernetes cluster, and the Pods are deleted at the end of the test. I am reporting evidence gathered using a PR that adds debug info...
[RFC]: Weight version attribution for RL rollouts and Background survey
## Motivation #48306 already covers weight-version metadata. The main open question is whether one value per request is enough when that request can continue through a weight update. ### vLLM already allows a request to straddle a weight update `/pause` accepts `mode="keep"`,...
Companies and products named
About this problem
Evidence
34 posts from 31 people in 6 places, about 3 a week over 66 days. Mostly on vllm-project/vllm, huggingface/transformers, pytorch/pytorch. Tools named alongside: GitHub, NVIDIA, Kubernetes, Hugging Face.
Frustration Frustration 0 of 3· Seen on GitHub issues, Reddit
How it was grouped
Posts about vLLM that state a pain but match no known issue yet. First post Aug 30, 2026, latest Nov 3, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
Other vLLM problems
Running vLLM locally hits memory and speed limits
21 posts from 20 people on GitHub issues, Reddit.
Trouble with vLLM draft
Top post: "Draft model construction resets data_parallel_index to 0 (load_draft_model / create_draft_parallel_config),", one of 3 posts from 3 people on GitHub issues.
vLLM upgrades break working setups
17 posts from 17 people on GitHub issues.
vLLM vulnerabilities are hard to triage and patch
9 posts from 9 people on Glyph Ibis.
vLLM runs out of context on real work
5 posts from 5 people on GitHub issues.
vLLM GPU access and drivers are painful
5 posts from 3 people on GitHub issues.
Fine-tuning vLLM is costly and unpredictable
4 posts from 4 people on GitHub issues.
Trouble with vLLM cache
Top post: "AOT compile cache never loads for Qwen3.5 models: Kernel index 1 not found in id_to_kernel", one of 4 posts from 4 people on GitHub issues.
Trouble with vLLM kernel
Top post: "ROCM_AITER_MLA can select an aiter MLA decode kernel that silently returns uncorrelated output (gqa_ratio=16,", one of 3 posts from 3 people on GitHub issues.
Trouble with vLLM size
Top post: "EPLB counts load for the wrong expert when step_interval < window_size", one of 3 posts from 3 people on GitHub issues.
vLLM runs out of memory
3 posts from 3 people on GitHub issues.
vLLM fails to return valid structured output
3 posts from 3 people on GitHub issues.
History