Data and AI struggles

2,114 problems · 377 companies

LLM apps

vLLM runs out of context on real work

7 posts from 7 people7d activeSteady, +1 this week

Posts per day

Posts per day7 posts, Aug 28 to Nov 21

The posts behind it

GitHub issuesvllm-project/vllm · waynehacking8Nov 16

[Bug]: flashinfer_b12x NVFP4 MoE returns garbage on SM120 since #64692 (the DeepGEMM warmup overwrites its FC2 input scale)

### Your current environment The output of python collect_env.py One RTX PRO 6000 Blackwell Max-Q (SM120) under WSL2, vLLM main at `543cb51c07` (11-15) installed from source with the precompiled kernels, torch 2.14.0+cu132, FlashInfer 0.7.4, nvidia-cutlass-dsl 4.8.0. No...

1 reactions1 replies
GitHub issueshuggingface/transformers · snn0703-coderNov 10

[RFC] Flash-Fused Entropy-Invariant Differential Attention (EIDA) in O(N) Memory

### Feature request Markdown Hello team! I've published a production-ready implementation and benchmark suite solving long-context entropic dilution and INT4 KV-cache noise using an O(N) memory distributive FlashAttention decomposition. You can inspect the full source code,...

1 replies
GitHub issuesvllm-project/vllm · ukanwatOct 28

[RFC]: Serve requests on top of an externally computed KV cache (PrecomputedKVConnector)

### Motivation More and more methods produce a KV cache *outside* the serving engine and then want to decode from it: - offline prefix caches: a long system prompt or document is encoded once (for example with Hugging Face Transformers) and reused across processes and...

GitHub issuesvllm-project/vllm · zksainxOct 27

[Performance]: FA3 runs long decodes without split-KV whenever a prefill chunk shares the batch

### Proposal to improve performance For FA3 batches that are not CUDA-graph eligible and mix decodes with prefills, run the decodes (already reordered to the front) as their own `max_seqlen_q=1` varlen call and the rest as a second call. Each call gets its own AOT scheduler...

GitHub issuesvllm-project/vllm · wuyeliangOct 16

[Bug]: vLLM EngineCore crash in native KV offloading for hybrid Attention/Mamba model

...model endpoint to fail and requires engine recovery/restart. ## Potential impact of the workaround - Prevents the observed EngineCore crash for this condition. - May reduce CPU KV offload or external prefix-cache hit rate for affected

GitHub issuesvllm-project/vllm · JartXOct 2

[RFC]: Octave KV, a native 3-bit KV cache for AMD GPUs

PR: https://github.com/vllm-project/vllm/pull/59774 ### Proposal to improve performance ## Where this comes from We serve Qwen3.8-27B (GPTQ W4A16) on boxes with four RX 7900 XTX, 96 GB in total, with tensor parallel 4 and MTP. Our users send long prompts: whole repositories,...

1 reactions2 replies
GitHub issuesvllm-project/vllm · shing100Aug 28

[Bug]: GLM-5.3-Flash — ModelOpt NVFP4 checkpoints emit invalid UTF-8 byte tokens on SM120, while a compressed-tensors NVFP4 of the same model is clean

### Your current environment Environment ### 🐛 Describe the bug Two ModelOpt-produced NVFP4 conversions of GLM-5.3-Flash emit **invalid UTF-8 byte-token sequences** during generation on SM120. A compressed-tensors NVFP4 conversion of the *same* model, served from the same image...

1 reactions3 replies

Companies and products named

7 posts
6 posts
2 posts
2 posts
About this problem

Evidence

7 posts from 7 people in 2 places, about 1 a week over 81 days. Mostly on vllm-project/vllm, huggingface/transformers. Tools named alongside: NVIDIA, GitHub, PyTorch, Hugging Face.

Frustration Frustration 2 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "context-window" rule about vLLM. First post Aug 28, 2026, latest Nov 16, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Context windows run out on real workloads

20 posts from 20 people on Hacker News, Reddit, GitHub issues.

Claude Code runs out of context on real work

13 posts from 13 people on GitHub issues, Reddit, Hacker News.

Claude runs out of context on real work

11 posts from 9 people on GitHub issues, Reddit, Hacker News.

OpenAI runs out of context on real work

9 posts from 9 people on GitHub issues, Community forums.

ChatGPT runs out of context on real work

8 posts from 8 people on GitHub issues, Reddit, Hacker News.

Ollama runs out of context on real work

7 posts from 7 people on GitHub issues, Reddit.

Codex runs out of context on real work

6 posts from 6 people on GitHub issues, Reddit, Hacker News.

Anthropic runs out of context on real work

5 posts from 5 people on GitHub issues.

Cursor runs out of context on real work

5 posts from 5 people on Community forums, Reddit, Hacker News.

Mistral runs out of context on real work

5 posts from 5 people on GitHub issues.

llama.cpp runs out of context on real work

4 posts from 4 people on GitHub issues.

Qwen runs out of context on real work

3 posts from 3 people on Reddit, GitHub issues, Hacker News.

Other vLLM problems

vLLM users keep filing bugs

Top post: "Qwen3.8-flash-next 0% MTP acceptance rate in disaggregated PD serving", one of 37 posts from 34 people on GitHub issues, Reddit.

Running vLLM locally hits memory and speed limits

27 posts from 26 people on GitHub issues, Reddit.

vLLM upgrades break working setups

21 posts from 21 people on GitHub issues.

vLLM vulnerabilities are hard to triage and patch

9 posts from 9 people on Glyph Ibis.

Trouble with vLLM cache

Top post: "AOT compile cache never loads for Qwen3.5 models: Kernel index 1 not found in id_to_kernel", one of 7 posts from 6 people on GitHub issues.

Fine-tuning vLLM is costly and unpredictable

7 posts from 7 people on GitHub issues.

vLLM GPU access and drivers are painful

5 posts from 3 people on GitHub issues.

vLLM fails to return valid structured output

5 posts from 5 people on GitHub issues.

Trouble with vLLM serve

Top post: "gpt-oss can't be served through `vllm serve` with any tool/reasoning-parser flags", one of 4 posts from 4 people on GitHub issues.

Trouble with vLLM performance

Top post: "Waiting requests re-walk their whole prefix-cache hit on every scheduling step", one of 3 posts from 3 people on GitHub issues, Reddit.

vLLM lock-in makes switching costly

3 posts from 3 people on GitHub issues.

RAG on vLLM returns irrelevant or missing context

3 posts from 3 people on GitHub issues.

History

Momentum went from fading to steady.

Nov 16, 2026

Momentum went from steady to fading.

Nov 4, 2026

Momentum went from rising to steady.

Nov 3, 2026

Momentum went from steady to rising.

Oct 30, 2026

Momentum went from fading to steady.

Oct 29, 2026

Momentum went from steady to fading.

Oct 23, 2026

Corroborated: 3 or more posts from 2 or more people or places.

Oct 20, 2026

Momentum went from fading to steady.

Oct 20, 2026

Momentum went from steady to fading.

Oct 9, 2026

First seen with 2 posts.

Oct 4, 2026