Data and AI struggles

2,114 problems · 377 companies

LLM apps

llama.cpp runs out of context on real work

4 posts from 4 people4d activeFading, +0 this week

Posts per day

Posts per day4 posts, Sep 1 to Nov 21

The posts behind it

GitHub issuesggml-org/llama.cpp · oinani0721Oct 7

qwen4exp: Metal decode depth-decay on M5 Max — free-form 26.9 t/s at 175K vs 73.9 t/s with n-gram spec vs 53.9 t/s MLX

### Name and Version version: 0.6.0-dev (build 11451, commit 2207c8e57) built with AppleClang 21.0.0.21000101 for Darwin arm64 (one number below is from b11429, noted where applicable) ### Operating systems Mac, macOS 26.5, Apple M5 Max, 128 GB unified memory ### Which llama.cpp...

1 reactions
GitHub issuesggml-org/llama.cpp · ariera-jOct 6

Vulkan: FA quantized-KV dequant scratch can exceed maxBufferSize at long context

...tokens. Any prompt that grows the KV cache beyond 1,048,576 entries should reproduce the issue with this model/configuration. The failure occurs during prompt evaluation, so generation settings are not relevant. With `-ub 512`, prompt processing succeeds through 1,048,576...

GitHub issuesggml-org/llama.cpp · simon3zSep 29

Feature Request: automatic slot save/restore by client session id

...breaks across server restarts unless the client also manages files. And when the client fails to save in time, the conversation's KV state is discarded and the whole prompt must be prefilled from scratch again - expensive on long contexts, where a single prefill can cost tens of...

GitHub issuesggml-org/llama.cpp · ky095nSep 1

Misc. bug: /slots restore yields no KV reuse on hybrid/recurrent and SWA models (context checkpoints are not persisted)

### Name and Version `b94041a` (2026-08-16); behaviour still present in `36b1015` (2026-09-01) - the code path is unchanged and no commit in between touches slot-save checkpoints. ### Operating systems Windows ### GGML backends SYCL ### Hardware 2x Intel Arc Pro B70, oneAPI...

8 reactions

Companies and products named

4 posts
2 posts
1 post
About this problem

Evidence

4 posts from 4 people in 1 place, about 1 a week over 37 days. Mostly on ggml-org/llama.cpp. Tools named alongside: GitHub, NVIDIA.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "context-window" rule about llama.cpp. First post Sep 1, 2026, latest Oct 7, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Context windows run out on real workloads

20 posts from 20 people on Hacker News, Reddit, GitHub issues.

Claude Code runs out of context on real work

13 posts from 13 people on GitHub issues, Reddit, Hacker News.

Claude runs out of context on real work

11 posts from 9 people on GitHub issues, Reddit, Hacker News.

OpenAI runs out of context on real work

9 posts from 9 people on GitHub issues, Community forums.

ChatGPT runs out of context on real work

8 posts from 8 people on GitHub issues, Reddit, Hacker News.

vLLM runs out of context on real work

7 posts from 7 people on GitHub issues.

Ollama runs out of context on real work

7 posts from 7 people on GitHub issues, Reddit.

Codex runs out of context on real work

6 posts from 6 people on GitHub issues, Reddit, Hacker News.

Anthropic runs out of context on real work

5 posts from 5 people on GitHub issues.

Cursor runs out of context on real work

5 posts from 5 people on Community forums, Reddit, Hacker News.

Mistral runs out of context on real work

5 posts from 5 people on GitHub issues.

Qwen runs out of context on real work

3 posts from 3 people on Reddit, GitHub issues, Hacker News.

Other llama.cpp problems

Trouble with llama.cpp misc

Top post: "Misc. bug: Severe drop in prompt processing speed after b10780 on Vulkan, RDNA3", one of 28 posts from 25 people on GitHub issues.

llama.cpp users keep filing bugs

Top post: "llama-ui : unable to open reasoning level selection menu on desktop", one of 16 posts from 16 people on GitHub issues, Reddit.

llama.cpp upgrades break working setups

7 posts from 7 people on GitHub issues.

Fine-tuning llama.cpp is costly and unpredictable

6 posts from 6 people on GitHub issues, Hacker News.

Trouble with llama.cpp compile

Top post: "Compile bug: rocm-6.3.4, no matching function for call to group_mask_warp_inplace", one of 6 posts from 5 people on GitHub issues.

llama.cpp runs out of memory

4 posts from 4 people on GitHub issues.

llama.cpp lock-in makes switching costly

3 posts from 3 people on GitHub issues.

Experiments on llama.cpp are hard to reproduce and track

3 posts from 3 people on GitHub issues.

llama.cpp tables suffer from small files and skew

2 posts from 2 people on GitHub issues. Unverified.

Trouble with llama.cpp flash

Top post: "Feature Request: Support GLM5.3 (flash)", one of 2 posts from 2 people on GitHub issues, Reddit. Unverified.

llama.cpp fails to return valid structured output

2 posts from 2 people on GitHub issues. Unverified.

Trouble with llama.cpp builds

Top post: "Feature Request: update ROCm release builds 10.0.0 -> 10.1.0", one of 2 posts from 2 people on GitHub issues. Unverified.

History

Momentum went from steady to fading.

Oct 14, 2026

Corroborated: 3 or more posts from 2 or more people or places.

Oct 12, 2026

Momentum went from fading to steady.

Oct 12, 2026

Momentum went from steady to fading.

Oct 6, 2026

First seen with 2 posts.

Oct 5, 2026