Data and AI struggles

2,114 problems · 377 companies

MLOps

Fine-tuning llama.cpp is costly and unpredictable

6 posts from 6 people6d activeFading, +0 this week

Posts per day

Posts per day6 posts, Sep 22 to Nov 21

The posts behind it

GitHub issuesggml-org/llama.cpp · kevinpark1217Nov 13

Feature Request: restrict qwen4exp QSA to a sequence's own cells with a unified KV cache

...and let it finish, then time a few short replies and compare with a fresh server. Both workarounds hurt: - `--cache-idle-slots` moves the idle conversation out, but then it re-reads its whole prompt when it comes back. - `--no-kv-unified` caps each slot at n_ctx / n_parallel....

GitHub issuesggml-org/llama.cpp · beaudamoreNov 12

Misc. bug: server slot discards the restored prompt if the LoRA is different for another call/model

### Name and Version $ ./llama-server --version version: 0.9.0-dev (build 12211, commit 7d7f02875) built with AppleClang 21.0.0.21000334 for Darwin arm64 ### Operating systems Mac ### Which llama.cpp modules do you know to be affected? llama-server ### Command line ### Problem...

1 replies
GitHub issuesggml-org/llama.cpp · reza-ebadiOct 8

Feature Request: Allow `--moe-cache-mib` to be comma separated per-device.

...takes a single value that is applied across all devices based on `--ts`. This makes it difficult to fine-tune MoE expert cache allocation when using multi-GPU setup (e.g. two GPUs of different generation/speed). In my case I have two GPUs with different generation and speed. I...

2 reactions1 replies
GitHub issuesggml-org/llama.cpp · joelteplyOct 7

Misc. bug: ggml-opt gradient accumulators are never cleared between optimizer steps (llama_opt_epoch / llama-finetune)

### What In the graph-per-step training path (`ggml_opt_prepare_alloc` + `ggml_opt_alloc` without static graphs, which is what `llama_context::opt_epoch_iter`, and so `llama_opt_epoch` / `llama-finetune`, uses), the gradient accumulators are never zeroed between optimizer steps....

GitHub issuesggml-org/llama.cpp · electroglyphOct 3

Feature Request: multi_logit_bias

...to logit_bias that allows me to blacklist phrases ### Motivation it's getting more difficult to make good anti-slop finetunes, so i figured why not do it here instead? generate a bunch of text, look for overused bigrams/trigrams (what i usually do when making a dataset for this)...

4 reactions4 replies

Companies and products named

6 posts
5 posts
1 post
About this problem

Evidence

6 posts from 6 people in 2 places, about 1 a week over 53 days. Mostly on ggml-org/llama.cpp, Hacker News comments. Tools named alongside: GitHub, NVIDIA.

Frustration Frustration 1 of 3· Seen on GitHub issues, Hacker News

How it was grouped

Posts that state a pain and match the "fine-tuning" rule about llama.cpp. First post Sep 22, 2026, latest Nov 13, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Fine-tuning is costly and results are unpredictable

17 posts from 17 people on Hacker News, Reddit, GitHub issues.

Fine-tuning Hugging Face is costly and unpredictable

12 posts from 12 people on GitHub issues, Reddit, Community forums.

Fine-tuning vLLM is costly and unpredictable

7 posts from 7 people on GitHub issues.

Fine-tuning GitHub is costly and unpredictable

6 posts from 6 people on Reddit, GitHub issues.

Fine-tuning PyTorch is costly and unpredictable

5 posts from 5 people on GitHub issues, Reddit.

Fine-tuning Qwen is costly and unpredictable

4 posts from 4 people on GitHub issues, Hacker News, Reddit.

Fine-tuning Claude is costly and unpredictable

4 posts from 4 people on Reddit, GitHub issues.

Fine-tuning Claude Code is costly and unpredictable

3 posts from 3 people on Reddit, GitHub issues.

Fine-tuning NVIDIA is costly and unpredictable

3 posts from 3 people on GitHub issues.

Fine-tuning OpenAI is costly and unpredictable

3 posts from 3 people on GitHub issues, Hacker News.

Fine-tuning Gemini is costly and unpredictable

2 posts from 2 people on Reddit, GitHub issues. Unverified.

Fine-tuning ChatGPT is costly and unpredictable

2 posts from 2 people on Reddit, Hacker News. Unverified.

Other llama.cpp problems

Trouble with llama.cpp misc

Top post: "Misc. bug: Severe drop in prompt processing speed after b10780 on Vulkan, RDNA3", one of 28 posts from 25 people on GitHub issues.

llama.cpp users keep filing bugs

Top post: "llama-ui : unable to open reasoning level selection menu on desktop", one of 16 posts from 16 people on GitHub issues, Reddit.

llama.cpp upgrades break working setups

7 posts from 7 people on GitHub issues.

Trouble with llama.cpp compile

Top post: "Compile bug: rocm-6.3.4, no matching function for call to group_mask_warp_inplace", one of 6 posts from 5 people on GitHub issues.

llama.cpp runs out of memory

4 posts from 4 people on GitHub issues.

llama.cpp runs out of context on real work

4 posts from 4 people on GitHub issues.

llama.cpp lock-in makes switching costly

3 posts from 3 people on GitHub issues.

Experiments on llama.cpp are hard to reproduce and track

3 posts from 3 people on GitHub issues.

llama.cpp tables suffer from small files and skew

2 posts from 2 people on GitHub issues. Unverified.

Trouble with llama.cpp flash

Top post: "Feature Request: Support GLM5.3 (flash)", one of 2 posts from 2 people on GitHub issues, Reddit. Unverified.

llama.cpp fails to return valid structured output

2 posts from 2 people on GitHub issues. Unverified.

Trouble with llama.cpp builds

Top post: "Feature Request: update ROCm release builds 10.0.0 -> 10.1.0", one of 2 posts from 2 people on GitHub issues. Unverified.

History

Momentum went from steady to fading.

Nov 20, 2026

Momentum went from rising to steady.

Nov 19, 2026

Momentum went from fading to rising.

Nov 17, 2026

Momentum went from steady to fading.

Oct 15, 2026

Corroborated: 3 or more posts from 2 or more people or places.

Oct 12, 2026

Momentum went from fading to steady.

Oct 12, 2026

Momentum went from steady to fading.

Oct 10, 2026

First seen with 2 posts.

Oct 5, 2026