Fine-tuning llama.cpp is costly and unpredictable
6 posts from 6 people6d activeFading, +0 this week
Posts per day
The posts behind it
Feature Request: restrict qwen4exp QSA to a sequence's own cells with a unified KV cache
...and let it finish, then time a few short replies and compare with a fresh server. Both workarounds hurt: - `--cache-idle-slots` moves the idle conversation out, but then it re-reads its whole prompt when it comes back. - `--no-kv-unified` caps each slot at n_ctx / n_parallel....
Misc. bug: server slot discards the restored prompt if the LoRA is different for another call/model
### Name and Version $ ./llama-server --version version: 0.9.0-dev (build 12211, commit 7d7f02875) built with AppleClang 21.0.0.21000334 for Darwin arm64 ### Operating systems Mac ### Which llama.cpp modules do you know to be affected? llama-server ### Command line ### Problem...
Feature Request: Allow `--moe-cache-mib` to be comma separated per-device.
...takes a single value that is applied across all devices based on `--ts`. This makes it difficult to fine-tune MoE expert cache allocation when using multi-GPU setup (e.g. two GPUs of different generation/speed). In my case I have two GPUs with different generation and speed. I...
Misc. bug: ggml-opt gradient accumulators are never cleared between optimizer steps (llama_opt_epoch / llama-finetune)
### What In the graph-per-step training path (`ggml_opt_prepare_alloc` + `ggml_opt_alloc` without static graphs, which is what `llama_context::opt_epoch_iter`, and so `llama_opt_epoch` / `llama-finetune`, uses), the gradient accumulators are never zeroed between optimizer steps....
Feature Request: multi_logit_bias
...to logit_bias that allows me to blacklist phrases ### Motivation it's getting more difficult to make good anti-slop finetunes, so i figured why not do it here instead? generate a bunch of text, look for overused bigrams/trigrams (what i usually do when making a dataset for this)...
Companies and products named
About this problem
Evidence
6 posts from 6 people in 2 places, about 1 a week over 53 days. Mostly on ggml-org/llama.cpp, Hacker News comments. Tools named alongside: GitHub, NVIDIA.
Frustration Frustration 1 of 3· Seen on GitHub issues, Hacker News
How it was grouped
Posts that state a pain and match the "fine-tuning" rule about llama.cpp. First post Sep 22, 2026, latest Nov 13, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Fine-tuning is costly and results are unpredictable
17 posts from 17 people on Hacker News, Reddit, GitHub issues.
Fine-tuning Hugging Face is costly and unpredictable
12 posts from 12 people on GitHub issues, Reddit, Community forums.
Fine-tuning vLLM is costly and unpredictable
7 posts from 7 people on GitHub issues.
Fine-tuning GitHub is costly and unpredictable
6 posts from 6 people on Reddit, GitHub issues.
Fine-tuning PyTorch is costly and unpredictable
5 posts from 5 people on GitHub issues, Reddit.
Fine-tuning Qwen is costly and unpredictable
4 posts from 4 people on GitHub issues, Hacker News, Reddit.
Fine-tuning Claude is costly and unpredictable
4 posts from 4 people on Reddit, GitHub issues.
Fine-tuning Claude Code is costly and unpredictable
3 posts from 3 people on Reddit, GitHub issues.
Fine-tuning NVIDIA is costly and unpredictable
3 posts from 3 people on GitHub issues.
Fine-tuning OpenAI is costly and unpredictable
3 posts from 3 people on GitHub issues, Hacker News.
Fine-tuning Gemini is costly and unpredictable
2 posts from 2 people on Reddit, GitHub issues. Unverified.
Fine-tuning ChatGPT is costly and unpredictable
2 posts from 2 people on Reddit, Hacker News. Unverified.
Other llama.cpp problems
Trouble with llama.cpp misc
Top post: "Misc. bug: Severe drop in prompt processing speed after b10780 on Vulkan, RDNA3", one of 28 posts from 25 people on GitHub issues.
llama.cpp users keep filing bugs
Top post: "llama-ui : unable to open reasoning level selection menu on desktop", one of 16 posts from 16 people on GitHub issues, Reddit.
llama.cpp upgrades break working setups
7 posts from 7 people on GitHub issues.
Trouble with llama.cpp compile
Top post: "Compile bug: rocm-6.3.4, no matching function for call to group_mask_warp_inplace", one of 6 posts from 5 people on GitHub issues.
llama.cpp runs out of memory
4 posts from 4 people on GitHub issues.
llama.cpp runs out of context on real work
4 posts from 4 people on GitHub issues.
llama.cpp lock-in makes switching costly
3 posts from 3 people on GitHub issues.
Experiments on llama.cpp are hard to reproduce and track
3 posts from 3 people on GitHub issues.
llama.cpp tables suffer from small files and skew
2 posts from 2 people on GitHub issues. Unverified.
Trouble with llama.cpp flash
Top post: "Feature Request: Support GLM5.3 (flash)", one of 2 posts from 2 people on GitHub issues, Reddit. Unverified.
llama.cpp fails to return valid structured output
2 posts from 2 people on GitHub issues. Unverified.
Trouble with llama.cpp builds
Top post: "Feature Request: update ROCm release builds 10.0.0 -> 10.1.0", one of 2 posts from 2 people on GitHub issues. Unverified.
History