Problemsby GlyphRunning DeepSeek locally hits memory and speed limits
8 posts from 8 people8d activeSteady, +2 this week
Posts per day
The posts behind it
...input tokens and produce an answer to a very simple request (translation). That's pretty unusable, to be honest. Is overthinking a trait of DeepSeek, or it's a low quantization issue? For comparison, the whole thing takes 1.8 sec on my RTX 6000 rig with GLM5.3-Flash, and 12.5...
...are not typically moved between VRAM and system RAM during inference, as it would be too slow. For some definition of "typical", sure, but it depends on what performance trade someone is willing to accept. Strata LLM is geared around streaming weights as needed. It lets me run...
Long replies break deepseek-r1 / phi3.5 until the model is unloaded (context shift rescales cached keys)
### What is the issue? after a long reply (or chat passing `num_ctx`), `deepseek-r1:8b`, `phi3.5`, `phi4-mini` and `phi3` 128k start giving worse answers, and it sticks until you `ollama stop` the model (looks like #7123). Below 23 GiB VRAM and on CPU, Ollama defaults to...
[Tracking] OffloadingConnector layout support
...for both CPU and Tiered offloading: document what works, gate what doesn't, and add the missing support. **Current issues and PRs** - [ ] #60004 / #60005 - LHBNC returns wrong output after CPU reload. The PR adds a guard; supporting the layout is still separate work. - [ ]...
[Bug]: Qwen3.5/3.6 MoE GPTQ checkpoint with BF16 fused MTP experts: `mtp.layers.0.mlp.experts has no parameter 'w2_weight'` (plain `gptq` path, related to #36954)
### Your current environment ### 🐛 Describe the bug MTP speculative decoding fails to load the draft layer of a GPTQ Qwen3.6-35B-A3B checkpoint. The main model loads correctly; the error is raised in `load_draft_model`: **Model:**...
...48GB Radeon VRAM GPU for training/inference via Vulkan (before someone claims it's too expensive). Wake up folks, your assumptions about what is hard to do and what is not is the wrong rationality here. Humans are sequential beings, therefore we think that a lot of tasks are...
Feature Request: Add support for K2 Horizon (0.9B, 3.7B, 7B, 32B, 36B MoVA)
### Prerequisites - [x] I am running the latest code. Mention the version if possible as well. - [x] I carefully followed the [README.md](https://github.com/ggml-org/llama.cpp/blob/master/README.md). - [x] I searched using keywords relevant to my issue to make sure that I am...
Add DeepSeek-V4.1-Flash (deepseek_v41)
### Model description **DeepSeek-V4.1-Flash** (released by DeepSeek, [`deepseek-ai/DeepSeek-V4.1-Flash`](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash), ~510 GB, fp8 / packed-fp4 weights). Text backbone of the V4 line with three additions over `deepseek_v4`: - **CSA2 -...
Companies and products named
About this problem
Evidence
8 posts from 8 people in 5 places, about 1 a week over 57 days. Mostly on Hacker News comments, vllm-project/vllm, huggingface/transformers. Tools named alongside: Hugging Face, llama.cpp, NVIDIA, GitHub.
Frustration Frustration 1 of 3· Seen on GitHub issues, Hacker News
How it was grouped
Posts that state a pain and match the "local-llm" rule about DeepSeek. First post Sep 13, 2026, latest Nov 8, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Running Ollama locally hits memory and speed limits
47 posts from 45 people on GitHub issues, Reddit.
Running llama.cpp locally hits memory and speed limits
45 posts from 41 people on GitHub issues, Reddit.
Running LLMs locally hits memory and speed limits
22 posts from 21 people on GitHub issues, Reddit, Hacker News.
Running vLLM locally hits memory and speed limits
21 posts from 20 people on GitHub issues, Reddit.
Running Qwen locally hits memory and speed limits
18 posts from 18 people on GitHub issues, Reddit, Stack Overflow.
Running Hugging Face locally hits memory and speed limits
17 posts from 15 people on GitHub issues, Reddit.
Running NVIDIA locally hits memory and speed limits
16 posts from 16 people on GitHub issues.
Running OpenAI locally hits memory and speed limits
13 posts from 13 people on GitHub issues.
Running Claude Code locally hits memory and speed limits
6 posts from 6 people on GitHub issues.
Running PyTorch locally hits memory and speed limits
5 posts from 5 people on GitHub issues, Hacker News.
Running GitHub locally hits memory and speed limits
4 posts from 4 people on Reddit, GitHub issues.
Running Perplexity locally hits memory and speed limits
3 posts from 3 people on GitHub issues.
Other DeepSeek problems
DeepSeek users keep filing bugs
Top post: "DeepSeek-V4.1-Flash cannot run on SM120 (RTX PRO 6000): compressed-layer page_block_size=32 has no FlashInfer", one of 18 posts from 18 people on GitHub issues, Reddit.
DeepSeek GPU access and drivers are painful
7 posts from 6 people on GitHub issues.
Trouble with DeepSeek cloud
Top post: "Request: Add cloud support for DeepSeek-V4.1-Flash", one of 5 posts from 5 people on GitHub issues.
DeepSeek rate limits and usage caps block work
3 posts from 3 people on Hacker News, GitHub issues, Reddit.
Trouble with DeepSeek flash
Top post: "Asymmetric P/D Deployment for DeepSeek-V4.1 Flash", one of 3 posts from 3 people on GitHub issues, Community forums.
DeepSeek lock-in makes switching costly
2 posts from 2 people on GitHub issues. Unverified.
DeepSeek upgrades break working setups
2 posts from 2 people on GitHub issues. Unverified.
DeepSeek fails to return valid structured output
2 posts from 2 people on GitHub issues. Unverified.
History