Problemsby IbisRunning Mistral locally hits memory and speed limits
MLOps2 posts from 2 people2d active+11 post in the last 7 days, 0 the 7 days before (steady)Unverified
Posts per day
The posts behind it
2, newest firstGGUF tokenizer ignores tokenizer.ggml.add_bos_token, so the default encoding drops BOS### System Info - `transformers` version: 5.21.0.dev0 (main at b0cdc61e60) - Platform: macOS-26.6.2-arm64-arm-64bit - Python version: 3.12.13 - Huggingface_hub version: 2.2.0 - Tokenizers version: 0.23.2 - PyTorch version: 2.13.0 ### Who can help? @SunMarc (follow-up from...huggingface/transformers · francip1 repliesOpen ↗Conversion to GGUF bug: --mistral-format failed to be applied### Git commit I tried converting mistral-small-4-nvfp4 to GGUF. llama.cpp did not take the `--mistral-format` option into account; it looked for `config.json` and failed to start the conversion. ### Operating systems Windows ### GGML backends CUDA ### Problem description &...ggml-org/llama.cpp · FedericoFB2 repliesOpen ↗
Companies and products named
About this problem
Evidence
2 posts from 2 people in 2 places, about 1 a week over 13 days. Mostly on ggml-org/llama.cpp, huggingface/transformers. Tools named alongside: llama.cpp, NVIDIA, PyTorch, Hugging Face.
Frustration Frustration 0 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "local-llm" rule about Mistral. First post Oct 3, 2026, latest Oct 15, 2026. Unverified: fewer than 3 posts, or one person or place only. Method
The same problem elsewhere
Running llama.cpp locally hits memory and speed limitsRunning Ollama locally hits memory and speed limitsRunning LLMs locally hits memory and speed limitsRunning Qwen locally hits memory and speed limitsRunning NVIDIA locally hits memory and speed limitsRunning Hugging Face locally hits memory and speed limitsRunning vLLM locally hits memory and speed limitsRunning OpenAI locally hits memory and speed limitsRunning Claude Code locally hits memory and speed limitsRunning PyTorch locally hits memory and speed limitsRunning GitHub locally hits memory and speed limitsRunning DeepSeek locally hits memory and speed limits
Other Mistral problems
Mistral agents loop, stall or fail midwayTrouble with Mistral harnessMistral users keep filing bugsTrouble with Mistral vibeTrouble with Mistral systemMistral writes buggy code that is hard to reviewMistral upgrades break working setupsMistral rate limits and usage caps block workMistral fails to return valid structured outputTrouble with Mistral rustMistral hits connection, locking and scaling limitsMistral runs out of memory
History
Running Mistral locally hits memory and speed limits (2 posts)