Problemsby GlyphNVIDIA GPU access and drivers are painful
12 posts from 12 people8d activeSteady, +2 this week
Posts per day
The posts behind it
[Feature]: CUDA 12 wheels for v0.33.x (torch 2.14.0 has cu126 only), and three stale CUDA gates that break CUDA 12.6 source builds
### 🚀 The feature, motivation and pitch **Request:** publish CUDA 12 wheels for the 0.33.x releases (a `+cu126` wheel would be enough), or say whether CUDA 12 builds from source are still supported. **Context.** #62575 removed CUDA 12.9 from the release pipeline, so v0.33.0...
nvidia pascal support broken (CUDA error) in ver 0.40.13
### What is the issue? What happened?: C:\a\ollama\ollama\build\llama-server-cuda_v12\_deps\llama_cpp-src\ggml\src\ggml-cuda\ggml-cuda.cu:110: CUDA error What did you expect to happen?: Model to execute normally, without CUDA error I'm using Ollama with Open WebUI v0.12.5...
How to deal with this `torch.AcceleratorError: CUDA error: device-side assert triggered` error?
...information matrix. I found this code that appears to do just that, but there are some issues with the code. First consider the code in its original form:. estimated_fisher = {} for n, p in model.named_parameters(): estimated_fisher[n] = torch.zeros_like(p) model.eval() for i,...
CUDA device-side assert triggered in 31 generation tests
...may have occurred in a prior kernel launch rather than in the `.to()` call itself. 31 failing CI tests in `generation`, all with `torch.AcceleratorError: CUDA error: device-side assert triggered`: -...
[Bug]: Qwen4Exp (Qwen3.8-Flash-Next) — single prefill crossing the 262,144 native boundary crashes the engine with CUDA illegal memory access (v0.32.0 and nightly, OK on v0.31.1)
...on liveness probes. - `--max-model-len 262144` (dropping the YaRN region) is not a workaround we can use - long-context 1M is a product requirement - so we rolled back to `0.31.1rc1.dev1+gbb87d227d`. - Full logs (three instances, including crash + restart cycles) available on...
[ROCm] On Windows, LibTorch C++ programs never load torch_hip.dll, so torch::cuda::is_available() is false
### 🐛 Describe the bug On Windows with a ROCm build of PyTorch, a C++ program that links LibTorch through CMake gets `torch::cuda::is_available() == false`, while `torch.cuda.is_available()` in Python on the same install is `True`. Minimal repro, `CMakeLists.txt`: `main.cpp`:...
ML on a shoestring? [D]
...for enabling that. . That being said, I am starting to accumulate GPU costs, and it got painful enough this morning for me to wonder if I am using my dollar budget in the best possible way. And I'm wondering what other people do, in similar situations. After granting myself...
Relax deterministic-mode NVLS disablement on qualified NVIDIA systems
### Summary `ProcessGroupNCCL` currently disables NCCL’s NVLS algorithm whenever deterministic algorithms are enabled and the user has not explicitly set `NCCL_ALGO`: [https://github.com/pytorch/pytorch/blob/main/torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp#L1010-L1025](https...
`nll_loss2d` (spatial `cross_entropy`) on CUDA: fp16 `mean` reduction returns inf/NaN and all-zero gradients once total weight exceeds fp16 range
### 🐛 Describe the bug With fp16 inputs outside autocast, `nll_loss` with `reduction="mean"` returns inf once the sum of the loss passes 65,504, and (with typical loss values) NaN once the `total_weight` passes 65,504. Once `total_weight` overflows, the gradient silently zeros....
Inductor StaticallyLaunchedTritonKernel finalization can invalidate CUDA graph capture via cuModuleUnload
### 🐛 Describe the bug `torch.compile(..., mode="max-autotune")`can leave static Inductor CUDA kernels whose Python finalizers run during a later CUDA graph capture. When this happens, `cuModuleUnload` is called while capture is active and invalidates the graph. The visible...
...It reads sensor data directly from the GPU.. The default output is human-readable but painful to parse:. bash nvidia-smi. For scripting, use the query flag with CSV output:. bash nvidia-smi...
torch.bmm on CUDA crashes with "fatal error: Python.h" in 2.14 when python3-dev is missing — _native Triton ops have no fallback when Triton cannot build its driver
### 🐛 Describe the bug Since 2.14, `torch._native` routes `aten::bmm` with outer-product shapes (`a: (B, M, 1)`, `b: (B, 1, N)`) to a Triton kernel on CUDA (`torch/_native/ops/bmm_outer_product/triton_impl.py`). Triton builds its CUDA driver shim...
Companies and products named
About this problem
Evidence
12 posts from 12 people in 7 places, about 2 a week over 51 days. Mostly on pytorch/pytorch, vllm-project/vllm, #machinelearning on Mastodon. Tools named alongside: PyTorch, GitHub, Hugging Face, vLLM.
Frustration Frustration 0 of 3· Seen on GitHub issues, Mastodon, Reddit, Stack Overflow
How it was grouped
Posts that state a pain and match the "gpu-access" rule about NVIDIA. First post Sep 14, 2026, latest Nov 3, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
PyTorch GPU access and drivers are painful
14 posts from 14 people on GitHub issues.
GPU access, drivers and CUDA versions are painful
7 posts from 7 people on GitHub issues, Reddit.
DeepSeek GPU access and drivers are painful
7 posts from 6 people on GitHub issues.
vLLM GPU access and drivers are painful
5 posts from 3 people on GitHub issues.
Claude Code GPU access and drivers are painful
3 posts from 3 people on GitHub issues.
Ollama GPU access and drivers are painful
3 posts from 3 people on Hacker News, Reddit, GitHub issues.
Chroma GPU access and drivers are painful
2 posts from 1 person on GitHub issues. Unverified.
Mistral GPU access and drivers are painful
2 posts from 2 people on GitHub issues. Unverified.
Other NVIDIA problems
Running NVIDIA locally hits memory and speed limits
16 posts from 16 people on GitHub issues.
NVIDIA users keep filing bugs
Top post: "`clef-flash` decision model always fails on `/v1/systemone`, Clef: non-finite logit (CUDA) / Clef: cannot", one of 41 posts from 38 people on GitHub issues, Reddit, Community forums.
Trouble with NVIDIA windows
Top post: "Windows CUDA prebuilt binaries stuck at b11828 while release tags keep advancing (b11846..b11852)", one of 3 posts from 3 people on GitHub issues.
NVIDIA vulnerabilities are hard to triage and patch
77 posts from 77 people on Glyph Ibis.
NVIDIA upgrades break working setups
11 posts from 11 people on GitHub issues.
Trouble with NVIDIA account
Top post: "NVIDIA Careers Workday Account Locked - Unable to Complete Employee Referral Application", one of 9 posts from 8 people on Community forums.
NVIDIA lock-in makes switching costly
5 posts from 5 people on GitHub issues, Hacker News, Bluesky.
NVIDIA runs out of memory
5 posts from 5 people on GitHub issues.
Trouble with NVIDIA slower
Top post: "CUDA training loop runs ~50x slower while a DataLoader iterates in a background thread, since 2.10.0", one of 4 posts from 4 people on GitHub issues.
Trouble with NVIDIA process
Top post: "`HuggingFaceEmbeddings(multi_process=True)` starts CPU workers on non-CUDA accelerators", one of 3 posts from 3 people on GitHub issues.
Trouble with NVIDIA phone
Top post: "Unable to receive OTP for Vietnam (+84) phone verification on build.nvidia.com", one of 3 posts from 3 people on Community forums.
Fine-tuning NVIDIA is costly and unpredictable
3 posts from 3 people on GitHub issues.
History