Data and AI struggles

2,114 problems · 377 companies

MLOps

Running PyTorch locally hits memory and speed limits

10 posts from 6 people6d activeRising, +5 this week

Posts per day

Posts per day10 posts, Sep 6 to Nov 21

The posts behind it

GitHub issuespytorch/pytorch · tangyuan499-commitsNov 15

[quantized] Empty BatchNorm inputs ignore requested output quantization parameters

### 🐛 Describe the bug ### Description `quantized::batch_norm1d`, `batch_norm2d`, and `batch_norm3d` return `qx.clone()` when the input is empty. This preserves the input quantization parameters instead of using the requested `output_scale` and `output_zero_point`. ###...

GitHub issuespytorch/pytorch · tangyuan499-commitsNov 15

[quantized] Embedding.from_float fails to preserve quint4x2 dtype

### 🐛 Describe the bug ### Description `nn.quantized.Embedding.from_float` fails when quantizing an Embedding with `torch.quint4x2` weights. The resulting weight is correctly quantized to `quint4x2`, but the created Embedding module retains the default `quint8` dtype. Its...

GitHub issuespytorch/pytorch · tangyuan499-commitsNov 15

[quantized] PReLU incorrectly scales positive inputs when weight scale is small

### 🐛 Describe the bug ### Description Quantized PReLU incorrectly scales positive inputs when the weight quantization range cannot represent 1.0. The positive branch of PReLU should be an identity operation, but the CPU kernel quantizes 1.0 using the weight's scale and zero...

GitHub issuespytorch/pytorch · tangyuan499-commitsNov 15

[quantized] EmbeddingBag silently ignores the mode argument

### 🐛 Describe the bug ### Description `nn.quantized.EmbeddingBag` silently ignores its `mode` argument and always performs sum reduction. The `mode` is stored by the constructor and copied by `from_float`, but `forward()` hardcodes `0` when calling both `embedding_bag_byte`...

GitHub issuespytorch/pytorch · tangyuan499-commitsNov 15

[quantization] Quantized MultiheadAttention uses bias_k qparams for bias_v

### 🐛 Describe the bug ### Description `nn.quantized.MultiheadAttention.from_observed()` incorrectly uses `bias_k` to calculate the quantization parameters for `bias_v`. When the two biases have significantly different value ranges, `bias_v` is severely quantized using...

GitHub issuespytorch/pytorch · mappygridNov 6

torch.equal still overreads quint2x4 storage in 2.11.0 (follow-up to #162801)

### Bug description `torch.equal()` appears to read past the backing storage of CPU `quint2x4` tensors. I can reproduce incorrect comparisons with the PyTorch 2.11.0 macOS arm64 CPU wheel. A separate guard-page test crashes when the read crosses the storage boundary. This looks...

GitHub issuespytorch/pytorch · zhengliyang01Oct 26

[FSDP2] uneven-sharding check in `all_gather_inputs` rejects multi-tensor `fsdp_pre_all_gather` payloads (quantized weights)

### 🐛 Describe the bug Under uneven dim-0 sharding, `FSDPParam.all_gather_inputs` (`torch/distributed/fsdp/_fully_shard/_fsdp_param.py`) rejects **any multi-tensor payload** returned by `fsdp_pre_all_gather`, even when every component is correctly padded along the sharded dim:...

GitHub issuespytorch/pytorch · anna-researcherSep 30

[torch.compile] AssertionError "n=copy_, n.args[0]=dequantize" when a graph input is mutated in place after torch.dequantize on a non-quantized tensor

### 🐛 Describe the bug Mutating a graph input in place after calling `torch.dequantize` on it (with a regular, non-quantized float tensor) works in eager mode but fails under `torch.compile` with an AOTAutograd functionalization assertion. The assertion comes from...

GitHub issuesvllm-project/vllm · ixcansSep 6

[Bug]: CPU MoE kernel produces NaN logits under torch.compile for Qwen3_5MoeForConditionalGeneration — reproduces with AND without GPTQ-Int4 quantization, fixed by --enforce-eager

...mode worth flagging on its own - a healthy pod/process gives zero signal that generation is broken. **Workaround: `--enforce-eager` completely fixes it, on both checkpoints.** With the identical checkpoint, config, and request, eager mode produces correct, coherent generations...

1 reactions1 replies

Companies and products named

10 posts
1 post
1 post
About this problem

Evidence

10 posts from 6 people in 3 places, about 1 a week over 71 days. Mostly on pytorch/pytorch, vllm-project/vllm, Hacker News comments. Tools named alongside: OpenAI, vLLM, Hugging Face.

Frustration Frustration 1 of 3· Seen on GitHub issues, Hacker News

Summary, model-written

PyTorch users encounter bugs including graph input compilation assertions, FSDP payload rejections, and out-of-bounds memory reads during tensor comparisons.

How it was grouped

Posts that state a pain and match the "local-llm" rule about PyTorch. First post Sep 6, 2026, latest Nov 15, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Running llama.cpp locally hits memory and speed limits

54 posts from 50 people on GitHub issues, Reddit, Stack Overflow.

Running Ollama locally hits memory and speed limits

48 posts from 46 people on GitHub issues, Reddit.

Running vLLM locally hits memory and speed limits

27 posts from 26 people on GitHub issues, Reddit.

Running LLMs locally hits memory and speed limits

24 posts from 23 people on GitHub issues, Reddit, Hacker News.

Running Qwen locally hits memory and speed limits

20 posts from 20 people on GitHub issues, Reddit, Stack Overflow.

Running Hugging Face locally hits memory and speed limits

19 posts from 16 people on GitHub issues, Reddit.

Running NVIDIA locally hits memory and speed limits

17 posts from 17 people on GitHub issues.

Running OpenAI locally hits memory and speed limits

14 posts from 14 people on GitHub issues.

Running DeepSeek locally hits memory and speed limits

11 posts from 11 people on GitHub issues, Hacker News.

Running Claude Code locally hits memory and speed limits

6 posts from 6 people on GitHub issues.

Running GitHub locally hits memory and speed limits

4 posts from 4 people on Reddit, GitHub issues.

Running Perplexity locally hits memory and speed limits

3 posts from 3 people on GitHub issues.

Other PyTorch problems

PyTorch writes buggy code that is hard to review

2 posts from 1 person on GitHub issues. Unverified.

PyTorch users keep filing bugs

Top post: "Meta-issue: torch.compile gaps found via the CPython test suite", one of 50 posts from 41 people on GitHub issues, Reddit, Stack Overflow.

PyTorch upgrades break working setups

29 posts from 23 people on GitHub issues, Reddit.

PyTorch GPU access and drivers are painful

17 posts from 17 people on GitHub issues, Reddit.

Trouble with PyTorch inductor

Top post: "`[Inductor][CPU] torch.var returns NaN for large-magnitude constant float64 inputs while eager returns 0`", one of 14 posts from 11 people on GitHub issues.

PyTorch builds are slow or flaky

11 posts from 2 people on GitHub issues.

Trouble with PyTorch true

Top post: "PaddleOCR-VL: position embedding interpolation changed from align_corners=False to True", one of 10 posts from 7 people on GitHub issues.

PyTorch deprecations force unplanned migrations

8 posts from 5 people on GitHub issues.

Trouble with PyTorch input

Top post: "Generation crashes with encoder_repetition_penalty when encoder_outputs are provided without input_ids", one of 7 posts from 7 people on GitHub issues.

Fine-tuning PyTorch is costly and unpredictable

5 posts from 5 people on GitHub issues, Reddit.

Trouble with PyTorch dynamic

Top post: "DISABLED test_weight_norm_1d_and_reduced_dtypes_dynamic_shapes (__main__.DynamicShapes)", one of 5 posts from 3 people on GitHub issues, Reddit.

Trouble with PyTorch windows

Top post: "Bundled cuDNN fails on first use since #202206 when no cuDNN is on PATH (Cannot load symbol cudnnGetVersion)", one of 4 posts from 4 people on GitHub issues.

History

Summary written by a model: PyTorch users encounter bugs including graph input compilation assertions, FSDP payload rejections, and out-of-bounds memory reads during tensor comparisons.

Nov 20, 2026

Momentum went from fading to rising.

Nov 18, 2026

Momentum went from steady to fading.

Nov 13, 2026

Momentum went from fading to steady.

Nov 8, 2026

Momentum went from steady to fading.

Nov 2, 2026

Momentum went from fading to steady.

Oct 30, 2026

Momentum went from steady to fading.

Oct 9, 2026

Momentum went from rising to steady.

Oct 7, 2026

Corroborated: 3 or more posts from 2 or more people or places.

Oct 5, 2026

Momentum went from steady to rising.

Oct 5, 2026

First seen with 2 posts.

Oct 3, 2026