Data and AI struggles

1,730 problems · 342 companies

Outages and change

PyTorch upgrades break working setups

26 posts from 21 people14d activeRising, +5 this week

Posts per day

Posts per day26 posts, Sep 3 to Nov 9

The posts behind it

GitHub issuespytorch/pytorch · 14H034160212Nov 7

[ONNX] Dynamo export of bool tensor addition emits Add(bool), which ONNX Runtime rejects

...in PyTorch, but exporting it to ONNX with the dynamo exporter produces a model that fails to load in ONNX Runtime. This matters because boolean masks are commonly combined using `+`, and the export appears to succeed despite producing an unusable model. #### AI-assisted analysis...

1 reactions2 replies
GitHub issuespytorch/pytorch · BlazeSR10hawkNov 7

[DTensor] nan_to_num incorrectly preserves Partial(max/min), silently changing reductions with NaN/Inf

### 🐛 Bug On current `main`, `torch/distributed/tensor/_ops/_pointwise_ops.py` registers `aten.nan_to_num.default`, `aten.nan_to_num_.default`, and `aten.nan_to_num.out` in `non_decreasing_unary_ops`. This adds `Partial(max) -> Partial(max)` and `Partial(min) -> Partial(min)`...

GitHub issuespytorch/pytorch · tangyuan499-commitsNov 6

[Bug] F.group_norm(num_groups=0) raises ZeroDivisionError before native validation

### 🐛 Describe the bug ## Description `torch.nn.functional.group_norm` raises a raw `ZeroDivisionError` for `num_groups=0` instead of reporting an invalid group count. **Reproduction:** **Expected behavior:** `F.group_norm(x, 0)` should reject the invalid `num_groups` parameter...

GitHub issuespytorch/pytorch · tangyuan499-commitsNov 6

[CPU] _standard_gamma and _sample_dirichlet silently accept invalid concentration values

### 🐛 Describe the bug ## Description `torch._standard_gamma` and `torch._sample_dirichlet` silently accept invalid concentration parameters, including negative values and NaN, producing apparently valid positive samples. ### Reproduction Observed on PyTorch 2.10.0+cpu: ###...

GitHub issueshuggingface/transformers · BlazeSR10hawkNov 3

AudioClassificationPipeline defaults to softmax for regression and multi-label models, corrupting prediction scores

### System Info - Transformers: `5.23.0.dev0` - Transformers source commit: `5572e70248d3286449771df61729a42ba3f5f4c7` - PyTorch: `2.11.0+cu130` - GPU: NVIDIA Tesla T4 - Environment: Linux - Backend: PyTorch / CUDA ### Who can help? @Rocketknight1 @eustlb ### Summary...

2 replies
GitHub issueshuggingface/transformers · the-cross-artNov 1

`_attn_implementation` setter does not reach encoder/decoder stacks built with `copy.deepcopy(config)`

### System Info - `transformers` version: 5.23.0.dev0 (main @ ca8c871ede) - Platform: macOS-27.0.1-arm64-arm-64bit - Python version: 3.12.11 - PyTorch version: 2.14.0 (GPU? False) - Using MPS: True ### Who can help? @Cyrilvallez @vasqu `git blame` around the...

6 replies
GitHub issueshuggingface/transformers · yuanx749Oct 25

DataCollatorForLanguageModeling fails with whole-word masking and dynamic padding

### System Info - `transformers` version: 5.21.0.dev0 - Platform: Linux-6.18.54.1-microsoft-standard-WSL2-x86_64-with-glibc2.39 - Python version: 3.12.12 - Huggingface_hub version: 1.33.0 - Safetensors version: 0.8.0 - Accelerate version: 1.15.0 - Accelerate config: not found -...

2 replies
GitHub issuespytorch/pytorch · prinwinterOct 25

get_ema_avg_fn introduces floating-point drift at fixed points

### 🐛 Describe the bug ## Summary `torch.optim.swa_utils.get_ema_avg_fn` can introduce floating-point drift even when the EMA parameter is exactly equal to the current model parameter. I independently ran the reproducer below and verified the observed behavior. I believe this...

2 replies
GitHub issueshuggingface/transformers · BlazeSR10hawkOct 23

FPS sampling returns no frames for short, non-empty videos

### System Info - `transformers` version: 5.21.0.dev0 - Platform: Windows-10-10.0.22631-SP0 - Python version: 3.11.17 - Huggingface_hub version: 1.33.0 - Safetensors version: 0.8.0 - Accelerate version: 1.15.0 - PyTorch version (accelerator?): 2.14.1+cpu (NA) ### Who can help?...

1 reactions
GitHub issueshuggingface/transformers · DManettiOct 23

SAM3: `Sam3ViTLayer.position_ids` buffer is cast to bf16 on loading, degrading RoPE in global-attention layers (regression since v5.17.0, PR #48105)

### System Info - `transformers` version: 5.20.2 (also reproduced with 5.17.0; 5.13.0-5.16.1 not affected) - Platform: Linux-7.0.0-1019-nvidia-aarch64-with-glibc2.39 - Python version: 3.12.3 - Huggingface_hub version: 1.33.0 - Safetensors version: 0.8.0 - Accelerate version:...

2 replies
GitHub issueshuggingface/transformers · niklas-palmOct 23

tp_plan="auto" adds backward hooks that break torch.compile graphs in inference

### System Info - transformers: main (e7b0341) - torch: 2.14.1 (CPU) - 2 processes, gloo backend - Python 3.12 ### Who can help? @3outeille @ArthurZucker ### Information - [ ] The official example scripts - [x] My own modified scripts ### Tasks - [ ] An officially supported task...

2 replies
GitHub issueshuggingface/transformers · BlazeSR10hawkOct 23

GOT-OCR2 produces incorrect bounding-box coordinates for NumPy images

### System Info - `transformers` version: 5.21.0.dev0 - Platform: Windows-10-10.0.22631-SP0 - Python version: 3.11.17 - Huggingface_hub version: 1.33.0 - Safetensors version: 0.8.0 - Accelerate version: 1.15.0 - PyTorch version (accelerator?): 2.14.1+cpu (NA) ### Who can help?...

4 replies
GitHub issueshuggingface/transformers · BlazeSR10hawkOct 23

default_data_collator incorrectly casts NumPy integer labels to float32

### System Info - `transformers` version: 5.21.0.dev0 - Platform: Windows-10-10.0.22631-SP0 - Python version: 3.11.17 - Huggingface_hub version: 1.33.0 - Safetensors version: 0.8.0 - Accelerate version: 1.15.0 - PyTorch version (accelerator): 2.14.1+cpu (NA) - CPU-only...

1 replies
GitHub issueshuggingface/transformers · digit50Oct 17

Expert-parallel token dispatch gives experts no gradient on torch 2.7 to 2.10

### System Info - `transformers` version: 5.20.1 - Platform: Linux-7.1.5-76070105-generic-x86_64-with-glibc2.39 - Python version: 3.12.3 - Huggingface_hub version: 2.2.0 - Safetensors version: 0.8.0 - Accelerate version: not installed - Accelerate config: not found - DeepSpeed...

1 replies
GitHub issuespytorch/pytorch · mullerhaiOct 17

**[BUG] Libtorch C++ API fails to export trained model to ONNX and AOT format, two separate export errors**

### 🐛 Describe the bug ## 🐛 Bug When exporting a trained model via Libtorch (C++ API), I hit two separate failures: 1. Export trained model to ONNX format failed. 2. Export trained model to AOT (torch.export / AOTInductor) format also failed. The model is trained in Python...

1 replies
GitHub issueshuggingface/transformers · DimensionSTPOct 9

AutoVideoProcessor eagerly imports unrelated video processors after backend registration update

### System Info - Python 3.12, Linux, PyTorch 2.11.0+cu129, torchvision 0.26.0+cu129. - Transformers main `fa40619176b63effb726ab0d0000925327cad7db`; the behavior is still present at `125f265a26c76c90e371b83386830cd61e5341fe`. - CPU reproduction; no model weights or GPU are...

2 replies
Redditr/datascience · u/Far-Media3683Oct 9

Sparse panel data, a small experiment

...costs a little accuracy vs. modelling each pollutant alone. A simple ±15day mean is hard to beat when data is dense. The network’s smooth time curve misses shared daily shocks. Writeup with code, figures, and results:....

GitHub issuespytorch/pytorch · ruihe774Sep 30

[Inductor][PDL] triton.enable_pdl: a kernel loads a buffer the previous kernel mutated in place before gdc_wait(), giving wrong results

### 🐛 Describe the bug The analysis was done with the assistance of AI and reviewed by me. With `torch.compile(..., options={"triton.enable_pdl": True})`, a kernel that reads a buffer the immediately preceding kernel mutated *in place* (`index_put_`, an indexed store, scatter,...

GitHub issuespytorch/pytorch · AdetsGithubSep 30

deserialize_torch_artifact retries every `weights_only=True` failure with `weights_only=False`

I was reading `deserialize_torch_artifact` on main (`2581b459d5546d212ad4a660bb57d163d436de35`). The first `torch.load` does not decide what gets loaded. ### Describe the bug `deserialize_torch_artifact` in `torch/_export/serde/serialize.py` does this: The `except` is bare...

GitHub issuespytorch/pytorch · anna-researcherSep 30

torch.isin raises "view size is not compatible" on CPU when test_elements is a non-contiguous multi-dimensional tensor (regression since 1.11)

### 🐛 Describe the bug On CPU, `torch.isin` raises a `RuntimeError` when `test_elements` is a multi-dimensional tensor that can't be flattened without a copy (e.g. a transposed matrix). The same call works with `test_elements.contiguous()`, `numpy.isin` returns the expected...

GitHub issueshuggingface/transformers · lucaluo925Sep 24

`SequenceBiasLogitsProcessor`: two edge cases (token id 0 rejected in list format; prefix equal to full context skipped)

### System Info - `transformers` version: 5.17.0 - Platform: macOS-26.6.2-arm64-arm-64bit-Mach-O - Python version: 3.14.7 - Huggingface_hub version: 1.33.0 - Safetensors version: 0.8.0 - Accelerate version: not installed - Accelerate config: not found - DeepSpeed version: not...

2 replies
GitHub issueshuggingface/transformers · UnitedSnakesSep 21

`save_pretrained(state_dict=...)` can duplicate tied weights after state-dict materialization

### System Info - transformers: main @ c587bc884db2c2e31fc2b8102314656b17aa07b1 - PyTorch: 2.14.0 - Python: 3.14 - Platform: macOS arm64 ### Who can help? @ArthurZucker @Cyrilvallez ### Information - [ ] The official example scripts - [x] My own modified scripts ### Tasks - [ ]...

4 replies
Redditr/MachineLearning · u/Dangerous-Pilot-6065Sep 14

[P] Wine synthesis using VAE [P]

.... Is the loss too large? I know that it never could reach perfect zero by how do I know if the loss is good enough? After reaching the plateau? I use MSELoss.. The repo itself: https://github.com/theaidenmax/tabular-vae-wine-generator . This is my second project in VAE (after...

GitHub issuespytorch/pytorch · Concode0Sep 10

matrix_exp T8 approximation uses an incorrect x7 coefficient

...by `compute_T8`. For a small 2D rotation, this causes a reproducible numerical error when a singleton matrix selects the T8 path. A minimal reproducer is: On macOS ARM64 CPU with PyTorch 2.14.0: The analytic reference is the closed-form exponential of which is the 2D rotation...

1 reactions2 replies
GitHub issuespytorch/pytorch · francesco-bertolottiSep 10

[BUG] DTensor RNG tracker is applied to CPU tensors under FSDP2 CPUOffloadPolicy, resetting the CPU RNG state

## 🐛 Describe the bug When a module is wrapped with `fully_shard(..., offload_policy=CPUOffloadPolicy())`, its parameters keep a CUDA-typed `DeviceMesh` even though the local shards live on CPU. Random ops on those parameters are still dispatched through the process-wide...

1 reactions
GitHub issuespytorch/pytorch · kshitij12345Sep 3

[regression] torch.compile(fullgraph=True) fails on dataclasses.replace for a sourceless instance of a sourced class

### 🐛 Describe the bug ### Issue `torch.compile(fullgraph=True)` fails to trace `dataclasses.replace(obj, **changes)` when `obj` is a locally-constructed (sourceless) instance of a module-level (sourced) dataclass, as long as `torch._dynamo.config.enable_trace_load_build_class`...

2 reactions2 replies

Companies and products named

26 posts
12 posts
7 posts
6 posts
1 post
1 post
About this problem

Evidence

26 posts from 21 people in 4 places, about 3 a week over 66 days. Mostly on huggingface/transformers, pytorch/pytorch, r/MachineLearning. Tools named alongside: Hugging Face, GitHub, NVIDIA, PyPI.

Frustration Frustration 0 of 3· Seen on GitHub issues, Reddit

How it was grouped

Posts that state a pain and match the "breaking-changes" rule about PyTorch. First post Sep 3, 2026, latest Nov 7, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

GitHub upgrades break working setups

87 posts from 75 people on GitHub issues, Hacker News, Stack Overflow.

Upgrades and new releases break working setups

76 posts from 62 people on GitHub issues, Reddit, App Store reviews.

Claude Code upgrades break working setups

67 posts from 60 people on GitHub issues, Hacker News, Reddit.

Codex upgrades break working setups

65 posts from 58 people on GitHub issues, Reddit.

DuckDB upgrades break working setups

37 posts from 28 people on GitHub issues.

Claude upgrades break working setups

35 posts from 34 people on GitHub issues, Reddit, Hacker News.

ClickHouse upgrades break working setups

31 posts from 20 people on GitHub issues.

Anthropic upgrades break working setups

27 posts from 22 people on GitHub issues.

ChatGPT upgrades break working setups

26 posts from 26 people on GitHub issues, Community forums, App Store reviews.

OpenAI upgrades break working setups

25 posts from 25 people on GitHub issues, Reddit, Community forums.

Open WebUI upgrades break working setups

25 posts from 2 people on GitHub issues.

Gemini upgrades break working setups

24 posts from 24 people on GitHub issues, Community forums, Reddit.

Other PyTorch problems

PyTorch GPU access and drivers are painful

14 posts from 14 people on GitHub issues.

PyTorch users keep filing bugs

Top post: "Meta-issue: torch.compile gaps found via the CPython test suite", one of 46 posts from 37 people on GitHub issues, Reddit.

Trouble with PyTorch values

Top post: "How can I sum elements of a tensor based on a condition of one out of two values in its index pair into a new", one of 3 posts from 3 people on GitHub issues, Stack Overflow.

Trouble with PyTorch causing

Top post: "PyVista plotting running slowly, resetting box size and causing collisions", one of 3 posts from 3 people on GitHub issues, Stack Overflow.

PyTorch deprecations force unplanned migrations

8 posts from 5 people on GitHub issues.

Trouble with PyTorch true

Top post: "PaddleOCR-VL: position embedding interpolation changed from align_corners=False to True", one of 9 posts from 6 people on GitHub issues.

Trouble with PyTorch inductor

Top post: "`[Inductor][CPU] torch.var returns NaN for large-magnitude constant float64 inputs while eager returns 0`", one of 12 posts from 9 people on GitHub issues.

PyTorch builds are slow or flaky

9 posts from 1 person on GitHub issues. Unverified.

Trouble with PyTorch input

Top post: "Generation crashes with encoder_repetition_penalty when encoder_outputs are provided without input_ids", one of 6 posts from 6 people on GitHub issues.

Running PyTorch locally hits memory and speed limits

5 posts from 5 people on GitHub issues, Hacker News.

PyTorch lock-in makes switching costly

4 posts from 4 people on GitHub issues.

Trouble with PyTorch windows

Top post: "Bundled cuDNN fails on first use since #202206 when no cuDNN is on PATH (Cannot load symbol cudnnGetVersion)", one of 4 posts from 4 people on GitHub issues.

History

Momentum went from steady to rising.

Nov 8, 2026

Momentum went from fading to steady.

Nov 7, 2026

Momentum went from rising to fading.

Oct 30, 2026

Momentum went from fading to rising.

Oct 29, 2026

Momentum went from rising to fading.

Oct 24, 2026

Momentum went from steady to rising.

Oct 23, 2026

Momentum went from fading to steady.

Oct 20, 2026

Momentum went from steady to fading.

Oct 16, 2026

Momentum went from fading to steady.

Oct 14, 2026

Momentum went from rising to fading.

Oct 7, 2026

Momentum went from fading to rising.

Oct 5, 2026

Momentum went from steady to fading.

Oct 2, 2026