Problemsby IbisTrouble with Hugging Face trainer
MLOps3 posts from 2 people3d active+00 posts in the last 7 days, 1 the 7 days before (fading)
Posts per day
The posts behind it
3, newest firstTrainer disables find_unused_parameters under any gradient checkpointing, not just reentrant### System Info transformers main (4cc2aa8430), torch 2.13.0+cu130, 2x B300. ### Who can help? @SunMarc ### Reproduction When `ddp_find_unused_parameters` is unset, `Trainer` uses `find_unused = not gradient_checkpointing`...huggingface/transformers · qgallouedecOpen ↗Trainer counts fp16 steps skipped by GradScaler as completed update steps (`global_step`, `max_steps`, `on_optimizer_step`)### System Info transformers 5.17.0 and main @ 96331a9; accelerate 1.15.0 and main @ 421af8b; torch 2.14.0; Python 3.13; Tesla T4, CUDA 13.0, `fp16=True`. Also reproduces on CPU with a `GradScaler` installed on the Accelerator (on CPU, `fp16=True` creates no scaler). ### Who can...huggingface/transformers · SeverinVisionary7 repliesOpen ↗Nothing destroys the process group, so every distributed Trainer run warns at exit### System Info transformers 5.18.0.dev0, accelerate 1.15.0, torch 2.13.0+cu129. ### Who can help? @SunMarc ### Reproduction Any distributed `Trainer` script that exits after training prints, once per rank: The Trainer sets distributed training up through accelerate and nothing...huggingface/transformers · qgallouedec2 repliesOpen ↗
Companies and products named
About this problem
Evidence
3 posts from 2 people in 1 place, about 1 a week over 23 days. Mostly on huggingface/transformers. Tools named alongside: GitHub, NVIDIA, Tesla.
Frustration Frustration 0 of 3· Seen on GitHub issues
How it was grouped
Posts about Hugging Face with no known issue, sharing the word "trainer". First post Sep 16, 2026, latest Oct 8, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
Other Hugging Face problems
Fine-tuning Hugging Face is costly and unpredictableTrouble with Hugging Face generateTrouble with Hugging Face sizeHugging Face users keep filing bugsTrouble with Hugging Face cruiseRunning Hugging Face locally hits memory and speed limitsHugging Face upgrades break working setupsTrouble with Hugging Face docsTrouble with Hugging Face tokenTrouble with Hugging Face gemmaTrouble with Hugging Face flightHugging Face integrations break or are missing
History
steady to fading
Trouble with Hugging Face trainer (3 posts)