Problemsby IbisTrouble with Hugging Face token
MLOps3 posts from 3 people3d active+11 post in the last 7 days, 1 the 7 days before (steady)
Posts per day
The posts behind it
3, newest firstTextStreamer re-decodes the whole line on every token (quadratic on long lines)### Feature request `TextStreamer.put` appends each token to `token_cache` and decodes the whole cache on every step. The cache only resets after a newline, so streaming a long line (code, JSON, long paragraphs, CJK) costs O(n²) decode work. With the Qwen2.5 tokenizer a...huggingface/transformers · deadstrobe5Open ↗SAM 3: text-only prompts drop the geometry CLS token that Meta's implementation always adds### System Info - `transformers` version: 5.19.0 (same code on main @5602e09) - Platform: macOS-26.6.2-arm64-arm-64bit - Python version: 3.12.13 - Huggingface_hub version: 2.2.0 - Safetensors version: 0.8.0 - PyTorch version (accelerator?): 2.14.1 (NA) - Using distributed or...huggingface/transformers · ju-ius3 repliesOpen ↗[zai-org/GLM-OCR] <|image|> token has special=False in Rust tokenizer backend — should it be True?## Problem The \` \` token (ID 59280) in \`zai-org/GLM-OCR\` is listed in \`tokenizer.all_special_tokens\` on the Python side, but its entry in the Rust (\`tokenizers.Tokenizer\`) backend has \`special=False\`: Output (verified on CI runner): ## Why the discrepancy exists The...huggingface/transformers · ydshieh1 reactions, 4 repliesOpen ↗
Companies and products named
About this problem
Evidence
3 posts from 3 people in 1 place, about 1 a week over 38 days. Mostly on huggingface/transformers. Tools named alongside: PyTorch, GitHub, Text.
Frustration Frustration 0 of 3· Seen on GitHub issues
How it was grouped
Posts about Hugging Face with no known issue, sharing the word "token". First post Sep 9, 2026, latest Oct 16, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
Other Hugging Face problems
Fine-tuning Hugging Face is costly and unpredictableTrouble with Hugging Face generateTrouble with Hugging Face sizeHugging Face users keep filing bugsTrouble with Hugging Face cruiseRunning Hugging Face locally hits memory and speed limitsHugging Face upgrades break working setupsTrouble with Hugging Face docsTrouble with Hugging Face trainerTrouble with Hugging Face gemmaTrouble with Hugging Face flightHugging Face integrations break or are missing
History
rising to steady
Trouble with Hugging Face token (3 posts)