Problemsby GlyphRedis rate limits and usage caps block work
6 posts from 6 people5d activeFading, +0 this week
Posts per day
The posts behind it
[Feature]: Queue requests at capacity by priority, with fair turns between tenants (priority has no effect under load today)
...regard to priority or tenant. So `priority` has no effect when the backend is the bottleneck. One tenant can starve the others, and the only protection is a hard per-tenant cap that wastes capacity when others are idle. Proposed behavior (opt-in, default off, so nothing changes...
[Bug]: Redis circuit breaker stays OPEN indefinitely on an idle instance — OPEN→HALF_OPEN only happens when a guarded call arrives
...guarded call. **Impact while a breaker is open:** - Every guarded call on that pod fast-fails to the in-memory tier, so its router cooldowns, session pins and shared rate-limit state stop propagating to other pods. - `litellm_redis_circuit_breaker_state{state="open"}` stays at...
[Bug]: Per-input embedding caching can fill a shared Redis and evict rate-limit/spend keys
### Description With the proxy response cache on (`litellm_settings.cache: true`, Redis), embedding responses are cached **per input item** (`caching_handler.py`, "a cache hit for some of the inputs in the list and a cache miss for others"). One batched `/v1/embeddings` request...
[Bug]: Scheduler never removes admitted requests from the priority queue, so prioritized requests fail during cooldown and, with Redis, under normal traffic
### Description When a request goes through the priority scheduler and the model group has a healthy deployment, `Scheduler.poll()` returns `True` without touching the queue, and the router only calls `remove_request` on the timeout branch. Every request admitted this way leaves...
[Bug]: sse_keepalive_ping_interval_seconds leaks a max_parallel_requests slot on every streaming request (ContextVar lost across asyncio.ensure_future)
...on their face - a TTFT keepalive and a concurrency limiter - which is what makes this hard to spot. Turning the keepalive on silently disables slot release for the v3 rate limiter. Observed in production: enabled at 06:08 UTC, first key lockout at 06:17. One key made exactly...
[Bug]: Completed batch can be skipped by model budget after logging cancellation
### Description A completed batch can escape its model budget charge if success logging is cancelled after `_claim_batch_charge` increments the marker and before `_increment_spend_for_key` updates spend. Later polls see the marker and skip the charge even though the spend...
Companies and products named
About this problem
Evidence
6 posts from 6 people in 1 place, about 1 a week over 43 days. Mostly on BerriAI/litellm. Tools named alongside: GitHub, Docker, Kubernetes, Grafana.
Frustration Frustration 1 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "rate-limits" rule about Redis. First post Sep 23, 2026, latest Nov 4, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
API rate limits and usage caps block real work
42 posts from 41 people on GitHub issues, Hacker News, Reddit.
Claude Code rate limits and usage caps block work
21 posts from 21 people on GitHub issues, Reddit, Hacker News.
Claude rate limits and usage caps block work
20 posts from 20 people on Reddit, App Store reviews, GitHub issues.
ChatGPT rate limits and usage caps block work
16 posts from 16 people on GitHub issues, Reddit, Stack Overflow.
Codex rate limits and usage caps block work
16 posts from 16 people on GitHub issues.
Anthropic rate limits and usage caps block work
12 posts from 9 people on GitHub issues, Reddit, Community forums.
GitHub rate limits and usage caps block work
11 posts from 10 people on GitHub issues, Hacker News.
Model Context Protocol rate limits and usage caps block work
9 posts from 6 people on GitHub issues, Hacker News.
Gemini rate limits and usage caps block work
9 posts from 9 people on GitHub issues, Reddit, App Store reviews.
MLflow rate limits and usage caps block work
6 posts from 5 people on GitHub issues.
OpenAI rate limits and usage caps block work
6 posts from 6 people on GitHub issues, Community forums.
Google Cloud rate limits and usage caps block work
5 posts from 4 people on Community forums, Reddit.
Other Redis problems
Redis upgrades break working setups
9 posts from 9 people on GitHub issues, Stack Overflow.
Redis users keep filing bugs
Top post: "7.0.0rc2 Flight SQL/ADBC validation: source-install dependency failures and metadata gaps", one of 3 posts from 1 person on GitHub issues.
History