Apache Airflow hits connection, locking and scaling limits
2 posts from 2 people2d activeNew, +2 this weekunverified
Posts per day
The posts behind it
Triggerer stops heartbeating while it uploads the logs of many finished triggers
When many deferred tasks finish at once with remote logging on, the triggerer uploads each finished trigger's log inline in its main loop, one after the other (`TriggerRunnerSupervisor._handle_request`, `TriggerStateChanges`: `upload_to_remote()` then `close()` per finished...
Task SDK: an exception during a supervisor request leaves its response for the next request
...value back, for example the previous `GetVariable`'s answer for a `GetConnection`, or fail to decode it. ### What you think should happen instead? A request never reads a response that belongs to another request. Possible fixes: check the response id and drain frames that belong...
Companies and products named
About this problem
Evidence
2 posts from 2 people in 1 place, about 2 a week over 2 days. Mostly on apache/airflow.
Frustration Frustration 3 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "db-scaling" rule about Apache Airflow. First post Nov 18, 2026, latest Nov 19, 2026. Unverified: fewer than 3 posts, or one person or place only. Method
The same problem elsewhere
Databases hit connection, locking and scaling limits
18 posts from 18 people on GitHub issues, Reddit, Hacker News.
GitHub hits connection, locking and scaling limits
6 posts from 6 people on GitHub issues.
MLflow hits connection, locking and scaling limits
4 posts from 4 people on GitHub issues.
PyTorch hits connection, locking and scaling limits
3 posts from 3 people on GitHub issues.
PostgreSQL hits connection, locking and scaling limits
3 posts from 3 people on Reddit, GitHub issues, Stack Overflow.
ClickHouse hits connection, locking and scaling limits
2 posts from 2 people on GitHub issues. Unverified.
Mistral hits connection, locking and scaling limits
2 posts from 2 people on GitHub issues. Unverified.
Anthropic hits connection, locking and scaling limits
2 posts from 2 people on GitHub issues. Unverified.
Polars hits connection, locking and scaling limits
2 posts from 2 people on GitHub issues. Unverified.
Other Apache Airflow problems
Apache Airflow backfills are slow and risky
9 posts from 9 people on GitHub issues.
Apache Airflow login, SSO or OAuth breaks
7 posts from 4 people on GitHub issues.
Apache Airflow users keep filing bugs
Top post: "Status of testing Providers that were prepared on October 20, 2026", one of 51 posts from 39 people on GitHub issues, Reddit.
Apache Airflow jobs fail silently or break often
16 posts from 15 people on GitHub issues, Stack Overflow.
Apache Airflow upgrades break working setups
14 posts from 12 people on GitHub issues.
Debugging Apache Airflow failures is hard
9 posts from 8 people on GitHub issues.
Trouble with Apache Airflow backend
Top post: "Trigger-written XComs bypass the configured XCom backend (raw rows mixed with externalized rows)", one of 4 posts from 4 people on GitHub issues.
Trouble with Apache Airflow dags
Top post: "DagModel.owners falls back to 'airflow' for Dags with only mapped tasks (LazyDeserializedDAG.owner ignores", one of 4 posts from 3 people on GitHub issues.
Apache Airflow permissions are confusing and error-prone
4 posts from 4 people on GitHub issues.
Apache Airflow lock-in makes switching costly
3 posts from 3 people on GitHub issues.
Trouble with Apache Airflow view
Top post: "Add Default DAG View preference to User Settings (Grid / Graph / Gantt)", one of 3 posts from 3 people on GitHub issues.
Trouble with Apache Airflow state
Top post: "Task state store writes from `on_retry_callback` fail with 404", one of 3 posts from 3 people on GitHub issues.
History