Data and AI struggles

2,149 problems · 379 companies

Databases

Apache Airflow hits connection, locking and scaling limits

2 posts from 2 people2d activeNew, +2 this weekunverified

Posts per day

Posts per day2 posts, Nov 18 to Nov 22

The posts behind it

GitHub issuesapache/airflow · seanghaeliNov 19

Triggerer stops heartbeating while it uploads the logs of many finished triggers

When many deferred tasks finish at once with remote logging on, the triggerer uploads each finished trigger's log inline in its main loop, one after the other (`TriggerRunnerSupervisor._handle_request`, `TriggerStateChanges`: `upload_to_remote()` then `close()` per finished...

1 replies
GitHub issuesapache/airflow · dablaNov 18

Task SDK: an exception during a supervisor request leaves its response for the next request

...value back, for example the previous `GetVariable`'s answer for a `GetConnection`, or fail to decode it. ### What you think should happen instead? A request never reads a response that belongs to another request. Possible fixes: check the response id and drain frames that belong...

Companies and products named

About this problem

Evidence

2 posts from 2 people in 1 place, about 2 a week over 2 days. Mostly on apache/airflow.

Frustration Frustration 3 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "db-scaling" rule about Apache Airflow. First post Nov 18, 2026, latest Nov 19, 2026. Unverified: fewer than 3 posts, or one person or place only. Method

The same problem elsewhere

Databases hit connection, locking and scaling limits

18 posts from 18 people on GitHub issues, Reddit, Hacker News.

GitHub hits connection, locking and scaling limits

6 posts from 6 people on GitHub issues.

MLflow hits connection, locking and scaling limits

4 posts from 4 people on GitHub issues.

PyTorch hits connection, locking and scaling limits

3 posts from 3 people on GitHub issues.

PostgreSQL hits connection, locking and scaling limits

3 posts from 3 people on Reddit, GitHub issues, Stack Overflow.

ClickHouse hits connection, locking and scaling limits

2 posts from 2 people on GitHub issues. Unverified.

Mistral hits connection, locking and scaling limits

2 posts from 2 people on GitHub issues. Unverified.

Anthropic hits connection, locking and scaling limits

2 posts from 2 people on GitHub issues. Unverified.

Polars hits connection, locking and scaling limits

2 posts from 2 people on GitHub issues. Unverified.

Other Apache Airflow problems

Apache Airflow backfills are slow and risky

9 posts from 9 people on GitHub issues.

Apache Airflow login, SSO or OAuth breaks

7 posts from 4 people on GitHub issues.

Apache Airflow users keep filing bugs

Top post: "Status of testing Providers that were prepared on October 20, 2026", one of 51 posts from 39 people on GitHub issues, Reddit.

Apache Airflow jobs fail silently or break often

16 posts from 15 people on GitHub issues, Stack Overflow.

Apache Airflow upgrades break working setups

14 posts from 12 people on GitHub issues.

Debugging Apache Airflow failures is hard

9 posts from 8 people on GitHub issues.

Trouble with Apache Airflow backend

Top post: "Trigger-written XComs bypass the configured XCom backend (raw rows mixed with externalized rows)", one of 4 posts from 4 people on GitHub issues.

Trouble with Apache Airflow dags

Top post: "DagModel.owners falls back to 'airflow' for Dags with only mapped tasks (LazyDeserializedDAG.owner ignores", one of 4 posts from 3 people on GitHub issues.

Apache Airflow permissions are confusing and error-prone

4 posts from 4 people on GitHub issues.

Apache Airflow lock-in makes switching costly

3 posts from 3 people on GitHub issues.

Trouble with Apache Airflow view

Top post: "Add Default DAG View preference to User Settings (Grid / Graph / Gantt)", one of 3 posts from 3 people on GitHub issues.

Trouble with Apache Airflow state

Top post: "Task state store writes from `on_retry_callback` fail with 404", one of 3 posts from 3 people on GitHub issues.

History

First seen with 2 posts.

Nov 22, 2026