Data and AI struggles

1,730 problems · 342 companies

Data engineering

Apache Airflow backfills are slow and risky

4 posts from 4 people4d activeFading, +0 this week

Posts per day

Posts per day4 posts, Aug 31 to Nov 9

The posts behind it

GitHub issuesapache/airflow · HydroYannOct 30

HA schedulers can start backfill DagRuns out of order with max_active_runs=1

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.2 ### What happened and how to reproduce it? Two scheduler instances can promote a later backfill DagRun before earlier queued runs even when the backfill's `max_active_runs=1` and...

2 replies
GitHub issuesapache/airflow · yuseok89Oct 9

Pick a Dag's latest run by run_after consistently in the UI and REST API

### Description The latest run of a Dag is picked in three different ways across these views. | Where | How the latest run is picked | |---|---| | Dashboard Failed Dags count (`ui/dashboard.py` `dag_stats`) | `logical_date DESC`, runs without a logical date skipped | | Dags list...

1 replies
GitHub issuesapache/airflow · xBis7Sep 9

Calendar view rework

### Description The issue is for covering the work of changing the design of the calendar view. We will start with a static mockup to get feedback from the community and then move on with the implementation. These are the current pages ### Use case/motivation The current...

3 replies
GitHub issuesapache/airflow · aeroyorchAug 31

Cancelling a backfill leaves its runs unschedulable forever

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.2.2 ### What happened and how to reproduce it? After a backfill is cancelled, clearing one of its DAG runs sends the run back to `queued`, where it stays. In this case it had still not...

Companies and products named

3 posts
2 posts
About this problem

Evidence

4 posts from 4 people in 1 place, about 1 a week over 61 days. Mostly on apache/airflow. Tools named alongside: GitHub, Docker, Kubernetes, PostgreSQL.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "backfills" rule about Apache Airflow. First post Aug 31, 2026, latest Oct 30, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Backfills are slow, costly and risky

10 posts from 10 people on GitHub issues, Reddit.

Dagster backfills are slow and risky

4 posts from 4 people on GitHub issues.

PostgreSQL backfills are slow and risky

3 posts from 3 people on GitHub issues.

Codex backfills are slow and risky

2 posts from 1 person on GitHub issues. Unverified.

MLflow backfills are slow and risky

2 posts from 2 people on GitHub issues. Unverified.

Claude backfills are slow and risky

2 posts from 2 people on GitHub issues. Unverified.

Other Apache Airflow problems

Apache Airflow users keep filing bugs

Top post: "Status of testing Providers that were prepared on October 20, 2026", one of 41 posts from 31 people on GitHub issues, Reddit.

Apache Airflow is too complex to run well

34 posts from 32 people on GitHub issues.

Apache Airflow jobs fail silently or break often

13 posts from 12 people on GitHub issues, Stack Overflow.

Apache Airflow upgrades break working setups

11 posts from 10 people on GitHub issues.

Debugging Apache Airflow failures is hard

7 posts from 6 people on GitHub issues.

Apache Airflow login, SSO or OAuth breaks

4 posts from 3 people on GitHub issues.

Trouble with Apache Airflow retry

Top post: "Task state store writes from `on_retry_callback` fail with 404", one of 3 posts from 3 people on GitHub issues.

Trouble with Apache Airflow dags

Top post: "DagModel.owners falls back to 'airflow' for Dags with only mapped tasks (LazyDeserializedDAG.owner ignores", one of 3 posts from 3 people on GitHub issues.

Trouble with Apache Airflow backend

Top post: "Support [workers] state_store_backend in language SDK tasks", one of 3 posts from 3 people on GitHub issues.

Apache Airflow lock-in makes switching costly

2 posts from 2 people on GitHub issues. Unverified.

Apache Airflow runs out of memory

2 posts from 2 people on GitHub issues. Unverified.

Apache Airflow permissions are confusing and error-prone

2 posts from 2 people on GitHub issues. Unverified.

History

Momentum went from steady to fading.

Nov 6, 2026

Momentum went from fading to steady.

Nov 4, 2026

Momentum went from steady to fading.

Oct 16, 2026

Corroborated: 3 or more posts from 2 or more people or places.

Oct 10, 2026

Momentum went from fading to steady.

Oct 10, 2026

First seen with 2 posts.

Sep 25, 2026