Data and AI struggles

1,730 problems · 342 companies

DevOps and infra

Apache Airflow is too complex to run well

34 posts from 32 people22d activeSteady, +1 this week

Posts per day

Posts per day34 posts, Aug 27 to Nov 9

The posts behind it

GitHub issuesapache/airflow · dablaNov 3

Keycloak auth manager: API returns 500 ('NoneType' object has no attribute 'access_token') when the Keycloak tokens of a browser session are missing or expired

### Under which category would you file this issue? Providers ### Apache Airflow version 3.3.2 (also reproduced against main; the code path is unchanged in 3.4.0rc with `apache-airflow-providers-keycloak` 0.11.2) ### What happened and how to reproduce it? With the Keycloak auth...

GitHub issuesapache/airflow · zach-overflowOct 29

`db clean --skip-archive` still copies deletion candidates into temporary archive tables prior to deletion.

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.2 ### What happened and how to reproduce it? On any Airflow 3.3.2 metadata database containing log rows older than the cutoff, run: You can see the following per-batch output: 1....

2 replies
GitHub issuesapache/airflow · jabberaOct 26

FAB auth: deep link redirects to home page when UI JWT expired but FAB session still valid (next is dropped)

...early-return (otherwise the SPA would immediately 401 again and loop back to login). Workaround we're using in `webserver_config.py`: ### Operating System Linux ### Deployment Other 3rd-party Helm chart / Kubernetes ### Apache Airflow Provider(s) fab ### Versions of Apache...

1 replies
GitHub issuesapache/airflow · martin-beranOct 26

Cannot enter letter "g" in Trigger Dag dialog, Configuration JSON field

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.1 ### What happened and how to reproduce it? In Dags view, select a Dag, then click on the Trigger button. In the Trigger Dag dialog, click on Advanced Options and start entering text...

1 replies
GitHub issuesapache/airflow · cbuffettOct 26

New Windows PT time zone results in UI error due to unresolved time zone

...to `selectedTimezone.toUpperCase()` where `selectedTimezone` is undefined due to being unable to resolve the OS time zone to an IANA name. Reproduction: 1. Set your Windows time zone to Pacific Time (UTC -7:00) 2. Clear browser local storage for the Airflow server 3. Log in to...

GitHub issuesapache/airflow · vuonghoangbnttOct 26

dag-processor --num-runs never exits after a Dag callback on a versioned bundle

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.1 ### What happened and how to reproduce it? **Issue Description** With a versioning bundle, `airflow dag-processor --num-runs N` never exits once it has handled a Dag callback. A...

2 replies
GitHub issuesapache/airflow · dhruv-techdevOct 24

BranchDateTimeOperator takes the false branch after midnight in an overnight time window

### Under which category would you file this issue? Providers ### Apache Airflow version main (commit 57a4751c96, airflow-core 3.4.0) ### What happened and how to reproduce it? **Issue Description** When `target_lower` / `target_upper` are `datetime.time` values and the window...

GitHub issuesapache/airflow · ChickenBennyOct 24

Serialized tasks repeat timedelta fields that match client_defaults

### Under which category would you file this issue? Airflow Core ### Apache Airflow version main ### What happened and how to reproduce it? Serialization omits a task field when it matches `client_defaults` or the schema default, but `timedelta` fields never match: the operator...

1 replies
GitHub issuesapache/airflow · AZSJTinusOct 23

Deadline Alert: No support for SmtpNotifier

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.2 ### What happened and how to reproduce it? The [DAG documentation](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/dags.html#deadline-alerts) mentions the...

1 replies
GitHub issuesapache/airflow · TJaniFOct 22

Dags button click does not lead to Dags page

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.4.0beta2 ### What happened and how to reproduce it? Tiny UI nit in the 3.4 beta 2. If you click on the Dags entry in the sidebar you don't automatically get the Dags page like in 3.3....

1 replies
GitHub issuesapache/airflow · PrabhodUrsOct 21

CeleryExecutor rejects "executeCallback" workloads, causing "SyncCallback" deadline alerts to fail

...above. The callback's `Deadline missed` log message is not emitted. ### 5. Test the workaround Downgrade `apache-airflow-providers-celery` to `3.18.0` on the scheduler and all workers, without changing the DAG or callback module. In my environment, the callback executes...

2 replies
GitHub issuesapache/airflow · rammmaaOct 16

.airflowignore rules are lost when a Dag folder symlink switches during a scan

### Under which category would you file this issue? Airflow Core ### Apache Airflow version main (91a8ab6950); initially observed on 2.11.2 ### What happened and how to reproduce it? When git-sync atomically replaces the symlink used by the Dag folder during file discovery,...

1 replies
GitHub issuesapache/airflow · devgonvarunOct 16

Metadata secret required for workers

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.2.2 ### What happened and how to reproduce it? Workers in Airflow 3 do not connect directly to the metadata db however they still require a metadata secret. Deploy a simple...

1 replies
GitHub issuesapache/airflow · sahilSENPAIIOct 15

GET /assets/events returns each event once per asset when name_prefix_pattern is ~

...because it is transformed to `%`, which is truthy, so the join is added. I have a failing test and a one-line fix ready and will open a PR. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code...

1 replies
GitHub issuesapache/airflow · atuls4441Oct 14

Clear task instance broken

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.4.0b1 ### What happened and how to reproduce it? 1. Try to clear a TI for any Dag which has a dagrun already. 2. Notice, no action done since request payload has dry_run=true Response:...

1 replies
GitHub issuesapache/airflow · atuls4441Oct 14

Dagrun state badge needs a page refresh to reflect correct state

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.4.0b1 ### What happened and how to reproduce it? On Dag list page, trigger a dag manually and see badge state is not updated even after dagrun finishes, needs page refresh to update the...

1 replies
GitHub issuesapache/airflow · AndrushikaOct 14

execution_timeout on @task.stub tasks is not enforced

### Under which category would you file this issue? Task SDK ### Apache Airflow version main ### What happened and how to reproduce it? A `@task.stub` task is declared in a Python Dag and runs in a language SDK. Its `execution_timeout` is set on the Python stub and is serialized...

3 replies
GitHub issuesapache/airflow · gavrikOct 9

Plugin 'kafka_event_producer' already registered

### Under which category would you file this issue? Providers ### Apache Airflow version 3.3.2 ### What happened and how to reproduce it? The fresh Airflow installation. Installation command pip install...

GitHub issuesapache/airflow · ashleyvanspOct 9

Gantt view instantly reverts to Grid when another browser tab is open on a page without a `run_id`

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.2 ### What happened and how to reproduce it? On a DAG run page, clicking the Gantt icon does not switch to the Gantt chart if another Airflow tab in the web browser is on a route with...

2 replies
GitHub issuesapache/airflow · per-andersenOct 8

DAG.test() fails for tasks whose task_id contains non ASCII characters

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.2, main ### What happened and how to reproduce it? Running a DAG with `DAG.test()` fails for any task whose `task_id` contains a non ASCII character, as soon as that task pushes a...

1 replies
GitHub issuesapache/airflow · LucasAnzmann-UIOct 7

Graph View layout became much more compressed and harder to read after upgrade from 3.1.5 to 3.3.2

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.2 ### What happened and how to reproduce it? After upgrading Airflow from 3.1.5 to 3.3.2, the layout of the asset/DAG dependency graph changed significantly. In Airflow 3.1.5, the...

3 replies
GitHub issuesapache/airflow · RNHTTROct 7

Docs for reproducing heartbeat timeout won't reproduce heartbeat timeout

### Under which category would you file this issue? Airflow Core ### Apache Airflow version n/a ### What happened and how to reproduce it? The [docs for reproducing heartbeat...

1 replies
GitHub issuesapache/airflow · deepnemesisOct 5

Git-Sync Containers Miss Default Security Hardening

### Under which category would you file this issue? Helm chart ### Apache Airflow version main (20f90a6e10) ### What happened and how to reproduce it? When DAG git-sync is enabled without an explicit container security context, both the git-sync sidecar and init container...

GitHub issuesapache/airflow · deepnemesisOct 5

OTel Collector Misses Default Container Security Context

### Under which category would you file this issue? Helm chart ### Apache Airflow version main (d339bb2767) ### What happened and how to reproduce it? When the OTel Collector is enabled without an explicit `otelCollector.securityContexts.container` value, its container renders...

1 replies
GitHub issuesapache/airflow · gang-zhOct 5

Grid "Number of Dag Runs" selection is lost on tab switch and not remembered since 3.3.0

### Under which category would you file this issue? UI ### Apache Airflow version 3.3.2 ### What happened and how to reproduce it? Since 3.3.0, the "Number of Dag Runs" selection on the Dag details page is lost almost immediately while browsing and is never remembered on the...

GitHub issuesapache/airflow · 1fanwangSep 27

Pool PATCH with update_mask still requires include_deferred and the pool name

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.2 ### What happened and how to reproduce it? A client can't update just the slots of a pool. `PATCH /api/v2/pools/{pool_name}?update_mask=slots` with `{"slots": 8}` returns 422...

1 replies
GitHub issuesapache/airflow · rjh-yextSep 25

Dag version inflation check runs against zip file instead of individual dags

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.1 ### What happened and how to reproduce it? Dags that are deployed as zip files do not trigger the dag version inflation check configured in dag_version_inflation_check_level because...

3 replies
GitHub issuesapache/airflow · KristoferFangratSep 22

Windows test collection fails with UnicodeDecodeError in provider dependency refresh

### Under which category would you file this issue? Airflow Core ### Apache Airflow version main checkout at commit 49674b776d ### What happened and how to reproduce it? On Windows, running a focused provider test can fail during pytest collection before any tests are collected....

6 replies
GitHub issuesapache/airflow · elingerSep 22

CelleryExecutor does not respect AIRFLOW__OPERATORS__DEFAULT_QUEUE for SyncCallback

...how to reproduce it? When I set a DAG deadline through a `SyncCallback`, the scheduler fails to execute the callback because my Airflow setup does not have a queue named default. I configure the default queue with `AIRFLOW__OPERATORS__DEFAULT_QUEUE`, but the CeleryExecutor does...

3 replies
GitHub issuesapache/airflow · emery-finkelsteinSep 15

Google Secrets Backend logs on every connection reference when using ADC

### Under which category would you file this issue? Providers ### Apache Airflow version Airflow 3.3.1 ### What happened and how to reproduce it? [Configure Google Secrets Backend...

1 reactions1 replies
GitHub issuesapache/airflow · romina-nikolovaSep 9

KubernetesExecutor: delete_pod fires before the self.running dedup check, causing repeated deletes per TaskInstanceKey

### Under which category would you file this issue? Providers ### Apache Airflow version 3.3.1 ### What happened and how to reproduce it? KubernetesExecutor issues 2-3 delete_pod API calls for the same finished worker pod instead of one. The first call succeeds; the rest return...

1 reactions1 replies
GitHub issuesapache/airflow · PolitePpSep 1

ti.get_previous_ti method is not optimized

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.3.0 ### What happened and how to reproduce it? When we execute `ti.get_previous_ti(state='success')` airflow collects all task instances from the database and then gets the latest on...

GitHub issuesapache/airflow · baha-boualiAug 31

UI: durations and relative times render in English in all non-English locales (dayjs.locale() is never set)

### Under which category would you file this issue? Airflow Core ### Apache Airflow version 3.2.0 ### What happened and how to reproduce it? The Dag deadline tooltip is only half translated. The sentence template and the reference point come out in the selected language, but the...

GitHub issuesapache/airflow · borevAug 27

BeamRunJavaPipelineOperator deferrable impersonation chain issue

### Under which category would you file this issue? Airflow Core ### Apache Airflow version Airflow 2.11.1 ### What happened and how to reproduce it? "When running _BeamRunJavaPipelineOperator_ with `deferrable=True `and `impersonation_chain='target_service_account',` I get an...

1 replies

Companies and products named

33 posts
31 posts
30 posts
2 posts
2 posts
About this problem

Evidence

34 posts from 32 people in 1 place, about 3 a week over 69 days. Mostly on apache/airflow. Tools named alongside: Kubernetes, GitHub, Docker, PostgreSQL.

Frustration Frustration 0 of 3· Seen on GitHub issues

Summary, model-written

Apache Airflow users face redundant Kubernetes pod deletion API calls, pipeline operator impersonation errors, partially translated tooltips, and inefficient database queries when fetching previous tasks.

How it was grouped

Posts that state a pain and match the "k8s-complexity" rule about Apache Airflow. First post Aug 27, 2026, latest Nov 3, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Kubernetes is too complex for most teams

13 posts from 13 people on GitHub issues, Reddit.

Kubernetes is too complex to run well

8 posts from 8 people on GitHub issues, Stack Overflow.

Claude Code is too complex to run well

5 posts from 4 people on GitHub issues.

Airbyte is too complex to run well

4 posts from 4 people on GitHub issues.

Google Cloud is too complex to run well

2 posts from 2 people on Reddit. Unverified.

Other Apache Airflow problems

Apache Airflow users keep filing bugs

Top post: "Status of testing Providers that were prepared on October 20, 2026", one of 41 posts from 31 people on GitHub issues, Reddit.

Apache Airflow jobs fail silently or break often

13 posts from 12 people on GitHub issues, Stack Overflow.

Apache Airflow upgrades break working setups

11 posts from 10 people on GitHub issues.

Debugging Apache Airflow failures is hard

7 posts from 6 people on GitHub issues.

Apache Airflow login, SSO or OAuth breaks

4 posts from 3 people on GitHub issues.

Apache Airflow backfills are slow and risky

4 posts from 4 people on GitHub issues.

Trouble with Apache Airflow retry

Top post: "Task state store writes from `on_retry_callback` fail with 404", one of 3 posts from 3 people on GitHub issues.

Trouble with Apache Airflow dags

Top post: "DagModel.owners falls back to 'airflow' for Dags with only mapped tasks (LazyDeserializedDAG.owner ignores", one of 3 posts from 3 people on GitHub issues.

Trouble with Apache Airflow backend

Top post: "Support [workers] state_store_backend in language SDK tasks", one of 3 posts from 3 people on GitHub issues.

Apache Airflow lock-in makes switching costly

2 posts from 2 people on GitHub issues. Unverified.

Apache Airflow runs out of memory

2 posts from 2 people on GitHub issues. Unverified.

Apache Airflow permissions are confusing and error-prone

2 posts from 2 people on GitHub issues. Unverified.

History

Momentum went from fading to steady.

Nov 9, 2026

Momentum went from steady to fading.

Nov 2, 2026

Momentum went from rising to steady.

Oct 31, 2026

Summary written by a model: Apache Airflow users face redundant Kubernetes pod deletion API calls, pipeline operator impersonation errors, partially translated tooltips, and inefficient database queries when fetching previous tasks.

Oct 28, 2026

Momentum went from fading to rising.

Oct 27, 2026

Momentum went from steady to fading.

Oct 21, 2026

Momentum went from fading to steady.

Oct 18, 2026

Momentum went from steady to fading.

Oct 15, 2026

Momentum went from rising to steady.

Oct 14, 2026

Momentum went from fading to rising.

Oct 10, 2026

Momentum went from rising to fading.

Oct 2, 2026

Momentum went from fading to rising.

Sep 28, 2026