Debugging orchestration and DAG failures is hard
5 posts from 5 people5d activeFading, +0 this week
Posts per day
The posts behind it
Feature: Bind Dag tags into task log context natively, so every task log line carries them
...and filtering (e.g. "send this team its failed-task logs") can't use Dag tags. The workaround today is a listener that calls `structlog.contextvars.bind_contextvars(...)` in `on_task_instance_running`. We run this in production and it mostly works, but it has two gaps that can't...
[BUG] Skipped ConditionalTask passes previous-run output to downstream context
### Description When the same `Crew` instance runs twice, a `ConditionalTask` that executes in the first run and is skipped in the second retains its previous `task.output`. A downstream task with explicit `context=[conditional_task]` then receives that previous output in its...
Runner SIGTERM handler leaves astop unawaited and waits for served flow subprocesses to finish
### Bug summary Sending SIGTERM to the parent process running `flow.serve()` emits `RuntimeWarning: coroutine 'Runner.astop' was never awaited`. The parent then stays alive until the running flow finishes naturally. The flow is not cancelled. Reproduced twice with Prefect 3.8.7...
Proposal: KAFKA CONNECT: JMX metrics for the iceberg-kafka-connect commit pipeline
...task happens to hold the coordinator role. There is no time series to alert on, and no way to see whether commits are getting slower before they start failing outright. 2. **Partial vs. full commits.** A timeout-driven partial commit (`iceberg.control.commit.timeout-ms`) means...
`dbt source freshness` logs are not associated with the enclosing task
### Bug summary When calling `PrefectDbtRunner.invoke(['source', 'freshness', ...])` inside a Prefect task, logs from source node events appear in the **flow run** only - not in the task that called `invoke()`. ### Steps to reproduce Source freshness logs are missing from the...
Companies and products named
About this problem
Evidence
5 posts from 5 people in 4 places, about 1 a week over 61 days. Mostly on PrefectHQ/prefect, apache/iceberg, crewAIInc/crewAI. Tools named alongside: Prefect, GitHub, Observe, dbt.
Frustration Frustration 2 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "orchestration-debugging" rule. First post Aug 31, 2026, latest Oct 30, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Debugging Apache Airflow failures is hard
7 posts from 6 people on GitHub issues.
History