Data and AI struggles

1,730 problems · 342 companies

Data engineering

Upstream schema changes break downstream pipelines

6 posts from 6 people5d activeFading, +0 this week

Posts per day

Posts per day6 posts, Sep 6 to Nov 9

The posts behind it

GitHub issuesmlflow/mlflow · avanish-gargOct 23

[BUG] SqlAlchemyStore stores NaN logged-model metrics as 0.0, so a diverged model matches `metrics.loss < 0.1` and sorts as the best

> [!WARNING] > Before submitting a PR, please make sure that: > - A maintainer has triaged this issue and applied the `ready` label > - This issue has no assignee > - No duplicate PR exists > > PRs not meeting these requirements may be automatically closed. ### Issues Policy...

2 replies
Redditr/databricks · u/Bhanuprakash_1947Oct 18

Have you ever changed a Delta table schema and only discovered the impact later?

...or changes its data type, the table itself might be fine, but downstream jobs can start failing or producing unexpected results.. How do you guys find all the dependent jobs and tables before making schema changes.. . Is Unity Catalog lineage enough in real projects or do you...

GitHub issuesCodeWithCJ/SparkyFitness · caillouxOct 13

[Feature]: Intervals.icu integration (pull activities + wellness, optional nutrition push)

### Are you willing to submit a PR? Yes, I'd like to submit a PR! ### What problem does this solve? I (and others) already send device data (Garmin, COROS, Polar, Wahoo, Zwift) to Intervals.icu. Today they have to connect each device to SparkyFitness separately, and not all of...

4 replies
GitHub issuesCodeWithCJ/SparkyFitness · DUC750Oct 10

[Feature]: Read mindfulness sessions from Health Connect and HealthKit

...example, Calm has write access to mindfulness in Health Connect, but SparkyFitness has no way to read what it records. #1786 asked for exactly this import; the closing comment there says the Health Connect / HealthKit sync would follow once the feature is stable, and the...

1 reactions1 replies
GitHub issuesduckdb/duckdb · gavinwahlOct 10

TOCTOU in postgres schema check; Concurrent schema modification results in silently incorrect results

### What happens? When reading from postgres, DuckDB first reads the table schema, then starts reading from it. If the schema changes between these two actions, DuckDB will read incorrect data or error. ### To Reproduce This is consistently reproducible: Setup: Then in DuckDB:...

1 replies
Redditr/PowerBI · u/TermRemarkable665Sep 22

Built a Power Query setup that handles schema changes instead of hoping next month’s file is identical

...monthly Power Query setup to make it a bit more resilient against source data drift and messy exports.. ​Instead of just covering standard basic cleanup or trying to build a "bulletproof" query that never breaks, the goal was to teach the query when to safely adapt vs. when it...

Companies and products named

2 posts
1 post
1 post
1 post
About this problem

Evidence

6 posts from 6 people in 5 places, about 1 a week over 32 days. Mostly on CodeWithCJ/SparkyFitness, r/PowerBI, duckdb/duckdb. Tools named alongside: GitHub, Excel, Power BI, PostgreSQL.

Frustration Frustration 0 of 3· Seen on GitHub issues, Reddit

How it was grouped

Posts that state a pain and match the "schema-drift" rule. First post Sep 22, 2026, latest Oct 23, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Apache Iceberg schema changes break downstream pipelines

3 posts from 3 people on GitHub issues.

Databricks schema changes break downstream pipelines

3 posts from 3 people on GitHub issues, Reddit.

Airbyte schema changes break downstream pipelines

2 posts from 2 people on GitHub issues. Unverified.

DuckDB schema changes break downstream pipelines

2 posts from 2 people on GitHub issues. Unverified.

Apache Flink schema changes break downstream pipelines

2 posts from 2 people on GitHub issues. Unverified.

ClickHouse schema changes break downstream pipelines

2 posts from 1 person on GitHub issues. Unverified.

History

Momentum went from steady to fading.

Oct 30, 2026

Momentum went from fading to steady.

Oct 26, 2026

Momentum went from steady to fading.

Oct 19, 2026

Momentum went from rising to steady.

Oct 17, 2026

Momentum went from fading to rising.

Oct 15, 2026

Momentum went from steady to fading.

Oct 14, 2026

Momentum went from fading to steady.

Oct 12, 2026

Momentum went from steady to fading.

Oct 9, 2026

Corroborated: 3 or more posts from 2 or more people or places.

Oct 7, 2026

Momentum went from fading to steady.

Oct 7, 2026

Momentum went from steady to fading.

Oct 2, 2026

Momentum went from fading to steady.

Sep 27, 2026