Data and AI struggles

2,149 problems · 379 companies

Data engineering

Hard to trace lineage and find trusted data in Apache Iceberg

3 posts from 3 people3d activeRising, +2 this week

Posts per day

Posts per day3 posts, Oct 11 to Nov 22

The posts behind it

GitHub issuesapache/iceberg · rishvinNov 20

GCSInputFile.getLength throws NullPointerException for missing blobs

### Apache Iceberg version 1.11.0 ### Query engine Spark ### Please describe the bug 🐞 I created an iceberg table with Polaris as the catalog, and then manually deleted the underlying manifest-list file from the Google-cloud-storage. Therefore, the manifest-list location was...

GitHub issuesapache/iceberg · regadasNov 16

Core: Expose status-filtered CloseableIterables on ManifestReader

### Feature Request / Improvement `Snapshot.addedDataFiles(FileIO)`, `removedDataFiles(FileIO)`, `addedDeleteFiles(FileIO)`, and `removedDeleteFiles(FileIO)` were deprecated in #15241 in favor of `SnapshotChanges`. However, `SnapshotChanges` eagerly materializes both added and...

GitHub issuesapache/iceberg · maomaodevOct 11

Spark: scala.MatchError in `RewriteOperationForRowLineage` for non-Iceberg tables

### Apache Iceberg version 1.10.1 ### Query engine Spark ### Please describe the bug 🐞 The rules `RewriteUpdateTableForRowLineage` and `RewriteMergeIntoTableForRowLineage` (both extending `RewriteOperationForRowLineage`) are injected into the Spark Analyzer by...

Companies and products named

About this problem

Evidence

3 posts from 3 people in 1 place, about 1 a week over 41 days. Mostly on apache/iceberg. Tools named alongside: GitHub, Google Cloud.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "lineage-catalog" rule about Apache Iceberg. First post Oct 11, 2026, latest Nov 20, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Hard to trace lineage and find trusted tables

19 posts from 19 people on GitHub issues, Reddit, Hacker News.

Hard to trace lineage and find trusted data in GitHub

8 posts from 7 people on GitHub issues, Reddit.

Hard to trace lineage and find trusted data in dbt

7 posts from 7 people on GitHub issues, Reddit.

Hard to trace lineage and find trusted data in Claude Code

3 posts from 3 people on GitHub issues, Reddit.

Hard to trace lineage and find trusted data in Databricks

3 posts from 3 people on Reddit, GitHub issues.

Hard to trace lineage and find trusted data in ChatGPT

3 posts from 3 people on GitHub issues.

Hard to trace lineage and find trusted data in Dagster

2 posts from 2 people on GitHub issues. Unverified.

Hard to trace lineage and find trusted data in Apache Airflow

2 posts from 2 people on GitHub issues. Unverified.

Hard to trace lineage and find trusted data in Grafana

2 posts from 2 people on GitHub issues. Unverified.

Hard to trace lineage and find trusted data in Gemini

2 posts from 2 people on Hacker News, GitHub issues. Unverified.

Hard to trace lineage and find trusted data in BigQuery

2 posts from 1 person on GitHub issues. Unverified.

Hard to trace lineage and find trusted data in vLLM

2 posts from 2 people on GitHub issues. Unverified.

Other Apache Iceberg problems

Trouble with Apache Iceberg partition

Top post: "Iceberg DROP PARTITION aborts or republishes parent delete counters after an overwrite snapshot", one of 4 posts from 4 people on GitHub issues.

Apache Iceberg upgrades break working setups

9 posts from 8 people on GitHub issues.

Apache Iceberg users keep filing bugs

Top post: "VARIANT does not work with older <= 4.1 HMS versions", one of 11 posts from 10 people on GitHub issues, Reddit.

Trouble with Apache Iceberg table

Top post: "Iceberg: `CREATE TABLE` writes a non-standard `required` key inside `list` types, breaking the engine after", one of 15 posts from 10 people on GitHub issues.

Trouble with Apache Iceberg spark

Top post: "Spark: WAP snapshot isolation broken for format-version 3 tables when deletion vectors are used (spark.wap.id)", one of 6 posts from 6 people on GitHub issues.

Trouble with Apache Iceberg rest

Top post: "Core: Negative delete-file-references in REST scan response cause IndexOutOfBoundsException", one of 6 posts from 5 people on GitHub issues, Reddit.

Apache Iceberg runs out of memory

4 posts from 3 people on GitHub issues.

Trouble with Apache Iceberg rows

Top post: "Parquet TIMESTAMP_MILLIS stats pruning can drop matching rows", one of 3 posts from 3 people on GitHub issues.

Trouble with Apache Iceberg core

Top post: "Core: RewriteManifests.addManifest mishandles v1-format manifests on v2+ tables", one of 3 posts from 3 people on GitHub issues.

Apache Iceberg schema changes break downstream pipelines

3 posts from 3 people on GitHub issues.

Apache Iceberg connectors and syncs are fragile

3 posts from 3 people on GitHub issues.

Bad data slips through Apache Iceberg unnoticed

3 posts from 3 people on GitHub issues.

History

First seen with 3 posts.

Nov 22, 2026