Data and AI struggles

1,512 problems · 320 companies

Data engineering

Bad data slips through Apache Iceberg unnoticed

3 posts from 3 people3d activeSteady, +2 this week

Posts per day

Posts per day3 posts, Oct 22 to Nov 3

The posts behind it

GitHub issuesapache/iceberg · Revanth14Oct 29

API: Evaluator matches null values for notEq and notIn

### Feature Request / Improvement ## Problem `Evaluator` compares through literal comparators that order null first, so `eq` is false and `notEq` is true for a null value. `in` returns false for null, so `notIn` is true. Under SQL three-valued logic, `x <> v` and `x NOT IN...

1 replies
GitHub issuesapache/iceberg · rajendragOct 28

Core: Metadata DELETE can incorrectly remove NULL date partitions

### Apache Iceberg version 1.10.0 ### Query engine Spark ### Please describe the bug 🐞 For a table partitioned by a nullable date column, a metadata DELETE using `<` can incorrectly remove rows from the NULL partition. The behavior depends on the manifest layout. For example,...

GitHub issuesapache/iceberg · zhengxiang378928908-codeOct 22

Spark: metadata-only DELETE can remove NULL rows for `<>`, single-value `NOT IN` and `NOT LIKE`

### Apache Iceberg version main (`e70a2efcd`) ### Query engine Spark 4.2. The same `NOT_EQ` and `NOT` conversion paths exist in the Spark 3.5, 4.0 and 4.1 source directories, but I haven't run the reproduction on those versions. ### Please describe the bug 🐞 `DELETE FROM t...

Companies and products named

About this problem

Evidence

3 posts from 3 people in 1 place, about 2 a week over 8 days. Mostly on apache/iceberg.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "data-quality" rule about Apache Iceberg. First post Oct 22, 2026, latest Oct 29, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Bad data slips into production unnoticed

21 posts from 21 people on GitHub issues, Reddit, Stack Overflow.

Bad data slips through Polars unnoticed

3 posts from 3 people on GitHub issues, Reddit.

Bad data slips through Snowflake unnoticed

3 posts from 3 people on Reddit, Hacker News.

Bad data slips through PostgreSQL unnoticed

2 posts from 2 people on Stack Overflow. Unverified.

Bad data slips through Databricks unnoticed

2 posts from 2 people on Reddit. Unverified.

Bad data slips through Apache Spark unnoticed

2 posts from 2 people on GitHub issues. Unverified.

Other Apache Iceberg problems

Trouble with Apache Iceberg table

Top post: "Parquet files written with the default CRS are not compatible with Iceberg Java EPSG:4326 table", one of 12 posts from 8 people on GitHub issues.

Apache Iceberg upgrades break working setups

6 posts from 5 people on GitHub issues.

Apache Iceberg users keep filing bugs

Top post: "ORC: Writer accepts an empty struct that reader cannot bind", one of 10 posts from 9 people on GitHub issues.

Trouble with Apache Iceberg spark

Top post: "Spark: WAP snapshot isolation broken for format-version 3 tables when deletion vectors are used (spark.wap.id)", one of 7 posts from 7 people on GitHub issues.

Apache Iceberg runs out of memory

4 posts from 3 people on GitHub issues.

Trouble with Apache Iceberg rest

Top post: "Core: Negative delete-file-references in REST scan response cause IndexOutOfBoundsException", one of 4 posts from 3 people on Reddit, GitHub issues.

Apache Iceberg connectors and syncs are fragile

3 posts from 3 people on GitHub issues.

Trouble with Apache Iceberg delete

Top post: "Iceberg DROP PARTITION aborts or republishes parent delete counters after an overwrite snapshot", one of 3 posts from 3 people on GitHub issues.

Trouble with Apache Iceberg pruning

Top post: "API: Predicate evaluation and file pruning treat -0.0 and 0.0 as different values", one of 2 posts from 2 people on GitHub issues. Unverified.

Apache Iceberg schema changes break downstream pipelines

2 posts from 2 people on GitHub issues. Unverified.

Apache Iceberg deprecations force unplanned migrations

2 posts from 2 people on GitHub issues. Unverified.

Apache Iceberg tables suffer from small files and skew

2 posts from 2 people on GitHub issues. Unverified.

History

First seen with 3 posts.

Nov 3, 2026