Problemsby GlyphApache Iceberg schema changes break downstream pipelines
3 posts from 3 people3d activeFading, +0 this week
Posts per day
The posts behind it
[Umbrella] Nightly fuzzer failures: SQLancer, SQLancer++ and the data lake fuzzer
...`tests/datalake_fuzzer/`). Each failure report has the writer, the seed, the failing check with query, settings and row diff, the shrunk operation list and a replay command; the minimal histories are in `datalake_fuzzer_logs.tar.zst`. How to handle a red run: 1. For each...
Key per-column table properties by field ID instead of column name
### Background Several table properties are keyed by **column name**, e.g.: - `write.parquet.compression-codec.column. ` / `write.parquet.compression-level.column. ` (added in #16094) - `write.parquet.bloom-filter-enabled.column. `, `...bloom-filter-fpp.column. `,...
[destination-starrocks] New Destination: StarRocks (Kotlin, Bulk CDK) — from discussion #53631
...(Feb 2025) requested a StarRocks destination. The suggested workaround (Iceberg / S3 Data Lake destinations) only covers clusters that sit behind an Iceberg catalog; it does not load into native StarRocks tables (PRIMARY KEY upsert/delete, Stream Load). In Feb 2026...
Companies and products named
About this problem
Evidence
3 posts from 3 people in 3 places, about 1 a week over 45 days. Mostly on airbytehq/airbyte, apache/iceberg, ClickHouse/ClickHouse. Tools named alongside: ClickHouse, Airbyte, PostgreSQL, MySQL.
Frustration Frustration 0 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "schema-drift" rule about Apache Iceberg. First post Sep 18, 2026, latest Nov 1, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Upstream schema changes break downstream pipelines
6 posts from 6 people on GitHub issues, Reddit.
Databricks schema changes break downstream pipelines
3 posts from 3 people on GitHub issues, Reddit.
Airbyte schema changes break downstream pipelines
2 posts from 2 people on GitHub issues. Unverified.
DuckDB schema changes break downstream pipelines
2 posts from 2 people on GitHub issues. Unverified.
Apache Flink schema changes break downstream pipelines
2 posts from 2 people on GitHub issues. Unverified.
ClickHouse schema changes break downstream pipelines
2 posts from 1 person on GitHub issues. Unverified.
Other Apache Iceberg problems
Trouble with Apache Iceberg table
Top post: "Parquet files written with the default CRS are not compatible with Iceberg Java EPSG:4326 table", one of 12 posts from 8 people on GitHub issues.
Apache Iceberg users keep filing bugs
Top post: "ORC: Writer accepts an empty struct that reader cannot bind", one of 11 posts from 10 people on GitHub issues, Reddit.
Trouble with Apache Iceberg spark
Top post: "Spark: WAP snapshot isolation broken for format-version 3 tables when deletion vectors are used (spark.wap.id)", one of 7 posts from 7 people on GitHub issues.
Apache Iceberg upgrades break working setups
6 posts from 5 people on GitHub issues.
Apache Iceberg runs out of memory
4 posts from 3 people on GitHub issues.
Trouble with Apache Iceberg rest
Top post: "Core: Negative delete-file-references in REST scan response cause IndexOutOfBoundsException", one of 4 posts from 3 people on Reddit, GitHub issues.
Apache Iceberg connectors and syncs are fragile
3 posts from 3 people on GitHub issues.
Trouble with Apache Iceberg delete
Top post: "Iceberg DROP PARTITION aborts or republishes parent delete counters after an overwrite snapshot", one of 3 posts from 3 people on GitHub issues.
Trouble with Apache Iceberg tables
Top post: "Iceberg tables are much slower than MergeTree in the performance tests: subcolumns, sort order, join", one of 3 posts from 3 people on GitHub issues.
Bad data slips through Apache Iceberg unnoticed
3 posts from 3 people on GitHub issues.
Apache Iceberg deprecations force unplanned migrations
2 posts from 2 people on GitHub issues. Unverified.
Trouble with Apache Iceberg pruning
Top post: "API: Predicate evaluation and file pruning treat -0.0 and 0.0 as different values", one of 2 posts from 2 people on GitHub issues. Unverified.
History