Data and AI struggles

1,787 problems · 347 companies

Data engineering

Trouble with Apache Iceberg table

12 posts from 8 people6d activeFading, +0 this week

Posts per day

Posts per day12 posts, Sep 7 to Nov 11

The posts behind it

GitHub issuesapache/iceberg · paleolimbotOct 28

Parquet files written with the "default" CRS are not compatible with Iceberg Java EPSG:4326 table

### Apache Iceberg version main (development) ### Query engine None ### Please describe the bug 🐞 After https://github.com/apache/iceberg/pull/17578, Parquet types are validated against the Iceberg type. This is great! In an attempt to reduce the number of readers that had to...

2 reactions3 replies
GitHub issuespola-rs/polars · kdn36Oct 28

SQL: `PARTITION BY` in `CREATE TABLE` for Iceberg catalog tables

### Description Support a partition spec in `CREATE TABLE` on an attached Iceberg catalog. Expected: the table is created with that partition spec (Iceberg's `year`, `month`, `day`, `hour`, `bucket(N, col)` and `truncate(W, col)` transforms), and the rows of `AS SELECT` are...

1 reactions
GitHub issuesapache/iceberg · LuciferYangOct 28

Core: Expiring unused schemas and adding a column in one transaction leaves REST table metadata unloadable

...and only schema 0. The `ops.current()` call at the end of `CatalogHandlers.commit` then fails to read it back: The client sees a failed commit, but the metadata pointer has already moved, so every later `loadTable` fails with the same error. The partition spec version (add a...

GitHub issuespola-rs/polars · djouallahOct 22

Parity gap: SQL time travel on an Iceberg table (FOR VERSION AS OF / FOR SYSTEM_TIME AS OF)

...`FOR SYSTEM_TIME AS OF '2100-01-01 00:00:00'` and `VERSION AS OF ` fail to parse as well. `polars-sql` already refuses a parsed version qualifier with "table version (time-travel) qualifiers are not supported". Expected: the query reads the table as of that snapshot (or...

1 replies
GitHub issuesduckdb/duckdb · haoxu0Oct 12

System table functions enumerate every schema even when the query pins one — O(schemas) RPCs on remote catalogs

...currently have none, so it is a design decision rather than a local change. ### Aside: hard to diagnose from the outside Neither `SET enable_http_logging=true` (no file produced) nor `SET enable_logging=true; SET enabled_log_types='HTTP'` followed by `SELECT * FROM duckdb_logs`...

GitHub issuespola-rs/polars · djouallahOct 12

SQL: CREATE TABLE in an Iceberg catalog

### Description I would like `CREATE TABLE` in Polars SQL to create a real table in an Iceberg catalog. Today `CREATE TABLE` only registers a frame in the `SQLContext`. Nothing is written to a catalog. Expected: the table exists in the catalog, and other engines (Spark, DuckDB)...

1 replies
GitHub issuespola-rs/polars · djouallahOct 12

SQL: INSERT INTO an Iceberg table

### Description I would like Polars SQL to support `INSERT INTO`, with `VALUES` and with `SELECT`, and to commit the rows to an Iceberg table. Today `INSERT` is not supported at all: `statement type is not supported`. Expected: one append commit to the Iceberg table per...

GitHub issuespola-rs/polars · djouallahOct 12

SQL: DROP TABLE in an Iceberg catalog

### Description I would like `DROP TABLE` in Polars SQL to drop the table from an Iceberg catalog. Today `DROP TABLE` only removes the name from the `SQLContext`. The table stays in the catalog. `DROP ... PURGE` is refused. Expected: the table is gone from the catalog. With...

GitHub issuespola-rs/polars · djouallahOct 12

SQL: DELETE FROM an Iceberg table

### Description I would like `DELETE FROM ... WHERE ...` in Polars SQL to delete the rows in an Iceberg table and commit. Today `DELETE` only returns a filtered frame. Nothing is written to the table. Expected: one commit to the Iceberg table (delete files, or rewritten data...

GitHub issuesapache/iceberg · ggershinskySep 24

Procedure for re-wrapping table encryption keys

### Feature Request / Improvement All encryption keys in a table, except for the master key, are wrapped (encrypted) by a parent key. Some security protocols require periodic re-wrapping with a fresh version of the parent keys. The period is typically long, one year or above. We...

1 replies
GitHub issuesClickHouse/ClickHouse · clickgapaiSep 22

Server abort on SELECT from Iceberg table with unparsable sort-order transform

### Describe what's wrong - Reading an Iceberg table aborts the server process with SIGABRT during query analysis, before any data is read - Trigger is a `sort-orders` field whose transform string ClickHouse cannot parse, for example `bucket[xyz]` in `metadata.json` - Expected a...

GitHub issuesapache/iceberg · smaheshwar-pltrSep 7

`rewrite_table_path` produces an unreadable copy of an encrypted table

### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 Currently, `rewrite_table_path` produces an unreadable copy of an encrypted table. The procedure completes normally and returns a file list, but reading the registered target fails: Root cause:...

Companies and products named

9 posts
6 posts
6 posts
About this problem

Evidence

12 posts from 8 people in 4 places, about 2 a week over 52 days. Mostly on pola-rs/polars, apache/iceberg, ClickHouse/ClickHouse. Tools named alongside: GitHub, DuckDB, Polars, ClickHouse.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts about Apache Iceberg with no known issue, sharing the word "table". First post Sep 7, 2026, latest Oct 28, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

Other Apache Iceberg problems

Apache Iceberg users keep filing bugs

Top post: "ORC: Writer accepts an empty struct that reader cannot bind", one of 11 posts from 10 people on GitHub issues, Reddit.

Trouble with Apache Iceberg spark

Top post: "Spark: WAP snapshot isolation broken for format-version 3 tables when deletion vectors are used (spark.wap.id)", one of 7 posts from 7 people on GitHub issues.

Apache Iceberg upgrades break working setups

6 posts from 5 people on GitHub issues.

Apache Iceberg runs out of memory

4 posts from 3 people on GitHub issues.

Trouble with Apache Iceberg rest

Top post: "Core: Negative delete-file-references in REST scan response cause IndexOutOfBoundsException", one of 4 posts from 3 people on Reddit, GitHub issues.

Apache Iceberg schema changes break downstream pipelines

3 posts from 3 people on GitHub issues.

Apache Iceberg connectors and syncs are fragile

3 posts from 3 people on GitHub issues.

Trouble with Apache Iceberg delete

Top post: "Iceberg DROP PARTITION aborts or republishes parent delete counters after an overwrite snapshot", one of 3 posts from 3 people on GitHub issues.

Trouble with Apache Iceberg tables

Top post: "Iceberg tables are much slower than MergeTree in the performance tests: subcolumns, sort order, join", one of 3 posts from 3 people on GitHub issues.

Bad data slips through Apache Iceberg unnoticed

3 posts from 3 people on GitHub issues.

Apache Iceberg deprecations force unplanned migrations

2 posts from 2 people on GitHub issues. Unverified.

Trouble with Apache Iceberg pruning

Top post: "API: Predicate evaluation and file pruning treat -0.0 and 0.0 as different values", one of 2 posts from 2 people on GitHub issues. Unverified.

History

Momentum went from rising to fading.

Nov 4, 2026

Momentum went from steady to rising.

Nov 3, 2026

Momentum went from fading to steady.

Nov 2, 2026

Momentum went from fading to steady.

Oct 26, 2026

Momentum went from rising to fading.

Oct 19, 2026

Momentum went from fading to rising.

Oct 15, 2026

Momentum went from rising to fading.

Oct 2, 2026

First seen with 3 posts; the top post now: "Parquet files written with the default CRS are not compatible with Iceberg Java EPSG:4326 table".

Sep 27, 2026