Problemsby IbisApache Flink tables suffer from small files and skew
Data engineering2 posts from 2 people2d active+00 posts in the last 7 days, 0 the 7 days before (fading)Unverified
Posts per day
The posts behind it
2, newest firstFlink: DynamicCommitter retry after an applied REST commit runs `cleanAll()` and deletes the committed snapshot's manifest list and manifests### Apache Iceberg version 1.11.0. That release includes `MaxCommittedCheckpointIdValidator` from #14517 (ported to `flink/v2.0` in #14637), and the bug is in how that validator interacts with the commit retry loop. The build we ran also cherry-picks #16648 into...apache/iceberg · rayhondo1 repliesOpen ↗Improve spark compaction scalability...performance for the table. When Flink job is under spikes, the Spark compactions are failing with OOM. **Root cause:** Compaction is using Spark iceberg reader, which loads equality deletes into memory in order to build a local filter. In hot partitions, there are huge amount of...apache/iceberg · qinghui-xu1 reactionsOpen ↗
Companies and products named
About this problem
Evidence
2 posts from 2 people in 1 place, about 1 a week over 13 days. Mostly on apache/iceberg. Tools named alongside: Apache Iceberg.
Frustration Frustration 0 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "file-formats" rule about Apache Flink. First post Sep 24, 2026, latest Oct 6, 2026. Unverified: fewer than 3 posts, or one person or place only. Method
The same problem elsewhere
Small files, skew and partitioning hurt performanceClaude Code tables suffer from small files and skewOpenAI tables suffer from small files and skewModel Context Protocol tables suffer from small files and skewVercel tables suffer from small files and skewGitHub tables suffer from small files and skewChatGPT tables suffer from small files and skewClaude tables suffer from small files and skew
Other Apache Flink problems
History
steady to fading
Apache Flink tables suffer from small files and skew (2 posts)