Problemsby IbisApache Iceberg tables suffer from small files and skew
Data engineering2 posts from 2 people2d active+11 post in the last 7 days, 0 the 7 days before (steady)Unverified
Posts per day
The posts behind it
2, newest firstSpark: node-local shuffle for pseudo-SPJ...via `MERGE INTO` with `id` as the only join key. Considering shuffle is the biggest bottleneck on the performance, I'd like to start adopting SPJ, allowing SPJ on subset of partition keys. However, with `preserve-data-grouping` enabled, Spark will load all files that make up the...apache/iceberg · ammarchalifahOpen ↗Support encryption in `rewrite_manifests`### Feature Request / Improvement Rewrite manifests writes plaintext manifests - this tracks supporting encryption for manifest compaction. ### Query engine None ### Willingness to contribute - [x] I can contribute this improvement/feature independently - [ ] I would be willing...apache/iceberg · smaheshwar-pltr1 repliesOpen ↗
Companies and products named
About this problem
Evidence
2 posts from 2 people in 1 place, about 1 a week over 44 days. Mostly on apache/iceberg.
Frustration Frustration 0 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "file-formats" rule about Apache Iceberg. First post Sep 7, 2026, latest Oct 20, 2026. Unverified: fewer than 3 posts, or one person or place only. Method
The same problem elsewhere
Small files, skew and partitioning hurt performanceClaude Code tables suffer from small files and skewDatabricks tables suffer from small files and skewOpenAI tables suffer from small files and skewChatGPT tables suffer from small files and skewModel Context Protocol tables suffer from small files and skewApache Flink tables suffer from small files and skewGitHub tables suffer from small files and skewVercel tables suffer from small files and skewCodex tables suffer from small files and skewClaude tables suffer from small files and skew
Other Apache Iceberg problems
Apache Iceberg upgrades break working setupsTrouble with Apache Iceberg sparkApache Iceberg users keep filing bugsTrouble with Apache Iceberg tableApache Iceberg runs out of memoryTrouble with Apache Iceberg restTrouble with Apache Iceberg deleteApache Iceberg connectors and syncs are fragileTrouble with Apache Iceberg coreApache Iceberg schema changes break downstream pipelinesApache Iceberg deprecations force unplanned migrationsTrouble with Apache Iceberg rows
History
Apache Iceberg tables suffer from small files and skew (2 posts)