Data and AI struggles

1,483 problems · 320 companies

Data engineering

Apache Kafka tables suffer from small files and skew

2 posts from 2 people2d activeFading, +0 this weekunverified

Posts per day

Posts per day2 posts, Oct 15 to Nov 2

The posts behind it

Redditr/apachekafka · u/ScheduledSilenceOct 19

Need advise on partitioning strategy for multi-tenancy

...data isolation needn't be very strict.. We need to effectively solve noisy neighbor problem with zero data loss is must. . Onboarding a tenant should've no downtime as well and a tenant if growing big should be allowed to be promoted or moved.. Have 2 options in mind:. Custom...

GitHub issuesapache/iceberg · skaushik15Oct 15

Plan to support table maintaence in kafka connect?

### Feature Request / Improvement Hey, Just curious, why not enhance kafka connector and include the compaction, snapshot expiry etc on the kafka connect itself? Connector's tasks are already running on the connect workers(compute), why not enhance the coordinator to start a...

Companies and products named

About this problem

Evidence

2 posts from 2 people in 2 places, about 2 a week over 5 days. Mostly on apache/iceberg, r/apachekafka. Tools named alongside: Apache Iceberg.

Frustration Frustration 0 of 3· Seen on GitHub issues, Reddit

How it was grouped

Posts that state a pain and match the "file-formats" rule about Apache Kafka. First post Oct 15, 2026, latest Oct 19, 2026. Unverified: fewer than 3 posts, or one person or place only. Method

The same problem elsewhere

Small files, skew and partitioning hurt performance

13 posts from 13 people on GitHub issues.

Claude Code tables suffer from small files and skew

8 posts from 8 people on GitHub issues, Reddit.

Codex tables suffer from small files and skew

4 posts from 4 people on GitHub issues.

OpenAI tables suffer from small files and skew

3 posts from 3 people on GitHub issues.

Databricks tables suffer from small files and skew

3 posts from 3 people on Reddit, GitHub issues.

ChatGPT tables suffer from small files and skew

3 posts from 3 people on GitHub issues.

Apache Flink tables suffer from small files and skew

2 posts from 2 people on GitHub issues. Unverified.

Apache Iceberg tables suffer from small files and skew

2 posts from 2 people on GitHub issues. Unverified.

GitHub tables suffer from small files and skew

2 posts from 2 people on GitHub issues. Unverified.

Vercel tables suffer from small files and skew

2 posts from 2 people on GitHub issues. Unverified.

Claude tables suffer from small files and skew

2 posts from 2 people on GitHub issues. Unverified.

Other Apache Kafka problems

Apache Kafka users keep asking for help

Top post: "How to reference a file inside the current jar to be loaded in a library", one of 11 posts from 11 people on Stack Overflow, GitHub issues, Reddit.

Trouble with Apache Kafka connect

Top post: "Kafka connect runtime 1.12.0 Azure Blob writes silently hang due to netty-tcnative/netty-handler", one of 3 posts from 3 people on GitHub issues.

Apache Kafka lock-in makes switching costly

2 posts from 2 people on Reddit, GitHub issues. Unverified.

Apache Kafka streams lag, drop or reorder events

2 posts from 2 people on Mastodon, Reddit. Unverified.

History

First seen with 2 posts.

Nov 2, 2026