Data engineering problems

11 problems, longest running first. The list grows every day.

Problem	Companies	Active	Seen on	This week
pandas users keep filing bugspandas, GitHub, PostgreSQL, SQLite, Prefect11d activeSeen on GitHub issues, Stack Overflow	pandas, GitHub, PostgreSQL, SQLite, Prefect	11d activesince Aug 27	Seen on GitHub issues, Stack Overflow	+33 posts in the last 7 days, 2 the 7 days before (steady)
Prefect users keep filing bugsPrefect, GitHub, Google Cloud, Nebius, Kubernetes9d activeSeen on GitHub issues	Prefect, GitHub, Google Cloud, Nebius, Kubernetes	9d activesince Aug 27	Seen on GitHub issues	+44 posts in the last 7 days, 2 the 7 days before (rising)
Small files, skew and partitioning hurt performanceGitHub, Apache Iceberg, llama.cpp, Claude Code, Microsoft Azure, Polars9d activeSeen on GitHub issues, Reddit	GitHub, Apache Iceberg, llama.cpp, Claude Code, Microsoft Azure, Polars	9d activesince Aug 28	Seen on GitHub issues, Reddit	+33 posts in the last 7 days, 0 the 7 days before (rising)
Airbyte users keep filing bugsAirbyte, GitHub, Kubernetes, Google Cloud, Apache Superset, AWS8d activeSeen on GitHub issues	Airbyte, GitHub, Kubernetes, Google Cloud, Apache Superset, AWS	8d activesince Sep 1	Seen on GitHub issues	+22 posts in the last 7 days, 3 the 7 days before (steady)
Migrating between platforms is slow and riskyNotion, Kubernetes, Elasticsearch, vLLM, Hugging Face, GitHub8d active+2Seen on Reddit, GitHub issues, App Store reviews, Hacker News, Community forums	Notion, Kubernetes, Elasticsearch, vLLM, Hugging Face, GitHub	8d activesince Aug 31	+1Seen on Reddit, GitHub issues, App Store reviews, Hacker News, Community forums	+33 posts in the last 7 days, 2 the 7 days before (steady)
Apache Iceberg users keep filing bugsApache Iceberg, GitHub, Amazon S3, Docker7d activeSeen on GitHub issues	Apache Iceberg, GitHub, Amazon S3, Docker	7d activesince Aug 30	Seen on GitHub issues	+22 posts in the last 7 days, 0 the 7 days before (rising)
Bad data slips into production unnoticedApache Iceberg, Snowflake, Excel, pandas, Apache Airflow, PostgreSQL7d active+1Seen on GitHub issues, DEV, Reddit, Stack Overflow	Apache Iceberg, Snowflake, Excel, pandas, Apache Airflow, PostgreSQL	7d activesince Sep 1	Seen on GitHub issues, DEV, Reddit, Stack Overflow	+33 posts in the last 7 days, 0 the 7 days before (rising)
Databricks users keep asking for helpDatabricks, Apache Flink, Delta Lake, Microsoft Azure, Azure Data Factory7d activeSeen on Reddit, GitHub issues	Databricks, Apache Flink, Delta Lake, Microsoft Azure, Azure Data Factory	7d activesince Sep 1	Seen on Reddit, GitHub issues	+33 posts in the last 7 days, 1 the 7 days before (rising)
Snowflake users keep asking for helpSnowflake, dbt, Apache Spark, Apache Airflow, AWS, Google Cloud5d activeSeen on Stack Overflow, Reddit, GitHub issues	Snowflake, dbt, Apache Spark, Apache Airflow, AWS, Google Cloud	5d activesince Aug 27	Seen on Stack Overflow, Reddit, GitHub issues	+44 posts in the last 7 days, 0 the 7 days before (rising)
Apache Flink users keep filing bugsApache Flink, Delta Lake, Apache Iceberg, GitHub5d activeSeen on GitHub issues, Reddit	Apache Flink, Delta Lake, Apache Iceberg, GitHub	5d activesince Aug 28	Seen on GitHub issues, Reddit	+22 posts in the last 7 days, 0 the 7 days before (rising)
Trouble with Prefect flowPrefect, GitHub5d activeSeen on GitHub issues	Prefect, GitHub	5d activesince Aug 28	Seen on GitHub issues	+11 post in the last 7 days, 2 the 7 days before (steady)

Data engineering problems

Rising problems by email