Skip to content

Apache Spark

Open source project7 problems16 posts+5 this week

Problem posts per week

Problem posts per week28 posts, week of Aug 24 to week of Sep 21

Problems

ProblemThis week
Hard to trace lineage and find trusted tablesData engineeringDagster, Apache Spark, Anthropic, OpenAI, Perplexity, DeepSeek4d activeSeen on GitHub issues+22 posts in the last 7 days, 0 the 7 days before (rising)
Snowflake users keep asking for helpData engineeringSnowflake, dbt, Apache Spark, Apache Airflow, AWS, Google Cloud5d activeSeen on Stack Overflow, Reddit, GitHub issues+44 posts in the last 7 days, 0 the 7 days before (rising)
Vector search quality and scaling are hard to get rightDatabasesDatabricks, Snowflake, Apache Spark, Apache Kafka, AWS, Lambda5d activeSeen on Reddit, GitHub issues, Hacker News+11 post in the last 7 days, 2 the 7 days before (steady)
Apache Spark users keep filing bugsData engineeringApache Spark, PostgreSQL, AWS, DuckDB, Databricks4d activeSeen on GitHub issues, Reddit+11 post in the last 7 days, 0 the 7 days before (steady)
Apache Spark upgrades break working setupsOutages and changeApache Spark, Polars, DuckDB, PostgreSQL3d activeSeen on GitHub issues+11 post in the last 7 days, 0 the 7 days before (steady)
BigQuery users keep asking for helpData engineeringBigQuery, GitHub, Apache Spark, Docker, Claude Code, Apache Airflow4d activeSeen on Stack Overflow, GitHub issues+00 posts in the last 7 days, 0 the 7 days before (fading)
Great Expectations users keep asking for helpData engineeringGreat Expectations, Databricks, Apache Spark1d activeSeen on Stack OverflowUnverified+00 posts in the last 7 days, 1 the 7 days before (fading)
About

Problems here are about Apache Spark or name it. It is tracked from the product list on the Method page. Website: spark.apache.org.

Rising problems by email

Mondays: the problems in data, tech and AI that grew fastest that week.

Double opt-in. Unsubscribe any time.