Skip to content
Problems.

Apache Spark upgrades break working setups

Outages and change3 posts from 3 people3d active+11 post in the last 7 days, 0 the 7 days before (steady)

Posts per day

Posts per day3 posts, Aug 28 to Sep 25

The posts behind it

3, newest first
PostDate
Shuffle tracking prevents dynamic allocation from removing executors after `collect()` / `toPandas()`When using dynamic allocation with shuffle tracking enabled, executors that participated in a shuffle remain alive indefinitely after a `collect()` or `toPandas()` action has completed. Configuration: For example: After the action completes, the executors that produced shuffle...apache/sparkaalopatinSep 24yesterday
Add linear regression aggregation functions (regr_slope, regr_intercept, regr_r2, regr_count)### Description I'd like to propose adding the core SQL-standard **linear regression aggregation functions** to Polars: - `pl.regr_slope(y, x)` - slope of the least-squares linear fit of `y` on `x` - `pl.regr_intercept(y, x)` - intercept of the fit - `pl.regr_r2(y, x)` -...pola-rs/polarss5dsn-eqeeAug 293 weeks ago
[SQL] State-store instance metrics are shipped in every task result## Summary Stateful operators currently create one state-store instance metric accumulator for every combination of: Every task then receives and registers all of those accumulators, even though it processes only one state partition. On task completion, all external accumulators...apache/sparkKimahrimanAug 284 weeks ago

Companies and products named

Company or productPosts naming it
Apache Spark3
Polars1
DuckDB1
PostgreSQL1
About this problem

Evidence

3 posts from 3 people in 2 places, about 1 a week over 28 days. Mostly on apache/spark, pola-rs/polars. Tools named alongside: Polars, DuckDB, PostgreSQL.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "breaking-changes" rule about Apache Spark. First post Aug 28, 2026, latest Sep 24, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

Other Apache Spark problems

History

  • 2026-09-25 Corroboration: unverified to corroborated
  • 2026-09-25 Added: Apache Spark upgrades break working setups (2 posts)

Rising problems by email

Mondays: the problems in data, tech and AI that grew fastest that week.

Double opt-in. Unsubscribe any time.