Data and AI struggles

2,114 problems · 377 companies

DevOps and infra

Apache Spark runs out of memory

2 posts from 2 people2d activeFading, +0 this weekunverified

Posts per day

Posts per day2 posts, Nov 11 to Nov 21

The posts behind it

GitHub issuesapache/spark · leewenshu36Nov 13

JVM OutOfMemoryError during PySpark DataFrame.collect() on PostgreSQL JDBC data

...whether this behavior is expected for large JDBC datasets or whether there is an issue with memory management during result serialization. Environment OS: Linux Python: 3.12 Apache Spark / PySpark version: [Please provide] Java version: [Please provide] PostgreSQL JDBC driver:...

1 replies
GitHub issuesapache/spark · jischengNov 11

[SQL][PYTHON] Rotate disk spill segments and support per-queue memory cap in HybridRowQueue

### Describe the bug / improvement `HybridRowQueue` buffers pass-through rows on the JVM while `EvalPythonExec` / `EvalPythonUDTFExec` feeds input batches to a Python worker and waits for responses. Currently, it has two structural issues under memory pressure or fast-producer /...

Companies and products named

About this problem

Evidence

2 posts from 2 people in 1 place, about 2 a week over 3 days. Mostly on apache/spark. Tools named alongside: PostgreSQL.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "oom" rule about Apache Spark. First post Nov 11, 2026, latest Nov 13, 2026. Unverified: fewer than 3 posts, or one person or place only. Method

The same problem elsewhere

Jobs and services run out of memory

16 posts from 16 people on GitHub issues, Stack Overflow, Reddit.

Gemini runs out of memory

14 posts from 13 people on GitHub issues.

GitHub runs out of memory

6 posts from 6 people on GitHub issues.

NVIDIA runs out of memory

5 posts from 5 people on GitHub issues.

llama.cpp runs out of memory

4 posts from 4 people on GitHub issues.

Claude Code runs out of memory

4 posts from 4 people on GitHub issues.

DuckDB runs out of memory

4 posts from 4 people on GitHub issues.

Apache Iceberg runs out of memory

4 posts from 3 people on GitHub issues.

BigQuery runs out of memory

3 posts from 3 people on GitHub issues.

Polars runs out of memory

3 posts from 3 people on GitHub issues.

vLLM runs out of memory

3 posts from 3 people on GitHub issues.

Ollama runs out of memory

3 posts from 3 people on GitHub issues.

Other Apache Spark problems

Apache Spark users keep filing bugs

Top post: "Is a broadcast variable shared across multiple concurrent Spark jobs?", one of 19 posts from 18 people on GitHub issues, Reddit, Stack Overflow.

Apache Spark upgrades break working setups

5 posts from 5 people on GitHub issues.

Trouble with Apache Spark rows

Top post: "scan_delta returns deleted rows from partitions whose path is percent-encoded", one of 4 posts from 4 people on GitHub issues, Stack Overflow.

Bad data slips through Apache Spark unnoticed

2 posts from 2 people on GitHub issues. Unverified.

History

Momentum went from steady to fading.

Nov 20, 2026

Momentum went from new to steady.

Nov 18, 2026

First seen with 2 posts.

Nov 14, 2026