Apache Spark runs out of memory
2 posts from 2 people2d activeFading, +0 this weekunverified
Posts per day
The posts behind it
JVM OutOfMemoryError during PySpark DataFrame.collect() on PostgreSQL JDBC data
...whether this behavior is expected for large JDBC datasets or whether there is an issue with memory management during result serialization. Environment OS: Linux Python: 3.12 Apache Spark / PySpark version: [Please provide] Java version: [Please provide] PostgreSQL JDBC driver:...
[SQL][PYTHON] Rotate disk spill segments and support per-queue memory cap in HybridRowQueue
### Describe the bug / improvement `HybridRowQueue` buffers pass-through rows on the JVM while `EvalPythonExec` / `EvalPythonUDTFExec` feeds input batches to a Python worker and waits for responses. Currently, it has two structural issues under memory pressure or fast-producer /...
Companies and products named
About this problem
Evidence
2 posts from 2 people in 1 place, about 2 a week over 3 days. Mostly on apache/spark. Tools named alongside: PostgreSQL.
Frustration Frustration 0 of 3· Seen on GitHub issues
How it was grouped
Posts that state a pain and match the "oom" rule about Apache Spark. First post Nov 11, 2026, latest Nov 13, 2026. Unverified: fewer than 3 posts, or one person or place only. Method
The same problem elsewhere
Jobs and services run out of memory
16 posts from 16 people on GitHub issues, Stack Overflow, Reddit.
Gemini runs out of memory
14 posts from 13 people on GitHub issues.
GitHub runs out of memory
6 posts from 6 people on GitHub issues.
NVIDIA runs out of memory
5 posts from 5 people on GitHub issues.
llama.cpp runs out of memory
4 posts from 4 people on GitHub issues.
Claude Code runs out of memory
4 posts from 4 people on GitHub issues.
DuckDB runs out of memory
4 posts from 4 people on GitHub issues.
Apache Iceberg runs out of memory
4 posts from 3 people on GitHub issues.
BigQuery runs out of memory
3 posts from 3 people on GitHub issues.
Polars runs out of memory
3 posts from 3 people on GitHub issues.
vLLM runs out of memory
3 posts from 3 people on GitHub issues.
Ollama runs out of memory
3 posts from 3 people on GitHub issues.
Other Apache Spark problems
Apache Spark users keep filing bugs
Top post: "Is a broadcast variable shared across multiple concurrent Spark jobs?", one of 19 posts from 18 people on GitHub issues, Reddit, Stack Overflow.
Apache Spark upgrades break working setups
5 posts from 5 people on GitHub issues.
Trouble with Apache Spark rows
Top post: "scan_delta returns deleted rows from partitions whose path is percent-encoded", one of 4 posts from 4 people on GitHub issues, Stack Overflow.
Bad data slips through Apache Spark unnoticed
2 posts from 2 people on GitHub issues. Unverified.
History