Small files, skew and partitioning hurt performance
14 posts from 14 people13d activeFading, +1 this week
Posts per day
The posts behind it
Iceberg Glue catalog: `INSERT` fails permanently with `Write into iceberg was not successful` after another writer (pyiceberg, Spark) commits, because ClickHouse names metadata files `v<N>.metadata.json` from the catalog pointer
### Describe what's wrong When a Glue Iceberg table is written both by ClickHouse (through a `DataLakeCatalog` database with `catalog_type = 'glue'`) and by any Iceberg Java or pyiceberg writer, ClickHouse writes stop working after a couple of alternating commits. Every further...
feat: context compaction should reuse the prompt cache (summary request is always a full cold read)
...and Discussion #30105. ### Examples or References _No response_ ### Alternatives or Workarounds I run a small proxy between Open WebUI and the engine that spots the compaction request, rebuilds it as described above from the chat's last request, and falls back to the original if...
[source-sftp-bulk] EMPTY_STREAM for root folder "/" with a relative glob (duplicate-slash path bug)
...discovers no files and fails with EMPTY_STREAM, even though a matching file exists. Failing configuration: { "folder_path": "/", "streams": [ { "name": "Lacey", "globs": ["downloaded/client2/Lacey*"] } ], "start_date": "2026-10-01T00:00:00.000000Z" } Target file:...
Offer option to Compress plan output as zstd instead of deflate
### Terraform Version ### Use Cases We currently run almost 1000 root modules in our environment. Our average plan file size is 30mb. This adds up to a lot of bandwidth used when we transfer the plans from our CI runners in our cloud environment to our source control. There is...
feat: hide toggles and buttons of disabled features, and let admins hide UI elements
**Problem** Open WebUI shows many toggles, buttons and menu entries for features that are turned off for the whole instance. Clicking them does nothing, opens an empty list or shows an error. Admins also cannot hide elements their users do not need without turning off the...
Segfault in PhysicalBatchCopyToFile on VARCHAR data during COPY ... ORDER BY TO parquet (1.5.2 and 1.5.6)
### What happens? `COPY (SELECT ... ORDER BY ...) TO '...parquet'` intermittently segfaults inside `PhysicalBatchCopyToFile` while handling `VARCHAR` data. The pattern looks like a dangling `string_t`. It reproduces on 1.5.2 and on 1.5.6, but the crash point differs between the...
[Bug]: delete_collection() leaves orphaned HNSW segment directory after successful deletion
### What happened? We are running ChromaDB in persistent HTTP/server mode and observed that `delete_collection()` successfully removes the collection from Chroma's logical catalog, but the corresponding persisted HNSW vector segment directory remains on disk. This was reproduced...
read_csv(use_pyarrow=True) fails or returns wrong values on quoted values with newlines
...passes user options to `pl.read_csv()` (pyathena-dev/PyAthena#1048). pandas has the same issue with `engine="pyarrow"` (pandas-dev/pandas#56396, pandas-dev/pandas#59009). ### Expected behavior `read_csv(..., use_pyarrow=True)` reads quoted values that contain newlines, as the...
Windows installer-signature bug: Ollama v0.35.1 Windows installer fail Authenticode with HashMismatch
...would defeat the purpose of the installer integrity validation and is not an appropriate workaround. ## Reproduction summary 1. On Windows, download `v0.35.1/OllamaSetup.exe`. 2. Run: 3. Observe: 4. Repeat with `v0.35.0/OllamaSetup.exe`. 5. Observe: This appears specific to the...
Compaction fails with 'Session lifecycle transition is active' error
When attempting to compact a session near the token limit, the operation fails with an error indicating a session lifecycle transition is already active. **Steps to Reproduce:** 1. Fill a session to near token limit (199k/200k tokens) 2. Execute `/compact` command 3. Observe...
UI: Ability to select model to be used for chat title summarization
### Prerequisites - [x] I am running the latest code. Mention the version if possible as well. - [x] I carefully followed the [README.md](https://github.com/ggml-org/llama.cpp/blob/master/README.md). - [x] I searched using keywords relevant to my issue to make sure that I am...
Perf: consolidate head/footer HTTP requests on Parquet, IPC
### Description Every cloud Parquet/IPC metadata read costs **two** round trips: a `HEAD` to learn the file size, then a ranged `GET` for the metadata footer. The HEAD transfers zero bytes and exists only to compute the range offset. MinIO trace, `scan_parquet` over 5 files,...
Introduce model-based correctness and linearizability testing for k8s storage
### What would you like to be added? We would like to introduce model-based correctness testing suite for Kubernetes storage (`k8s.io/apiserver/pkg/storage`), modeled after the [robustness testing approach used in...
Companies and products named
About this problem
Evidence
14 posts from 14 people in 12 places, about 1 a week over 70 days. Mostly on pola-rs/polars, open-webui/open-webui, ggml-org/llama.cpp. Tools named alongside: GitHub, Polars, llama.cpp, Microsoft Azure.
Frustration Frustration 0 of 3· Seen on GitHub issues, Hacker News
How it was grouped
Posts that state a pain and match the "file-formats" rule. First post Aug 28, 2026, latest Nov 5, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Claude Code tables suffer from small files and skew
9 posts from 9 people on GitHub issues, Reddit.
ChatGPT tables suffer from small files and skew
5 posts from 5 people on GitHub issues.
GitHub tables suffer from small files and skew
5 posts from 5 people on GitHub issues.
Claude tables suffer from small files and skew
4 posts from 4 people on GitHub issues, Hacker News.
Codex tables suffer from small files and skew
4 posts from 4 people on GitHub issues.
OpenAI tables suffer from small files and skew
3 posts from 3 people on GitHub issues.
Databricks tables suffer from small files and skew
3 posts from 3 people on Reddit, GitHub issues.
Model Context Protocol tables suffer from small files and skew
3 posts from 2 people on GitHub issues.
Microsoft Fabric tables suffer from small files and skew
2 posts from 2 people on GitHub issues, Stack Overflow. Unverified.
Apache Flink tables suffer from small files and skew
2 posts from 2 people on GitHub issues. Unverified.
Apache Iceberg tables suffer from small files and skew
2 posts from 2 people on GitHub issues. Unverified.
Vercel tables suffer from small files and skew
2 posts from 2 people on GitHub issues. Unverified.
History