Skip to content
Problems.

Experiments on ClickHouse are hard to reproduce and track

MLOps3 posts from 1 person3d active+33 posts in the last 7 days, 0 the 7 days before (new)Unverified

Posts per day

Posts per day3 posts, Sep 21 to Sep 27

The posts behind it

3, newest first
PostDate
failpoints_disabled_in_tests check wrongly exempts no-parallel-replicas tests### Describe what's wrong - The new check_failpoints_are_disabled check fails to flag tests that run in parallel but disable all fail points. - Triggered by any test tagged `no-parallel-replicas` without `no-parallel`, using `SYSTEM DISABLE ALL FAILPOINTS` as cleanup. - Such...ClickHouse/ClickHouseclickgapaiSep 234 days ago
Empty `\Q\E` absorbs a quantifier: `replaceRegexpAll` still becomes `...One`### Describe what's wrong - `replaceRegexpAll`/`regexp_replace` drop the trailing replacement when the pattern ends with an empty `\Q\E` before a quantifier and ` Experiments on ClickHouse are hard to reproduce and track
. - Triggered at default settings with `optimize_rewrite_regexp_functions = 1`, e.g. pattern `o\Q\E* Experiments on ClickHouse are hard to reproduce and track
on `'foo'`....ClickHouse/ClickHouseclickgapai
Sep 225 days ago
uniqTheta: 5-byte state blob allocates 1 GiB, unrefused by max_memory_usage### Describe what's wrong - Deserializing a crafted `uniqTheta` state allocates and zeroes 1 GiB of memory before validating the payload. - Triggered by casting a 5-byte hex literal to `AggregateFunction(uniqTheta, UInt64)` and calling `finalizeAggregation`. - Expected an...ClickHouse/ClickHouseclickgapaiSep 216 days ago

Companies and products named

Company or productPosts naming it
ClickHouse3
GitHub2
About this problem

Evidence

3 posts from 1 person in 1 place, about 3 a week over 3 days. Mostly on ClickHouse/ClickHouse. Tools named alongside: GitHub.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "ml-reproducibility" rule about ClickHouse. First post Sep 21, 2026, latest Sep 23, 2026. Unverified: fewer than 3 posts, or one person or place only. Method

History

  • 2026-09-27 Added: Experiments on ClickHouse are hard to reproduce and track (3 posts)

Rising problems by email

Mondays: the problems in data, tech and AI that grew fastest that week.

Double opt-in. Unsubscribe any time.

Skip to content
Problems.

Experiments on ClickHouse are hard to reproduce and track

MLOps3 posts from 1 person3d active+33 posts in the last 7 days, 0 the 7 days before (new)Unverified

Posts per day

Posts per day3 posts, Sep 21 to Sep 27

The posts behind it

3, newest first
PostDate
failpoints_disabled_in_tests check wrongly exempts no-parallel-replicas tests### Describe what's wrong - The new check_failpoints_are_disabled check fails to flag tests that run in parallel but disable all fail points. - Triggered by any test tagged `no-parallel-replicas` without `no-parallel`, using `SYSTEM DISABLE ALL FAILPOINTS` as cleanup. - Such...ClickHouse/ClickHouseclickgapaiSep 234 days ago
Empty `\Q\E` absorbs a quantifier: `replaceRegexpAll` still becomes `...One`### Describe what's wrong - `replaceRegexpAll`/`regexp_replace` drop the trailing replacement when the pattern ends with an empty `\Q\E` before a quantifier and ` Experiments on ClickHouse are hard to reproduce and track
. - Triggered at default settings with `optimize_rewrite_regexp_functions = 1`, e.g. pattern `o\Q\E* Experiments on ClickHouse are hard to reproduce and track
on `'foo'`....ClickHouse/ClickHouseclickgapai
Sep 225 days ago
uniqTheta: 5-byte state blob allocates 1 GiB, unrefused by max_memory_usage### Describe what's wrong - Deserializing a crafted `uniqTheta` state allocates and zeroes 1 GiB of memory before validating the payload. - Triggered by casting a 5-byte hex literal to `AggregateFunction(uniqTheta, UInt64)` and calling `finalizeAggregation`. - Expected an...ClickHouse/ClickHouseclickgapaiSep 216 days ago

Companies and products named

Company or productPosts naming it
ClickHouse3
GitHub2
About this problem

Evidence

3 posts from 1 person in 1 place, about 3 a week over 3 days. Mostly on ClickHouse/ClickHouse. Tools named alongside: GitHub.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "ml-reproducibility" rule about ClickHouse. First post Sep 21, 2026, latest Sep 23, 2026. Unverified: fewer than 3 posts, or one person or place only. Method

History

  • 2026-09-27 Added: Experiments on ClickHouse are hard to reproduce and track (3 posts)

Rising problems by email

Mondays: the problems in data, tech and AI that grew fastest that week.

Double opt-in. Unsubscribe any time.

. - Triggered at default settings with `optimize_rewrite_regexp_functions = 1`, e.g. pattern `o\\Q\\E* Experiments on ClickHouse are hard to reproduce and track
Skip to content
Problems.

Experiments on ClickHouse are hard to reproduce and track

MLOps3 posts from 1 person3d active+33 posts in the last 7 days, 0 the 7 days before (new)Unverified

Posts per day

Posts per day3 posts, Sep 21 to Sep 27

The posts behind it

3, newest first
PostDate
failpoints_disabled_in_tests check wrongly exempts no-parallel-replicas tests### Describe what's wrong - The new check_failpoints_are_disabled check fails to flag tests that run in parallel but disable all fail points. - Triggered by any test tagged `no-parallel-replicas` without `no-parallel`, using `SYSTEM DISABLE ALL FAILPOINTS` as cleanup. - Such...ClickHouse/ClickHouseclickgapaiSep 234 days ago
Empty `\Q\E` absorbs a quantifier: `replaceRegexpAll` still becomes `...One`### Describe what's wrong - `replaceRegexpAll`/`regexp_replace` drop the trailing replacement when the pattern ends with an empty `\Q\E` before a quantifier and ` Experiments on ClickHouse are hard to reproduce and track
. - Triggered at default settings with `optimize_rewrite_regexp_functions = 1`, e.g. pattern `o\Q\E* Experiments on ClickHouse are hard to reproduce and track
on `'foo'`....ClickHouse/ClickHouseclickgapai
Sep 225 days ago
uniqTheta: 5-byte state blob allocates 1 GiB, unrefused by max_memory_usage### Describe what's wrong - Deserializing a crafted `uniqTheta` state allocates and zeroes 1 GiB of memory before validating the payload. - Triggered by casting a 5-byte hex literal to `AggregateFunction(uniqTheta, UInt64)` and calling `finalizeAggregation`. - Expected an...ClickHouse/ClickHouseclickgapaiSep 216 days ago

Companies and products named

Company or productPosts naming it
ClickHouse3
GitHub2
About this problem

Evidence

3 posts from 1 person in 1 place, about 3 a week over 3 days. Mostly on ClickHouse/ClickHouse. Tools named alongside: GitHub.

Frustration Frustration 0 of 3· Seen on GitHub issues

How it was grouped

Posts that state a pain and match the "ml-reproducibility" rule about ClickHouse. First post Sep 21, 2026, latest Sep 23, 2026. Unverified: fewer than 3 posts, or one person or place only. Method

History

  • 2026-09-27 Added: Experiments on ClickHouse are hard to reproduce and track (3 posts)

Rising problems by email

Mondays: the problems in data, tech and AI that grew fastest that week.

Double opt-in. Unsubscribe any time.

on `'foo'`....","author":"clickgapai","day":"2026-09-22","score":0,"comments":0,"entities":["clickhouse","github"],"frustration":0,"sourceName":"ClickHouse/ClickHouse"},{"id":"github:clickhouse/clickhouse#121410","kind":"github","source":"gh:clickhouse/clickhouse","url":"https://github.com/ClickHouse/ClickHouse/issues/121410","title":"uniqTheta: 5-byte state blob allocates 1 GiB, unrefused by max_memory_usage","excerpt":"### Describe what's wrong - Deserializing a crafted `uniqTheta` state allocates and zeroes 1 GiB of memory before validating the payload. - Triggered by casting a 5-byte hex literal to `AggregateFunction(uniqTheta, UInt64)` and calling `finalizeAggregation`. - Expected an...","author":"clickgapai","day":"2026-09-21","score":0,"comments":0,"entities":["clickhouse","github"],"frustration":0,"sourceName":"ClickHouse/ClickHouse"}],"related":[{"id":"ml-reproducibility","statement":"ML experiments are hard to reproduce and track","category":"mlops","issue":"ml-reproducibility","entity":"","entityName":"","domain":"","method":"pattern","posts":7,"authors":7,"places":4,"engagement":8,"last7":3,"prev7":2,"momentum":"steady","daysActive":6,"severity":0,"firstPost":"2026-09-02","lastPost":"2026-09-24","firstSeen":"2026-09-25 12:39:59","companies":[{"slug":"github","name":"GitHub","domain":"github.com","n":4},{"slug":"mlflow","name":"MLflow","domain":"mlflow.org","n":3},{"slug":"llamaindex","name":"LlamaIndex","domain":"llamaindex.ai","n":1},{"slug":"openai","name":"OpenAI","domain":"openai.com","n":1},{"slug":"claude","name":"Claude","domain":"anthropic.com","n":1},{"slug":"bedrock","name":"Amazon Bedrock","domain":"aws.amazon.com","n":1}],"kinds":["github"],"spark":[0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,2,0,0,1,0,0,0],"top":[{"url":"https://github.com/mlflow/mlflow/issues/25898","title":"[BUG] AI Gateway: `send_stream_request` drops the upstream error message unless the body is OpenAI-shaped","kind":"github","source":"mlflow/mlflow","author":"rhamzatov","day":"2026-09-16","score":0},{"url":"https://github.com/mlflow/mlflow/issues/25525","title":"[FR] Expose Helm health probes as configuration variables","kind":"github","source":"mlflow/mlflow","author":"jeremyeder","day":"2026-09-02","score":0},{"url":"https://github.com/crewAIInc/crewAI/issues/7684","title":"[BUG] QdrantEdgeStorage crashes on Windows during orphaned-shard cleanup (os.kill signal 0 unsupported)","kind":"github","source":"crewAIInc/crewAI","author":"josephkamau32","day":"2026-09-21","score":0}],"verified":true,"rising":3.8,"persistent":true,"opportunity":"7 posts from 7 people in 4 places, about 2 a week over 23 days. Mostly on mlflow/mlflow, crewAIInc/crewAI, run-llama/llama_index. Tools named alongside: GitHub, MLflow, LlamaIndex, OpenAI.","summary":"","summaryBy":""}],"sameEntity":[{"id":"clickhouse-breaking-changes","statement":"ClickHouse upgrades break working setups","category":"reliability","issue":"breaking-changes","entity":"clickhouse","entityName":"ClickHouse","domain":"clickhouse.com","method":"pattern","posts":6,"authors":5,"places":1,"engagement":5,"last7":6,"prev7":0,"momentum":"new","daysActive":4,"severity":0,"firstPost":"2026-09-21","lastPost":"2026-09-25","firstSeen":"2026-09-27 05:42:20","companies":[{"slug":"clickhouse","name":"ClickHouse","domain":"clickhouse.com","n":6},{"slug":"github","name":"GitHub","domain":"github.com","n":2},{"slug":"text","name":"Text","domain":"investor.text.com","n":1}],"kinds":["github"],"spark":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,3,1,0,1,0,0],"top":[{"url":"https://github.com/ClickHouse/ClickHouse/issues/121509","title":"Reuse checksum-matching detached parts when bootstrapping a fresh replica","kind":"github","source":"ClickHouse/ClickHouse","author":"patelsmit32123","day":"2026-09-22","score":1},{"url":"https://github.com/ClickHouse/ClickHouse/issues/122333","title":"hasAny over Array(String) compares every element with every constant needle: a needle hash set cuts a 32-needle hasAny from 5.4 s to 0.52 s","kind":"github","source":"ClickHouse/ClickHouse","author":"snelheidai","day":"2026-09-25","score":1},{"url":"https://github.com/ClickHouse/ClickHouse/issues/121691","title":"Text index prunes matching rows: decline keys on needle padding, not value's","kind":"github","source":"ClickHouse/ClickHouse","author":"clickgapai","day":"2026-09-23","score":0}],"verified":true,"rising":13.2,"persistent":false,"opportunity":"6 posts from 5 people in 1 place, about 6 a week over 5 days. Mostly on ClickHouse/ClickHouse. Tools named alongside: GitHub, Text.","summary":"","summaryBy":""},{"id":"clickhouse-issues","statement":"ClickHouse users keep filing bugs","category":"databases","issue":"general","entity":"clickhouse","entityName":"ClickHouse","domain":"clickhouse.com","method":"entity","posts":10,"authors":7,"places":1,"engagement":5,"last7":10,"prev7":0,"momentum":"new","daysActive":2,"severity":0,"firstPost":"2026-09-21","lastPost":"2026-09-22","firstSeen":"2026-09-27 05:42:20","companies":[{"slug":"clickhouse","name":"ClickHouse","domain":"clickhouse.com","n":10},{"slug":"github","name":"GitHub","domain":"github.com","n":5},{"slug":"docker","name":"Docker","domain":"docker.com","n":1}],"kinds":["github"],"spark":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,6,0,0,0,0,0],"top":[{"url":"https://github.com/ClickHouse/ClickHouse/issues/121314","title":"Smaller binary via lld ICF of functions with identical exception tables (llvm-project#224929)","kind":"github","source":"ClickHouse/ClickHouse","author":"thevar1able","day":"2026-09-21","score":1},{"url":"https://github.com/ClickHouse/ClickHouse/issues/121577","title":"Segmentation fault (STID: 4085-568b)","kind":"github","source":"ClickHouse/ClickHouse","author":"Algunenano","day":"2026-09-22","score":0},{"url":"https://github.com/ClickHouse/ClickHouse/issues/121580","title":"parseQueryToJSON writes a large Float64 literal that formatQueryFromJSON cannot read back (integer overflow in Poco JSON parser)","kind":"github","source":"ClickHouse/ClickHouse","author":"qoega","day":"2026-09-22","score":0}],"verified":true,"rising":8.5,"persistent":false,"opportunity":"10 posts from 7 people in 1 place, about 10 a week over 2 days. Mostly on ClickHouse/ClickHouse. Tools named alongside: GitHub, Docker.","summary":"","summaryBy":""},{"id":"clickhouse-kw-table","statement":"Trouble with ClickHouse table","category":"databases","issue":"kw:table","entity":"clickhouse","entityName":"ClickHouse","domain":"clickhouse.com","method":"keyword","posts":4,"authors":3,"places":1,"engagement":2,"last7":4,"prev7":0,"momentum":"new","daysActive":2,"severity":0,"firstPost":"2026-09-21","lastPost":"2026-09-22","firstSeen":"2026-09-27 05:42:20","companies":[{"slug":"clickhouse","name":"ClickHouse","domain":"clickhouse.com","n":4},{"slug":"github","name":"GitHub","domain":"github.com","n":3}],"kinds":["github"],"spark":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,1,0,0,0,0,0],"top":[{"url":"https://github.com/ClickHouse/ClickHouse/issues/121304","title":"clickhouse local: table/dictionary names longer than NAME_MAX raise std::filesystem_error (code 1001) instead of UNKNOWN_TABLE","kind":"github","source":"ClickHouse/ClickHouse","author":"qoega","day":"2026-09-21","score":0},{"url":"https://github.com/ClickHouse/ClickHouse/issues/121562","title":"Cache dictionary table read fabricates a DEFAULT row for a deleted source key","kind":"github","source":"ClickHouse/ClickHouse","author":"clickgapai","day":"2026-09-22","score":0},{"url":"https://github.com/ClickHouse/ClickHouse/issues/121347","title":"Distributed-destination INSERT SELECT still forwards the plain table function","kind":"github","source":"ClickHouse/ClickHouse","author":"clickgapai","day":"2026-09-21","score":0}],"verified":true,"rising":5.9,"persistent":false,"opportunity":"4 posts from 3 people in 1 place, about 4 a week over 2 days. Mostly on ClickHouse/ClickHouse. Tools named alongside: GitHub.","summary":"","summaryBy":""},{"id":"clickhouse-kw-column","statement":"Trouble with ClickHouse column","category":"databases","issue":"kw:column","entity":"clickhouse","entityName":"ClickHouse","domain":"clickhouse.com","method":"keyword","posts":4,"authors":3,"places":1,"engagement":1,"last7":4,"prev7":0,"momentum":"new","daysActive":2,"severity":0,"firstPost":"2026-09-21","lastPost":"2026-09-22","firstSeen":"2026-09-27 05:42:20","companies":[{"slug":"clickhouse","name":"ClickHouse","domain":"clickhouse.com","n":4},{"slug":"github","name":"GitHub","domain":"github.com","n":2}],"kinds":["github"],"spark":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,3,0,0,0,0,0],"top":[{"url":"https://github.com/ClickHouse/ClickHouse/issues/121560","title":"Logical error: Sizes of nested column and null map of Nullable column are not equal after deserialization when reading a Wide part with a pe","kind":"github","source":"ClickHouse/ClickHouse","author":"alexey-milovidov","day":"2026-09-22","score":0},{"url":"https://github.com/ClickHouse/ClickHouse/issues/121617","title":"WITH FILL emits rows outside FROM..TO when STEP overflows the column type","kind":"github","source":"ClickHouse/ClickHouse","author":"clickgapai","day":"2026-09-22","score":0},{"url":"https://github.com/ClickHouse/ClickHouse/issues/121483","title":"`ALTER UPDATE/DELETE` on a column shadowing `_part`: NOT_FOUND_COLUMN_IN_BLOCK","kind":"github","source":"ClickHouse/ClickHouse","author":"clickgapai","day":"2026-09-22","score":0}],"verified":true,"rising":5.4,"persistent":false,"opportunity":"4 posts from 3 people in 1 place, about 4 a week over 2 days. Mostly on ClickHouse/ClickHouse. Tools named alongside: GitHub.","summary":"","summaryBy":""},{"id":"clickhouse-structured-output","statement":"ClickHouse fails to return valid structured output","category":"llm-apps","issue":"structured-output","entity":"clickhouse","entityName":"ClickHouse","domain":"clickhouse.com","method":"pattern","posts":2,"authors":1,"places":1,"engagement":0,"last7":2,"prev7":0,"momentum":"new","daysActive":1,"severity":3,"firstPost":"2026-09-22","lastPost":"2026-09-22","firstSeen":"2026-09-27 05:42:20","companies":[{"slug":"clickhouse","name":"ClickHouse","domain":"clickhouse.com","n":2}],"kinds":["github"],"spark":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0],"top":[{"url":"https://github.com/ClickHouse/ClickHouse/issues/121581","title":"formatQueryFromJSON rejects parseQueryToJSON output for a partially named Tuple(...) type (empty element name)","kind":"github","source":"ClickHouse/ClickHouse","author":"qoega","day":"2026-09-22","score":0},{"url":"https://github.com/ClickHouse/ClickHouse/issues/121583","title":"formatQueryFromJSON rejects parseQueryToJSON output for TTL expr GROUP BY without grouping keys","kind":"github","source":"ClickHouse/ClickHouse","author":"qoega","day":"2026-09-22","score":0}],"verified":false,"rising":2.5,"persistent":false,"opportunity":"2 posts from 1 person in 1 place, about 2 a week over 1 days. Mostly on ClickHouse/ClickHouse.","summary":"","summaryBy":""}],"history":[{"at":"2026-09-27 05:42:21","type":"problem","item":"clickhouse-ml-reproducibility","field":"added","old":"","new":"Experiments on ClickHouse are hard to reproduce and track (3 posts)","source":"https://github.com/ClickHouse/ClickHouse/issues/121676","by":"daily"}],"sourceUrl":"https://github.com/ClickHouse/ClickHouse/issues/121676","firstSeenAt":"2026-09-27 05:42:20","lastSeenAt":"2026-09-27 05:42:20","verifiedBy":""}}}