Problemsby GlyphData engineering problems
Dagster users keep filing bugs
Top post: "Allow customizing the content/layout of Dagster+ alert notification message", one of 6 posts from 6 people on GitHub issues.
Trouble with Polars join
Top post: "Coalescing full join on Int32-vs-Int64 keys with `unique` incorrect", one of 6 posts from 6 people on GitHub issues.
Hard to trace lineage and find trusted data in dbt
6 posts from 6 people on GitHub issues, Reddit.
Trouble with Polars instead
Top post: "Series.to_list() panics on out-of-range datetimes instead of raising an error", one of 6 posts from 6 people on GitHub issues.
Upstream schema changes break downstream pipelines
6 posts from 6 people on GitHub issues, Reddit.
Apache Flink users keep filing bugs
Top post: "Clarify that write.*.isolation-level properties do not apply to append/INSERT", one of 5 posts from 5 people on GitHub issues, Reddit, Stack Overflow.
Debugging orchestration and DAG failures is hard
5 posts from 5 people on GitHub issues.
Trouble with pandas read
Top post: "CI: sanitizers environment has no pyarrow, so the read_csv pyarrow string path is never sanitized", one of 5 posts from 5 people on GitHub issues.
Trouble with pandas raises
Top post: "read_csv with usecols raises IndexError in low-memory mixed-dtype warning path", one of 5 posts from 5 people on GitHub issues.
Databricks jobs fail silently or break often
5 posts from 5 people on GitHub issues, Reddit.
Trouble with pandas series
Top post: "Series arithmetic ops with PyArrow dtypes and scalars can upcast", one of 5 posts from 4 people on GitHub issues, Stack Overflow.
Trouble with Apache Spark rows
Top post: "scan_delta returns deleted rows from partitions whose path is percent-encoded", one of 4 posts from 4 people on GitHub issues, Stack Overflow.
Trouble with Polars group
Top post: "Datetime out-of-range in `group_by_dynamic` streaming engine", one of 4 posts from 3 people on GitHub issues.
Trouble with dbt fusion
Top post: "With dbt Fusion and no dbt-core, every definitions load runs `dbt list` and leaves a target directory behind", one of 4 posts from 4 people on GitHub issues.
Trouble with Databricks catalog
Top post: "Spark support for Unity Catalog Iceberg", one of 4 posts from 4 people on Reddit, GitHub issues.
Trouble with Apache Iceberg rest
Top post: "Core: Negative delete-file-references in REST scan response cause IndexOutOfBoundsException", one of 4 posts from 3 people on Reddit, GitHub issues.
Trouble with pandas instead
Top post: "`pd.merge` gives `float`s instead of integer extensions", one of 4 posts from 4 people on GitHub issues.
PostgreSQL connectors and syncs are fragile
4 posts from 4 people on GitHub issues, Hacker News.
Trouble with Delta Lake protocol
Top post: "Concurrent Identity Columns (CIC)", one of 4 posts from 4 people on GitHub issues.
Apache Iceberg schema changes break downstream pipelines
3 posts from 3 people on GitHub issues.
Trouble with Apache Iceberg tables
Top post: "Iceberg tables are much slower than MergeTree in the performance tests: subcolumns, sort order, join", one of 3 posts from 3 people on GitHub issues.
Trouble with Apache Flink column
Top post: "A dropped field is rejected as Cannot drop clustering column when a clustering column under another parent", one of 3 posts from 3 people on GitHub issues.
Bad data slips through Apache Iceberg unnoticed
3 posts from 3 people on GitHub issues.
Databricks schema changes break downstream pipelines
3 posts from 3 people on GitHub issues, Reddit.
Trouble with dbt fail
Top post: "`TIMESTAMPTZ_NS` cannot be compared with `TIMESTAMPTZ` or take an `INTERVAL`, so 1.5 queries on nanosecond", one of 3 posts from 3 people on GitHub issues.
Trouble with Polars returns
Top post: "GPU `max_horizontal` returns NaN for a finite value followed by NaN", one of 3 posts from 2 people on GitHub issues.
Trouble with dbt false
Top post: "SQL linting (dbt sqlfluff lint) does not respect enabled: false (dynamic +enabled)", one of 3 posts from 3 people on GitHub issues.
Redshift users keep filing bugs
Top post: "dbt-oss 2.0.8 still does not retry transient Redshift query errors", one of 3 posts from 3 people on GitHub issues, Reddit.
Testing data pipelines and SQL is hard
3 posts from 3 people on Reddit, GitHub issues.
Trouble with Apache Iceberg delete
Top post: "Iceberg DROP PARTITION aborts or republishes parent delete counters after an overwrite snapshot", one of 3 posts from 3 people on GitHub issues.
Trouble with Apache Airflow retry
Top post: "Task state store writes from `on_retry_callback` fail with 404", one of 3 posts from 3 people on GitHub issues.
Trouble with Databricks spark
Top post: "replaceUsing on partitioned tables fails with DELTA_FILE_TO_OVERWRITE_NOT_FOUND on local file:// paths", one of 3 posts from 3 people on GitHub issues.
Databricks tables suffer from small files and skew
3 posts from 3 people on Reddit, GitHub issues.
Trouble with Snowflake query
Top post: "SELECT SQL query returning no columns", one of 3 posts from 3 people on GitHub issues, Stack Overflow.
Trouble with Dagster asset
Top post: "Asset daemon: `execution_failed()` on unpartitioned assets issues one un-batched query per asset per tick", one of 3 posts from 3 people on GitHub issues.
Trouble with Polars value
Top post: "polars.read_excel() value error on very large date", one of 3 posts from 3 people on GitHub issues.
Trouble with Polars true
Top post: "wrap_numerical=True saturates floats at the dtype maximum", one of 3 posts from 3 people on GitHub issues.
Model Context Protocol tables suffer from small files and skew
3 posts from 2 people on GitHub issues.
Trouble with Polars read
Top post: "`read_database` raises `ImportError` with SQLAlchemy 2.1 when `greenlet` is not installed", one of 3 posts from 3 people on GitHub issues.
Trouble with dbt semantic
Top post: "`NodeNotFoundOrDisabled` when a package's model and its semantic_model are both disabled at project level", one of 3 posts from 3 people on GitHub issues.
Trouble with BigQuery fail
Top post: "Unit tests fail with protected to the package for public models from a `restrict-access: true` package", one of 3 posts from 3 people on GitHub issues.