Data and AI struggles

1,730 problems · 342 companies

Data engineering

Backfills are slow, costly and risky

10 posts from 10 people10d activeFading, +0 this week

Posts per day

Posts per day10 posts, Sep 9 to Nov 9

The posts behind it

GitHub issuesCodeWithCJ/SparkyFitness · DUC750Oct 30

[Feature]: Write endpoint for passive health metric samples

### Are you willing to submit a PR? Yes, I'd like to submit a PR! ### What problem does this solve? I sync a Xiaomi watch into SparkyFitness with my own script (Xiaomi cloud → SparkyFitness API). Daily values such as steps, sleep and resting heart rate have a write path. Passive...

1 reactions1 replies
GitHub issuesdagster-io/dagster · eelcodoornbosOct 22

Make the 90-day default partition window in the Materialize dialog configurable or remembered

### What's the use case? Since #26898 (4a573d4c73, in 1.13.x), the partition selection dialog for time-window partitioned assets opens with a "Last 90 days" date window. The user's choice of a different range is held only in component state, so every new dialog starts at 90 days...

GitHub issuesCodeWithCJ/SparkyFitness · Bubbles840Oct 19

[Feature]: Mood check-ins with energy, focus, journal and multiple entries per day

...I need a few quick ratings through the day, not one number at night. Right now there's no way to record energy or focus, and a second mood entry overwrites the first. ### Describe the feature you'd like Upgrade the mood check-in. Every part stays optional: - **Energy** and...

3 replies
GitHub issuespandas-dev/pandas · xjclOct 14

BUG: `bfill`/`ffill` with `axis=1` ignore `inplace=True` and leave the original DataFrame unchanged

### Pandas version checks - [x] I have checked that this issue has not already been reported. - [x] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas. - [x] I have confirmed this bug exists on the [main...

7 replies
GitHub issuesmicrosoft/vscode · dessingyOct 7

Session Sync Silent Error

### Summary With `chat.sessionSync.enabled` and `github.copilot.chat.localIndex.enabled` both true, the Session Sync status shows 0 synced sessions and no sessions appear on https://github.com/copilot/agents. No error is shown in the UI or in the default VS Code logs. Two...

1 reactions
GitHub issuesClickHouse/ClickHouse · zlareb1Oct 5

GraphiteMergeTree: part minmax index is built from the rows before the insert-time rollup, so queries on the time column silently skip the part

**TL;DR:** With `optimize_on_insert` (the default), an `INSERT` into a `GraphiteMergeTree` computes the part's minmax index from the rows *before* the insert-time rollup rewrites `Time` downward, so the written part's minmax range excludes the rows the part actually stores....

GitHub issuesairbytehq/airbyte · ianadanielSep 15

Source Recharge: `events` stream's initial start_datetime crashes the entire sync (not clamped to API's 7-day limit)

### Connector Name source-recharge ### Connector Version 3.0.11 ### What step the error happened? During the sync ### Relevant information ## What happened? When the connector's global `start_date` config is older than 7 days (e.g. a typical historical backfill start date like...

Redditr/dataengineering · u/lostmyway573Sep 14

Pending job offer - devil you know vs devil you don't know? (details inside)

...a data engineering point of view, our data isn't terribly complex, but our warehouse and lack of design practice over the past 10+ years has overcomplicated a lot of seemingly simple things. We have basically zero documentation, our entire warehouse relies on tribal knowledge....

GitHub issueslangchain-ai/langchain · long2bui-andpadSep 11

v3 stream: parallel tool calls collide on the positional fallback key and one is silently dropped

### Submission checklist - [x] This is a bug, not a usage question. - [x] I added a clear and descriptive title that summarizes this issue. - [x] I used the GitHub search to find a similar question and didn't find it. - [x] I am sure that this is a bug in LangChain rather than...

7 replies
GitHub issuesapache/iceberg · JeonDaehongSep 9

Spark: vectorized reads probe the position delete index once per row - 43–58% of scan CPU on narrow projections

### Summary Spark's vectorized reader asks the position delete index "is row N deleted?" once per row, 5,000 times per batch, although the positions in a batch are a contiguous ascending range. The index can be traversed once for the whole range instead. - Delete-check CPU drops...

Companies and products named

About this problem

Evidence

10 posts from 10 people in 9 places, about 1 a week over 52 days. Mostly on CodeWithCJ/SparkyFitness, r/dataengineering, langchain-ai/langchain. Tools named alongside: Snowflake, Azure Data Factory, SQL Server, Anthropic.

Frustration Frustration 1 of 3· Seen on GitHub issues, Reddit

How it was grouped

Posts that state a pain and match the "backfills" rule. First post Sep 9, 2026, latest Oct 30, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

The same problem elsewhere

Apache Airflow backfills are slow and risky

4 posts from 4 people on GitHub issues.

Dagster backfills are slow and risky

4 posts from 4 people on GitHub issues.

PostgreSQL backfills are slow and risky

3 posts from 3 people on GitHub issues.

Codex backfills are slow and risky

2 posts from 1 person on GitHub issues. Unverified.

MLflow backfills are slow and risky

2 posts from 2 people on GitHub issues. Unverified.

Claude backfills are slow and risky

2 posts from 2 people on GitHub issues. Unverified.

History

Momentum went from steady to fading.

Nov 6, 2026

Momentum went from fading to steady.

Nov 3, 2026

Momentum went from steady to fading.

Oct 29, 2026

Momentum went from fading to steady.

Oct 23, 2026

Momentum went from steady to fading.

Oct 21, 2026

Momentum went from fading to steady.

Oct 16, 2026

Momentum went from steady to fading.

Oct 14, 2026

Momentum went from rising to steady.

Oct 12, 2026

Momentum went from steady to rising.

Oct 9, 2026

Momentum went from fading to steady.

Oct 8, 2026

Momentum went from steady to fading.

Oct 2, 2026

Momentum went from fading to steady.

Sep 28, 2026