Problemsby GlyphData pipelines fail silently or break often
19 posts from 19 people18d activeFading, +2 this week
Posts per day
The posts behind it
[ROCm][CI] Machines under the label amd-dpx-linux.rocm.gpu.gfx950.N are undergoing GPU driver upgrades.
...finish by end of day today (EOD, 2026-11-07, US Eastern time). ### Prevention/followups *How do we prevent issues like this in the future?* N/A Made with Cursor cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @jataylo @hongxiayang @naromero77amd @pragupta...
workflow-engine: step timeout over ~24.8 days (or Infinity) fires after 1ms
`executeWithTimeout` in `rivetkit-typescript/packages/workflow-engine/src/context.ts` uses a plain `setTimeout(reject, timeoutMs)`. Node can't wait longer than 2^31-1 ms (~24.8 days) that way: anything bigger, including `Infinity`, is clamped to 1ms with a...
[BUG] MLFLOW_SQLALCHEMYSTORE_POOL_SIZE=0 and MAX_OVERFLOW=0 are silently ignored
...and SQLAlchemy falls back to its defaults (pool_size=5, max_overflow=10). This makes it impossible to control these specific connection limits using environment variables, which defeats the purpose of these variables in production deployments. ### How would a user encounter this...
Text-generation pipeline crashes with batched generation metadata and silently drops beam scores
### System Info - Transformers: 5.23.0.dev0 (development version installed from GitHub main) - PyTorch: 2.11.0+cu128 - Python: 3.13 - Platform: Linux - GPU: NVIDIA Tesla T4 - Models tested: `openai-community/gpt2`, `facebook/opt-125m` - Precision: FP32 ### Reproduction While...
Code location fails to load (`KeyError` in `_attempt_resolve_node_cycles`) when two `@dbt_assets` ops depend on each other and a dbt test references an ephemeral model
## What's the issue? If two `@dbt_assets` ops depend on each other (op A -> op B -> op A, through different models), Dagster is meant to split the op into subsets. When a dbt test of a model in that cycle also references an **ephemeral** model, building the asset job fails...
What to do with my paper, for MLsys conference? [R]
...only the offline results, or should I include the failed deployment?. If I include it, how do I frame it so it adds to the paper instead of sinking it?. Why might a tiny median deviation still lead to task failure? My guesses are tail errors and error compounding in closed loop,...
[Bug]: iOS nutrition sync fails for existing installs after caffeine was added (permission never re-requested)
### Are you willing to submit a PR? Yes, I'd like to submit a PR! ### What happened? Since 1.7.0, every HealthKit sync with Nutrition enabled logs: The sync is marked as having errors, so `lastSyncedTime` is never updated: the Sync screen shows the same "Last synced" date...
AWS Glue 6.0 (Spark 4.1) throws No plan for TableReference exception during MERGE INTO operation
...version.. . I tried via disabling adaptive.sql but the error persists no matter what.. Is there any way to bypass this while staying on Glue 6.0?.
[inductor] Non-CUDA/XPU devices silently get CUDACodeCache and device-less arch errors
### 🐛 Describe the bug Inductor repeatedly encodes the assumption "only CUDA and XPU exist" as "anything that is not XPU is CUDA". That is invisible while only those two backends are in tree, but it turns every out-of-tree backend (NPU / HPU / …) into a silent failure with a...
A new problem with New Outlook
This has got to be the worst one so far, as it is costing them actual customers and missing vendor deadlines. . Somehow a user snuck past us and is using New Outlook. A report came in that the first time he emails someone new, it fails. The 2nd time or any reply, it works.....
[Spark/ORC] NullPointerException in SparkOrcWriter when inserting VARIANT data into ORC-backed Iceberg v3 tables
### Apache Iceberg version 1.11.0 ### Query engine Spark ### Please describe the bug 🐞 In Apache Iceberg v3, creating a table with a VARIANT column using ORC as the default write format ('write.format.default' = 'orc') succeeds during DDL creation without validation warnings....
[source-salesforce] REST streams fail with INVALID_SESSION_ID when refresh token rotation ends the session mid-pagination
### Connector Name source-salesforce ### Connector Version 2.8.1 (observed). The affected code is unchanged in 2.9.2. ### What step the error happened? During the sync ### Relevant information Syncs fail when Salesforce Refresh Token Rotation (RTR) is enabled and a REST-path...
Made a huge mistake during work and now my team is having to fix it.
...The principals devs got it handled, they’re fixing it all right now. But man I feel terrible. They’re having to use their time to fix my fuck up. To make it worse I just had a convo with my boss about how to eventually become senior, and he said autonomy was a big component. Not...
Fabric Shortcut → Warehouse SQL Sync Suddenly Failing
Hi everyone,. I have D365 tables coming into Fabric through Dataverse. I created shortcuts from the D365 workspace to both DEV and UAT workspaces.. DEV: Shortcuts work fine in both Lakehouse and Warehouse.. UAT: Shortcut is visible in the Lakehouse, but Warehouse sync fails:....
Destination Clickhouse syncs fail before inserting records
### Connector Name desination-clickhouse ### Connector Version 2.1.24 ### What step the error happened? During the sync ### Relevant information Logging on behalf of a community member from Slack: We are using Airbyte OSS 2.0.1 with ClickHouse destination connector 2.1.24. After...
I built a zero-dependency CLI that scans LangChain and CrewAI code for silent failures and runaway loops
...issues where our agents reported successful executions despite the underlying tools failing. The most frustrating case was a custom tool that caught SendGrid API exceptions and returned "Email sent" to the agent when SendGrid was actually returning 401 Unauthorized. The agent...
Agent Host (Copilot): hooks from marketplace-installed agent plugins are never invoked since 1.139
...`~/Library/Application Support/Code/agentPlugins/…` is byte-identical to the source. ## Workaround Registering the same command as a user-level hook in `~/.copilot/hooks/ .json` (absolute paths, `PLUGIN_ROOT` set via `env`) works: the command is invoked for...
Workers are not honoring deployment concurrency limits during poll time
...actually start. The backlog would drain in order without the retry storm. We looked at workarounds, like one work queue per deployment with a queue-level concurrency limit. That reduces the churn, but queue priorities have to be unique, so we'd lose our small set of shared...
[source-todoist] REST API v2 shutdown returns 410 Gone
### Connector Name source-todoist ### Connector Version 0.3.43 ### What step the error happened? Configuring a new connector ### Relevant information Todoist shut down REST API v2 on 2026-02-10. The connector still calls `https://api.todoist.com/rest/v2`, which returns **410...
Companies and products named
About this problem
Evidence
19 posts from 19 people in 16 places, about 3 a week over 47 days. Mostly on airbytehq/airbyte, pytorch/pytorch, PrefectHQ/prefect. Tools named alongside: GitHub, Airbyte, PyTorch, Slack.
Frustration Frustration 2 of 3· Seen on GitHub issues, Reddit, Stack Overflow
How it was grouped
Posts that state a pain and match the "pipeline-failures" rule. First post Sep 22, 2026, latest Nov 7, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method
The same problem elsewhere
Apache Airflow jobs fail silently or break often
13 posts from 12 people on GitHub issues, Stack Overflow.
Gemini jobs fail silently or break often
8 posts from 1 person on GitHub issues. Unverified.
PostgreSQL jobs fail silently or break often
5 posts from 4 people on GitHub issues.
Databricks jobs fail silently or break often
5 posts from 5 people on GitHub issues, Reddit.
GitHub jobs fail silently or break often
4 posts from 4 people on Reddit, GitHub issues.
ChatGPT jobs fail silently or break often
3 posts from 3 people on GitHub issues, Community forums.
Claude Code jobs fail silently or break often
3 posts from 3 people on GitHub issues.
GitLab jobs fail silently or break often
2 posts from 2 people on Stack Overflow. Unverified.
Docker jobs fail silently or break often
2 posts from 2 people on GitHub issues. Unverified.
History