Data and AI struggles

1,730 problems · 342 companies

Data engineering

BigQuery users keep asking for help

14 posts from 14 people12d activeFading, +0 this week

Posts per day

Posts per day14 posts, Aug 30 to Nov 9

The posts behind it

Stack OverflowStack Overflow [google-bigquery] · JohnDataIntOct 29

MySQL updates sometimes not making it to BigQuery

I've been syncing a MySQL table to BigQuery every hour and noticed something weird. New rows are coming in fine, but some of the existing ones don't seem to update. I checked a few in MySQL and the updated_at values are newer, so I don't get why they're being missed.. The...

GitHub issuesdbt-labs/dbt · b-perOct 28

[BigQuery] ADBC driver does not expose job statistics or CTAS row count

Part of #16932. Related: #16396, #14462. ### Problem BigQuery tables and incremental models show no row count in dbt v2. The default ADBC BigQuery driver (arrow-adbc `go/adbc/driver/bigquery`) attaches only `BIGQUERY:query_id` to the result schema metadata, so none of the job...

GitHub issuesairbytehq/airbyte · nasri-hOct 22

Shopify 4.3.0 — existing refunds remain PENDING after becoming SUCCESS

Hi Airbyte team, We’re seeing refund status changes missed by the `transactions` stream on **source-shopify 4.3.0**, using **Incremental Append + Deduped** with BigQuery. On October 22, we verified 20 refunds as SUCCESS through Shopify Admin GraphQL by **17:12 Paris time**. They...

Redditr/googlecloud · u/Pale-Stranger-9743Oct 20

Incident details | Google Cloud Service Health (europe-west2)

...down and affecting a bunch of services, in case you're wondering why something is not working. . Incident affecting AlloyDB for PostgreSQL, BigQuery Data Transfer Service, Cloud Data Fusion, Cloud Filestore, Cloud Firestore, Cloud Memorystore, Cloud Run, Cloud Spanner, Google...

GitHub issuesdbt-labs/dbt · jeremychiaOct 19

[1.x Bug] buildable indirect selection drops every test that reads a seed outside the selection

...in production on bigquery. the cause is in dbt-core's selector, not in the adapter. - **workarounds:** for pattern 2, add `-- depends_on: {{ ref(' ') }}` to the model. it renders as a sql comment, so the query does not change. pattern 1 has no model to put the ref in. the only...

2 replies
Stack OverflowStack Overflow [google-bigquery] · TomOct 13

How to use the JOIN clause

I don't quite understand how the JOIN clause works. For my course, I need to combine rows from two different tables. Table 1 holds information about the total order value, and table 2 holds information about the order date. I need to combine these two tables using the common...

GitHub issuesairbytehq/airbyte · sabburiOct 13

Please add a Services stream to the Hubspot Source connector

### Connector Name source-hubspot ### Connector Version 6.12.1 ### What step the error happened? Configuring a new connector ### Relevant information There is no services stream. We have data in the services object of Hubspot that we can't replicate to any other object. It would...

GitHub issuesapache/iceberg · patnaikankitOct 10

BigQuery: Retry transient network exceptions wrapped by RuntimeIOException

### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 ## Description `BigQueryMetastoreClientImpl` configures retries for `ConnectException` and `UnknownHostException`, but its [[exception...

1 replies
GitHub issuesdbt-labs/dbt · cfcalabreseOct 7

[v2 Bug] BigQuery: insert_overwrite + copy_partitions on a DATETIME partition column fails with "none has no method named strftime" (Storage-API-disabled client returns None for DATETIME)

...by comparing the two clients on literals (see Steps To Reproduce): The temp table in the failing runs has **no NULL partition values**, so this is not the NULL case described in dbt-labs/dbt-adapters#2086 (same symptom, different cause); filtering NULLs would not fix it. It...

GitHub issuesapache/iceberg · fanruan-tucoSep 11

API: Clarify atomicity requirements for SupportsNamespaces.dropNamespace

### Query engine None - custom catalog implementation backed by FoundationDB ### Question I am implementing a FoundationDB-backed catalog and would like to clarify the concurrency contract of `SupportsNamespaces.dropNamespace` before choosing the implementation semantics. The...

Redditr/bigquery · u/Expensive-Insect-317Sep 7

BigQuery slot contention is often not a SQL problem.

If a query suddenly takes 5-10x longer without any SQL or data changes, the bottleneck may be workload contention, capacity, or concurrency.. A practical guide on how to diagnose it using JOBS , JOBS_TIMELINE , total_slot_ms and period_estimated_runnable_units ..

Redditr/bigquery · u/datapopcornSep 7

Is Manage Permission in Bigquery Broken

Get "Error loading permissions for selected resources." error when I click selected table -> share -> Manage permissions.. Anyone know what has updated ?. https://preview.redd.it/1n3i1qpwd0oh1.png?width=1154&format=png&auto=webp&s=1592eb720b3843c451e90305a3b09b8716a37b69 .

Stack OverflowStack Overflow [apache-spark] · keisuke nebashiSep 4

Why does --packages work with spark-submit, but spark.jars.packages does not work with SparkSession.builder?

I am using PySpark to load data from BigQuery using the Spark BigQuery Connector .. When I pass the connector using --packages to spark-submit , everything works as expected:. spark-submit \ --packages com.google.cloud.spark:spark-4.1-bigquery:0.45.0 \ ./src/bigquery.py However,...

1 votes1 replies
Stack OverflowStack Overflow [google-bigquery] · Amber FarmerSep 1

Query doesn't recognize a table name

I am trying to join two tables in BigQuery. However, my code is generating an error that one of the table names is not recognized and I don't know why. Can you explain why this is happening, and fix my code so that it runs?. I have two tables:. DailyActivity3 DailyActivity4 I...

2 replies

Companies and products named

14 posts
4 posts
3 posts
2 posts
About this problem

Evidence

14 posts from 14 people in 7 places, about 2 a week over 59 days. Mostly on Stack Overflow [google-bigquery], dbt-labs/dbt, apache/iceberg. Tools named alongside: GitHub, dbt, Apache Iceberg, Airbyte.

Frustration Frustration 0 of 3· Seen on GitHub issues, Stack Overflow, Reddit

How it was grouped

Posts about BigQuery that state a pain but match no known issue yet. First post Sep 1, 2026, latest Oct 29, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

Other BigQuery problems

BigQuery error messages are cryptic

3 posts from 3 people on GitHub issues.

Trouble with BigQuery fail

Top post: "Unit tests fail with protected to the package for public models from a `restrict-access: true` package", one of 3 posts from 3 people on GitHub issues.

BigQuery runs out of memory

2 posts from 2 people on GitHub issues. Unverified.

BigQuery upgrades break working setups

2 posts from 2 people on GitHub issues. Unverified.

BigQuery permissions are confusing and error-prone

2 posts from 2 people on GitHub issues. Unverified.

Hard to trace lineage and find trusted data in BigQuery

2 posts from 1 person on GitHub issues. Unverified.

Trouble with BigQuery tests

One post on GitHub issues: "Add missing tests for the `google.cloud.utils` BigQuery helpers". Unverified.

History

Renamed from "BigQuery users keep filing bugs".

Nov 9, 2026

Momentum went from steady to fading.

Nov 5, 2026

Momentum went from fading to steady.

Oct 31, 2026

Momentum went from steady to fading.

Oct 29, 2026

Momentum went from fading to steady.

Oct 27, 2026

Momentum went from steady to fading.

Oct 26, 2026

Renamed from "BigQuery users keep asking for help".

Oct 23, 2026

Momentum went from fading to steady.

Oct 23, 2026

Renamed from "BigQuery users keep filing bugs".

Oct 21, 2026

Momentum went from steady to fading.

Oct 20, 2026

Momentum went from rising to steady.

Oct 17, 2026

Renamed from "BigQuery users keep asking for help".

Oct 16, 2026