Data and AI struggles

1,730 problems · 342 companies

Data engineering

pandas users keep filing bugs

26 posts from 19 people18d activeFading, +1 this week

Posts per day

Posts per day26 posts, Aug 27 to Nov 9

The posts behind it

Redditr/dataengineering · u/Salt_Mud3004Nov 4

Need Advice on an Automated Data Collection System

Hello developers!. I am developing a B2B buyer intelligence project.. I want to automatically collect, clean, store and update company information every day.. I am considering Python, Pandas, MongoDB and APScheduler.. Which architecture and open-source technologies would you...

GitHub issuespandas-dev/pandas · github-actions[bot]Nov 2

CI: pandas-stubs tests fail against pandas nightly

@pandas-dev/pandas-stubs-team the pandas-stubs tests are failing against pandas nightly. This issue is managed by the `stubs_tests` job in `.github/workflows/wheels.yml` and closes on the next passing run. Closing it by hand while the tests still fail opens a new one on the next...

GitHub issuespandas-dev/pandas · aulemahalOct 29

ENH: `ListAccessor.contains`

### Feature Type - [x] Adding new functionality to pandas - [ ] Changing existing functionality in pandas - [ ] Removing existing functionality in pandas ### Problem Description Since pandas 3.1, a Polars Dataframe columns with dtypes like `list [pyarrow]` have their dtype...

2 replies
GitHub issuesapache/superset · bmuraykhiOct 27

"Show full range for time shift" fails when the shifted period has no data (7.0.0rc2)

### Bug description With a time shift and **Show full range for time shift** (#41334) on, a chart fails with "Data error: One or more chart-data queries failed" when the shifted period has no rows. With the option off, the same chart renders fine. Present in `7.0.0rc2` and...

GitHub issuespandas-dev/pandas · jorisvandenbosscheOct 22

RLS: 3.1.x releases

Issue to track the 3.1.x bug-fix releases

GitHub issuespandas-dev/pandas · warwickmmOct 22

BUG: Logical OR does not obey Kleene three-valued logic

...: None IPython : None adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : None bottleneck : None fastparquet : None fsspec : None html5lib : None gcsfs : None jinja2 : None lxml.etree : None matplotlib : None numba : None numexpr : None odfpy : None openpyxl : None...

5 replies
GitHub issuespandas-dev/pandas · hrwOct 21

BUG: on RISC-V NaN goes to INT_MAX

...IPython : 9.17.1 adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : 4.15.0 bottleneck : 1.6.0 fastparquet : None fsspec : 2026.9.0 html5lib : 1.1 hypothesis : 6.151.9 gcsfs : None jinja2 : 3.1.6 lxml.etree : 6.1.2 matplotlib : 3.11.2 numba : None numexpr : 2.14.1 odfpy...

2 replies
GitHub issuespola-rs/polars · kdn36Oct 20

`dt.offset_by` keeps the sorted flag when the result overflows and wraps around

### Checks - [x] I have checked that this issue has not already been reported. - [x] I have confirmed this bug exists on the [latest version](https://pypi.org/project/polars/) of Polars. ### Reproducible example ### Log output ### Issue description 2262-04-11 + 1 day doesn't fit...

GitHub issuespandas-dev/pandas · 2xBOct 19

DOC: pyarrow.Array.to_pandas vs ArrowExtensionArray

### Pandas version checks - [x] I have checked that the issue still exists on the latest versions of the docs on `main` [here](https://pandas.pydata.org/docs/dev/) ### Location of the documentation https://pandas.pydata.org/docs/user_guide/pyarrow.html ### Documentation problem...

2 replies
GitHub issuespandas-dev/pandas · github-actions[bot]Oct 15

CI: pandas-stubs tests fail against pandas nightly

[Workflow Run URL](https://github.com/pandas-dev/pandas/actions/runs/38677170119) Python 3.13 Test Summary

3 replies
Stack OverflowStack Overflow [pandas] · StephanieOct 10

Does loadTable().to_pandas() transfer all data or only metadata first?

When using the Python API, I want to understand the data transfer behavior of loadTable vs to_pandas . The database table has millions of rows and I don't want to accidentally trigger a full transfer.. import dolphindb as ddb s = ddb.session() s.connect("localhost", 8848,...

2 votes1 replies
GitHub issuespandas-dev/pandas · github-actions[bot]Oct 10

CI: pandas-stubs tests fail against pandas nightly

[Workflow Run URL](https://github.com/pandas-dev/pandas/actions/runs/38201489513) Python 3.13 Test Summary

3 replies
GitHub issuespandas-dev/pandas · natmokvalOct 5

DOC: Fix docs signature for various offset constructors

### Pandas version checks - [x] I have checked that the issue still exists on the latest versions of the docs on `main` [here](https://pandas.pydata.org/docs/dev/) ### Location of the documentation Multiple offset classes in pandas.tseries.offsets. Examples: - BusinessDay -...

7 replies
Redditr/dataengineering · u/Ok_Procedure199Oct 5

How should I best structure my data for use for my analytics

Hello data engineers, I am here in order to get some help for a hobby. I'm trying to figure out how to structure my data. I'll try to explain the use-case in order for you guys to understand.. I analyze numbers, from different sources to create stories that explain what is going...

GitHub issuespandas-dev/pandas · cmp0xffOct 1

BUG: `pandas` allows comparing complex numbers

...None IPython : None adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : 4.15.0 bottleneck : None fastparquet : None fsspec : 2026.7.0 html5lib : 1.1 hypothesis : None gcsfs : None jinja2 : 3.1.6 lxml.etree : 6.1.1 matplotlib : 3.11.1 numba : None numexpr : 2.14.2 odfpy...

3 replies
GitHub issuespandas-dev/pandas · utsab345Sep 24

t2

b

GitHub issuespandas-dev/pandas · utsab345Sep 24

t

b

Stack OverflowStack Overflow [pandas] · SwankyTigerSep 17

Pandas 'drop_duplicates' not removing rows

I am reading a csv into a dataFrame with the goal of filtering duplicate entries and storing their id's. csv: . Name,Hash,Id,Tag,Rarity,Tier,Type,Source,Archetype,Tertiary Stat,Tuning Stat AION Adapter Bond,119770768,"""6917530158183792231""",,Legendary,5,Warlock...

GitHub issuespandas-dev/pandas · jose-matosSep 8

BUG: .to_latex() throws an exception in some cases

...IPython : 9.16.1 adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : 4.15.0 bottleneck : 1.6.0 fastparquet : None fsspec : 2026.7.0 html5lib : 1.1 hypothesis : 6.151.9 gcsfs : None jinja2 : 3.1.6 lxml.etree : 6.1.1 matplotlib : 3.11.1 numba : None numexpr : 2.14.1 odfpy...

10 replies
GitHub issuespandas-dev/pandas · jbrockmendelSep 4

TST: pandas/tests/api/test_api.py::TestApi::test_api fails when that directory is run on its own

`TestApi.allowed_api_dirs` in `pandas/tests/api/test_api.py` lists `"internals"`, and `test_api` compares that list against `dir(pandas.api)`. But `pandas/api/__init__.py` does not import the `internals` submodule, and a submodule only shows up in its parent's `dir()` once...

1 replies
GitHub issuespandas-dev/pandas · jbrockmendelSep 4

BUILD: window/meson.build never passes --shared=pandas._libs._cyutility to its extensions

`pandas/_libs/window/meson.build` opens with but both `py.extension_module` calls in that file pass `cython_args: cy_args`. The `+=` lands on `cython_args`, the variable inherited from `pandas/_libs/meson.build` (which already appends the same flag there), so the flag never...

1 replies
GitHub issuespandas-dev/pandas · jbrockmendelSep 4

BUILD: pyproject.toml banned-api comment is wrong — TID251 does resolve np.random through the alias

`pyproject.toml` carries this in `[tool.ruff.lint.flake8-tidy-imports.banned-api]`: The premise is false. TID251 resolves qualified names through aliased module imports, so with `import numpy as np` a `"numpy.random"` entry does flag `np.random.rand()` at the call site. Verified...

2 replies
GitHub issuesPrefectHQ/prefect · gabelsohnSep 1

Pandas is eagerly imported as a result of cache_policies.py

### Bug summary [`_register_stable_transforms()`](https://github.com/PrefectHQ/prefect/blob/3a128c2b55d57bf30f9db28c2148bba5e5b6c145/src/prefect/cache_policies.py#L404) in `cache_policies.py` eagerly imports pandas if a user happens to have it installed in their environment...

1 reactions
GitHub issuespandas-dev/pandas · jnettelsSep 1

BUG: IndexError with certain subplot order in stacked barchart

...IPython : 9.16.1 adbc-driver-postgresql: None adbc-driver-sqlite : None bs4 : 4.15.0 bottleneck : None fastparquet : None fsspec : None html5lib : None hypothesis : None gcsfs : None jinja2 : 3.1.6 lxml.etree : 6.1.1 matplotlib : 3.11.1 numba : 0.67.0 numexpr : None odfpy : None...

6 replies
GitHub issuespandas-dev/pandas · jbrockmendelSep 1

BUG: IntervalDtype.__from_arrow__ ignores the struct-level validity bitmap

### Reproducible Example ### Issue Description The second entry is null in the arrow array but reads back as `(0.0, 0.0]` instead of NA. `IntervalDtype.__from_arrow__` reads only `arr.field("left")` / `arr.field("right")`. pyarrow's `StructArray.field()` does not merge the...

3 replies
GitHub issuespandas-dev/pandas · jnettelsAug 28

BUG: Issues with stacked/grouped bars in subplot charts

### Pandas version checks - [x] I have checked that this issue has not already been reported. - [x] I have confirmed this bug exists on the [latest version](https://pandas.pydata.org/docs/whatsnew/index.html) of pandas. - [x] I have confirmed this bug exists on the [main...

2 replies

Companies and products named

26 posts
8 posts
6 posts
6 posts
2 posts
1 post
About this problem

Evidence

26 posts from 19 people in 6 places, about 3 a week over 69 days. Mostly on pandas-dev/pandas, Stack Overflow [pandas], r/dataengineering. Tools named alongside: GitHub, PostgreSQL, SQLite, Polars.

Frustration Frustration 0 of 3· Seen on GitHub issues, Stack Overflow, Reddit

Summary, model-written

Pandas library users face unwanted eager imports by external tools, unclear data transfer behaviors, and formatting issues when loading CSV files into dataframes.

How it was grouped

Posts about pandas that state a pain but match no known issue yet. First post Aug 28, 2026, latest Nov 4, 2026. Corroborated: 3 or more posts from 2 or more people or places. Method

Other pandas problems

pandas upgrades break working setups

10 posts from 10 people on GitHub issues.

Trouble with pandas dataframe

Top post: "HDFStore.select with where=cat == None returns empty DataFrame for Categorical columns instead of matching", one of 8 posts from 7 people on GitHub issues, Stack Overflow.

pandas deprecations force unplanned migrations

7 posts from 6 people on GitHub issues.

Trouble with pandas series

Top post: "Series arithmetic ops with PyArrow dtypes and scalars can upcast", one of 5 posts from 4 people on GitHub issues, Stack Overflow.

Trouble with pandas read

Top post: "CI: sanitizers environment has no pyarrow, so the read_csv pyarrow string path is never sanitized", one of 5 posts from 5 people on GitHub issues.

Trouble with pandas raises

Top post: "read_csv with usecols raises IndexError in low-memory mixed-dtype warning path", one of 5 posts from 5 people on GitHub issues.

pandas error messages are cryptic

5 posts from 5 people on GitHub issues.

Trouble with pandas index

Top post: "DOC: document `pd.Index.to_flat_index`", one of 4 posts from 4 people on GitHub issues.

Trouble with pandas instead

Top post: "`pd.merge` gives `float`s instead of integer extensions", one of 4 posts from 4 people on GitHub issues.

Critical work in pandas is fragile and manual

4 posts from 4 people on GitHub issues.

pandas builds are slow or flaky

4 posts from 1 person on GitHub issues. Unverified.

Trouble with pandas dtype

Top post: "pd.to_numeric with errors=coerce fails to produce actual NA values when applied to dtype string[pyarrow]", one of 3 posts from 3 people on GitHub issues.

History

Momentum went from steady to fading.

Nov 9, 2026

Momentum went from fading to steady.

Nov 4, 2026

Momentum went from steady to fading.

Nov 3, 2026

Momentum went from fading to steady.

Nov 2, 2026

Momentum went from steady to fading.

Oct 29, 2026

Momentum went from rising to steady.

Oct 27, 2026

Summary written by a model: Pandas library users face unwanted eager imports by external tools, unclear data transfer behaviors, and formatting issues when loading CSV files into dataframes.

Oct 24, 2026

Momentum went from fading to rising.

Oct 24, 2026

Momentum went from steady to fading.

Oct 18, 2026

Momentum went from rising to steady.

Oct 17, 2026

Momentum went from steady to rising.

Oct 16, 2026

Momentum went from fading to steady.

Oct 7, 2026