vadimkantorov opened a new issue, #46151:
URL: https://github.com/apache/arrow/issues/46151
### Describe the bug, including details regarding any error messages,
version, and platform.
I was doing something like this:
```python
import numpy as np
import pyarrow.parquet as pq
tab = pq.read_table("non-existing.parquet")
```
This failed with:
```
tab = pq.read_table("train-00000-of-00004.parquet")
File
"/home/inferencer/.local/lib/python3.10/site-packages/pyarrow/parquet/core.py",
line 1793, in read_table
dataset = ParquetDataset(
File
"/home/inferencer/.local/lib/python3.10/site-packages/pyarrow/parquet/core.py",
line 1291, in __init__
import pyarrow.dataset as ds
File
"/home/inferencer/.local/lib/python3.10/site-packages/pyarrow/dataset.py", line
24, in <module>
from pyarrow._dataset import ( # noqa
File "pyarrow/_dataset.pyx", line 159, in init pyarrow._dataset
File "pyarrow/_compute.pyx", line 2715, in
pyarrow._compute.Expression._scalar
File "pyarrow/scalar.pxi", line 1277, in pyarrow.lib.scalar
File "pyarrow/array.pxi", line 57, in pyarrow.lib._is_array_like
File "pyarrow/pandas-shim.pxi", line 126, in
pyarrow.lib._PandasAPIShim._have_pandas_internal
File "pyarrow/types.pxi", line 105, in
pyarrow.lib._datatype_to_pep3118
File "pyarrow/pandas-shim.pxi", line 108, in
pyarrow.lib._PandasAPIShim._check_import
File "pyarrow/pandas-shim.pxi", line 50, in
pyarrow.lib._PandasAPIShim._import_pandas
File
"/home/inferencer/.local/lib/python3.10/site-packages/pandas/__init__.py", line
22, in <module>
from pandas.compat import is_numpy_dev as _is_numpy_dev
File
"/home/inferencer/.local/lib/python3.10/site-packages/pandas/compat/__init__.py",
line 15, in <module> from
pandas.compat.numpy import (
File
"/home/inferencer/.local/lib/python3.10/site-packages/pandas/compat/numpy/__init__.py",
line 4, in <module> from
pandas.util.version import Version
File
"/home/inferencer/.local/lib/python3.10/site-packages/pandas/util/__init__.py",
line 1, in <module> from
pandas.util._decorators import ( # noqa:F401
File
"/home/inferencer/.local/lib/python3.10/site-package
s/pandas/util/_decorators.py", line 14, in <module>
from pandas._libs.properties import cache_readonly #
noqa:F401
File
"/home/inferencer/.local/lib/python3.10/site-packages/pandas/_libs/__init__.py",
line 13, in <module> from
pandas._libs.interval import Interval
File
"pandas/_libs/interval.pyx", line 1, in init pandas._libs.interval
ValueError: numpy.dtype size changed, may indicate binary incompatibility.
Expected 96 from C header, got 88 from PyObject
```
This is quite strange, because I hoped that pyarrow does not depend on
pandas / pandas installed correctly. And it was strange that this threw only at
`pq.read_table` call, and even before checking if file existed.
I thought that pandas depended on pyarrow and not in reverse. Is it not
true? And in any case, some explicit error message with minimal required pandas
version might be more clear to the user.
This was with: `pandas 1.4.3 and pyarrow-19.0.1`
After I did `python -m pip uinstall pyarrow pandas` and reinstalled again
(got `pandas-2.2.3 pyarrow-19.0.1`), the problem fixed itself.
Thanks!
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
