kurtrwall opened a new issue, #45369: URL: https://github.com/apache/arrow/issues/45369
### Describe the bug, including details regarding any error messages, version, and platform. I expect installing `pyarrow` on my Apple M3 chipped computer to work. There's even a wheel that I expect to package prebuilt libs for my system @ https://files.pythonhosted.org/packages/f5/b9/ba07ed3dd6b6e4f379b78e9c47c50c8886e07862ab7fa6339ac38622d755/pyarrow-19.0.0-cp313-cp313-macosx_12_0_arm64.whl I think there's missing libraries, since the error complains about not finding `libhdfs.dylib` where it's looking. Hadoop doesn't install `lib/native` via brew anymore, which is the other location I'd expect to find it. **Can someone provide instruction or assist?** ### Versions - pyarrow: 19.0.0 - python: 3.12 - arch: arm64 - os: OSX Sequoia 15.2 - chip: Apple M3 Max ### Reproduce 1. Set CLASSPATH ```shell $ export CLASSPATH=$(hdfs classpath --glob) ``` 2. Execute python ```python import pyarrow.fs hdfs = pyarrow.fs.HadoopFileSystem('default', 0) hdfs.ls('/') ``` ### Error output ``` Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/toro/code/bll-python/bll-fs/src/bll/fs/hdfs.py", line 68, in ls return self.hdfs.get_file_info(select) ^^^^^^^^^ File "/Users/toro/code/bll-python/bll-fs/src/bll/fs/hdfs.py", line 20, in hdfs self._hdfs = pyarrow.fs.HadoopFileSystem(host, port=port, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pyarrow/_hdfs.pyx", line 97, in pyarrow._hdfs.HadoopFileSystem.__init__ File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status OSError: Unable to load libhdfs dlopen(libhdfs.dylib) failed: dlopen(libhdfs.dylib, 0x0006): tried: 'libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibhdfs.dylib' (no such file), '/Users/toro/.pyenv/versions/3.12.7/envs/airflow/lib/python3.12/site-packages/pyarrow/libhdfs.dylib' (no such file), '/Users/toro/.pyenv/versions/3.12.7/lib/libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/toro/.pyenv/versions/3.12.7/lib/libhdfs.dylib' (no such file), '/opt/homebrew/lib/libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/libhdfs.dylib' (no such file), '/usr/lib/libhdfs.dylib' (no such file, not in dyld cache), 'libhdfs.dylib' (no such file) dlopen(./libhdfs.dylib) failed: dlopen(./libhdfs.dylib, 0x0006): tried: './libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS./libhdfs.dylib' (no such file), '/Users/toro/.pyenv/versions/3.12.7/envs/airflow/lib/python3.12/site-packages/pyarrow/./libhdfs.dylib' (no such file), '/Users/toro/.pyenv/versions/3.12.7/lib/./libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/toro/.pyenv/versions/3.12.7/lib/./libhdfs.dylib' (no such file), '/opt/homebrew/lib/./libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/lib/./libhdfs.dylib' (no such file), '/usr/lib/./libhdfs.dylib' (no such file, not in dyld cache), './libhdfs.dylib' (no such file), '/Users/toro/code/airflow/libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/toro/code/airflow/libhdfs.dylib' (no such file), '/Users/toro/code/airflow/libhdfs.dylib' (no such file) dlopen(/opt/hadoop/lib/native/libhdfs.dylib) failed: dlopen(/opt/hadoop/lib/native/libhdfs.dylib, 0x0006): tried: '/opt/hadoop/lib/native/libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/hadoop/lib/native/libhdfs.dylib' (no such file), '/opt/hadoop/lib/native/libhdfs.dylib' (no such file), '/opt/hadoop-3.3.5/lib/native/libhdfs.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/hadoop-3.3.5/lib/native/libhdfs.dylib' (no such file), '/opt/hadoop-3.3.5/lib/native/libhdfs.dylib' (no such file) ``` ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
