anentropic opened a new issue, #47587: URL: https://github.com/apache/arrow/issues/47587
### Describe the bug, including details regarding any error messages, version, and platform. I am using pyarrow in a Python 3.11 AWS Lambda function. This Python version means the OS is Amazon Linux 2, which in turn implies a particular GCC version. I started getting failures in my CI pipeline when we allowed pyarrow to update to latest 21.0.0 version: ``` 19.18 ✔ Install numpy 2.2.6 successful ... 38.85 ✖ Install pyarrow 21.0.0 failed 38.86 38.86 ERRORS: 38.86 add pyarrow failed: 38.86 Traceback (most recent call last): 38.86 File "/var/lang/lib/python3.11/concurrent/futures/thread.py", line 58, in run 38.86 result = self.fn(*self.args, **self.kwargs) 38.86 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/installers/synchronizers.py", 38.86 line 29, in install_candidate 38.86 self.manager.install(can) 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/installers/manager.py", line 38.86 33, in install 38.86 prepared.build(), 38.86 ^^^^^^^^^^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/models/candidates.py", line 38.86 426, in build 38.86 self._cached = Path(builder.build(build_dir, 38.86 metadata_directory=self._metadata_dir)) 38.86 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 38.86 ^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/builders/base.py", line 84, 38.86 in wrapper 38.86 return func(*args, **kwargs) 38.86 ^^^^^^^^^^^^^^^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/builders/wheel.py", line 23, 38.86 in build 38.86 self.install(self._requires, shared=****) 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/builders/base.py", line 323, 38.86 in install 38.86 install_requirements(missing, env, allow_uv=*****) 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/installers/core.py", line 45, 38.86 in install_requirements 38.86 syncer.synchronize() 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/installers/base.py", line 38.86 221, in synchronize 38.86 manager.install(can) 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/installers/manager.py", line 38.86 33, in install 38.86 prepared.build(), 38.86 ^^^^^^^^^^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/models/candidates.py", line 38.86 426, in build 38.86 self._cached = Path(builder.build(build_dir, 38.86 metadata_directory=self._metadata_dir)) 38.86 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 38.86 ^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/builders/base.py", line 84, 38.86 in wrapper 38.86 return func(*args, **kwargs) 38.86 ^^^^^^^^^^^^^^^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/builders/wheel.py", line 26, 38.86 in build 38.86 filename = self._hook.build_wheel(out_dir, self.config_settings, 38.86 metadata_directory) 38.86 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 38.86 ^^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pyproject_hooks/_impl.py", line 38.86 256, in build_wheel 38.86 return self._call_hook( 38.86 ^^^^^^^^^^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pyproject_hooks/_impl.py", line 38.86 392, in _call_hook 38.86 self._subprocess_runner( 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/builders/base.py", line 281, 38.86 in subprocess_runner 38.86 return log_subprocessor(cmd, cwd, extra_environ=env) 38.86 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 38.86 File "/var/lang/lib/python3.11/site-packages/pdm/builders/base.py", line 128, 38.86 in log_subprocessor 38.86 raise build_error(e) from None 38.86 pdm.exceptions.BuildError: Build backend raised error: Showing the last 10 lines 38.86 of the build output: 38.86 C linker for the host machine: cc ld.bfd 2.29.1-31 38.86 C++ compiler for the host machine: c++ (gcc 7.3.1 "c++ (GCC) 7.3.1 20180712 (Red 38.86 Hat 7.3.1-17)") 38.86 C++ linker for the host machine: c++ ld.bfd 2.29.1-31 38.86 Cython compiler for the host machine: cython (cython 3.1.4) 38.86 Host machine cpu family: aarch64 38.86 Host machine cpu: aarch64 38.86 38.86 ../meson.build:28:4: ERROR: Problem encountered: NumPy requires GCC >= 9.3 ``` AFAIK this issue is associated with numpy `2.3.0` which starts to depend on GCC >= 9.3 We are using [pdm](https://pdm-project.org/) as our installer. In our `pyproject.toml` we already pin `"numpy<2.3.0"` The build output reports: ``` 19.18 ✔ Install numpy 2.2.6 successful ... 38.85 ✖ Install pyarrow 21.0.0 failed ``` However if we also pin `"pyarrow<21.0.0"` then the build succeeds. I didn't see anything obvious in the [release notes](https://arrow.apache.org/release/21.0.0.html) so I am hoping this can be avoided. ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
