DylanLukes opened a new issue, #38391: URL: https://github.com/apache/arrow/issues/38391
### Describe the bug, including details regarding any error messages, version, and platform. ### Minimal Reproduction A minimal reproduction and run is available here: https://github.com/DylanLukes/pyarrow-ci-macos13-minimal-repro/actions/runs/6606055123/job/17941866516 ### Issue Description As PyArrow 13.0.0 has no wheels, I've been having to compile the Cython extensions for CI. The Arrow 13.0.0 header files make use of C++ features which are only available on macOS from 13 onwards, for example: ```c++ /usr/local/include/arrow/type.h:1844:29: error: 'get<std::string, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' is unavailable: introduced in macOS 10.13 return IsName() ? &std::get<std::string>(impl_) : NULLPTR; ``` It turns out that the `macos-latest` runner is, in fact, not the latest, that would be `macos-13`. Switching the runner to `macos-13` however doesn't fix the issue! What's going on? Inspecting the logs for my own project's failed CI: ``` Installing project in development mode error: subprocess-exited-with-error × Building wheel for pyarrow (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [468 lines of output] <string>:34: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html WARNING setuptools_scm.pyproject_reading toml section missing 'pyproject.toml does not contain a tool.setuptools_scm section' running bdist_wheel running build running build_py creating build creating build/lib.macosx-10.9-universal2-cpython-[31](https://github.com/DylanLukes/renkon/actions/runs/6600268842/job/17930093683#step:9:32)2 creating build/lib.macosx-10.9-universal2-cpython-312/pyarrow copying pyarrow/orc.py -> build/lib.macosx-10.9-universal2-cpython-312/pyarrow copying pyarrow/conftest.py -> build/lib.macosx-10.9-universal2-cpython-312/pyarrow ``` Mac OS X... **_10.9?_** **It's not clear why, but `setuptools` seems to be inferring the incorrect system version, which in turn propagates down to CMake, which then results in the aforementioned compilation error.** Checking the system version in a few ways I get: ``` > uname -a Darwin Mac-1697921125444.local 22.6.0 Darwin Kernel Version 22.6.0: Fri Sep 15 13:39:52 PDT 2023; root:xnu-8796.141.3.700.8~1/RELEASE_X86_64 x86_64 > gcc -dumpmachine x86_64-apple-darwin22.6.0 > python -c "import platform; print(platform.platform())" macOS-13.6-x86_64-i386-64bit ``` ### Workaround While it doesn't fix `setuptools` thinking it's building a wheel for Mac OS X 10.9 rather than macOS 13, we _can_ get CMake back on the right page with the following extra step: ```yml ... - if: runner.os == 'macOS' run: echo "MACOSX_DEPLOYMENT_TARGET=$(sw_vers -productVersion)" >> $GITHUB_ENV ... ``` ### Diagnostic Progress So Far I was pretty baffled by this. I'm not sure where `setuptools` is getting 10.9 from. I've not had any issue like this on a local Mac running macOS 14, it _seems_ specific to the GitHub Actions runner environment? I then went poking around the details of `pip inspect` and the contents of the packaged Python the setup step installs. AT first glance it appears to have been built against yet another different version (macOS 11), as it's marked as `python-3.12.0-macos11.pkg` in the accompanying `setup.sh`. But, if we actually run the package (or expand it with `pkgutil --expand` and take a look inside...), we find that in the accompanying license, readme, and welcome messages it refers to 10.9! > ``` > This package will install Python 3.12.0 for macOS 10.9 or later. > ``` Aha! That explains it. ### Thoughts It seems to me that despite this being the case, building pyarrow _should work_. Yes, 10.9 is not supported... but, we're not on 10.9. And this build _can_ work perfectly well, so long as CMake is not misled about which `MACOSX_DEPLOYMENT_TARGET` to use. So, if Arrow 13.0.0 _requires_ macOS 13 in its header files (it won't work on older systems anyways), it seems reasonable that `arrow/setup.py` should ensure that rather than deferring to a Python installation which may be misleading it. ### Component(s) C++, Continuous Integration, Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
