On 07.11.24 22:40, Ross Burton wrote:
On 6 Nov 2024, at 18:30, Konrad Weihmann via lists.openembedded.org
<[email protected]> wrote:
On Tue, Nov 5, 2024 at 01:56 PM, Ross Burton wrote:
I find that patch incredibly risky, as there is no really opt-out.
As the main beneficiary is pandas (which is not in core), which not add it
there as a patch?
You can opt out by setting PARALLEL_MAKE to -j1. Pandas is where I noticed,
but I expect it will make a difference for any package that builds extensions
with setuptools.
Also other suggest that all that can be achieved with --global-option="build_ext"
--global-option="-j5"
set by PEP517_BUILD_OPTS, which would find the much better options here.
Something like
PARALLEL_PEP517_BUILD ?= " --global-option="build_ext" --global-option="-j<job
count>""
PEP517_BUILD_OPTS:append = " ${PARALLEL_PEP517_BUILD}”
That’s possibly a neater solution, yes. I tried using global-option but
couldn’t get it to work right.
Ross
Here's a monkey patch to expose the missing option
diff --git a/setuptools/command/bdist_wheel.py
b/setuptools/command/bdist_wheel.py
index 8cf91538f..b692a7aa3 100644
--- a/setuptools/command/bdist_wheel.py
+++ b/setuptools/command/bdist_wheel.py
@@ -234,6 +234,11 @@ class bdist_wheel(Command):
"result of calling the PEP517 'prepare_metadata_for_build_wheel' "
"method)",
),
+ (
+ "parallel=",
+ "j",
+ "number of parallel build jobs",
+ ),
]
boolean_options = ["keep-temp", "skip-build", "relative", "universal"]
@@ -259,6 +264,7 @@ class bdist_wheel(Command):
self.build_number: str | None = None
self.py_limited_api: str | Literal[False] = False
self.plat_name_supplied = False
+ self.parallel = None
def finalize_options(self) -> None:
if not self.bdist_dir:
@@ -398,6 +404,7 @@ class bdist_wheel(Command):
build_ext = self.reinitialize_command("build_ext")
build_ext.inplace = False
+ build_ext.parallel = self.parallel
if not self.skip_build:
self.run_command("build")
feel free to propose that to upstream, as I think it's the better option in the
long run.
And yeah I have to agree for pandas it's like a day and night difference, on
20+ cores it's
almost fun to compile it now :-)
Cheers
Konrad
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#206856):
https://lists.openembedded.org/g/openembedded-core/message/206856
Mute This Topic: https://lists.openembedded.org/mt/109415037/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
