On Mon, Jun 3, 2024 at 10:49 AM Khem Raj <[email protected]> wrote: > > On Mon, Jun 3, 2024 at 9:35 AM Richard Purdie > <[email protected]> wrote: > > > > On Mon, 2024-06-03 at 09:25 -0700, Khem Raj via lists.openembedded.org > > wrote: > > > loader expects it to be called with 'linux-gnu' e.g > > > rpds.cpython-312-x86_64-linux-musl.so is created with musl > > > but loader expects it to called rpds.cpython-312-x86_64-linux-gnu.so > > > > > > Lets create the symlink to please the loader. > > > > > > Fixes > > > ImportError while importing test module '/usr/lib/python3-rpds- > > > py/ptest/tests/test_hash_trie_map.py'. > > > Hint: make sure your test modules/packages have valid Python names. > > > Traceback: > > > ../../python3.12/importlib/__init__.py:90: in import_module > > > return _bootstrap._gcd_import(name[level:], package, level) > > > tests/test_hash_trie_map.py:36: in <module> > > > from rpds import HashTrieMap > > > ../../python3.12/site-packages/rpds/__init__.py:1: in <module> > > > from .rpds import * > > > E ModuleNotFoundError: No module named 'rpds.rpds' > > > ERROR: tests/test_hash_trie_map.py:tests/test_hash_trie_map.py > > > > > > Signed-off-by: Khem Raj <[email protected]> > > > --- > > > .../recipes-devtools/python/python3-rpds-py_0.18.1.bb | 11 > > > +++++++++++ > > > 1 file changed, 11 insertions(+) > > > > > > diff --git a/meta/recipes-devtools/python/python3-rpds-py_0.18.1.bb > > > b/meta/recipes-devtools/python/python3-rpds-py_0.18.1.bb > > > index f46df1115c8..b1f0e100332 100644 > > > --- a/meta/recipes-devtools/python/python3-rpds-py_0.18.1.bb > > > +++ b/meta/recipes-devtools/python/python3-rpds-py_0.18.1.bb > > > @@ -27,4 +27,15 @@ do_install_ptest() { > > > cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/ > > > } > > > > > > +do_install:append() { > > > + for f in ${D}/${PYTHON_SITEPACKAGES_DIR}/rpds/rpds.cpython*.so > > > + do > > > + fname=`basename $f` > > > + lname=`echo $fname | sed 's/musl/gnu/'` > > > + if [ "$fname" != "$lname" ]; then > > > + mv $f ${D}/${PYTHON_SITEPACKAGES_DIR}/rpds/$lname > > > + fi > > > + done > > > +} > > > + > > > BBCLASSEXTEND = "native nativesdk" > > > > > > > This feels like the kind of workaround that will live forever. Can we > > not fix the loader to use the right name? > > OE's python interpreter is cross-built and we end up using OSABI which > is detected during build 'linux-gnu', this seems to work fine for most of > python modules since they get the OSABI from the sysconfigdata so it > all works even though the OSABI is called linux-gnu, however some modules > like this one which are using maturin to build, seems to get the OSABI > equivalent differently, I am no expert on maturin but maybe someone knows > how it happens. This problem is seen with several python modules built with > OE, see > > https://github.com/meta-homeassistant/meta-homeassistant/issues/89 > https://github.com/pypa/auditwheel/issues/349 > > also see how it is fixed in python3-pydantic-core on meta-python > https://git.openembedded.org/meta-openembedded/tree/meta-python/recipes-devtools/python/python3-pydantic-core_2.16.3.bb#n38 > > Hopefully musllinux ABI tag will be sorted in pypa and this will work better > however, we also need to see how we are encoding OSABI in python builds > > e.g. on qemux86-64 musl target I get > > root@qemux86-64:~# python3 > Python 3.12.3 (main, Apr 9 2024, 08:09:14) [Clang 19.0.0 > (/home/kraj/work/llvm-project 4152c5f5ae88d8d5fe99d20b525b35f14db2 on > linux > Type "help", "copyright", "credits" or "license" for more information. > >>> import sysconfig > >>> sysconfig.get_config_var('SOABI') > 'cpython-312-x86_64-linux-gnu' >
I looked further in pyo and maturin infrastructure and it is using ustc --target=<tuple> --print cfg to compute EXT_SUFFIX equivalent value which is then used to build the C extension (if any) inside the wheel, thats why we get cpython-312-x86_64-linux-musl for suffix, what pyo/maturin is doing is right, however how OE does it in python itself seems to be wrong to me, its synthesizing the OSABI incorrectly. where 'environment' value is not right. However, this is same everywhere so it works, so long as the modules are built using setuptools. since SOABI is part of "_sysconfigdata" and its an OE generated file, Perhaps it can consider the musl/gnu difference and emit SOABI accordingly. Something like below https://snips.sh/f/L6QspcdxjL might be able to fix the python interpreter's logic. > > > > Also, there is not even a comment above saying why we need to do this. > > > > I added the log showing how it does not work, I thought that is showing how > it's not working at all when using musl. If you have better idea what I should > add to commit msg to make it better, let me know I will send a v2. > > > Cheers, > > > > Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#200289): https://lists.openembedded.org/g/openembedded-core/message/200289 Mute This Topic: https://lists.openembedded.org/mt/106465307/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
