Thanks for getting to the root of the issue.
Alex On Wed 5. Jun 2024 at 9.47, Khem Raj via lists.openembedded.org <raj.khem= [email protected]> wrote: > On Mon, Jun 3, 2024 at 9:56 PM Khem Raj <[email protected]> wrote: > > > > On Mon, Jun 3, 2024 at 10:49 AM Khem Raj <[email protected]> wrote: > > > > > > On Mon, Jun 3, 2024 at 9:35 AM Richard Purdie > > > <[email protected]> wrote: > > > > > > > > On Mon, 2024-06-03 at 09:25 -0700, Khem Raj via > lists.openembedded.org > > > > wrote: > > > > > loader expects it to be called with 'linux-gnu' e.g > > > > > rpds.cpython-312-x86_64-linux-musl.so is created with musl > > > > > but loader expects it to called > rpds.cpython-312-x86_64-linux-gnu.so > > > > > > > > > > Lets create the symlink to please the loader. > > > > > > > > > > Fixes > > > > > ImportError while importing test module '/usr/lib/python3-rpds- > > > > > py/ptest/tests/test_hash_trie_map.py'. > > > > > Hint: make sure your test modules/packages have valid Python names. > > > > > Traceback: > > > > > ../../python3.12/importlib/__init__.py:90: in import_module > > > > > return _bootstrap._gcd_import(name[level:], package, level) > > > > > tests/test_hash_trie_map.py:36: in <module> > > > > > from rpds import HashTrieMap > > > > > ../../python3.12/site-packages/rpds/__init__.py:1: in <module> > > > > > from .rpds import * > > > > > E ModuleNotFoundError: No module named 'rpds.rpds' > > > > > ERROR: tests/test_hash_trie_map.py:tests/test_hash_trie_map.py > > > > > > > > > > Signed-off-by: Khem Raj <[email protected]> > > > > > --- > > > > > .../recipes-devtools/python/python3-rpds-py_0.18.1.bb | 11 > > > > > +++++++++++ > > > > > 1 file changed, 11 insertions(+) > > > > > > > > > > diff --git a/meta/recipes-devtools/python/ > python3-rpds-py_0.18.1.bb > > > > > b/meta/recipes-devtools/python/python3-rpds-py_0.18.1.bb > > > > > index f46df1115c8..b1f0e100332 100644 > > > > > --- a/meta/recipes-devtools/python/python3-rpds-py_0.18.1.bb > > > > > +++ b/meta/recipes-devtools/python/python3-rpds-py_0.18.1.bb > > > > > @@ -27,4 +27,15 @@ do_install_ptest() { > > > > > cp -rf ${S}/tests/* ${D}${PTEST_PATH}/tests/ > > > > > } > > > > > > > > > > +do_install:append() { > > > > > + for f in ${D}/${PYTHON_SITEPACKAGES_DIR}/rpds/rpds.cpython*.so > > > > > + do > > > > > + fname=`basename $f` > > > > > + lname=`echo $fname | sed 's/musl/gnu/'` > > > > > + if [ "$fname" != "$lname" ]; then > > > > > + mv $f ${D}/${PYTHON_SITEPACKAGES_DIR}/rpds/$lname > > > > > + fi > > > > > + done > > > > > +} > > > > > + > > > > > BBCLASSEXTEND = "native nativesdk" > > > > > > > > > > > > > This feels like the kind of workaround that will live forever. Can we > > > > not fix the loader to use the right name? > > > > > > OE's python interpreter is cross-built and we end up using OSABI which > > > is detected during build 'linux-gnu', this seems to work fine for most > of > > > python modules since they get the OSABI from the sysconfigdata so it > > > all works even though the OSABI is called linux-gnu, however some > modules > > > like this one which are using maturin to build, seems to get the OSABI > > > equivalent differently, I am no expert on maturin but maybe someone > knows > > > how it happens. This problem is seen with several python modules built > with > > > OE, see > > > > > > https://github.com/meta-homeassistant/meta-homeassistant/issues/89 > > > https://github.com/pypa/auditwheel/issues/349 > > > > > > also see how it is fixed in python3-pydantic-core on meta-python > > > > https://git.openembedded.org/meta-openembedded/tree/meta-python/recipes-devtools/python/python3-pydantic-core_2.16.3.bb#n38 > > > > > > Hopefully musllinux ABI tag will be sorted in pypa and this will work > better > > > however, we also need to see how we are encoding OSABI in python builds > > > > > > e.g. on qemux86-64 musl target I get > > > > > > root@qemux86-64:~# python3 > > > Python 3.12.3 (main, Apr 9 2024, 08:09:14) [Clang 19.0.0 > > > (/home/kraj/work/llvm-project 4152c5f5ae88d8d5fe99d20b525b35f14db2 on > > > linux > > > Type "help", "copyright", "credits" or "license" for more information. > > > >>> import sysconfig > > > >>> sysconfig.get_config_var('SOABI') > > > 'cpython-312-x86_64-linux-gnu' > > > > > > > I looked further in pyo and maturin infrastructure and it is using > > ustc --target=<tuple> --print cfg to compute EXT_SUFFIX equivalent > > value which is then used to build the C extension (if any) inside the > > wheel, thats why we get cpython-312-x86_64-linux-musl for suffix, > > what pyo/maturin is doing is right, however how OE does it in python > > itself seems to be wrong to me, its synthesizing the OSABI > > incorrectly. > > where 'environment' value is not right. However, this is same > > everywhere so it works, so long as the modules are built using > > setuptools. > > since SOABI is part of "_sysconfigdata" and its an OE generated file, > > Perhaps it can consider the musl/gnu difference and emit SOABI > > accordingly. Something like below > > > > https://snips.sh/f/L6QspcdxjL > > > > might be able to fix the python interpreter's logic. > > To close this thread. I have sent an alternative patch which fixes it > in python3 itself. > > https://lore.kernel.org/openembedded-core/[email protected]/T/#t > > > > > > > > > > > Also, there is not even a comment above saying why we need to do > this. > > > > > > > > > > I added the log showing how it does not work, I thought that is > showing how > > > it's not working at all when using musl. If you have better idea what > I should > > > add to commit msg to make it better, let me know I will send a v2. > > > > > > > Cheers, > > > > > > > > Richard > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#200356): https://lists.openembedded.org/g/openembedded-core/message/200356 Mute This Topic: https://lists.openembedded.org/mt/106465307/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
