[Python-Dev] Cross compiling Python (for Android)

[Frank, Matthew I]

This email is about my experience getting CPython (3.4.1) to
cross-compile and run on x86 Android (4.4.2 with sdk 19 and ndk-r9).
I know that Android is not a supported architecture (and I won't
regale you with stories about the complete locale and mbstowcs support
I had to borrow from FreeBSD to get it working).  The purpose of this
email is that several things I found are arguably "bugs" in the Python
build system or code when it comes to cross-compiling that are exposed
by Android's poor Posix support.  I'd like some advice about what kind
of patch (if any) would be most suitable for fixing the problems on
the Python side.

Just to be complete:  I'm configuring with

 CPPFLAGS=-I../my-locale ../Python-3.4.1/configure --enable-shared
 --prefix=/path/to/install/dir --build=x86_64-linux-gnu
 --host=i686-linux-android --disable-ipv6 ac_cv_file__dev_ptmx=no
 ac_cv_file__dev_ptc=no ac_cv_little_endian_double=yes

(The CPPFLAGS addition is to get the headers for my locale fixes
instead of the default Android ones.  ac_cv_file__dev_ptmx=no and
ac_cv_file__dev_ptc=no are because I don't have /dev/whatever on my
build machine.  ac_cv_little_endian_double is because configure for
cross builds can't figure out the endianness of doubles on the host
(because it is running on the build machine not the host.)  (For ARM
it would be ac_cv_mixed_endian_double=yes.)

I've gotten to the point where `make; make install` succeeds up to the
point of building something that runs on my Android system (from the
command line) and `python -m test` runs 388 tests, with 321 ok, 24
test failures and 43 tests skipped (the skips mostly due, I think, to
me not yet having installed the right cross-building support for
things like bz2 and dbm.)

1. `make` succeeds but `make install` always fails at the end with
   something having to do with being unable to run "ensurepip"
   (presumably because ensurepip requires modules that only run on the
   host, not the build module.)  So it seems this should be wrapped in
   a test for cross compilation, but I haven't looked at exactly what
   yet.  The error is:

   /linux-python/bin/python3.4: Error while finding spec for
   'ensurepip.__main__' (:
   /build-directory/build/lib.linux-i686-3.4/time.cpython-34m.so:
   wrong ELF class: ELFCLASS32); 'ensurepip' is a package and cannot
   be directly executed make: *** [install] Error 1

2. setup.py is missing -lm flag for several modules.  On Linux this
   problem is hidden because libm is already loaded by the executable
   calling dlopen(), but Android's loader deals with unknown symbols
   differently (searches only the libs explicitly linked against the
   module being loaded.)  http://bugs.python.org/issue21668 reports
   the problem for selectmodule (can't find ceil()) and timemodule
   (fmod() and floor()).  But there are at least two more: audioop
   fails to load because it needs floor() and ctypes_test fails to
   load because it needs sqrt().  I'll happily update the patch in
   21668.

   Is there any fundamental objection to adding the -lm flag to the
   link step where it is necessary?

3. What is ossaudiodev?  It tries to include "sys/soundcard.h", which
   I don't have on my system.   (The rule in setup.py is
   wrapped in a test for host of Linux/FreeBSD/Darwin, but Android x86
   gets configured with --host=i686-linux-android so to turn it off
   requires an extra test for "and not cross_compiling".)

   Can I just turn off ossaudiodev for cross compiling or might
   someone want it in a different type of cross build?  (In which case
   I think I'll have to write some kind autoconf rule for it, which I
   don't quite know how to do yet.)

4. Module _decimal is failing to compile.  The problem is that it has
   a header called memory.h.  Android's libc has the problem that
   /usr/include/stdlib.h includes .  But the build system
   puts -I. on the include path before the system dirs (as it should)
   so when compiling _decimal, Modules/_decimal/libmpdec/memory.h gets
   found instead of /usr/include/memory.h.  Shiz has a patch here:
   https://github.com/rave-engine/python3-android/blob/master/mk/python/3.3.5/p\
ython-3.3.5-android-libmpdec.patch
   (which renames memory.h -> mpmemory.h) but I don't know

   a.  Is there a tracker for this yet?  and
   b.  Is Shiz's fix the desired one or should I be looking for
   another approach?  (Maybe modifying the -I flags for the build
   of just the build of _decimal or something?)

5. I'm not sure what test configure is actually doing for gethostby*()
   in a cross-compile environment.  In any case Android has a bug
   where gethostbyaddr_r() is declared in the headers, but not
   actually implemented in libc.  So I have to modify my pyconfig.h by
   hand to define HAVE_GETHOSTBYNAME and undef HAVE_GETHOSTBYNAME_R
   and HAVE_GETHOSTBYNAME_R_6_ARG.

   Is there a variable (like ac_cv_little_endian_double) that I can
   give to `configure` to make it set HAVE_GETHOSTB

Re: [Python-Dev] Undefined dlopen When Building Module On Android

[Frank, Matthew I]

Android's dlopen() works slightly differently than the normal Unix dlopen() in 
at least two different ways.  I haven't seen your particular problem, but 
that's probably because I'm cross-building CPython (building everything I need 
on a Linux machine, and then copying the install directory to the Android 
machine.)

(1) When building for Android you need to explicitly include the "-ldl" flag on 
the command line for the link step.  (In Linux the dlopen() routine is included 
in libc, so "-ldl" is not necessary).  I suspect that some part of your 
distutils was not linked correctly.  (Since I'm cross-building I've never run 
distutils on the Android side, which is probably why I've not seen this.)  Your 
best bet would be to run under a debugger and figure out which line of C code 
in which .so file is throwing the error.  Then going back to the build scripts 
and looking at how that .so file is getting linked.  (For an example of someone 
else having a similar problem see for example, 
http://stackoverflow.com/questions/25846927/git-built-on-android-throws-undefined-reference-to-dlopen-error).

(2) The other possibility has to do with a quirk in Android's dlopen() 
implementation.  On most legacy Unix systems (including Linux) when you 
dlopen() a library Z then all the already loaded libraries A, B, C, ... are 
searched for any dependences of Z (even if Z was not explicitly linked against 
any of A, B, C,...).  On Android (perhaps for security reasons) that's not 
true, if Z depends on A, then you need to have "-lA" when you link Z.  An 
example (and patch) for this problem is http://bugs.python.org/issue21668.

-Matt

> -Original Message-
> From: Cyd Haselton [mailto:chasel...@gmail.com]
> Sent: Friday, January 23, 2015 10:50 AM
> To: Brett Cannon
> Cc: Guido van Rossum; Python-Dev
> Subject: Re: [Python-Dev] Undefined dlopen When Building Module On Android
> 
> I guess I'll keep waiting...given the zero responses I've gotten from the 
> android side.
> :-(.
> 
> In the meantime...in case anyone is interested... since I have the working 
> binary, I
> executed it and went through each command in setup.py to see what throws the
> 'undefined reference to dlopen' error.
> Turns out that the distutils module is the culprit...specifically 
> distutils.core.
> 
> 
> On Thu, Jan 22, 2015 at 8:33 AM, Brett Cannon  wrote:
> > A mobile SIG is being formed, but it doesn't have a mailing list yet,
> > else that would be a good place to ask this question.
> >
> > On Wed Jan 21 2015 at 5:54:39 PM Guido van Rossum  wrote:
> >>
> >> Maybe try a list focused on Android development? Few people in the
> >> Python core development community have any Android experience. But
> >> the issues and context you offer seem to be related to the Android world, 
> >> not to
> Python.
> >> (dlopen is used by a lot of systems, not just Python.)
> >>
> >> On Wed, Jan 21, 2015 at 2:43 PM, Cyd Haselton  wrote:
> >>>
> >>> On Mon, Jan 19, 2015 at 5:23 PM, Cyd Haselton 
> >>> wrote:
> >>> > On Mon, Jan 19, 2015 at 8:51 AM, Cyd Haselton
> >>> > 
> >>> > wrote:
> >>> >> Hello,
> >>> >> I'm struggling with a build issue on Android; I've posted to the
> >>> >> general python list with no result, so I'm re-posting here in
> >>> >> hopes that someone can help.  If this is the wrong place feel
> >>> >> free to let me know.
> >>> >>
> >>> >> I'm attempting to build Python 2.7.8 on my Android device; I'm
> >>> >> using an environment that simulates a Linux filesystem within the
> >>> >> Android terminal using a port of fakechroot.  Within that
> >>> >> environment I've ported and/or bootstrapped a number of Linux
> >>> >> utilities (curl, git, openssl, gcc)
> >>> >>
> >>> >> I run ./configure, then make, and the executable and library are
> >>> >> built.  The problem occurs when build_ext is run; the newly built
> >>> >> python executable builds, then links _struct, and immediately
> >>> >> afterwards I get an 'undefined reference to dlopen' error.
> >>> >>
> >>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
> >>> >> --libraries='c dl m' -f, the 'undefined reference to dlopen'
> >>> >> error is thrown again.
> >>> >>
> >>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
> >>> >> --libraries='-lc -ldl -lm' -f the build continues past
> >>> >> _struct...even though ld throws the expected 'unable to find -l-lc' 
> >>> >> and other
> errors.
> >>> >>
> >>> >> Let me know if you need me to provide additional information.
> >>> >> Any help would be greatly appreciated.
> >>> >>
> >>> >> Cyd
> >>> >
> >>> >
> >>> > Additionally I took a strace of the error producing command. The
> >>> > following is (hopefully) a relevant portion minus the various 'no
> >>> > such file' lines before the correct lib is found (which it always
> >>> > is)
> >>> >
> >>> > 16513
> >>> > open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-
> 2.7.8/Lib/distutils/unixccompiler.py",
> >>> > O_RDONLY|O_LARGEFILE) = 3

Re: [Python-Dev] Import Fails in setup.py On Android

[Frank, Matthew I]

There’s now (as of a couple days ago) a python mobile-sig 
(https://mail.python.org/mailman/listinfo/mobile-sig).  While that group is 
much smaller than python-dev and probably not as knowledgeable about the 
Cpython source base, that’s where you’re going to find folks who have a vested 
interest in getting Python working on mobile platforms like Android.  (And who 
would be interested in hearing about your experiences, no matter whether they 
can help you or not.)

What you are doing (trying to run the entire C compiler toolchain on an Android 
machine, instead of using the cross-compilers that come in Google’s Android NDK 
(native development kit)) is way outside the mainstream.  (I.e., you’re not 
going to find very many people trying to do this here or anywhere.)

Since you compiled all the libraries that you are linking against, there is 
some possibility (likelihood) that the problem is in one of those libraries, 
not anywhere in the CPython source.  (Some other library was compiled without 
the necessary –dl flag.)  The right way to track down this problem (no matter 
where it is) is to run under gdb and type “where” when the program crashes.

The reason I suspect that the problem is in one of the libraries you compiled 
before building Python, rather than a problem in the CPython sources or build 
scripts, is that I don’t have this problem when I cross-compile Python 3.4.2 on 
a Linux machine, then take the result and copy it over to the Android machine.  
I’ve posted instructions and patches for successfully performing this cross 
compilation (for Android KitKat running on an x86) here: 
https://github.com/wandering-logic/android_x86_python-3.4.

Best,
-Matt

From: Cyd Haselton [mailto:chasel...@gmail.com]
Sent: Monday, February 02, 2015 3:25 PM
To: Ryan Gonzalez
Cc: Python-Dev
Subject: Re: [Python-Dev] Import Fails in setup.py On Android

No traceback unfortunately...the fakechroot in the environment throws the error 
and setup.py fails.

I'll roll back that change...any idea where I could find info about the 
original method?
On February 2, 2015 3:17:54 PM CST, Ryan Gonzalez 
mailto:rym...@gmail.com>> wrote:
In reality, things just got broken even more. I don't know when that patch was 
created, but it's now very out of date: importlib._bootstrap has no load 
function. That's what the error you're getting is telling you. Since it isn't 
getting to load anything, the issue seems "solved". Not really.

What's the full traceback for the undefined reference exception?

On Mon, Feb 2, 2015 at 2:04 PM, Cyd Haselton 
mailto:chasel...@gmail.com>> wrote:
Update: While waiting for replies I made the change referenced here: 
https://bugs.python.org/review/5309/diff2/12811:12826/setup.py?context=3&column_width=80

specifically changing
importlib. _bootstrap._SpecMethods(spec).load()
to
importlib._bootstrap.load(spec)

I no longer get a terminating 'undefined reference to dlopen' error, but I do 
get 'importing extensions failed' errors for each extension...like this:

*** WARNING: importing extension "_pickle" failed with : 'module' object has no attribute 'load'

On February 2, 2015 1:36:29 PM CST, Cyd Haselton 
mailto:chasel...@gmail.com>> wrote:
After fixing a segfault issue (many thanks Ryan) I'm back to the same issue I 
was having with Python 2.7.8; the newly built python throws an undefined 
reference to dlopen when running setup.py...specifically when 
importing just-built extensions

I've managed to narrow the problem down to the following line:

importlib._bootstrap._SpecMethods(spec).load()

Googling this brings up a few hits from bugs.python.org 
and not much else. I'm new to Python; any ideas what this does...or where I can 
read up on it for troubleshooting purposes?

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com



--
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's 
becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was 
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. - srean
Check out my website: http://kirbyfan64.github.io/

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com