[Python-Dev] Code review for PEP 448

[Neil Girdhar]

Hello all,

PEP 448 (https://www.python.org/dev/peps/pep-0448/) is mostly implemented
now based on some early implementations by twouters (in 2008) and fhahn
(2013) and recently by Joshua and I.

The issue tracker http://bugs.python.org/issue2292  has:
* a working patch, and
* discussion and updates to the PEP (the most conservative interpretations
were taken).

I was wondering if anyone would mind reviewing the patch or even just
trying it out to let us know if there are any corner cases that we missed.
 (I need to get back to my actual work, so it would be good to have this
reviewed before I forget everything if possible.)

Thank you,

Neil
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

Absolutely.  Good thing I have addr2line on device

/bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0 0008bbc8
_PyMem_RawStrdup
/bld/python/Python-3.4.2/Objects/obmalloc.c:323
/bld/python/Python-3.4.2 $



On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
> Could you try the steps at http://stackoverflow.com/a/11369475/2097780? They
> allow you to get a better idea of where libc is crashing.
>
> Cyd Haselton  wrote:
>>
>> Managed to get this out of logcat:
>> F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>> 11914 (python)  (libc)
>>
>> [ 01-29 19:30:55.855 23373:23373 F/libc ]
>> Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373 (python)
>>
>> Less detail than strace but it seems to be that python is segfaulting
>> libc...
>>
>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez  wrote:
>>>
>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum 
>>> wrote:


  What I see in the strace:

  ... load libpython3.4m.so.1.0
  ... load libm
  ... open /dev/__properties__ and do something to it
 (what?)
  ... get current time
  ... allocate memory
  ... getuid
  ... segfault

  That's not a lot to go on, but it doesn't look as if it has started to
  load modules yet.

  Does /dev/__properties__ ring a bell? Not to me.
>>>
>>>
>>>
>>>
>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>>  is the code that works with that file.
>>>
>>>  This explains it a bit (slides 24-29). Looks like something to do with
>>>  interprocess communication. Likely has nothing to do with Python itself.
>>>
>>>  Maybe this would be useful?
>>>

  That stack trace would be really helpful.

  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton 
 wrote:
>
>
>  Apologies...I'm not sure what a stack track is, but I do have the
>  strace.  Nearest I can tell, it happens due to an open call, though I
>  am probably wrong.
>  Attaching the strace output to this email.  I'm going to head back to
>  the documentation and to back out of some Android-related changes in
>  _localemodule.c
>
>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum 
>  wrote:
>>
>>  There could be a million differences relevant (unicode, ints, ...).
>>  Perhaps
>>  the importlib bootstrap is failing. Perhaps the dynamic loading code
>>  changed. Did you get a stack track? (IIRC strace shows a syscall
>> trace
>>  --
>>  also useful, but doesn't tell you precisely how
>> it segfaulted.)
>>
>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton 
>>  wrote:
>>>
>>>
>>>  All,
>>>  I recently ditched my attempts to port Python 2.7.8 to Android in
>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>>> configure
>>>  options in the same environment, and modifying the setup.py as
>>> needed,
>>>  the newly built binary throws a segfault when the
>>> generate-posix-vars
>>>  portion of the build is reached...and when it is run as well (i.e.
>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>>>
>>>  I took a strace of ./python, however I'm a bit lost when reviewing
>>> it.
>>>  Any ideas as to what may be going on...i.e. why Python 2.7 works but
>>>  3.x throws a segfault?
>>>
>>>  Thanks in advance,
>>>  Cyd
>>> 
>>>
>>>  Python-Dev mailing list
>>>
>>> Python-Dev@python.org
>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>  Unsubscribe:
>>>
>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>>
>>
>>
>>  --
>>  --Guido van Rossum (python.org/~guido)





  --
  --Guido van Rossum (python.org/~guido)

 

  Python-Dev mailing list
  Python-Dev@python.org
  https://mail.python.org/mailman/listinfo/python-dev
  Unsubscribe:
  https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>>
>>>
>>>
>>>
>>>
>>>  --
>>>  Ryan
>>>  If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>>>  nul-terminated."
>>>  Personal reality distortion fields are immune to contradictory evidence.
>>> -
>>>  srean
>>>  Check out my website: http://kirbyfan64.github.io/
>
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> Check out my website: http://kirbyfan64.github.io/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

There's a related strdup patch for readline.c, mentioned
here:http://bugs.python.org/issue21390 and here
https://github.com/rave-engine/python3-android/issues/2.
There's a patch, but I'm not sure how to modify it for obmalloc.c, as
(I think) the functions all belong to Python...they're all prefixed
with _PyXx

On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton  wrote:
> Absolutely.  Good thing I have addr2line on device
>
> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0 
> 0008bbc8
> _PyMem_RawStrdup
> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> /bld/python/Python-3.4.2 $
>
>
>
> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
>> Could you try the steps at http://stackoverflow.com/a/11369475/2097780? They
>> allow you to get a better idea of where libc is crashing.
>>
>> Cyd Haselton  wrote:
>>>
>>> Managed to get this out of logcat:
>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>>> 11914 (python)  (libc)
>>>
>>> [ 01-29 19:30:55.855 23373:23373 F/libc ]
>>> Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373 (python)
>>>
>>> Less detail than strace but it seems to be that python is segfaulting
>>> libc...
>>>
>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez  wrote:

  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum 
 wrote:
>
>
>  What I see in the strace:
>
>  ... load libpython3.4m.so.1.0
>  ... load libm
>  ... open /dev/__properties__ and do something to it
> (what?)
>  ... get current time
>  ... allocate memory
>  ... getuid
>  ... segfault
>
>  That's not a lot to go on, but it doesn't look as if it has started to
>  load modules yet.
>
>  Does /dev/__properties__ ring a bell? Not to me.




 https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
  is the code that works with that file.

  This explains it a bit (slides 24-29). Looks like something to do with
  interprocess communication. Likely has nothing to do with Python itself.

  Maybe this would be useful?

>
>  That stack trace would be really helpful.
>
>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton 
> wrote:
>>
>>
>>  Apologies...I'm not sure what a stack track is, but I do have the
>>  strace.  Nearest I can tell, it happens due to an open call, though I
>>  am probably wrong.
>>  Attaching the strace output to this email.  I'm going to head back to
>>  the documentation and to back out of some Android-related changes in
>>  _localemodule.c
>>
>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum 
>>  wrote:
>>>
>>>  There could be a million differences relevant (unicode, ints, ...).
>>>  Perhaps
>>>  the importlib bootstrap is failing. Perhaps the dynamic loading code
>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
>>> trace
>>>  --
>>>  also useful, but doesn't tell you precisely how
>>> it segfaulted.)
>>>
>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton 
>>>  wrote:


  All,
  I recently ditched my attempts to port Python 2.7.8 to Android in
  favor of Python 3.4.2.  Unfortunately, after using the same
 configure
  options in the same environment, and modifying the setup.py as
 needed,
  the newly built binary throws a segfault when the
 generate-posix-vars
  portion of the build is reached...and when it is run as well (i.e.
  ./python --help, ./python -E -S -m sysconfig, or similar)

  I took a strace of ./python, however I'm a bit lost when reviewing
 it.
  Any ideas as to what may be going on...i.e. why Python 2.7 works but
  3.x throws a segfault?

  Thanks in advance,
  Cyd
 

  Python-Dev mailing list

 Python-Dev@python.org
  https://mail.python.org/mailman/listinfo/python-dev
  Unsubscribe:

 https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>>
>>>
>>>
>>>
>>>  --
>>>  --Guido van Rossum (python.org/~guido)
>
>
>
>
>
>  --
>  --Guido van Rossum (python.org/~guido)
>
> 
>
>  Python-Dev mailing list
>  Python-Dev@python.org
>  https://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe:
>  https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com





  --
  Ryan
  If anybody ever asks me why I prefer C++ to C, my answer will be simple:
  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
  nul-terminated."
  Personal reality distortion fields are immune to contradictory evidence.

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2015-01-23 - 2015-01-30)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open4764 ( +4)
  closed 30351 (+44)
  total  35115 (+48)

Open issues with patches: 2224 


Issues opened (38)
==

#21076: Turn signal.SIG* constants into enums
http://bugs.python.org/issue21076  reopened by serhiy.storchaka

#23306: Within zipfile, use of zlib.crc32 raises OverflowError at argu
http://bugs.python.org/issue23306  opened by Danny.Yoo

#23308: a bug in Instructions section 4.1
http://bugs.python.org/issue23308  opened by Dmot

#23309: Hang on interpreter shutdown if daemon thread prints to stdout
http://bugs.python.org/issue23309  opened by marienz

#23310: MagicMock constructor configuration fails for magic methods
http://bugs.python.org/issue23310  opened by berdario

#23311: Update PC/example_nt and extending/windows.rst
http://bugs.python.org/issue23311  opened by steve.dower

#23312: google thinks the docs are mobile unfriendly
http://bugs.python.org/issue23312  opened by benjamin.peterson

#23314: Disabling CRT asserts in debug build
http://bugs.python.org/issue23314  opened by steve.dower

#23315: tempfile.mkdtemp fails with non-ascii paths on Python 2
http://bugs.python.org/issue23315  opened by akira

#23316: Incorrect evaluation order of function arguments with *args
http://bugs.python.org/issue23316  opened by Joshua.Landau

#23317: Incorrect description of descriptor invocation in Python Langu
http://bugs.python.org/issue23317  opened by Justin.Eldridge

#23319: Missing SWAP_INT in I_set_sw
http://bugs.python.org/issue23319  opened by mgautierfr

#23320: devguide should mention rules about "paragraph reflow" in the 
http://bugs.python.org/issue23320  opened by akira

#23321: Crash in str.decode() with special error handler
http://bugs.python.org/issue23321  opened by serhiy.storchaka

#23322: parser module docs missing second example
http://bugs.python.org/issue23322  opened by Devin Jeanpierre

#23323: Issue with imaplib and append messages passing a tuple with fl
http://bugs.python.org/issue23323  opened by Pilessio

#23324: Nonblocking serial io using Arduino and Ubuntu 14.10 (Python 3
http://bugs.python.org/issue23324  opened by MrYsLab

#23325: Turn SIG_DFL and SIG_IGN into functions
http://bugs.python.org/issue23325  opened by serhiy.storchaka

#23326: Remove redundant __ne__ implementations
http://bugs.python.org/issue23326  opened by serhiy.storchaka

#23327: zipimport to import from non-ascii pathname on Windows
http://bugs.python.org/issue23327  opened by amswap

#23328: urllib2 fails for proxy credentials that contain a '/' charact
http://bugs.python.org/issue23328  opened by Andy.Reitz

#23330: h2py.py regular expression missing
http://bugs.python.org/issue23330  opened by Thomas.Roos

#23331: Add non-interactive version of Bdb.runcall
http://bugs.python.org/issue23331  opened by etanol

#23332: datetime.isoformat() -> explicitly mark UTC string as such
http://bugs.python.org/issue23332  opened by mirkovogt

#2: asyncio: add a new Protocol.connection_failed() method
http://bugs.python.org/issue2  opened by haypo

#23336: Thread.LockType is misnamed
http://bugs.python.org/issue23336  opened by cindykrafft

#23337: Run python with restricted rights
http://bugs.python.org/issue23337  opened by marcd

#23338: PyErr_Format in ctypes uses invalid parameter
http://bugs.python.org/issue23338  opened by mkato

#23342: run() - unified high-level interface for subprocess
http://bugs.python.org/issue23342  opened by takluyver

#23344: Faster marshalling
http://bugs.python.org/issue23344  opened by serhiy.storchaka

#23345: test_ssl fails on OS X 10.10.2 with latest patch level of Open
http://bugs.python.org/issue23345  opened by ned.deily

#23346: shutil.rmtree doesn't work correctly on FreeBSD.
http://bugs.python.org/issue23346  opened by negval

#23348: distutils.LooseVersion fails to compare two valid versions
http://bugs.python.org/issue23348  opened by maethor

#23349: PyBuffer_ToContiguous() off-by-one error for non-contiguous bu
http://bugs.python.org/issue23349  opened by rhansen

#23350: Content-length is incorrect when request body is a list or tup
http://bugs.python.org/issue23350  opened by demian.brecht

#23351: socket.settimeout(5.0) does not have any effect
http://bugs.python.org/issue23351  opened by piotrjurkiewicz

#23352: PyBuffer_IsContiguous() sometimes returns wrong result if subo
http://bugs.python.org/issue23352  opened by rhansen

#23353: gnerator bug with exception: tstate->exc_value is not cleared 
http://bugs.python.org/issue23353  opened by haypo



Most recent 15 issues with no replies (15)
==

#23349: PyBuffer_ToContiguous() off-by-one error for non-contiguous bu
http://bugs.python.org/issue23349

#23344: Faster marshalling
http://bugs.python.org/issue23344

#2333

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Ryan Gonzalez]

Is it possible at all to get a stack trace of the crash using gdb? Try the
steps here .

That way we can see where Python's own strdup function is getting called.

On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton  wrote:

> Absolutely.  Good thing I have addr2line on device
>
> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
> 0008bbc8
> _PyMem_RawStrdup
> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> /bld/python/Python-3.4.2 $
>
>
>
> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
> > Could you try the steps at http://stackoverflow.com/a/11369475/2097780?
> They
> > allow you to get a better idea of where libc is crashing.
> >
> > Cyd Haselton  wrote:
> >>
> >> Managed to get this out of logcat:
> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
> >> 11914 (python)  (libc)
> >>
> >> [ 01-29 19:30:55.855 23373:23373 F/libc ]
> >> Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373 (python)
> >>
> >> Less detail than strace but it seems to be that python is segfaulting
> >> libc...
> >>
> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
> wrote:
> >>>
> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum 
> >>> wrote:
> 
> 
>   What I see in the strace:
> 
>   ... load libpython3.4m.so.1.0
>   ... load libm
>   ... open /dev/__properties__ and do something to it
>  (what?)
>   ... get current time
>   ... allocate memory
>   ... getuid
>   ... segfault
> 
>   That's not a lot to go on, but it doesn't look as if it has started
> to
>   load modules yet.
> 
>   Does /dev/__properties__ ring a bell? Not to me.
> >>>
> >>>
> >>>
> >>>
> >>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >>>  is the code that works with that file.
> >>>
> >>>  This explains it a bit (slides 24-29). Looks like something to do with
> >>>  interprocess communication. Likely has nothing to do with Python
> itself.
> >>>
> >>>  Maybe this would be useful?
> >>>
> 
>   That stack trace would be really helpful.
> 
>   On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton 
>  wrote:
> >
> >
> >  Apologies...I'm not sure what a stack track is, but I do have the
> >  strace.  Nearest I can tell, it happens due to an open call, though
> I
> >  am probably wrong.
> >  Attaching the strace output to this email.  I'm going to head back
> to
> >  the documentation and to back out of some Android-related changes in
> >  _localemodule.c
> >
> >  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum  >
> >  wrote:
> >>
> >>  There could be a million differences relevant (unicode, ints, ...).
> >>  Perhaps
> >>  the importlib bootstrap is failing. Perhaps the dynamic loading
> code
> >>  changed. Did you get a stack track? (IIRC strace shows a syscall
> >> trace
> >>  --
> >>  also useful, but doesn't tell you precisely how
> >> it segfaulted.)
> >>
> >>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton  >
> >>  wrote:
> >>>
> >>>
> >>>  All,
> >>>  I recently ditched my attempts to port Python 2.7.8 to Android in
> >>>  favor of Python 3.4.2.  Unfortunately, after using the same
> >>> configure
> >>>  options in the same environment, and modifying the setup.py as
> >>> needed,
> >>>  the newly built binary throws a segfault when the
> >>> generate-posix-vars
> >>>  portion of the build is reached...and when it is run as well (i.e.
> >>>  ./python --help, ./python -E -S -m sysconfig, or similar)
> >>>
> >>>  I took a strace of ./python, however I'm a bit lost when reviewing
> >>> it.
> >>>  Any ideas as to what may be going on...i.e. why Python 2.7 works
> but
> >>>  3.x throws a segfault?
> >>>
> >>>  Thanks in advance,
> >>>  Cyd
> >>> 
> >>>
> >>>  Python-Dev mailing list
> >>>
> >>> Python-Dev@python.org
> >>>  https://mail.python.org/mailman/listinfo/python-dev
> >>>  Unsubscribe:
> >>>
> >>>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >>
> >>
> >>
> >>
> >>
> >>  --
> >>  --Guido van Rossum (python.org/~guido)
> 
> 
> 
> 
> 
>   --
>   --Guido van Rossum (python.org/~guido)
> 
>  
> 
>   Python-Dev mailing list
>   Python-Dev@python.org
>   https://mail.python.org/mailman/listinfo/python-dev
>   Unsubscribe:
> 
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>  --
> >>>  Ryan
> >>>  If anybody ever asks me why I prefer C++ to C, my answer will be
> simple:
> >>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
> was
> >>>  nul-terminated."
> >>>  Personal

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

Alternatively, is there a hassle-free way to find out what changed in
obmalloc.c between 2.7.x and 3.4.x?


On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton  wrote:
> There's a related strdup patch for readline.c, mentioned
> here:http://bugs.python.org/issue21390 and here
> https://github.com/rave-engine/python3-android/issues/2.
> There's a patch, but I'm not sure how to modify it for obmalloc.c, as
> (I think) the functions all belong to Python...they're all prefixed
> with _PyXx
>
> On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton  wrote:
>> Absolutely.  Good thing I have addr2line on device
>>
>> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0 
>> 0008bbc8
>> _PyMem_RawStrdup
>> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> /bld/python/Python-3.4.2 $
>>
>>
>>
>> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
>>> Could you try the steps at http://stackoverflow.com/a/11369475/2097780? They
>>> allow you to get a better idea of where libc is crashing.
>>>
>>> Cyd Haselton  wrote:

 Managed to get this out of logcat:
 F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
 11914 (python)  (libc)

 [ 01-29 19:30:55.855 23373:23373 F/libc ]
 Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373 (python)

 Less detail than strace but it seems to be that python is segfaulting
 libc...

 On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez  wrote:
>
>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum 
> wrote:
>>
>>
>>  What I see in the strace:
>>
>>  ... load libpython3.4m.so.1.0
>>  ... load libm
>>  ... open /dev/__properties__ and do something to it
>> (what?)
>>  ... get current time
>>  ... allocate memory
>>  ... getuid
>>  ... segfault
>>
>>  That's not a lot to go on, but it doesn't look as if it has started to
>>  load modules yet.
>>
>>  Does /dev/__properties__ ring a bell? Not to me.
>
>
>
>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>  is the code that works with that file.
>
>  This explains it a bit (slides 24-29). Looks like something to do with
>  interprocess communication. Likely has nothing to do with Python itself.
>
>  Maybe this would be useful?
>
>>
>>  That stack trace would be really helpful.
>>
>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton 
>> wrote:
>>>
>>>
>>>  Apologies...I'm not sure what a stack track is, but I do have the
>>>  strace.  Nearest I can tell, it happens due to an open call, though I
>>>  am probably wrong.
>>>  Attaching the strace output to this email.  I'm going to head back to
>>>  the documentation and to back out of some Android-related changes in
>>>  _localemodule.c
>>>
>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum 
>>>  wrote:

  There could be a million differences relevant (unicode, ints, ...).
  Perhaps
  the importlib bootstrap is failing. Perhaps the dynamic loading code
  changed. Did you get a stack track? (IIRC strace shows a syscall
 trace
  --
  also useful, but doesn't tell you precisely how
 it segfaulted.)

  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton 
  wrote:
>
>
>  All,
>  I recently ditched my attempts to port Python 2.7.8 to Android in
>  favor of Python 3.4.2.  Unfortunately, after using the same
> configure
>  options in the same environment, and modifying the setup.py as
> needed,
>  the newly built binary throws a segfault when the
> generate-posix-vars
>  portion of the build is reached...and when it is run as well (i.e.
>  ./python --help, ./python -E -S -m sysconfig, or similar)
>
>  I took a strace of ./python, however I'm a bit lost when reviewing
> it.
>  Any ideas as to what may be going on...i.e. why Python 2.7 works but
>  3.x throws a segfault?
>
>  Thanks in advance,
>  Cyd
> 
>
>  Python-Dev mailing list
>
> Python-Dev@python.org
>  https://mail.python.org/mailman/listinfo/python-dev
>  Unsubscribe:
>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org





  --
  --Guido van Rossum (python.org/~guido)
>>
>>
>>
>>
>>
>>  --
>>  --Guido van Rossum (python.org/~guido)
>>
>> 
>>
>>  Python-Dev mailing list
>>  Python-Dev@python.org
>>  https://mail.python.org/mailman/listinfo/python-dev
>>  Unsubscribe:
>>  https://mail.python.org/mailman/options/python-dev/rymg19%4

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

I don't have gdb on device; does the following tell you where Python's
strdup is called?

>> _PyMem_RawStrdup
>> /bld/python/Python-3.4.2/Objects/obmalloc.c:323

On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez  wrote:
> Is it possible at all to get a stack trace of the crash using gdb? Try the
> steps here.
>
> That way we can see where Python's own strdup function is getting called.
>
> On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton  wrote:
>>
>> Absolutely.  Good thing I have addr2line on device
>>
>> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>> 0008bbc8
>> _PyMem_RawStrdup
>> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> /bld/python/Python-3.4.2 $
>>
>>
>>
>> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
>> > Could you try the steps at http://stackoverflow.com/a/11369475/2097780?
>> > They
>> > allow you to get a better idea of where libc is crashing.
>> >
>> > Cyd Haselton  wrote:
>> >>
>> >> Managed to get this out of logcat:
>> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>> >> 11914 (python)  (libc)
>> >>
>> >> [ 01-29 19:30:55.855 23373:23373 F/libc ]
>> >> Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373 (python)
>> >>
>> >> Less detail than strace but it seems to be that python is segfaulting
>> >> libc...
>> >>
>> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
>> >> wrote:
>> >>>
>> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum 
>> >>> wrote:
>> 
>> 
>>   What I see in the strace:
>> 
>>   ... load libpython3.4m.so.1.0
>>   ... load libm
>>   ... open /dev/__properties__ and do something to it
>>  (what?)
>>   ... get current time
>>   ... allocate memory
>>   ... getuid
>>   ... segfault
>> 
>>   That's not a lot to go on, but it doesn't look as if it has started
>>  to
>>   load modules yet.
>> 
>>   Does /dev/__properties__ ring a bell? Not to me.
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >>>  is the code that works with that file.
>> >>>
>> >>>  This explains it a bit (slides 24-29). Looks like something to do
>> >>> with
>> >>>  interprocess communication. Likely has nothing to do with Python
>> >>> itself.
>> >>>
>> >>>  Maybe this would be useful?
>> >>>
>> 
>>   That stack trace would be really helpful.
>> 
>>   On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton 
>>  wrote:
>> >
>> >
>> >  Apologies...I'm not sure what a stack track is, but I do have the
>> >  strace.  Nearest I can tell, it happens due to an open call, though
>> > I
>> >  am probably wrong.
>> >  Attaching the strace output to this email.  I'm going to head back
>> > to
>> >  the documentation and to back out of some Android-related changes
>> > in
>> >  _localemodule.c
>> >
>> >  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> > 
>> >  wrote:
>> >>
>> >>  There could be a million differences relevant (unicode, ints,
>> >> ...).
>> >>  Perhaps
>> >>  the importlib bootstrap is failing. Perhaps the dynamic loading
>> >> code
>> >>  changed. Did you get a stack track? (IIRC strace shows a syscall
>> >> trace
>> >>  --
>> >>  also useful, but doesn't tell you precisely how
>> >> it segfaulted.)
>> >>
>> >>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> 
>> >>  wrote:
>> >>>
>> >>>
>> >>>  All,
>> >>>  I recently ditched my attempts to port Python 2.7.8 to Android in
>> >>>  favor of Python 3.4.2.  Unfortunately, after using the same
>> >>> configure
>> >>>  options in the same environment, and modifying the setup.py as
>> >>> needed,
>> >>>  the newly built binary throws a segfault when the
>> >>> generate-posix-vars
>> >>>  portion of the build is reached...and when it is run as well
>> >>> (i.e.
>> >>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>> >>>
>> >>>  I took a strace of ./python, however I'm a bit lost when
>> >>> reviewing
>> >>> it.
>> >>>  Any ideas as to what may be going on...i.e. why Python 2.7 works
>> >>> but
>> >>>  3.x throws a segfault?
>> >>>
>> >>>  Thanks in advance,
>> >>>  Cyd
>> >>> 
>> >>>
>> >>>  Python-Dev mailing list
>> >>>
>> >>> Python-Dev@python.org
>> >>>  https://mail.python.org/mailman/listinfo/python-dev
>> >>>  Unsubscribe:
>> >>>
>> >>>
>> >>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>  --
>> >>  --Guido van Rossum (python.org/~guido)
>> 
>> 
>> 
>> 
>> 
>>   --
>>   --Guido van Rossum (python.org/~guido)
>> 
>>  
>> 
>>   Python-Dev mailing list
>

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Ryan Gonzalez]

I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes when
calling strlen. It's that whatever is calling it is likely asking it to
duplicate a null pointer. Basically, it's probably the caller's fault.

You could always try modifying _PyMem_RawStrdup to return NULL when given a
null pointer and see where it then segfaults.

On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton  wrote:

> Alternatively, is there a hassle-free way to find out what changed in
> obmalloc.c between 2.7.x and 3.4.x?
>
>
> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton  wrote:
> > There's a related strdup patch for readline.c, mentioned
> > here:http://bugs.python.org/issue21390 and here
> > https://github.com/rave-engine/python3-android/issues/2.
> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
> > (I think) the functions all belong to Python...they're all prefixed
> > with _PyXx
> >
> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
> wrote:
> >> Absolutely.  Good thing I have addr2line on device
> >>
> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
> 0008bbc8
> >> _PyMem_RawStrdup
> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> /bld/python/Python-3.4.2 $
> >>
> >>
> >>
> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
> >>> Could you try the steps at http://stackoverflow.com/a/11369475/2097780?
> They
> >>> allow you to get a better idea of where libc is crashing.
> >>>
> >>> Cyd Haselton  wrote:
> 
>  Managed to get this out of logcat:
>  F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>  11914 (python)  (libc)
> 
>  [ 01-29 19:30:55.855 23373:23373 F/libc ]
>  Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
> (python)
> 
>  Less detail than strace but it seems to be that python is segfaulting
>  libc...
> 
>  On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
> wrote:
> >
> >  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <
> gu...@python.org>
> > wrote:
> >>
> >>
> >>  What I see in the strace:
> >>
> >>  ... load libpython3.4m.so.1.0
> >>  ... load libm
> >>  ... open /dev/__properties__ and do something to it
> >> (what?)
> >>  ... get current time
> >>  ... allocate memory
> >>  ... getuid
> >>  ... segfault
> >>
> >>  That's not a lot to go on, but it doesn't look as if it has
> started to
> >>  load modules yet.
> >>
> >>  Does /dev/__properties__ ring a bell? Not to me.
> >
> >
> >
> >
> >
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >  is the code that works with that file.
> >
> >  This explains it a bit (slides 24-29). Looks like something to do
> with
> >  interprocess communication. Likely has nothing to do with Python
> itself.
> >
> >  Maybe this would be useful?
> >
> >>
> >>  That stack trace would be really helpful.
> >>
> >>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton  >
> >> wrote:
> >>>
> >>>
> >>>  Apologies...I'm not sure what a stack track is, but I do have the
> >>>  strace.  Nearest I can tell, it happens due to an open call,
> though I
> >>>  am probably wrong.
> >>>  Attaching the strace output to this email.  I'm going to head
> back to
> >>>  the documentation and to back out of some Android-related changes
> in
> >>>  _localemodule.c
> >>>
> >>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <
> gu...@python.org>
> >>>  wrote:
> 
>   There could be a million differences relevant (unicode, ints,
> ...).
>   Perhaps
>   the importlib bootstrap is failing. Perhaps the dynamic loading
> code
>   changed. Did you get a stack track? (IIRC strace shows a syscall
>  trace
>   --
>   also useful, but doesn't tell you precisely how
>  it segfaulted.)
> 
>   On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <
> chasel...@gmail.com>
>   wrote:
> >
> >
> >  All,
> >  I recently ditched my attempts to port Python 2.7.8 to Android
> in
> >  favor of Python 3.4.2.  Unfortunately, after using the same
> > configure
> >  options in the same environment, and modifying the setup.py as
> > needed,
> >  the newly built binary throws a segfault when the
> > generate-posix-vars
> >  portion of the build is reached...and when it is run as well
> (i.e.
> >  ./python --help, ./python -E -S -m sysconfig, or similar)
> >
> >  I took a strace of ./python, however I'm a bit lost when
> reviewing
> > it.
> >  Any ideas as to what may be going on...i.e. why Python 2.7
> works but
> >  3.x throws a segfault?
> >
> >  Thanks in advance,
> >  Cyd
> > _

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Ryan Gonzalez]

No...

...but I think I found the issue with grep. Try applying the attached patch
to the Python/frozenmain.c. It comments out the locale handling.

It seems that Python always calls its strdup function on the locale string.
On Android, this can apparently be null (as seen in the bug report you
linked to).

On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton  wrote:

> I don't have gdb on device; does the following tell you where Python's
> strdup is called?
>
> >> _PyMem_RawStrdup
> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>
> On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez  wrote:
> > Is it possible at all to get a stack trace of the crash using gdb? Try
> the
> > steps here.
> >
> > That way we can see where Python's own strdup function is getting called.
> >
> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
> wrote:
> >>
> >> Absolutely.  Good thing I have addr2line on device
> >>
> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
> >> 0008bbc8
> >> _PyMem_RawStrdup
> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> /bld/python/Python-3.4.2 $
> >>
> >>
> >>
> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
> >> > Could you try the steps at
> http://stackoverflow.com/a/11369475/2097780?
> >> > They
> >> > allow you to get a better idea of where libc is crashing.
> >> >
> >> > Cyd Haselton  wrote:
> >> >>
> >> >> Managed to get this out of logcat:
> >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
> >> >> 11914 (python)  (libc)
> >> >>
> >> >> [ 01-29 19:30:55.855 23373:23373 F/libc ]
> >> >> Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
> (python)
> >> >>
> >> >> Less detail than strace but it seems to be that python is segfaulting
> >> >> libc...
> >> >>
> >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
> >> >> wrote:
> >> >>>
> >> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <
> gu...@python.org>
> >> >>> wrote:
> >> 
> >> 
> >>   What I see in the strace:
> >> 
> >>   ... load libpython3.4m.so.1.0
> >>   ... load libm
> >>   ... open /dev/__properties__ and do something to it
> >>  (what?)
> >>   ... get current time
> >>   ... allocate memory
> >>   ... getuid
> >>   ... segfault
> >> 
> >>   That's not a lot to go on, but it doesn't look as if it has
> started
> >>  to
> >>   load modules yet.
> >> 
> >>   Does /dev/__properties__ ring a bell? Not to me.
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >> >>>  is the code that works with that file.
> >> >>>
> >> >>>  This explains it a bit (slides 24-29). Looks like something to do
> >> >>> with
> >> >>>  interprocess communication. Likely has nothing to do with Python
> >> >>> itself.
> >> >>>
> >> >>>  Maybe this would be useful?
> >> >>>
> >> 
> >>   That stack trace would be really helpful.
> >> 
> >>   On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <
> chasel...@gmail.com>
> >>  wrote:
> >> >
> >> >
> >> >  Apologies...I'm not sure what a stack track is, but I do have the
> >> >  strace.  Nearest I can tell, it happens due to an open call,
> though
> >> > I
> >> >  am probably wrong.
> >> >  Attaching the strace output to this email.  I'm going to head
> back
> >> > to
> >> >  the documentation and to back out of some Android-related changes
> >> > in
> >> >  _localemodule.c
> >> >
> >> >  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
> >> > 
> >> >  wrote:
> >> >>
> >> >>  There could be a million differences relevant (unicode, ints,
> >> >> ...).
> >> >>  Perhaps
> >> >>  the importlib bootstrap is failing. Perhaps the dynamic loading
> >> >> code
> >> >>  changed. Did you get a stack track? (IIRC strace shows a syscall
> >> >> trace
> >> >>  --
> >> >>  also useful, but doesn't tell you precisely how
> >> >> it segfaulted.)
> >> >>
> >> >>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
> >> >> 
> >> >>  wrote:
> >> >>>
> >> >>>
> >> >>>  All,
> >> >>>  I recently ditched my attempts to port Python 2.7.8 to Android
> in
> >> >>>  favor of Python 3.4.2.  Unfortunately, after using the same
> >> >>> configure
> >> >>>  options in the same environment, and modifying the setup.py as
> >> >>> needed,
> >> >>>  the newly built binary throws a segfault when the
> >> >>> generate-posix-vars
> >> >>>  portion of the build is reached...and when it is run as well
> >> >>> (i.e.
> >> >>>  ./python --help, ./python -E -S -m sysconfig, or similar)
> >> >>>
> >> >>>  I took a strace of ./python, however I'm a bit lost when
> >> >>> reviewing
> >> >>> it.
> >> >>>  Any ideas as to what may be going on...i.e. why Python 2.7
> works
> >> >>> but
> >> >>>  3.x throws a segf

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

  Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
currently returning NULL when given a null pointer.

>From obmalloc.c

_PyMem_RawStrdup(const char *str)
{
size_t size;
char *copy;
size = strlen(str) + 1;
copy = PyMem_RawMalloc(size);
if (copy == NULL)
return NULL;
memcpy(copy, str, size);
return copy;
  }

On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez  wrote:
> I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes when
> calling strlen. It's that whatever is calling it is likely asking it to
> duplicate a null pointer. Basically, it's probably the caller's fault.
>
> You could always try modifying _PyMem_RawStrdup to return NULL when given a
> null pointer and see where it then segfaults.
>
> On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton  wrote:
>>
>> Alternatively, is there a hassle-free way to find out what changed in
>> obmalloc.c between 2.7.x and 3.4.x?
>>
>>
>> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton  wrote:
>> > There's a related strdup patch for readline.c, mentioned
>> > here:http://bugs.python.org/issue21390 and here
>> > https://github.com/rave-engine/python3-android/issues/2.
>> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
>> > (I think) the functions all belong to Python...they're all prefixed
>> > with _PyXx
>> >
>> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
>> > wrote:
>> >> Absolutely.  Good thing I have addr2line on device
>> >>
>> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>> >> 0008bbc8
>> >> _PyMem_RawStrdup
>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> /bld/python/Python-3.4.2 $
>> >>
>> >>
>> >>
>> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
>> >>> Could you try the steps at
>> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >>> allow you to get a better idea of where libc is crashing.
>> >>>
>> >>> Cyd Haselton  wrote:
>> 
>>  Managed to get this out of logcat:
>>  F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>>  11914 (python)  (libc)
>> 
>>  [ 01-29 19:30:55.855 23373:23373 F/libc ]
>>  Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
>>  (python)
>> 
>>  Less detail than strace but it seems to be that python is segfaulting
>>  libc...
>> 
>>  On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
>>  wrote:
>> >
>> >  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> > 
>> > wrote:
>> >>
>> >>
>> >>  What I see in the strace:
>> >>
>> >>  ... load libpython3.4m.so.1.0
>> >>  ... load libm
>> >>  ... open /dev/__properties__ and do something to it
>> >> (what?)
>> >>  ... get current time
>> >>  ... allocate memory
>> >>  ... getuid
>> >>  ... segfault
>> >>
>> >>  That's not a lot to go on, but it doesn't look as if it has
>> >> started to
>> >>  load modules yet.
>> >>
>> >>  Does /dev/__properties__ ring a bell? Not to me.
>> >
>> >
>> >
>> >
>> >
>> > https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >  is the code that works with that file.
>> >
>> >  This explains it a bit (slides 24-29). Looks like something to do
>> > with
>> >  interprocess communication. Likely has nothing to do with Python
>> > itself.
>> >
>> >  Maybe this would be useful?
>> >
>> >>
>> >>  That stack trace would be really helpful.
>> >>
>> >>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> 
>> >> wrote:
>> >>>
>> >>>
>> >>>  Apologies...I'm not sure what a stack track is, but I do have the
>> >>>  strace.  Nearest I can tell, it happens due to an open call,
>> >>> though I
>> >>>  am probably wrong.
>> >>>  Attaching the strace output to this email.  I'm going to head
>> >>> back to
>> >>>  the documentation and to back out of some Android-related changes
>> >>> in
>> >>>  _localemodule.c
>> >>>
>> >>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >>> 
>> >>>  wrote:
>> 
>>   There could be a million differences relevant (unicode, ints,
>>  ...).
>>   Perhaps
>>   the importlib bootstrap is failing. Perhaps the dynamic loading
>>  code
>>   changed. Did you get a stack track? (IIRC strace shows a syscall
>>  trace
>>   --
>>   also useful, but doesn't tell you precisely how
>>  it segfaulted.)
>> 
>>   On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>>  
>>   wrote:
>> >
>> >
>> >  All,
>> >  I recently ditched my attempts to port Python 2.7.8 to Android
>> > in
>> >  favor of Python 3.4.2.  Unfortunately, after using the same
>> > configure
>> >  options in the same environment, and mo

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Ryan Gonzalez]

No, it returns NULL if malloc gives it a raw pointer. It unconditionally
checks the length of the (possibly null) string argument first.

Please try the patch I attached in the last email. It *might* fix the
issue. Android has crappy locale handling.

On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton  wrote:

>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
> currently returning NULL when given a null pointer.
>
> From obmalloc.c
>
> _PyMem_RawStrdup(const char *str)
> {
> size_t size;
> char *copy;
> size = strlen(str) + 1;
> copy = PyMem_RawMalloc(size);
> if (copy == NULL)
> return NULL;
> memcpy(copy, str, size);
> return copy;
>   }
>
> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez  wrote:
> > I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes
> when
> > calling strlen. It's that whatever is calling it is likely asking it to
> > duplicate a null pointer. Basically, it's probably the caller's fault.
> >
> > You could always try modifying _PyMem_RawStrdup to return NULL when
> given a
> > null pointer and see where it then segfaults.
> >
> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton 
> wrote:
> >>
> >> Alternatively, is there a hassle-free way to find out what changed in
> >> obmalloc.c between 2.7.x and 3.4.x?
> >>
> >>
> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton 
> wrote:
> >> > There's a related strdup patch for readline.c, mentioned
> >> > here:http://bugs.python.org/issue21390 and here
> >> > https://github.com/rave-engine/python3-android/issues/2.
> >> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
> >> > (I think) the functions all belong to Python...they're all prefixed
> >> > with _PyXx
> >> >
> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
> >> > wrote:
> >> >> Absolutely.  Good thing I have addr2line on device
> >> >>
> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
> /lib/libpython3.4m.so.1.0
> >> >> 0008bbc8
> >> >> _PyMem_RawStrdup
> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> >> /bld/python/Python-3.4.2 $
> >> >>
> >> >>
> >> >>
> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
> >> >>> Could you try the steps at
> >> >>> http://stackoverflow.com/a/11369475/2097780? They
> >> >>> allow you to get a better idea of where libc is crashing.
> >> >>>
> >> >>> Cyd Haselton  wrote:
> >> 
> >>  Managed to get this out of logcat:
> >>  F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
> >>  11914 (python)  (libc)
> >> 
> >>  [ 01-29 19:30:55.855 23373:23373 F/libc ]
> >>  Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
> >>  (python)
> >> 
> >>  Less detail than strace but it seems to be that python is
> segfaulting
> >>  libc...
> >> 
> >>  On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
> >>  wrote:
> >> >
> >> >  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
> >> > 
> >> > wrote:
> >> >>
> >> >>
> >> >>  What I see in the strace:
> >> >>
> >> >>  ... load libpython3.4m.so.1.0
> >> >>  ... load libm
> >> >>  ... open /dev/__properties__ and do something to it
> >> >> (what?)
> >> >>  ... get current time
> >> >>  ... allocate memory
> >> >>  ... getuid
> >> >>  ... segfault
> >> >>
> >> >>  That's not a lot to go on, but it doesn't look as if it has
> >> >> started to
> >> >>  load modules yet.
> >> >>
> >> >>  Does /dev/__properties__ ring a bell? Not to me.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >> >  is the code that works with that file.
> >> >
> >> >  This explains it a bit (slides 24-29). Looks like something to do
> >> > with
> >> >  interprocess communication. Likely has nothing to do with Python
> >> > itself.
> >> >
> >> >  Maybe this would be useful?
> >> >
> >> >>
> >> >>  That stack trace would be really helpful.
> >> >>
> >> >>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
> >> >> 
> >> >> wrote:
> >> >>>
> >> >>>
> >> >>>  Apologies...I'm not sure what a stack track is, but I do have
> the
> >> >>>  strace.  Nearest I can tell, it happens due to an open call,
> >> >>> though I
> >> >>>  am probably wrong.
> >> >>>  Attaching the strace output to this email.  I'm going to head
> >> >>> back to
> >> >>>  the documentation and to back out of some Android-related
> changes
> >> >>> in
> >> >>>  _localemodule.c
> >> >>>
> >> >>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
> >> >>> 
> >> >>>  wrote:
> >> 
> >>   There could be a million differences relevant (unicode, ints,
> >>  ...).
> >>   Perhaps
> >>   the importlib bootstrap is failing. Perhaps the dynamic
> loading
> >>  code

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Victor Stinner]

Android provides a minimal support of locales. Most functions return a
fake result, do nothing. I'm not sure that it supports any codec. To
support Android, we may force UTF-8 for the filesystem encoding, as
done on Mac OS X.

Victor

2015-01-30 19:04 GMT+01:00 Ryan Gonzalez :
> No...
>
> ...but I think I found the issue with grep. Try applying the attached patch
> to the Python/frozenmain.c. It comments out the locale handling.
>
> It seems that Python always calls its strdup function on the locale string.
> On Android, this can apparently be null (as seen in the bug report you
> linked to).
>
> On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton  wrote:
>>
>> I don't have gdb on device; does the following tell you where Python's
>> strdup is called?
>>
>> >> _PyMem_RawStrdup
>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>
>> On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez  wrote:
>> > Is it possible at all to get a stack trace of the crash using gdb? Try
>> > the
>> > steps here.
>> >
>> > That way we can see where Python's own strdup function is getting
>> > called.
>> >
>> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
>> > wrote:
>> >>
>> >> Absolutely.  Good thing I have addr2line on device
>> >>
>> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>> >> 0008bbc8
>> >> _PyMem_RawStrdup
>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> /bld/python/Python-3.4.2 $
>> >>
>> >>
>> >>
>> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
>> >> > Could you try the steps at
>> >> > http://stackoverflow.com/a/11369475/2097780?
>> >> > They
>> >> > allow you to get a better idea of where libc is crashing.
>> >> >
>> >> > Cyd Haselton  wrote:
>> >> >>
>> >> >> Managed to get this out of logcat:
>> >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>> >> >> 11914 (python)  (libc)
>> >> >>
>> >> >> [ 01-29 19:30:55.855 23373:23373 F/libc ]
>> >> >> Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
>> >> >> (python)
>> >> >>
>> >> >> Less detail than strace but it seems to be that python is
>> >> >> segfaulting
>> >> >> libc...
>> >> >>
>> >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
>> >> >> wrote:
>> >> >>>
>> >> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >>> 
>> >> >>> wrote:
>> >> 
>> >> 
>> >>   What I see in the strace:
>> >> 
>> >>   ... load libpython3.4m.so.1.0
>> >>   ... load libm
>> >>   ... open /dev/__properties__ and do something to it
>> >>  (what?)
>> >>   ... get current time
>> >>   ... allocate memory
>> >>   ... getuid
>> >>   ... segfault
>> >> 
>> >>   That's not a lot to go on, but it doesn't look as if it has
>> >>  started
>> >>  to
>> >>   load modules yet.
>> >> 
>> >>   Does /dev/__properties__ ring a bell? Not to me.
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >>>  is the code that works with that file.
>> >> >>>
>> >> >>>  This explains it a bit (slides 24-29). Looks like something to do
>> >> >>> with
>> >> >>>  interprocess communication. Likely has nothing to do with Python
>> >> >>> itself.
>> >> >>>
>> >> >>>  Maybe this would be useful?
>> >> >>>
>> >> 
>> >>   That stack trace would be really helpful.
>> >> 
>> >>   On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >>  
>> >>  wrote:
>> >> >
>> >> >
>> >> >  Apologies...I'm not sure what a stack track is, but I do have
>> >> > the
>> >> >  strace.  Nearest I can tell, it happens due to an open call,
>> >> > though
>> >> > I
>> >> >  am probably wrong.
>> >> >  Attaching the strace output to this email.  I'm going to head
>> >> > back
>> >> > to
>> >> >  the documentation and to back out of some Android-related
>> >> > changes
>> >> > in
>> >> >  _localemodule.c
>> >> >
>> >> >  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> > 
>> >> >  wrote:
>> >> >>
>> >> >>  There could be a million differences relevant (unicode, ints,
>> >> >> ...).
>> >> >>  Perhaps
>> >> >>  the importlib bootstrap is failing. Perhaps the dynamic loading
>> >> >> code
>> >> >>  changed. Did you get a stack track? (IIRC strace shows a
>> >> >> syscall
>> >> >> trace
>> >> >>  --
>> >> >>  also useful, but doesn't tell you precisely how
>> >> >> it segfaulted.)
>> >> >>
>> >> >>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >> 
>> >> >>  wrote:
>> >> >>>
>> >> >>>
>> >> >>>  All,
>> >> >>>  I recently ditched my attempts to port Python 2.7.8 to Android
>> >> >>> in
>> >> >>>  favor of Python 3.4.2.  Unfortunately, after using the same
>> >> >>> configure
>> >> >>>  options in the same environment, and modifying the setup.py as
>> >> >>> needed,
>>

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Victor Stinner]

Oh, I found my old patch to force UTF-8 on Android. I didn't test it:
see attached file.

It would be nice to start a wiki page to collect all informations on
the Python port to Android.

Victor

2015-01-30 21:04 GMT+01:00 Victor Stinner :
> Android provides a minimal support of locales. Most functions return a
> fake result, do nothing. I'm not sure that it supports any codec. To
> support Android, we may force UTF-8 for the filesystem encoding, as
> done on Mac OS X.
>
> Victor
>
> 2015-01-30 19:04 GMT+01:00 Ryan Gonzalez :
>> No...
>>
>> ...but I think I found the issue with grep. Try applying the attached patch
>> to the Python/frozenmain.c. It comments out the locale handling.
>>
>> It seems that Python always calls its strdup function on the locale string.
>> On Android, this can apparently be null (as seen in the bug report you
>> linked to).
>>
>> On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton  wrote:
>>>
>>> I don't have gdb on device; does the following tell you where Python's
>>> strdup is called?
>>>
>>> >> _PyMem_RawStrdup
>>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>
>>> On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez  wrote:
>>> > Is it possible at all to get a stack trace of the crash using gdb? Try
>>> > the
>>> > steps here.
>>> >
>>> > That way we can see where Python's own strdup function is getting
>>> > called.
>>> >
>>> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
>>> > wrote:
>>> >>
>>> >> Absolutely.  Good thing I have addr2line on device
>>> >>
>>> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>>> >> 0008bbc8
>>> >> _PyMem_RawStrdup
>>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>> >> /bld/python/Python-3.4.2 $
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
>>> >> > Could you try the steps at
>>> >> > http://stackoverflow.com/a/11369475/2097780?
>>> >> > They
>>> >> > allow you to get a better idea of where libc is crashing.
>>> >> >
>>> >> > Cyd Haselton  wrote:
>>> >> >>
>>> >> >> Managed to get this out of logcat:
>>> >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>>> >> >> 11914 (python)  (libc)
>>> >> >>
>>> >> >> [ 01-29 19:30:55.855 23373:23373 F/libc ]
>>> >> >> Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
>>> >> >> (python)
>>> >> >>
>>> >> >> Less detail than strace but it seems to be that python is
>>> >> >> segfaulting
>>> >> >> libc...
>>> >> >>
>>> >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
>>> >> >> wrote:
>>> >> >>>
>>> >> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>>> >> >>> 
>>> >> >>> wrote:
>>> >> 
>>> >> 
>>> >>   What I see in the strace:
>>> >> 
>>> >>   ... load libpython3.4m.so.1.0
>>> >>   ... load libm
>>> >>   ... open /dev/__properties__ and do something to it
>>> >>  (what?)
>>> >>   ... get current time
>>> >>   ... allocate memory
>>> >>   ... getuid
>>> >>   ... segfault
>>> >> 
>>> >>   That's not a lot to go on, but it doesn't look as if it has
>>> >>  started
>>> >>  to
>>> >>   load modules yet.
>>> >> 
>>> >>   Does /dev/__properties__ ring a bell? Not to me.
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>> >> >>>  is the code that works with that file.
>>> >> >>>
>>> >> >>>  This explains it a bit (slides 24-29). Looks like something to do
>>> >> >>> with
>>> >> >>>  interprocess communication. Likely has nothing to do with Python
>>> >> >>> itself.
>>> >> >>>
>>> >> >>>  Maybe this would be useful?
>>> >> >>>
>>> >> 
>>> >>   That stack trace would be really helpful.
>>> >> 
>>> >>   On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>>> >>  
>>> >>  wrote:
>>> >> >
>>> >> >
>>> >> >  Apologies...I'm not sure what a stack track is, but I do have
>>> >> > the
>>> >> >  strace.  Nearest I can tell, it happens due to an open call,
>>> >> > though
>>> >> > I
>>> >> >  am probably wrong.
>>> >> >  Attaching the strace output to this email.  I'm going to head
>>> >> > back
>>> >> > to
>>> >> >  the documentation and to back out of some Android-related
>>> >> > changes
>>> >> > in
>>> >> >  _localemodule.c
>>> >> >
>>> >> >  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>>> >> > 
>>> >> >  wrote:
>>> >> >>
>>> >> >>  There could be a million differences relevant (unicode, ints,
>>> >> >> ...).
>>> >> >>  Perhaps
>>> >> >>  the importlib bootstrap is failing. Perhaps the dynamic loading
>>> >> >> code
>>> >> >>  changed. Did you get a stack track? (IIRC strace shows a
>>> >> >> syscall
>>> >> >> trace
>>> >> >>  --
>>> >> >>  also useful, but doesn't tell you precisely how
>>> >> >> it segfaulted.)
>>> >> >>
>>> >> >>  On Wed, Jan 28, 2015 at 6:43 AM, 

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

Ok...that makes sense.. apologies I do not do a lot of debugging. My
goal was to get Python (then spidermonkey) on my device then start
learning languages where i'd need to learn debugging.

Tried patch, see my reply, agree about Android's locale...at least
where native codeis concerned

On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez  wrote:
> No, it returns NULL if malloc gives it a raw pointer. It unconditionally
> checks the length of the (possibly null) string argument first.
>
> Please try the patch I attached in the last email. It *might* fix the issue.
> Android has crappy locale handling.
>
> On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton  wrote:
>>
>>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
>> currently returning NULL when given a null pointer.
>>
>> From obmalloc.c
>>
>> _PyMem_RawStrdup(const char *str)
>> {
>> size_t size;
>> char *copy;
>> size = strlen(str) + 1;
>> copy = PyMem_RawMalloc(size);
>> if (copy == NULL)
>> return NULL;
>> memcpy(copy, str, size);
>> return copy;
>>   }
>>
>> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez  wrote:
>> > I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes
>> > when
>> > calling strlen. It's that whatever is calling it is likely asking it to
>> > duplicate a null pointer. Basically, it's probably the caller's fault.
>> >
>> > You could always try modifying _PyMem_RawStrdup to return NULL when
>> > given a
>> > null pointer and see where it then segfaults.
>> >
>> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton 
>> > wrote:
>> >>
>> >> Alternatively, is there a hassle-free way to find out what changed in
>> >> obmalloc.c between 2.7.x and 3.4.x?
>> >>
>> >>
>> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton 
>> >> wrote:
>> >> > There's a related strdup patch for readline.c, mentioned
>> >> > here:http://bugs.python.org/issue21390 and here
>> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
>> >> > (I think) the functions all belong to Python...they're all prefixed
>> >> > with _PyXx
>> >> >
>> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
>> >> > wrote:
>> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >>
>> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> /lib/libpython3.4m.so.1.0
>> >> >> 0008bbc8
>> >> >> _PyMem_RawStrdup
>> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> /bld/python/Python-3.4.2 $
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
>> >> >>> Could you try the steps at
>> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >>> allow you to get a better idea of where libc is crashing.
>> >> >>>
>> >> >>> Cyd Haselton  wrote:
>> >> 
>> >>  Managed to get this out of logcat:
>> >>  F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>> >>  11914 (python)  (libc)
>> >> 
>> >>  [ 01-29 19:30:55.855 23373:23373 F/libc ]
>> >>  Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
>> >>  (python)
>> >> 
>> >>  Less detail than strace but it seems to be that python is
>> >>  segfaulting
>> >>  libc...
>> >> 
>> >>  On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
>> >>  wrote:
>> >> >
>> >> >  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> > 
>> >> > wrote:
>> >> >>
>> >> >>
>> >> >>  What I see in the strace:
>> >> >>
>> >> >>  ... load libpython3.4m.so.1.0
>> >> >>  ... load libm
>> >> >>  ... open /dev/__properties__ and do something to it
>> >> >> (what?)
>> >> >>  ... get current time
>> >> >>  ... allocate memory
>> >> >>  ... getuid
>> >> >>  ... segfault
>> >> >>
>> >> >>  That's not a lot to go on, but it doesn't look as if it has
>> >> >> started to
>> >> >>  load modules yet.
>> >> >>
>> >> >>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >  is the code that works with that file.
>> >> >
>> >> >  This explains it a bit (slides 24-29). Looks like something to
>> >> > do
>> >> > with
>> >> >  interprocess communication. Likely has nothing to do with Python
>> >> > itself.
>> >> >
>> >> >  Maybe this would be useful?
>> >> >
>> >> >>
>> >> >>  That stack trace would be really helpful.
>> >> >>
>> >> >>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >> 
>> >> >> wrote:
>> >> >>>
>> >> >>>
>> >> >>>  Apologies...I'm not sure what a stack track is, but I do have
>> >> >>> the
>> >> >>>  strace.  Nearest I can tell, it happens due to an open call,
>> >> >>> though I
>> >> >>>  am probably wrong.
>> >> >>>  Attaching the

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

Unfortunately it is still reporting the same function :-/.

On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez  wrote:
> Yes...
>
> Can you check if it's crashing in a different function now?
>
> On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton  wrote:
>>
>> Yes I did.  I did have to enter all the information in manually...I'm
>> not familiar with automated patch application tools and even if I
>> were, I'm pretty sure I wouldn't have them on my device.
>>
>> Just so that I'm absolutely sure I got everything correct...you wanted
>> all of the lines in the patch commented out, correct?  Basically
>> everything referencing oldloc?
>>
>> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez  wrote:
>> > Are you sure the patch was applied correctly? I was SO sure it would
>> > work!
>> >
>> > FYI, you tried the patch I attached to the email message, right?
>> >
>> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton 
>> > wrote:
>> >>
>> >> Update:  I did try the patch after getting it installed correctly, but
>> >> I'm still getting a segfault on the newly built binary.
>> >> Will post info this afternoon.
>> >>
>> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez 
>> >> wrote:
>> >> > No, it returns NULL if malloc gives it a raw pointer. It
>> >> > unconditionally
>> >> > checks the length of the (possibly null) string argument first.
>> >> >
>> >> > Please try the patch I attached in the last email. It *might* fix the
>> >> > issue.
>> >> > Android has crappy locale handling.
>> >> >
>> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton 
>> >> > wrote:
>> >> >>
>> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
>> >> >> currently returning NULL when given a null pointer.
>> >> >>
>> >> >> From obmalloc.c
>> >> >>
>> >> >> _PyMem_RawStrdup(const char *str)
>> >> >> {
>> >> >> size_t size;
>> >> >> char *copy;
>> >> >> size = strlen(str) + 1;
>> >> >> copy = PyMem_RawMalloc(size);
>> >> >> if (copy == NULL)
>> >> >> return NULL;
>> >> >> memcpy(copy, str, size);
>> >> >> return copy;
>> >> >>   }
>> >> >>
>> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez 
>> >> >> wrote:
>> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
>> >> >> > crashes
>> >> >> > when
>> >> >> > calling strlen. It's that whatever is calling it is likely asking
>> >> >> > it
>> >> >> > to
>> >> >> > duplicate a null pointer. Basically, it's probably the caller's
>> >> >> > fault.
>> >> >> >
>> >> >> > You could always try modifying _PyMem_RawStrdup to return NULL
>> >> >> > when
>> >> >> > given a
>> >> >> > null pointer and see where it then segfaults.
>> >> >> >
>> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>> >> >> > 
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Alternatively, is there a hassle-free way to find out what
>> >> >> >> changed
>> >> >> >> in
>> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>> >> >> >>
>> >> >> >>
>> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>> >> >> >> 
>> >> >> >> wrote:
>> >> >> >> > There's a related strdup patch for readline.c, mentioned
>> >> >> >> > here:http://bugs.python.org/issue21390 and here
>> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> >> >> > There's a patch, but I'm not sure how to modify it for
>> >> >> >> > obmalloc.c,
>> >> >> >> > as
>> >> >> >> > (I think) the functions all belong to Python...they're all
>> >> >> >> > prefixed
>> >> >> >> > with _PyXx
>> >> >> >> >
>> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>> >> >> >> > 
>> >> >> >> > wrote:
>> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >> >> >>
>> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> >> >> /lib/libpython3.4m.so.1.0
>> >> >> >> >> 0008bbc8
>> >> >> >> >> _PyMem_RawStrdup
>> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> >> >> /bld/python/Python-3.4.2 $
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan 
>> >> >> >> >> wrote:
>> >> >> >> >>> Could you try the steps at
>> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >> >> >>> allow you to get a better idea of where libc is crashing.
>> >> >> >> >>>
>> >> >> >> >>> Cyd Haselton  wrote:
>> >> >> >> 
>> >> >> >>  Managed to get this out of logcat:
>> >> >> >>  F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1),
>> >> >> >>  thread
>> >> >> >>  11914 (python)  (libc)
>> >> >> >> 
>> >> >> >>  [ 01-29 19:30:55.855 23373:23373 F/libc ]
>> >> >> >>  Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>> >> >> >>  23373
>> >> >> >>  (python)
>> >> >> >> 
>> >> >> >>  Less detail than strace but it seems to be that python is
>> >> >> >>  segfaulting
>> >> >> >>  libc...
>> >> >> >> 
>> >> >> >>  On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>> >> >> >>  
>> >> >> >>  wrote:
>> >> >> >> >
>> >> >> >> >  On Wed, Jan 28, 201

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

Update:  I did try the patch after getting it installed correctly, but
I'm still getting a segfault on the newly built binary.
Will post info this afternoon.

On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez  wrote:
> No, it returns NULL if malloc gives it a raw pointer. It unconditionally
> checks the length of the (possibly null) string argument first.
>
> Please try the patch I attached in the last email. It *might* fix the issue.
> Android has crappy locale handling.
>
> On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton  wrote:
>>
>>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
>> currently returning NULL when given a null pointer.
>>
>> From obmalloc.c
>>
>> _PyMem_RawStrdup(const char *str)
>> {
>> size_t size;
>> char *copy;
>> size = strlen(str) + 1;
>> copy = PyMem_RawMalloc(size);
>> if (copy == NULL)
>> return NULL;
>> memcpy(copy, str, size);
>> return copy;
>>   }
>>
>> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez  wrote:
>> > I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes
>> > when
>> > calling strlen. It's that whatever is calling it is likely asking it to
>> > duplicate a null pointer. Basically, it's probably the caller's fault.
>> >
>> > You could always try modifying _PyMem_RawStrdup to return NULL when
>> > given a
>> > null pointer and see where it then segfaults.
>> >
>> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton 
>> > wrote:
>> >>
>> >> Alternatively, is there a hassle-free way to find out what changed in
>> >> obmalloc.c between 2.7.x and 3.4.x?
>> >>
>> >>
>> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton 
>> >> wrote:
>> >> > There's a related strdup patch for readline.c, mentioned
>> >> > here:http://bugs.python.org/issue21390 and here
>> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
>> >> > (I think) the functions all belong to Python...they're all prefixed
>> >> > with _PyXx
>> >> >
>> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
>> >> > wrote:
>> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >>
>> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> /lib/libpython3.4m.so.1.0
>> >> >> 0008bbc8
>> >> >> _PyMem_RawStrdup
>> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> /bld/python/Python-3.4.2 $
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
>> >> >>> Could you try the steps at
>> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >>> allow you to get a better idea of where libc is crashing.
>> >> >>>
>> >> >>> Cyd Haselton  wrote:
>> >> 
>> >>  Managed to get this out of logcat:
>> >>  F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
>> >>  11914 (python)  (libc)
>> >> 
>> >>  [ 01-29 19:30:55.855 23373:23373 F/libc ]
>> >>  Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
>> >>  (python)
>> >> 
>> >>  Less detail than strace but it seems to be that python is
>> >>  segfaulting
>> >>  libc...
>> >> 
>> >>  On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
>> >>  wrote:
>> >> >
>> >> >  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> > 
>> >> > wrote:
>> >> >>
>> >> >>
>> >> >>  What I see in the strace:
>> >> >>
>> >> >>  ... load libpython3.4m.so.1.0
>> >> >>  ... load libm
>> >> >>  ... open /dev/__properties__ and do something to it
>> >> >> (what?)
>> >> >>  ... get current time
>> >> >>  ... allocate memory
>> >> >>  ... getuid
>> >> >>  ... segfault
>> >> >>
>> >> >>  That's not a lot to go on, but it doesn't look as if it has
>> >> >> started to
>> >> >>  load modules yet.
>> >> >>
>> >> >>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >  is the code that works with that file.
>> >> >
>> >> >  This explains it a bit (slides 24-29). Looks like something to
>> >> > do
>> >> > with
>> >> >  interprocess communication. Likely has nothing to do with Python
>> >> > itself.
>> >> >
>> >> >  Maybe this would be useful?
>> >> >
>> >> >>
>> >> >>  That stack trace would be really helpful.
>> >> >>
>> >> >>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >> 
>> >> >> wrote:
>> >> >>>
>> >> >>>
>> >> >>>  Apologies...I'm not sure what a stack track is, but I do have
>> >> >>> the
>> >> >>>  strace.  Nearest I can tell, it happens due to an open call,
>> >> >>> though I
>> >> >>>  am probably wrong.
>> >> >>>  Attaching the strace output to this email.  I'm going to head
>> >> >>> back to
>> >> >>>  the documentation and to back out of some An

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

I've got a page of all of the modifications to source files I've used
in my attempts to get 3.4.2 working...I can post it somewhere if the
port ever succeeds.

On Fri, Jan 30, 2015 at 2:07 PM, Victor Stinner
 wrote:
> Oh, I found my old patch to force UTF-8 on Android. I didn't test it:
> see attached file.
>
> It would be nice to start a wiki page to collect all informations on
> the Python port to Android.
>
> Victor
>
> 2015-01-30 21:04 GMT+01:00 Victor Stinner :
>> Android provides a minimal support of locales. Most functions return a
>> fake result, do nothing. I'm not sure that it supports any codec. To
>> support Android, we may force UTF-8 for the filesystem encoding, as
>> done on Mac OS X.
>>
>> Victor
>>
>> 2015-01-30 19:04 GMT+01:00 Ryan Gonzalez :
>>> No...
>>>
>>> ...but I think I found the issue with grep. Try applying the attached patch
>>> to the Python/frozenmain.c. It comments out the locale handling.
>>>
>>> It seems that Python always calls its strdup function on the locale string.
>>> On Android, this can apparently be null (as seen in the bug report you
>>> linked to).
>>>
>>> On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton  wrote:

 I don't have gdb on device; does the following tell you where Python's
 strdup is called?

 >> _PyMem_RawStrdup
 >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323

 On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez  wrote:
 > Is it possible at all to get a stack trace of the crash using gdb? Try
 > the
 > steps here.
 >
 > That way we can see where Python's own strdup function is getting
 > called.
 >
 > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton 
 > wrote:
 >>
 >> Absolutely.  Good thing I have addr2line on device
 >>
 >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
 >> 0008bbc8
 >> _PyMem_RawStrdup
 >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
 >> /bld/python/Python-3.4.2 $
 >>
 >>
 >>
 >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
 >> > Could you try the steps at
 >> > http://stackoverflow.com/a/11369475/2097780?
 >> > They
 >> > allow you to get a better idea of where libc is crashing.
 >> >
 >> > Cyd Haselton  wrote:
 >> >>
 >> >> Managed to get this out of logcat:
 >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1), thread
 >> >> 11914 (python)  (libc)
 >> >>
 >> >> [ 01-29 19:30:55.855 23373:23373 F/libc ]
 >> >> Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
 >> >> (python)
 >> >>
 >> >> Less detail than strace but it seems to be that python is
 >> >> segfaulting
 >> >> libc...
 >> >>
 >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez 
 >> >> wrote:
 >> >>>
 >> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
 >> >>> 
 >> >>> wrote:
 >> 
 >> 
 >>   What I see in the strace:
 >> 
 >>   ... load libpython3.4m.so.1.0
 >>   ... load libm
 >>   ... open /dev/__properties__ and do something to it
 >>  (what?)
 >>   ... get current time
 >>   ... allocate memory
 >>   ... getuid
 >>   ... segfault
 >> 
 >>   That's not a lot to go on, but it doesn't look as if it has
 >>  started
 >>  to
 >>   load modules yet.
 >> 
 >>   Does /dev/__properties__ ring a bell? Not to me.
 >> >>>
 >> >>>
 >> >>>
 >> >>>
 >> >>>
 >> >>>
 >> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
 >> >>>  is the code that works with that file.
 >> >>>
 >> >>>  This explains it a bit (slides 24-29). Looks like something to do
 >> >>> with
 >> >>>  interprocess communication. Likely has nothing to do with Python
 >> >>> itself.
 >> >>>
 >> >>>  Maybe this would be useful?
 >> >>>
 >> 
 >>   That stack trace would be really helpful.
 >> 
 >>   On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
 >>  
 >>  wrote:
 >> >
 >> >
 >> >  Apologies...I'm not sure what a stack track is, but I do have
 >> > the
 >> >  strace.  Nearest I can tell, it happens due to an open call,
 >> > though
 >> > I
 >> >  am probably wrong.
 >> >  Attaching the strace output to this email.  I'm going to head
 >> > back
 >> > to
 >> >  the documentation and to back out of some Android-related
 >> > changes
 >> > in
 >> >  _localemodule.c
 >> >
 >> >  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
 >> > 
 >> >  wrote:
 >> >>
 >> >>  There could be a million differences relevant (unicode, ints,
 >> >> ...).
 >> >>  Perhaps
 >> >>  th

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Ryan Gonzalez]

Are you sure the patch was applied correctly? I was SO sure it would work!

FYI, you tried the patch I attached to the email message, right?

On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton  wrote:

> Update:  I did try the patch after getting it installed correctly, but
> I'm still getting a segfault on the newly built binary.
> Will post info this afternoon.
>
> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez  wrote:
> > No, it returns NULL if malloc gives it a raw pointer. It unconditionally
> > checks the length of the (possibly null) string argument first.
> >
> > Please try the patch I attached in the last email. It *might* fix the
> issue.
> > Android has crappy locale handling.
> >
> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton 
> wrote:
> >>
> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
> >> currently returning NULL when given a null pointer.
> >>
> >> From obmalloc.c
> >>
> >> _PyMem_RawStrdup(const char *str)
> >> {
> >> size_t size;
> >> char *copy;
> >> size = strlen(str) + 1;
> >> copy = PyMem_RawMalloc(size);
> >> if (copy == NULL)
> >> return NULL;
> >> memcpy(copy, str, size);
> >> return copy;
> >>   }
> >>
> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez 
> wrote:
> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes
> >> > when
> >> > calling strlen. It's that whatever is calling it is likely asking it
> to
> >> > duplicate a null pointer. Basically, it's probably the caller's fault.
> >> >
> >> > You could always try modifying _PyMem_RawStrdup to return NULL when
> >> > given a
> >> > null pointer and see where it then segfaults.
> >> >
> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton 
> >> > wrote:
> >> >>
> >> >> Alternatively, is there a hassle-free way to find out what changed in
> >> >> obmalloc.c between 2.7.x and 3.4.x?
> >> >>
> >> >>
> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton 
> >> >> wrote:
> >> >> > There's a related strdup patch for readline.c, mentioned
> >> >> > here:http://bugs.python.org/issue21390 and here
> >> >> > https://github.com/rave-engine/python3-android/issues/2.
> >> >> > There's a patch, but I'm not sure how to modify it for obmalloc.c,
> as
> >> >> > (I think) the functions all belong to Python...they're all prefixed
> >> >> > with _PyXx
> >> >> >
> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton  >
> >> >> > wrote:
> >> >> >> Absolutely.  Good thing I have addr2line on device
> >> >> >>
> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
> >> >> >> /lib/libpython3.4m.so.1.0
> >> >> >> 0008bbc8
> >> >> >> _PyMem_RawStrdup
> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> >> >> /bld/python/Python-3.4.2 $
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan  wrote:
> >> >> >>> Could you try the steps at
> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
> >> >> >>> allow you to get a better idea of where libc is crashing.
> >> >> >>>
> >> >> >>> Cyd Haselton  wrote:
> >> >> 
> >> >>  Managed to get this out of logcat:
> >> >>  F(11914) Fatal signal 11 (SIGSEGV) at 0x (code=1),
> thread
> >> >>  11914 (python)  (libc)
> >> >> 
> >> >>  [ 01-29 19:30:55.855 23373:23373 F/libc ]
> >> >>  Fatal signal 11 (SIGSEGV) at 0x (code=1), thread 23373
> >> >>  (python)
> >> >> 
> >> >>  Less detail than strace but it seems to be that python is
> >> >>  segfaulting
> >> >>  libc...
> >> >> 
> >> >>  On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <
> rym...@gmail.com>
> >> >>  wrote:
> >> >> >
> >> >> >  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
> >> >> > 
> >> >> > wrote:
> >> >> >>
> >> >> >>
> >> >> >>  What I see in the strace:
> >> >> >>
> >> >> >>  ... load libpython3.4m.so.1.0
> >> >> >>  ... load libm
> >> >> >>  ... open /dev/__properties__ and do something to it
> >> >> >> (what?)
> >> >> >>  ... get current time
> >> >> >>  ... allocate memory
> >> >> >>  ... getuid
> >> >> >>  ... segfault
> >> >> >>
> >> >> >>  That's not a lot to go on, but it doesn't look as if it has
> >> >> >> started to
> >> >> >>  load modules yet.
> >> >> >>
> >> >> >>  Does /dev/__properties__ ring a bell? Not to me.
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >> >> >  is the code that works with that file.
> >> >> >
> >> >> >  This explains it a bit (slides 24-29). Looks like something to
> >> >> > do
> >> >> > with
> >> >> >  interprocess communication. Likely has nothing to do with
> Python
> >> >> > itself.
> >> >> >
> >> >> >  Maybe this would be useful?
> >> >> >
> >> >> >>
> >> >> >>  That stack trace would be really helpful.
> >> >> >>
> >> >> 

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Ryan Gonzalez]

Do you have the Android SDK and NDK installed? If so...

Using Google, I've created this series of steps, which may (or may not)
work:

1. Make sure you have a copy of Python on your computer and make sure that
it's built with debug symbols.

2. Run the following commands from a shell with your phone plugged in:

  adb forward tcp:5039 tcp:5039
  adb shell /system/bin/gdbserver tcp:5039 
  /arm-linux-androideabi-gdb

Now, GDB should have opened, so type the following commands:

  set solib-search-path 
  file 
  target remote :5055
  run

Now, wait for the program to crash and type:

  backtrace

You should now see a complete backtrace in your terminal. To GDB, type:

  quit

*crosses fingers*

On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton  wrote:

> Unfortunately it is still reporting the same function :-/.
>
> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez  wrote:
> > Yes...
> >
> > Can you check if it's crashing in a different function now?
> >
> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton 
> wrote:
> >>
> >> Yes I did.  I did have to enter all the information in manually...I'm
> >> not familiar with automated patch application tools and even if I
> >> were, I'm pretty sure I wouldn't have them on my device.
> >>
> >> Just so that I'm absolutely sure I got everything correct...you wanted
> >> all of the lines in the patch commented out, correct?  Basically
> >> everything referencing oldloc?
> >>
> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez 
> wrote:
> >> > Are you sure the patch was applied correctly? I was SO sure it would
> >> > work!
> >> >
> >> > FYI, you tried the patch I attached to the email message, right?
> >> >
> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton 
> >> > wrote:
> >> >>
> >> >> Update:  I did try the patch after getting it installed correctly,
> but
> >> >> I'm still getting a segfault on the newly built binary.
> >> >> Will post info this afternoon.
> >> >>
> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez 
> >> >> wrote:
> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
> >> >> > unconditionally
> >> >> > checks the length of the (possibly null) string argument first.
> >> >> >
> >> >> > Please try the patch I attached in the last email. It *might* fix
> the
> >> >> > issue.
> >> >> > Android has crappy locale handling.
> >> >> >
> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton <
> chasel...@gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
> >> >> >> currently returning NULL when given a null pointer.
> >> >> >>
> >> >> >> From obmalloc.c
> >> >> >>
> >> >> >> _PyMem_RawStrdup(const char *str)
> >> >> >> {
> >> >> >> size_t size;
> >> >> >> char *copy;
> >> >> >> size = strlen(str) + 1;
> >> >> >> copy = PyMem_RawMalloc(size);
> >> >> >> if (copy == NULL)
> >> >> >> return NULL;
> >> >> >> memcpy(copy, str, size);
> >> >> >> return copy;
> >> >> >>   }
> >> >> >>
> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez  >
> >> >> >> wrote:
> >> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
> >> >> >> > crashes
> >> >> >> > when
> >> >> >> > calling strlen. It's that whatever is calling it is likely
> asking
> >> >> >> > it
> >> >> >> > to
> >> >> >> > duplicate a null pointer. Basically, it's probably the caller's
> >> >> >> > fault.
> >> >> >> >
> >> >> >> > You could always try modifying _PyMem_RawStrdup to return NULL
> >> >> >> > when
> >> >> >> > given a
> >> >> >> > null pointer and see where it then segfaults.
> >> >> >> >
> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
> >> >> >> > 
> >> >> >> > wrote:
> >> >> >> >>
> >> >> >> >> Alternatively, is there a hassle-free way to find out what
> >> >> >> >> changed
> >> >> >> >> in
> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
> >> >> >> >> 
> >> >> >> >> wrote:
> >> >> >> >> > There's a related strdup patch for readline.c, mentioned
> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
> >> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
> >> >> >> >> > There's a patch, but I'm not sure how to modify it for
> >> >> >> >> > obmalloc.c,
> >> >> >> >> > as
> >> >> >> >> > (I think) the functions all belong to Python...they're all
> >> >> >> >> > prefixed
> >> >> >> >> > with _PyXx
> >> >> >> >> >
> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
> >> >> >> >> > 
> >> >> >> >> > wrote:
> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
> >> >> >> >> >>
> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
> >> >> >> >> >> /lib/libpython3.4m.so.1.0
> >> >> >> >> >> 0008bbc8
> >> >> >> >> >> _PyMem_RawStrdup
> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> >> >> >> >> /bld/python/Python-3.4.2 $
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> On Thu, Jan 2

Re: [Python-Dev] Can Python Be Built Without Distutils

[Gregory P. Smith]

On Sun Jan 25 2015 at 7:08:53 AM Antoine Pitrou  wrote:

> On Sun, 25 Jan 2015 05:22:48 +
> "Gregory P. Smith"  wrote:
> > Why doesn't our Makefile supply that flag with the make parallelism level
> > in the sharedmods step?
>
> If I run "make -j4" here, it works (on the default branch).
>

Thanks, glad to see that this was added! :)

This looks like a feature that was added in 3.5.  Rather than passing the
flag to setup.py from the Makefile (where I had been looking for it),
setup.py looks at the MAKEFLAGS environment variable and turns on
distutils.common.build_ext's parallel flag if "-j" is found in that. By
default it will then do a parallel build using os.cpu_count() number of
tasks.

-gps


>
> Regards
>
> Antoine.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> greg%40krypto.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Cyd Haselton]

This is going to take some time...here's why:

Downloading and installing the NDK/SDK won't be too hard...I have to
clear some space...but my primary machine is running Windows 7 and I'd
rather swallow hot coals than install Cygwin.  I've got next to no
experience with it, other than knowing that the NDK recommends against
it.

I've got a CentOS VM...but it's currently in tarball form on an
external drive for space reasons...and it only has the NDK installed.
If I am able to get it back up and running, there's still the task of
getting  adb connected to my device. from the VM.

Not saying it's impossible...just that it'll take time...and I'll
probably have to tackle it tomorrow (earliest) or Sunday (latest).  In
the meantime I'll also check to see if there's anything that can a)
run in an Android terminal and b) can take a stack trace; it would be
far, far, far easier than either option above.



On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez  wrote:
> Do you have the Android SDK and NDK installed? If so...
>
> Using Google, I've created this series of steps, which may (or may not)
> work:
>
> 1. Make sure you have a copy of Python on your computer and make sure that
> it's built with debug symbols.
>
> 2. Run the following commands from a shell with your phone plugged in:
>
>   adb forward tcp:5039 tcp:5039
>   adb shell /system/bin/gdbserver tcp:5039 
>   /arm-linux-androideabi-gdb
>
> Now, GDB should have opened, so type the following commands:
>
>   set solib-search-path 
>   file 
>   target remote :5055
>   run
>
> Now, wait for the program to crash and type:
>
>   backtrace
>
> You should now see a complete backtrace in your terminal. To GDB, type:
>
>   quit
>
> *crosses fingers*
>
> On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton  wrote:
>>
>> Unfortunately it is still reporting the same function :-/.
>>
>> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez  wrote:
>> > Yes...
>> >
>> > Can you check if it's crashing in a different function now?
>> >
>> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton 
>> > wrote:
>> >>
>> >> Yes I did.  I did have to enter all the information in manually...I'm
>> >> not familiar with automated patch application tools and even if I
>> >> were, I'm pretty sure I wouldn't have them on my device.
>> >>
>> >> Just so that I'm absolutely sure I got everything correct...you wanted
>> >> all of the lines in the patch commented out, correct?  Basically
>> >> everything referencing oldloc?
>> >>
>> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez 
>> >> wrote:
>> >> > Are you sure the patch was applied correctly? I was SO sure it would
>> >> > work!
>> >> >
>> >> > FYI, you tried the patch I attached to the email message, right?
>> >> >
>> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton 
>> >> > wrote:
>> >> >>
>> >> >> Update:  I did try the patch after getting it installed correctly,
>> >> >> but
>> >> >> I'm still getting a segfault on the newly built binary.
>> >> >> Will post info this afternoon.
>> >> >>
>> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez 
>> >> >> wrote:
>> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>> >> >> > unconditionally
>> >> >> > checks the length of the (possibly null) string argument first.
>> >> >> >
>> >> >> > Please try the patch I attached in the last email. It *might* fix
>> >> >> > the
>> >> >> > issue.
>> >> >> > Android has crappy locale handling.
>> >> >> >
>> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>> >> >> > 
>> >> >> > wrote:
>> >> >> >>
>> >> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
>> >> >> >> currently returning NULL when given a null pointer.
>> >> >> >>
>> >> >> >> From obmalloc.c
>> >> >> >>
>> >> >> >> _PyMem_RawStrdup(const char *str)
>> >> >> >> {
>> >> >> >> size_t size;
>> >> >> >> char *copy;
>> >> >> >> size = strlen(str) + 1;
>> >> >> >> copy = PyMem_RawMalloc(size);
>> >> >> >> if (copy == NULL)
>> >> >> >> return NULL;
>> >> >> >> memcpy(copy, str, size);
>> >> >> >> return copy;
>> >> >> >>   }
>> >> >> >>
>> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>> >> >> >> 
>> >> >> >> wrote:
>> >> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
>> >> >> >> > crashes
>> >> >> >> > when
>> >> >> >> > calling strlen. It's that whatever is calling it is likely
>> >> >> >> > asking
>> >> >> >> > it
>> >> >> >> > to
>> >> >> >> > duplicate a null pointer. Basically, it's probably the caller's
>> >> >> >> > fault.
>> >> >> >> >
>> >> >> >> > You could always try modifying _PyMem_RawStrdup to return NULL
>> >> >> >> > when
>> >> >> >> > given a
>> >> >> >> > null pointer and see where it then segfaults.
>> >> >> >> >
>> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>> >> >> >> > 
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> Alternatively, is there a hassle-free way to find out what
>> >> >> >> >> changed
>> >> >> >> >> in
>> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>> 

Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

[Ryan Gonzalez]

Do you have just the SDK (which doesn't require Cygwin) or a rooted phone?
If so, I can upload instructions that don't use the NDK.

On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton  wrote:

> This is going to take some time...here's why:
>
> Downloading and installing the NDK/SDK won't be too hard...I have to
> clear some space...but my primary machine is running Windows 7 and I'd
> rather swallow hot coals than install Cygwin.  I've got next to no
> experience with it, other than knowing that the NDK recommends against
> it.
>
> I've got a CentOS VM...but it's currently in tarball form on an
> external drive for space reasons...and it only has the NDK installed.
> If I am able to get it back up and running, there's still the task of
> getting  adb connected to my device. from the VM.
>
> Not saying it's impossible...just that it'll take time...and I'll
> probably have to tackle it tomorrow (earliest) or Sunday (latest).  In
> the meantime I'll also check to see if there's anything that can a)
> run in an Android terminal and b) can take a stack trace; it would be
> far, far, far easier than either option above.
>
>
>
> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez  wrote:
> > Do you have the Android SDK and NDK installed? If so...
> >
> > Using Google, I've created this series of steps, which may (or may not)
> > work:
> >
> > 1. Make sure you have a copy of Python on your computer and make sure
> that
> > it's built with debug symbols.
> >
> > 2. Run the following commands from a shell with your phone plugged in:
> >
> >   adb forward tcp:5039 tcp:5039
> >   adb shell /system/bin/gdbserver tcp:5039 
> >   /arm-linux-androideabi-gdb
> >
> > Now, GDB should have opened, so type the following commands:
> >
> >   set solib-search-path 
> >   file 
> >   target remote :5055
> >   run
> >
> > Now, wait for the program to crash and type:
> >
> >   backtrace
> >
> > You should now see a complete backtrace in your terminal. To GDB, type:
> >
> >   quit
> >
> > *crosses fingers*
> >
> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton 
> wrote:
> >>
> >> Unfortunately it is still reporting the same function :-/.
> >>
> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez 
> wrote:
> >> > Yes...
> >> >
> >> > Can you check if it's crashing in a different function now?
> >> >
> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton 
> >> > wrote:
> >> >>
> >> >> Yes I did.  I did have to enter all the information in manually...I'm
> >> >> not familiar with automated patch application tools and even if I
> >> >> were, I'm pretty sure I wouldn't have them on my device.
> >> >>
> >> >> Just so that I'm absolutely sure I got everything correct...you
> wanted
> >> >> all of the lines in the patch commented out, correct?  Basically
> >> >> everything referencing oldloc?
> >> >>
> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez 
> >> >> wrote:
> >> >> > Are you sure the patch was applied correctly? I was SO sure it
> would
> >> >> > work!
> >> >> >
> >> >> > FYI, you tried the patch I attached to the email message, right?
> >> >> >
> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
> chasel...@gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Update:  I did try the patch after getting it installed correctly,
> >> >> >> but
> >> >> >> I'm still getting a segfault on the newly built binary.
> >> >> >> Will post info this afternoon.
> >> >> >>
> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez  >
> >> >> >> wrote:
> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
> >> >> >> > unconditionally
> >> >> >> > checks the length of the (possibly null) string argument first.
> >> >> >> >
> >> >> >> > Please try the patch I attached in the last email. It *might*
> fix
> >> >> >> > the
> >> >> >> > issue.
> >> >> >> > Android has crappy locale handling.
> >> >> >> >
> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
> >> >> >> > 
> >> >> >> > wrote:
> >> >> >> >>
> >> >> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup
> is
> >> >> >> >> currently returning NULL when given a null pointer.
> >> >> >> >>
> >> >> >> >> From obmalloc.c
> >> >> >> >>
> >> >> >> >> _PyMem_RawStrdup(const char *str)
> >> >> >> >> {
> >> >> >> >> size_t size;
> >> >> >> >> char *copy;
> >> >> >> >> size = strlen(str) + 1;
> >> >> >> >> copy = PyMem_RawMalloc(size);
> >> >> >> >> if (copy == NULL)
> >> >> >> >> return NULL;
> >> >> >> >> memcpy(copy, str, size);
> >> >> >> >> return copy;
> >> >> >> >>   }
> >> >> >> >>
> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
> >> >> >> >> 
> >> >> >> >> wrote:
> >> >> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
> >> >> >> >> > crashes
> >> >> >> >> > when
> >> >> >> >> > calling strlen. It's that whatever is calling it is likely
> >> >> >> >> > asking
> >> >> >> >> > it
> >> >> >> >> > to
> >> >> >> >> > duplicate a null pointer. Basically, it's probably the
> caller's
> >> >> >> >> > fault.

Re: [Python-Dev] Results of the python 2.x and 3.x use survey, 2014 edition

[Tetsuya Morimoto]

Thank you for your hard work. That's very helpful!

thanks,
Tetsuya

On Fri, Jan 30, 2015 at 2:49 AM, Bruno Cauet  wrote:

> Hi!
> Finally, here are the results:
> http://blog.frite-camembert.net/python-survey-2014.html
> Here is the auto-generated Google Forms recap:
>
> https://docs.google.com/forms/d/1DqxkNi4GvyTCu54usSdE1DjW29zw1tc52iMeH3z4heg/viewanalytics
> (more elegant than my matplotlib graphs - I'd have no future as a
> designer).
>
> Overall people started writing more python 3: +15 points in "I ever
> wrote python 3 code", +10 points in "I write more python 3 than 2".
> Transition is still ongoing and I hope a tipping point will be soon be
> attained.
> Users definitely seem to want to switch to python 3, but dependencies
> keep them with 2.7 (I weep for the few ones still on 2.5).
>
> I also posted the results on HN
> (https://news.ycombinator.com/item?id=8967645) and reddit
> (
> http://www.reddit.com/r/Python/comments/2u3oh0/results_of_the_python_2x_and_3x_use_survey_2014/
> ).
> Have a nice day, and a year full of python 3!
>
> Bruno Cauet
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/tetsuya.morimoto%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com