[issue8478] tokenize.untokenize first token missing failure case
New submission from rb :
When altering tokens and thus not providing token location information,
tokenize.untokenize sometimes misses out the first token. Failure case below.
Expected output: 'import foo ,bar\n'
Actual output: 'foo ,bar\n'
$ python
Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import StringIO, tokenize
>>>
>>> def strip(iterable):
... for t_type, t_str, (srow, scol), (erow, ecol), line in iterable:
... yield t_type, t_str
...
>>> source = StringIO.StringIO('import foo, bar\n')
>>> print
>>> repr(tokenize.untokenize(strip(tokenize.generate_tokens(source.readline
'foo ,bar \n'
>>> source.seek(0)
>>> print repr(tokenize.untokenize(tokenize.generate_tokens(source.readline)))
'import foo, bar\n'
>>>
--
components: Library (Lib)
messages: 103799
nosy: rb
severity: normal
status: open
title: tokenize.untokenize first token missing failure case
versions: Python 2.6
___
Python tracker
<http://bugs.python.org/issue8478>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8478] tokenize.untokenize first token missing failure case
Changes by rb : -- type: -> behavior ___ Python tracker <http://bugs.python.org/issue8478> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
New submission from rb <[EMAIL PROTECTED]>:
Python cannot be embedded in shared library due to dependency problems
with lib-dynload.
I am implementing a shared library (in C) that implements a specific API
as a dynamically loadable plugin to an application. To implement this
library, I am calling out to Python via the C API.
When my code tries to load a Python module, it fails. This is because
lib-dynload/time.so (for example) cannot resolve symbols that are in
libpython2.5.so because it never looks there.
In this test case I'm trying to import "time", but it seems to fail on
other modules too - presumably anything that is in lib-dynload. It also
fails if I import a pure Python module that then tries to import time.
The error produced is: ImportError:
/usr/lib/python2.5/lib-dynload/time.so: undefined symbol: PyExc_ValueError
>From LD_DEBUG:
29682:file=/usr/lib/python2.5/lib-dynload/time.so [0]; needed by
/usr/lib/libpython2.5.so.1.0 [0]
...
29682:relocation processing: /usr/lib/python2.5/lib-dynload/time.so
29682:symbol=PyExc_ValueError; lookup in file=./myprog [0]
29682:symbol=PyExc_ValueError; lookup in file=/lib/libdl.so.2 [0]
29682:symbol=PyExc_ValueError; lookup in file=/lib/libc.so.6 [0]
29682:symbol=PyExc_ValueError; lookup in
file=/lib64/ld-linux-x86-64.so.2 [0]
29682:symbol=PyExc_ValueError; lookup in
file=/usr/lib/python2.5/lib-dynload/time.so [0]
29682:symbol=PyExc_ValueError; lookup in file=/lib/libm.so.6 [0]
29682:symbol=PyExc_ValueError; lookup in file=/lib/libpthread.so.0 [0]
29682:symbol=PyExc_ValueError; lookup in file=/lib/libc.so.6 [0]
29682:symbol=PyExc_ValueError; lookup in
file=/lib64/ld-linux-x86-64.so.2 [0]
29682:/usr/lib/python2.5/lib-dynload/time.so: error: symbol lookup
error:
undefined symbol: PyExc_ValueError (fatal)
$ nm -D /usr/lib/libpython2.5.so.1|grep PyExc_ValueError
0033a7e0 D PyExc_ValueError
$ ldd /usr/lib/python2.5/lib-dynload/time.so
libm.so.6 => /lib/libm.so.6 (0x2afe014c9000)
libpthread.so.0 => /lib/libpthread.so.0 (0x2afe0174a000)
libc.so.6 => /lib/libc.so.6 (0x2afe01967000)
/lib64/ld-linux-x86-64.so.2 (0x4000)
I am told that the problem is that lib-dynload/*.so should depend on
libpython2.5.so.1.
Test case attached. mylib.c is the library that I'm implementing reduced
to the problem case. myprog.c is a stub program that loads mylib.c to
demonstrate the problem. The "real" myprog would be any third-party
application that I have no control over that expects to be able to
dlopen() mylib.so and call functions within it.
I have been given the following workaround: in mylib.c, before
PyInitialize() I can call dlopen("libpython2.5.so", RTLD_LAZY |
RTLD_GLOBAL);
This works, but I believe that lib-dynload/*.so should depend on
libpython2.5.so.1 so this hack should not be necessary.
I am using Ubuntu 8.04 with Python version 2.5.2-2ubuntu4.1.
--
components: Library (Lib)
files: mylib.c
messages: 76454
nosy: rb
severity: normal
status: open
title: Embedding into a shared library fails
type: behavior
versions: Python 2.5
Added file: http://bugs.python.org/file12134/mylib.c
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue4434>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
Changes by rb <[EMAIL PROTECTED]>: Added file: http://bugs.python.org/file12135/myprog.c ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4434> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
rb <[EMAIL PROTECTED]> added the comment: What is the purpose of linking to the static library if you're still going to depend on shared objects in lib-dynload? ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4434> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
rb <[EMAIL PROTECTED]> added the comment: The problem, and the reason for the existence of this bug, is that I cannot build a shared object that links to libpython2.5.so.1 and works. Please don't mark this bug invalid until this problem is fixed. My proposal of adding dependencies to lib-dynload/*.so is just a suggestion on how to go about fixing this bug. If this is not a suitable solution, please suggest a solution that is. In response to your discussion on why the static library is currently useful, you haven't really answered the question. In what circumstance would someone actually want to build against the static library and then dynamically load other libraries? "Because it works" isn't an answer, as if this bug were fixed, then building against the shared library would validly have the same response. Why static over dynamic, if when static you have to link to dynamic anyway? ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4434> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
rb <[EMAIL PROTECTED]> added the comment: Ralf, I'm sure it does work, but what is the point of linking statically to libpython.a but then having other dependencies, for example on lib-dynload/time.so? Why not just link to libpython2.5.so in the first place? ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4434> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31861] add aiter() and anext() functions to operator module
Change by rb : -- nosy: +rb ___ Python tracker <https://bugs.python.org/issue31861> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28167] remove platform.linux_distribution()
rb added the comment: > Maintaining the necessary logic Python is not really possible in the stdlib. > It's better to have a PyPI module for this which can be updated much more > easily. The /etc/os-release syntax is stable, the file is implemented nearly everywhere and is unlikely to change, so I don't see why it'll be difficult to maintain. Just parse and give us the key/value pairs, please! It's disappointing to have to add Yet Another Dependency to get this functionality, contrary to Python's "batteries included" philosophy. -- nosy: +rb ___ Python tracker <https://bugs.python.org/issue28167> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28167] remove platform.linux_distribution()
rb added the comment: Apologies for my tone. I wasn't aware that starting out with a PyPI module is the only accepted process for getting functionality into stdlib. That certainly is a lot of work for what would be a trivial handful of lines to parse key/value pairs and handle error paths, etc (and corresponding tests of course). As a distribution developer, from my perspective we've already (finally) managed to reach widespread standardization and unification on a simple key/value pair specification, so it is particularly surprising to me that as a Python developer I cannot easily get to it. IMHO, forcing the PyPI route seems like a recipe for scope creep to me in this case. Looking at that module, it tries to do so much more. /etc/os-release is widely accepted nowadays, and I think it would be widely useful to the majority of users to simply pass the values through to a Python dictionary, reflecting "this is what the OS says it is" over "this is my promised stable view". Many of the issues there are because it's trying to be "clever", IMHO, rather than just passing through the provided information from the platform. My proposed API promise would simply be "this is what os-release says" or "os-release information not available". Of course the most popular PyPI module to solve a particular problem will always be the one with the widest scope, so there seems little point in writing a parallel minimal one. The cost of adding Yet Another Dependency has to be paid in either case. -- ___ Python tracker <https://bugs.python.org/issue28167> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5509] cPickle - module object has no attribute
rb added the comment: I've just been stung by this. I've noticed that this seems to apply to both cPickle and pickle. Even worse, it causes different behaviour when a program is run under pdb because __main__ is suddenly pdb rather than the program itself. So, in summary, neither pickle nor cPickle can pickle a class if it is not defined in its own module? Surely this is an obvious deficiency of pickle and if it is not going to be fixed it should at least be documented as such? -- nosy: +rb ___ Python tracker <http://bugs.python.org/issue5509> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5518] cPickle produces inconsistent output
New submission from rb : The documentation states that the output of pickle and cPickle may be different. However it is implied that the output of a particular module will always be consistent within itself. This expectation fails for the case below. I am using the output of cPickle in order to generate a key to use for external storage where the key is abstracted to a generic Python (immutable) object. Without consistency this breaks for me; pickle is too slow so I need to use cPickle. $ python Python 2.5.2 (r252:60911, Oct 5 2008, 19:29:17) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import cPickle >>> key = (1, u'foo') >>> cPickle.dumps(key) '(I1\nVfoo\ntp1\n.' >>> cPickle.dumps((1, u'foo')) '(I1\nVfoo\np1\ntp2\n.' PythonWin 2.6.1 (r261:67517, Dec 4 2008, 16:51:00) [MSC v.1500 32 bit (Intel)] on win32. Portions Copyright 1994-2008 Mark Hammond - see 'Help/About PythonWin' for further copyright information. >>> import cPickle >>> key = (1,u'foo') >>> cPickle.dumps(key) '(I1\nVfoo\ntp1\n.' >>> cPickle.dumps((1,u'foo')) '(I1\nVfoo\np1\ntp2\n.' Expected results: the output of the two dumps calls should be the same. -- components: Library (Lib) messages: 83814 nosy: rb severity: normal status: open title: cPickle produces inconsistent output versions: Python 2.5, Python 2.6 ___ Python tracker <http://bugs.python.org/issue5518> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5518] cPickle produces inconsistent output
rb added the comment: Martin, Sorry, I don't follow. I realise that the refcounts will be different; but pickling an object should surely be independent of the refcount as there is no need to include the refcount in the output? What other way (using pickle or not) can I convert a generic immutable Python object to a string to use as a key in external storage? Currently the documentation points out that the output may be different between pickle and cPickle which implies that the output will be consistent for a single module. If pickle is not required to produce consistent output for the same input (and refcount isn't really part of the input in this case; it is a side issue) than can this be documented? -- ___ Python tracker <http://bugs.python.org/issue5518> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4434] Embedding into a shared library fails
rb added the comment: Original reporter here. lib-dynload is part of Python's dynamic loading mechanism. Perhaps somewhere like Python/dynload_dl.c is the relevant code? Lib seemed like the right place to put the bug at the time, since it was the stdlib module files that were missing dependency declarations. Although perhaps Extension Modules would have been better? In any case, I've just tried to reproduce the bug with 2.7 (on Debian Squeeze pulling version 2.7-2 of python2.7 and python2.7-dev from experimental), and failed (both 32- and 64-bit). So it seems that this bug has been fixed at some point since 2.5. -- ___ Python tracker <http://bugs.python.org/issue4434> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25230] Unix datagram sockets not supported
New submission from rb: AF_UNIX, SOCK_DGRAM sockets are valid, but asyncio doesn't appear to support them. I've tried combinations of create_connection, create_datagram_endpoint and create_unix_connection, creating a socket myself and passing in sock, and equivalent methods at the server end. There seem to be implicit assumptions about addresses being 2-tuples (instead of strings) and transports being hardcoded to be constructed as either stream or datagram transports. create_unix_connection makes the assumption that it will be a stream, and create_datagram_endpoint that it will be AF_INET or AF_INET6 with (host, port) addressing. I used 3.4.3, but looking at the docs it doesn't look like this was addressed in 3.5 either. I'd like this because message boundaries are preserved (unlike in SOCK_STREAM), which if it Just Worked would make local system IPC (eg. with JSON-RPC) extremely trivial to implement in asyncio. -- components: asyncio messages: 251549 nosy: gvanrossum, haypo, rb, yselivanov priority: normal severity: normal status: open title: Unix datagram sockets not supported type: enhancement versions: Python 3.4 ___ Python tracker <http://bugs.python.org/issue25230> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
