Re: [Python-Dev] how to find the file path to an extension module at init time?
"Martin v. Löwis", 13.11.2011 21:46: I'm asking specifically because I'd like to properly implement __file__ in Cython modules at module init time. Why do you need to implement __file__? Python will set it eventually to its correct value, no? Well, yes, eventually. However, almost all real world usages are at module init time, not afterwards. So having CPython set it after running through the module global code doesn't help much. Another problem is that package local imports from __init__.py no longer work when it's compiled Does it actually work to have __init__ be an extension module? I'm just starting to try it, and the problems I found so far were __file__ (in general), __path__ and relative imports (specifically). Any ideas how this could currently be achieved? Currently, for Cython? I don't think that can work. Hmm, it might work to put an empty module next to the 'real' extension and to import it to figure out the common directory of both. As long as it's still there after installation and the right one gets imported, that is. A relative import should help on versions that support it. Although that won't help in the __init__ case because a relative import will likely depend on __path__ being available first. Chicken and egg... Support in CPython would definitely help. Or could this become a new feature in the future? Certainly. An approach similar to _Py_PackageContext should be possible. Yes, and a "_Py_ModuleImportContext" would be rather trivial to do. Could that go into 3.3? What about 2.7? Could an exception be made there regarding new "features"? It's not likely to break anything, but it would help Cython. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] peps: And now for something completely different.
If the PEP 404 lists important changes between Python 2 and Python 3, the removal of old-style classes should also be mentioned because it is a change in the core language. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Delivery reports about your e-mail
The original message was received at Mon, 14 Nov 2011 12:56:42 +0200 from ghaering.de [173.1.122.19] - The following addresses had permanent fatal errors - - Transcript of session follows - while talking to python.org.: >>> MAIL From:g...@ghaering.de <<< 501 g...@ghaering.de... Refused ÿþS m a l l B u s i n e s s S e r v e r h a s r e m o v e d p o t e n t i a l l y u n s a f e e - m a i l a t t a c h m e n t ( s ) f r o m t h i s m e s s a g e : d o c u m e n t . b a t B e c a u s e c o m p u t e r v i r u s e s a r e c o m m o n l y s p r e a d t h r o u g h f i l e s a t t a c h e d t o e - m a i l m e s s a g e s , c e r t a i n t y p e s o f f i l e s w i l l n o t b e d e l i v e r e d t o y o u r m a i l b o x . F o r m o r e i n f o r m a t i o n , c o n t a c t t h e p e r s o n r e s p o n s i b l e f o r y o u r n e t w o r k . ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] how to find the file path to an extension module at init time?
On Mon, Nov 14, 2011 at 6:18 PM, Stefan Behnel wrote: >> Certainly. An approach similar to _Py_PackageContext should be possible. > > Yes, and a "_Py_ModuleImportContext" would be rather trivial to do. Could > that go into 3.3? What about 2.7? Could an exception be made there regarding > new "features"? It's not likely to break anything, but it would help Cython. Hmm, interesting call - fixing this doesn't actually require a new public API, since it's just a new data attribute protected by the import lock that is used to pass state information from importdl.c to moduleobject.c. I'm inclined to say "no" myself, but it's a close run thing. So +1 for fixing it in 3.3, and -0 for calling it a bug rather than a missing feature and also fixing it in 2.7 Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] how to find the file path to an extension module at init time?
On 14 November 2011 08:18, Stefan Behnel wrote: > "Martin v. Löwis", 13.11.2011 21:46: >>> >>> I'm asking specifically because I'd like to properly implement __file__ >>> in Cython modules at module init time. >> >> Why do you need to implement __file__? Python will set it eventually to >> its correct value, no? > > Well, yes, eventually. However, almost all real world usages are at module > init time, not afterwards. So having CPython set it after running through > the module global code doesn't help much. > Perhaps Cython could detect use of __file__ at module scope (if this package context function is not available), and if it's used it tries to use something akin to imp.find_module(__name__) to find the path to the file and set __file__ manually. It should handle dots out of the box and perhaps not rely on any __path__ attributes of packages (I think not many people change __path__ or use pkgutil anyway). Would this be feasible for python < 3.3? >>> Another problem is that package local imports from __init__.py no longer >>> work when it's compiled >> >> Does it actually work to have __init__ be an extension module? > > I'm just starting to try it, and the problems I found so far were __file__ > (in general), __path__ and relative imports (specifically). > > >>> Any ideas how this could currently be achieved? >> >> Currently, for Cython? I don't think that can work. > > Hmm, it might work to put an empty module next to the 'real' extension and > to import it to figure out the common directory of both. As long as it's > still there after installation and the right one gets imported, that is. A > relative import should help on versions that support it. Although that won't > help in the __init__ case because a relative import will likely depend on > __path__ being available first. Chicken and egg... > > Support in CPython would definitely help. > > >>> Or could this become a new feature in the future? >> >> Certainly. An approach similar to _Py_PackageContext should be possible. > > Yes, and a "_Py_ModuleImportContext" would be rather trivial to do. Could > that go into 3.3? What about 2.7? Could an exception be made there regarding > new "features"? It's not likely to break anything, but it would help Cython. > > Stefan > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/markflorisson88%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] how to find the file path to an extension module at init time?
mark florisson, 14.11.2011 12:55: On 14 November 2011 08:18, Stefan Behnel wrote: "Martin v. Löwis", 13.11.2011 21:46: I'm asking specifically because I'd like to properly implement __file__ in Cython modules at module init time. Why do you need to implement __file__? Python will set it eventually to its correct value, no? Well, yes, eventually. However, almost all real world usages are at module init time, not afterwards. So having CPython set it after running through the module global code doesn't help much. Perhaps Cython could detect use of __file__ at module scope (if this package context function is not available), and if it's used it tries to use something akin to imp.find_module(__name__) to find the path to the file and set __file__ manually. It's problematic. Depending on the import hooks that are in use, a second search for the already loaded but not yet instantiated module may potentially trigger a second try to load it. And if the module is already put into sys.modules, a second search may not return a helpful result. Also, running a full import of "imp" along the way may have additional side effects, and the C-API doesn't have a way to just search for the path of a module. Would this be feasible for python < 3.3? It would certainly not be more than a work-around, but it could be made to work "well enough" in many cases. I'd definitely prefer having something that really works in both 2.7.x and 3.3+ over a half-baked solution in all of Py2.x. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Fix memory leak with FLUFL-related syntax errors (!)
> changeset: 0feb5a5dbaeb > user:Antoine Pitrou > date:Sun Nov 13 01:02:02 2011 +0100 > summary: > Fix memory leak with FLUFL-related syntax errors (!) I don’t think it is allowed to criticize FLUFL-related code. As the FLUFL is flawless, so are flufly things. I hear the PSU has been notified; a messenger will come t ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] urllib.request.urlopen struggling in Windows 7
Previously, in python 2.6, I had made a lot of use of urllib.urlopen to capture
web page content and then post process the data from the site I was downloading.
Now, those routines, and the new routines I am trying to use for python 3.2 are
running into what seems to be a windows only (maybe even windows 7 only
problem).
Using the following code with python 3.2.2 (64) on windows 7 ...
import urllib.request
fp = urllib.request.urlopen(URL_string_that_I_use)
string = fp.read()
fp.close()
print(string.decode("utf8"))
I get the following message:
Traceback (most recent call last):
File "TATest.py", line 5, in
string = fp.read()
File "d:\python32\lib\http\client.py", line 489, in read
return self._read_chunked(amt)
File "d:\python32\lib\http\client.py", line 553, in _read_chunked
self._safe_read(2) # toss the CRLF at the end of the chunk
File "d:\python32\lib\http\client.py", line 592, in _safe_read
raise IncompleteRead(b''.join(s), amt)
http.client.IncompleteRead: IncompleteRead(0 bytes read, 2 more expected)
Using the following code instead ...
import urllib.request
fp = urllib.request.urlopen(URL_string_that_I_use)
for Line in fp:
print(Line.decode("utf8").rstrip('\n'))
fp.close()
I get a fair amount of the web page's content, but then the rest of the capture
is thwarted by ...
Traceback (most recent call last):
File "TATest.py", line 9, in
for Line in fp:
File "d:\python32\lib\http\client.py", line 489, in read
return self._read_chunked(amt)
File "d:\python32\lib\http\client.py", line 545, in _read_chunked
self._safe_read(2) # toss the CRLF at the end of the chunk
File "d:\python32\lib\http\client.py", line 592, in _safe_read
raise IncompleteRead(b''.join(s), amt)
http.client.IncompleteRead: IncompleteRead(0 bytes read, 2 more expected)
Trying to read another page yields ...
Traceback (most recent call last):
File "TATest.py", line 11, in
print(Line.decode("utf8").rstrip('\n'))
File "d:\python32\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\x92' in position
21: character maps to
I do believe this is a windows issue, but can python be made more robust to deal
with what is causing it? When trying similar code on Linux, we do not encounter
the problem.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] urllib.request.urlopen struggling in Windows 7
> I do believe this is a windows issue, but can python be made more robust to > deal > with what is causing it? I can't believe that it's a Windows issue, and neither can I believe that it's a Python issue (although this is more likely). Most likely, it's a server issue, i.e. the server somehow closes the connection without providing all the data it ought to provide. If that's the issue, Python can do nothing about it - you need to fix the server. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] how to find the file path to an extension module at init time?
>> Currently, for Cython? I don't think that can work. > > Hmm, it might work to put an empty module next to the 'real' extension > and to import it to figure out the common directory of both. As long as > it's still there after installation and the right one gets imported, > that is. A relative import should help on versions that support it. > Although that won't help in the __init__ case because a relative import > will likely depend on __path__ being available first. Chicken and egg... If there was an actual __init__.py that just had import __cinit__ then __cinit__ could copy __path__ from the already-loaded __init__, no? >> Certainly. An approach similar to _Py_PackageContext should be possible. > > Yes, and a "_Py_ModuleImportContext" would be rather trivial to do. > Could that go into 3.3? If somebody contributes a patch: sure. > What about 2.7? Certainly not. It would be a new feature, and there can't be new features in 2.7. > Could an exception be made there regarding new "features"? > It's not likely to break anything, but it would help Cython. "Not likely to break anything" really means it practice "it probably will break somebody's code". Policies are there to be followed, and this one I personally feel strongly about. If it means that users can get certain Cython features only with Python 3.x, the better. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
