Re: [Python-Dev] Why does python use relative instead of absolute path when calling LoadLibrary*
Thank you both for your answers. I will go away with this modification, and see how it goes. David On Thu, Mar 12, 2015 at 2:41 AM, Wes Turner wrote: > > On Mar 11, 2015 3:36 PM, "David Cournapeau" wrote: > > > > Hi, > > > > While looking at the import code of python for C extensions, I was > wondering why we pass a relative path instead of an absolute path to > LoadLibraryEx (see bottom for some context). > > > > In python 2.7, the full path existence was even checked before calling > into LoadLibraryEx ( > https://github.com/python/cpython/blob/2.7/Python/dynload_win.c#L189), > but it looks like this check was removed in python 3.x branch. > > > > Is there any defined behaviour that depends on this path to be relative ? > > Just a guess: does it have to do with resolving symlinks (w/ POSIX > filesystems)? > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 488: elimination of PYO files
Here is the latest draft of the PEP. I have closed the issue of file name
formatting thanks to the informal poll results being very clear on the
preferred format and also closed the idea of embedding the optimization
level in the bytecode file metadata (that can be another PEP if someone
cares to write that one). The only remaining open issue is whether to
special-case non-optimized bytecode file names and completely leave out the
optimization level in that instance.
I think this PEP is close enough to be finished to ask whether Guido wants
to wield his BDFL stick for this PEP or if he wants to delegate to a BDFAP.
PEP: 488
Title: Elimination of PYO files
Version: $Revision$
Last-Modified: $Date$
Author: Brett Cannon
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 20-Feb-2015
Post-History:
2015-03-06
2015-03-13
Abstract
This PEP proposes eliminating the concept of PYO files from Python.
To continue the support of the separation of bytecode files based on
their optimization level, this PEP proposes extending the PYC file
name to include the optimization level in bytecode repository
directory (i.e., the ``__pycache__`` directory).
Rationale
=
As of today, bytecode files come in two flavours: PYC and PYO. A PYC
file is the bytecode file generated and read from when no
optimization level is specified at interpreter startup (i.e., ``-O``
is not specified). A PYO file represents the bytecode file that is
read/written when **any** optimization level is specified (i.e., when
``-O`` is specified, including ``-OO``). This means that while PYC
files clearly delineate the optimization level used when they were
generated -- namely no optimizations beyond the peepholer -- the same
is not true for PYO files. Put in terms of optimization levels and
the file extension:
- 0: ``.pyc``
- 1 (``-O``): ``.pyo``
- 2 (``-OO``): ``.pyo``
The reuse of the ``.pyo`` file extension for both level 1 and 2
optimizations means that there is no clear way to tell what
optimization level was used to generate the bytecode file. In terms
of reading PYO files, this can lead to an interpreter using a mixture
of optimization levels with its code if the user was not careful to
make sure all PYO files were generated using the same optimization
level (typically done by blindly deleting all PYO files and then
using the `compileall` module to compile all-new PYO files [1]_).
This issue is only compounded when people optimize Python code beyond
what the interpreter natively supports, e.g., using the astoptimizer
project [2]_.
In terms of writing PYO files, the need to delete all PYO files
every time one either changes the optimization level they want to use
or are unsure of what optimization was used the last time PYO files
were generated leads to unnecessary file churn. The change proposed
by this PEP also allows for **all** optimization levels to be
pre-compiled for bytecode files ahead of time, something that is
currently impossible thanks to the reuse of the ``.pyo`` file
extension for multiple optimization levels.
As for distributing bytecode-only modules, having to distribute both
``.pyc`` and ``.pyo`` files is unnecessary for the common use-case
of code obfuscation and smaller file deployments.
Proposal
To eliminate the ambiguity that PYO files present, this PEP proposes
eliminating the concept of PYO files and their accompanying ``.pyo``
file extension. To allow for the optimization level to be unambiguous
as well as to avoid having to regenerate optimized bytecode files
needlessly in the `__pycache__` directory, the optimization level
used to generate a PYC file will be incorporated into the bytecode
file name. Currently bytecode file names are created by
``importlib.util.cache_from_source()``, approximately using the
following expression defined by PEP 3147 [3]_, [4]_, [5]_::
'{name}.{cache_tag}.pyc'.format(name=module_name,
cache_tag=sys.implementation.cache_tag)
This PEP proposes to change the expression to::
'{name}.{cache_tag}.opt-{optimization}.pyc'.format(
name=module_name,
cache_tag=sys.implementation.cache_tag,
optimization=str(sys.flags.optimize))
The "opt-" prefix was chosen so as to provide a visual separator
from the cache tag. The placement of the optimization level after
the cache tag was chosen to preserve lexicographic sort order of
bytecode file names based on module name and cache tag which will
not vary for a single interpreter. The "opt-" prefix was chosen over
"o" so as to be somewhat self-documenting. The "opt-" prefix was
chosen over "O" so as to not have any confusion with "0" while being
so close to the interpreter version number.
A period was chosen over a hyphen as a separator so as to distinguish
clearly that the optimization level is not part of the interpreter
version as specified by the cache tag. It also
Re: [Python-Dev] Request for Pronouncement: PEP 441 - Improving Python ZIP Application Support
On Sun, Mar 8, 2015 at 10:35 AM Brett Cannon wrote: > > On Sun, Mar 8, 2015, 08:40 Paul Moore wrote: > > On 26 February 2015 at 21:48, Paul Moore wrote: > > On 26 February 2015 at 21:34, Guido van Rossum wrote: > >> Accepted! > >> > >> Thanks for your patience, Paul, and thanks everyone for their feedback. > >> > >> I know there are still a few small edits to the PEP, but those don't > affect > >> my acceptance. Congrats! > > > > Excellent, thanks to everyone for the helpful comments throughout. > > I'm still looking for someone who has the time to commit the final > patch on http://bugs.python.org/issue23491. Python 3.5a2 is due out > today (at least according to the PEP) so I've probably missed that, > but it would be nice if it could be in for alpha 3 at least. > > If no one gets to it I can submit on Friday. > Submitted in https://hg.python.org/cpython/rev/d1e9f337fea1 . Thanks for the hard work, Paul! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2015-03-06 - 2015-03-13)
Python tracker at http://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open4817 (+13)
closed 30604 (+49)
total 35421 (+62)
Open issues with patches: 2266
Issues opened (49)
==
#11726: clarify that linecache only works on files that can be decoded
http://bugs.python.org/issue11726 reopened by r.david.murray
#23566: RFE: faulthandler.register() should support file descriptors
http://bugs.python.org/issue23566 reopened by haypo
#23599: single and double quotes stripped upon paste with MacPorts lib
http://bugs.python.org/issue23599 opened by Jeff Doak
#23600: tizinfo.fromutc changed for tzinfo wih StdOffset=0, DstOffset=
http://bugs.python.org/issue23600 opened by peterjclaw
#23601: use small object allocator for dict key storage
http://bugs.python.org/issue23601 opened by jtaylor
#23602: Implement __format__ for Fraction
http://bugs.python.org/issue23602 opened by tuomas.suutari
#23603: Embedding Python3.4 - PyUnicode_Check fails (MinGW-W64)
http://bugs.python.org/issue23603 opened by Ashish Sadanandan
#23605: Use the new os.scandir() function in os.walk()
http://bugs.python.org/issue23605 opened by haypo
#23606: ctypes.util.find_library("c") no longer makes sense
http://bugs.python.org/issue23606 opened by steve.dower
#23607: Inconsistency in datetime.utcfromtimestamp(Decimal)
http://bugs.python.org/issue23607 opened by serhiy.storchaka
#23609: Export PyModuleObject in moduleobject.h
http://bugs.python.org/issue23609 opened by h.venev
#23611: Pickle nested names with protocols < 4
http://bugs.python.org/issue23611 opened by serhiy.storchaka
#23612: 3.5.0a2 Windows installer does not remove 3.5.0a1
http://bugs.python.org/issue23612 opened by steve.dower
#23614: Opaque error message on UTF-8 decoding to surrogates
http://bugs.python.org/issue23614 opened by Rosuav
#23616: Idle: conflict between loop execution and undo shortcut.
http://bugs.python.org/issue23616 opened by Davide Okami
#23618: PEP 475: handle EINTR in the socket module
http://bugs.python.org/issue23618 opened by haypo
#23619: Python 3.5.0a2 installer fails on Windows
http://bugs.python.org/issue23619 opened by pmoore
#23621: Uninstalling Python 3.5 removes a "py.exe" that was installed
http://bugs.python.org/issue23621 opened by pmoore
#23622: Deprecate unrecognized backslash+letter escapes
http://bugs.python.org/issue23622 opened by serhiy.storchaka
#23623: Python 3.5 docs need to clarify how to set PATH, etc
http://bugs.python.org/issue23623 opened by pmoore
#23624: str.center inconsistent with format "^"
http://bugs.python.org/issue23624 opened by Vedran.ÄaÄiÄ
#23626: Windows per-user install of 3.5a2 doesn't associate .py files
http://bugs.python.org/issue23626 opened by pmoore
#23628: See if os.scandir() could help speed up importlib
http://bugs.python.org/issue23628 opened by brett.cannon
#23630: support multiple hosts in create_server/start_server
http://bugs.python.org/issue23630 opened by sebastien.bourdeauducq
#23631: 3.5 (a2) traceback regression snarls Idle
http://bugs.python.org/issue23631 opened by terry.reedy
#23632: memoryview doesn't allow tuple-indexing
http://bugs.python.org/issue23632 opened by pitrou
#23633: Improve py launcher help, index, and doc
http://bugs.python.org/issue23633 opened by terry.reedy
#23634: os.fdopen reopening a read-only fd on windows
http://bugs.python.org/issue23634 opened by mattip
#23636: Add scgi to urllib.parse.uses_netloc
http://bugs.python.org/issue23636 opened by anthonyryan1
#23637: Warnings error with non-ascii chars.
http://bugs.python.org/issue23637 opened by Lukáš.NÄmec
#23639: Not documented special names
http://bugs.python.org/issue23639 opened by serhiy.storchaka
#23640: Enum.from_bytes() is broken
http://bugs.python.org/issue23640 opened by serhiy.storchaka
#23641: Got rid of bad dunder names
http://bugs.python.org/issue23641 opened by serhiy.storchaka
#23642: Interaction of ModuleSpec and C Extension Modules
http://bugs.python.org/issue23642 opened by dabeaz
#23644: g++ module compile fails with â_Atomicâ does not name a ty
http://bugs.python.org/issue23644 opened by Joshua.J.Cogliati
#23645: Incorrect doc for __getslice__
http://bugs.python.org/issue23645 opened by gdementen
#23646: PEP 475: handle EINTR in the time module, retry sleep()
http://bugs.python.org/issue23646 opened by haypo
#23647: imaplib.py MAXLINE value is too low for gmail
http://bugs.python.org/issue23647 opened by arnt
#23648: PEP 475 meta issue
http://bugs.python.org/issue23648 opened by haypo
#23649: tarfile not re-entrant for multi-threading
http://bugs.python.org/issue23649 opened by sgnn7
#23652: ifdef uses of EPOLLxxx macros so we can compile on systems tha
http://bugs.python.org/issue23652 opened by WanderingLogic
#23653: Make inspect._em
Re: [Python-Dev] Why does python use relative instead of absolute path when calling LoadLibrary*
On 3/11/15 9:36 PM, David Cournapeau wrote: Hi, While looking at the import code of python for C extensions, I was wondering why we pass a relative path instead of an absolute path to LoadLibraryEx (see bottom for some context). In python 2.7, the full path existence was even checked before calling into LoadLibraryEx (https://github.com/python/cpython/blob/2.7/Python/dynload_win.c#L189), but it looks like this check was removed in python 3.x branch. Is there any defined behaviour that depends on this path to be relative ? My two cents. According to the MSDN documentation, passing a relative path is actually undefined https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179%28v=vs.85%29.aspx LOAD_WITH_ALTERED_SEARCH_PATH 0x0008 If this value is used and lpFileName specifies an absolute path, the system uses the alternate file search strategy discussed in the Remarks section to find associated executable modules that the specified module causes to be loaded. If this value is used and lpFileName specifies a relative path, the behavior is undefined. --- meaning that wpathname _must_ be absolute. https://github.com/python/cpython/blob/master/Python/dynload_win.c#L222 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
