I've updated PEP 397 - "Python launcher for Windows" based on recent
discussions and Vinay's implementation.
http://www.python.org/dev/peps/pep-0397/ and a copy is attached for your
convenience.
The main changes are:
* All mentions of the Python reference implementation have been removed
now that a C implementation exists and is a more accurate implementation
of the PEP than the Python version.
* Some "implementation details" have been removed and links added to the
launcher docs. This was done mainly so the implementation is free to
make changes based on user feedback and still stay true to the PEP.
Note that the launcher doc link doesn't exist right now, but should
magically appear over the next couple of days as Vinay pushes some
changes I just sent him.
* The recommendation to install into System32 has been changed to a
recommendation to install directly into \Windows, as the System32
directory is not on the default path for 32bit processes on a 64bit
Windows. Vinay even has a version of an MSI installer which installs
into this directory. The PEP also gives installers more leeway on where
to install the launcher if this directory is not writable.
I think this PEP is getting close to being finalized - please suggest
whatever changes you feel are necessary ASAP as soon I'll be asking for
pronouncement.
Thanks - especially to Vinay for taking on the C implementation!
Mark
PEP: 397
Title: Python launcher for Windows
Version: $Revision$
Last-Modified: $Date$
Author: Mark Hammond <mhamm...@skippinet.com.au>
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 15-Mar-2011
Post-History: 21-July-2011, 17-May-2011, 15-Mar-2011
Abstract
This PEP describes a Python launcher for the Windows platform. A
Python launcher is a single executable which uses a number of
heuristics to locate a Python executable and launch it with a
specified command line.
Rationale
Windows provides "file associations" so an executable can be associated
with an extension, allowing for scripts to be executed directly in some
contexts (eg., double-clicking the file in Windows Explorer.) Until now,
a strategy of "last installed Python wins" has been used and while not
ideal, has generally been workable due to the conservative changes in
Python 2.x releases. As Python 3.x scripts are often syntactically
incompatible with Python 2.x scripts, a different strategy must be used
to allow files with a '.py' extension to use a different executable based
on the Python version the script targets. This will be done by borrowing
the existing practices of another operating system - scripts will be able
to nominate the version of Python they need by way of a "shebang" line, as
described below.
Unix-like operating systems (referred to simply as "Unix" in this
PEP) allow scripts to be executed as if they were executable images
by examining the script for a "shebang" line which specifies the
actual executable to be used to run the script. This is described in
detail in the evecve(2) man page [1] and while user documentation will
be created for this feature, for the purposes of this PEP that man
page describes a valid shebang line.
Additionally, these operating systems provide symbolic-links to
Python executables in well-known directories. For example, many
systems will have a link /usr/bin/python which references a
particular version of Python installed under the operating-system.
These symbolic links allow Python to be executed without regard for
where Python it actually installed on the machine (eg., without
requiring the path where Python is actually installed to be
referenced in the shebang line or in the PATH.) PEP 394 'The "python"
command on Unix-Like Systems' [2] describes additional conventions
for more fine-grained specification of a particular Python version.
These 2 facilities combined allow for a portable and somewhat
predictable way of both starting Python interactively and for allowing
Python scripts to execute. This PEP describes an implementation of a
launcher which can offer the same benefits for Python on the Windows
platform and therefore allows the launcher to be the executable
associated with '.py' files to support multiple Python versions
concurrently.
While this PEP offers the ability to use a shebang line which should
work on both Windows and Unix, this is not the primary motivation for
this PEP - the primary motivation is to allow a specific version to be
specified without inventing new syntax or conventions to describe it.
An overview of the launcher.
This PEP outlines the general functionality and key guidelines of a
launcher for Windows. It is accompanied by an implementation [3],
written in C, which defines the detailed implementation. Over
time, changes to the implementation may be desired - if the changes
adhere to the guidelines in this PEP and have been made following
the standard Python development model this PEP need not change.
In other words, this PEP makes no attempt to describe in detail every
feature offered by the launcher but instead to offer guidelines the
launcher should adhere to.
The launcher will come in 2 versions - one which is a console program and
one which is a "windows" (ie., GUI) program. These 2 launchers correspond
to the 'python.exe' and 'pythonw.exe' executables which currently ship
with Python. The console launcher will be named 'py.exe' and the Windows
one named 'pyw.exe'. The "windows" (ie., GUI) version of the launcher
will attempt to locate and launch pythonw.exe even if a virtual shebang
line nominates simply "python" - infact, the trailing 'w' notation will
not be supported in the virtual shebang line at all.
The launcher will be distributed with all future versions of Python
and if possible, should be installed directly into the Windows directory
(note that the System32 directory is not a good option as this directory
is not on the default PATH for 32bit processes on a 64bit Windows.) If
the launcher can't be installed in the Windows directory, the installer
can suggest or choose an alternative, but it will be the responsibility
of the user to ensure this directory is on the PATH.
Once installed, the "console" version of the launcher should be
associated with .py files and the "windows" version associated with .pyw
files.
The launcher will not be tied to a specific version of Python - eg., a
launcher distributed with Python 3.3 should be capable of locating and
executing any Python 2.x and Python 3.x version. Future versions of the
launcher should remain backwards compatible with older versions, so later
versions of Python can install an updated version of the launcher without
impacting how the previously installed version of the launcher is used.
The launcher may offer some conveniences for Python developers working
interactively - for example, starting the launcher with no command-line
arguments will launch the default Python with no command-line arguments.
Further, command-line arguments will be supported to allow a specific
Python version to be launched interactively - however, these conveniences
must not detract from the primary purpose of launching scripts and must
be easy to avoid if desired.
Guidelines for a Python launcher.
The Python launcher described in this PEP will intentionally be
constrained to the use-cases described in the Rationale section
above. It will not attempt to be a general purpose script launcher
or shebang processor.
The launcher should support for format of shebang lines as described
in [1], including all restrictions listed.
The launcher should support shebang lines commonly found on Unix.
For example, a shebang line of '#! /usr/bin/python' should work even
though there is unlikely to be an executable in the relative Windows
directory "\usr\bin". This means that many scripts can use a single
shebang line and be likely to work on both Unix and Windows without
modification.
The launcher will support fully-qualified paths to executables.
While this will make the script inherently non-portable, it is a
feature offered by Unix and would be useful for Windows users in
some cases.
The launcher will be capable of supporting implementations other than
CPython, such as jython and IronPython, but given both the absence of
common links on Unix (such as "/usr/bin/jython") and the inability for the
launcher to automatically locate the installation location of these
implementations on Windows, the launcher will support this via
customization options. Scripts taking advantage of this will not be
portable (as these customization options must be set to reflect the
configuration of the machine on which the launcher is running) but this
ability is nonetheless considered worthwhile.
On Unix, the user can control which specific version of Python is used
by adjusting the links in /usr/bin to point to the desired version. As
the launcher on Windows will not use Windows links, cutomization options
(exposed via both environment variables and INI files) will be used to
override the semantics for determining what version of Python will be
used. For example, while a shebang line of "/usr/bin/python2" will
automatically locate a Python 2.x implementation, an environment variable
can override exactly which Python 2.x implementation will be chosen.
Similarly for "/usr/bin/python" and "/usr/bin/python3". This is
specified in detail later in this PEP.
Shebang line parsing
If the first command-line argument does not start with a dash ('-')
character, an attempt will be made to open that argument as a file
and parsed for a shebang line according to the rules in [1]. Once
parsed, the command will be categorized according to the following rules:
* If the command starts with the definition of a customized command
followed by a space character, the customized command will be used.
See below for a description of customized commands.
* The launcher will define a set of strings which are considered Unix
compatible commands to launch Python, such as '/usr/bin/python' etc.
If a command matching one of these strings will be treated as a
'virtual command' and the rules described in Python Version Qualifiers
(below) will be used to locate the executable to use.
* Otherwise the command is assumed to be directly ready to execute - ie.
a fully-qualified path (or a reference to an executable on the PATH)
optionally followed by arguments. The contents of the string will not
be parsed - it will be passed directly to the Windows CreateProcess
function after appending the name of the script and the launcher
command-line arguments. This means that the rules used by
CreateProcess will be used, including how relative path names and
executable references without extensions are treated. Notably, the
Windows command processor will not be used, so special rules used by the
command processor (such as automatic appending of extensions other than
'.exe', support for batch files, etc) will not be used.
The use of 'virtual' shebang lines will be encouraged as this should
allow for portable shebang lines to be specified which work on
multiple operating systems and different installations of the same
operating system.
If the first argument can not be opened as a file or if no valid
shebang line can be found, the launcher will act as if a shebang line of
'#!python' was found - ie., a default Python interpreter will be
located and the arguments passed to that. However, if a valid
shebang line is found but the process specified by that line can not
be started, the default interpreter will not be started - the error
to create the specified child process will cause the launcher to display
an appropriate message and terminate with a specific exit code.
Virtual commands in shebang lines:
Virtual Commands are shebang lines which start with strings which would
be expected to work on Unix platforms - examples include
'/usr/bin/python', '/usr/bin/env python' and 'python'. Optionally, the
virtual command may be suffixed with a version qualifier (see below),
such as '/usr/bin/python2' or '/usr/bin/python3.2'. The command executed
is based on the rules described in Python Version Qualifiers below.
Customized Commands:
The launcher will support the ability to define "Customized Commands" in a
Windows .ini file (ie, a file which can be parsed by the Windows function
GetPrivateProfileString). A section called '[commands]' can be created
with key names defining the virtual command and the value specifying the
actual command-line to be used for this virtual command.
For example, if an INI file has the contents:
[commands]
vpython=c:\bin\vpython.exe -foo
Then a shebang line of '#! vpython' in a script named 'doit.py' will
result in the launcher using the command-line 'c:\bin\vpython.exe -foo
doit.py'
The precise details about the names, locations and search order of the
.ini files is in the launcher documentation [4]
Python Version Qualifiers
Some of the features described allow an optional Python version qualifier
to be used.
A version qualifier starts with a major version number and can optionally
be followed by a period ('.') and a minor version specifier. If the minor
qualifier is specified, it may optionally be followed by "-32" to indicate
the 32bit implementation of that version be used. Note that no "-64"
qualifier is necessary as this is the default implementation (see below).
On 64bit Windows with both 32bit and 64bit implementations of the
same (major.minor) Python version installed, the 64bit version will
always be preferred. This will be true for both 32bit and 64bit
implementations of the launcher - a 32bit launcher will prefer to
execute a 64bit Python installation of the specified version if
available. This is so the behavior of the launcher can be predicted
knowing only what versions are installed on the PC and without
regard to the order in which they were installed (ie, without knowing
whether a 32 or 64bit version of Python and corresponding launcher was
installed last). As noted above, an optional "-32" suffix can be used
on a version specifier to change this behaviour.
The launcher will support various customization options to allow
fine-grained control over which specific Python version is chosen given
a partial or empty version qualifier - see the launcher documentation [4]
for details.
Command-line handling
Only the first command-line argument will be checked for a shebang line
and only if that argument does not start with a '-'.
If the only command-line argument is "-h" or "--help", the launcher will
print a small banner and command-line usage, then pass the argument to
the default Python. This will cause help for the launcher being printed
followed by help for Python itself. The output from the launcher will
clearly indicate the extended help information is coming from the
launcher and not Python.
As a concession to interactively launching Python, the launcher will
support the first command-line argument optionally being a dash ("-")
followed by a version qualifier, as described above, to nominate a
specific version be used. For example, while "py.exe" may locate and
launch the latest Python 2.x implementation installed, a command-line such
as "py.exe -3" could specify the latest Python 3.x implementation be
launched, while "py.exe -2.6-32" could specify a 32bit implementation
Python 2.6 be located and launched. If a Python 2.x implementation is
desired to be launched with the -3 flag, the command-line would need to be
similar to "py.exe -2 -3" (or the specific version of Python could
obviously be launched manually without use of this launcher.) Note that
this feature can not be used with shebang processing as the file scanned
for a shebang line and this argument must both be the first argument and
therefore are mutually exclusive.
All other arguments will be passed untouched to the child Python process.
Process Launching
Ideally, the launcher process would execute Python directly inside
the same process, primarily so the parent of the launcher process could
terminate the launcher and have the Python interpreter terminate. If the
launcher executes Python as a sub-process and the parent of the launcher
terminates the launcher, the Python process will be unaffected.
However, there are a number of practical problems associated with this
approach. Windows does not support the execv* family of Unix functions,
so this could only be done by the launcher dynamically loading the Python
DLL, but this would have a number of side-effects. The most serious
side effect of this is that the value of sys.executable would refer to the
launcher instead of the Python implementation. Many Python scripts use the
value of sys.executable to launch child processes, and these scripts may
fail to work as expected if the launcher is used. Consider a "parent"
script with a shebang line of '#! /usr/bin/python3' which attempts to
launch a child script (with no shebang) via sys.executable - currently the
child is launched using the exact same version running the parent script.
If sys.executable referred to the launcher the child would be likely
executed using a Python 2.x version and would be likely to fail with a
SyntaxError.
Another hurdle is the support for alternative Python implementations
using the "customized commands" feature described above, where loading
the command dynamically into a running executable is not possible.
The final hurdle is the rules above regarding 64bit and 32bit programs -
a 32bit launcher would be unable to load the 64bit version of Python and
vice-versa.
Given these considerations, the launcher will execute it's command in a
child process, remaining alive while the child process is executing, then
terminate with the same exit code as returned by the child. To address
concerns regarding the termination of the launcher not killing the child,
the Win32 Job API will be used to arrange so that the child process is
automatically killed when the parent is terminated (although children of
that child process will continue as is the case now.) As this Windows API
is available in Windows XP and later, this launcher will not work on
Windows 2000 or earlier.
References
[1] http://linux.die.net/man/2/execve
[2] http://www.python.org/dev/peps/pep-0394/
[3] https://bitbucket.org/vinay.sajip/pylauncher
[4] https://bitbucket.org/vinay.sajip/pylauncher/src/tip/Docs/launcher.rst
Copyright
This document has been placed in the public domain.
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
