Re: [Python-Dev] Python initialization and embedded Python

[Eric Snow]

On Mon, Nov 20, 2017 at 3:03 PM, Victor Stinner
 wrote:
> To statically initialize PyMemAllocatorEx fields, you need to export a
> lot of allocator functions. I would prefer to not do that.
>
> [snip]
>
> The rules to choose the allocator to each domain are also complex
> depending if pymalloc is enabled, debug hooks are enabled by default,
> etc. The memory allocator is also linked to _PyMem_Debug which is not
> currently in Include/internals/ but Objects/obmalloc.c.

I'm not suggesting supporting the full machinery.  Rather, as my PR
demonstrates, we can statically initialize the minimum needed to
support pre-init use of PyMem_RawMalloc() and PyMem_RawFree().  The
allocators will be fully initialized once the runtime is initialized
(i.e. once Py_Initialize() is called), just as they are now.

FWIW, I'm not sure that's the best approach.  See my notes in
https://bugs.python.org/issue32096.

>
> I understand that moving global variables to _PyRuntime helps to
> clarify how these variables are initialized and then finalized, but
> memory allocators are a complex corner case.

Agreed.  I spent a large portion of my time getting the allocators
right when working on the original _PyRuntime patch.  It's tricky
code.

-eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 563: Postponed Evaluation of Annotations (Draft 3)

[Lukasz Langa]

Based on the feedback I gather in early November,
I'm publishing the third draft for consideration on python-dev.
I hope you like it!

A nicely formatted rendering is available here:
https://www.python.org/dev/peps/pep-0563/

The full list of changes between this version and the previous draft
can be found here:
https://github.com/ambv/static-annotations/compare/python-dev1...python-dev2

- Ł



PEP: 563
Title: Postponed Evaluation of Annotations
Version: $Revision$
Last-Modified: $Date$
Author: Łukasz Langa 
Discussions-To: Python-Dev 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 8-Sep-2017
Python-Version: 3.7
Post-History: 1-Nov-2017, 21-Nov-2017
Resolution:


Abstract


PEP 3107 introduced syntax for function annotations, but the semantics
were deliberately left undefined.  PEP 484 introduced a standard meaning
to annotations: type hints.  PEP 526 defined variable annotations,
explicitly tying them with the type hinting use case.

This PEP proposes changing function annotations and variable annotations
so that they are no longer evaluated at function definition time.
Instead, they are preserved in ``__annotations__`` in string form.

This change is going to be introduced gradually, starting with a new
``__future__`` import in Python 3.7.


Rationale and Goals
===

PEP 3107 added support for arbitrary annotations on parts of a function
definition.  Just like default values, annotations are evaluated at
function definition time.  This creates a number of issues for the type
hinting use case:

* forward references: when a type hint contains names that have not been
  defined yet, that definition needs to be expressed as a string
  literal;

* type hints are executed at module import time, which is not
  computationally free.

Postponing the evaluation of annotations solves both problems.

Non-goals
-

Just like in PEP 484 and PEP 526, it should be emphasized that **Python
will remain a dynamically typed language, and the authors have no desire
to ever make type hints mandatory, even by convention.**

This PEP is meant to solve the problem of forward references in type
annotations.  There are still cases outside of annotations where
forward references will require usage of string literals.  Those are
listed in a later section of this document.

Annotations without forced evaluation enable opportunities to improve
the syntax of type hints.  This idea will require its own separate PEP
and is not discussed further in this document.

Non-typing usage of annotations
---

While annotations are still available for arbitrary use besides type
checking, it is worth mentioning that the design of this PEP, as well
as its precursors (PEP 484 and PEP 526), is predominantly motivated by
the type hinting use case.

In Python 3.8 PEP 484 will graduate from provisional status.  Other
enhancements to the Python programming language like PEP 544, PEP 557,
or PEP 560, are already being built on this basis as they depend on
type annotations and the ``typing`` module as defined by PEP 484.
In fact, the reason PEP 484 is staying provisional in Python 3.7 is to
enable rapid evolution for another release cycle that some of the
aforementioned enhancements require.

With this in mind, uses for annotations incompatible with the
aforementioned PEPs should be considered deprecated.


Implementation
==

In Python 4.0, function and variable annotations will no longer be
evaluated at definition time.  Instead, a string form will be preserved
in the respective ``__annotations__`` dictionary.  Static type checkers
will see no difference in behavior, whereas tools using annotations at
runtime will have to perform postponed evaluation.

The string form is obtained from the AST during the compilation step,
which means that the string form might not preserve the exact formatting
of the source.  Note: if an annotation was a string literal already, it
will still be wrapped in a string.

Annotations need to be syntactically valid Python expressions, also when
passed as literal strings (i.e. ``compile(literal, '', 'eval')``).
Annotations can only use names present in the module scope as postponed
evaluation using local names is not reliable (with the sole exception of
class-level names resolved by ``typing.get_type_hints()``).

Note that as per PEP 526, local variable annotations are not evaluated
at all since they are not accessible outside of the function's closure.

Enabling the future behavior in Python 3.7
--

The functionality described above can be enabled starting from Python
3.7 using the following special import::

from __future__ import annotations

A reference implementation of this functionality is available
`on GitHub `_.


Resolving Type Hints at Runtime
===

To resolve an annotation at runtime from its string for

[Python-Dev] Script bootstrapping executables on Windows

[Thomas Mansencal]

Hi,

This is a Windows specific question, to give a bit of context I'm working
in a studio where depending on the project we use different Python
interpreters installed in different locations, e.g. Python 2.7.13, Python
2.7.14, Python 3.6. We set PATH, PYTHONHOME and PYTHONPATH accordingly
depending the interpreter in use.

Our Python packages are living atomically on the network and are added to
the environment on a per project basis by extending PYTHONPATH. This is in
contrast to using a monolithic virtual environment built with virtualenv or
conda. Assuming it is compatible, a Python package might be used with any
of the 3 aforementioned interpreters, e.g. yapf (a code formatter).

Now, on Windows, if you for example *pip install yapf*, a yapf.exe
boostrapping executable and a yapf-script.py file are being generated. The
boostrapping executable seems to look for the yapf-script.py file and
launch it using the absolute hardcoded interpreter path of the
yapf-script.py shebang.

Given the above we run into issues if for example yapf was deployed using
Python 2.7.13 but the Python 2.7.14 interpreter is being used in the
environment instead. We get a "failed to create process." error in that
case.

What we would like to do is not being tied to the absolute interpreter path
but have it defined with a variable or just use #!python. I have tried to
search for the above error in cpython source code and the installation
directory without luck. I would like to know what module/package is
responsible for generating the boostrapping executables to understand how
it works and see if we can doctor it for our usage.

Bests,

Thomas
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com