Re: [Python-Dev] PEP 467: last round (?)

[Nick Coghlan]

On 4 September 2016 at 08:11, Random832  wrote:
> On Sat, Sep 3, 2016, at 18:06, Koos Zevenhoven wrote:
>> I guess one reason I don't like bchr (nor chrb, really) is that they
>> look just like a random sequence of letters in builtins, but not
>> recognizable the way asdf would be.
>>
>> I guess I have one last pair of suggestions for the name of this
>> function: bytes.chr or bytes.char.

The PEP started out with a classmethod, and that proved problematic
due to length and the expectation of API symmetry with bytearray. A
new builtin paralleling chr avoids both of those problems.

> What about byte? Like, not bytes.byte, just builtins.byte.

The main problem with "byte" as a name is that "bytes" is *not* an
iterable of these - it's an iterable of ints. That concern doesn't
arise with chr/str as they're both abbreviated singular nouns rather
than one being the plural form of the other (it also doesn't hurt that
str actually is an iterable of chr results).

If we wanted a meaningful noun (other than byte) for the bchr concept,
then the alternative term that most readily comes to mind for me is
"octet", but I don't know how intuitive or memorable that would be for
folks without an embedded systems or serial communications background
(especially given that we already have 'oct', which does something
completely different).

That said, the PEP does propose "getbyte()" and "iterbytes()" for
bytes-oriented indexing and iteration, so there's a reasonable
consistency argument in favour of also proposing "byte" as the builtin
factory function:

* data.getbyte(idx) would be a more efficient alternative to byte(data[idx])
* data.iterbytes() would be a more efficient alternative to map(byte, data)

With bchr, those mappings aren't as clear (plus there's a potentially
unwanted "text" connotation arising from the use of the "chr"
abbreviation).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Mark Shannon]

On 02/09/16 19:04, Steven D'Aprano wrote:

On Fri, Sep 02, 2016 at 08:10:24PM +0300, Koos Zevenhoven wrote:


A good checker should be able to infer that x is a union type at the
point that it's passed to spam, even without the type annotation. For
example:

def eggs(cond:bool):
if cond:
x = 1
else:
x = 1.5
spam(x)   # a good type checker infers that x is of type Union[int, float]


Oh I really hope not. I wouldn't call that a *good* type checker. I
would call that a type checker that is overly permissive.
Why would that be overly permissive? It infers the most precise type 
possible.




Maybe you think that it's okay because ints and floats are somewhat
compatible. But suppose I wrote:

if cond:
x = HTTPServer(*args)
else:
x = 1.5

Would you want the checker to infer Union[HTTPServer, float]? I
wouldn't. I would want the checker to complain that the two branches of
the `if` result in different types for x. If I really mean it, then I
can give a type-hint.
Yes, the checker would infer that the type of x (strictly, all uses of x 
that are  defined by these definitions) is Union[HTTPServer, float].


You example is incomplete, what do you do with x?
If you pass x to a function that takes  Union[HTTPServer, float] then 
there is no error.

If you pass it to a function that takes a number then you get an error:
"Cannot use HTTPServer (from line 2) as Number (line ...)"
as one would expect.

When it comes to checkers, people hate false positives. Flagging correct 
code as erroneous because it is bad 'style' is really unpopular.




In any case, this PEP isn't about specifying when to declare variable
types, it is for picking syntax. Do you have a better idea for variable
syntax?


No. I think that defining the type of variables, rather than expressions 
is a bad idea.


Cheers,
Mark.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Koos Zevenhoven]

On Sun, Sep 4, 2016 at 12:51 PM, Nick Coghlan  wrote:
> On 4 September 2016 at 08:11, Random832  wrote:
>> On Sat, Sep 3, 2016, at 18:06, Koos Zevenhoven wrote:
>>> I guess one reason I don't like bchr (nor chrb, really) is that they
>>> look just like a random sequence of letters in builtins, but not
>>> recognizable the way asdf would be.
>>>
>>> I guess I have one last pair of suggestions for the name of this
>>> function: bytes.chr or bytes.char.
>
> The PEP started out with a classmethod, and that proved problematic
> due to length and the expectation of API symmetry with bytearray. A
> new builtin paralleling chr avoids both of those problems.
>
>> What about byte? Like, not bytes.byte, just builtins.byte.
>
> The main problem with "byte" as a name is that "bytes" is *not* an
> iterable of these - it's an iterable of ints. That concern doesn't
> arise with chr/str as they're both abbreviated singular nouns rather
> than one being the plural form of the other (it also doesn't hurt that
> str actually is an iterable of chr results).
>

Since you agree with me about this...

[...]
>
> That said, the PEP does propose "getbyte()" and "iterbytes()" for
> bytes-oriented indexing and iteration, so there's a reasonable
> consistency argument in favour of also proposing "byte" as the builtin
> factory function:
>
> * data.getbyte(idx) would be a more efficient alternative to byte(data[idx])
> * data.iterbytes() would be a more efficient alternative to map(byte, data)
>

.. I don't understand the argument for having 'byte' in these names.
They should have 'char' or 'chr' in them for exacly the same reason
that the proposed builtin should have 'chr' in it instead of 'byte'.
If 'bytes' is an iterable of ints, then get_byte should probably
return an int

I'm sorry, but this argument comes across as "were're proposing the
wrong thing here, so for consistency, we might want to do the wrong
thing in this other part too".

And didn't someone recently propose deprecating iterability of str
(not indexing, or slicing, just iterability)? Then str would also need
a way to provide an iterable or sequence view of the characters. For
consistency, the str functionality would probably need to mimic the
approach in bytes. IOW, this PEP may in fact ultimately dictate how to
get a iterable/sequence from a str object.

-- Koos


> With bchr, those mappings aren't as clear (plus there's a potentially
> unwanted "text" connotation arising from the use of the "chr"
> abbreviation).
>

Which mappings?

> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/k7hoven%40gmail.com


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Mark Shannon]

On 02/09/16 19:19, Steven D'Aprano wrote:

On Fri, Sep 02, 2016 at 10:47:41AM -0700, Steve Dower wrote:

"I'm not seeing what distinction you think you are making here. What
distinction do you see between:

x: int = func(value)

and

x = func(value)  #type: int"

Not sure whether I agree with Mark on this particular point, but the
difference I see here is that the first describes what types x may
ever contain, while the latter describes what type of being assigned
to x right here. So one is a variable annotation while the other is an
expression annotation.


Ultimately Python is a dynamically typed language, and that's not
changing. This means types are fundamentally associated with *values*,
not *variables* (names). But in practice, you can go a long way by
pretending that it is the variable that carries the type. That's the
point of the static type checker: if you see that x holds an int here,
then assume (unless told differently) that x should always be an int.
Because in practice, most exceptions to that are due to bugs, or at
least sloppy code.

Of course, it is up to the type checker to decide how strict it wants to
be, whether to treat violations as a warning or a error, whether to
offer the user a flag to set the behaviour, etc. None of this is
relevant to the PEP. The PEP only specifies the syntax, leaving
enforcement or non-enforcement to the checker, and it says:

PEP 484 introduced type hints, a.k.a. type annotations. While its
main focus was function annotations, it also introduced the notion
of type comments to annotate VARIABLES [emphasis added]


If I recall, Guido and I agreed to differ on that point. We still do, it 
seems. We did manage to agree on the syntax though.




not expressions. And:

This PEP aims at adding syntax to Python for annotating the types
of variables and attributes, instead of expressing them through
comments

which to me obviously implies that the two ways (type comment, and
variable type hint) are intended to be absolutely identical in
semantics, at least as far as the type-checker is concerned.


The key difference is in placement.
PEP 484 style
variable = value # annotation

Which reads to me as if the annotation refers to the value.
PEP 526
variable: annotation = value

Which reads very much as if the annotation refers to the variable.
That is a change in terms of semantics and a change for the worse, in 
terms of expressibility.


Cheers,
Mark.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Chris Angelico]

On Sun, Sep 4, 2016 at 8:52 PM, Mark Shannon  wrote:
> The key difference is in placement.
> PEP 484 style
> variable = value # annotation
>
> Which reads to me as if the annotation refers to the value.
> PEP 526
> variable: annotation = value
>
> Which reads very much as if the annotation refers to the variable.
> That is a change in terms of semantics and a change for the worse, in terms
> of expressibility.

So what you have is actually a change in *implication* (since the PEP
doesn't stipulate semantics); and the old way (the comment) implies
something contrary to the semantics of at least one of the type
checkers that uses it (MyPy). Are there any current type checkers that
actually do associate that with the value? That is, to have "variable
= func() # str" indicate the same type check as "def func() -> str"?
If not, this is a strong argument in favour of the PEP, since it would
synchronize the syntax with the current best-of-breed checkers.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patch reviews

[Christian Heimes]

On 2016-09-01 23:15, Victor Stinner wrote:
> 2016-08-31 22:31 GMT+02:00 Christian Heimes :
>> https://bugs.python.org/issue27744
>> Add AF_ALG (Linux Kernel crypto) to socket module
> 
> This patch adds a new socket.sendmsg_afalg() method on Linux.
> 
> "afalg" comes from AF_ALG which means "Address Family Algorithm". It's
> documented as "af_alg: User-space algorithm interface" in
> crypto/af_alg.c.
> 
> IHMO the method should be just "sendmsg_alg()", beacuse "afalg" is
> redundant. The AF_ prefix is only used to workaround a C limitation:
> there is no namespace in the language, all symbols are in one single
> giant namespace.
> 
> I don't expect that a platform will add a new sendmsg_alg() C
> function. If it's the case, we will see how to handle the name
> conflict ;-)

Hi,

afalg is pretty much the standard name for Linux Kernel crypto. For
example OpenSSL 1.1.0 introduced a crypto engine to offload AES. The
engine is called 'afalg' [1]. Other documentations refer to the
interface as either afalg or AF_ALG, too. I prefer to use an established
name for the method.

Christian

[1]
https://github.com/openssl/openssl/blob/master/engines/afalg/e_afalg.c#L88

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Mark Shannon]

On 02/09/16 20:33, Guido van Rossum wrote:

On Fri, Sep 2, 2016 at 10:47 AM, Steve Dower  wrote:

"I'm not seeing what distinction you think you are making here. What
distinction do you see between:

x: int = func(value)

and

x = func(value)  # type: int"

Not sure whether I agree with Mark on this particular point, but the
difference I see here is that the first describes what types x may ever
contain, while the latter describes what type of being assigned to x right
here. So one is a variable annotation while the other is an expression
annotation.


But that's not what type comments mean! They don't annotate the
expression. They annotate the variable. The text in PEP 484 that
introduces them is clear about this (it never mentions expressions,
only variables).


In PEP 484, the section on type comments says:
(Quoting verbatim)
"""
No first-class syntax support for explicitly marking variables as being 
of a specific type is added by this PEP. To help with type inference in 
complex cases, a comment of the following format may be used...

"""

Some mentions of the type of a variable are made in other places in the 
PEP, but those were all added *after* I had approved the PEP.


In other words PEP 484 specifically states that annotations are to help 
with type inference. As defined in PEP 526, I think that type 
annotations become a hindrance to type inference.


Cheers,
Mark.




Personally, I prefer expression annotations over variable annotations, as
there are many other languages I'd prefer if variable have fixed types (e.g.
C++, where I actually enjoy doing horrible things with implicit casting ;)
).

Variable annotations appear to be inherently restrictive, so either we need
serious clarification as to why they are not, or they actually are and we
ought to be more sure that it's the direction we want the language to go.


At runtime the variable annotations are ignored. And a type checker
will only ask for them when it cannot infer the type. So I think we'll
be fine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Ivan Levkivskyi]

On 4 September 2016 at 12:52, Mark Shannon  wrote:

> The key difference is in placement.
> PEP 484 style
> variable = value # annotation
>
> Which reads to me as if the annotation refers to the value.
> PEP 526
> variable: annotation = value
>
> Which reads very much as if the annotation refers to the variable.
> That is a change in terms of semantics and a change for the worse, in
> terms of expressibility.


I still think it is better to leave the decision to type checkers.
The proposed syntax allows two forms:

variable: annotation = value

and

variable: annotation

The first form still could be interpreted by type checkers
as annotation for value (a cast to more precise type):

variable = cast(annotation, value) # visually also looks similar

and PEP says that annotations "are intended to help with type inference in
complex cases".
Such interpretation could be useful for function local variables
(the implementation is also optimised for such use case).
While the second form (without value)
indeed looks like annotation of variable, and is equivalent to:

__annotations__['variable'] = annotation

This form could be useful for annotating instance/class variables.
Or just for documentation (I really like this).

In addition, expression annotations are allowed

expression: annotation [= value]

This form is not used by 3rd party tools (as far as I know) but one of the
possible use cases
could be "check-points" for values:

somedict[somefunc(somevar)]: annotation # type checker could flag this if
something went wrong.

Finally, I would like to reiterate, both interpretations (annotating value
vs annotating variable)
are possible and we (OK at least me, but it looks like Guido also agree)
don't want to force 3rd party tools to use only one of those.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Mark Shannon]

On 04/09/16 11:57, Chris Angelico wrote:

On Sun, Sep 4, 2016 at 8:52 PM, Mark Shannon  wrote:

The key difference is in placement.
PEP 484 style
variable = value # annotation

Which reads to me as if the annotation refers to the value.
PEP 526
variable: annotation = value

Which reads very much as if the annotation refers to the variable.
That is a change in terms of semantics and a change for the worse, in terms
of expressibility.


So what you have is actually a change in *implication* (since the PEP
doesn't stipulate semantics); and the old way (the comment) implies
something contrary to the semantics of at least one of the type
checkers that uses it (MyPy).
Do we really want to make a major, irrevocable change to the language 
just because MyPy does something? MyPy is very far from complete (it 
doesn't even support Optional types yet).


Are there any current type checkers that

actually do associate that with the value? That is, to have "variable
= func() # str" indicate the same type check as "def func() -> str"?
If not, this is a strong argument in favour of the PEP, since it would
synchronize the syntax with the current best-of-breed checkers.
I believe pytype uses value, rather the variable, tracking. It is thus 
more precise. Of course, it is more of an inferencer than a checker.
We (semmle.com) do precise value tracking to infer a lot of "type" 
errors in unannotated code (as the vast majority of Python code is).


It would be a real shame if PEP 526 mandates against checkers doing as 
good as job as possible. Forcing all uses of a variable to have the same 
type is a major and, IMO crippling, limitation.


E.g.
def foo(x:Optional[int])->int:
if x is None:
   return -1
return x + 1

If the type of the *variable* 'x' is Optional[int] then 'return x + 1' 
doesn't type check. If the type of the *parameter* 'x' is Optional[int] 
then a checker can readily verify the above code.


I want a checker to check my code and, with minimal annotations, give me 
confidence that my code is correct.



Cheers,
Mark.



ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/mark%40hotpy.org


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Steven D'Aprano]

On Sun, Sep 04, 2016 at 12:31:26PM +0100, Mark Shannon wrote:

> In other words PEP 484 specifically states that annotations are to help 
> with type inference. As defined in PEP 526, I think that type 
> annotations become a hindrance to type inference.

I'm pretty sure that they don't.

Have you used any languages with type inference? Any type-checkers? If 
so, can you give some actual concrete examples of how annotating a 
variable hinders type inference? It sounds like you are spreading FUD at 
the moment.

The whole point of type annotations is that you use them to deliberately 
over-ride what the checker would infer (if it infers the wrong thing, or 
cannot infer anything). I cannot see how you conclude from this that 
type annotations will be a hindrance to type inference.

If you don't want to declare the type of a variable, simply DON'T 
declare the type, and let the checker infer whatever it can (which may 
be nothing, or may be the wrong type).



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Ivan Levkivskyi]

On 4 September 2016 at 13:30, Mark Shannon  wrote:

> It would be a real shame if PEP 526 mandates against checkers doing as
> good as job as possible. Forcing all uses of a variable to have the same
> type is a major and, IMO crippling, limitation.
>
> E.g.
> def foo(x:Optional[int])->int:
> if x is None:
>return -1
> return x + 1
>
> If the type of the *variable* 'x' is Optional[int] then 'return x + 1'
> doesn't type check. If the type of the *parameter* 'x' is Optional[int]
> then a checker can readily verify the above code.
>

Mark,

First, in addition to the quote from my previous e-mail, I would like to
show another quote from PEP 526
"This PEP does not require type checkers to change their type checking
rules. It merely provides a more readable syntax to replace type comments"

Second, almost exactly your example has been added to PEP 484:

class Reason(Enum):
timeout = 1
error = 2

def process(response: Union[str, Reason] = '') -> str:
if response is Reason.timeout:
return 'TIMEOUT'
elif response is Reason.error:
return 'ERROR'
else:
# response can be only str, all other possible values exhausted
return 'PROCESSED: ' + response

I think mypy either already supports this or will support very soon (and
the same for Optional)

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Koos Zevenhoven]

On Sun, Sep 4, 2016 at 1:52 PM, Mark Shannon  wrote:
[...]
>
> The key difference is in placement.
> PEP 484 style
> variable = value # annotation
>
> Which reads to me as if the annotation refers to the value.
> PEP 526
> variable: annotation = value
>
> Which reads very much as if the annotation refers to the variable.
> That is a change in terms of semantics and a change for the worse, in terms
> of expressibility.
>

You have probably noticed this already, but in the semantics which I
have now explained more precisely on python-ideas

https://mail.python.org/pipermail/python-ideas/2016-September/042076.html

an annotation like

variable: annotation = value

is a little closer to an expression annotation. I.e. it does not say
that 'variable' should *always* have the type given by 'annotation'.

-- Koos

>
> Cheers,
> Mark.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/k7hoven%40gmail.com



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Steven D'Aprano]

On Sun, Sep 04, 2016 at 12:30:18PM +0100, Mark Shannon wrote:

> It would be a real shame if PEP 526 mandates against checkers doing as 
> good as job as possible. Forcing all uses of a variable to have the same 
> type is a major and, IMO crippling, limitation.

This is approaching FUD.

Guido has already stated that the section of the PEP which implied that 
*any* change of type of a variable would be a warning (not an error) is 
too strong:

https://mail.python.org/pipermail/python-dev/2016-September/146064.html

and indeed the relevant section of the PEP has already been changed:

Duplicate type annotations will be ignored. However, static type 
checkers may issue a warning for annotations of the same variable 
by a different type:

a: int
a: str  # Static type checker may or may not warn about this.


This PEP does not mandate any behaviour for type-checkers. It describes 
the syntax for type annotations in Python code. What type-checkers do 
with that information is up to them.



> E.g.
> def foo(x:Optional[int])->int:
> if x is None:
>return -1
> return x + 1
> 
> If the type of the *variable* 'x' is Optional[int] then 'return x + 1' 
> doesn't type check.

That makes no sense. Why wouldn't it type check?

It may be that some simple-minded type-checkers are incapable of 
checking that code because it is too complex. If so, that's a limitation 
of that specific checker, not of type-checkers in general. MyPy already 
can type-check that code. See below.


> If the type of the *parameter* 'x' is Optional[int] 
> then a checker can readily verify the above code.

This makes even less sense, since the parameter "x" is, of course, 
precisely the same as the variable "x".

Here's MyPy in action, successfully checking code that you state can't 
be checked:

[steve@ando ~]$ cat test.py
from typing import Optional

def foo(x:Optional[int])->int:
if x is None:
return -1
return x + 1

def bar(x:Optional[int])->int:
y = x  # the type of y must be inferred
if y is None:
return y + 1
return len(y)

[steve@ando ~]$ mypy --strict-optional test.py
test.py: note: In function "bar":
test.py:11: error: Unsupported operand types for + (None and "int")
test.py:12: error: Argument 1 to "len" has incompatible type "int"; expected 
"Sized"


foo passes the type check; bar fails.


> I want a checker to check my code and, with minimal annotations, give me 
> confidence that my code is correct.

Don't we all.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Steven D'Aprano]

Referring to the alternative syntax forms:

# Proposed
x: int = func(value)

# Already accepted
x = func(value)  #type: int


On Sun, Sep 04, 2016 at 11:52:24AM +0100, Mark Shannon wrote:

> The key difference is in placement.
> PEP 484 style
> variable = value # annotation
> 
> Which reads to me as if the annotation refers to the value.

Both Guido and the PEP have stated that it doesn't refer to the value, 
but to the variable.

But what does it even mean to say that it refers to the value in the 
context of *static type-checking*? I know what it means in the context 
of dynamic type-checking, but I don't see how that has any relevance to 
a static checker.


I have seen a number of people commenting that the comment annotation 
"applies to the expression", but I don't understand what this is 
supposed to mean. How is that different from applying it to the 
variable? (That's not a rhetorical question.) Suppose I write this:

mylist = []
x = False or None or (mylist + [1])  #type: List[int]
pass  # stand-in for arbitrary code
x.append("definitely not an int")

Should the type-checker flag the call to x.append as an error? I hope we 
all agree that it should.

But it can only do that if it knows the type of the variable `x`. This 
is a *static* type-checker, it doesn't know what value x *actually* has 
at run-time because it isn't running at run-time. As far as the static 
checker is concerned, it can only flag that append as an error if it 
knows that `x` must be a list of ints.

If you distinguish the two cases:

"the expression `False or None or (mylist + [1])` is List[int]"

versus:

"the variable `x` is List[int]"

I don't even see what the first case could possible mean. But whatever 
it means, if it is different from the second case, then the type-checker 
is pretty limited in what it can do.



> PEP 526
> variable: annotation = value
> 
> Which reads very much as if the annotation refers to the variable.

Since the PEP makes it clear that the two forms are to be treated the 
same, I think that whatever difference you think they have is not 
relevant. They are *defined* to mean the same thing.




-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Ivan Levkivskyi]

On 4 September 2016 at 15:07, Steven D'Aprano  wrote:

> > PEP 526
> > variable: annotation = value
> >
> > Which reads very much as if the annotation refers to the variable.
>
> Since the PEP makes it clear that the two forms are to be treated the
> same, I think that whatever difference you think they have is not
> relevant. They are *defined* to mean the same thing.
>

Steve,
This has been discussed in the python/typing tracker. When you say "mean"
you are talking about semantics, but:

Me:
"""
The title of the PEP contains "Syntax", not "Semantics" because we don't
want to impose any new type semantics (apart from addition of ClassVar)
"""

Guido:
"""
I have nothing to add to what @ilevkivskyi 
said about the semantics of redefinition -- that is to be worked out
between type checkers.
(Much like PEP 484 doesn't specify how type checkers should behave -- while
it gives examples of suggested behavior, those examples are not normative,
and there are huge gray areas where the PEP doesn't give any guidance. It's
the same for PEP 526.)
"""

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Koos Zevenhoven]

On Sun, Sep 4, 2016 at 3:43 PM, Steven D'Aprano  wrote:
[...]
> [steve@ando ~]$ cat test.py
> from typing import Optional
>
> def foo(x:Optional[int])->int:
> if x is None:
> return -1
> return x + 1
>
> def bar(x:Optional[int])->int:
> y = x  # the type of y must be inferred
> if y is None:
> return y + 1
> return len(y)
>
> [steve@ando ~]$ mypy --strict-optional test.py
> test.py: note: In function "bar":
> test.py:11: error: Unsupported operand types for + (None and "int")
> test.py:12: error: Argument 1 to "len" has incompatible type "int"; expected 
> "Sized"
>
>
> foo passes the type check; bar fails.
>

That's great. While mypy has nice features, these examples have little
to do with PEP 526 as they don't have variable annotations, not even
using comments.

For some reason, pip install --upgrade mypy fails for me at the
moment, but at least mypy version 0.4.1 does not allow this:

from typing import Callable

def foo(cond: bool, bar : Callable, baz : Callable) -> float:
if cond:
x = bar() # type: int
else:
x = baz() # type: float
return x / 2

and complains that

test.py:7: error: Name 'x' already defined". Maybe someone can confirm
this with a newer version.

Here,

def foo(cond: bool) -> float:
if cond:
x = 1
else:
x = 1.5
return x / 2

you get a different error:

test.py:5: error: Incompatible types in assignment (expression has
type "float", variable has type "int")

Maybe someone can confirm this with a newer version, but IIUC this is
still the case.

>> I want a checker to check my code and, with minimal annotations, give me
>> confidence that my code is correct
>
> Don't we all.
>

I would add *with minimal restrictions on how the code is supposed to
be written* for type checking to work. It's not at all obvious that
everyone thinks that way. Hence, the "Semantics for type checking"
thread on python-ideas.

-- Koos

>
>
> --
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/k7hoven%40gmail.com



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Nick Coghlan]

On 4 September 2016 at 20:43, Koos Zevenhoven  wrote:
> On Sun, Sep 4, 2016 at 12:51 PM, Nick Coghlan  wrote:
>> That said, the PEP does propose "getbyte()" and "iterbytes()" for
>> bytes-oriented indexing and iteration, so there's a reasonable
>> consistency argument in favour of also proposing "byte" as the builtin
>> factory function:
>>
>> * data.getbyte(idx) would be a more efficient alternative to byte(data[idx])
>> * data.iterbytes() would be a more efficient alternative to map(byte, data)
>>
>
> .. I don't understand the argument for having 'byte' in these names.
> They should have 'char' or 'chr' in them for exacly the same reason
> that the proposed builtin should have 'chr' in it instead of 'byte'.
> If 'bytes' is an iterable of ints, then get_byte should probably
> return an int
>
> I'm sorry, but this argument comes across as "were're proposing the
> wrong thing here, so for consistency, we might want to do the wrong
> thing in this other part too".

There are two self-consistent sets of names:

bchr
bytes.getbchr, bytearray.getbchr
bytes.iterbchr, bytearray.iterbchr

byte
bytes.getbyte, bytearray.getbyte
bytes.iterbytes, bytearray.iterbytes

The former set emphasises the "stringiness" of this behaviour, by
aligning with the chr() builtin

The latter set emphasises that these APIs are still about working with
arbitrary binary data rather than text, with a Python "byte"
subsequently being a length 1 bytes object containing a single integer
between 0 and 255, rather than "What you get when you index or iterate
over a bytes instance".

Having noticed the discrepancy, my personal preference is to go with
the latter option (since it better fits the "executable pseudocode"
ideal and despite my reservations about "bytes objects contain int
objects rather than byte objects", that shouldn't be any more
confusing in the long run than explaining that str instances are
containers of length-1 str instances). The fact "byte" is much easier
to pronounce than bchr (bee-cher? bee-char?) also doesn't hurt.

However, I suspect we'll need to put both sets of names in front of
Guido and ask him to just pick whichever he prefers to get it resolved
one way or the other.

> And didn't someone recently propose deprecating iterability of str
> (not indexing, or slicing, just iterability)? Then str would also need
> a way to provide an iterable or sequence view of the characters. For
> consistency, the str functionality would probably need to mimic the
> approach in bytes. IOW, this PEP may in fact ultimately dictate how to
> get a iterable/sequence from a str object.

Strings are not going to become atomic objects, no matter how many
times people suggest it.

>> With bchr, those mappings aren't as clear (plus there's a potentially
>> unwanted "text" connotation arising from the use of the "chr"
>> abbreviation).
>
> Which mappings?

The mapping between the builtin name and the method names.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patch reviews

[Nick Coghlan]

On 4 September 2016 at 20:57, Christian Heimes  wrote:
> On 2016-09-01 23:15, Victor Stinner wrote:
>> 2016-08-31 22:31 GMT+02:00 Christian Heimes :
>>> https://bugs.python.org/issue27744
>>> Add AF_ALG (Linux Kernel crypto) to socket module
>>
>> This patch adds a new socket.sendmsg_afalg() method on Linux.
>>
>> "afalg" comes from AF_ALG which means "Address Family Algorithm". It's
>> documented as "af_alg: User-space algorithm interface" in
>> crypto/af_alg.c.
>>
>> IHMO the method should be just "sendmsg_alg()", beacuse "afalg" is
>> redundant. The AF_ prefix is only used to workaround a C limitation:
>> there is no namespace in the language, all symbols are in one single
>> giant namespace.
>>
>> I don't expect that a platform will add a new sendmsg_alg() C
>> function. If it's the case, we will see how to handle the name
>> conflict ;-)
>
> Hi,
>
> afalg is pretty much the standard name for Linux Kernel crypto. For
> example OpenSSL 1.1.0 introduced a crypto engine to offload AES. The
> engine is called 'afalg' [1]. Other documentations refer to the
> interface as either afalg or AF_ALG, too. I prefer to use an established
> name for the method.

Right, there's a confusability problem here not just at the API level,
but at a general terminology level: "alg" is just short for
"algorithm", which is way to generic to be meaningful. Once you put
the "af" qualifier in front though, it's clear you're not just talking
about algorithms in general, you're referring to AF_ALG in particular.

Putting the alternatives into Google and seeing which one gives more
relevant results also suggests afalg as the clear winner, since the
first link returned is the OpenSSL engine for it, while even
qualifying "alg" with "ssl" doesn't get you relevant information:

* https://www.google.com.au/search?q=afalg
* https://www.google.com.au/search?q=alg
* https://www.google.com.au/search?q=alg#q=alg+ssl

(that afalg repo unfortunately doesn't have a useful README, but at
least the first link is relevant, unlike the results for "alg" and
"alg ssl")

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Nick Coghlan]

On 4 September 2016 at 21:32, Ivan Levkivskyi  wrote:
> The first form still could be interpreted by type checkers
> as annotation for value (a cast to more precise type):
>
> variable = cast(annotation, value) # visually also looks similar

I think a function based spelling needs to be discussed further, as it
seems to me that at least some of the goals of the PEP could be met
with a suitable definition of "cast" and "declare", with no syntactic
changes to Python. Specifically, consider:

def cast(value, annotation):
return value

def declare(annotation):
return object()

The idea here is that "cast" would be used as a hint to type checkers
to annotate an *expression*, with the runtime semantic impact being
exactly nil - the value just gets passed through unmodified.

Annotated initialisations would then look like:

from typing import cast
primes = cast([], List[int])

class Starship:
stats = cast({}, ClassVar[Dict[str, int]])

This preserves the relative annotation order of current type hints,
where the item being annotated (parameter, function declaration,
assignment statement) is on the left, and the annotation is on the
right.

In cases where the typechecker is able to infer a type for the
expression, it may complain here when there's a mismatch between the
type inference and the explicit declaration, so these would also be a
form of type assertion.

That leaves the case of declarations, where the aim is to provide a
preemptive declaration that all assignments to a particular variable
will include an implicit casting of the RHS. That would look like:

from typing import declare

captain = declare(str)

Until it left the scope, or saw a new target declaration, a
typechecker would then interpret future assignments to "captain" as if
they had been written:

captain = cast(RHS, str)

With the above definition, this would have the runtime consequence of
setting "captain" to a unique object() instance until the first
assignment took place. Both that assignment, and the runtime overhead
of evaluating the declaration, can be avoided by moving the
declaration into otherwise dead code:

if 0: captain = declare(str)

Considering the goals and problems listed in the PEP, this would be
sufficient to address many of them:

* These are real expressions, and hence will be highlighted appropriately
* declare() allows annotations of undefined variables (sort of)
* These annotations will be in the AST, just as function arguments,
rather than as custom nodes
* In situations where normal comments and type comments are used
together, it is difficult to distinguish them

For the other goals, the function-based approach may not help:

* For conditional branches, it's only arguably an improvement

if 0: my_var = declare(Logger)
if some_value:
my_var = function()
else:
my_var = another_function()

* Readability for typeshed might improve for module level and class
level declarations, but functions would likely want the leading "if
0:" noise for performance reasons

* Since the code generator remains uninvolved, this still wouldn't
allow annotation access at runtime (unless typing implemented some
sys._getframe() hacks in declare() and cast())

However, exploring this possibility still seems like a good idea to
me, as it should allow many of the currently thorny semantic questions
to be resolved, and a future syntax-only PEP for 3.7+ can just be
about defining syntactic sugar for semantics that can (by then)
already be expressed via appropriate initialisers.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Koos Zevenhoven]

On Sun, Sep 4, 2016 at 7:43 PM, Nick Coghlan  wrote:
> On 4 September 2016 at 21:32, Ivan Levkivskyi  wrote:
>> The first form still could be interpreted by type checkers
>> as annotation for value (a cast to more precise type):
>>
>> variable = cast(annotation, value) # visually also looks similar
>
> I think a function based spelling needs to be discussed further, as it
> seems to me that at least some of the goals of the PEP could be met
> with a suitable definition of "cast" and "declare", with no syntactic
> changes to Python. Specifically, consider:
>
> def cast(value, annotation):
> return value
>

typing.cast already exists.

-- Koos
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Ivan Levkivskyi]

On 4 September 2016 at 18:43, Nick Coghlan  wrote:

> On 4 September 2016 at 21:32, Ivan Levkivskyi 
> wrote:
> > The first form still could be interpreted by type checkers
> > as annotation for value (a cast to more precise type):
> >
> > variable = cast(annotation, value) # visually also looks similar
>
> I think a function based spelling needs to be discussed further, as it
> seems to me that at least some of the goals of the PEP could be met
> with a suitable definition of "cast" and "declare", with no syntactic
> changes to Python. Specifically, consider:
>
> def cast(value, annotation):
> return value
>
> def declare(annotation):
> return object()
>

Nick, If I understand you correctly, this idea is very similar to Undefined.
It was proposed a year and half ago, when PEP 484 was discussed.

At that time it was abandoned, it reappeared during the discussion
of this PEP, but many people (including me) didn't like this,
so that we decided to put it in the list of rejected ideas to this PEP.

Some downsides of this approach:

* People will start to expect Undefined (or whatever is returned by
declare())
everywhere (as in Javascript) even if we prohibit this.

* Some runtime overhead is still present: annotation gets evaluated
at every call to cast, and many annotations involve creation of
class objects (especially generics) that are very costly.
Because of this overhead, such use of generics was prohibited in PEP 484:

x = Node[int]() # prohibited by PEP 484
x = Node() # type: Node[int] # this was allowed

* Readability will be probably even worse than with comments:
many types already have brackets and parens, so that two more form cast()
is not good. Plus some noise of the if 0: that you mentioned, plus
"cast" everywhere.

However, exploring this possibility still seems like a good idea to
> me, as it should allow many of the currently thorny semantic questions
> to be resolved, and a future syntax-only PEP for 3.7+ can just be
> about defining syntactic sugar for semantics that can (by then)
> already be expressed via appropriate initialisers.
>

I think that motivation of the PEP is exactly opposite, this is why it has
"Syntax" not "Semantics" in title. Also quoting Guido:

> But I'm not in a hurry for that -- I'm only hoping to get the basic
> syntax accepted by Python 3.6 beta 1 so that we can start using this
> in 5 years from now rather than 7 years from now.

I also think that semantics should be up to the type checkers.
Maybe it is not a perfect comparison, but prohibiting all type semantics
except one is like prohibiting all Python web frameworks except one.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Nick Coghlan]

On 4 September 2016 at 21:51, Steven D'Aprano  wrote:
> On Sun, Sep 04, 2016 at 12:31:26PM +0100, Mark Shannon wrote:
>
>> In other words PEP 484 specifically states that annotations are to help
>> with type inference. As defined in PEP 526, I think that type
>> annotations become a hindrance to type inference.
>
> I'm pretty sure that they don't.
>
> Have you used any languages with type inference? Any type-checkers? If
> so, can you give some actual concrete examples of how annotating a
> variable hinders type inference? It sounds like you are spreading FUD at
> the moment.

Steven, this kind of credential checking is uncalled for - Mark is
significantly more qualified than most of us to comment on this topic,
since he actually works on a shipping software quality analysis
product that does type inference on Python code (hence
https://semmle.com/semmle-analysis-now-includes-python/ ), and was
nominated as the BDFL-Delegate for PEP 484 because Guido trusted him
to critically review the proposal and keep any insurmountable problems
from getting through.

Getting accused of spreading FUD when a topic is just plain hard to
explain (due to the large expert/novice gap that needs to be bridged)
is one of the reasons python-dev and python-ideas can end up feeling
hostile to domain experts. We have the SIG system to help mitigate
that problem, but it's vastly preferable if such folks also feel their
expertise is welcomed on the main lists, rather than having it be
rejected as an inconvenient complication.

> The whole point of type annotations is that you use them to deliberately
> over-ride what the checker would infer (if it infers the wrong thing, or
> cannot infer anything). I cannot see how you conclude from this that
> type annotations will be a hindrance to type inference.

The problem arises for the "bare annotation" case, as that looks a
*lot* like traditional declarations in languages where initialisation
(which can specify a type) and assignment (which can't) are different
operations.

Consider this case:

if arg is not None:
x = list(arg)
# Type of "x" is inferred as List[Any] or something more specific here
if other_arg is not None:
# This is fine, we know "x" is a list at this point
x.extend(other_arg)
else:
x = None
# Type of "x" is inferred as type(None) here
# Type of "x" is inferred as Optional[List[Any]] from here on out

Now, consider that case with PEP 526:

x: Optional[List[Any]]
# Oops, this is the type of "x" *after* the if statement, not *during* it
if arg is not None:
x = list(arg)
if other_arg is not None:
# If we believe the type declaration here, this code will
(incorrectly) be flagged
# (as None has no "extend" method)
x.extend(other_arg)
else:
x = None

The "pre-declaration as documentation" proposal turns out to be
problematic in this case, as it misses the fact that different
branches of the if statement contribute different types to what
ultimately becomes a Union type for the rest of the code.

In order to cover the entire code block, we have to make the
pre-declaration match the state after the if statement, but then it's
overly broad for any *particular* branch.

So in this case, attempting to entirely defer specification of the
semantics creates a significant risk of type checkers written on the
assumption of C++ or Java style type declarations actively inhibiting
the dynamism of Python code, suggesting that the PEP would be well
advised to declare not only that the PEP 484 semantics are unchanged,
but also that a typechecker that flags the example above as unsafe is
wrong to do so.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Nick Coghlan]

On 5 September 2016 at 03:23, Ivan Levkivskyi  wrote:
> On 4 September 2016 at 18:43, Nick Coghlan  wrote:
>>
>> On 4 September 2016 at 21:32, Ivan Levkivskyi 
>> wrote:
>> > The first form still could be interpreted by type checkers
>> > as annotation for value (a cast to more precise type):
>> >
>> > variable = cast(annotation, value) # visually also looks similar
>>
>> I think a function based spelling needs to be discussed further, as it
>> seems to me that at least some of the goals of the PEP could be met
>> with a suitable definition of "cast" and "declare", with no syntactic
>> changes to Python. Specifically, consider:
>>
>> def cast(value, annotation):
>> return value
>>
>> def declare(annotation):
>> return object()
>
>
> Nick, If I understand you correctly, this idea is very similar to Undefined.
> It was proposed a year and half ago, when PEP 484 was discussed.

Not quite, as it deliberately doesn't create a singleton, precisely to
avoid the problems a new singleton creates - if you use declare() as
written, there's no way to a priori check for it at runtime (since
each call produces a new object), so you have to either get the
TypeError when you try to use it as whatever type it's supposed to be,
or else use a static checker to find cases where you try to use it
without initialising it properly first.

Folks can also put it behind an "if 0:" or "if typing.TYPE_CHECKING"
guard so it doesn't actually execute at runtime, and is only visible
to static analysis.

> At that time it was abandoned, it reappeared during the discussion
> of this PEP, but many people (including me) didn't like this,
> so that we decided to put it in the list of rejected ideas to this PEP.
>
> Some downsides of this approach:
>
> * People will start to expect Undefined (or whatever is returned by
> declare())
> everywhere (as in Javascript) even if we prohibit this.

Hence why I didn't use a singleton.

> * Some runtime overhead is still present: annotation gets evaluated
> at every call to cast, and many annotations involve creation of
> class objects (especially generics) that are very costly.

Hence the commentary about using an explicit guard to prevent
execution ("if 0:" in my post for the dead code elimination, although
"if typing.TYPE_CHECKING:" would be more self-explanatory).

> * Readability will be probably even worse than with comments:
> many types already have brackets and parens, so that two more form cast()
> is not good. Plus some noise of the if 0: that you mentioned, plus
> "cast" everywhere.

I mostly agree, but the PEP still needs to address the fact that it's
only a subset of the benefits that actually require new syntax, since
it's that subset which provides the rationale for rejecting the use of
a function based approach, while the rest provided the incentive to
start looking for a way to replace the type comments.

>> However, exploring this possibility still seems like a good idea to
>> me, as it should allow many of the currently thorny semantic questions
>> to be resolved, and a future syntax-only PEP for 3.7+ can just be
>> about defining syntactic sugar for semantics that can (by then)
>> already be expressed via appropriate initialisers.
>
> I think that motivation of the PEP is exactly opposite, this is why it has
> "Syntax" not "Semantics" in title. Also quoting Guido:
>
>> But I'm not in a hurry for that -- I'm only hoping to get the basic
>> syntax accepted by Python 3.6 beta 1 so that we can start using this
>> in 5 years from now rather than 7 years from now.
>
> I also think that semantics should be up to the type checkers.
> Maybe it is not a perfect comparison, but prohibiting all type semantics
> except one is like prohibiting all Python web frameworks except one.

It's the semantics that worry people though, and it's easy for folks
actively working on typecheckers to think it's just as easy for the
rest of us to make plausible assumptions about the kind of code that
well-behaved typecheckers are going to allow as it is for you. That's
not the case, which means folks get concerned, especially those
accustomed to instititutional environments where decisions about tool
use are still made by folks a long way removed from the day to day
experience of software development, rather than being delegated to the
engineering teams themselves.

I suspect you'll have an easier time of it on that front if you
include some examples of dynamically typed code that a well-behaved
type-checker *must* report as correct Python code, such as:

x: Optional[List[Any]]
# This is the type of "x" *after* the if statement, not *during* it
if arg is not None:
x = list(arg)
if other_arg is not None:
# A well-behaved typechecker should allow this due to
# the more specific initialisation in this particular branch
x.extend(other_arg)
else:
x = None

A typechecker could abide by that restriction by ignoring variabl

Re: [Python-Dev] Please reject or postpone PEP 526

[Ivan Levkivskyi]

On 4 September 2016 at 19:29, Nick Coghlan  wrote:

> So in this case, attempting to entirely defer specification of the
> semantics creates a significant risk of type checkers written on the
> assumption of C++ or Java style type declarations actively inhibiting
> the dynamism of Python code, suggesting that the PEP would be well
> advised to declare not only that the PEP 484 semantics are unchanged,
> but also that a typechecker that flags the example above as unsafe is
> wrong to do so.
>

I don't think that a dedicated syntax will increase the
risk more than the existing type comment syntax. Moreover,
mainstream type checkers (mypy, pytype, etc) are far
from C++ or Java, and as far as I know they are not going
to change semantics.

As I understand, the main point of Mark is that such syntax suggests
visually a variable annotation, more than a value annotation.
However, I think that the current behavior of type checkers will
have more influence on perception of people rather than a visual
appearance of annotation.

Anyway, I think it is worth adding an explicit statement to the PEP
that both interpretations are possible (maybe even add that value
semantics is inherent to Python). But I don't think that we should
*prohibit* something in the PEP.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Ivan Levkivskyi]

On 4 September 2016 at 19:59, Nick Coghlan  wrote:

Nick,

Thank you for good suggestions.


> I mostly agree, but the PEP still needs to address the fact that it's
> only a subset of the benefits that actually require new syntax, since
> it's that subset which provides the rationale for rejecting the use of
> a function based approach, while the rest provided the incentive to
> start looking for a way to replace the type comments.
>

I think I agree.


> I suspect you'll have an easier time of it on that front if you
> include some examples of dynamically typed code that a well-behaved
> type-checker *must* report as correct Python code, such as:
>
> x: Optional[List[Any]]
> # This is the type of "x" *after* the if statement, not *during* it
> if arg is not None:
> x = list(arg)
> if other_arg is not None:
> # A well-behaved typechecker should allow this due to
> # the more specific initialisation in this particular branch
> x.extend(other_arg)
> else:
> x = None


There are very similar examples in PEP 484 (section on singletons in
unions),
we could just copy those or use this example,
but I am sure Guido will not agree to word "must" (although "should" maybe
possible :-)


> Similarly, it would be reasonable to say that these three snippets
> should all be equivalent from a typechecking perspective:
>
> x = None # type: Optional[T]
>
> x: Optional[T] = None
>
> x: Optional[T]
> x = None
>

Nice idea, explicit is better than implicit.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Nick Coghlan]

On 5 September 2016 at 04:13, Ivan Levkivskyi  wrote:
> On 4 September 2016 at 19:59, Nick Coghlan  wrote:
>> I suspect you'll have an easier time of it on that front if you
>> include some examples of dynamically typed code that a well-behaved
>> type-checker *must* report as correct Python code, such as:
>>
>> x: Optional[List[Any]]
>> # This is the type of "x" *after* the if statement, not *during* it
>> if arg is not None:
>> x = list(arg)
>> if other_arg is not None:
>> # A well-behaved typechecker should allow this due to
>> # the more specific initialisation in this particular branch
>> x.extend(other_arg)
>> else:
>> x = None
>
> There are very similar examples in PEP 484 (section on singletons in
> unions),
> we could just copy those or use this example,
> but I am sure Guido will not agree to word "must" (although "should" maybe
> possible :-)

"Should" would be fine by me :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Koos Zevenhoven]

On Sun, Sep 4, 2016 at 9:13 PM, Ivan Levkivskyi  wrote:
> On 4 September 2016 at 19:59, Nick Coghlan  wrote:
[...]
>>
>> Similarly, it would be reasonable to say that these three snippets
>> should all be equivalent from a typechecking perspective:
>>
>> x = None # type: Optional[T]
>>
>> x: Optional[T] = None
>>
>> x: Optional[T]
>> x = None
>
>
> Nice idea, explicit is better than implicit.
>

How is it going to help that these are equivalent within one checker,
if the meaning may differ across checkers?

-- Koos


> --
> Ivan
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/k7hoven%40gmail.com
>



-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Guido van Rossum]

Everybody please stop panicking. PEP 526 does not make a stand on the
behavior of type checkers (other than deferring to PEP 484). If you
want to start a discussion about constraining type checkers please do
it over at python-ideas. There is no rush as type checkers are not
affected by the feature freeze.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Koos Zevenhoven]

On Sun, Sep 4, 2016 at 6:38 PM, Nick Coghlan  wrote:
>
> There are two self-consistent sets of names:
>

Let me add a few. I wonder if this is really used so much that
bytes.chr is too long to type (and you can do bchr = bytes.chr if you
want to):

bytes.chr (or bchr in builtins)
bytes.chr_at, bytearray.chr_at
bytes.iterchr, bytearray.iterchr

bytes.chr (or bchr in builtins)
bytes.chrview, bytearray.chrview (sequence views)

bytes.char(or bytes.chr or bchr in builtins)
bytes.chars, bytearray.chars (sequence views)


> bchr
> bytes.getbchr, bytearray.getbchr
> bytes.iterbchr, bytearray.iterbchr
>
> byte
> bytes.getbyte, bytearray.getbyte
> bytes.iterbytes, bytearray.iterbytes
>
> The former set emphasises the "stringiness" of this behaviour, by
> aligning with the chr() builtin
>
> The latter set emphasises that these APIs are still about working with
> arbitrary binary data rather than text, with a Python "byte"
> subsequently being a length 1 bytes object containing a single integer
> between 0 and 255, rather than "What you get when you index or iterate
> over a bytes instance".
>
> Having noticed the discrepancy, my personal preference is to go with
> the latter option (since it better fits the "executable pseudocode"
> ideal and despite my reservations about "bytes objects contain int
> objects rather than byte objects", that shouldn't be any more
> confusing in the long run than explaining that str instances are
> containers of length-1 str instances). The fact "byte" is much easier
> to pronounce than bchr (bee-cher? bee-char?) also doesn't hurt.
>
> However, I suspect we'll need to put both sets of names in front of
> Guido and ask him to just pick whichever he prefers to get it resolved
> one way or the other.
>
>> And didn't someone recently propose deprecating iterability of str
>> (not indexing, or slicing, just iterability)? Then str would also need
>> a way to provide an iterable or sequence view of the characters. For
>> consistency, the str functionality would probably need to mimic the
>> approach in bytes. IOW, this PEP may in fact ultimately dictate how to
>> get a iterable/sequence from a str object.
>
> Strings are not going to become atomic objects, no matter how many
> times people suggest it.
>

You consider all non-iterable objects atomic? If str.__iter__ raises
an exception, it does not turn str somehow atomic. I wouldn't be
surprised by breaking changes of this nature to python at some point.
The breakage will be quite significant, but easy to fix.

-- Koos
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Greg Ewing]

On Sun, Sep 04, 2016 at 12:31:26PM +0100, Mark Shannon wrote:

As defined in PEP 526, I think that type 
annotations become a hindrance to type inference.


In Haskell-like languages, type annotations have no
ability to influence whether types can be inferred.
The compiler infers a type for everything, whether
you annotate or not. The annotations serve as
assertions about what the inferred types should be.
If they don't match, it means the programmer has
made a mistake somewhere.

I don't think it's possible for an annotation to
prevent the compiler from being able to infer a
type where it could have inferred one without the
annotation.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Random832]

On Sun, Sep 4, 2016, at 16:42, Koos Zevenhoven wrote:
> On Sun, Sep 4, 2016 at 6:38 PM, Nick Coghlan  wrote:
> >
> > There are two self-consistent sets of names:
> >
> 
> Let me add a few. I wonder if this is really used so much that
> bytes.chr is too long to type (and you can do bchr = bytes.chr if you
> want to):
> 
> bytes.chr (or bchr in builtins)
> bytes.chr_at, bytearray.chr_at

Ugh, that "at" is too reminiscent of java. And it just feels wrong to
spell it "chr" rather than "char" when there's a vowel elsewhere in the
name.

Hmm... how offensive to the zen of python would it be to have "magic" to
allow both bytes.chr(65) and b'ABCDE'.chr[0]? (and possibly also
iter(b'ABCDE'.chr)? That is, a descriptor which is callable on the
class, but returns a view on instances?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Please reject or postpone PEP 526

[Nick Coghlan]

On 5 September 2016 at 04:40, Koos Zevenhoven  wrote:
> On Sun, Sep 4, 2016 at 9:13 PM, Ivan Levkivskyi  wrote:
>> On 4 September 2016 at 19:59, Nick Coghlan  wrote:
> [...]
>>>
>>> Similarly, it would be reasonable to say that these three snippets
>>> should all be equivalent from a typechecking perspective:
>>>
>>> x = None # type: Optional[T]
>>>
>>> x: Optional[T] = None
>>>
>>> x: Optional[T]
>>> x = None
>>
>>
>> Nice idea, explicit is better than implicit.
>
> How is it going to help that these are equivalent within one checker,
> if the meaning may differ across checkers?

For typechecker developers, it provides absolute clarity that the
semantics of the new annotations should match the behaviour of
existing type comments when there's an initialiser present, or of a
parameter annotation when there's no initialiser present.

For folks following along without necessarily keeping up with all the
nuances, it makes it more explicit what Guido means when he says "PEP
526 does not make a stand on the
behavior of type checkers (other than deferring to PEP 484)."

For example, the formulation of the divergent initialisation case
where I think the preferred semantics are already implied by PEP 484
can be looked at this way:

x = None # type: Optional[List[T]]
if arg is not None:
x = list(arg)
if other_arg is not None:
x.extend(arg)

It would be a strange typechecker indeed that handled that case
differently from the new spellings made possible by PEP 526:

x: Optional[List[T]] = None
if arg is not None:
x = list(arg)
if other_arg is not None:
x.extend(arg)

x: Optional[List[T]]
if arg is None:
x = None
else:
x = list(arg)
if other_arg is not None:
x.extend(arg)

x: Optional[List[T]]
if arg is not None:
x = list(arg)
if other_arg is not None:
x.extend(arg)
else:
x = None

Or from the semantics of PEP 484 parameter annotations:

def my_func(arg:Optional[List[T]], other_arg=None):
# other_arg is implicitly Optional[Any]
if arg is not None and other_arg is not None:
# Here, "arg" can be assumed to be List[T]
# while "other_arg" is Any
arg.extend(other_arg)

A self-consistent typechecker will either allow all of the above, or
prohibit all of the above, while a typechecker that *isn't*
self-consistent would be incredibly hard to use.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 467: last round (?)

[Nick Coghlan]

On 5 September 2016 at 06:42, Koos Zevenhoven  wrote:
> On Sun, Sep 4, 2016 at 6:38 PM, Nick Coghlan  wrote:
>>
>> There are two self-consistent sets of names:
>>
>
> Let me add a few. I wonder if this is really used so much that
> bytes.chr is too long to type (and you can do bchr = bytes.chr if you
> want to)
>
> bytes.chr (or bchr in builtins)

The main problem with class method based spellings is that we need to
either duplicate it on bytearray or else break the bytearray/bytes
symmetry and propose "bytearray(bytes.chr(x))" as the replacement for
current cryptic "bytearray([x])"

Consider:

bytearray([x])
bytearray(bchr(x))
bytearray(byte(x))
bytearray(bytes.chr(x))

Folks that care about maintainability are generally willing to trade a
few extra characters at development time for ease of reading later,
but there are limits to how large a trade-off they can be asked to
make if we expect the alternative to actually be used (since overly
verbose code can be a readability problem in its own right).

> bytes.chr_at, bytearray.chr_at
> bytes.iterchr, bytearray.iterchr

These don't work for me because I'd expect iterchr to take encoding
and errors arguments and produce length 1 strings.

You also run into a searchability problem as "chr" will get hits for
both the chr builtin and bytes.chr, similar to the afalg problem that
recently came up in another thread. While namespaces are a honking
great idea, the fact that search is non-hierarchical means they still
don't give API designers complete freedom to reuse names at will.

> bytes.chr (or bchr in builtins)
> bytes.chrview, bytearray.chrview (sequence views)
>
> bytes.char(or bytes.chr or bchr in builtins)
> bytes.chars, bytearray.chars (sequence views)

The views are already available via memoryview.cast if folks really
want them, but encouraging their use in general isn't a great idea, as
it means more function developers now need to ask themselves "What if
someone passes me a char view rather than a normal bytes object?".

>> Strings are not going to become atomic objects, no matter how many
>> times people suggest it.
>
> You consider all non-iterable objects atomic? If str.__iter__ raises
> an exception, it does not turn str somehow atomic.

"atomic" is an overloaded word in software design, but it's still the
right one for pointing out that something people want strings to be
atomic, and sometimes they don't - it depends on what they're doing.

In particular, you can look up the many, many, many discussions of
providing a generic flatten operation for iterables, and how it always
founders on the question of types like str and bytes, which can both
be usefully viewed as an atomic unit of information, *and* as
containers of smaller units of information (NumPy arrays are another
excellent example of this problem).

> I wouldn't be
> surprised by breaking changes of this nature to python at some point.

I would, and you should be to:
http://www.curiousefficiency.org/posts/2014/08/python-4000.html

> The breakage will be quite significant, but easy to fix.

Please keep in mind that we're already 10 years into a breaking change
to Python's text handling model, with another decade or so still to go
before the legacy Python 2 text model is spoken of largely in terms
similar to the way COBOL is spoken of today. There is no such thing as
a "significant, but easy to fix" change when it comes to adjusting how
a programming language handles text data, as text handling is a
fundamental part of defining how a language is used to communicate
with people.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

[Steve Dower]

I posted a minor update to PEP 528 at 
https://github.com/python/peps/blob/master/pep-0528.txt and a diff below.


While there are likely to be technical and compatibility issues to 
resolve after the changes are applied, I don't believe they impact the 
decision to accept the change at the PEP-level (everyone who has raised 
potential issues has also been supportive of the change). Without real 
experience during the beta period, it's really hard to determine whether 
fixes should be made on our side or their side, so I think it's worth 
going ahead with the change, even if specific implementation details 
change between now and release.


Cheers,
Steve

---

@@ -21,8 +21,7 @@
 This PEP proposes changing the default standard stream implementation 
on Windows
 to use the Unicode APIs. This will allow users to print and input the 
full range

 of Unicode characters at the default Windows console. This also requires a
-subtle change to how the tokenizer parses text from readline hooks, 
that should

-have no backwards compatibility issues.
+subtle change to how the tokenizer parses text from readline hooks.

 Specific Changes
 
@@ -46,7 +45,7 @@

 The use of an ASCII compatible encoding is required to maintain 
compatibility
 with code that bypasses the ``TextIOWrapper`` and directly writes 
ASCII bytes to
-the standard streams (for example, [process_stdinreader.py]_). Code 
that assumes
+the standard streams (for example, `Twisted's 
process_stdinreader.py`_). Code that assumes
 a particular encoding for the standard streams other than ASCII will 
likely

 break.

@@ -78,8 +77,9 @@
 Alternative Approaches
 ==

-The ``win_unicode_console`` package [win_unicode_console]_ is a pure-Python
-alternative to changing the default behaviour of the console.
+The `win_unicode_console package`_ is a pure-Python alternative to 
changing the

+default behaviour of the console. It implements essentially the same
+modifications as described here using pure Python code.

 Code that may break
 ===
@@ -94,21 +94,21 @@

 Code that assumes that the encoding required by ``sys.stdin.buffer`` or
 ``sys.stdout.buffer`` is ``'mbcs'`` or a more specific encoding may 
currently be
-working by chance, but could encounter issues under this change. For 
example::
+working by chance, but could encounter issues under this change. For 
example:


-sys.stdout.buffer.write(text.encode('mbcs'))
-r = sys.stdin.buffer.read(16).decode('cp437')
+>>> sys.stdout.buffer.write(text.encode('mbcs'))
+>>> r = sys.stdin.buffer.read(16).decode('cp437')

 To correct this code, the encoding specified on the ``TextIOWrapper`` 
should be

-used, either implicitly or explicitly::
+used, either implicitly or explicitly:

-# Fix 1: Use wrapper correctly
-sys.stdout.write(text)
-r = sys.stdin.read(16)
+>>> # Fix 1: Use wrapper correctly
+>>> sys.stdout.write(text)
+>>> r = sys.stdin.read(16)

-# Fix 2: Use encoding explicitly
-sys.stdout.buffer.write(text.encode(sys.stdout.encoding))
-r = sys.stdin.buffer.read(16).decode(sys.stdin.encoding)
+>>> # Fix 2: Use encoding explicitly
+>>> sys.stdout.buffer.write(text.encode(sys.stdout.encoding))
+>>> r = sys.stdin.buffer.read(16).decode(sys.stdin.encoding)

 Incorrectly using the raw object
 
@@ -117,32 +117,57 @@
 writes may be affected. This is particularly important for reads, 
where the
 number of characters read will never exceed one-fourth of the number 
of bytes
 allowed, as there is no feasible way to prevent input from encoding as 
much

-longer utf-8 strings::
+longer utf-8 strings.

->>> stdin = open(sys.stdin.fileno(), 'rb')
->>> data = stdin.raw.read(15)
+>>> raw_stdin = sys.stdin.buffer.raw
+>>> data = raw_stdin.read(15)
 abcdefghijklm
 b'abc'
 # data contains at most 3 characters, and never more than 12 bytes
 # error, as "defghijklm\r\n" is passed to the interactive prompt

 To correct this code, the buffered reader/writer should be used, or 
the caller

-should continue reading until its buffer is full.::
+should continue reading until its buffer is full.

-# Fix 1: Use the buffered reader/writer
->>> stdin = open(sys.stdin.fileno(), 'rb')
+>>> # Fix 1: Use the buffered reader/writer
+>>> stdin = sys.stdin.buffer
 >>> data = stdin.read(15)
 abcedfghijklm
 b'abcdefghijklm\r\n'

-# Fix 2: Loop until enough bytes have been read
->>> stdin = open(sys.stdin.fileno(), 'rb')
+>>> # Fix 2: Loop until enough bytes have been read
+>>> raw_stdin = sys.stdin.buffer.raw
 >>> b = b''
 >>> while len(b) < 15:
-... b += stdin.raw.read(15)
+... b += raw_stdin.read(15)
 abcedfghijklm
 b'abcdefghijklm\r\n'

+Using the raw object with small buffers
+---
+
+Code that uses the raw IO object and attempts to read less than four 
characters
+will

Re: [Python-Dev] PEP 529: Change Windows filesystem encoding to UTF-8

[Steve Dower]

I posted an update to PEP 529 at 
https://github.com/python/peps/blob/master/pep-0529.txt and a diff below. The 
update includes more detail on the affected code within CPython - including a 
number of references to broken code that would be resolved with the change - 
and more details about the necessary changes.

As with PEP 528, I don't think it's possible to predict the impact better than 
I already have, and the beta period will be essential to determine whether this 
change is completely unworkable. I am fully prepared to back out the change if 
necessary prior to RC.

Cheers,
Steve

---


@@ -16,7 +16,8 @@
 operating system, often via C Runtime functions. However, these have been long
 discouraged in favor of the UTF-16 APIs. Within the operating system, all text
 is represented as UTF-16, and the ANSI APIs perform encoding and decoding using
-the active code page.
+the active code page. See `Naming Files, Paths, and Namespaces`_ for
+more details.
 
 This PEP proposes changing the default filesystem encoding on Windows to utf-8,
 and changing all filesystem functions to use the Unicode APIs for filesystem
@@ -27,10 +28,10 @@
 characters outside of the user's active code page.
 
 Notably, this does not impact the encoding of the contents of files. These will
-continue to default to locale.getpreferredencoding (for text files) or plain
-bytes (for binary files). This only affects the encoding used when users pass a
-bytes object to Python where it is then passed to the operating system as a 
path
-name.
+continue to default to ``locale.getpreferredencoding()`` (for text files) or
+plain bytes (for binary files). This only affects the encoding used when users
+pass a bytes object to Python where it is then passed to the operating system 
as
+a path name.
 
 Background
 ==
@@ -44,9 +45,10 @@
 
 When paths are passed between the filesystem and the application, they are
 either passed through as a bytes blob or converted to/from str using
-``os.fsencode()`` or ``sys.getfilesystemencoding()``. The result of encoding a
-string with ``sys.getfilesystemencoding()`` is a blob of bytes in the native
-format for the default file system.
+``os.fsencode()`` and ``os.fsdecode()`` or explicit encoding using
+``sys.getfilesystemencoding()``. The result of encoding a string with
+``sys.getfilesystemencoding()`` is a blob of bytes in the native format for the
+default file system.
 
 On Windows, the native format for the filesystem is utf-16-le. The recommended
 platform APIs for accessing the filesystem all accept and return text encoded 
in
@@ -83,11 +85,11 @@
 canonical representation. Even if the encoding is "incorrect" by some standard,
 the file system will still map the bytes back to the file. Making use of this
 avoids the cost of decoding and reencoding, such that (theoretically, and only
-on POSIX), code such as this may be faster because of the use of `b'.'` 
compared
-to using `'.'`::
+on POSIX), code such as this may be faster because of the use of ``b'.'``
+compared to using ``'.'``::
 
 >>> for f in os.listdir(b'.'):
-... os.stat(f)
+... os.stat(f)
 ...
 
 As a result, POSIX-focused library authors prefer to use bytes to represent
@@ -105,32 +107,31 @@
 Currently the default filesystem encoding is 'mbcs', which is a meta-encoder
 that uses the active code page. However, when bytes are passed to the 
filesystem
 they go through the \*A APIs and the operating system handles encoding. In this
-case, paths are always encoded using the equivalent of 'mbcs:replace' - we have
-no ability to change this (though there is a user/machine configuration option
-to change the encoding from CP_ACP to CP_OEM, so it won't necessarily always
-match mbcs...)
+case, paths are always encoded using the equivalent of 'mbcs:replace' with no
+opportunity for Python to override or change this.
 
 This proposal would remove all use of the \*A APIs and only ever call the \*W
-APIs. When Windows returns paths to Python as str, they will be decoded from
+APIs. When Windows returns paths to Python as ``str``, they will be decoded 
from
 utf-16-le and returned as text (in whatever the minimal representation is). 
When
-Windows returns paths to Python as bytes, they will be decoded from utf-16-le 
to
-utf-8 using surrogatepass (Windows does not validate surrogate pairs, so it is
-possible to have invalid surrogates in filenames). Equally, when paths are
-provided as bytes, they are decoded from utf-8 into utf-16-le and passed to the
-\*W APIs.
+Python code requests paths as ``bytes``, the paths will be transcoded from
+utf-16-le into utf-8 using surrogatepass (Windows does not validate surrogate
+pairs, so it is possible to have invalid surrogates in filenames). Equally, 
when
+paths are provided as ``bytes``, they are trasncoded from utf-8 into utf-16-le
+and passed to the \*W APIs.
 
-The use of utf-8 will not be configurable, with the possible exception of a
-"legacy mode" environment variable or X-flag.