Re: [Python-Dev] PEP 561 rework

[Sebastian Rittau]

Hello everyone,


Am 14.11.2017 um 00:29 schrieb Guido van Rossum:
This is a nice piece of work. I expect to accept it pretty much 
verbatim (with some small edits, see 
https://github.com/python/peps/pull/467). I agree with Nick that we 
don't have to do anything specifically about control of foo_stubs 
packages -- nor do I think we need to worry about foo_stubs vs. foo-stubs.


Everyone else: if you think this should not go through, now's the time 
to reply-all here!
I am really looking forward to the implementation of this PEP and I am 
glad that it is close to acceptance. One thing that is not really clear 
to me is how module-only packages are handled. Say I have a package 
"foo" that installs the file "foo.py" to site-packages, where would I 
install "foo.pyi" and py.typed to? Or is this case not supported and I 
have to convert the foo module into a package containing just __init__.py?


 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

[Sebastian Rittau]

Am 14.11.2017 um 02:38 schrieb Guido van Rossum:
On Mon, Nov 13, 2017 at 3:50 PM, Sebastian Rittau <mailto:srit...@rittau.biz>> wrote:


I am really looking forward to the implementation of this PEP and
I am glad that it is close to acceptance. One thing that is not
really clear to me is how module-only packages are handled. Say I
have a package "foo" that installs the file "foo.py" to
site-packages, where would I install "foo.pyi" and py.typed to? Or
is this case not supported and I have to convert the foo module
into a package containing just __init__.py?


Good call. I think that conversion to a package is indeed the best 
approach -- it doesn't seem worth it to add more special-casing for 
this scenario.


Ethan, if you agree, you should just add a sentence about this to the PEP.

This seems like the best solution, especially since setuptools does not 
really support installing package data for pure modules.


 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Sebastian Rittau]

On 25.11.2017 22:06, Eric V. Smith wrote:
The updated version should show up at 
https://www.python.org/dev/peps/pep-0557/ shortly.


This PEP looks very promising and will make my life quite a bit easier, 
since we are using a pattern involving data classes. Currently, we write 
the constructor by hand.



The major changes from the previous version are:

- Add InitVar to specify initialize-only fields. 


This is the only feature that does not sit right with me. It looks very 
obscure and "hacky". From what I understand, we are supposed to use the 
field syntax to define constructor arguments. I'd argue that the name 
"initialize-only fields" is a misnomer, which only hides the fact that 
this has nothing to do with fields at all. Couldn't dataclassses just 
pass *args and **kwargs to __post_init__()? Type checkers need to be 
special-cases for InitVar anyway, couldn't they instead be special cased 
to look at __post_init__ argument types?


 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Sebastian Rittau]

On 27.11.2017 12:01, Sebastian Rittau wrote:



The major changes from the previous version are:

- Add InitVar to specify initialize-only fields. 


This is the only feature that does not sit right with me. It looks 
very obscure and "hacky". From what I understand, we are supposed to 
use the field syntax to define constructor arguments. I'd argue that 
the name "initialize-only fields" is a misnomer, which only hides the 
fact that this has nothing to do with fields at all. Couldn't 
dataclassses just pass *args and **kwargs to __post_init__()? Type 
checkers need to be special-cases for InitVar anyway, couldn't they 
instead be special cased to look at __post_init__ argument types? 
I am sorry for the double post, but I thought a bit more about why this 
does not right with me:


 * As written above, InitVars look like fields, but aren't.
 * InitVar goes against the established way to pass through arguments,
   *args and **kwargs. While type checking those is an unsolved
   problem, from what I understand, I don't think we should introduce a
   second way just for dataclasses.
 * InitVars look like a way to satisfy the type checker without
   providing any benefit to the programmer. Even when I'm not
   interested in type checking, I have to declare init vars.
 * InitVars force me to repeat myself. I have the InitVar declaration
   and then I have the repeat myself in the signature of
   __post_init__(). This has all the usual problems of repeated code.

I hope I did not misunderstood the purpose of InitVar.

 - Sebastian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Sebastian Rittau]

On 27.11.2017 13:23, Eric V. Smith wrote:
I had something like your suggestion half coded up, except I inspected 
the args to __post_init__() and added them to __init__, avoiding the 
API-unfriendly *args and **kwargs.
I understand your concerns with *args and **kwargs. I think we need to 
find a solution for that eventually.


One other thing about InitVar: it lets you control where the init-only 
parameter goes in the __init__ call. This is especially important with 
default values:


This is indeed a nice property. I was thinking about that myself and how 
to best handle it. One use case that could occur in out codebase is 
passing in a "context" argument. By convention, this is always the first 
argument to the constructor, so it would be nice if this would also work 
for dataclasses.


 - Sebastian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Sebastian Rittau]

On 16.07.2018 19:21, Adam Cataldo via Python-Dev wrote:

*

 *

One thing we care about in particular, given the implementation of
pytype, is the detailed definition of what goes in a .pyi file. Do
folks think this makes sense to include as part of PEP 484, or
would this be better in a separate PEP? We’d love to get your
thoughts.

*
It would be useful to define - on a semantic, not syntactic level - what 
constructs a type checker is expected to understand. Not only for 
implementers, but more importantly for stub authors. Sometimes it's hard 
to judge, what constructs type checkers will understand. And while by 
now I think I personally have a solid understanding of what mypy 
understands, I have no idea whether that also applies to pytype, 
PyCharm, or other type checkers.


For example, in one of my first pull requests for typeshed, I tried to 
use the following construct, expecting type checkers to understand it:


    class Foo:
    def bar(self) -> None:
    raise NotImplementedError()

It seems they don't, but mypy understands:

    class Foo:
    @abstractmethod
    def bar(self) -> None: ...

But do I need to import abstractmethod? Would @abc.abstractmethod also 
work? Can I have an assignment "_am = abc.abstractmethod" and then @_am 
would work? Can I alias functions by using assignments in stubs or 
should I use a second function definition? How complex can Python 
version checks be? There are many more of those questions.


If these expectations would be documents, implementers of type checkers 
can still decide not to support certain constructs (or not support them 
yet), or even to support more constructs. But at least such a deviation 
could be documented, so users know what to expect. On the other hand, 
stub authors will know what constructs will likely not work and should 
be avoided.


 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Sebastian Rittau]

On 17.07.2018 17:05, Guido van Rossum wrote:
This is a good point. I presume specifying this unambiguously would be 
a huge amount of work, and it would mostly codify mypy's current 
behavior. I don't think that's within the scope of PEP 484, but it 
could well be done as a separate PEP (perhaps an informational one?). 
I hope you understand that I am not volunteering.
An informational PEP sounds about right to me. Such a PEP could also 
include style recommendations like those from typeshed's CONTRIBUTING 
file (https://github.com/python/typeshed/blob/master/CONTRIBUTING.md).


I guess I just volunteered to help with such a PEP, although I feel that 
someone from mypy's core team should take the lead on that. And if I 
understood this thread correctly, the pytype team is also willing to 
help out?


 - Sebastian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove tempfile.mktemp()

[Sebastian Rittau]

Am 19.03.19 um 14:53 schrieb Victor Stinner:


When I write tests, I don't really care of security, but
NamedTemporaryFile caused me many troubles on Windows: you cannot
delete a file if it's still open in a another program. It's way more
convenient to use tempfile.mktemp().

O_EXCL, open(tmpname, "wx"), os.open(tmpname, os.O_CREAT | os.O_EXCL |
os.O_WRONLY), etc. can be used to get an error if the file already
exists.

I agree that for production code where security matters,
tempfile.mktemp() must be avoided. But I would prefer to keep it for
tests.


If there are valid use cases for mktemp(), I recommend renaming
it to mkname_unsafe() or something equally obvious. Experience
(and the list of packages still using mktemp() posted here) shows
that just adding a warning to documentation is not enough. Users
often discover functions by experimentation or looking at examples
on the internet.

mktemp() is also unfortunately named, as it does not create a temp
file as implied. This can also add to the impression that it is the
proper function to use.

Adding a new function and following the deprecation process for the
old one should only be a minor inconvenience for existing users that
need it, should wake up existing users that should not use it in the
first place, and still allows it to be used for relevant use cases.

I believe for security reasons sometimes inconvenient changes like
this are necessary.

 - Sebastian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove tempfile.mktemp()

[Sebastian Rittau]

Am 19.03.19 um 17:23 schrieb Giampaolo Rodola':

@Sebastian

If there are valid use cases for mktemp(), I recommend renaming
it to mkname_unsafe() or something equally obvious.

I'm -1 about adding an alias (there should be one and preferably only
one way to do it). Also mkstemp() and mkdtemp() are somewhat poorly
named IMO, but I wouldn't add an alias for them either.


Just to clarify: I was not suggesting creating an alias, I was suggesting
renaming the function, but keeping the old name for a normal
deprecation cycle.

But I had another thought: If I understand correctly, the exploitability
of mktemp() relies on the fact that between determining whether the
file exists and creation an attacker can create the file themselves.
Couldn't this problem be solved by generating a filename of sufficient
length using the secrets module? This way the filename should be
"unguessable" and safe.

 - Sebastian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove tempfile.mktemp()

[Sebastian Rittau]

Am 20.03.19 um 09:47 schrieb Anders Munch:

Greg Ewing:

So use NamedTemporaryFile(delete = False) and close it before passing it to the 
other program.

That's effectively the same as calling tempfile.mktemp.   While it does waste 
time opening and closing an unused file, that doesn't help with security.  If 
anything, it might worsen security.


That is not actually true. The important difference is that with 
NamedTemporaryFile the file exists with appropriate access right (0600). 
This denies access of that file to other users. With mktemp() no file is 
created, so another user can "hijack" that name and cause programs to 
write potentially privileged data into or read manipulated data from 
that file.


 - Sebastian


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

[Sebastian Rittau]

Am 21.05.19 um 13:39 schrieb André Malo:


So what I hear is, this battery is definitely not dead, which is what the
PEP is all about.
it's just half charged (or discharged, depending on your POV), so to speak.

Substitute: "none" should read pypi then?


Every single module on this list will have some users. To me the 
question is: Are there enough users to justify the extra burden on the 
maintainers, but also users not using that package? (The latter will 
have to download the package, have it installed on their systems, it 
makes it harder to find other things in the documentation etc.) If there 
not many users left, I believe these should either vendor the last 
official package or to band together to maintain the package themselves 
on pypi.


 - Sebastian


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: Typing syntax and ecosystem

[Sebastian Rittau]

Am 16.04.21 um 03:21 schrieb Barry Warsaw:
Actually, I think it’s time for a comprehensive guide to type 
annotations. Just anecdotally, I was trying to annotate a library of 
mine and was having an impossible time of it, until a chat with Guido 
lead me to @typing.overload. That solved my problem intuitively and 
easily, but I just didn’t know about it. Right now, there’s 
information spread out all over the place, the stdlib documentation, 
tool documentation, StackOverflow :D etc. It’s a complicated topic 
that I think a comprehensive guide, a tutorial, etc. could really help 
with.


As a typeshed maintainer, I agree. Currently, the typing documentation 
is spread over various PEPs, the typing module documentation, and the 
mypy documentation. We also have a style guide for stub in the typeshed 
documentation. That said, at least for type stubs, we are working on a 
PEP (still looking for a sponsor) that is supposed to be a comprehensive 
document. [1][2]


 - Sebastian

[1] 
https://mail.python.org/archives/list/typing-...@python.org/message/VNMC3JPWWP3O4TNMJZMSEH6UU5BPN4ZJ/

[2] https://github.com/srittau/type-stub-pep/blob/type-stub-pep/pep-.rst


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SDEQGGWKL5MHBPZURDVQSV6VC22MWY3E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Relaxing the annotation syntax

[Sebastian Rittau]

On Sun, Apr 11, 2021 at 1:31 PM Barry Warsaw > wrote:

[snip]

This is something the SC has been musing about, but as it’s not a
fully formed idea, I’m a little hesitant to bring it up.  That
said, it’s somewhat relevant: We wonder if it may be time to in a
sense separate the typing syntax from Python’s regular syntax. 
TypeGuards are a case where if typing had more flexibility to
adopt syntax that wasn’t strictly legal “normal” Python, maybe
something more intuitive could have been proposed.  I wonder if
the typing-sig has discussed this possibility (in the future, of
course)?

I am strongly in favor of diverging type annotation syntax from Python 
syntax. Currently, type annotations are a very useful tool, but often 
clunky to use. Enhancements have been made, but design space is limited 
when working within existing Python syntax. Type annotations have a 
different set of rules, needs, and constraints than general-purpose 
Python code. This is similar to other domain specific languages like 
regular expressions. Ideally, Python itself would not check the syntax 
of annotations, except as needed for determining the end of an 
annotation. PEP 563 is a step in that direction.


As far as I understand the arguments against PEP 563 and in favor of PEP 
649 mostly boil down to "annotations are used outside of typing, these 
uses would need to use eval() in the future and eval() is slow". (At 
least from a user's perspective, there are more arguments from a Python 
maintainer's perspective that I can't comment on.) Are there benchmarks 
to verify that using eval() has a non-negligible effect for this use 
case? Overall, I don't find this to be a compelling argument when 
compared to the problem that PEP 649 would close all design space for 
type annotation syntax enhancements.


 - Sebastian


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RWLOLMWLPZ3O5VO7OQZSHTQGR4ICXDJV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Sebastian Rittau]

Am 20.04.2021 um 19:03 schrieb Mark Shannon:
PEP 544 supports structural typing, but to declare a structural type 
you must inherit from Protocol.

That smells a lot like nominal typing to me.


I'm not sure what inheriting from Protocol has to do with nominal 
typing, even though I would personally prefer to have a separate keyword 
for declaring protocols. But current typing philosophy is to prefer 
structural over nominal typing:


 * Use abstract types like "Iterable" or "Sequence" over concrete types
   list "list".
 * Use protocols instead of instances where it makes sense (although
   pragmatism very often means using an existing concrete class instead
   of defining a protocol).
 * We are slowly replacing typing.IO et al. with more restricted protocols.

Personally I think that the typing infrastructure should move even more 
towards structural typing than it does at the moment and there is 
certainly a lot of room to grow. But overall I feel that typing is 
moving towards allowing more duck typing than it does at the moment.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4SDOHDJENVLNDZVJYZN47JSHWJMN4SHV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Sebastian Rittau]

Am 22.04.21 um 10:42 schrieb Chris Angelico:

File-like objects are used VERY frequently in the stdlib, and actual
open file objects have quite a large interface. The use-case is a
pretty real one: "if I were to create a simulant file object to pass
to json.load(), what methods do I need?".

Maybe in some cases, the "smaller protocols" option is practical, but
it would need to have a useful name. For instance, if it needs to be
readable, iterable, closeable, and autocloseable via
__enter__/__exit__, that's ... uhh a readable, iterable, closeable
context manager? Not an improvement over "file-like object".


Experience from typeshed shows that many functions in the stdlib and 
third-party libraries only use one or very few methods, very often just 
read() or write(). From a practical standpoint, small protocols seem 
quite feasible. These are quite an improvement over "file-like" objects, 
where no one knows what that actually means.


 - Sebastian


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H33OH6GGIGHR7SVE6NL32I7OBO7B7F7E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Anyone else gotten bizarre personal replies to mailing list posts?

[Sebastian Rittau]

Am 23.04.21 um 17:38 schrieb Nathaniel Smith:
I just got the reply below sent directly to my personal account, and 
I'm confused about what's going on. If it's just a one off I'll chalk 
it up to random internet weirdness, but if other folks are getting 
these too it might be something the list admins should look into? 
Or... something?


I also got one of these from the same sender, Same rambling, incoherent 
style, which I have always connected to mental issues. To protect both 
the sender and other list members, I would suggest to the list owners to 
manually unsubscribe the sender from the mailing list.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6SRZW4BI2ONGD4NG2AID5F5GEYNMLAJE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Keeping Python a Duck Typed Language.

[Sebastian Rittau]

Am 24.04.2021 um 01:26 schrieb Gregory P. Smith:
Practically speaking, one issue I have is how easy it is to write 
isinstance or issubclass checks. It has historically been much more 
difficult to write and maintain a check that something looks like a duck.


 `if hasattr(foo, 'close') and hasattr(foo, 'seek') and hasattr(foo, 
'read'):`


Just does not roll off the figurative tongue and that is a relatively 
simple example of what is required for a duck check.


To prevent isinstance use when a duck check would be better, we're 
missing an easy builtin elevated to the isinstance() availability 
level behaving as lookslikeaduck() that does matches against a (set 
of) declared typing.Protocol shape(s). An implementation of this 
exists - 
https://www.python.org/dev/peps/pep-0544/#runtime-checkable-decorator-and-narrowing-types-by-isinstance 
 
- but it requires the protocols to declare runtime checkability and 
has them work with isinstance similar to ABCs...  technically accurate 
/BUT via isinstance/? Doh!  It promotes the use of isinstance when it 
really isn't about class hierarchy at all...


Edit: Maybe that's okay, isinstance can be read leniently to mean "is 
an instance of something that one of these things over here says it 
matches" rather than meaning "a parent class type is..."?  From a past 
experience user perspective I don't read "isinstance" as "looks like a 
duck" when I read code.  I assume I'm not alone.


I'm using isinstance from time to time, mostly to satisfy the type 
checker in some cases. I think having a "hasshape()" builtin would be a 
huge win. In addition it would be useful for type checkers to better 
support hasattr() for distinguishing between separate types. For 
example, mypy has a problem with the following:


    class A: pass
    class B:
    x: int

    def foo(x: A | B) -> None:
    if hasattr(x, "b"):
    accepts_b(x)
    else:
    accepts_a(x)

As Nathaniel indicated, how deep do we want to go down this rabbit 
hole of checking?  just names? signatures and types on those?  What 
about exceptions (something our type system has no way to declare at 
all)?  and infinite side effects?  At the end of the day we're 
required to trust the result of whatever check we use and any 
implementation may not conform to our desires no matter how much 
checking we do. Unless we solve the halting problem. :P
I think a PEP to discuss these questions would be appropriate. Things 
like also checking types could also be optional.
Not quite.  A Protocol is merely a way to describe a structural type.  
You do not /need/ to have your /implementations/ of anything inherit 
from typing.Protocol.  I'd /personally/ advise people /do not inherit/ 
from Protocol in their implementation.


+1

Luciano notes that it is preferred to define your protocols as narrow 
and define them in places *where they're used*, to follow a golang 
interface practice.  My thinking aligns with that.


My background is with TypeScript, not go. But TypeScript uses 
"interfaces" extensively to describe the shape of expected objects. I 
agree mostly with you and Luciano here, but it can make sense to define 
some protocols in a more visible location. Examples are the protocols in 
collections.abc. In typeshed we collected a few more common protocols in 
the type-check only module _typeshed. 
(https://github.com/python/typeshed/tree/master/stdlib/_typeshed) These 
are a bit experimental, but could eventually find their way into the 
standard library.


A bit off-topic: But intersection types (as discussed here: 
https://github.com/python/typing/issues/213) could also make a nice 
addition to quickly compose ad-hoc protocols from pre-made protocols:


    def read_stuff(f: HasRead & HasSeek) -> None: ...

That inheritance is used in the /declaration/ of the protocol is an 
implementation detail because our language has never had a syntax for 
declaring an interface. 544 fit within our existing language syntax.


Jukka Lehtosalo proposed a new "type" keyword over at typing-sig: 
https://mail.python.org/archives/list/typing-...@python.org/thread/LV22PX454W4VDTXY6NDJV7NZD4LFK464/ 
This could also be used to define protocols a bit more succintly, and 
with additional syntax constraints, making protocols feel a bit more 
"first class" than they feel now.


 - Sebastian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VJEBP3U37RFN36TJDNLDQTHKCW7D4KHJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Origins of iterators and iterables

[Sebastian Rittau]

Am 30.05.21 um 19:08 schrieb Guido van Rossum:
Returning "self" as the iterator was originally only intended to paper 
over the case where you want to write


it = iter(a)

for x in it:
    ...

-- basically we wanted 'for x in iter(a)' and 'for x in a' to have the 
same meaning.


The above use case (iterator being iterable themselves) was a very good 
design decision. In fact, I have a blog post dating back from 2006 where 
I berated Java from not doing the same: 
https://rittau.org/2006/11/java-iterators-are-not-iterable/. To take my 
example from there converted to python:


class Tree:
    def depth_first() -> Iterator[...]: ...
    def breath_first() -> Iterator[...]: ...

for item in tree.depth_first(): ...

This example would not work if "iter(it)" would not return "self".

 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VWLREDZR6HXXKBGTWRXV6H3FU7QZQWOQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: declare "unstable APIs"

[Sebastian Rittau]

Am 03.06.21 um 20:11 schrieb Gregory P. Smith:
The ideal way to declare an API as unstable is to constantly change it 
in a breaking manner.  With every release and potentially even within 
some patch releases when the point really needs to be made.  Even when 
you didn't have a reason to change anything.  If you don't do that, 
people are going to find it convenient, discover stability, assume it 
exists, depend on it, and complain about breakage no matter what was 
stated. https://www.hyrumslaw.com/ 


There is certainly value in having some stability guarantees, even in 
"unstable" APIs. For example, while it is expected that the ast module 
breaks with each new minor Python version (3.7, 3.8 etc.), it's still 
stable during each such version. This makes the ast module quite useful 
for lots of applications that work on Python source code and that are 
understood to need changes for each Python version anyway. Breaking the 
compatibility just for the sake of it would be counterproductive.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7BBKPPUD7BX6THFFBWZXCUK3TBIFNNQS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Roundup to GitHub Issues migration

[Sebastian Rittau]

Am 22.06.21 um 10:00 schrieb Tiziano Zito:
I think it is important to notice that GitHub actively blocks user 
registration and activity from countries that are sanctioned by the US 
government. At least in 2019 GitHub was blocking users from IPs 
located in Cuba, North Corea, Syria, Crimea, Iran, etc (see for 
example [1]). They block, of course, users of any nationality, if they 
happen to be traveling or living in those countries.


I could not find any clear official statement from GitHub, but I think 
this is something to consider nonetheless, especially now that the 
Python community is making great efforts to become more welcoming and 
diverse. The fact of excluding a significant part of the potential 
contributors based on a random list by a random government over which 
the Python community as a whole has no influence whatsoever seems a 
move in the wrong direction. 


I was overall in favor of moving Python issues over to GitHub, for 
convenience, easier access, and a more usable interface. But I think the 
issue above is a showstopper. This problem of course already exists for 
pull requests, but discriminating against users based on their place of 
residence is absolutely unacceptable to me. In fact, it is directly in 
violation to the PSF's mission statement that says in part: "... to 
support and facilitate the growth of a diverse and international 
community of Python programmers." This issue hasn't been addresses in 
PEP 581, so I believe it wasn't considered when accepting the PEP. But 
it's serious enough that I would like to ask the steering council to 
reconsider their decision to accept PEP 581.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JWNL7FSDK6727ED73EKSQGELITP3F4I2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Roundup to GitHub Issues migration

[Sebastian Rittau]

(For some weird reason, I didn't get this mail via the list, only the 
private copy, so I might break threading by replying via my private copy.)


Am 22.06.21 um 14:01 schrieb Nathaniel Smith:


As much as we might wish otherwise, the PSF is also a US entity and
has to comply with US laws. GitHub's official policy at

https://docs.github.com/en/github/site-policy/github-and-trade-controls

gives the impression that they're reading the law as narrowly as
possible, and allowing access to every person that they legally can.
In particular, that policy page claims that there are no restrictions
on users from Cuba or Iran, and that users from Syria and Crimea are
allowed to participate in OSS projects, just not give GitHub money.
(They do disallow use by North Koreans and "Specially Designated
Nationals".)

It is even possible for the PSF to do better without breaking the law?
I'm not an expert in this area at all, so happy to be educated if
so...

-n

I'm not much interested in theory. Are people currently blocked from 
participating in Python issues due to their country of residence? Will 
people be blocked after the move to GitHub? If the answer to the second 
question is "yes", the move would violate one of the core principles of 
the PSF. If the answer to the first question is "yes", we need to 
consider transferring the bug tracker to an organization in a country 
whose laws better align with the goals of the PSF.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NG4OCTORMKAKXC5SSUC4TAJEOZN5ZAWF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Sebastian Rittau]

Am 20.10.21 um 15:18 schrieb Thomas Wouters:


By and large, the SC views PEP 649 as a better way forward. If PEP 563 
had never come along, it would be a fairly easy decision to accept PEP 
649. We are still inclined to accept PEP 649. That would leave the 
consideration about what to do with PEP 563 and existing from 
__future__ import annotationsdirectives. As far as we can tell, there 
are two reasons for code to want to use PEP 563: being able to 
conveniently refer to names that aren’t available until later in the 
code (i.e. forward references), and reducing the overhead of type 
annotations. If PEP 649 satisfies allof the objectives of PEP 563, is 
there a reason to keep supporting PEP 563’s stringified annotations?  
Are there any practical, real uses of stringified annotations that 
would not be served by PEP 649's deferred annotations?



These are my reasons for using from __future__ import annotations:

 * Forward references and cyclic imports (with if TYPE_CHECKING: from
   foo import Bar) while avoiding quotes.
 * Using typing features from future Python versions. This becomes less
   important the more typing stabilizes.

 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KZK7Q4LT7CDBUDSAPQH4236AA5KABSGM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Sebastian Rittau]

I'm sorry, I sent my last mail too early. Here are the rest of my thoughts.

Am 20.10.21 um 15:18 schrieb Thomas Wouters:


Keeping the future import and stringified annotations around is 
certainly an option, but we’re worried about the cost of the 
implementation, the support cost, and the confusion for users 
(specifically, it is a future import that will never become the 
future). If we do keep them, how long would we keep them around? 
Should we warn about their use? If we warn about the future import, is 
the noise and confusion this generates going to be worth it? If we 
don't warn about them, how will we ever be able to turn them off?


Personally, I think that stringified annotations should be deprecated 
and eventually be removed. This opens the design space to use quotes for 
different purposes, for example getting rid of the cumbersome need to 
use Literal for literals.


One thing we’re thinking of specifically for the future import, and 
for other deprecations in Python, is to revisit the deprecation and 
warning policy. We think it’s pretty clear that the policy we have 
right now doesn’t exactly work. We used to have noisy 
DeprecationWarnings, which were confusing to end users when they were 
not in direct control of the code. We now have silent-by-default 
DeprecationWarnings, where the expectation is that test frameworks 
surface these warnings. This avoids the problem of end users being 
confused, but leaves the problem of the code’s dependencies triggering 
the warning, and thus still warns users (developers) not necessarily 
in a position to fix the problem, which in turn leads to them 
silencing the warning and moving on. We need a better way to reach the 
users in a position to update the code.



+1


1.

If we do need a warning, how loud, and how long should it be
around? At the end of the deprecation period, should the future
import be an error, or simply be ignored?

The future import should remain active and should continue to work as it 
does now (without warning) as long as there are Python versions 
supported that have not implemented PEP 649, i.e. Python 3.10. 
Otherwise, existing code that provides supports for these older versions 
(especially libraries) have to regress their typing support. For 
example, a library supporting 3.8+ could have the following code:


from __future__ import annotations

def foo(x: int | str): pass

If newer versions stop supporting the future import or warn about it, 
the library would have to go back to using typing.Union here. Other 
constructs would even be impossible to use.


After that, I would recommend to start the normal "warn"/"remove" cycle 
for the future import. Don't keep it around if it's doing nothing.


 - Sebastian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RHSQFTTCTPIH4X3XCQBYXFSAO3I6CTBN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Proposal: Allow non-default after default arguments

[Sebastian Rittau]

Currently, Python doesn't allow non-default arguments after default 
arguments:


>>> def foo(x=None, y): pass
  File "", line 1
    def foo(x=None, y): pass
 ^
SyntaxError: non-default argument follows default argument

I believe that at the time this was introduced, no use cases for this 
were known and this is is supposed to prevent a source of bugs. I have 
two use cases for this, one fringe, but valid, the other more important:


The fringe use case: Suppose you have a function that takes a 2D 
coordinate value as separate "x" and "y" arguments. The "x" argument is 
optional, the "y" argument isn't. Currently there are two ways to do 
this, none of them particularly great:


def foo(y, x):  # reverse x and y arguments, confusing
    ...
def foo(x, y=None):  # Treat the x argument as y if only one argument is 
provided

    if y is None:
    x, y = y, x
    ...

To me, the "natural" solution looks like this:

def foo(x=None, y): ...
# Called like this:
foo(1, 2)
foo(y=2)

This could also be useful when evolving APIs. For example there is a 
function "bar" that takes two required arguments. In a later version, 
the first argument gains a useful default, the second doesn't. There is 
no sensible way to evolve the API at the moment.


The more important use case involves @overloads. A condensed example of 
a function where the return type depends on an "encoding" parameter, 
followed by further parameters that could be called like this:


foo(123, "utf-8")  # would return bytes
foo(encoding="utf-8")
foo(123, None)  # would return str
foo(encoding=None)
foo(x=123)  # would return str
foo()

This could ideally be written as:

@overload
def foo(x: int = ..., encoding: None = ...) -> str: ...
@overload
def foo(x: int = ..., encoding: str) -> bytes: ...
# plus the actual implementation

But due to the syntax constraint, this needs to be hacked around with a 
third overload:


@overload
def foo(x: int = ... encoding: None = ...) -> str: ...
@overload
def foo(x: int, encoding: str) -> bytes: ...  # for foo(123, "utf-8")
@overload
def foo(*, encoding: str) -> bytes: ...  # for foo(encoding="utf-8")

Not only is this hard to read, real examples in typeshed are usually 
more complex, with many arguments before or after the affected argument 
or even multiple affected arguments. This often becomes too complex to 
write or maintain. Here is one example from the wild: 
https://github.com/python/typeshed/blob/b95b729b9e07ab21d252701af0f5b7404672b952/stubs/redis/redis/client.pyi#L51


Allowing non-default arguments after default arguments would solve both 
use cases above and eliminates a special case. I'm also not sure what 
exactly the current SyntaxError really protects us from. Adding a 
non-default after a default argument can't really lead bugs.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7W5RMJVBRN6NE6A3LUP5Y4BMOZKQRYWH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Sebastian Rittau]

Am 09.11.21 um 10:50 schrieb Chris Angelico:

On Tue, Nov 9, 2021 at 8:38 PM Sebastian Rittau  wrote:

Currently, Python doesn't allow non-default arguments after default
arguments:

  >>> def foo(x=None, y): pass
File "", line 1
  def foo(x=None, y): pass
   ^
SyntaxError: non-default argument follows default argument

I believe that at the time this was introduced, no use cases for this
were known and this is is supposed to prevent a source of bugs. I have
two use cases for this, one fringe, but valid, the other more important:

The fringe use case: Suppose you have a function that takes a 2D
coordinate value as separate "x" and "y" arguments. The "x" argument is
optional, the "y" argument isn't. Currently there are two ways to do
this, none of them particularly great:

def foo(y, x):  # reverse x and y arguments, confusing
  ...
def foo(x, y=None):  # Treat the x argument as y if only one argument is
provided
  if y is None:
  x, y = y, x
  ...

To me, the "natural" solution looks like this:

def foo(x=None, y): ...
# Called like this:
foo(1, 2)
foo(y=2)

This could also be useful when evolving APIs. For example there is a
function "bar" that takes two required arguments. In a later version,
the first argument gains a useful default, the second doesn't. There is
no sensible way to evolve the API at the moment.


What would this mean, though:

foo(2)

Is that legal?


No. This would be equal to foo(x=2) (same as now), meaning the required 
argument "y" is missing.



I would instead recommend making the parameters keyword-only, which
would allow any of them to have defaults or not have defaults. In
terms of useful API design, this is usually more helpful than having
an early parameter omitted.


This might be better API design (although I don't think Python should be 
opinionated about this outside the standard library), but this still 
leaves the API change example and the very real problem of @overloads 
unsolved.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SNH7R36XOG2VZHKJJ6ENN5GR4Y5NULP3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Sebastian Rittau]

Am 09.11.21 um 13:44 schrieb Petr Viktorin:
And for the "encoding" case: IMO, varying the return type based on an 
optional "encoding" argument" is a holdover from the pre-typing era, 
when return types were only specified in the documentation -- just 
like "addch" is a holdover from the days when function signatures were 
only described in the docs. Nowadays, I'd consider it bad API design. 
The @overloads are ugly but they work -- just like the API itself. IMO 
we shouldn't add special cases to encourage more of it.


"encoding" arguments might be the most common case, but it's certainly 
not limited to that. And the fact remains that however much we desire it 
to be different, there are loads of APIs out there, both in the standard 
library and in (well received and not so well received) third-party 
libraries that use these constructs. This is not going to change. Also, 
API design is always in the eye of the beholder. Personally I prefer my 
example foo(x=None, y) to the design of range() of using some 
*args/**kwargs hacks as was suggested.


I don't believe Python's users are best served by artificially limiting 
possible API design, especially if it causes genuine problems (as it in 
the case of @overload).


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2FRLNZDC3EUEARCGKAWDCWC66V24L4YK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal: Allow non-default after default arguments

[Sebastian Rittau]

Am 09.11.21 um 19:26 schrieb Terry Reedy:
The signature of Sebastian's function with honest parameter names is 
foo(x_or_y, required_y=_NotGiven, /).  It is the 2nd argument, not the 
first, that is optional, as with range.  If required_y is not given, 
than x_or_y must be y, and x is given a default that is not part of 
the signature because it is explicitly bound when called. If 
required_y *is* given, then x_or_y can be x. 


Just to clarify: This proposal works differently than how range() works. 
foo(3) would be illegal as the required second parameter ("y") is 
missing. This can be solved by either supplying both "x" and "y", e.g. 
foo(None, 3), or by using a named parameter for "y": foo(y=3). Therefore 
the honest names are foo(x=None, y) in my proposal.


 - Sebastian


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MN24EZEYEWUSC2WM2SKXR6BYK3Z3XH4Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Sebastian Rittau]

Am 26.11.21 um 01:48 schrieb Rob Cliffe via Python-Dev:
ISTM that typing/annotations have been allowed to drift without a 
clear end in view, rather like a ship that is steered in one direction 
by one person, then in another by someone else, with nobody knowing 
what the destination port is.


I believe the problem is less within the typing community itself 
(whether static or runtime), but a fundamental conflict between the 
typing community and what Stephen called the "typing-suspicious crowd". 
typing in Python has always been hampered by sometimes valid, often 
unfounded fears that typing is "taking over" and somehow becomes a 
requirement. For example, PEP 484 made no changes to Python or the 
stdlib, except the introduction of a new module, often to the detriment 
of readability and usability of type annotations. And while progress is 
made slowly (for example with generics in standard collections or the 
new callable syntax proposal), I'm sure the typing community would love 
to progress faster and with less obstacles.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HC43AFLA2RCTBXZ3K5O2IIYGPCMYQXOX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Sebastian Rittau]

Am 30.11.21 um 13:39 schrieb Oscar Benjamin:

Others have mentioned the pressure on libraries to adopt typing and
I've certainly noticed this with SymPy. I think type hints could be
good for SymPy for internal use but it seems that a lot of users want
it for external reasons that I don't always understand but that also
seems to come partly from editors as well.


Please note that users of you library usually won't care that the 
library uses type hints. It's more important that there are type hints 
for the API, which can also be supplied using a stub file.



Some people apparently want to add type hints that look completely
useless to me like
 def f() -> Union[MyClass, Any]
As I understand it this does not give any meaningful information to a
type checker but it apparently makes vscode work better:
https://github.com/sympy/sympy/pull/22180#discussion_r718917577


We have several resources that can help with questions like these:

 * https://github.com/python/typing/discussions is a forum for
   questions about typing.
 * The forum also has a separate section where users can ask for
   reviews of type annotations, for example in stub files.
 * https://gitter.im/python/typing is a chat for typing-related questions.

We are also working on a documentation hub at 
https://typing.readthedocs.io/, but there is not much to see at the moment.



There are other open "issues" like this for SymPy where the
presumption is that not having type hints is now to be considered a
deficiency of the library regardless of whether the hints have any
benefit for internal use. I don't object to adding the hints but it's
a huge amount of work that someone would have to do and I don't want
to add useless/inaccurate hints temporarily (as some have suggested to
do).


Please note that users of you library usually won't care that the 
library uses type hints. It's more important that there are type hints 
for the API, which can also be supplied using a stub file. This is 
usually quite a bit easier than typing an existing code base. 
Alternatively, just adding type annotations to the public API, but not 
using type checking yourself, can be useful for your users.


 - Sebastian


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PUV2ODPSN27JN6SXLVUSY3O54G7H2ONB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The current state of typing PEPs

[Sebastian Rittau]

Am 01.12.21 um 13:36 schrieb Paul Moore:

On Wed, 1 Dec 2021 at 12:08, Sebastian Rittau  wrote:


Please note that users of you library usually won't care that the library uses 
type hints. It's more important that there are type hints for the API, which 
can also be supplied using a stub file.

I tried that route, but I was informed that if I do that, mypy will
not check my stubs against the source code. Which means that there's
no easy way of testing that the stubs are correct - is that accurate?


mypy includes a "stubtest" tool that compares a stub to introspected 
runtime data. We use it in typeshed as part of the CI process and it's a 
very powerful tool to help keeping stubs up-to-date and correct.




PS I appreciate the links you posted to various typing forums, but IMO
the most critical missing resource is a central set of typing
documentation that includes examples, FAQs and best practices as well
as reference materials.


Completely agree. This is what typing.readthedocs.io is supposed to 
become, but time constraints mean that it's very incomplete at the moment.



PPS Sorry if this sounds negative. TBH, I'd quite happily not use
typing if I didn't want to and stay quiet. A lot of the frustration I
see being expressed here (including my own) seems to come from the
fact that it's so difficult to actually take that sort of "I can
ignore it if I don't use it" attitude, whether that's because of
community pressure, tool requirements, or whatever.


I completely understand this pressure, especially for library authors. 
Providing high quality stubs and the best user experience is not easy. 
But I believe that referring people to typeshed can help. While we of 
course prefer high quality stubs or type annotations shipped with the 
package in question, typeshed can provide a fairly low barrier of entry 
for projects that don't have the resources to maintain type annotations 
themselves. It can also be used as an "incubator", where stubs are 
created and improved iteratively, until they are deemed ready for 
inclusion in an upstream package.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3Z7MCNNQUH3S5IWXA5MXVAUKPYYWKFZO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Request to revert unittest and configparser incompatible changes in Python 3.11

[Sebastian Rittau]

Am 18.01.22 um 22:57 schrieb Victor Stinner:

At the end of my first email, I also suggest thinking about
incompatible changes differently, try to make affected projects
compatible in advance. The problem are not the changes themselves, but
how they are introduced in Python, and more globally how they are
introduced "in the Python ecosystem" (!).

I believe that some (semi-) automated way to actively test and notify 
important projects of deprecations/removals before a release would be a 
great addition to the Python ecosystem. This is a complicated issue, but 
well worth it. (I'm not volunteering. :) )


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4QVAHO4U5QC44QWMJNCONNNE5HVDQPX3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Increase of Spammy PRs and PR reviews

[Sebastian Rittau]

Am 01.02.22 um 01:31 schrieb Nikita Sobolev:

Hi, my name is Nikita and I think that I am the person behind these spammy PRs.
Link: https://github.com/python/cpython/pulls/sobolevn


As a typeshed maintainer, Nikita also "spams" typeshed with PRs. I 
highly appreciate those PRs, which I am sure take a lot of effort, as 
they improved the typeshed annotations significantly over the last few 
months. Besides more substantial PRs, Nikita also submits a lot of PRs 
that improve and modernize small bits and pieces that needed 
improvement, but that no one found the time yet to tackle. I am glad 
that the PRs are fairly small and therefore easy to review. I can't 
speak to the situation of CPython, though.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z45WOV7G5ESMKRB7GANHU5UKG2LQ4TPD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

[Sebastian Rittau]

Am 03.02.22 um 22:41 schrieb Gregory P. Smith:
datapoint: an internal code search turns up blender, multidict, and 
typed_ast as open source users of _Py_IDENTIFIER .  Easy to clean up 
as PRs.  There are a couple of internal uses as well, all of which are 
similarly easy to address and are only in code that is expected to 
need API cleanup tweaks between CPython versions.


typed_ast is mostly legacy, to be used for parsing Python code < Python 
3.8. It's end of life is scheduled when Python 3.7 end of lifes in June 
2023. I believe we can easily get away with just copying the 
_Py_IDENTIFIER macros over to its source code in typed_ast.


 - Sebastian


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NYRNJRDSRPSABNSSQCU7NGJLUTZPENGM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: New PEP website is horrible to read on mobile device

[Sebastian Rittau]

Am 15.03.22 um 19:40 schrieb Zachary Ware:

On Tue, Mar 15, 2022 at 1:31 PM Nathan Cook  wrote:

Please make https://peps.python.org/ more responsive to various form factors

See attached screenshot from Chrome version 99.0.4844.58 on my Pixel 3aXL 
running Android 12

I can't reproduce this without zooming in.  If you're still convinced
there's a problem with the site itself, please open an issue at
https://github.com/python/peps/issues.


And just to confirm: With nearly the same phone (Pixel 3, but not XL) 
and the same software versions everything looks fine for me.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EOX3T2XYHCUNOJ2TRSUIUW2EPW6WOYHB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Sebastian Rittau]

Am 27.03.22 um 18:11 schrieb Christopher Barker:

On Sun, Mar 27, 2022 at 3:08 AM Paul Moore  wrote:

> > 3. Overall, I think the days where "battery included" was a
positive argument are over
>
> I strongly disagree.  Being able to download something and
immediately get something to work and see results is hugely
> rewarding; on the other hand, having to research, find, compare
& contrast available third-party modules (especially for
> new-comers) can be extremely discouraging.


exactly - let's say someone needs to write some JSON for the first time.


I think the truth is somewhere in the middle. Python should include 
support for commonly needed functionality. This includes all kinds of 
language support (functools, collections, asyncio, wsgiref, re etc.), 
common file formats like XML, JSON, tar, zip etc., OS support (os, sys, 
stat etc.), basic networking (TCP/IP, making HTTP requests, sending 
mail), but also basic math modules like decimal, fractions, or 
statistics. I don't think it should support esoteric, niche, or long 
obsolete file formats or networking protocols like FTP, sunau etc. I 
also don't think that Python should come with "basic" servers that can't 
be used for anything useful. The same is true for domain-specific 
applications. For example, I wouldn't want a complex numerics package as 
part of Python (although having some more basic data types would make 
sense).


Another problem are "optional" modules like sqlite or tk/tcl. Because 
they are optional, you can't depend on them being present, but you also 
can't ensure their presence by "pip install"ing them.


In fact, this is an example, I think, of where we should put some 
effort into making the included batteries better -- it's great to have 
a JSON lib built in, but it's too bad that it's not best-of-bread by 
pretty much any definition (speed, memory performance, flexibility) -- 
there are quite a few feature requests open for it -- it would be nice 
to actually implement some of those. (but yes, that's a lot of work 
that someone(s) would have to do)


Back to the topic at hand, rather than remove urllib, maybe it could 
be made better -- an as-easy-to-use-as-requests package in the stdlib 
would be really great.


I agree 100%. On the one hand the stdlib is missing some functionality I 
would consider "basic" (for example, async file and HTTP fetching 
support), on the other hand some of the existing modules would benefit 
from a more modern API that makes the common case easy and the uncommon 
case possible.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RZFGUDPO2DIASHI72GUF2NP3IR3DLQGO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python3 -bb and hash collisions

[Sebastian Rittau]

Am 11.09.19 um 15:34 schrieb Daniel Holth:


It's different. One hint is that there's already an option to disable 
the feature. The old style of error will occasionally reveal itself 
with decode errors but the new style error happens silently, you 
discover it somehow, then enable the -bb option, track down the source 
of the error, and deal with the fallout. The proposed change would 
allow `print(bytes)` for (de)bugging by letting you toggle `python3 
-bb` behavior at runtime instead of only at the command line. Or you 
could debug more explicitly by `print(bytes.decode('ebcdic'))` or 
`print(repr(bytes))`


I didn't realize you could override __builtins__.str. That's interesting.


Being able to call str() on everything is such a fundamental assumption 
that changing the behavior of str(bytes) would break Python. Porting 
from Python 2 to Python 3 is a big task and especially the 
str/unicode/bytes handling needs extra care, and this is one of those 
corner cases that might prove problematic when porting. That doesn't 
justify breaking Python, especially not for those users that have 
decided to port to Python 3 in a timely manner.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PUDMIH55ZCVWNWLTCIC4PLYJM2XZSN22/

[Python-Dev] Re: PEP 616 -- String methods to remove prefixes and suffixes

[Sebastian Rittau]

Am 26.03.20 um 06:28 schrieb Cameron Simpson:

On 24Mar2020 18:49, Brett Cannon  wrote:

-1 on "cut*" because my brain keeps reading it as "cute".
+1 on "trim*" as it is clear what's going on and no confusion with 
preexisting methods.

+1 on "remove*" for the same reasons as "trim*".


I reiterate my huge -1 on "trim" because it will confuse every PHP 
user who comes to us from the dark side. Over there "trim" means what 
our "strip" means.


I've got (differing) opinions about the others, but "trim" is a big 
one to me. 


As a full stack developer with terrible memory, I agree. JavaScript also 
uses trim() like Python's strip(), and this would quickly get confusing.


 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RNVBORUJDVWBQPELPSPYMNHBVLCZ4B5B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Duck-typing self

[Sebastian Rittau]

Hi!

I am curious why the following will not work in Python:

  class foo(object):
  def bar(self):
  print self.attr
  
  class duck(object):
  attr = 3.14
  
  foo.bar(duck())

Is it a design decision that duck-typing self does not work or is there a
technical reason? From a practical standpoint it seems that being able to
duck-type self has merit, for example in unit testing complex classes.

 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Duck-typing self

[Sebastian Rittau]

On Wed, Feb 18, 2009 at 11:32:09PM +0100, Sebastian Rittau wrote:

> I am curious why the following will not work in Python:
> 
>   class foo(object):
>   def bar(self):
>   print self.attr
>   
>   class duck(object):
>   attr = 3.14
>   
>   foo.bar(duck())

Thanks to everybody who has responded on-list or in private. Actually
Guido blogged about this design decision (among other things) in his
latest post to the "The History of Python" blog:
<http://tinyurl.com/c6qya5>.

 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Getting values stored inside sets

[Sebastian Rittau]

Hello,

On Fri, Apr 03, 2009 at 02:07:02PM +0200, Hrvoje Niksic wrote:

> But I can't seem to find a way to retrieve the element corresponding to  
> 'foo', at least not without iterating over the entire set.  Is this an  
> oversight or an intentional feature?  Or am I just missing an obvious  
> way to do this?

I am missing a simple way to retrieve the "first" element of any
iterable in python that matches a certain condition anyway. Something
like this:

  def first(iter, cb):
  for el in iter:
  if cb(el):
  return el
  raise IndexError()

Or (shorter, but potentially slower):

  def first(iter, cb):
  return [el for el in iter if cb(el)][0]

To be used like this:

  my_el = first(my_set, lambda el: el == "foobar")

This is something I need from time to time and this also seems to solve
your problem.

 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3144 review.

[Sebastian Rittau]

On Tue, Sep 15, 2009 at 01:16:06PM -0400, Scott Dial wrote:

> I have to concur with the opinions above. I was very confused by the
> following error:
> 
> >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0")
> ...
> ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not
> a valid address (hint, it's probably a network)
> 
> Because, it *is* a address of a host on a network.

To me, 10.1.2.3/255.255.240.0 is not a host address, but specifies a
network. I.e., 10.1.2.3/255.255.240.0 == 10.1.0.0/255.255.240.0 ==
10.1.35.200/20.

> >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0")
> 
> But then, I was dumbfounded as to how I could get the gateway IP from
> this IPNetwork object.

Well, you can't. There is no way to determine a gateway, without querying
the network topology. This is clearly outside the scope of this module.
The only two known host addresses of a network are the network address
(10.1.0.0 in the example you gave) and the broadcast address (10.1.15.255).

> It took me a while to figure out that you can
> iterate over IPNetwork instances:
> 
> >>> gateway = net[1]
> 
> I was then confused, because:
> 
> >>> print(type(gateway))
> 
> 
> Which sorta blew my mind.. I fully expected to receive an IPNetwork back
> from that operation. It is unclear to me why the network information
> gets chucked by that operation.

This makes perfect sense to me. An IP network consists of a list of IP
addresses. Returning /32 networks seems kind of pointless to me.

> I foresee having to work around that in
> real applications by doing something obnoxious like:
> 
> >>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask))

But a gateway is not an IP address plus hostmask. A gateway is just a single
IP address. What is the use of adding a hostmask to the gateway IP address
or some other IP address inside the network?

 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] conceptual clarity

[Sebastian Rittau]

On Thu, Sep 17, 2009 at 02:04:11PM -0400, R. David Murray wrote:

> I mean, eg, IPv4Network.fromHostIP('192.168.1.1/24').

I'd actually suggest to use

  >>> net, host = parse_network_and_host("192.168.111.33/24")
  (IPv4Network('192.168.111.0/24'), IPv4Address('192.168.111.33'))
  >>> 

I think this helps the use case of the short network+gateway notation,
while keeping the concepts of network and host address cleanly separate.

> I would have IPv4Address itself be strict, and thus the new constructors
> would compute the network address and call the regular IPv4Address
> constructor.(*)

Then the regular IPv4Network constructor could be strict and can be used
for validation.

 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3147: PYC Repository Directories

[Sebastian Rittau]

On Sun, Jan 31, 2010 at 12:44:33PM +0100, Georg Brandl wrote:

> At least to me, this does not explain why an "unwanted" (why unwanted? If
> it's unwanted, set PYTHONDONTWRITEBYTECODE=1) directory is worse than an
> "unwanted" file.

A directory "feels" different than. For example, typing "ls" in my shell
regular print files in black, but directories in bold and blue. File
managers and IDE also highlight directories differently. In tree views,
directories have expander buttons that also make them stand out.

As a concrete example, have a look at these two screenshots:

  http://tinyurl.com/yz2fr6c and http://tinyurl.com/yg38uqt

In the first one, the subpackages stand out, while in the second one they
are hard to make out among the *.pyr directories. A directory just adds
more clutter than a file.

But overall I like the idea of using just a single __pycache__ or
__pyr__ directory per path. This would also reduce the *.pyc clutter.

 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] unittest Suggestions

[Sebastian Rittau]

[I just saw the other post about unit testing, while I was writing this.
A strange conincidence.]

Hello,

since this is the first time I post to this list, I'd like to introduce
myself briefly. As you can see my name is Sebastian. I am working as a
software developer both as a professional and as a hobbyist. During the
years I've used many programming languages, but in the last few years
I've fallen in love with Python. (Blame Martin v. Löwis for that - I
started using Python after participating in a seminar held by him at
university.)

I'd like to propose a few changes to the unittest module. These changes
are supposed to ease usage, enable future changes and enhancements, and
bring unittest more in line with other modern xUnit variants (such as
JUnit 4 and NUnit). I have implemented most of those changes in a custom
library that hacks around the current unittest behaviour. I've published
the relevant package at [1]. All proposed changes are (mostly) backward
compatible.

Change unittest into a package
--

Currently unittest is a module. Changing it into a package would enable
us to separate concerns and move code out into separate modules. For
example the unittest.asserts proposal below requires this change. If a
mocking framework would be included in the Python standard library at
some point, it could go into unittest.mock etc.

@test, @before, and @after decorators
-

Currently unittest uses "magic" names starting with "test_" for detecting
the test methods in a test case class. Following the "explicit is better
than implicit" guideline, I propose to add a @test decorator to
explicitly mark test methods:

  class MyTest(TestCase):

  @test
  def attributes(self):
  """Do stuff."""

I've been bitten in the past by naming methods test_xyz that were not
supposed to be called by the test runner. For now those methods still
need to be recognized, since they are widely used. But enabling (and
recommending) an alternative is a first step.

The decorator syntax makes is possible to introduce alternative
decorators as well, like @disabled_test. In the example package I
included a @performance_test decorator that takes a time argument where
the test fails if it takes more than a given amount of time.

Similarily I recommend to introduce @before and @after decorators to
supplement the setUp and tearDown methods. The decorators were named
after the decorators in JUnit 4, although I wonder whether @setup and
@teardown would be a better name. @before and @after methods are called
in any order, but @before methods in a super class are guaranteed to be
called before @before methods in a derived class, while it's vice versa
with @after methods. Introducing these has the added benefit of being
able to get rid of the mixedCase setUp and tearDown methods in my own
code.

unittest.asserts


Currently all assertions reside as methods on the TestCase class. This
means that all testing functions, including utility functions, need to be
on a TestCase-derived class. This seems wrong to me, especially since it
combines different concerns on a class: Testing the functionality of a
SUT, running a test suite, and providing test functions. This proposal
will not try to separate the first two concerns, but only the third one.
In the long term I think it's worthwhile to make tests work on any
class, not only on classes derived from TestCase, but as I said, this is
not part of this proposal.

JUnit 4 moved the assertions to a static class in a separate module. In
Python I'd propose to use functions in a separate module. A typical test
suite would then look like this:

  from unittest import TestCase
  from unittest.asserts import *

  class FooTest(TestCase):

  """Tests for the Foo class."""

  @before
  def setup(self):
  self._sut = create_an_object_to_test()

  @test
  def construct__some_attribute(self):
  assert_equals("foobar", self._sut.some_attribute)

  ...

Again, as a side effect this removes inconsistent naming (mixedCase and
three different version of each test) from my tests. There is one
regression, though. Currently it's possible to change the assertion
exception (which defaults to AssertionError) by setting an attribute of
the TestCase instance. I am not sure how relevant that is, though. I've
never used this feature myself, not even when testing the unittest
module and its extensions, and this approach wouldn't work anyway, if
you use classes with utility assertions etc.

Anyway, I am happy about any comments.

 - Sebastian

[1] http://www.rittau.biz/~srittau/unittest, though I will move it to
http://www.rittau.org/python/unittest, once I'm home.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com