[Python-Dev] Re: Remove asyncore, asynchat and smtpd modules

[Petr Viktorin]

On 11. 11. 21 13:31, Victor Stinner wrote:

Hi,

The asyncore module is a very old module of the Python stdlib for
asynchronous programming, usually to handle network sockets
concurrently. It's a common event loop, but its design has many flaws.

The asyncio module was added to Python 3.4 with a well designed
architecture. Twisted developers, who have like 10 to 20 years of
experience in asynchronous programming, helped to design the asyncio
API. By design, asyncio doesn't have flaws which would be really hard
to fix in asyncore and asynchat.

It was decided to start deprecating the asyncore, asynchat and smtpd
modules in Python 3.6 released in 2016, 5 years ago. Python 3.10 emits
DeprecationWarning.


Wait, only Python 3.10?
According to the policy, the warning should be there for *at least* two 
releases. (That's a minimum, for removing entire modules it might make 
sense to give people even more time.)




asynchat and smtpd are implemented with asyncore.
Open issues in asyncore, asynchat and smtpd have been closed as "wont
fix" because these modules are deprecated. These modules are basically
no longer maintained.

I propose to remove asyncore, aynchat and smtpd in Python 3.11 to
reduce the Python maintenance burden, while asyncio remains available
in stdlib and is maintained:

* asyncore and asynchat can be replaced with asyncio
* smtpd can be replaced with aiosmtpd which is based on asyncio:
https://aiosmtpd.readthedocs.io/

If someone wants to continue using asyncore, asynchat or smtpd, it's
trivial to copy Python 3.10 asyncore.py, asynchat.py and smtpd.py to
their project, and maintain these files there. Someone is also free to
continue maintaining these modules as third-party projects on PyPI.

The removal is discussed at:
https://bugs.python.org/issue28533

I wrote a PR to remove the 3 modules:
https://github.com/python/cpython/pull/29521


... in short, the intent is to move the asyncore, asynchat and smtpd
maintenance outside the Pyhon project ;-) (if anyone still use them)


Victor


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2Q22LYRMTYRDSCXXLM2DMVTT3VVRQF5B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Do we need to remove everything that's deprecated?

[Petr Viktorin]

We're rebuilding many popular projects with Python 3.11 alpha, and I see 
many failures like:


  AttributeError: module 'configparser' has no attribute 
'SafeConfigParser'. Did you mean: 'RawConfigParser'?

(bpo-45173)

  ImportError: cannot import name 'formatargspec' from 'inspect'
(bpo-45320)

  AttributeError: '[...]Tests' object has no attribute 'failUnless'
(bpo-45162)

Are these changes necessary?
Does it really cost us that much in maintainer effort to keep a 
well-tested backwards compatibility alias name, or a function that has a 
better alternative?


I think that rather than helping our users, changes like these are 
making Python projects painful to maintain.
If we remove them to make Python easier for us to develop, is it now 
actually that much easier to maitain?


The current backwards compatibility policy (PEP 387) sets a *minimum* 
timeline for deprecations and removals -- "deprecation period must last 
at least two years."
But it seems like it's not treated as a minimum: if any contributor 
sends an issue/PR to remove deprecated functionality, it's merged 
without much discussion. And it's very easy to "solve" these "issues", 
since everything is already marked for deletion; I fear we get the same 
kind of bikeshed/powerplant problem https://bikeshed.com/ for changes 
that explains for discussion. It's just so much easier to do "spring 
cleaning" than solve other problems.


Note that I am criticizing the *process*; the examples I gave have some 
people's names attached, and I have no doubt the people acted with best 
intentions.
I'm also not talking about code that's buggy, insecure, or genuinely 
hard to maintain.



If deprecation now means "we've come up with a new way to do things, and 
you have two years to switch", can we have something else that means 
"there's now a better way to do things; the old way is a bit worse but 
continues to work as before"?


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AYJOQL36SK3EK5VMEKT5L5BVH25HVY4G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Remove asyncore, asynchat and smtpd modules

[Victor Stinner]

> > It was decided to start deprecating the asyncore, asynchat and smtpd
> > modules in Python 3.6 released in 2016, 5 years ago. Python 3.10 emits
> > DeprecationWarning.
>
> Wait, only Python 3.10?
> According to the policy, the warning should be there for *at least* two
> releases. (That's a minimum, for removing entire modules it might make
> sense to give people even more time.)

The PEP 387 says "Similarly a feature cannot be removed without notice
between any two consecutive releases."

It is the case here. The 3 modules are marked as deprecated for 4
releases in the documentation: Python 3.6, 3.7, 3.9 and 3.10. Example:
https://docs.python.org/3.6/library/asyncore.html

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/R5LTSKKEGDDO3TTLLZFDTAKOH3ACBTMV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we need to remove everything that's deprecated?

[Victor Stinner]

> The current backwards compatibility policy (PEP 387) sets a *minimum*
> timeline for deprecations and removals -- "deprecation period must last
> at least two years."

About the PEP 387 process and the 3 examples.

On Fri, Nov 12, 2021 at 11:58 AM Petr Viktorin  wrote:
>AttributeError: module 'configparser' has no attribute
> 'SafeConfigParser'. Did you mean: 'RawConfigParser'?
> (bpo-45173)

SafeConfigParser was not even documented, was deprecated since Python
3.2, and emitted a DeprecationWarning.

>ImportError: cannot import name 'formatargspec' from 'inspect'
> (bpo-45320)

It was deprecated in the doc since Python 3.5, and emitted a DeprecationWarning.

>AttributeError: '[...]Tests' object has no attribute 'failUnless'
> (bpo-45162)

Deprecated in the doc since Python 3.1. It emitted a DeprecationWarning.

> But it seems like it's not treated as a minimum
> (...)
> Note that I am criticizing the *process*

On these examples, the functions were deprecated for way longer than a
minimum of 2 Python versions, no?

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L5K4H5O3ZW5DIA6K5XKOAOL4GHMQL532/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we need to remove everything that's deprecated?

[Petr Viktorin]

On 12. 11. 21 13:51, Victor Stinner wrote:

The current backwards compatibility policy (PEP 387) sets a *minimum*
timeline for deprecations and removals -- "deprecation period must last
at least two years."


About the PEP 387 process and the 3 examples.

On Fri, Nov 12, 2021 at 11:58 AM Petr Viktorin  wrote:

AttributeError: module 'configparser' has no attribute
'SafeConfigParser'. Did you mean: 'RawConfigParser'?
(bpo-45173)


SafeConfigParser was not even documented, was deprecated since Python
3.2, and emitted a DeprecationWarning.


ImportError: cannot import name 'formatargspec' from 'inspect'
(bpo-45320)


It was deprecated in the doc since Python 3.5, and emitted a DeprecationWarning.


AttributeError: '[...]Tests' object has no attribute 'failUnless'
(bpo-45162)


Deprecated in the doc since Python 3.1. It emitted a DeprecationWarning.


But it seems like it's not treated as a minimum
(...)
Note that I am criticizing the *process*


On these examples, the functions were deprecated for way longer than a
minimum of 2 Python versions, no?


Yes. And as far as I know, they haven't really caused problems in all 
that time.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S3BDHURSCE3ODZBFOKEW3P2UWIZPEU7R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we need to remove everything that's deprecated?

[Victor Stinner]

For me, deprecated functions cause me a lot of thinking when I met
them as a Python maintainer and as a Python user. Why is it still
there? What is its purpose? Is there a better alternative? It's
related to the Chesterton's fence principle. Sometimes, reading the
doc is enough. Sometimes, I have to dig into the bug tracker and the
Git history.

In Python, usually, there is a better alternative. A recent example is
the asyncore module that I'm proposing to remove. This module has
multiple design flaws which cause bugs in corner cases. It's somehow
dangerous to use this module. Deprecating the module doesn't help
users who continue to use it and may get bugs in production. Removing
the module forces user to think about why they chose asyncore and if
they can switch to a better alternative. It's supposed to help users
to avoid bugs.

The gray area is more about "deprecated aliases" and having two ways
to do the same things, but one way is deprecated. One example is the
removal of collections.MutableMapping: you must now use
collections.abc.MutableMapping. Another example is the removal the "U"
mode in the open() function: the flag was simply ignored since Python
3.0. So far, the trend is to remove these "aliases" and force users to
upgrade this code. Not removing these aliases has been discussed, and
it seems like each time, it was decided to remove them. Usually, the
"old way" is deprecated for many Python versions, like 5 years if not
longer.

Using deprecated functions is a threat in terms of technical debt. An
application using multiple deprecated functions will break with a
future Python version. It's safe to avoid deprecated functions
whenever possible. Some deprecated functions have been removed but
then restored for 1 or 2 more Python releases, to give more time to
users to upgrade their code. At the end, the deprecated code is
removed.

We can warn developers to pay attention to DeprecationWarning
warnings, but sadly, in my experience, the removal is the only trigger
which works for everybody.

Do you have to repeat "You should check for DeprecationWarning in your
code" in every "What's New in Python X.Y?" document? Python 3.9 has
such section:
https://docs.python.org/dev/whatsnew/3.9.html#you-should-check-for-deprecationwarning-in-your-code

Victor

On Fri, Nov 12, 2021 at 11:58 AM Petr Viktorin  wrote:
>
> We're rebuilding many popular projects with Python 3.11 alpha, and I see
> many failures like:
>
>AttributeError: module 'configparser' has no attribute
> 'SafeConfigParser'. Did you mean: 'RawConfigParser'?
> (bpo-45173)
>
>ImportError: cannot import name 'formatargspec' from 'inspect'
> (bpo-45320)
>
>AttributeError: '[...]Tests' object has no attribute 'failUnless'
> (bpo-45162)
>
> Are these changes necessary?
> Does it really cost us that much in maintainer effort to keep a
> well-tested backwards compatibility alias name, or a function that has a
> better alternative?
>
> I think that rather than helping our users, changes like these are
> making Python projects painful to maintain.
> If we remove them to make Python easier for us to develop, is it now
> actually that much easier to maitain?
>
> The current backwards compatibility policy (PEP 387) sets a *minimum*
> timeline for deprecations and removals -- "deprecation period must last
> at least two years."
> But it seems like it's not treated as a minimum: if any contributor
> sends an issue/PR to remove deprecated functionality, it's merged
> without much discussion. And it's very easy to "solve" these "issues",
> since everything is already marked for deletion; I fear we get the same
> kind of bikeshed/powerplant problem https://bikeshed.com/ for changes
> that explains for discussion. It's just so much easier to do "spring
> cleaning" than solve other problems.
>
> Note that I am criticizing the *process*; the examples I gave have some
> people's names attached, and I have no doubt the people acted with best
> intentions.
> I'm also not talking about code that's buggy, insecure, or genuinely
> hard to maintain.
>
>
> If deprecation now means "we've come up with a new way to do things, and
> you have two years to switch", can we have something else that means
> "there's now a better way to do things; the old way is a bit worse but
> continues to work as before"?
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/AYJOQL36SK3EK5VMEKT5L5BVH25HVY4G/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/

[Python-Dev] Re: Do we need to remove everything that's deprecated?

[Petr Viktorin]

On 12. 11. 21 14:18, Victor Stinner wrote:

For me, deprecated functions cause me a lot of thinking when I met
them as a Python maintainer and as a Python user. Why is it still
there? What is its purpose? Is there a better alternative? It's
related to the Chesterton's fence principle. Sometimes, reading the
doc is enough. Sometimes, I have to dig into the bug tracker and the
Git history.


Could you just add a comment when you find the answer? And a note in the 
docs, for the users?



In Python, usually, there is a better alternative. A recent example is
the asyncore module that I'm proposing to remove. This module has
multiple design flaws which cause bugs in corner cases. It's somehow
dangerous to use this module. Deprecating the module doesn't help
users who continue to use it and may get bugs in production. Removing
the module forces user to think about why they chose asyncore and if
they can switch to a better alternative. It's supposed to help users
to avoid bugs.


Right. If something's an attractive-looking trap, that's a reasonable 
reason to think about removing it.

But I'm not talking about that there.



The gray area is more about "deprecated aliases" and having two ways
to do the same things, but one way is deprecated. One example is the
removal of collections.MutableMapping: you must now use
collections.abc.MutableMapping. Another example is the removal the "U"
mode in the open() function: the flag was simply ignored since Python
3.0. So far, the trend is to remove these "aliases" and force users to
upgrade this code. Not removing these aliases has been discussed, and
it seems like each time, it was decided to remove them. Usually, the
"old way" is deprecated for many Python versions, like 5 years if not
longer.

Using deprecated functions is a threat in terms of technical debt. An
application using multiple deprecated functions will break with a
future Python version. 


But "will break with a future Python version" just means that people's 
code breaks because *we break it*. If we stopped doing that (in the 
simple cases of name aliases or functions that are older but not 
dangerous), then their code wouldn't break.




It's safe to avoid deprecated functions
whenever possible. Some deprecated functions have been removed but
then restored for 1 or 2 more Python releases, to give more time to
users to upgrade their code. At the end, the deprecated code is
removed.

We can warn developers to pay attention to DeprecationWarning
warnings, but sadly, in my experience, the removal is the only trigger
which works for everybody.

Do you have to repeat "You should check for DeprecationWarning in your
code" in every "What's New in Python X.Y?" document? Python 3.9 has
such section:
https://docs.python.org/dev/whatsnew/3.9.html#you-should-check-for-deprecationwarning-in-your-code


Clearly, that's not working. Python users want to write commits that 
either bring value, or that are fun. Mass-replacing "failUnless" with 
"assertTrue" just because someone decided it's a better name is neither.
Same with a forced move to the latest version of a function, if you 
don't use the bells and whistles it added.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IRBNWJ23KTL7YCSHQ6IMNHQPNDS3AU63/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Paul Ganssle]

I think Richard's point was two-fold: People usually don't want or need 
this kind of thing /except/ when they have some very specific 
performance requirements, in which case they probably want to also be 
very specific about the kind of container they are using rather than 
using an abstract "sorted container".


It is true that people sensitive to performance may really care about 
the way dict is implemented, but there are a great many uses for 
associative arrays in general. I knew about sortedcontainers and I also 
don't remember ever seeing a situation where I needed one or recommended 
its use. Maybe it would be useful in leetcode-style interview questions 
(which may by itself be a good reason to include it in the standard 
library, since a lot of those interviewers will let you use 
`collections.deque` or `collections.OrderedDict` without implementing 
any sort of linked listed, but they won't let you import a module that 
provides a trie or something)?


I'm fairly neutral on this proposal. I do think the standard library is 
a good place for these sorts of fundamental data structures (even 
abstract ones), but I do agree with Richard that they are pretty niche. 
In almost any situation where you'd want / need something like this, I 
feel like adding a PyPI dependency would not be a big deal.


One thing that I do think might be valuable is if the plan were to 
re-implement these as C extensions. Maintaining and distributing C 
extensions on PyPI is in many ways more difficult than bundling them 
directly into CPython. That said, doing so adds maintenance burden and 
implementation cost (and it's a different proposition than merely 
adopting an existing library), so I'm probably -0.0 on the whole thing — 
the absence of these types of containers in the standard library is not 
an obvious lack, and getting them from PyPI in the situations where you 
actually need them doesn't seem like a big deal, particularly when it's 
a pure Python impementation.


On 11/11/21 21:43, Steven D'Aprano wrote:

On Thu, Nov 11, 2021 at 11:01:32AM -0800, Richard Levasseur wrote:


Should the stdlib have e.g. SortedList? Probably not, because the use cases
of such data types are too niche to a one-size-fits-all implementation, and
there are too many implementations with too many of their own settings.
Should it have e.g. BTree, RedBlackTree, SortedArrayList, etc? Probably so,
because those are somewhat fundamental data structures and implementing,
testing, and verifying them is very much non-trivial. While niche, having
them at your immediate disposal is very powerful.

By that reasoning, we shouldn't have dicts, for exactly the same reason:
for anyone wanting an associative array, there are so many implementation
variants to choose from:

- hash table with linear probing
- hash table with chaining
- AVL tree
- red-black tree
- judy array
- splay tree
- treap
- scapegoat tree

and many more, and most of them can be tuned.

If Python didn't already have dict, your argument against SortedList
would equally apply to it: they are "fundamental data structures and
implementing, testing, and verifying them is very much non-trivial".

So if your argument is correct, that would imply that standardizing on
one single dict implementation, one that isn't even tunable by the
caller, was a mistake. We should have provided a dozen different hash
tables, trees and treaps.

But... we didn't, and I don't think that Python is a worse language
because we only have one associative array implementation in the stdlib.

Whenever I need an associative array, I don't lose sleep over whether I
could get 2% better performance for hits, at a cost of 17% worse
performance for misses, by swapping over to some other implementation. I
just reach for dict, knowing that it will almost always be good enough.



Last year, for fun, after wishing there was a SortedSet in the standard
lib, I ended up implementing a Red-Black Tree and BTree based sorted
dictionary/set[1]. After then trying to use them for my use case[2], I
found that, in order to fully and truly exploit their benefits, the basic
Sequence/Collection/Set/Dict APIs didn't really suffice. I needed APIs that
would let me, e.g. binary search to a particular spot and then iterate, or
give me a range between two points, etc.

I believe that sortedcontainers.SortedSet provides that functionality
via the irange() method.

http://www.grantjenks.com/docs/sortedcontainers/sortedlist.html#sortedcontainers.SortedList.irange

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HI3IW4XEANYTUOEWVRMH2WYZLGBSQEPD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Steven D'Aprano]

On Fri, Nov 12, 2021 at 10:07:13AM -0500, Paul Ganssle wrote:

> I knew about sortedcontainers and I also don't remember ever seeing a 
> situation where I needed one or recommended its use.

We have a very odd situation where apparently sortedcontainers is one 
of the most well-known, popular, most heavily downloaded libraries on 
PyPI. According to here:

https://hugovk.github.io/top-pypi-packages/

sortedcontainers is the 290th most popular package on PyPI, ahead of 
such luminaries as pylint, black, selenium, mypy, django and nose.

And yet, nobody(?) admits to either using it or knowing what it could be 
used for. How very curious :-/


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7JVLJE4G6ZWM6PKXTMSZL7MVAIUWGZRH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Chris Angelico]

On Sat, Nov 13, 2021 at 3:20 AM Steven D'Aprano  wrote:
>
> On Fri, Nov 12, 2021 at 10:07:13AM -0500, Paul Ganssle wrote:
>
> > I knew about sortedcontainers and I also don't remember ever seeing a
> > situation where I needed one or recommended its use.
>
> We have a very odd situation where apparently sortedcontainers is one
> of the most well-known, popular, most heavily downloaded libraries on
> PyPI. According to here:
>
> https://hugovk.github.io/top-pypi-packages/
>
> sortedcontainers is the 290th most popular package on PyPI, ahead of
> such luminaries as pylint, black, selenium, mypy, django and nose.
>
> And yet, nobody(?) admits to either using it or knowing what it could be
> used for. How very curious :-/
>

I think that proves the value of download counts: very little. The
highest on the list is a thing called botocore, which I've never heard
of. What is it? It's a dependency of a number of Amazon web services.
My guess is that it's a dependency of popular tools - or maybe of
automatically-installed tools, even - while not itself being well
known. Actually, quite a few of the most popular packages on that list
are Amazon-related, so quite possibly they're all being installed as a
set in response to one single *actual* dependency.

(Download counts DO most likely indicate that something is heavily
used, though, so if people are trying to plan out a benchmark suite,
then that might be more useful. But I'm not really surprised that a
heavily-downloaded package isn't itself well known.)

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LF622NRJBFBNHKZR2B7P6X4D2GE4WSDX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Marc-Andre Lemburg]

On 12.11.2021 17:10, Steven D'Aprano wrote:
> On Fri, Nov 12, 2021 at 10:07:13AM -0500, Paul Ganssle wrote:
> 
>> I knew about sortedcontainers and I also don't remember ever seeing a 
>> situation where I needed one or recommended its use.
> 
> We have a very odd situation where apparently sortedcontainers is one 
> of the most well-known, popular, most heavily downloaded libraries on 
> PyPI. According to here:
> 
> https://hugovk.github.io/top-pypi-packages/
> 
> sortedcontainers is the 290th most popular package on PyPI, ahead of 
> such luminaries as pylint, black, selenium, mypy, django and nose.
> 
> And yet, nobody(?) admits to either using it or knowing what it could be 
> used for. How very curious :-/

Those download stats can be misleading. Packages are often pulled in
as a dependency of other packages which are popular or used a lot
in CI/CD setups.

Perhaps there's a reverse dependency graph we could use to find out
why the package is downloaded this often. I remember having seen
a project which does this, but have lost the URL.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 12 2021)
>>> Python Projects, Coaching and Support ...https://www.egenix.com/
>>> Python Product Development ...https://consulting.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   https://www.egenix.com/company/contact/
 https://www.malemburg.com/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FJT4SSWISWWRFB5FGRPXSLDLDZVMUIXF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Bob Fang]

> And yet, nobody(?) admits to either using it or knowing what it could be 
> used for. How very curious :-/

trio (which IMHO is a somewhat high profile uses it):

https://github.com/python-trio/trio/blob/master/trio/_core/_run.py#L27 


And so if I am not mistaken any project that depends on trio will automatically 
need to have sortedcontainers as its dependency? I am not saying this alone 
will explain why sortedcontainers have large number of downloads, but with one 
or two projects like this it will push the number up maybe?

Best,
Bob


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VW6ZJWGZZEBDHN4VQ4RSXJTUEUPOQHOV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Bob Fang]

> And yet, nobody(?) admits to either using it or knowing what it could be 
> used for. How very curious :-/

trio (which IMHO is a somewhat high profile uses it):

https://github.com/python-trio/trio/blob/master/trio/_core/_run.py#L27 


And so if I am not mistaken any project that depends on trio will automatically 
need to have sortedcontainers as its dependency? I am not saying this alone 
will explain why sortedcontainers have large number of downloads, but with one 
or two projects like this it will push the number up maybe?

Best,
Bob

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UWZ2S7UDXIBV4P4UEQIS2ZB7BLBKY6ZX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Damian Shaw]

Yeah, a datapoint I didn't see mentioned is searching on public github
repos for import sortedcontainers (which includes from sortedcontainers
import).

Obviously it's just one datapoint but it shows a very small count compared
to other packages mentioned when looking at many of the stats available:

https://github.com/search?q=import+sortedcontainers+language%3APython&type=code
https://github.com/search?q=import+pylint+language%3APython&type=code
https://github.com/search?q=import+black+language%3APython&type=code
https://github.com/search?q=import+mypy+language%3APython&type=code
https://github.com/search?q=import+django+language%3APython&type=code

Damian


On Fri, Nov 12, 2021 at 11:39 AM Marc-Andre Lemburg  wrote:

> On 12.11.2021 17:10, Steven D'Aprano wrote:
> > On Fri, Nov 12, 2021 at 10:07:13AM -0500, Paul Ganssle wrote:
> >
> >> I knew about sortedcontainers and I also don't remember ever seeing a
> >> situation where I needed one or recommended its use.
> >
> > We have a very odd situation where apparently sortedcontainers is one
> > of the most well-known, popular, most heavily downloaded libraries on
> > PyPI. According to here:
> >
> > https://hugovk.github.io/top-pypi-packages/
> >
> > sortedcontainers is the 290th most popular package on PyPI, ahead of
> > such luminaries as pylint, black, selenium, mypy, django and nose.
> >
> > And yet, nobody(?) admits to either using it or knowing what it could be
> > used for. How very curious :-/
>
> Those download stats can be misleading. Packages are often pulled in
> as a dependency of other packages which are popular or used a lot
> in CI/CD setups.
>
> Perhaps there's a reverse dependency graph we could use to find out
> why the package is downloaded this often. I remember having seen
> a project which does this, but have lost the URL.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Experts (#1, Nov 12 2021)
> >>> Python Projects, Coaching and Support ...https://www.egenix.com/
> >>> Python Product Development ...https://consulting.egenix.com/
> 
>
> ::: We implement business ideas - efficiently in both time and costs :::
>
>eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>Registered at Amtsgericht Duesseldorf: HRB 46611
>https://www.egenix.com/company/contact/
>  https://www.malemburg.com/
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/FJT4SSWISWWRFB5FGRPXSLDLDZVMUIXF/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KIKXWW2LLRT2YCNE2OOW7ZJ64N6OHOML/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Bob Fang]

> On 12 Nov 2021, at 16:32, Marc-Andre Lemburg  wrote:
> 
> Perhaps there's a reverse dependency graph we could use to find out
> why the package is downloaded this often. I remember having seen
> a project which does this, but have lost the URL.


I believe this is the URL we are looking for: 

https://www.wheelodex.org/projects/sortedcontainers/rdepends/?page=1 



Best,
Bob___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IT6XF34TJQQANG24AKPEC6RSWMA7BPSR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Eric V. Smith]

On 11/12/2021 11:31 AM, Chris Angelico wrote:


I think that proves the value of download counts: very little. The
highest on the list is a thing called botocore, which I've never heard
of. What is it? It's a dependency of a number of Amazon web services.
My guess is that it's a dependency of popular tools - or maybe of
automatically-installed tools, even - while not itself being well
known. Actually, quite a few of the most popular packages on that list
are Amazon-related, so quite possibly they're all being installed as a
set in response to one single *actual* dependency.


botocore and boto3 are the Amazon-provided low- and high-level 
interfaces, respectively, to AWS. They're extremely import and well 
known in that environment.


Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/V4GSZINECFIAZ4MGDZNM3L354LJDSX2Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2021-11-05 - 2021-11-12)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7282 (+23)
  closed 50241 (+38)
  total  57523 (+61)

Open issues with patches: 2886 


Issues opened (46)
==

#44439: stdlib wrongly uses len() for bytes-like object
https://bugs.python.org/issue44439  reopened by malin

#45354: test_winconsoleio fails on Windows 11
https://bugs.python.org/issue45354  reopened by vstinner

#45708: PEP 515-style formatting with underscores does not seem to wor
https://bugs.python.org/issue45708  reopened by serhiy.storchaka

#45732: Update python.org Windows and macOS installers to use Tk 8.6.1
https://bugs.python.org/issue45732  opened by ned.deily

#45733: importlib.abc.Traversable.name does not match
https://bugs.python.org/issue45733  opened by jmg

#45735: Promise the long-time truth that `args=list` works
https://bugs.python.org/issue45735  opened by tim.peters

#45736: 2to3 does not support integer division fixing
https://bugs.python.org/issue45736  opened by February291948

#45737: assertLogs to optionally not disable existing handlers
https://bugs.python.org/issue45737  opened by dandiez

#45738: 3.11 exec raises SystemError instead of SyntaxError on char af
https://bugs.python.org/issue45738  opened by pewscorner

#45739: The Python implementation of Decimal does not support the "N" 
https://bugs.python.org/issue45739  opened by serhiy.storchaka

#45742: python -h can't  find -R option
https://bugs.python.org/issue45742  opened by 180909

#45743: Cleanup and simplify setup.py
https://bugs.python.org/issue45743  opened by christian.heimes

#45744: Fix Flawfinder C Errors
https://bugs.python.org/issue45744  opened by osdotsystem

#45752: copy module doc wrongly says it doesn't copy arrays
https://bugs.python.org/issue45752  opened by Stefan Pochmann

#45753: Further speed up Python-to-Python calls.
https://bugs.python.org/issue45753  opened by Mark.Shannon

#45755: Mock spec with a specialized generic class does not mock class
https://bugs.python.org/issue45755  opened by kjamieson

#45756: mock raises exception when using a spec with an attribute that
https://bugs.python.org/issue45756  opened by kjamieson

#45757: compiler emits EXTENDED_ARG + NOP sequence in 3.10
https://bugs.python.org/issue45757  opened by rok.mandeljc

#45758: Crash on Py_DecRef'ing builtin object from previous run
https://bugs.python.org/issue45758  opened by Victor Milovanov

#45759: Improve error messages for non-matching `elif`/`else` statemen
https://bugs.python.org/issue45759  opened by February291948

#45760: Remove "PyNumber_InMatrixMultiply"
https://bugs.python.org/issue45760  opened by brandtbucher

#45761: recursive ProcessPoolExecutor invoke unable to return result
https://bugs.python.org/issue45761  opened by evan0greenup

#45762: Missing `list` symbols in the object inventory
https://bugs.python.org/issue45762  opened by Akarys

#45764: Parse error improvement forgetting ( after def
https://bugs.python.org/issue45764  opened by Carl.Friedrich.Bolz

#45765: importlib.metadata fails to find distributions in empty path
https://bugs.python.org/issue45765  opened by jaraco

#45766: Add direct proportion option to statistics.linear_regression()
https://bugs.python.org/issue45766  opened by rhettinger

#45767: Fix types for dev_t processing in posix module
https://bugs.python.org/issue45767  opened by AMDmi3

#45771: urllib.request.urlopen can leak socket.timeout to callers
https://bugs.python.org/issue45771  opened by thorpe-dev

#45772: socket.socket should be a class instead of a function
https://bugs.python.org/issue45772  opened by arhadthedev

#45773: Compiler hangs during jump elimination
https://bugs.python.org/issue45773  opened by brandtbucher

#45774: Detect SQLite in configure.ac
https://bugs.python.org/issue45774  opened by erlendaasland

#45775: Implementation of colorsys.rgb_to_yuv and vice versa
https://bugs.python.org/issue45775  opened by thomas.stolarski

#45776: abc submodule not an attribute of collections on Python 3.10.0
https://bugs.python.org/issue45776  opened by stcanny

#45779: multiprocessing initializer error with CPython 3.9.6 on Apple 
https://bugs.python.org/issue45779  opened by zhangzheng

#45780: dict. keys view behaviour diverges from set()
https://bugs.python.org/issue45780  opened by gregory.p.smith

#45781: Deleting __debug__ should be an SyntaxError
https://bugs.python.org/issue45781  opened by Carl.Friedrich.Bolz

#45783: test_freeze fails if a file is removed
https://bugs.python.org/issue45783  opened by petr.viktorin

#45786: Avoid allocating when exiting frame; it may be unsafe.
https://bugs.python.org/issue45786  opened by Mark.Shannon

#45787: HOWTO for socket programming and select documentation contradi
https://bugs.python.org/issue45787  opened by AlexWaygood

#45788: sys.prefix include direct

[Python-Dev] Re: Do we need to remove everything that's deprecated?

[Serhiy Storchaka]

12.11.21 12:55, Petr Viktorin пише:
>   AttributeError: '[...]Tests' object has no attribute 'failUnless'
> (bpo-45162)

This one caused me troubles more then one time. It is so easy to make a
typo and write assertEquals instead of assertEqual or assertRaisesRegexp
instead of assertRaisesRegex. Tests are passed, and warnings are
ignored. Then I run tests with -Werror to test new warnings and get a
lot of unrelated failures because of PRs merged at last half-year. I am
very glad that these long time ago deprecated aliases are finally removed.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BUFGDRDX336TODI5I6JAA5SB42WYFFRX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Marc-Andre Lemburg]

On 12.11.2021 17:46, Bob Fang wrote:
> 
> 
>> On 12 Nov 2021, at 16:32, Marc-Andre Lemburg > > wrote:
>>
>> Perhaps there's a reverse dependency graph we could use to find out
>> why the package is downloaded this often. I remember having seen
>> a project which does this, but have lost the URL.
> 
> 
> I believe this is the URL we are looking for: 
> 
> https://www.wheelodex.org/projects/sortedcontainers/rdepends/?page=1

Yes, thanks, that was the one.

The listing doesn't seem to solve the puzzle either, though. There
are a few high profile projects using the package, but those are not
high up on the download ranking, so don't explain the high download
counts of the package.

BTW: This reverse dependency ranking on the website gives a good
idea of how important a package is for the Python package eco system:

https://www.wheelodex.org/rdepends-leaders/

(sort of like the Page rank for Python packages)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Nov 12 2021)
>>> Python Projects, Coaching and Support ...https://www.egenix.com/
>>> Python Product Development ...https://consulting.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   https://www.egenix.com/company/contact/
 https://www.malemburg.com/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/732T3MNBB2PMFP7TVXSDDMW6KNI5CEQS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do we need to remove everything that's deprecated?

[Christopher Barker]

I sympathize with the OP, but I think never removing deprecated names is
the wrong solution.

If never removing those names is the appropriate action, then they never
should have been changed in the first place. That is, we should be ( and I
think are) very careful about gratuitously changing things.

Frankly, changing names in your code is a light lift. If you really want
your code to not be touched, keep using an older version of Python.

BTW, tools can help a lot here. I make heavy use of Pytest, and it recently
changed its default warning policy so that I get the depreciation warnings
early and often. It’s just not that hard to keep up if the code is seeing
any maintenance at all.


It’s a simple reality that code needs to be maintained.

 Mass-replacing "failUnless" with
> "assertTrue" just because someone decided it's a better name


If there really was no better reason than that, the change never should
have been made. But once made, keeping multiple names around forever is not
a good option.

-CHB
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HRD3NSIAI57CDRCP2CNXBL4722CUNGLX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Update colorsys with YUV conversions?

[Terry Reedy]

https://github.com/python/cpython/pull/29512
adds what appears to be well written conversions between RGB and YUV 
color systems, with tests.  Are we allowing such improvements?  If not, 
a module docstring note or comment should be added.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FV73RE3TSWDYBOJLZIONUJCX6FA32SZB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Update colorsys with YUV conversions?

[Victor Stinner]

I guess that you should agree on constants and then stick to them.
Otherwise, we might have to add a parameter later to chose the
conversion standard.

The PR uses "ATSC BT.709 standard constant".

Wikipedia says:
"When encoding Y’CBCR video, BT.709 creates gamma-encoded luma (Y’)
using matrix coefficients 0.2126, 0.7152, and 0.0722 (together they
add to 1). BT.709-1 used slightly different 0.2125, 0.7154, 0.0721
(changed to standard ones in BT.709-2)."
https://en.wikipedia.org/wiki/Rec._709#Luma_coefficients

Victor

On Fri, Nov 12, 2021 at 11:28 PM Terry Reedy  wrote:
>
> https://github.com/python/cpython/pull/29512
> adds what appears to be well written conversions between RGB and YUV
> color systems, with tests.  Are we allowing such improvements?  If not,
> a module docstring note or comment should be added.
>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/FV73RE3TSWDYBOJLZIONUJCX6FA32SZB/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZWI3UTU3FADBHF4CP3WLTUQVLWNQKPUY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Update colorsys with YUV conversions?

[Guido van Rossum]

There was talk of deprecating colorsys, but PEP 594 now lists it under
"modules to keep" so I don't see why we couldn't improve it. The OP's
reasons for doing so seem solid. I don't know anything about this topic so
I'll leave it at that.

On Fri, Nov 12, 2021 at 2:22 PM Terry Reedy  wrote:

> https://github.com/python/cpython/pull/29512
> adds what appears to be well written conversions between RGB and YUV
> color systems, with tests.  Are we allowing such improvements?  If not,
> a module docstring note or comment should be added.
>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/FV73RE3TSWDYBOJLZIONUJCX6FA32SZB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U66ZB7GBURYA5MFBC3ZBHEACDHF2WBMH/
Code of Conduct: http://python.org/psf/codeofconduct/