Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.
On Fri, 24 May 2019 14:23:21 +0200 Thomas Wouters wrote: > On Thu, May 23, 2019 at 5:15 PM Steve Dower wrote: > > > On 23May2019 0542, Inada Naoki wrote: > > > 1. perf shows 95% of CPU time is eaten by _PyObject_Free, not kernel > > space. > > > 2. This loop is cleary hot: > > > > > https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L1816-L1819 > > > > > > > > I can attach the process by gdb and I confirmed many arenas have > > > same nfreepools. > > > > It's relatively easy to test replacing our custom allocators with the > > system ones, yes? Can we try those to see whether they have the same > > characteristic? > > > > Given the relative amount of investment over the last 19 years [1], I > > wouldn't be surprised if most system ones are at least as good for our > > needs now. Certainly Windows HeapAlloc has had serious improvements in > > that time to help with fragmentation and small allocations. > > > > FYI, and I've mentioned this at PyCon to a few people (might've been you, > Steve, I don't remember) -- but at Google we've experimented with disabling > obmalloc when using TCMalloc (a faster and thread-aware malloc, which makes > a huge difference within Google when dealing with multi-threaded C++ > libraries), both using the usual Python benchmarks and real-world code with > real-world workloads (a core part of YouTube, for example), all on Linux. > There's still a significant benefit to using obmalloc when using glibc's > malloc, and also a noticeable benefit when using TCMalloc. There are > certainly cases where it doesn't matter much, and there may even be cases > where the overhead of obmalloc isn't worth it, but I do believe it's still > a firm net benefit. Interesting that a 20-year simple allocator (obmalloc) is able to do better than the sophisticated TCMalloc. (well, of course, obmalloc doesn't have to worry about concurrent scenarios, which explains some of the simplicity) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.
On 5/22/19 12:15 PM, Tim Peters wrote: There's a Stackoverflow report[1] I suspect is worth looking into, but it requires far more RAM (over 80GB) than I have). [...] But, for a start, it would be good to know if anyone else can actually reproduce the problem. [1] https://stackoverflow.com/questions/56228799/python-hangs-indefinitely-trying-to-delete-deeply-recursive-object I have a computer with two Xeon CPUs and 256GB of RAM. So, even though it's NUMA, I still have 128GB of memory per CPU. It's running a "spin" of Ubuntu 18.10. I compiled a fresh Python 3.7.3 --with-optimizations. I copied the sample program straight off the StackOverflow page. The program ran for about five and a half hours then exited normally. During the run it printed: This gets printed! This doesn't get printed Statistics reported by "time": 19811.05s user 123.56s system 99% cpu 5:32:15.04 total Checking in on it now and then, peak observed memory usage (as reported by "top") was just under 80GB. I take it that the interesting part was confirming that "This doesn't get printed" gets printed when you have enough RAM for the program to run to completion. So I guess there's no bug here? Just an observation about CPython's garbage collector being kinda slow? Or maybe CPython gc + swap = super bombad slow? //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 594 - a proposal for unmaintained modules
On Fri, 24 May 2019 09:54:05 -0700 Steve Dower wrote: > > All in all, this is basically where we are today, with the exception > that we haven't officially said that we no longer support these modules. > PEP 594 is this official statement, and our usual process for things we > don't support is to remove them in two versions time. It doesn't have to > be so controversial - either the people who are saying "we rely on this" > are also willing to help us maintain them, or they're not. And if > they're not, they clearly don't rely on it (or realize the cost of > relying on volunteer-maintained software). This is an absurd statement. You can rely on something without having the time or competence required to maintain it yourself. Division of labour exists for a reason. Obviously this doesn't mean that the CPython development team must absorb any development activity that may be useful to the greater public. But widely-used software packages such as a Python and its stdlib have a tremendous enabling effect. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 558: Defined semantics for locals()
(And this time I will remember to remove the SPAM label...) On Sun, May 26, 2019 at 08:44:33AM +1000, Chris Angelico wrote: > From my reading of the description, you could also "assert a is b" - > is that correct? Yes, that's already the behaviour. py> def demo(): ... a = locals() ... b = locals() ... print(a is b) ... py> demo() True -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [SPAM?] Re: PEP 558: Defined semantics for locals()
Richard, your email seems to have introduced a spurious "SPAM" label
to this thread, which may confuse some email clients into treating it
as spam. Can you teach your email program that this mailing list is ham,
not spam, or failing that, at least edit the subject line to remove the
label? Thanks. I've done so for this response, but please take care that
you don't re-introduce the label again, thanks.
On Sat, May 25, 2019 at 06:37:22PM -0400, Richard Damon wrote:
> To me that is a static snapshot of a dynamic environment, not a dynamic
> snapshot. The snapshot you get at THAT moment in time won't change, as
> time progresses, so that snapshot itself isn't dynamic.
Actually, it does change -- but the confusing part is that it doesn't
change automatically but only when you call the locals() function again.
This already CPython's behaviour, so that is not changing.
def demo1():
a = b = c = 1
a = locals()
print(a)
b = 999
print(a)
def demo2():
a = b = c = 1
a = locals()
print(a)
b = 999
locals() # call function but throw the result away
print(a)
And running those two functions in Python 3.5:
py> demo1() # No change to the dict.
{'a': 1, 'b': 1, 'c': 1}
{'a': 1, 'b': 1, 'c': 1}
py> demo2() # Dict magically updates!
{'a': 1, 'b': 1, 'c': 1}
{'a': {...}, 'b': 999, 'c': 1}
I know this is the backwards-compatible behaviour, but I would like to
question whether we want to enshrine it in stone. This seems to me to
be the worst possible combinations of features:
- writing to the locals() dict doesn't write changes back to the
actual local variables;
- the dict returned isn't a fixed snapshot, but the PEP calls it a
snapshot despite not being one (naming things is hard);
- the "snapshop" can change as a side-effect of another operation.
If this wasn't already the behaviour, would we want it?
--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 558: Defined semantics for locals()
On Sun, May 26, 2019 at 08:04:11PM +1000, Steven D'Aprano wrote: > Richard, your email seems to have introduced a spurious "SPAM" label [...] > edit the subject line to remove the > label? Thanks. I've done so for this response Oops. Done now. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 558: Defined semantics for locals()
On Sun, May 26, 2019 at 8:07 PM Steven D'Aprano wrote: > On Sun, May 26, 2019 at 08:44:33AM +1000, Chris Angelico wrote: > > > From my reading of the description, you could also "assert a is b" - > > is that correct? > > Yes, that's already the behaviour. > > py> def demo(): > ... a = locals() > ... b = locals() > ... print(a is b) > ... > py> demo() > True > Sure, but this PEP is all about defining things that weren't previously defined, so I wanted to clarify intent rather than current behaviour. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 558: Defined semantics for locals()
On Sun, 26 May 2019 at 12:23, Chris Angelico wrote: > > On Sun, May 26, 2019 at 8:07 PM Steven D'Aprano wrote: > > On Sun, May 26, 2019 at 08:44:33AM +1000, Chris Angelico wrote: > > > > > From my reading of the description, you could also "assert a is b" - > > > is that correct? > > > > Yes, that's already the behaviour. > > > > py> def demo(): > > ... a = locals() > > ... b = locals() > > ... print(a is b) > > ... > > py> demo() > > True > > > > Sure, but this PEP is all about defining things that weren't > previously defined, so I wanted to clarify intent rather than current > behaviour. +1 on the PEP being explicit over this. Even though it's current behaviour, it's surprising and making a clear and definitive statement that the PEP intends to make the behaviour part of the language definition and not just a CPython detail, is IMO worthwhile. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 558: Defined semantics for locals()
On Sun, May 26, 2019 at 09:20:53PM +1000, Chris Angelico wrote: > Sure, but this PEP is all about defining things that weren't > previously defined, so I wanted to clarify intent rather than current > behaviour. As I understand it, the intent is to: - fix some annoyances/bugs involved when you have a trace hook; - make the current CPython behaviour a language behaviour. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 594 - a proposal for unmaintained modules
On 24.05.2019 9:55, Steven D'Aprano wrote: I don't know if this is a good idea or a terrible idea or somewhere in between, so I'm throwing it out to see if anyone likes it. Let's add a third option to PEP 594 between "keep" and "remove": explicitly flagging a module as unmaintained. Unmaintained modules: - will raise a warning when imported that they are unmaintained; - will have their tests disabled; - possibly we move them into a seperate namespace: ``from unmaintained import aardvark`` - bug reports without patches will be closed Will Not Fix; - bug reports with patches *may* be accepted if some core dev is willing to review and check it in, but there is no obligation to do so; - should it turn out that someone is willing to maintain the module, it can be returned to regular status. Plus side: - reduce the maintenance burden (if any) from the module; - while still distributing the module and allowing users to use it: "no promises, but here you go"; - other implementations are under no obligation to distribute unmaintained modules. Minus side: - this becomes a self-fulfilling prophesy: with tests turned off, bit-rot will eventually set in and break modules that currently aren't broken. Thoughts? This would greately damage Python's image as a mature, reliable product, something that you can bank on for your critical tasks. By including something into the offer, we implicitly endorse it and claim that we want out users to use that rather than competing offers. You may look at the archives where I relied on TKinter for an important project and it let me down big time. I would very much prefer Python to honestly admit that there's not enough interest in this module and drop it so I would look elsewhere right off the bat and don't. You may argue that marking it as unsupported would do the same but it wouldn't. This mean I'll have to be constantly on the lookout for hidden notes and such, I will forever lose confidence that I can just take whatever is in Python and run with it, being confident that the Python team has my back. -- Regards, Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Accepting PEP 587 (Python Initialization Configuration)
As PEP-delegate, I'm accepting version 5 of PEP 587 (Python Initialization Configuration). I don't think Victor has posted the latest version, but it contains a couple of minor renames, and a new private API to allow us to experiment with two-phase initialisation of Python. There's still time to debate exact names and work on the wording of the PEP, but I'm satisfied the design is solid, a clear improvement over what we currently have, and a step in the right direction for some of the more extensive changes that have been proposed (e.g. using Python *code* for the Python configuration. If we do get to that point, the PEP contains a few things that are superfluous, but I'd consider them minor transitionary warts at that point -- if we ever get there). Thanks, Victor, for pushing this massive amount of work through; it won't affect most people using Python (at least until the rest of PEP 432 lands) but it's a giant boon to those of us who embed Python :) -- Thomas Wouters Hi! I'm an email virus! Think twice before sending your email to help me spread! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Getting #6641 merged
Hi, I'd like to get https://github.com/python/cpython/pull/6641 merged. I keep having to rebase it and regenerate all the importlib header files, which is becoming a bit annoying. So, I can I ask that if you are going to modify Python/ceval.c can you hold on just a little while, until #6641 is merged. Thanks, Mark. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.
[Larry Hastings ] > I have a computer with two Xeon CPUs and 256GB of RAM. So, even > though it's NUMA, I still have 128GB of memory per CPU. It's running a > "spin" of Ubuntu 18.10. > > I compiled a fresh Python 3.7.3 --with-optimizations. I copied the sample > program straight off the StackOverflow page. The program ran for about > five and a half hours then exited normally. Thanks, Larry! > During the run it printed: > > This gets printed! > This doesn't get printed > > Statistics reported by "time": > > 19811.05s user 123.56s system 99% cpu 5:32:15.04 total > > Checking in on it now and then, peak observed memory usage (as reported > by "top") was just under 80GB. All of that is consistent with what others reported, although you've given more quantified details, and that's helpful. > I take it that the interesting part was confirming that "This doesn't get > printed" > gets printed when you have enough RAM for the program to run to completion. That was the primary point at first, yes, > So I guess there's no bug here? Just an observation about CPython's > garbage collector being kinda slow? Or maybe CPython gc + swap = > super bombad slow? No need to confirm this: if you ran it again with a bit more output, you'd find that the vast bulk of the time was spent between the two output lines. That is, it takes hours from the time the train() function starts to return and completes returning. That's all consumed as a side effect of `tree` losing its last reference. We know why now, and it's hard to say whether it's "a bug". It's functioning as designed, but the design sucks ;-) So I'd call it a design flaw. The heuristics that were introduced to try to make it likely we could return more empty arenas to C were designed when machines had far less RAM, and can burn time quadratic in the number of arenas. That really didn't matter in a world with a few hundred arenas, but can be a timing disaster in a world with hundreds of thousands (or more) of arenas. The Stackoverflow test case was whittled way down from the OP's real program, which ran "for days" without completing. I've sketched two (albeit closely related, both to each other and to what we currently do) ways the worst-case overall time could be cut from quadratic to linear in the number of arenas, which is asymptotically optimal. More ambitious would be to dream up an entirely different heuristic. Keeping the list of usable arenas fully sorted in order of number of free pools seems a very strong requirement for a poke-and-hope heuristic. But, best I can tell, the person who came up with this heuristic to begin with didn't check in any motivating test programs. So we'd be starting over from scratch. For this specific Stackoverflow case, I've confirmed that it doesn't make a real difference if we didn't bother to sort the list of arenas _at all_ (then it still returns about the same number of arenas to C, both at the point the tree finishes building, and at the time it completes tearing down the tree). So I have a program showing the current scheme can be a disaster, but none where it shows it actually helps anything ;-) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 594: update 1
On Sat, May 25, 2019 at 8:00 PM Daniel Moisset wrote: > Hi, thanks for the work on this proposal, I think this is at least a tip > of the iceberg and a good start for the bigger question of how the stdlib > should evolve.. > > I think that the PEP should include an idea of what should happen if > existing stdlib pieces depend on this deprecated modules. For example, > email.mime.audio is part of a non-deprecated module but it depends on > sndhdr which is being planned for deprecation. Should that part of the > functionality be deprecated too? rewritten to not depend on the deprecated > module (but keep the pieces of relevant code? depend on optional 3rd party > modules and degrade if those are not available? > It's a case-by-case situation. In some instances we may copy the relevant code over and keep it private. In other situations some functionality may get deprecated. It all depends and thus why you can't really write a general solution that applies uniformly. -Brett > > The sndhdr case is just an example but I can imagine there are others > (optparse is no longer schedule for removal in your PEP, but would have > been another case, being used by the profile module). > > Best, > Daniel > > On Tue, 21 May 2019 at 15:15, Christian Heimes > wrote: > >> Hi, >> >> I have updated the PEP with feedback from discussions. The highlights are: >> >> * Deprecate parser module >> * Keep fileinput module >> * Elaborate why crypt and spwd are dangerous and bad >> * Improve sections for cgitb, colorsys, nntplib, and smtpd modules >> * The colorsys, crypt, imghdr, sndhdr, and spwd sections now list >> suitable substitutions. >> * Mention that socketserver is going to stay for http.server and >> xmlrpc.server >> * The future maintenance section now states that the deprecated modules >> may be adopted by Python community members. >> >> https://github.com/python/peps/compare/7799178a...2d536899?diff=unified#diff-ae358c21fa7968ee3b6c64479e051574 >> >> >> I'll be traveling the next couple of days and will only have limited >> opportunities to respond on feedback. >> >> Christian >> >> --- >> PEP: 594 >> Title: Removing dead batteries from the standard library >> Author: Christian Heimes >> Status: Draft >> Type: Standards Track >> Content-Type: text/x-rst >> Created: 20-May-2019 >> Post-History: 21-May-2019 >> >> >> Abstract >> >> >> This PEP proposed a list of standard library modules to be removed from >> the >> standard library. The modules are mostly historic data formats and APIs >> that >> have been superseded a long time ago, e.g. Mac OS 9 and Commodore. >> >> >> Rationale >> = >> >> Back in the early days of Python, the interpreter came with a large set of >> useful modules. This was often refrained to as "batteries included" >> philosophy and was one of the corner stones to Python's success story. >> Users didn't have to figure out how to download and install separate >> packages in order to write a simple web server or parse email. >> >> Times have changed. The introduction of the cheese shop (PyPI), >> setuptools, >> and later pip, it became simple and straight forward to download and >> install >> packages. Nowadays Python has a rich and vibrant ecosystem of third party >> packages. It's pretty much standard to either install packages from PyPI >> or >> use one of the many Python or Linux distributions. >> >> On the other hand, Python's standard library is piling up cruft, >> unnecessary >> duplication of functionality, and dispensable features. This is >> undesirable >> for several reasons. >> >> * Any additional module increases the maintenance cost for the Python core >> development team. The team has limited resources, reduced maintenance >> cost >> frees development time for other improvements. >> * Modules in the standard library are generally favored and seen as the >> de-facto solution for a problem. A majority of users only pick 3rd party >> modules to replace a stdlib module, when they have a compelling reason, >> e.g. >> lxml instead of `xml`. The removal of an unmaintained stdlib module >> increases the chances of a community contributed module to become widely >> used. >> * A lean and mean standard library benefits platforms with limited >> resources >> like devices with just a few hundred kilobyte of storage (e.g. BBC >> Micro:bit). Python on mobile platforms like BeeWare or WebAssembly >> (e.g. pyodide) also benefit from reduced download size. >> >> The modules in the PEP have been selected for deprecation because their >> removal is either least controversial or most beneficial. For example >> least controversial are 30 years old multimedia formats like ``sunau`` >> audio format, which was used on SPARC and NeXT workstations in the late >> 1980ties. The ``crypt`` module has fundamental flaws that are better >> solved >> outside the standard library. >> >> This PEP also designates some m
Re: [Python-Dev] Accepting PEP 587 (Python Initialization Configuration)
On 5/26/2019 10:02 AM, Thomas Wouters wrote: As PEP-delegate, I'm accepting version 5 of PEP 587 (Python Initialization Configuration). ... Thanks, Victor, for pushing this massive amount of work through; And thanks to you for reviewing and approving. it won't affect most people using Python (at least until the rest of PEP 432 lands) but it's a giant boon to those of us who embed Python :) It affects us when apps decided to embed or not embed Python as their exposed scripting language. The open-source Godot 2D/3D game engine started with Lua, switched to Squirrel (???), and tried to embed Python but ran into 'problems'. Instead they developed a very Python-like GDScript, with a subset of Python syntax and expanded built-ins. (So it is still easy for a Pythonista to use.) It might be worthwhile to ask them what the fatal problems were. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 594 - a proposal for unmaintained modules
On Sun, May 26, 2019 at 04:03:11PM +0300, Ivan Pozdeev via Python-Dev wrote: > On 24.05.2019 9:55, Steven D'Aprano wrote: > >I don't know if this is a good idea or a terrible idea or somewhere in > >between, so I'm throwing it out to see if anyone likes it. [...] > This would greately damage Python's image as a mature, reliable product, > something that you can bank on for your critical tasks. > > By including something into the offer, we implicitly endorse it and claim > that we want out users to use that rather than competing offers. Do we? I don't think that is the case for everything -- we certainly don't want people using private or undocumented functions, nor do we want them using deprecated code. [...] > You may argue that marking it as unsupported would do the same but it > wouldn't. This mean I'll have to be constantly on the lookout for hidden > notes and such, Is "from unmaintained import module" enough to tell you that the module is unmaintained? -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 594 - a proposal for unmaintained modules
Thanks Steve for your comments, I appreciate them. As I said I don't know if this is a good idea or not so please read my responses below as part of a friendly debate aimed at reaching consensus, not an argument. (The argument is in Room 12 with Mr. Barnard.) On Fri, May 24, 2019 at 09:54:05AM -0700, Steve Dower wrote: > On 23May2019 2355, Steven D'Aprano wrote: > >I don't know if this is a good idea or a terrible idea or somewhere in > >between, so I'm throwing it out to see if anyone likes it. > > > >Let's add a third option to PEP 594 between "keep" and "remove": > >explicitly flagging a module as unmaintained. Unmaintained modules: > > > >- will raise a warning when imported that they are unmaintained; > > Nobody reads warnings. If nobody reads warnings, we should just remove the warnings module and be done with it. That should probably be a PEP. > >- will have their tests disabled; > > I'm fine with this. Reducing time to run the test suite is the biggest > immediate win from removing unmaintained modules. Can somebody quantify how much time will be saved? Will we be saving 5 seconds in a 20 minute test suite, or 1 minute in a 2 minute test suite? > >- possibly we move them into a seperate namespace: > > ``from unmaintained import aardvark`` > > May as well move them all the way to PyPI and leave the import name the > same. No, that won't be the same. Keeping the import name the same is only valuable if we're not breaking backwards compatibility. If we remove the module from the std lib, we've broken backwards compatibility and people's code will still be broken, same name or not. > >- bug reports without patches will be closed Will Not Fix; > > By whom? By anyone with the ability to close bug reports. On bugs.python.org that's anyone with an account, you don't need to be a core developer, and some volunteers help out by triaging or closing poor-quality bug reports long before the core developers get involved. I assume the same applies on github -- if it doesn't, that's a huge usability regression. > >- bug reports with patches *may* be accepted if some core dev is > > willing to review and check it in, but there is no obligation > > to do so; > > Historically, nobody has been willing to review and merge these modules > for years. Is that true? That's certainly the *impression* people are giving, but the PEP is light on examples or details. The one example I looked into, binhex, didn't seem particularly troublesome. https://mail.python.org/pipermail/python-dev/2019-May/157673.html Although I may have missed some examples. > How long do we have to wait? How long do we wait before closing any other ticket? Less than that, because these would be unmaintained. > >- should it turn out that someone is willing to maintain the module, > > it can be returned to regular status. > > Or we can give them the keys to the PyPI package. Or help pip implement > the "Obsoleted By" header and redirect to a fork. Indeed we could. The question is whether we should. > >Plus side: > > > >- reduce the maintenance burden (if any) from the module; > > Apart from having to read, review, and decide on bug reports, CVEs, and > documentation. They're unmaintained. Nobody is promising to read or review bug reports. Volunteers will surely do the bulk of triaging (as I think they already do now). And besides, if it is correct that nobody is using these modules, who will be making the bug reports? > >- while still distributing the module and allowing users to use > > it: "no promises, but here you go"; > > a.k.a. PyPI? No, nothing like PyPI, for reasons discussed many times before including some of my earlier posts. Every Python user has access to whatever is in the std lib (exception: some Linux distros split out parts of the std lib into packages that need to be apt-installed or equivalent). Not all Python users can use third-party libraries on PyPI. Moving things to PyPI is an implicit decision to cut some users out. > >- other implementations are under no obligation to distribute > > unmaintained modules. > > a.k.a. PyPI? No, nothing like PyPI. > >Minus side: > > > >- this becomes a self-fulfilling prophesy: with tests turned off, > > bit-rot will eventually set in and break modules that currently > > aren't broken. > > True. And then we have the added maintenance burden of explaining > repeatedly that we don't care that it's broken, but we want to put it on > your machine anyway. That's not a "maintenance burden" since there's no maintenance involved. At worst its a small annoyance to the volunteers who close invalid bug reports on the tracker. If there is a flood of bug reports for unmaintained modules, that will be objective evidence that the module is broken and we can remove it for real. [...] > either the people who are saying "we rely on this" > are also willing to help us maintain them, or they're not. And if > they'r
[Python-Dev] Overloading comparison operator for lists
NumPy arrays have this awesome feature, where array == 3 does an element-wise comparison and returns a list. For example: np.array([1,2,3,4,5])==3 returns [False,False,True,False,False] It would be cool if Python had similar functionality for lists. If that is not possible, perhaps we could consider allowing developers to overload operators on built-in types within the context of a project or module. For example, an overload in one module would have no effect on the same operator in a different module (such as any Python standard modules.) Additionally, let's then give the developers the option to explicitly import an overload from other modules. So, people could develop a module with the purpose of providing overloads that make complete sense within a certain context. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Overloading comparison operator for lists
This belongs on python-ideas, not python-dev. I've directed replies to this message there. Comments below. On 26May2019 21:52, Montana Burr wrote: NumPy arrays have this awesome feature, where array == 3 does an element-wise comparison and returns a list. For example: np.array([1,2,3,4,5])==3 returns [False,False,True,False,False] It would be cool if Python had similar functionality for lists. map(lamdba item: item==3, [1,2,3,4,5]) I'm not sure this rates extra Python features. Personally I'm -1 on this suggestion because == traditionally returns a Boolean, NumPy notwithstanding. Your example above doesn't return a Boolean. If that is not possible, perhaps we could consider allowing developers to overload operators on built-in types within the context of a project or module. For example, an overload in one module would have no effect on the same operator in a different module (such as any Python standard modules.) This is usally done by overloading dunder methods on classes. if you class subclasses a builtin eg int or list then the instances get the special behaviour. Additionally, let's then give the developers the option to explicitly import an overload from other modules. So, people could develop a module with the purpose of providing overloads that make complete sense within a certain context. If you go the subclass route you could do this with a mixin class (a class providing methods but little else, intended to be part of the MRO of a subclass). Cheers, Cameron Simpson ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
