[Python-Dev] PEP 594 -- Bundling libraries?

2019-05-24 Thread Inada Naoki
When removing libraries from stdlib, can we bundle
removed libraries and install it like ensurepip does?

Ruby does similar thing, called "Gemification".
See https://rubykaigi.org/2017/presentations/hsbt.html

When people don't use venv, scripts importing nntplib
or aifc runs correctly.

When people use venv, they need to `pip install nntplib`
(we may be able to use PyPI's namespace beta feature here).
But I assume `pip install` is not a problem for people using venv.

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-24 Thread Inada Naoki
> > >
> > > It's relatively easy to test replacing our custom allocators with the
> > > system ones, yes? Can we try those to see whether they have the same
> > > characteristic?
> > >
> >
> > Yes.
> >
> > PYTHONMALLOC=malloc LD_PRELOAD=/path/to/jemalloc.so python script.py
> >
> > I will try it tomorrow.
>
> Can you simply test with the system allocator rather than jemalloc?
>

For the record, result for 10M nodes, Ubuntu 18.04 on AWS r5a.4xlarge:

$ local/bin/python3 t1.py  # default
1138.1123778309993 -- end train, start del
688.7927911250008 -- end

$ arena-1m/bin/python3 t1.py  # Changed ARENA_SIZE to 1MiB
1085.3363994170013 -- end train, start del
84.57135540099989 -- end

$ PYTHONMALLOC=malloc local/bin/python3 t1.py
1157.4882792789995 -- end train, start del
27.91983470674 -- end

$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
PYTHONMALLOC=malloc local/bin/python3 t1.py
1098.4383037820007 -- end train, start del
117.93938426599925 -- end

In this case, glibc malloc is the fastest.
glibc is know to weak about fragmentation.
But algorithm to avoid fragmentation is just an overhead in this script.

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-24 Thread Victor Stinner
Le ven. 24 mai 2019 à 09:41, Inada Naoki  a écrit :
> For the record, result for 10M nodes, Ubuntu 18.04 on AWS r5a.4xlarge:
>
> $ local/bin/python3 t1.py  # default
> 1138.1123778309993 -- end train, start del
> 688.7927911250008 -- end
>
> $ arena-1m/bin/python3 t1.py  # Changed ARENA_SIZE to 1MiB
> 1085.3363994170013 -- end train, start del
> 84.57135540099989 -- end

688 => 84 looks like an interesting speedup. Would it be technically
possible to make ARENA_SIZE configurable at runtime?

Using the new PEP 587 preinitialization, it shouldn't be too hard to
hard a new command line option and/or an environment variable to tune
the memory allocator:
https://www.python.org/dev/peps/pep-0587/#preinitialization-with-pypreconfig

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-24 Thread Thomas Wouters
On Thu, May 23, 2019 at 5:15 PM Steve Dower  wrote:

> On 23May2019 0542, Inada Naoki wrote:
> > 1. perf shows 95% of CPU time is eaten by _PyObject_Free, not kernel
> space.
> > 2. This loop is cleary hot:
> >
> https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L1816-L1819
> >
> > I can attach the process by gdb and I confirmed many arenas have
> > same nfreepools.
>
> It's relatively easy to test replacing our custom allocators with the
> system ones, yes? Can we try those to see whether they have the same
> characteristic?
>
> Given the relative amount of investment over the last 19 years [1], I
> wouldn't be surprised if most system ones are at least as good for our
> needs now. Certainly Windows HeapAlloc has had serious improvements in
> that time to help with fragmentation and small allocations.
>

FYI, and I've mentioned this at PyCon to a few people (might've been you,
Steve, I don't remember) -- but at Google we've experimented with disabling
obmalloc when using TCMalloc (a faster and thread-aware malloc, which makes
a huge difference within Google when dealing with multi-threaded C++
libraries), both using the usual Python benchmarks and real-world code with
real-world workloads (a core part of YouTube, for example), all on Linux.
There's still a significant benefit to using obmalloc when using glibc's
malloc, and also a noticeable benefit when using TCMalloc. There are
certainly cases where it doesn't matter much, and there may even be cases
where the overhead of obmalloc isn't worth it, but I do believe it's still
a firm net benefit.


>
> Cheers,
> Steve
>
> [1]:
>
> https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L769
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/thomas%40python.org
>


-- 
Thomas Wouters 

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python in next Windows 10 update

2019-05-24 Thread Baptiste Carvello
Hello,

Le 21/05/2019 à 22:30, Steve Dower a écrit :
>
> [...]
> 
> * the Python 3.7 installed from the store will not auto-update to 3.8,
> but when 3.8 is released we (Microsoft) will update the redirect to
> point at it
> * if you pass arguments to the redirect command, it just exits with an
> error code - you only get the Store page if you run it without arguments

I was wondering how those 2 bullet points combine. Say a user installs
3.7 from the store, then uses "python.exe" with arguments, in a shebang
line or a batch script.

Does it mean the script might break unexpectedly when 3.8 is released?
Then, would it make sense for the redirect command to proxy to the 3.7
install when arguments are passed?

Cheers,
Baptiste


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done

2019-05-24 Thread Ben Cail
First, thanks to all the python core devs (and others) who work on 
Python - many, many people (myself included) benefit from your work. 
Suggestion below...


On 5/23/19 9:02 PM, Barry Warsaw wrote:

On May 23, 2019, at 14:17, Brett Cannon  wrote:


I'm personally viewing it as a first step in addressing the maintenance burden 
we have with such a large stdlib. Christian started this work over a year ago 
and I think it's worth seeing through. After that we should probably have a 
discussion as a team about how we view the stdlib long-term and how that ties 
into maintaining it so that people's opinion of the stdlib's quality goes up 
rather than viewing the quality of it as varying module-to-module.

+1 :)



Why not have the PSF hire someone (or multiple people) to be paid to 
work on the maintenance burden? This could be similar to the Django 
fellows: 
https://www.djangoproject.com/fundraising/#who-is-the-django-fellow. It 
seems like a good thing for Django, and Python is used by many more 
people than Django. Why not pay someone to do the work that others don't 
want to do? The person in this position could be guided by the PSF 
and/or the Steering Council, to do the work most necessary for the good 
of the language as a whole (like maintaining old modules that other core 
devs don't want to work on).


You could market it together with the maintenance burden: "you want to 
use all these old modules, but we don't want to maintain them. So pay us 
some money, and we'll hire someone to maintain them."


If this idea has already been discussed in this context and rejected, 
could it be added to the PEP, with the reasons why it won't work?


Thanks again for your work.

-Ben

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python in next Windows 10 update

2019-05-24 Thread Steve Dower

On 24May2019 0220, Baptiste Carvello wrote:

Hello,

Le 21/05/2019 à 22:30, Steve Dower a écrit :


[...]

* the Python 3.7 installed from the store will not auto-update to 3.8,
but when 3.8 is released we (Microsoft) will update the redirect to
point at it
* if you pass arguments to the redirect command, it just exits with an
error code - you only get the Store page if you run it without arguments


I was wondering how those 2 bullet points combine. Say a user installs
3.7 from the store, then uses "python.exe" with arguments, in a shebang
line or a batch script.

Does it mean the script might break unexpectedly when 3.8 is released?
Then, would it make sense for the redirect command to proxy to the 3.7
install when arguments are passed?


This is a very big complicated situation (just like on Linux), but the 
answer is that if you are worried about your script breaking, then don't 
use "python.exe" in the shebang line or batch file. You can use 
"python3.7" just fine (well, no shebang lines on Windows).


As for the redirect command, if you install Python from the store, it 
will be replaced completely. Uninstalling Python doesn't even bring it 
back (another thing we considered and decided wasn't worth the effort - 
if you've installed Python once then you probably know how to find it 
again).


And if you replace it with something higher in PATH then it's just like 
shadowing any other command, including the case where one day Microsoft 
adds a new version of a command you were using (e.g. ssh or bash :) ).


So there's really no change to your scenario due to the redirect, just 
the same old "if you don't specify the version of Python you want then 
you'll get what you get". (Refer to the linux-sig list for detailed 
discussion.)


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 594 - a proposal for unmaintained modules

2019-05-24 Thread Steve Dower

On 23May2019 2355, Steven D'Aprano wrote:

I don't know if this is a good idea or a terrible idea or somewhere in
between, so I'm throwing it out to see if anyone likes it.

Let's add a third option to PEP 594 between "keep" and "remove":
explicitly flagging a module as unmaintained. Unmaintained modules:

- will raise a warning when imported that they are unmaintained;


Nobody reads warnings.


- will have their tests disabled;


I'm fine with this. Reducing time to run the test suite is the biggest 
immediate win from removing unmaintained modules.



- possibly we move them into a seperate namespace:
   ``from unmaintained import aardvark``


May as well move them all the way to PyPI and leave the import name the 
same.



- bug reports without patches will be closed Will Not Fix;


By whom?


- bug reports with patches *may* be accepted if some core dev is
   willing to review and check it in, but there is no obligation
   to do so;


Historically, nobody has been willing to review and merge these modules 
for years. How long do we have to wait?



- should it turn out that someone is willing to maintain the module,
   it can be returned to regular status.


Or we can give them the keys to the PyPI package. Or help pip implement 
the "Obsoleted By" header and redirect to a fork.



Plus side:

- reduce the maintenance burden (if any) from the module;


Apart from having to read, review, and decide on bug reports, CVEs, and 
documentation.



- while still distributing the module and allowing users to use
   it: "no promises, but here you go";


a.k.a. PyPI?


- other implementations are under no obligation to distribute
   unmaintained modules.


a.k.a. PyPI?


Minus side:

- this becomes a self-fulfilling prophesy: with tests turned off,
   bit-rot will eventually set in and break modules that currently
   aren't broken.


True. And then we have the added maintenance burden of explaining 
repeatedly that we don't care that it's broken, but we want to put it on 
your machine anyway.


All in all, this is basically where we are today, with the exception 
that we haven't officially said that we no longer support these modules. 
PEP 594 is this official statement, and our usual process for things we 
don't support is to remove them in two versions time. It doesn't have to 
be so controversial - either the people who are saying "we rely on this" 
are also willing to help us maintain them, or they're not. And if 
they're not, they clearly don't rely on it (or realize the cost of 
relying on volunteer-maintained software).


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-24 Thread Tim Peters
[Inada Naoki ]
> For the record, result for 10M nodes, Ubuntu 18.04 on AWS r5a.4xlarge:

I'm unclear on what "nodes" means.  If you mean you changed 27M to 10M
in this line:

for token in random_strings(27_000_000):

that's fine, but there are about 40 times more than that `Node`
objects created by the program.  So if you did change that to
10_000_000, you'd have about 400M `Node` objects, consuming about 80x
that many bytes of RAM (or about 32GB).

> $ local/bin/python3 t1.py  # default
> 1138.1123778309993 -- end train, start del
> 688.7927911250008 -- end
>
> $ arena-1m/bin/python3 t1.py  # Changed ARENA_SIZE to 1MiB
> 1085.3363994170013 -- end train, start del
> 84.57135540099989 -- end

Nice!  I assume these are seconds.  On Stackoverflow, the OP reported
that boosting ARENA_SIZE the same way cut deallocation time in his
original program to about 13 minutes.


> $ PYTHONMALLOC=malloc local/bin/python3 t1.py
> 1157.4882792789995 -- end train, start del
> 27.91983470674 -- end

While the OP reported, for the original program:

"""
With PYTHONMALLOC=malloc, insertion takes nearly twice as long, but
deallocation only takes 2 minutes!
"""

The "nearly twice as long" for building the tree is in sharp contrast
to what you saw, but there's about 3x as much of everything in the
original program, and, e.;g., it's certainly possible malloc is
tripping over fragmentation then that obmalloc avoids.



> $ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
> PYTHONMALLOC=malloc local/bin/python3 t1.py
> 1098.4383037820007 -- end train, start del
> 117.93938426599925 -- end
>
> In this case, glibc malloc is the fastest.
> glibc is know to weak about fragmentation.
> But algorithm to avoid fragmentation is just an overhead in this script.

I can't say.

I'll note that the approach I very briefly sketched before
(restructure the list of arenas to partition it into multiple lists
partitioned by number of free pools) "should make" obmalloc
competitive with malloc here (it would eliminate all searches, except
perhaps (depending on how gonzo the code is) a rare need to search for
"the first" non-empty list).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Python in next Windows 10 update

2019-05-24 Thread Giampaolo Rodola'
On Wed, 22 May 2019 at 03:30, Steve Dower  wrote:

> Hi all
>
> Just sharing this here because I think it's important for us to be aware
> of it - I'm not trying to promote or sell anything here :) (Those who
> were at the language summit have seen this already.)
>
> In the next Windows 10 update that starts rolling out today, we
> (Microsoft) have added "python.exe" and "python3.exe" commands that are
> installed on PATH *by default* and will open the Microsoft Store at the
> page where we (Python core team) publish our build.
>
> This makes it a 1-2 click process to get from a clean machine to having
> a usable Python install ("python.exe" -> opens Store -> "Get it Free" ->
> "python.exe" now works!)
>
> The associated blog post:
>
>
> https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update/
>
> Here are answers to a few questions that I assume will come up, at least
> from this audience that understands the issues better than most:
>
> * if someone had installed Python and put it on PATH with our installer,
> this new command *does not* interfere
> * if someone had manually modified their own PATH, they *may* see some
> interference (but we [Microsoft] decided this was an acceptable risk)
> * the Python 3.7 installed from the store will not auto-update to 3.8,
> but when 3.8 is released we (Microsoft) will update the redirect to
> point at it
> * if you pass arguments to the redirect command, it just exits with an
> error code - you only get the Store page if you run it without arguments
> * once the Store package is installed, the redirect command is replaced
> (this required a new feature in the OS). If you install with the regular
> installer and update PATH, or active a venv, it will add it *before* the
> redirect. So these scenarios should be all good.
>
> I'm happy to answer other questions here. The long-term contact for this
> integration is python (at) microsoft.com, which right now will come to me.
>
> And on a personal note, I'm very excited that we (Microsoft) got the
> approval to do this. Getting *anything* added to Windows is a big task,
> so it's a reflection of the popularity and support for Python that's
> growing within Microsoft that we were able to make this happen. That's
> due to every contributor, both to the core runtime and the ecosystem. I
> hope this will only help us improve the availability of Python for users
> and make it an easier choice for dev tasks in the future.
>
> Cheers,
> Steve
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com



>
I am really glad this happened. I think that in a sense this could be
considered sort of historical.
-- 
Giampaolo - http://grodola.blogspot.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done

2019-05-24 Thread Nathaniel Smith
On Fri, May 24, 2019, 08:08 Ben Cail  wrote:

>
> Why not have the PSF hire someone (or multiple people) to be paid to
> work on the maintenance burden? This could be similar to the Django
> fellows:
> https://www.djangoproject.com/fundraising/#who-is-the-django-fellow. It
> seems like a good thing for Django, and Python is used by many more
> people than Django. Why not pay someone to do the work that others don't
> want to do? The person in this position could be guided by the PSF
> and/or the Steering Council, to do the work most necessary for the good
> of the language as a whole (like maintaining old modules that other core
> devs don't want to work on).
>
> You could market it together with the maintenance burden: "you want to
> use all these old modules, but we don't want to maintain them. So pay us
> some money, and we'll hire someone to maintain them."
>

I think the basic idea here is a good one, but:

- transitioning from an all-volunteer project to a mixed
paid-staff+volunteers project is a big step, and we'll need to take time to
figure out what that would look like before people are comfortable with it.

- even if we have folks paid to help with maintenance, it won't mean we
suddenly have infinite resources and can do everything. We'll still need to
pick which things to prioritize. And I think if you asked 100 people to
name the most critical issues facing Python today, most of them would not
say "maintaining xdrlib".

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Summary of Python tracker Issues

2019-05-24 Thread Python tracker


ACTIVITY SUMMARY (2019-05-17 - 2019-05-24)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7067 ( +0)
  closed 41721 (+88)
  total  48788 (+88)

Open issues with patches: 2849 


Issues opened (59)
==

#34651: Disallow fork in a subinterpreter.
https://bugs.python.org/issue34651  reopened by vstinner

#36950: test.support: add an helper to wait for an event with a timeou
https://bugs.python.org/issue36950  opened by vstinner

#36953: Remove collections ABCs?
https://bugs.python.org/issue36953  opened by p-ganssle

#36954: test_recursive_repr breaks tracing in test_xml_etree
https://bugs.python.org/issue36954  opened by gphemsley

#36956: Calling "functions" used to implement generators/comps easily 
https://bugs.python.org/issue36956  opened by bup

#36959: ISO date errors in _strptime are jumbled
https://bugs.python.org/issue36959  opened by gphemsley

#36960: Make datetime docs more user-friendly
https://bugs.python.org/issue36960  opened by bsolomon1124

#36967: Eliminate unnecessary check in _strptime when determining AM/P
https://bugs.python.org/issue36967  opened by gphemsley

#36968: Top level transient modal windows stopping deiconify on window
https://bugs.python.org/issue36968  opened by Daniel Law

#36969: pdb.do_args: display keyword-only and positional only argument
https://bugs.python.org/issue36969  opened by blueyed

#36971: Add subsections in C API "Common Object Structures" page
https://bugs.python.org/issue36971  opened by jdemeyer

#36973: test_json.test_recursion.TestPyRecursion.test_endless_recursio
https://bugs.python.org/issue36973  opened by xtreak

#36974: Implement PEP 590
https://bugs.python.org/issue36974  opened by jdemeyer

#36976: email: AttributeError
https://bugs.python.org/issue36976  opened by alter-bug-tracer

#36977: SharedMemoryManager should relase its resources when its paren
https://bugs.python.org/issue36977  opened by pierreglaser

#36979: ncurses extension uses wrong include path
https://bugs.python.org/issue36979  opened by chargr

#36982: Add support for extended color functions in ncurses 6.1
https://bugs.python.org/issue36982  opened by websurfer5

#36983: typing.__all__ has drifted from actual contents
https://bugs.python.org/issue36983  opened by Anthony Sottile

#36984: typing docs "versionadded" is inaccurate for many attributes
https://bugs.python.org/issue36984  opened by Anthony Sottile

#36985: typing.ForwardRef is undocumented
https://bugs.python.org/issue36985  opened by Anthony Sottile

#36986: tarfile: unexpected IsADirectoryError on extraction
https://bugs.python.org/issue36986  opened by alter-bug-tracer

#36988: zipfile: string IndexError on extract
https://bugs.python.org/issue36988  opened by alter-bug-tracer

#36989: test_thread fails because symbol is (no longer) exported
https://bugs.python.org/issue36989  opened by Michael.Felt

#36990: test_asyncio.test_create_connection_ipv6_scope fails(in mock t
https://bugs.python.org/issue36990  opened by Michael.Felt

#36996: unittest.mock.patch decorator doesn't work with async function
https://bugs.python.org/issue36996  opened by xtreak

#36997: Document that spwd is considered harmful
https://bugs.python.org/issue36997  opened by christian.heimes

#36998: distutils sdist command fails to create MANIFEST if any filena
https://bugs.python.org/issue36998  opened by a.badger

#36999: Expose the coroutine object in asyncio task objects
https://bugs.python.org/issue36999  opened by alex.gronholm

#37001: symtable.symtable doesn't accept bytes which leads to a mismat
https://bugs.python.org/issue37001  opened by dino.viehland

#37002: PyType_FromSpec can't create immutable types
https://bugs.python.org/issue37002  opened by dino.viehland

#37003: ast unparse does not support f-string new debug format.
https://bugs.python.org/issue37003  opened by mbussonn

#37004: SequenceMatcher.ratio() noncommutativity not well-documented
https://bugs.python.org/issue37004  opened by Dennis Sweeney

#37005: bz2 module doesn't write end-of-stream marker
https://bugs.python.org/issue37005  opened by Dobatymo

#37006: Add top level await statement support for doctest
https://bugs.python.org/issue37006  opened by xtreak

#37007: Implement socket.if_{nametoindex,indextoname} for Windows
https://bugs.python.org/issue37007  opened by dtantsur

#37009: Threading and THREAD_SAFE for AIX
https://bugs.python.org/issue37009  opened by Michael.Felt

#37010: Review performance of inspect.getfullargspec
https://bugs.python.org/issue37010  opened by ncoghlan

#37011: pdb: restore original tracing function instead of sys.settrace
https://bugs.python.org/issue37011  opened by blueyed

#37012: Possible crash due to PyType_FromSpecWithBases()
https://bugs.python.org/issue37012  opened by petr.viktorin

#37013: Fatal Python error in socket.if_indextoname()
https://bugs.python.org/is

Re: [Python-Dev] [python-committers] PEP 595: Improving bugs.python.org

2019-05-24 Thread Gregory P. Smith
-cc: committers to avoid crossposting.

I have feedback for roundup as experienced on BPO that should be
represented within PEP 595 if we are going to have a summary of "improving
roundup for BPO" captured in a PEP (presumably already rejected given 581?
But good to have documented regardless so _thank you_ for doing this
writeup even though I realize our plan of record may be demoralizing for
you).

> **Flexible UI.**  While Roundup UI might look dated, it is convenient and
flexible.

I wholly disagree with this statement.

The BPO roundup UI drives me nuts. every. single. time. I have to use it.
It is not optimized for common workflows users actually need to accomplish
when using a bug tracker.  Two example usability issues (of many): Users
can't read the latest update to a bug of length because it is hidden within
the *middle* of the scrolling region, they must hunt for it.  After
reading, the text box to add to the discussion is oddly located near the
*top* of the scrolling region so that a user cannot see context of a bug
discussion they are responding to as they type.  I file things like this
under "User experience design is needed for the common workflows of all
classes of users".

Roundup needs a modern responsive web interface, not GET/POST request based
interface seen on BPO.  As a result of that, roundup *feels* like is
belongs in the Pre-2004 era interface wise by being web form and full page
reload server for everything. A responsive modern "async javascript
requests happen in the background of the UI" system that we all expect of
any web UI is needed. Not just tweaking the existing thing to have a mobile
friendly version of the web form. This includes persistent connections so
that updates to an issue show up live as they happen instead of getting an
error message "someone/something else has updated this issue since you
started typing, the action you wanted to take such as submitting that
comment or editing that field is now invalid and cannot be completed
without a lot of manual work figuring out what happened, cut and pasting,
and fixing things up on the you the users part".

I'm not going to try proposing a PR to this PEP encapsulating that, I'll
leave that up to anyone willing to wrangle such a PEP.  The list archive
has it regardless now. :)

-Greg


On Thu, May 23, 2019 at 1:17 PM Ezio Melotti  wrote:

> Hello,
> Berker and I have been working on a PEP that suggests we keep using
> and improving bugs.python.org and Roundup instead of switching to
> GitHub Issues as proposed by PEP 581.
>
> The PEP covers:
> * What are the advantages of Roundup over GitHub issues;
> * What features are missing in Roundup and how can we add them;
> * Issues with PEP 581;
> * Issues with the migration plan proposed by PEP 588;
>
> The rendered version of PEP 595 is available at
> https://www.python.org/dev/peps/pep-0595/
>
> For reference, you can consult PEP 581 and 588 at
> https://www.python.org/dev/peps/pep-0581/ and
> https://www.python.org/dev/peps/pep-0588/
>
> The full text of the PEP is include below.  We are planning to update
> the PEP to include the feedback we receive and to update the status of
> features as we implement them (we also have a Google Summer of Code
> students working on it).
>
> Best Regards,
> Ezio Melotti
>
>
> 
> PEP: 595
> Title: Improving bugs.python.org
> Author: Ezio Melotti , Berker Peksag
> 
> Status: Draft
> Type: Process
> Content-Type: text/x-rst
> Created: 12-May-2019
>
>
> Abstract
> 
>
> This PEP proposes a list of improvements to make bugs.python.org
> more usable for contributors and core developers.  This PEP also
> discusses why remaining on Roundup should be preferred over
> switching to GitHub Issues, as proposed by :pep:`581`.
>
>
> Motivation
> ==
>
> On May 14th, 2019 :pep:`581` has been accepted [#]_ without much
> public discussion and without a clear consensus [#]_.  The PEP
> contains factual errors and doesn't address some of the
> issues that the migration to GitHub Issues might present.
>
> Given the scope of the migration, the amount of work required,
> and how it will negatively affect the workflow during the
> transition phase, this decision should be re-evaluated.
>
>
> Roundup advantages over GitHub Issues
> =
>
> This section discusses reasons why Roundup should be preferred
> over GitHub Issues and Roundup features that are not available
> on GitHub Issues.
>
> * **Roundup is the status quo.**  Roundup has been an integral
>   part of the CPython workflow for years.  It is a stable product
>   that has been tested and customized to adapt to our needs as the
>   workflow evolved.
>
>   It is possible to gradually improve it and avoid the disruption
>   that a switch to a different system would inevitabily bring to
>   the workflow.
>
> * **Open-source and Python powered.**  Roundup is an open-source
>   project and is written in Python.  By using it and supporting
>   it, we

Re: [Python-Dev] PEP 594: update 1

2019-05-24 Thread Brett Cannon
On Thu, May 23, 2019 at 5:45 PM Steven D'Aprano  wrote:

> On Thu, May 23, 2019 at 02:06:13PM -0700, Brett Cannon wrote:
> > On Wed, May 22, 2019 at 1:23 PM Sean Wallitsch <
> > sean.wallit...@dreamworks.com> wrote:
> >
> > > My apologies for that oversight. My understanding is that many of the
> > > methods present in aifc depend heavily on audioop for reading and
> writing.
> > >
> >
> > But are people using audioop directly?
>
> Yes they are.
>
>
> https://old.reddit.com/r/Python/comments/brgl8v/pep_594_removing_dead_batteries_from_the_standard/eodvexl/
>
> Shouldn't the PEP be responsable for establishing (as well as any
> negative can be proven) that these modules aren't used, rather than
> merely assuming that they aren't?
>

> Of course it is hard to establish that a module isn't being used. Not
> all code can be searched for on the Internet, there's huge amounts of
> non-open source code written by users that never leaves their computer,
> or behind corporate firewalls.
>

Exactly, hence why we are going through the PEP process on this and not
simply deprecating everything outright without public discussion.


>
> The difficulty of proving this negative requires that we be especially
> conservative about removal. At the very least, we should demonstrate
> that the package is *an active burden*.
>
> Its not enough to say "nobody has touched this module for ages" since
> stable, working code doesn't rot unless we change the language (or the
> std lib!) and break it.
>
> So far, the PEPs record hasn't been too good: out of the 31 modules
> listed in the original draft, the PEP marks four as "keep", leaving 27
> to be deprecated. So far, we've found:
>

I don't think it's fair to be saying Christian isn't doing "too good"
simply because he took a stab at trying to figure out some way to know
which modules would make sense to remove and he got it wrong for some of
them from some people's perspective. As you pointed out, there's no
scientific way to do this ahead of time due to closed source code (e.g. the
VFX industry is not publishing all of their asset pipeline tools so there
was no way to know ahead of time without asking like we are with the PEP),
so he had to start somewhere. And this initial list was discussed at the
PyCon US 2018 language summit as a good one to start from so he isn't
entirely guessing without some initial support to try this list out.


>
> - the sound libraries are in heavy use by hobbyists and the professional
> audio-visual industry; aifc in particular is a mature, stable library
> that isn't break and doesn't need much attention;
>
> - cgi and cgitb are used by people who don't need a heavyweight HTML
> solution (see Reddit);
>
> - colorsys, fileinput and nntplib (at least) have been added to the
> "keep" list;
>
> - the removal of spwd (and crypt?) has been controversial.
>
> So far, nearly 30% of the so-called "dead batteries" turn out to be not
> so dead after all.
>
> I may have missed some. Nor have I even talked much about modules which
> I personally use occasionally, like binhex, because it's not about *me*
> its about the community. As much as I would prefer binhex to remain, if
> it is removed I will be disappointed but I'll make do.
>

I personally think it's about both us and the community. The community can
and does ask for stuff all the time, but we have to balance that with what
we as a team are capable of handling and in my opinion the stdlib is too
big right now for us to maintain appropriately. Plus there's an asymmetric
ask here when the community says they want something while we have to keep
it going.


>
> Speaking of binhex, I tried to get a sense of how much of a burden it
> actually is. There was a comment by Guido in *2007*:
>
>
> https://github.com/python/cpython/commit/34a042d301d6ab88645046a6dfa6c38265ca4b39
>
> "This is the last time I fix binhex. If it breaks again it goes in the
> dustbin"
>
> which of course is Guido's perogative to say Won't Fix. Its now 2019 and
> it hasn't broken again, or at least it hasn't broken again sufficiently
> to make any difference. Serhey found one issue:
>
> https://bugs.python.org/issue29566
>
> but I'm not sure if that's actually a bug or not. There was a
> documentation bug (in the source code) in 2017:
>
> https://bugs.python.org/issue29557
>
> and one enhancement request to clean up the code:
>
> https://bugs.python.org/issue34063
>
> Other than that, Serhey touched binhex.py as part of a mass patch to
> almost the whole stdlib to modernise file handling to use "with".
>
> So by my count, in 12 years since Guido's frustrated comment about
> binhex, it has seen:
>
> - removal of an obsolete and ambiguous comment from the source code;
> - addition of a few with blocks to modernise file handling;
> - one enhancement request still outstanding;
> - one possibly a bug (or maybe not) still outstanding.
>
>
> I may have missed some, but we're talking about one issue per three
> years. How is this a maintenance bur

Re: [Python-Dev] PEP 594 -- Bundling libraries?

2019-05-24 Thread Brett Cannon
On Fri, May 24, 2019 at 12:20 AM Inada Naoki  wrote:

> When removing libraries from stdlib, can we bundle
> removed libraries and install it like ensurepip does?
>

I think that would require people picking those modules up and maintaining
them. But even then I don't know how easy it would be to communicate that
those modules are no longer under our care and so please report bugs and
feature requests to that module's own repository before they open an issue
on bpo.

-Brett


>
> Ruby does similar thing, called "Gemification".
> See https://rubykaigi.org/2017/presentations/hsbt.html
>
> When people don't use venv, scripts importing nntplib
> or aifc runs correctly.
>
> When people use venv, they need to `pip install nntplib`
> (we may be able to use PyPI's namespace beta feature here).
> But I assume `pip install` is not a problem for people using venv.
>
> Regards,
> --
> Inada Naoki  
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done

2019-05-24 Thread Brett Cannon
On Fri, May 24, 2019 at 11:07 AM Nathaniel Smith  wrote:

> On Fri, May 24, 2019, 08:08 Ben Cail  wrote:
>
>>
>> Why not have the PSF hire someone (or multiple people) to be paid to
>> work on the maintenance burden? This could be similar to the Django
>> fellows:
>> https://www.djangoproject.com/fundraising/#who-is-the-django-fellow. It
>> seems like a good thing for Django, and Python is used by many more
>> people than Django. Why not pay someone to do the work that others don't
>> want to do? The person in this position could be guided by the PSF
>> and/or the Steering Council, to do the work most necessary for the good
>> of the language as a whole (like maintaining old modules that other core
>> devs don't want to work on).
>>
>> You could market it together with the maintenance burden: "you want to
>> use all these old modules, but we don't want to maintain them. So pay us
>> some money, and we'll hire someone to maintain them."
>>
>
> I think the basic idea here is a good one, but:
>
> - transitioning from an all-volunteer project to a mixed
> paid-staff+volunteers project is a big step, and we'll need to take time to
> figure out what that would look like before people are comfortable with it.
>
> - even if we have folks paid to help with maintenance, it won't mean we
> suddenly have infinite resources and can do everything. We'll still need to
> pick which things to prioritize. And I think if you asked 100 people to
> name the most critical issues facing Python today, most of them would not
> say "maintaining xdrlib".
>

I'll just quickly say that the steering council has gotten out from under
the PEP backlog enough to start a conversation with the PSF about staffing.
It's initially around PM-type things (e.g. Python 2 sunset, PEP 588), but
if that works out then I would assume we would expand the conversation to
other types of staffing. IOW it's too early to start talking about this
topic but just know the steering council is aware of what the DSF does and
the general idea of potentially hiring folks to help on the dev-side of
things (and Nathaniel's point about setting expectations I think is spot-on
:) .
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] we will probably be having an difficult discussion about the stdlib after PEP 594 is done (was: PEP 594: Removing dead batteries from the standard library)

2019-05-24 Thread Nathaniel Smith
On Thu, May 23, 2019 at 2:18 PM Brett Cannon  wrote:
> I'm personally viewing it as a first step in addressing the maintenance 
> burden we have with such a large stdlib. Christian started this work over a 
> year ago and I think it's worth seeing through. After that we should probably 
> have a discussion as a team about how we view the stdlib long-term and how 
> that ties into maintaining it so that people's opinion of the stdlib's 
> quality goes up rather than viewing the quality of it as varying 
> module-to-module.

I started a thread on discourse to discuss some "what if" scenarios
here, in the hopes it will help us make more informed decisions:
https://discuss.python.org/t/1738

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] [python-committers] PEP 595: Improving bugs.python.org

2019-05-24 Thread Ezio Melotti
On Fri, May 24, 2019, 20:23 Gregory P. Smith  wrote:

> -cc: committers to avoid crossposting.
>

+1 (I wanted to include committers, since the announcement about PEP 581
was posted there too, but it's better to keep the discussion here)


> I have feedback for roundup as experienced on BPO that should be
> represented within PEP 595 if we are going to have a summary of "improving
> roundup for BPO" captured in a PEP (presumably already rejected given 581?
> But good to have documented regardless so _thank you_ for doing this
> writeup even though I realize our plan of record may be demoralizing for
> you).
>

We would like people to re-evaluate the decision, but if that doesn't
happen I think the PEP is still useful, since it provides a fair view of
Roundup capabilities and discusses things that we will have to take into
account if we proceed with the migration -- that's why we decided to go
ahead and write the PEP.


> > **Flexible UI.**  While Roundup UI might look dated, it is convenient
> and flexible.
>
> I wholly disagree with this statement.
>

> The BPO roundup UI drives me nuts. every. single. time. I have to use it.
> It is not optimized for common workflows users actually need to accomplish
> when using a bug tracker.  Two example usability issues (of many): Users
> can't read the latest update to a bug of length because it is hidden within
> the *middle* of the scrolling region, they must hunt for it.
>

This came up in the past, and two solutions have been proposed already:
1) keyboard shortcuts have been added in the issue page to quickly jump to
the first/previous/next/last message and to the response box [0].  They
support both mnemonic (f/p/n/l/r) and vim-style (h/k/j/l/i) combinations.
You can find a summary table in the left sidebar of the issue page, under
help -> keyboard shortcuts.
2) a patch to collapse the history by default (so that the last message was
at the end of the page) was proposed and merged [1], but it was reverted
after a few days because some devs wanted direct access to the history
without having to do an extra click every time to expand it.

After reading, the text box to add to the discussion is oddly located near
> the *top* of the scrolling region so that a user cannot see context of a
> bug discussion they are responding to as they type.
>

This has also been discussed and different people had different opinion.
Some suggested to reverse the order of the messages so that the last
message is at the top near the reply box (like Twitter does), but other
said it's unnatural to read.  Some suggested to put the reply box at the
bottom; however if the other fields are left at the top you would have to
go back up to set them, and if they are moved down you won't easily see
them at the top when you open an existing issue.  Another solution is
duplicating the fields and response box at the top and bottom.


What I mean with *flexible* when I wrote the PEP, is that, unlike GitHub
(where they decide), we can customize bpo however we want (as long as we
agree on what we want -- we could even have per-user settings if we really
want to :)


I think I last heard discussion on the position of the response box in 2011
(when shortcuts and history collapsing were discussed).  Maybe people
didn't care enough about it so they didn't bother bringing it up or they
didn't know it could be changed.  If people do speak up, we can change
bpo/Roundup.


  I file things like this under "User experience design is needed for the
> common workflows of all classes of users".
>
> Roundup needs a modern responsive web interface, not GET/POST request
> based interface seen on BPO.  As a result of that, roundup *feels* like
> is belongs in the Pre-2004 era interface wise by being web form and full
> page reload server for everything. A responsive modern "async javascript
> requests happen in the background of the UI" system that we all expect of
> any web UI is needed. Not just tweaking the existing thing to have a mobile
> friendly version of the web form. This includes persistent connections so
> that updates to an issue show up live as they happen instead of getting an
> error message "someone/something else has updated this issue since you
> started typing, the action you wanted to take such as submitting that
> comment or editing that field is now invalid and cannot be completed
> without a lot of manual work figuring out what happened, cut and pasting,
> and fixing things up on the you the users part".
>

This is a good point and I think it can be done now that Roundup has a REST
API.


> I'm not going to try proposing a PR to this PEP encapsulating that, I'll
> leave that up to anyone willing to wrangle such a PEP.  The list archive
> has it regardless now. :)
>

Thanks a lot for the feedback, I'll update the PEP once I get back to a PC
(using mobile now).

Best Regards,
Ezio Melotti

[0]:
https://bitbucket.org/python/tracker-cpython/commits/dbe3912f93895cd1c44fd353a0fd5db1b467a075
[1]:
https

Re: [Python-Dev] [python-committers] PEP 595: Improving bugs.python.org

2019-05-24 Thread Gregory P. Smith
On Fri, May 24, 2019 at 1:48 PM Ezio Melotti  wrote:

>
> On Fri, May 24, 2019, 20:23 Gregory P. Smith  wrote:
>
>> -cc: committers to avoid crossposting.
>>
>
> +1 (I wanted to include committers, since the announcement about PEP 581
> was posted there too, but it's better to keep the discussion here)
>
>
>> I have feedback for roundup as experienced on BPO that should be
>> represented within PEP 595 if we are going to have a summary of "improving
>> roundup for BPO" captured in a PEP (presumably already rejected given 581?
>> But good to have documented regardless so _thank you_ for doing this
>> writeup even though I realize our plan of record may be demoralizing for
>> you).
>>
>
> We would like people to re-evaluate the decision, but if that doesn't
> happen I think the PEP is still useful, since it provides a fair view of
> Roundup capabilities and discusses things that we will have to take into
> account if we proceed with the migration -- that's why we decided to go
> ahead and write the PEP.
>
>
>> > **Flexible UI.**  While Roundup UI might look dated, it is convenient
>> and flexible.
>>
>> I wholly disagree with this statement.
>>
>
>> The BPO roundup UI drives me nuts. every. single. time. I have to use
>> it.  It is not optimized for common workflows users actually need to
>> accomplish when using a bug tracker.  Two example usability issues (of
>> many): Users can't read the latest update to a bug of length because it is
>> hidden within the *middle* of the scrolling region, they must hunt for it.
>>
>
> This came up in the past, and two solutions have been proposed already:
> 1) keyboard shortcuts have been added in the issue page to quickly jump to
> the first/previous/next/last message and to the response box [0].  They
> support both mnemonic (f/p/n/l/r) and vim-style (h/k/j/l/i) combinations.
> You can find a summary table in the left sidebar of the issue page, under
> help -> keyboard shortcuts.
> 2) a patch to collapse the history by default (so that the last message
> was at the end of the page) was proposed and merged [1], but it was
> reverted after a few days because some devs wanted direct access to the
> history without having to do an extra click every time to expand it.
>
> After reading, the text box to add to the discussion is oddly located near
>> the *top* of the scrolling region so that a user cannot see context of a
>> bug discussion they are responding to as they type.
>>
>
> This has also been discussed and different people had different opinion.
> Some suggested to reverse the order of the messages so that the last
> message is at the top near the reply box (like Twitter does), but other
> said it's unnatural to read.  Some suggested to put the reply box at the
> bottom; however if the other fields are left at the top you would have to
> go back up to set them, and if they are moved down you won't easily see
> them at the top when you open an existing issue.  Another solution is
> duplicating the fields and response box at the top and bottom.
>

The two other modern bugtracker UIs I use on a regular basis do this by
having all such non-conversation metainfo in a persistent top and side bar
boxes such that it is always present.  Keeping the conversation and
metainfo changes listed in order (latest message at the bottom, metainfo
deltas interspersed in between messages all ordered/grouped by timestamp,
and reply text entry box below that.  These two are typically big-screen
engineering UIs (github being one of them), if mobile is desired i expect
these would effectively wind up as a multi-pane UI.  There's a third ticket
that I use for non-engineering stuff which does things in reverse order
with the text entry up top and messages in reverse chronological order
below that.  This reversal always annoys me; probably because I spend most
of my time in the others so it forces me to do headstands. I don't know if
there is a *right* answer between the two approaches, I just know what I
prefer. But the common theme is keeping new the update UI elements right
next to the most recent entries which is what BPO lacks today.

What I mean with *flexible* when I wrote the PEP, is that, unlike GitHub
> (where they decide), we can customize bpo however we want (as long as we
> agree on what we want -- we could even have per-user settings if we really
> want to :)
>

Definitely true in that sense.  Custom things, no matter where, are custom
code.  diverge from an upstream project in our own fork and maintenance and
updates become harder.  Author a pile of bots to automate tasks within a
standard API or UI (as is working great for our Github PR workflow) and the
equivalent maintenance burden falls on keeping the bots working.

I think I last heard discussion on the position of the response box in 2011
> (when shortcuts and history collapsing were discussed).  Maybe people
> didn't care enough about it so they didn't bother bringing it up or they
> didn't know it could be changed.  If peo

Re: [Python-Dev] [python-committers] PEP 595: Improving bugs.python.org

2019-05-24 Thread Ezio Melotti
On Fri, May 24, 2019, 23:14 Gregory P. Smith  wrote:

>
>
> On Fri, May 24, 2019 at 1:48 PM Ezio Melotti 
> wrote:
>
>>
>> On Fri, May 24, 2019, 20:23 Gregory P. Smith  wrote:
>>
>>> -cc: committers to avoid crossposting.
>>>
>>
>> +1 (I wanted to include committers, since the announcement about PEP 581
>> was posted there too, but it's better to keep the discussion here)
>>
>>
>>> I have feedback for roundup as experienced on BPO that should be
>>> represented within PEP 595 if we are going to have a summary of "improving
>>> roundup for BPO" captured in a PEP (presumably already rejected given 581?
>>> But good to have documented regardless so _thank you_ for doing this
>>> writeup even though I realize our plan of record may be demoralizing for
>>> you).
>>>
>>
>> We would like people to re-evaluate the decision, but if that doesn't
>> happen I think the PEP is still useful, since it provides a fair view of
>> Roundup capabilities and discusses things that we will have to take into
>> account if we proceed with the migration -- that's why we decided to go
>> ahead and write the PEP.
>>
>>
>>> > **Flexible UI.**  While Roundup UI might look dated, it is convenient
>>> and flexible.
>>>
>>> I wholly disagree with this statement.
>>>
>>
>>> The BPO roundup UI drives me nuts. every. single. time. I have to use
>>> it.  It is not optimized for common workflows users actually need to
>>> accomplish when using a bug tracker.  Two example usability issues (of
>>> many): Users can't read the latest update to a bug of length because it is
>>> hidden within the *middle* of the scrolling region, they must hunt for it.
>>>
>>
>> This came up in the past, and two solutions have been proposed already:
>> 1) keyboard shortcuts have been added in the issue page to quickly jump
>> to the first/previous/next/last message and to the response box [0].  They
>> support both mnemonic (f/p/n/l/r) and vim-style (h/k/j/l/i) combinations.
>> You can find a summary table in the left sidebar of the issue page, under
>> help -> keyboard shortcuts.
>> 2) a patch to collapse the history by default (so that the last message
>> was at the end of the page) was proposed and merged [1], but it was
>> reverted after a few days because some devs wanted direct access to the
>> history without having to do an extra click every time to expand it.
>>
>> After reading, the text box to add to the discussion is oddly located
>>> near the *top* of the scrolling region so that a user cannot see context of
>>> a bug discussion they are responding to as they type.
>>>
>>
>> This has also been discussed and different people had different opinion.
>> Some suggested to reverse the order of the messages so that the last
>> message is at the top near the reply box (like Twitter does), but other
>> said it's unnatural to read.  Some suggested to put the reply box at the
>> bottom; however if the other fields are left at the top you would have to
>> go back up to set them, and if they are moved down you won't easily see
>> them at the top when you open an existing issue.  Another solution is
>> duplicating the fields and response box at the top and bottom.
>>
>
> The two other modern bugtracker UIs I use on a regular basis do this by
> having all such non-conversation metainfo in a persistent top and side bar
> boxes such that it is always present.  Keeping the conversation and
> metainfo changes listed in order (latest message at the bottom, metainfo
> deltas interspersed in between messages all ordered/grouped by timestamp,
> and reply text entry box below that.  These two are typically big-screen
> engineering UIs (github being one of them), if mobile is desired i expect
> these would effectively wind up as a multi-pane UI.  There's a third ticket
> that I use for non-engineering stuff which does things in reverse order
> with the text entry up top and messages in reverse chronological order
> below that.  This reversal always annoys me; probably because I spend most
> of my time in the others so it forces me to do headstands. I don't know if
> there is a *right* answer between the two approaches, I just know what I
> prefer. But the common theme is keeping new the update UI elements right
> next to the most recent entries which is what BPO lacks today.
>
> What I mean with *flexible* when I wrote the PEP, is that, unlike GitHub
>> (where they decide), we can customize bpo however we want (as long as we
>> agree on what we want -- we could even have per-user settings if we really
>> want to :)
>>
>
> Definitely true in that sense.  Custom things, no matter where, are custom
> code.  diverge from an upstream project in our own fork and maintenance and
> updates become harder.  Author a pile of bots to automate tasks within a
> standard API or UI (as is working great for our Github PR workflow) and the
> equivalent maintenance burden falls on keeping the bots working.
>

To clarify, Roundup allows you by design to define instances where you can
customize the

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-24 Thread Tim Peters
[Tim]
> I'll note that the approach I very briefly sketched before
> (restructure the list of arenas to partition it into multiple lists
> partitioned by number of free pools) "should make" obmalloc
> competitive with malloc here ...

But it's also intrusive, breaking up a simple linked list into a
mutli-headed beast.  That affects all code using it, not just the
parts mutating it.

So instead I suggest leaving `usable_arenas` entirely alone, but add a
vector of "search fingers", say,

static struct arenaobject* nfp2lasta[MAX_NUMBER_OF_FREE_POOLS_POSSIBLE + 1];

mapping a count of free pools to (a pointer to) the last arena object
in usable_arenas with that number of free pools.

Then the search loop in py_malloc_free() can be replaced with a single
array lookup.:  it leaps directly to where the search loop ends now.
For example, if we free a pool in an arena that had 23 free pools,
then the arena object's count of free pools has to be increased to 24,
and the arena object unlinked from its current position in
usable_arenas and inserted immediately after nfp2lasta[23].  Note that
usable_arenas is a doubly linked list, so there's no _need_ at all to
iterate over it.  Each node in the list knows what's immediately
before and after it.  And since a free pool count can only increase by
1, it's always correct to move the arena immediately after the last
arena with the same original free pool count.

Key invariants:

1. nfp2lasta[n] is NULL if and only if no arena in usable_arenas has
nfreepools == n.

2. nfp2lasta[pa->nfreepools] == pa if and only if pa is the only arena
in usable_arenas with that many free pools.

So, in the example above,  nfp2lasta[23] has to be set to NULL if and
only if the arena in question was the only one with 23 free pools
(which can be determined cheaply via invariant #2).

And nfp2lasta[24] has to be set to point to the arena in question if
and only if nfp2lasta[24] is NULL.

Tricky bit:  if the arena in question is the only one with a given
original free pool count, no pointers in arena objects need to be
changed at all.  Following the sketch above literally would leave you
trying to insert it after itself, which wouldn't end well  ;-)

Anyway, this "feels like a right solution" to me, eliminating all
searches with simple (albeit delicate) constant-time code, and
requiring code changes only where an arena's number of free pools can
change.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-24 Thread Tim Peters
[Tim]
> Key invariants:
> ...
> 2. nfp2lasta[pa->nfreepools] == pa if and only if pa is the only arena
> in usable_arenas with that many free pools.

Ack!  Scratch that.  I need a nap :-(

In fact if that equality holds, it means that nfp2lasta entry has to
change if pa is moved and pa->prevarena has the same count of free
pools.

So forget about the explanation and just think about the goal - the
details will work themselves out ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 594: update 1

2019-05-24 Thread Random832
On Thu, May 23, 2019, at 15:27, Steve Holden wrote:
> Besides which, it would be lovely to have a major release that didn't 
> involve any pain at all for the majority of users!
> 
> Our erstwhile BDFL always eschewed two-digit version identifiers- due 
> to the possibility for confusion about collating sequence, I beleive.. 
> We should honour his preferences by going from 3.9 to 4.0.

Is 4.0 still gonna install as "python3", pip3, etc, executable as "py -3" on 
windows? I can see that being a pain point even if nothing else is.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 594: update 1

2019-05-24 Thread Glenn Linderman

On 5/24/2019 9:09 PM, Random832 wrote:

On Thu, May 23, 2019, at 15:27, Steve Holden wrote:

Besides which, it would be lovely to have a major release that didn't
involve any pain at all for the majority of users!

Our erstwhile BDFL always eschewed two-digit version identifiers- due
to the possibility for confusion about collating sequence, I beleive..
We should honour his preferences by going from 3.9 to 4.0.

Is 4.0 still gonna install as "python3", pip3, etc, executable as "py -3" on 
windows? I can see that being a pain point even if nothing else is.


Maybe by then it can go back to just being python, as 2.x will be past 
end-of-life.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com