[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Paul Moore 
On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett  wrote:
>
> Whenever I've used except Exception or stronger, it was a sanitary barrier 
> around code that might well do unpredictable or even stupid things.  Adding a 
> new kind of exception that I hadn't predicted -- including ExceptionGroup -- 
> would certainly fit this description, and I want my driver loop to do what I 
> told it.  (Probably log an unexpected exception, and continue with the next 
> record.  I honestly don't even want a page, let alone a crash, because data 
> from outside that barrier ... is often bad, and almost never in ways the 
> on-call person can safely fix.  And if they don't have time to find it in the 
> logs, then it isn't a priority that week.)

This is my biggest concern. Disclaimer: I've yet to read the PEP,
because async makes my head hurt, but I am aware of some of the
background with Trio. Please take this as the perspective of someone
thinking "I don't use async/await in my code, can I assume this
doesn't affect me?"

For me, the biggest requirement I would have is that if I have code like:

def safe_fn():
try:
do_something_funky()
return True
except Exception:
print("Well, that was a shame...")
return False

then I am intending to guarantee that calling safe_fn() will never
raise an exception. Obviously, the example is a toy (in reality, I'd
log the error, skip a record, whatever) but that's not the point here
- the point is that I consider it reasonable to expect `except
Exception` to be a hard barrier that allows me to be sure I've covered
everything.

My worry is that if ExceptionGroup exceptions are *not* trapped by
`except Exception`, then code like this is no longer safe. And by
making ExceptionGroup a BaseException subclass, that's what will
happen.

Ultimately, there's a genuine use case for code that says "whatever
happens in the code I call, this is my last line of defense and I want
to be as near to 100% sure as I can be that I regain control at this
point". At the moment, that is spelled `except Exception`. If you're
proposing to change that, even if it's just to require that it be
spelled differently, you're going to break quite a lot of code - and
worse, the code you're breaking will be code that has taken care to
ensure it's written safely, so you're penalising people who *tried to
get stuff right*, which IMO is the worst type of breakage.

I will at some point read the PEP fully, to better understand the
async side of the story, but for now please consider this as the
perspective of someone who doesn't expect to care about async, and
therefore wants to feel safe in assuming this PEP won't affect them.

Paul

PS If you're thinking "but if you were using async, you'd know about
it", consider that I could be using a PyPI library that, for example,
returned stock values. That library switches to using async, and has a
bug that lets an exception group escape. My `except Exception` code is
designed *precisely* to protect me against such bugs, without me
needing to know anything about how the library works...
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SKU4RIRPRRGESKRMP55ZPZNYL4XWFFUR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

2021-02-27 Thread Paul Moore 
On Sat, 27 Feb 2021 at 01:08, Oscar Benjamin  wrote:
>
> On Fri, 26 Feb 2021 at 23:06, Jim J. Jewett  wrote:
> >
> > I think his point is that most of his students (economics or business, 
> > rather than comp sci) will never need to use Perl or C or Java.  Python is 
> > friendly enough to be useful, but this is still a major pain point.
>
> Thanks Jim, that is the situation and yes they are not CS students.

I've been in this situation with people who are professional
developers (but in environments where "development" is done in a
self-contained application IDE). I completely understand and
sympathise with the idea that suddenly having to learn command line
skills when all you'd signed up for was to learn Python is both
daunting and off-putting.

> The other point though is that it doesn't need to be like this. If the
> issue was just installing Python and then setting up your PATH then
> that's manageable. The problem is that even after doing those things
> there isn't a "one way" to invoke Python from the command line. All
> possible invocations (python, python3, py, ...) will fail for some
> users. That's a problem for beginners but it's also a problem in any
> situation where you want to write a command line that should run on
> multiple platforms (e.g. in a Makefile or some other kind of script).

This is 100% true. (At least the part where you say that "it's a
problem". I'm not sure if "it doesn't need to be like this" is true -
can you point to a language where the command line "just works" the
way you suggest?)

> I see that the official Python tutorial now suggests typing
> "python3.9" [1]. Is that what is recommended now? Obviously that would
> fail for someone who had installed 3.8.
>
> It would be great if Python could converge around a single way for
> users to invoke the Python that they have installed and would want to
> use. Python 2.x is beginning to fade away so python vs python3 makes
> less sense now but maybe standardising on py for all platforms and
> installs would be better (since py also has features for selecting
> different versions and is not currently used for any "system python").

It *would* be great. I don't think there's anyone who disagrees with
this point. But just thinking "it would be great" isn't of much
practical use. People have been trying to fix this issue for many
years (quite apart from all of the core Python debates, it's an
endless, frustrating topic in the packaging community - I can't tell
you how many times we've tried to come up with a good way to tell
people how to invoke pip).

In my view, for people learning Python in a context where command line
skills are *not* the norm, it would be ideal to focus on one (or more)
"integrated environment", in much the same way as no-one learns
command-line R, they all use R Studio. Unfortunately, the Python
community isn't good at getting behind a single solution like this, so
we have IDLE, Jupyter, Spyder, and probably a host of others.

The command is *hard*. People used to it tend to forget that. I know
people who are good developers (in an IDE) who struggle with how to
specify options to commands, and how to quote arguments. (Not
surprising, as both of those depend on what OS and what shell you're
using, as well as being entirely learned information - there's no
"intuitive" reason why options start with a dash, or you quote with
double quotes not single quotes in Windows). I don't think we should
be pointing beginners at the command line *at all*. But as a
community, we don't really have any better option. (Individual
trainers or organisations can make their own choices, but that's not
easy in the absence of a community consensus).

Paul

PS Having a standard "integrated environment" that people learn in has
its own problems, of course, as we risk training a new generation of
Python users who don't know how to use their Python code outside of
that IDE, and who therefore can't work with git or automated tests,
etc, etc... I'm looking at Java and the "if my IDE doesn't do it, I'm
lost" mindset here. I don't know if R has fallen into a similar trap
with R Studio.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3FEHMUITOB4CW3TFQKFBAS2AEEXJEUXE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

2021-02-27 Thread Oscar Benjamin 
On Sat, 27 Feb 2021 at 11:04, Paul Moore  wrote:
>
> On Sat, 27 Feb 2021 at 01:08, Oscar Benjamin  
> wrote:
> >
> > The other point though is that it doesn't need to be like this. If the
> > issue was just installing Python and then setting up your PATH then
> > that's manageable. The problem is that even after doing those things
> > there isn't a "one way" to invoke Python from the command line. All
> > possible invocations (python, python3, py, ...) will fail for some
> > users. That's a problem for beginners but it's also a problem in any
> > situation where you want to write a command line that should run on
> > multiple platforms (e.g. in a Makefile or some other kind of script).
>
> This is 100% true. (At least the part where you say that "it's a
> problem". I'm not sure if "it doesn't need to be like this" is true -
> can you point to a language where the command line "just works" the
> way you suggest?)

Python from 15 years ago :)

Maybe I'm misremembering but when I first started with Python there
didn't seem to be any ambiguity about what to type in the terminal.
The first split as I remember it was on Linux when python3 came along
but python kept referring to 2.x on (most) Linux. At the same time
python3 was not added to Windows and instead python became 3.x. Then
there was the py launcher for Windows. Perhaps this problem always
existed on OSX although I didn't use it at all at the time.

More recently I've been exploring Julia. You could compare these pages:

https://docs.python.org/3/tutorial/interpreter.html#invoking-the-interpreter
https://docs.julialang.org/en/v1/manual/getting-started/

When installing julia you select the installer for your OS and there
are some instructions for setting up PATH (although I didn't need to
do that manually on OSX). After setup the instruction is that you type
"julia" in the terminal and then Ctrl-D to exit. There are no caveats
or mentions of different operating systems or words like "if" and
"usually". The setup process is platform specific but then the usage
instructions are not.

--
Oscar
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BTDEIUAVZN2GWRVQY34FJHRF6YOWKZKT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

2021-02-27 Thread Steven D'Aprano 
On Sat, Feb 27, 2021 at 12:56:17PM +, Oscar Benjamin wrote:
> On Sat, 27 Feb 2021 at 11:04, Paul Moore  wrote:

> > This is 100% true. (At least the part where you say that "it's a
> > problem". I'm not sure if "it doesn't need to be like this" is true -
> > can you point to a language where the command line "just works" the
> > way you suggest?)
> 
> Python from 15 years ago :)

Fifteen years ago, if I remember correctly, I had Python 1.5, 2.4 and 
2.5 installed. (I may have some of the versions mixed up.)

When I typed `python` at the command line, the OS read the faint 
electrical currents at my finger tips through the keyboard, extrapolated 
the state of my brain, predicted the interpreter I wanted, and 
unfailingly guessed the right one.

Nah just kidding. I'm pretty sure it ran 2.4, always.


> When installing julia you select the installer for your OS and there
> are some instructions for setting up PATH (although I didn't need to
> do that manually on OSX). After setup the instruction is that you type
> "julia" in the terminal and then Ctrl-D to exit. There are no caveats
> or mentions of different operating systems or words like "if" and
> "usually". The setup process is platform specific but then the usage
> instructions are not.

So what happens when I have two different Julia versions 
installed?


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T3HHANHVUZOM67TA72R6LZFTMOZHJ2DM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

2021-02-27 Thread Paul Moore 
On Sat, 27 Feb 2021 at 13:03, Oscar Benjamin  wrote:
> More recently I've been exploring Julia. You could compare these pages:
>
> https://docs.python.org/3/tutorial/interpreter.html#invoking-the-interpreter
> https://docs.julialang.org/en/v1/manual/getting-started/
>
> When installing julia you select the installer for your OS and there
> are some instructions for setting up PATH (although I didn't need to
> do that manually on OSX). After setup the instruction is that you type
> "julia" in the terminal and then Ctrl-D to exit. There are no caveats
> or mentions of different operating systems or words like "if" and
> "usually". The setup process is platform specific but then the usage
> instructions are not.

Sorry, I couldn't resist replying to this.

I went to the Julia page. That linked me to the download page. Which
offered me multiple versions. Which should I take? Stable, LTS or
"upcoming"? As a beginner without the experience I have hitting stuff
like this, how would I know? Let's assume "stable". OK, download and
run an installer is easy enough for me on Windows, but let's say I'm
doing this on Linux, I get the choice of "Generic Linux on x86" or
"Generic Linux on ARM". Huh? I'm on Ubuntu, not "Generic", so I'm
confused. Then I have "64-bit (GPG)" or "64-bit (musl) (GPG)" Or AArch
if I chose "ARM". How the heck would I know. Any of those get me a
.tar.gz file, which I don't know how to use...

Going to the Linux help, I get told to use wget, not the file I just
downloaded. And then I unpack it, and now I'm into setting PATH to
/tmp/julia-something/bin (because I did this in /tmp, as I have no
idea where I want to put this long term).

OK, my linux thought experiment isn't going well. Let's go back to
Windows, where I was lucky enough to pick "installer" rather than
"portable", so I avoided a world of pain about where to put the
package, just like I hit on Linux. The installer puts Julia in my
personal apps directory, just like Python does. I'm going to do
something "advanced" here ;-) and deselect "desktop shortcut" because
I hate clutter on my desktop. OK, it's complete, let's select "run
julia". That got me a prompt. Cool. Start menu item does the same.
Also cool.

BUT... The python installer on windows is *exactly the same*. Installs
to user's area, start menu entry that runs the REPL. So that's no
worse.

Suppose I go to a command prompt on my own and try to run Julia. Open
a terminal, "julia". No good, I need to set PATH. Hmm, wasn't there
something in the help on the Julia website about that? Off we go to
check. Weren't we here before when we were using Python? And the
instructions on the Julia website are about as complex as I'd expect,
and much the same as for Python. So still no worse for Python. But the
Python installer has "add Python to PATH" that does this stuff for you
(not selected by default, because of reasons that you may or may not
agree with, but at least it's *there*). And there's the py launcher,
but let's ignore that, as it works round some complexity at the cost
of some different complexity, so it's a distraction at this stage.

Honestly, I think Julia is overall slightly *worse* than Python (at
least on Windows). But it's almost identical in practice.

The one thing that *is* substantially worse for Python, is the
circumlocutions needed in the documentation to say how to run Python.
But that's 100% down to us not being willing to say "just type the
command python". And the reason for *that* is mostly historical,
related to the Python 3 transition and what happened on Linux over the
python/python2/python3 commands, and to a lesser extent the
introduction of the launcher on Windows ("lesser", because using the
launcher on the command line wasn't recommended until some time after
its introduction, at which point the python2/python3 split was
established)...

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7AVAVWMGNRDLAEQHDLWWTPHMLJG4V7RD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

2021-02-27 Thread Jim J. Jewett 
[They are not CS students]

> Why is that relevant?

Because for many users, python is NOT a programming language; it is an 
application like any other.  It happens to be very powerful and flexible, but 
the point isn't to program; it is to produce better reports.

If the hurdle to get started (either initially, or after a while away) is too 
high, then it is not a useful tool.  It is OK to have all sorts of power-user 
knobs, but the default instructions should just work ... they shouldn't have to 
say "ensure you have proper settings for these 6 things you don't care about or 
understand", because the default install should just choose reasonable default 
settings and take care of them.

> there are some basic fundemental skills that every amateur or
> professional programmer needs to know, such as:

My point is that plenty of python users have no intention of being a programmer.

> how to edit files and save them
(fair ... but plenty of programs don't even *let* you do this with an external 
editor; there is no reason the _default_ should force people to pick and 
configure an external editor.  IDLE tends to do OK here.)

> which file am I actually running?
> which interpreter am I actually running?
> how do I tell the computer to use a different interpreter?

If you need to care about any of these, then the environment is fighting you -- 
and the application probably stinks.  Programmers have to deal with it because 
of bootstrapping, but there is no reason that we should assume all python users 
need that flexibility or want that responsibility.

> Writing code to run on any platform is a hard problem that 
> requires complex solutions.

Thus the preference for solving it once in the library/official 
documentation/reference installer.  Experts can build on it or opt out, but 
non-programmers shouldn't have to worry about cross-platform issues just to use 
python.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FPPE35KUME5JDDRSZ44RXRDN366EJNAC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Jim J. Jewett 
> In fact, if you're happy with RuntimeError here, making
> ExceptionGroup inherit from Exception (instead of
> BaseException) would do just as well -- after all RuntimeError
> is pretty arbitrary

Agreed, and I think it should inherit from Exception.

> (in fact, it's wrong, because the problem is a static bug in the
> code, not something that went wrong at run time).

Something went wrong deep inside an individual task.  The only clear static bug 
is that the async library changed how it reported that, and my except clause 
hasn't done the make-work to keep in step.

> Let's see how this compares to the alternatives. First let's define 
> three key examples.

> Example 1:

> try:
>raise ExceptionGroup(ValueError)
> except Exception:

> Example 2:

> try:
> raise ExceptionGroup(ValueError)
> except ValueError:

Example 2(a) :

 try:
 raise ExceptionGroup(ValueError,OtherError)
 except ValueError:
 
> Example 3:

> try:
>raise ExceptionGroup(asyncio.CancelledError)
> except Exception:

I would prefer that the except clause be executed in all of the examples, but I 
admit that 2a is a problem, because it could end up silently losing OtherError. 
 And I admit getting example 2 to do the right thing if 2a doesn't might be too 
much of a contortion.

For example 3, I may be missing a subtle point, but I feel that by the time you 
get to code which doesn't expect asyncio.CancelledError, then you have already 
cancelled as far as you should.  Cancelling may want to bubble up through 
several other tasks, but it shouldn't kill the whole server, just because the 
trampoline got sloppy.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RHL3E2IBCIW2SP5L5W3BHWIWB4DV6U55/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Guido van Rossum 
On Sat, Feb 27, 2021 at 2:39 AM Paul Moore  wrote:

> On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett  wrote:
> >
> > Whenever I've used except Exception or stronger, it was a sanitary
> barrier around code that might well do unpredictable or even stupid
> things.  Adding a new kind of exception that I hadn't predicted --
> including ExceptionGroup -- would certainly fit this description, and I
> want my driver loop to do what I told it.  (Probably log an unexpected
> exception, and continue with the next record.  I honestly don't even want a
> page, let alone a crash, because data from outside that barrier ... is
> often bad, and almost never in ways the on-call person can safely fix.  And
> if they don't have time to find it in the logs, then it isn't a priority
> that week.)
>
> This is my biggest concern. Disclaimer: I've yet to read the PEP,
> because async makes my head hurt, but I am aware of some of the
> background with Trio. Please take this as the perspective of someone
> thinking "I don't use async/await in my code, can I assume this
> doesn't affect me?"
>

The PEP doesn't actually do anything with async -- it just uses asyncio as
one of the motivating use cases. The spec itself does not use async or
asyncio. At some point in the future we might consider adding new APIs to
asyncio that can raise ExceptionGroup, but other uses of 'async def' will
definitely not be affected.

Our goal for the PEP is that *unless* you're going to use APIs that are
documented to raise ExceptionGroup, you won't have to use `except *` nor
will you have to deal with ExceptionGroup otherwise.


> For me, the biggest requirement I would have is that if I have code like:
>
> def safe_fn():
> try:
> do_something_funky()
> return True
> except Exception:
> print("Well, that was a shame...")
> return False
>
> then I am intending to guarantee that calling safe_fn() will never
> raise an exception. Obviously, the example is a toy (in reality, I'd
> log the error, skip a record, whatever) but that's not the point here
> - the point is that I consider it reasonable to expect `except
> Exception` to be a hard barrier that allows me to be sure I've covered
> everything.
>

The modification I proposed where we have both BaseExceptionGroup and
ExceptionGroup will satisfy this need. It basically means that *unless* you
are explicitly using an API that is documented to raise
[Base]ExceptionGroup (such as a future variant of asyncio.gather()), you
don't have to care about it. The other exception is if you're writing a
library for formatting exceptions (like the stdlib traceback.py module).


> My worry is that if ExceptionGroup exceptions are *not* trapped by
> `except Exception`, then code like this is no longer safe. And by
> making ExceptionGroup a BaseException subclass, that's what will
> happen.
>

Right, and that's why I am proposing to change the PEP so that your code
will remain safe.

Note that the class headings are like this:
```
class BaseExceptionGroup(BaseException): ...
class ExceptionGroup(BaseExceptionGroup, Exception): ...
```
which produces the following MROs:
```
ExceptionGroup -> BaseExceptionGroup -> Exception -> BaseException -> object
  BaseExceptionGroup ->  BaseException -> object
```


Ultimately, there's a genuine use case for code that says "whatever
> happens in the code I call, this is my last line of defense and I want
> to be as near to 100% sure as I can be that I regain control at this
> point". At the moment, that is spelled `except Exception`. If you're
> proposing to change that, even if it's just to require that it be
> spelled differently, you're going to break quite a lot of code - and
> worse, the code you're breaking will be code that has taken care to
> ensure it's written safely, so you're penalising people who *tried to
> get stuff right*, which IMO is the worst type of breakage.
>

I got you.


> I will at some point read the PEP fully, to better understand the
> async side of the story, but for now please consider this as the
> perspective of someone who doesn't expect to care about async, and
> therefore wants to feel safe in assuming this PEP won't affect them.
>

There really is no async side. And you will be safe.


> Paul
>
> PS If you're thinking "but if you were using async, you'd know about
> it", consider that I could be using a PyPI library that, for example,
> returned stock values. That library switches to using async, and has a
> bug that lets an exception group escape. My `except Exception` code is
> designed *precisely* to protect me against such bugs, without me
> needing to know anything about how the library works...
>

And it will.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@pyth

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Guido van Rossum 
On Sat, Feb 27, 2021 at 1:09 PM Jim J. Jewett  wrote:

> > In fact, if you're happy with RuntimeError here, making
> > ExceptionGroup inherit from Exception (instead of
> > BaseException) would do just as well -- after all RuntimeError
> > is pretty arbitrary
>
> Agreed, and I think it should inherit from Exception.
>
> > (in fact, it's wrong, because the problem is a static bug in the
> > code, not something that went wrong at run time).
>
> Something went wrong deep inside an individual task.  The only clear
> static bug is that the async library changed how it reported that, and my
> except clause hasn't done the make-work to keep in step.
>
> > Let's see how this compares to the alternatives. First let's define
> > three key examples.
>
> > Example 1:
>
> > try:
> >raise ExceptionGroup(ValueError)
> > except Exception:
>
> > Example 2:
>
> > try:
> > raise ExceptionGroup(ValueError)
> > except ValueError:
>
> Example 2(a) :
>
>  try:
>  raise ExceptionGroup(ValueError,OtherError)
>  except ValueError:
>
> > Example 3:
>
> > try:
> >raise ExceptionGroup(asyncio.CancelledError)
> > except Exception:
>
> I would prefer that the except clause be executed in all of the examples,
> but I admit that 2a is a problem, because it could end up silently losing
> OtherError.  And I admit getting example 2 to do the right thing if 2a
> doesn't might be too much of a contortion.
>
> For example 3, I may be missing a subtle point, but I feel that by the
> time you get to code which doesn't expect asyncio.CancelledError, then you
> have already cancelled as far as you should.  Cancelling may want to bubble
> up through several other tasks, but it shouldn't kill the whole server,
> just because the trampoline got sloppy.
>

If you're using 'except Exception:' and an asyncio.CancelledError  (or
trio.Cancelled) exception escapes, you're already dead, because those don't
inherit from Exception, only from BaseException. And everything inherits
from BaseException, including [Base]ExceptionGroup.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TJGM3DBVXOUJU4LCD2ULICN4I537HHCY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

2021-02-27 Thread Nick Coghlan 
On Wed, 24 Feb 2021 at 10:49, Random832  wrote:
>
> I was reading a discussion thread 
>  about 
> various issues with the Debian packaged version of Python, and the following 
> statement stood out for me as shocking:
>
> Christian Heimes wrote:
> > Core dev and PyPA has spent a lot of effort in promoting venv because we 
> > don't want users to break their operating system with sudo pip install.
>
> I don't think sudo pip install should break the operating system. And I think 
> if it does, that problem should be solved rather than merely advising users 
> against using it. And why is it, anyway, that distributions whose package 
> managers can't coexist with pip-installed packages don't ever seem to get the 
> same amount of flak for "damaging python's brand" as Debian is getting from 
> some of the people in the discussion thread? Why is it that this community is 
> resigned to recommending a workaround when distributions decide the 
> site-packages directory belongs to their package manager rather than pip, 
> instead of bringing the same amount of fiery condemnation of that practice as 
> we apparently have for *checks notes* splitting parts of the stdlib into 
> optional packages? Why demand that pip be present if we're not going to 
> demand that it works properly?

The reason venv is promoted as heavily as it is is because it's the
only advice that can be given that is consistently correct regardless
of the operating system the user is running locally, whereas safely
using a system-wide Python installation varies a lot depending on
whether you're on Windows, Mac OS X, or Linux (let alone some other
platform outside the big 3 desktop clients).

conda is also popular for the same reason: while the instructions for
installing conda in the first place are OS-dependent, once it is up
and running you can use consistent platform independent conda commands
rather than having to caveat all your documentation with
platform-specific instructions.

Apple moved all of their dynamic language interpreter implementations
to inaccessible-by-default locations so Mac OS X users would stop
using them to run their own code.

Alongside that, we *have* worked with the Linux distro vendors to help
make "sudo pip install" safe (e.g [1]), but that only helps if a user
is running a new enough version of a distro that has participated in
that work.

However, while the option of running "platform native" environments
will never go away, and work will continue to make it less error
prone, the level of knowledge of your specific OS's idiosyncrasies
that it requires is almost certainly going to remain too high for it
to ever again become the default recommendation that it used to be.

Cheers,
Nick.

[1] https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe (Note:
this change mitigated some aspects of the problem in a way similar to
what Debian does, but still doesn't solve it completely, as custom
Python builds may still make arbitrary changes)

P.S. "But what about user site-packages?" you ask. Until relatively
recently, Debian didn't put the user's local bin directory on the
system path by default, so commands provided by user level package
installs didn't work without the user adjusting their PATH. The
CPython Windows installer also doesn't adjust PATH by default (for
good reasons). And unlike a venv, "python -m" doesn't let you ensure
that the code executed is the version installed in user site-packages
- it could be coming from a directory earlier in sys.path.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RDLEH6DUF57UB6U4HNL2QRVAJY4KDSSJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Cameron Simpson 
On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett  wrote:
> Whenever I've used except Exception or stronger, it was a sanitary
> barrier around code that might well do unpredictable or even stupid
> things.  Adding a new kind of exception that I hadn't predicted --
> including ExceptionGroup -- would certainly fit this description, and 
> I want my driver loop to do what I told it.

This is my concern as well. I've got plenty of "except Exception" and 
IMO none consititutes "abuse" - they're there for the same reason Jim 
cites above: a barrier around _arbitrary_ callback code - so that the 
outer control facility continues to roll on as expected.

All of these are handler frameworks for use by other code - they 
inherently call an unknown and arbitrary thing at some point and expect 
to catch an Exception raised and suitably handle it (log it, record it 
for the caller to see later, whatever).

Now _all_ those handlers will need a special except* handler for 
ExceptionGroup, because they DO NOT KNOW whether the called code might 
raise it. And "except Exception", the longstanding recommended way to 
catch "everything (well almost everything)" no longer works.

Now, I've long wanted something like MultiError or ExceptionGroup to 
gather miltiple failures for raising together at a suitable 
slightly-later point.

[...]
On 27Feb2021 19:06, Guido van Rossum  wrote:
>Our goal for the PEP is that *unless* you're going to use APIs that are
>documented to raise ExceptionGroup, you won't have to use `except *` nor
>will you have to deal with ExceptionGroup otherwise.

But.. this isn't Java, where the (hah!) type annotations document the 
exceptions it raises.

Jim again, catching Exception to protect calling code:
>> def safe_fn():
>> try:
>> do_something_funky()
>> return True
>> except Exception:
>> print("Well, that was a shame...")
>> return False
>>
>> then I am intending to guarantee that calling safe_fn() will never
>> raise an exception. [...]

Guido:
>The modification I proposed where we have both BaseExceptionGroup and
>ExceptionGroup will satisfy this need. It basically means that *unless* you
>are explicitly using an API that is documented to raise
>[Base]ExceptionGroup (such as a future variant of asyncio.gather()), you
>don't have to care about it. [...] that's why I am proposing to change the PEP 
>so that your 
>code will remain safe.

That would be welcome to me, too.

Cheers,
Cameron Simpson 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IKGK5YEPUPPWCAQ6NQEVGMKA63KPFUJQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Larry Hastings 


On 2/27/21 2:37 AM, Paul Moore wrote:

On Fri, 26 Feb 2021 at 23:36, Jim J. Jewett  wrote:

Whenever I've used except Exception or stronger, it was a sanitary barrier 
around code that might well do unpredictable or even stupid things.  Adding a 
new kind of exception that I hadn't predicted -- including ExceptionGroup -- 
would certainly fit this description, and I want my driver loop to do what I 
told it.  (Probably log an unexpected exception, and continue with the next 
record.  I honestly don't even want a page, let alone a crash, because data 
from outside that barrier ... is often bad, and almost never in ways the 
on-call person can safely fix.  And if they don't have time to find it in the 
logs, then it isn't a priority that week.)

This is my biggest concern. Disclaimer: I've yet to read the PEP,
because async makes my head hurt, but I am aware of some of the
background with Trio. Please take this as the perspective of someone
thinking "I don't use async/await in my code, can I assume this
doesn't affect me?"


I haven't read the PEP either.  But I assume it could (should?) affect 
anyone managing multiple simultaneous /things/ in Python:


 * async code, "fibers", "greenlets", Stackless "microthreads",
   "cooperative multitasking", or any other userspace mechanism where
   you manage multiple "threads" of execution with multiple stacks
 * code managing multiple OS-level threads
 * code managing multiple processes

It seems to me that any of those could raise multiple heterogeneous 
exceptions, and Python doesn't currently provide a mechanism to manage 
this situation.  My dim understanding is that ExceptionGroup proposes a 
mechanism to handle exactly this thing.



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2EDUBSWHB6QP5ITFSDV6OPHUAJ2NXMKC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Cameron Simpson 
On 26Feb2021 02:44, Irit Katriel  wrote:
>On Fri, Feb 26, 2021 at 2:00 AM Guido van Rossum  wrote:
>> OT: Is ExceptionGroup *really* immutable in the current 
>> implementation? As
>> long as the 'errors' field is a list, I think one could mutate the list
>> directly.
>
>It's not, but we were going to make it an immutable tuple.

Could you say why? Other than wanting to discourage mutation happy code 
getting out there?

The reason I ask is that the scenario which comes to my mind is 
something like:

except *OSError as e:

AIUI "e" is an ExceptionGroup containing only OSErrors. So, one common 
thing in my own code is this:

try:
do something with a file
except OSError as e:
if e.errno == ENOENT:
# file is missing, but that is ok
# because we treat it like an empty file
elif ... some other ok situation ...
else:
raise

My natural inclination with an ExceptionGroup would be to winnow the 
OSErrors I'm handed, and push the _unhandled_ errors back into the 
original ExceptionGroup. That way, after the various except* clauses, a 
nonempty ExceptionGroup would remain with the unhandled errors, and it 
might perhaps be reraised then.

Cheers,
Cameron Simpson 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FZI5JRQKTSMF7BYSIM2PJTO4E6IHSPDA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

2021-02-27 Thread Guido van Rossum 
We really don’t want users pushing non-exceptions into the list, nor do we
want e.g. KeyboardInterrupt to be added to a (non-Base-) ExceptionGroup.

There’s a pattern for what you propose using the split() method and a
lambda, or you can keep the exceptions in a list and re-wrap them at the
end.

On Sat, Feb 27, 2021 at 20:41 Cameron Simpson  wrote:

> On 26Feb2021 02:44, Irit Katriel  wrote:
> >On Fri, Feb 26, 2021 at 2:00 AM Guido van Rossum 
> wrote:
> >> OT: Is ExceptionGroup *really* immutable in the current
> >> implementation? As
> >> long as the 'errors' field is a list, I think one could mutate the list
> >> directly.
> >
> >It's not, but we were going to make it an immutable tuple.
>
> Could you say why? Other than wanting to discourage mutation happy code
> getting out there?
>
> The reason I ask is that the scenario which comes to my mind is
> something like:
>
> except *OSError as e:
>
> AIUI "e" is an ExceptionGroup containing only OSErrors. So, one common
> thing in my own code is this:
>
> try:
> do something with a file
> except OSError as e:
> if e.errno == ENOENT:
> # file is missing, but that is ok
> # because we treat it like an empty file
> elif ... some other ok situation ...
> else:
> raise
>
> My natural inclination with an ExceptionGroup would be to winnow the
> OSErrors I'm handed, and push the _unhandled_ errors back into the
> original ExceptionGroup. That way, after the various except* clauses, a
> nonempty ExceptionGroup would remain with the unhandled errors, and it
> might perhaps be reraised then.
>
> Cheers,
> Cameron Simpson 
>
-- 
--Guido (mobile)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KLQTLTUCUGQ4ZKQA5NFW4SR7MIDPUME4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

2021-02-27 Thread Stephen J. Turnbull 
Jim J. Jewett writes:

 > > which file am I actually running?
 > > which interpreter am I actually running?
 > > how do I tell the computer to use a different interpreter?
 > 
 > If you need to care about any of these, then the environment is
 > fighting you -- and the application probably stinks.  Programmers
 > have to deal with it because of bootstrapping, but there is no
 > reason that we should assume all python users need that flexibility
 > or want that responsibility.

Be careful!  I agree about need for flexibility and desire for
responsibility, but in fact these problems present themselves
frequently when helping people with their work.  "Which file am I
actually [edit]ing?" is one that comes up in just about every meeting:
documents are undated (or dated with the deadline date), people are
looking at different versions, and not infrequently the presenter
themself has a Documents/Downloads confusion and gets it wrong.  Then
"where is the one I want to be editing?" requires an oracle. :-)
"Which program am I running?" doesn't come up that often, but it does
come up when helping people with "doing something", especially trying
to find menu items across major versions.  Interestingly enough, "how
do I tell the computer to use a different interpreter?"  doesn't come
up for me outside of Python (and even there rarely) -- everybody who
consults me knows how to use "Open With".

So I agree with the basic principle that programs should Just Work by
default, but it's not always possible.  Eg, there's good reason why
Downloads and Documents both exist.  So it helps to have users know
something about implementations and sysadminning stuff, especially
when remote troubleshooting.

The question for Python is given the history and the plethora of ways
to invoke Python and find packages and programs, can we do better with
a minimum of backward incompatibility?  Humans are generally better at
learning this stuff than bags of bits on spinning platters are!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OAYWDLL26YTSIFTKGL6LWP7TU4PXUH6K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

2021-02-27 Thread Stephen J. Turnbull 
Paul Moore writes:

 > The one thing that *is* substantially worse for Python, is the
 > circumlocutions needed in the documentation to say how to run Python.
 > But that's 100% down to us not being willing to say "just type the
 > command python". And the reason for *that* is mostly historical,
 > related to the Python 3 transition

I don't think that's entirely true, and the roots are much earlier
than Python 3.  We had the same problem with Linux distributions
(especially Red Hat) and version 1 vs. version 2.  This was crucial
for those of us handling multibyte languages: getting Unicode Inside,
even in the non-str form of Python 2, really changed things.  But
Python 2.0 was a .0 release.  For a lot of developers dealing with
unibyte languages, 1.5.2 was the workhorse until about Python 2.2.

 > and what happened on Linux over the python/python2/python3
 > commands, and to a lesser extent the introduction of the launcher
 > on Windows

I don't know about Windows.  It might have been possible to arrange
that just "python" was the right way to do things, and have "-2" and
"-3" flags for specifying compatibility.  But it just wasn't possible
on *nix systems because /usr/bin/python was typically used for system
software.  They shouldn't have done that (they should have kept in it
in an OS-specific place and used full shebangs), but that's not how it
went 

I think those are important things to keep in mind when comparing with
a very young language like Julia (Julia didn't make *any* backward
compatibility promises until 1.0 in August 2018).  A lot of Python's
problems occur *because* of its multi-use potential and *because* of
its wide adoption in practice.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S7H6W75N62YGW5L2WYQH53BTFDJJWRZK/
Code of Conduct: http://python.org/psf/codeofconduct/