date:20190809

On Fri, Aug 09, 2019 at 01:16:16AM +0100, Rob Cliffe via Python-Dev wrote:

> >The name is perfectly self-descriptive: TargetScopeError means that
> >there's an error with the scope of the target. What else could it mean?

> That's easy for an experienced programmer to say.

Correct. Why should we be embarrassed to be experienced programmers?
For the record, I'm not a professional programmer, so my experience is 
probably a thousandth or less of that of some of the people here.

> IMHO it's about as helpful to a 
> newbie as saying GrobKunkError means that there's an error with the 
> kunk of the grob.

To a programmer who is not a native English speaker, "ValueError" and 
"SyntaxError" makes as much sense as PHP's PAAMAYIM NEKUDOTAYIM 
operator makes to non-Hebrew speakers. If they can cope with learning 
what "Value" means, I'm sure we English-speakers can learn what "Target" 
and "Scope" mean.

Developers are newbies for a tiny fraction of their programming life. 
Why should we ban technical terms because somebody in the first month of 
their programming life doesn't know the terms? They're going to learn 
them, and pretty quickly, as soon as they start talking to more 
experienced programmers, or reading answers on Stackoverflow, etc.

"Scope" and "target" are standard terms of art, like class, function, 
variable, etc. Saying that we shouldn't use them makes as little sense 
as saying that chefs shouldn't use the terms "sauté", "braise" or 
"blanche" because newbie cooks don't know what they are.

Of course they don't. Not knowing the things they need to know is the 
very definition of a newbie.

Here are some exceptions which "make as much sense to a newbie as..." 

- BrokenPipeError 
  (is my plumbing leaking? what do pipes have to do with programming?)

- ChildProcessError (Child? Process?)

- FutureWarning (something to do with Marty McFly or the Terminator?)

- the confusingly named UnicodeDecodeError and UnicodeEncodeError 
  which differ by two letters but mean completely different things, 
  the complete opposite of each other.

to mention only a few.

> I think AmbiguousScopeError is somewhat better than TargetScopeError,

I would be fine with AmbiguousScopeError too, except for the 
unfortunately double s. But now we're bike-shedding. Either name is much 
more informative than "syntax error".

[...]
> >One of the frustrations I have is that it's near to impossible to
> >programmatically distinguish wrong number of arguments from bad keyword
> >arguments from providing the wrong type of argument without relying on
> >the error message.
[...]

> I'm mildly curious as to why you couldn't rely on the error message, but 
> I'm sure you had a good reason.

Error messages are not part of the API and could change at any time, 
including in bug fix releases. Over the last few years, we've made a lot 
of progress in making error messages more informative:

python2.4 -c "len(None)"
TypeError: len() of unsized object

python2.7 -c "len(None)"
TypeError: object of type 'NoneType' has no len()

and they could easily continue to change in the future:

python4000 -c "len(None)"
TypeError: 'None' has no length

-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WJD7ZSYCPHHR3DTKERLOQVEM2KY2PTYR/

[Python-Dev] Re: PEP 572 TargetScopeError

On Thu, Aug 08, 2019 at 04:28:28PM -0700, Barry Warsaw wrote:
> On Aug 8, 2019, at 14:58, Steven D'Aprano  wrote:
> 
> > It's not a syntax error. There's nothing wrong with the syntax per-say:
> > we still have ``target := expression``. There's a problem with the
> > *semantics* not the syntax.
> 
> I’m not sure that distinction is meaningful though.  What you wrote is 
> disallowed, so you have to change your code (i.e. syntax) to avoid the 
> error.

I find it difficult to imagine a more meaningful distinction than that 
between syntax and semantics.

I can write something with invalid syntax that you can nevertheless 
predict the intended semantics:

ipmort sys

and write something with correct syntax that is almost impossible to 
determine the semantics without a huge amount of effort:

https://github.com/davekch/b/blob/master/b.py

(and they say you can't write one-liners in Python...)

Should x/0 be a SyntaxError? It too is "disallowed", and we have to 
change our code.

Should None[x] be a SyntaxError? It too is "disallowed", and we have to 
change our code.

I cannot think of a single programming error that doesn't require us to 
"change your code (i.e. syntax) to avoid the error".

> > The name is perfectly self-descriptive: TargetScopeError means that
> > there's an error with the scope of the target. What else could it mean?
> 
> What’s the scope?  What’s *a* scope (in this context)?  What’s the 
> target?  Nothing about the exception name tells you that it’s a 
> problem with the assignment expression you wrote.

It isn't a requirement for the exception *type* _alone_ to give a 
complete and detailed explanation of the error and all the background 
knowledge you need to make sense of it. No other exception lives up to 
that high standard.

Not even ValueError. (If you think "value" is easy enough, you ought to 
read some of the debates on comp.lang.python.)

What's "syntax" mean in the context of programming? (If you can assume 
the reader doesn't know what scope and target mean, I can assume the 
reader doesn't know what syntax is.) What's wrong with the syntax? 
(Nothing. The syntax is fine. Its the semantics we are prohibiting.)

[...]
> Once you learned what a key was — and you have to do that pretty early
> on in your Python experience — KeyError is perfectly descriptive.

You can cross out "key" and replace it with "(assignment) target" and 
"scope", and your comment still applies.

These are basic, simple terms in common use, like assignment, global and 
class. "Scope" is used in the tutorial's fourth chapter:

https://docs.python.org/3/tutorial/controlflow.html

"Key" in the sense of a dict not until the fifth:

https://docs.python.org/3/tutorial/datastructures.html

We're not talking about obscure terminology known only to a tiny handful 
of specialists here.

> So let’s imagine what the user experience would be like with a better 
> named exception to see whether a subclass is worth it.  For now let’s 
> call it AssignmentExpressionDisallowedError.  If you saw that it would 
> definitely be more descriptive.

It's certainly *longer*, but I'm not so sure it describes the problem 
better. The assignment expression is disallowed. That's implied by 
the fact that we have an exception in the first place. The important 
question is why it has been disallowed?

- Because it's Tuesday, and they are never allowed on Tuesdays?

- Because the assignment target is a read-only name binding?
  (Python doesn't have those, but a beginner might not know that.)

- Because access controls have prevented the assignment?

- Or because there's a problem with the scope of the assignment target?

> Is that valuable in the error output?

Its certainly more valuable than SyntaxError. I wouldn't choose 
"Assignment Disallowed" over "Target Scope", but let's go with it.

Which conveys more information about the error?

"I'm getting a syntax error."

"I'm getting an assignment disallowed error."

I'm a beginner, my google-fu is not strong, and I get this exception. If 
I google for SyntaxError without the error message, will I get good 
quality results?

How about if I google for AssignmentDisallowedError?

> > Please let's not use SyntaxError for something that's not an error
> > with the syntax.
> 
> PEP 572 does propose TargetScopeError as a subclass of SyntaxError, so 
> you’re getting a SyntaxError anyway.

I know, and I'm not super happy about that, but at least it's a 
subclass with a descriptive name.

-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WOBDBUPSHG3JL5R73SHYNY7VUPBZEQCD/

[Python-Dev] Re: What to do about invalid escape sequences

On Wed, Aug 07, 2019 at 07:47:45PM +1000, Chris Angelico wrote:
> On Wed, Aug 7, 2019 at 7:33 PM Steven D'Aprano  wrote:
> > What's the rush? Let's be objective here: what benefit are we going to
> > get from this change? Is there anyone hanging out desperately for "\d"
> > and "\-" to become SyntaxErrors, so they can... do what?
> 
> So that problems can start to be detected. Time and again, Python
> users on Windows get EXTREMELY confused by the way their code worked
> perfectly with one path, then bizarrely fails with another. That is a
> very real problem, and the problem is that it appeared to work when
> actually it was wrong.

And this change won't fix that, because *good* paths that currently work 
today will fail in the future, but *bad* paths that silently do the 
wrong thing will continue to silently do the wrong thing.

py> filename = "location\data"  # will work correctly
:1: SyntaxWarning: invalid escape sequence \d

py> filename = "location\temp"  # doesn't work as expected, but no error
py>

Effectively, we are hoping that Windows users will infer from the 
failure of "\d" (say) that they shouldn't use "\t" even though it 
doesn't raise. Perhaps some of them will, but I maintain we're talking 
about a small, incremental improvement, not something that will once and 
for all fix the problem.

I don't think this is a benefit for users of any operating system except 
Windows users. For Linux, Unix, Mac users, one could argue strongly that 
we're making the string escape experience a tiny bit *worse*, not 
better. Raymond's example of ASCII art for example.

I think the subset of users that this will help is quite small:

- users on Windows;

- who haven't read or paid attention to the innumerable recommendations
  on the web and the documentation that they always use forwards slashes
  in paths;

- who happen to use an escape like \d rather than \t;

- and will read and understand the eventual SyntaxWarning/Error;

- and infer from that error that they should change their path to use
  forward slashes instead of backslashes;

- and all this happens *before* they get bitten by the \t problem and
  they learn the hard way not to use backslashes in paths.

I'm not saying this isn't worth doing. I'm saying it's a small benefit 
that *right now* is a lot less than the cost to library authors and users.

> Python has a history of fixing these problems. It used to be that
> b"\x61\x62\x63\x64" was equal to u"abcd", but now Python sees these as
> fundamentally different.

Yes, and we fixed that over a 10+ year period involving no fewer than 
three full releases in the Python 2.x series and eight full releases in 
the Python 3.x series, and the transition period is not over yet since 
2.7 is not yet EOLed.

> Data-dependent bugs caused by a syntactic
> oddity are a language flaw that needs to be fixed.

There is always a tradeoff between the severity of the flaw and how much 
pain we are willing to accept to fix it. I think Raymond has made a good 
case that in this instance, the pain of fixing it *now* is greater than 
the benefit.

(I don't think he has made the case to reverse the depreciation 
altogether.)

If the benefit versus pain never moves into the black, then we should 
keep the status quo indefinitely, like any other language wart or 
misfeature we're stuck with due to backwards compatibility.

("Although never is often better than *right* now.")

But having said that, I'm confident that given an improved deprecation 
process that makes it easier for library authors to see the warning 
before end-users, we will be able to move forward in a release or two.

> > Because our processes don't work the way we assumed, it turns out that
> > in practice we haven't given developers the deprecation period we
> > thought we had. Read Nathaniel's post, if you haven't already done so:
> >
> > https://mail.python.org/archives/list/python-dev@python.org/message/E7QCC74OBYEY3PVLNQG2ZAVRO653LD5K/
> >
> > He makes a compelling case that while we might have had the promised
> > deprecation period by the letter of the law, in practice most developers
> > will have never seen it, and we will be breaking the spirit of the
> > promise if we continue with the unmodified plan.
> 
> Yes, that's a fair complaint. But merely pushing the deprecation back
> by a version is not solving it. There has to be SOMETHING done
> differently.

"We must do SOMETHING!!! This is something, therefore we must do it!!!"

I agree that we ought to fix the problem with the deprecation warnings.

What I don't agree with is the demand that unless I can give a fix for 
the deprecation warning issue *right now* we must stay the course no 
matter how annoying and painful it is for users and library authors.

> > And yet here we are rushing through a breaking change in an accelerated
> > manner, for a change of marginal benefit.
> 
> It's not a marginal benefit. For people who try to teach Python on
> multiple operating systems, this is

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Chris Angelico

On Fri, Aug 9, 2019 at 11:22 PM Steven D'Aprano  wrote:
>
> And this change won't fix that, because *good* paths that currently work
> today will fail in the future, but *bad* paths that silently do the
> wrong thing will continue to silently do the wrong thing.

Except that many paths can be both "good" and "bad", because paths
have multiple components. So the warning has a VERY high probability
of happening.

But I've given up on this debate. No more posts from me. Some things
aren't worth fighting for. With the number of words posted in this
thread saying "we need convenience, not correctness", I'm done
arguing.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/K4T7V5Z5GSGGK7HO73ZMFGTTAGMKHDE3/

[Python-Dev] Re: PEP 572 TargetScopeError

Short version of my more detailed answer below:

While my rationale is different from Barry's, I've convinced myself
that the right thing to do for Python 3.8 is to remove the new
TargetScopeError and have these cases all just raise SyntaxError.

I do still see potential value in a new "AssignmentScopeError"
subclass, but it would apply to more than just these new exceptions,
and can be considered on a more leisurely basis for Python 3.9.

On Fri, 9 Aug 2019 at 03:07, Barry Warsaw  wrote:
>
> bpo-37757: https://bugs.python.org/issue37757
>
> This issue describes some corner cases in PEP 572 (assignment expressions) 
> syntax that haven’t been implemented yet.  The PEP currently says:
>
> "The two invalid cases listed above raise TargetScopeError, a new subclass of 
> SyntaxError (with the same signature).”
>
> The PEP doesn’t really go into the rationale for why a new exception is being 
> defined, and in the issue I’ve argued that we should just raise SyntaxError 
> in those cases.  To me, “TargetScopeError” is pretty obscure and doesn’t give 
> users an obvious clue as to what’s going wrong, without searching the 
> interwebs.
>
> Nick argues (apologies if I’m misrepresenting you!):
>
> "I believe our main motivation for separating it out was the fact that even 
> though TargetScopeError is a compile-time error, the affected code is 
> syntactically fine - there are just issues with unambiguously inferring the 
> intended read/write location for the affected target names. (Subclassing 
> SyntaxError is then a pragmatic concession to the fact that "SyntaxError" 
> also de facto means "CompilationError”)”

This accurately conveys what I wrote on the PR and issue, but what I
wrote there didn't capture the full background for when
TargetScopeError gets raised.

The main motivating case that led to the new exception being defined
is the one that was already implemented as part of the initial PR
adding assignment expressions: the fact we can't readily make PEP
572's parent local scoping for comprehensions work when that parent
scope is a class scope. Hence, the TargetScopeError raised in this
case:

>>> class C:
... [x := 10 for i in range(5)]
...
 File "", line 2
TargetScopeError: named expression within a comprehension cannot
be used in a class body

Syntactically, the individual pieces of this code are perfectly fine -
the problem is that the calculated target for "x" would be in the
class body, and there isn't currently a way for the compiler to make
that name accessible both in the class scope and in the comprehension
body. (We're only able to make zero-arg super work because we don't
need to make `__class__` accessible while the class is executing).

The new PR then adds the additional checks to disallow the cases where
we don't want to enshrine "What CPython's current implementation
handles this code" as "The way this code must behave in all Python
implementations":

>>> [i := 10 for i in range(5)]
  File "", line 1
TargetScopeError: named expression cannot rebind comprehension
iteration variable

>>> [True for i in range(5) if (j := True) for j in range(5)]
  File "", line 1
TargetScopeError: comprehension inner loop cannot rebind named
expression target

>>> [True for i in (iterable := range(5))]
  File "", line 1
TargetScopeError: named expression cannot be used in comprehension
iterable expression

I agree that it's a problem that "Assignment" doesn't appear anywhere
in the exception name - if we're going to have a specific name for
this, then something like "AssignmentScopeError" would be better.

The main counterargument to this approach would be that other similar
scoping issues currently just raise SyntaxError:

>>> def f(x): global x
...
 File "", line 1
SyntaxError: name 'x' is parameter and global
>>> def f(x): nonlocal x
...
 File "", line 1
SyntaxError: name 'x' is parameter and nonlocal
>>> def f(): global x; nonlocal x
...
 File "", line 1
SyntaxError: name 'x' is nonlocal and global
>>> def f(): nonlocal x
   ...
 File "", line 1
SyntaxError: no binding for nonlocal 'x' found

(Those also highlight that I should tweak the new name specific
binding conflict errors to mention the variable name)

With a revised exception name though, I think even those existing
errors would be improved by switching them over to the subclass:

>>> def f(x): global x
...
 File "", line 1
AssignmentScopeError: name 'x' is parameter and global
>>> def f(x): nonlocal x
...
 File "", line 1
AssignmentScopeError: name 'x' is parameter and nonlocal
>>> def f(): global x; nonlocal x
...
 File "", line 1
AssignmentScopeError: name 'x' is nonlocal and global
>>> def f(): nonlocal x
   ...
 File "", line 1
AssignmentScopeError: no binding for nonlocal 'x' found

However, a change like that would make the most sense when considered
ind

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Rob Cliffe via Python-Dev

On 09/08/2019 12:50:14, Steven D'Aprano wrote:

On Thu, Aug 08, 2019 at 04:28:28PM -0700, Barry Warsaw wrote:
I find it difficult to imagine a more meaningful distinction than that
between syntax and semantics.

The distinction between syntax and semantics can sometimes be blurred.
Certainly in spoken languages. What about program code?

Should x/0 be a SyntaxError? It too is "disallowed", and we have to
change our code.
No, because x might be an instance of a class (extended numbers?) that
allows division by zero.

Should None[x] be a SyntaxError? It too is "disallowed", and we have to
change our code.

It's allowable syntax with defined runtime semantics: raise a TypeError
if x exists,
and a NameError if it isn't. Conceivably some programmer might find a
use for it.
But since it always raises an exception, it would probably be more
helpful in the long
run if it *were* a SyntaxError. It *isn't*, because (I assume)
detecting this and similar

idioms, e.g. {}[0] , would complicate the parser/compiler too much.

Your examples show, I claim, that even in program code the distinction
between

syntax and semantics is not always obvious.

I suggest the only watertight definition of a syntax error is something
which raises
an exception at compile time. Obviously this can depend not only on the
code,

but also on the compiler.

Rob Cliffe
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/FKZSJS2JZ3DXENYOI3AFPGDH3TOH5C5U/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread brian . skinn

> This whole thread would be an excellent justification for following 3.9
> with 4.0. It's as near as we ever want to get to a breaking change, and a
> major version number would indicate the need to review. If increasing
> strictness of escape code interpretation in string literals is the only
> incompatibility there would surely be general delight.
> 
> Kind regards,
> Steve Holden

I rather doubt that allowing breaking changes into a Python 4.0 would end up 
with this as the only proposed incompatibility. Once word got out, a flood of 
incompat requests would probably get raised. I personally have a change I'd 
like made to doctest (https://bugs.python.org/issue36714), and I know of 
another in argparse (https://bugs.python.org/issue33109) that I'm personally 
neutral on but that others have stronger feelings about.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SEBJF7C7RRG3Q3MFD5D6CTOFZUX7DNSE/

[Python-Dev] Core review is needed by bpo-34788

2019-08-09 Thread Sasha Pavlyuk

Hello

2019-06-03 I have created PR https://github.com/python/cpython/pull/13772 ,
which adds IPv6 scoped addresses support to ipaddress module.
It is very critical to everyone, who is dealing with IPv6  networking. For
example, in salt project they use patched module.
So, it would be very nice to have my changes merged and avoid such a
workarounds.

Pull request passed review, but stopped at core review stage.
I wrote reminder comment at bugtracker without any result, also I can not
reach pmoody (a maintainer of ipaddress module).
Here is link to the bug - https://bugs.python.org/issue34788.

Please, pay attention to this issue.

KR
Oleksandr Pavliuk
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZNURFRSF5OP5IYYKOGBG3JOGX66FEA6J/

[Python-Dev] Re: PEP 572 TargetScopeError

I don't see how this debate can avoid a vote in the Steering Council.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PLOQMIQYWOJCDXD6LQ5IZT2FV6Q2PXWY/

[Python-Dev] Re: What is a public API?

On Wed, 24 Jul 2019 at 06:32, Kyle Stanley  wrote:
>
> Steve Dower wrote:
> > So I apologise for mentioning that people care about import performance.
> > Let's ignore them/that issue for now and worry instead about making sure
> > people (including us!) know what the canonical reference for
> > public/internal is.
>
> Good point, the discussion about __all__, adding the @public decorator, and 
> worrying about performance impacts might be jumping too far ahead.
>
> For now, if most of the core devs are in agreement with the current unwritten 
> rule of  "unless explicitly documented public, all imports are private even 
> if not prefixed with an underscore", I think the first priority should be to 
> document it officially somewhere. That way, other developers and any 
> potential confused users can be referred to it.

It's not an unwritten rule, as it already has its own subsection in
PEP 8: https://www.python.org/dev/peps/pep-0008/#public-and-internal-interfaces

The main question in this thread is what to do about standard library
modules that were written before those documented guidelines were put
in place, and hence have multiple internal APIs that lack the leading
underscore (and I don't think that's a question with a generic
answer).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6TNG7J4QEOBCVLGSZ6VBOKKQQBHKPZPF/

[Python-Dev] Re: What to do about invalid escape sequences

This discussion looks like there's no end in sight. Maybe the Steering
Council should take a vote?

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5BYIHT2BV7TPDHP6F5W44K4JKN5PHQQ3/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Tim Peters

[Guido]
> I don't see how this debate can avoid a vote in the Steering Council.

FWIW, I found Nick's last post wholly persuasive:  back off to
SyntaxError for now, and think about adding a more specific exception
later for _all_ cases (not just walrus) in which a scope conflict
isn't allowed (e.g., his example of declaring a function's formal
argument `global` in the function).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7ILVNCFSYMJ4HP5INVA34N4DVEKSEYAU/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Eric V. Smith


On 8/9/2019 11:41 AM, Tim Peters wrote:

[Guido]

I don't see how this debate can avoid a vote in the Steering Council.


FWIW, I found Nick's last post wholly persuasive:  back off to
SyntaxError for now, and think about adding a more specific exception
later for _all_ cases (not just walrus) in which a scope conflict
isn't allowed (e.g., his example of declaring a function's formal
argument `global` in the function).


Also FWIW: I agree with Nick, except I don't see the point of having an 
exception that likely no one is ever going to catch. What's the scenario 
under which you'd want to catch a ScopeConflictException? If it only 
exists to give more info to a human, just improve the message that goes 
with SyntaxError in that case.


But to answer Guido's question: I think this is a fine issue for the SC 
to vote on.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2TDXXUCOEAZYFPCVGOVUWZSCQ24DNGMZ/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Christian Tismer

On 09.08.19 15:44, Nick Coghlan wrote:
...

> However, a change like that would make the most sense when considered
> independently of the specific case of assignment expressions -
> instead, we would want to ask "What kinds of exceptions does the
> symbol table analysis pass raise?", and then consider whether it might
> make sense to define subclasses of syntax error for them.
> 
> Given the naming issue, and the fact that a potential new SyntaxError
> subclass would be relevant for more than just the new PEP 572
> exceptions, I think the right near term approach is to go with the
> generic SyntaxError as Barry suggests. I'll update my PRs accordingly.


Totally agree. It is fine to have SyntaxError now and go for
one or more new subclasses for a whole bunch of errors at
a later time, fixing more things in a more consistent way.

-- 
Christian Tismer :^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B2YAMST5CQWPIHPM5DS56TV4CYZDTZ73/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Serhiy Storchaka


09.08.19 18:30, Guido van Rossum пише:
This discussion looks like there's no end in sight. Maybe the Steering 
Council should take a vote?


Possible options:

1. SyntaxWarning in 3.8+ (the current status).
2. DeprecationWarning in 3.8, SyntaxWarning in 3.9+ (revert changes in 
3.8 only).

3. DeprecationWarning in 3.8 and 3.9 (revert changes in master and 3.8).
4. No warnings at all.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TRZHNEITUZTDEEHSFWV5SUEXNRHTU3KQ/

[Python-Dev] Re: What to do about invalid escape sequences

On Sat, 10 Aug 2019 at 01:44, Guido van Rossum  wrote:
>
> This discussion looks like there's no end in sight. Maybe the Steering 
> Council should take a vote?

I find the "Our deprecation warnings were even less visible than
normal" argument for extending the deprecation period compelling.

I also think the UX of the warning itself could be reviewed to provide
a more explicit nudge towards using raw strings when folks want to
allow arbitrary embedded backslashes. Consider:

SyntaxWarning: invalid escape sequence \,

vs something like:

SyntaxWarning: invalid escape sequence \, (Note: adding the raw
string literal prefix, r, will accept all non-trailing backslashes)

After all, the habit we're trying to encourage is "If I want to
include backslashes without escaping them all, I should use a raw
string", not "I should memorize the entire set of valid escape
sequences" or even "I should always escape backslashes".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N7Q4R3GX5RBF3FPGWMMKWYB4LOI7GVOC/

[Python-Dev] Re: PEP 572 TargetScopeError

On Sat, 10 Aug 2019 at 02:05, Eric V. Smith  wrote:
>
> On 8/9/2019 11:41 AM, Tim Peters wrote:
> > [Guido]
> >> I don't see how this debate can avoid a vote in the Steering Council.
> >
> > FWIW, I found Nick's last post wholly persuasive:  back off to
> > SyntaxError for now, and think about adding a more specific exception
> > later for _all_ cases (not just walrus) in which a scope conflict
> > isn't allowed (e.g., his example of declaring a function's formal
> > argument `global` in the function).
>
> Also FWIW: I agree with Nick, except I don't see the point of having an
> exception that likely no one is ever going to catch. What's the scenario
> under which you'd want to catch a ScopeConflictException? If it only
> exists to give more info to a human, just improve the message that goes
> with SyntaxError in that case.

It's mainly the idea of being able to attach the variable name to the
exception as structured data, which an IDE could then use to highlight
the offending name without needing to parse the human-readable
exception string. Actually making the change would require IDE authors
agreeing that it  would be helpful to them, though, and they may not
need it if they already have their own symbol analysers running.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RGZVXAUIJVOHMDYDU2IXUSSHXAOOLNNW/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Steve Dower


On 09Aug2019 0905, Serhiy Storchaka wrote:

09.08.19 18:30, Guido van Rossum пише:
This discussion looks like there's no end in sight. Maybe the Steering 
Council should take a vote?


Possible options:

1. SyntaxWarning in 3.8+ (the current status).
2. DeprecationWarning in 3.8, SyntaxWarning in 3.9+ (revert changes in 
3.8 only).

3. DeprecationWarning in 3.8 and 3.9 (revert changes in master and 3.8).
4. No warnings at all.


I also posted another possible option that helps solve the real problem 
faced by users, and not just the "we want to have a warning" problem 
that is purely ours.



* change the SyntaxWarning into a default-silenced one that fires every time a 
.pyc is loaded (this is the hard part, but it's doable)
* change pathlib.PureWindowsPath, os.fsencode and os.fsdecode to explicitly 
warn when the path contains control characters
* change the PyErr_SetExcFromWindowsErrWithFilenameObjects function to append (or chain) an extra message when either of the filenames contains control characters (or change OSError to do it, or the default sys.excepthook) 


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GZJPZ55OR2CIERO5Q4ETPZPAQZSFAEDD/

[Python-Dev] Re: What is a public API?

2019-08-09 Thread Christian Tismer

On 16.07.19 00:32, Barry Warsaw wrote:
> On Jul 13, 2019, at 19:09, Raymond Hettinger  
> wrote:
>>
>> In some modules, we've been careful to use both __all__ and to use an 
>> underscore prefix to indicate private variables and helper functions 
>> (collections and random for example).  IMO, when a module has shown that 
>> care, future maintainers should stick with that practice.
>>
>> The calendar module is an example of where that care was taken for many 
>> years and then a recent patch went against that practice.  This came to my 
>> attention when an end-user questioned which functions were for internal use 
>> only and posted their question on Twitter.  On the tracker, I then made a 
>> simple request to restore the module's convention but you seem steadfastly 
>> resistant to the suggestion.
>>
>> When we do have evidence of user confusion (as in the case with the calendar 
>> module), we should just fix it.  IMO, it would be an undue burden on the 
>> user to have to check every method in dir() against the contents of __all__ 
>> to determine what is public (see below).  Also, as a maintainer of the 
>> module, I would not have found it obvious whether the functions were public 
>> or not.  The non-public functions look just like the public ones.
>>
>> It's true that the practices across the standard library have historically 
>> been loose and varied (__all__ wasn't always used and wasn't always kept 
>> up-to-date, some modules took care with private underscore names and some 
>> didn't).  To me this has mostly worked out fine and didn't require a strict 
>> rule for all modules everywhere.  IMO, there is no need to sweep through the 
>> library and change long-standing policies on existing modules.
> 
> EIBTI 
> 
> Shameless plug: https://public.readthedocs.io/en/latest/


Hey, what a fantastic little module!
I'll hurry and use that a lot! Especially the builtins idea
is really great :-P

Cheers - Chris


p.s.: How about adding @private as well?
There are cases where I would like to do the opposite:

__all__ = dir()

@private
_some_private_func_1(...): ...
...
@private
_some_private_func_n(...): ...

not-too-seriously yours - Chris
-- 
Christian Tismer :^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HUSOXEYFEBJRMQPC5CSFRB2DDMAARBFV/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Paul Moore

On Fri, 9 Aug 2019 at 17:55, Steve Dower  wrote:
> > * change the SyntaxWarning into a default-silenced one that fires every 
> > time a .pyc is loaded (this is the hard part, but it's doable)
> > * change pathlib.PureWindowsPath, os.fsencode and os.fsdecode to explicitly 
> > warn when the path contains control characters
> > * change the PyErr_SetExcFromWindowsErrWithFilenameObjects function to 
> > append (or chain) an extra message when either of the filenames contains 
> > control characters (or change OSError to do it, or the default 
> > sys.excepthook)

The second and third art of this seem like they are both independent
of the first, and useful improvements in their own right.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6L65KDXMRTTLHX7HWAU4WLRMHEH7GXFA/

[Python-Dev] Re: What is a public API?

2019-08-09 Thread Barry Warsaw

Interesting idea!

https://gitlab.com/warsaw/public/issues/3

-Barry

> On Aug 9, 2019, at 09:55, Christian Tismer  wrote:
> 
> Signed PGP part
> On 16.07.19 00:32, Barry Warsaw wrote:
>> On Jul 13, 2019, at 19:09, Raymond Hettinger  
>> wrote:
>>> 
>>> In some modules, we've been careful to use both __all__ and to use an 
>>> underscore prefix to indicate private variables and helper functions 
>>> (collections and random for example).  IMO, when a module has shown that 
>>> care, future maintainers should stick with that practice.
>>> 
>>> The calendar module is an example of where that care was taken for many 
>>> years and then a recent patch went against that practice.  This came to my 
>>> attention when an end-user questioned which functions were for internal use 
>>> only and posted their question on Twitter.  On the tracker, I then made a 
>>> simple request to restore the module's convention but you seem steadfastly 
>>> resistant to the suggestion.
>>> 
>>> When we do have evidence of user confusion (as in the case with the 
>>> calendar module), we should just fix it.  IMO, it would be an undue burden 
>>> on the user to have to check every method in dir() against the contents of 
>>> __all__ to determine what is public (see below).  Also, as a maintainer of 
>>> the module, I would not have found it obvious whether the functions were 
>>> public or not.  The non-public functions look just like the public ones.
>>> 
>>> It's true that the practices across the standard library have historically 
>>> been loose and varied (__all__ wasn't always used and wasn't always kept 
>>> up-to-date, some modules took care with private underscore names and some 
>>> didn't).  To me this has mostly worked out fine and didn't require a strict 
>>> rule for all modules everywhere.  IMO, there is no need to sweep through 
>>> the library and change long-standing policies on existing modules.
>> 
>> EIBTI 
>> 
>> Shameless plug: https://public.readthedocs.io/en/latest/
> 
> 
> Hey, what a fantastic little module!
> I'll hurry and use that a lot! Especially the builtins idea
> is really great :-P
> 
> Cheers - Chris
> 
> 
> p.s.: How about adding @private as well?
> There are cases where I would like to do the opposite:
> 
> __all__ = dir()
> 
> @private
> _some_private_func_1(...): ...
> ...
> @private
> _some_private_func_n(...): ...
> 
> not-too-seriously yours - Chris
> --
> Christian Tismer :^)   tis...@stackless.com
> Software Consulting  : http://www.stackless.com/
> Karl-Liebknecht-Str. 121 : https://github.com/PySide
> 14482 Potsdam: GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
> 
> 
> 



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T4UVFGL5K4GI2VFD4Z3GDTI6CIKVTMUO/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Barry Warsaw

Thanks Nick.  I find yours (and Eric’s) argument compelling.  I’m totally 
aligned with using SyntaxError in 3.8 and thinking through the more general 
problem for 3.9.

Cheers,
-Barry

> On Aug 9, 2019, at 06:44, Nick Coghlan  wrote:
> 
> Short version of my more detailed answer below:
> 
> While my rationale is different from Barry's, I've convinced myself
> that the right thing to do for Python 3.8 is to remove the new
> TargetScopeError and have these cases all just raise SyntaxError.
> 
> I do still see potential value in a new "AssignmentScopeError"
> subclass, but it would apply to more than just these new exceptions,
> and can be considered on a more leisurely basis for Python 3.9.



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CI4U5VOEKZFJ26IETN3FYRZOZRDWDCP3/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Barry Warsaw

Nick and I are now on the same page, so I don’t think we *have* to have a 
formal SC vote.

Cheers,
-Barry

> On Aug 9, 2019, at 08:28, Guido van Rossum  wrote:
> 
> I don't see how this debate can avoid a vote in the Steering Council.
> 
> --
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him/his (why is my pronoun here?)
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/PLOQMIQYWOJCDXD6LQ5IZT2FV6Q2PXWY/



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DDRM3KM5Z5WANWQC2ZIDNJ7WJUXLZ6B5/

[Python-Dev] Summary of Python tracker Issues

2019-08-09 Thread Python tracker



ACTIVITY SUMMARY (2019-08-02 - 2019-08-09)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7136 ( -3)
  closed 42419 (+61)
  total  49555 (+58)

Open issues with patches: 2897 


Issues opened (43)
==

#37747: _markupbase.py fails with TypeError on invalid keyword in mark
https://bugs.python.org/issue37747  opened by bp256r1

#37749: ipaddress - is_global method all multicast addresses and netwo
https://bugs.python.org/issue37749  opened by bjames

#37750: PyBuffer_FromContiguous not documented
https://bugs.python.org/issue37750  opened by shihai1991

#37751: In codecs,  function 'normalizestring' should convert both spa
https://bugs.python.org/issue37751  opened by qigangxu

#37752: Redundant Py_CHARMASK called in some files
https://bugs.python.org/issue37752  opened by qigangxu

#37754: alter size of segment using multiprocessing.shared_memory
https://bugs.python.org/issue37754  opened by vinay0410

#37755: pydoc topics, keywords and symbols always use pager instead of
https://bugs.python.org/issue37755  opened by eht16

#37756: Error 0x80070643 when installing
https://bugs.python.org/issue37756  opened by bfbfbfb bfbfbf

#37757: TargetScopeError not raised for comprehension scope conflict
https://bugs.python.org/issue37757  opened by ncoghlan

#37758: unicodedata checksum-tests only test 1/17th of Unicode's codep
https://bugs.python.org/issue37758  opened by Greg Price

#37759: Polish whatsnew for 3.8
https://bugs.python.org/issue37759  opened by rhettinger

#37760: Refactor makeunicodedata.py: dedupe parsing, use dataclass
https://bugs.python.org/issue37760  opened by Greg Price

#37762: IDLE very slow due a super long line output in chunks
https://bugs.python.org/issue37762  opened by Bernie

#37763: Need setup.py to pick up -isystem flags from CPPFLAGS
https://bugs.python.org/issue37763  opened by jherland

#37764: email.Message.as_string infinite loop
https://bugs.python.org/issue37764  opened by mytran

#37765: IDLE: Include keywords in __main__ autocomplete list
https://bugs.python.org/issue37765  opened by xtreak

#37766: IDLE autocomplete: revise fetch_completions, add htest
https://bugs.python.org/issue37766  opened by terry.reedy

#37768: IDLE: Show help(object) output in a text viewer
https://bugs.python.org/issue37768  opened by taleinat

#37769: Windows Store installer should warn user about MAX_PATH
https://bugs.python.org/issue37769  opened by Jonas Binding

#37771: No equivalent of `inspect.getcoroutinestate` for PyAsyncGenASe
https://bugs.python.org/issue37771  opened by GeeVye

#37772: zipfile.Path.iterdir() outputs sub directories many times or n
https://bugs.python.org/issue37772  opened by joernheissler

#37773: ValueError: I/O operation on closed file. in ZipFile destructo
https://bugs.python.org/issue37773  opened by joernheissler

#37774: Micro-optimize vectorcall using PY_LIKELY
https://bugs.python.org/issue37774  opened by jdemeyer

#37775: update doc of compileall
https://bugs.python.org/issue37775  opened by shihai1991

#37776: Test Py_Finalize() from a subinterpreter
https://bugs.python.org/issue37776  opened by nanjekyejoannah

#37779: configparser: add documentation about several read() behaviour
https://bugs.python.org/issue37779  opened by sblondon

#37781: Use "z" for PY_FORMAT_SIZE_T
https://bugs.python.org/issue37781  opened by inada.naoki

#37784: Compiling Python 3 with sqlite impossible when sqlite installa
https://bugs.python.org/issue37784  opened by manuco

#37785: argparse uses %s in gettext calls causing xgettext warnings
https://bugs.python.org/issue37785  opened by kulikjak

#37788: fix for bpo-36402 (threading._shutdown() race condition) cause
https://bugs.python.org/issue37788  opened by anselm.kruis

#37789: Update doc strings for test.bytecode_helper
https://bugs.python.org/issue37789  opened by nanjekyejoannah

#37790: subprocess.Popen() is extremely slow
https://bugs.python.org/issue37790  opened by Alexander.Pyhalov

#37792: xml.etree.ElementTree.Element.__eq__ does compare only objects
https://bugs.python.org/issue37792  opened by Marco Sulla

#37794: Replace /Ox with /O2
https://bugs.python.org/issue37794  opened by Kojoley

#37796: ModuleFinder does not resolve ".." correctly
https://bugs.python.org/issue37796  opened by mkleehammer

#37797: Add name attribute to NameError
https://bugs.python.org/issue37797  opened by skreft

#37798: Add C fastpath for statistics.NormalDist.inv_cdf()
https://bugs.python.org/issue37798  opened by rhettinger

#37799: Renaming Doc/reference/ to Doc/language/
https://bugs.python.org/issue37799  opened by maggyero

#37800: Clean up the documentation on module attributes
https://bugs.python.org/issue37800  opened by maggyero

#37801: Compilation on MINGW64 fails (CODESET,wcstok,...)
https://bugs.python.org/issue37801  opened by vengelson

#37802: micro-optimization o

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Jonathan Goble

On Fri, Aug 9, 2019 at 12:34 PM Nick Coghlan  wrote:
>
> I find the "Our deprecation warnings were even less visible than
> normal" argument for extending the deprecation period compelling.

Outsider's 2 cents from reading this discussion (with no personal
experience with this warning):

I am perplexed at the opinion, seemingly espoused by multiple people
in this thread, that because a major part of the problem is that the
warnings were not visible enough, somehow the proposed solution is
making them not visible enough again? It's too late, in my
understanding, in the 3.8 cycle to add a new feature like a change to
how these warnings are produced (it seems a significant change to the
.pyc structure is needed to emit them at runtime), so this supposed
"solution" is nothing but kicking the can down the road. When 3.9
rolls around, public exposure to the problem of invalid escape
sequences will still be approximately what it is now (because if
nobody saw the warnings in 3.7, they certainly won't see them in 3.8
with this "fix"), so you'll end up with the same complaints about
SyntaxWarning that started this discussion, end up back on
DeprecationWarning for 3.9 (hopefully with support for emitting them
at runtime instead of just compile-time), then have to wait until
3.10/4.0 for SyntaxWarning and eventually the next version to actually
make them errors.

It seems to me, in my humble but uneducated opinion, that if people
are not seeing the warnings, then continuing to give them warnings
they won't see isn't a solution to anything. Put the warning front and
center. The argument of third-party packages will always be an issue,
even if we wait ten years. So put these warnings front and center now
so package and code maintainers actually see it, and I'll bet the
problematic escape sequences get fixed rather quickly.

What am I missing here?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6ZBX2PULRGIRUBQ735ONGV2RZU2LP3WQ/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Eric V. Smith


On 8/9/2019 2:28 PM, Jonathan Goble wrote:

On Fri, Aug 9, 2019 at 12:34 PM Nick Coghlan  wrote:

I find the "Our deprecation warnings were even less visible than
normal" argument for extending the deprecation period compelling.

Outsider's 2 cents from reading this discussion (with no personal
experience with this warning):

I am perplexed at the opinion, seemingly espoused by multiple people
in this thread, that because a major part of the problem is that the
warnings were not visible enough, somehow the proposed solution is
making them not visible enough again? It's too late, in my
understanding, in the 3.8 cycle to add a new feature like a change to
how these warnings are produced (it seems a significant change to the
.pyc structure is needed to emit them at runtime), so this supposed
"solution" is nothing but kicking the can down the road. When 3.9
rolls around, public exposure to the problem of invalid escape
sequences will still be approximately what it is now (because if
nobody saw the warnings in 3.7, they certainly won't see them in 3.8
with this "fix"), so you'll end up with the same complaints about
SyntaxWarning that started this discussion, end up back on
DeprecationWarning for 3.9 (hopefully with support for emitting them
at runtime instead of just compile-time), then have to wait until
3.10/4.0 for SyntaxWarning and eventually the next version to actually
make them errors.


Yes, I think that's the idea: Deprecation warning in 3.9, but more 
visible that what 3.7 has. That is, not just at compile time but at run 
time. What's required to make that happen is an open question.



It seems to me, in my humble but uneducated opinion, that if people
are not seeing the warnings, then continuing to give them warnings
they won't see isn't a solution to anything. Put the warning front and
center. The argument of third-party packages will always be an issue,
even if we wait ten years. So put these warnings front and center now
so package and code maintainers actually see it, and I'll bet the
problematic escape sequences get fixed rather quickly.

What am I missing here?


Hopefully the warnings in 3.9 would be more visible that what we saw in 
3.7, so that library authors can take notice and do something about it 
before 3.10 rolls around.


Eric
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GGZY7B2WFHVXRQ7NVTHGC2F4L5RJIKDI/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Jonathan Goble

On Fri, Aug 9, 2019 at 2:36 PM Eric V. Smith  wrote:
>
> On 8/9/2019 2:28 PM, Jonathan Goble wrote:
> > On Fri, Aug 9, 2019 at 12:34 PM Nick Coghlan  wrote:
> >> I find the "Our deprecation warnings were even less visible than
> >> normal" argument for extending the deprecation period compelling.
> > Outsider's 2 cents from reading this discussion (with no personal
> > experience with this warning):
> >
> > I am perplexed at the opinion, seemingly espoused by multiple people
> > in this thread, that because a major part of the problem is that the
> > warnings were not visible enough, somehow the proposed solution is
> > making them not visible enough again? It's too late, in my
> > understanding, in the 3.8 cycle to add a new feature like a change to
> > how these warnings are produced (it seems a significant change to the
> > .pyc structure is needed to emit them at runtime), so this supposed
> > "solution" is nothing but kicking the can down the road. When 3.9
> > rolls around, public exposure to the problem of invalid escape
> > sequences will still be approximately what it is now (because if
> > nobody saw the warnings in 3.7, they certainly won't see them in 3.8
> > with this "fix"), so you'll end up with the same complaints about
> > SyntaxWarning that started this discussion, end up back on
> > DeprecationWarning for 3.9 (hopefully with support for emitting them
> > at runtime instead of just compile-time), then have to wait until
> > 3.10/4.0 for SyntaxWarning and eventually the next version to actually
> > make them errors.
>
> Yes, I think that's the idea: Deprecation warning in 3.9, but more
> visible that what 3.7 has. That is, not just at compile time but at run
> time. What's required to make that happen is an open question.
>
> > It seems to me, in my humble but uneducated opinion, that if people
> > are not seeing the warnings, then continuing to give them warnings
> > they won't see isn't a solution to anything. Put the warning front and
> > center. The argument of third-party packages will always be an issue,
> > even if we wait ten years. So put these warnings front and center now
> > so package and code maintainers actually see it, and I'll bet the
> > problematic escape sequences get fixed rather quickly.
> >
> > What am I missing here?
>
> Hopefully the warnings in 3.9 would be more visible that what we saw in
> 3.7, so that library authors can take notice and do something about it
> before 3.10 rolls around.

OK, so I'm at least understanding the plan correctly. I just don't get
the idea of kicking the can down the road on the hope that in 3.9
people will see the warning (knowing that you are still using a
warning that is disabled by default and thus has a high chance of not
being seen until 3.10), when we already have the ability to push out a
visible-by-default warning now in 3.8 and get people to take notice
two whole feature releases (= about 3 years) earlier.

The SyntaxWarning disruption (or SyntaxError disruption) has to happen
eventually, and while I support the idea of making compile-time
DeprecationWarnings be emitted at runtime, I really don't think that a
disabled-by-default warning is going to change a whole lot. Sure, the
major packages will likely see it and update their code, but lots of
smaller specialty packages and independent developers won't see it in
3.9. The bulk of the change isn't going to happen until we go to
SyntaxWarning, so why not just get it over with instead of dragging it
out for three years?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CWM2KO5IA24UCBSAYJP735EYKXIRRQRG/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread brian . skinn

Eric V. Smith wrote:
>  Hopefully the warnings in 3.9 would be more visible that what we saw in 
> 3.7, so that library authors can take notice and do something about it 
> before 3.10 rolls around.
> Eric

Apologies for the ~double-post on the thread, but: the SymPy team has figured 
out the right pytest incantation to expose these warnings. Given the extensive 
adoption of pytest, perhaps it would be good to combine (1) a FR on pytest to 
add a convenience flag enabling this mix of options with (2) an aggressive 
"marketing push", encouraging library maintainers to add it to their testing/CI.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S2464WJ3QCDE4CBM6AWITHMFCISA6O75/

[Python-Dev] Re: What to do about invalid escape sequences


On 8/9/2019 9:08 AM, Nick Coghlan wrote:

On Sat, 10 Aug 2019 at 01:44, Guido van Rossum  wrote:

This discussion looks like there's no end in sight. Maybe the Steering Council 
should take a vote?

I find the "Our deprecation warnings were even less visible than
normal" argument for extending the deprecation period compelling.

I also think the UX of the warning itself could be reviewed to provide
a more explicit nudge towards using raw strings when folks want to
allow arbitrary embedded backslashes. Consider:

 SyntaxWarning: invalid escape sequence \,

vs something like:

 SyntaxWarning: invalid escape sequence \, (Note: adding the raw
string literal prefix, r, will accept all non-trailing backslashes)

After all, the habit we're trying to encourage is "If I want to
include backslashes without escaping them all, I should use a raw
string", not "I should memorize the entire set of valid escape
sequences" or even "I should always escape backslashes".

Cheers,
Nick.

The reason I never use raw strings is in the documentation, it is 
because \ still has a special meaning, and the first several times I 
felt the need for raw strings, it was for directory names that wanted to 
end with \ and couldn't. Quoted below. Also relevant to the discussion 
is the "benefit" of leaving the backslash in the result of an illegal 
escape, which no one has mentioned in this huge thread.


Unlike Standard C, all unrecognized escape sequences are left in the 
string unchanged, i.e., /the backslash is left in the result/. (This 
behavior is useful when debugging: if an escape sequence is mistyped, 
the resulting output is more easily recognized as broken.) It is also 
important to note that the escape sequences only recognized in string 
literals fall into the category of unrecognized escapes for bytes 
literals.


Changed in version 3.6: Unrecognized escape sequences produce a
DeprecationWarning. In some future version of Python they will be
a SyntaxError.

Even in a raw literal, quotes can be escaped with a backslash, but the 
backslash remains in the result; for example, |r"\""| is a valid 
string literal consisting of two characters: a backslash and a double 
quote; |r"\"| is not a valid string literal (even a raw string cannot 
end in an odd number of backslashes). Specifically, /a raw literal 
cannot end in a single backslash/ (since the backslash would escape 
the following quote character). Note also that a single backslash 
followed by a newline is interpreted as those two characters as part 
of the literal, /not/ as a line continuation.




___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HKLW5VBZK46TOP6WURFH767YCHRFOYNN/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Nathaniel Smith

On Fri, Aug 9, 2019 at 12:07 PM  wrote:
>
> Eric V. Smith wrote:
> >  Hopefully the warnings in 3.9 would be more visible that what we saw in
> > 3.7, so that library authors can take notice and do something about it
> > before 3.10 rolls around.
> > Eric
>
> Apologies for the ~double-post on the thread, but: the SymPy team has figured 
> out the right pytest incantation to expose these warnings. Given the 
> extensive adoption of pytest, perhaps it would be good to combine (1) a FR on 
> pytest to add a convenience flag enabling this mix of options with (2) an 
> aggressive "marketing push", encouraging library maintainers to add it to 
> their testing/CI.

Unfortunately, their solution isn't a pytest incantation, it's a
separate 'compileall' invocation they run on their source tree. I'm
not sure how you'd convert this into a pytest feature, because I don't
think pytest always know which parts of your code are your code versus
which parts are supporting libraries.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H36DMKUODHOQOYIIZCKW6LYKSGJLXTT4/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread brian . skinn

Nathaniel Smith wrote:
> Unfortunately, their solution isn't a pytest incantation, it's a
> separate 'compileall' invocation they run on their source tree. I'm
> not sure how you'd convert this into a pytest feature, because I don't
> think pytest always know which parts of your code are your code versus
> which parts are supporting libraries.
> -n

Ahh, did not appreciate this. :-( Nevermind, then!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VFFV7MJUZKMSD6FS3OONSEN5XLBOLT5R/

[Python-Dev] Re: What to do about invalid escape sequences

On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:

> The reason I never use raw strings is in the documentation, it is 
> because \ still has a special meaning, and the first several times I 
> felt the need for raw strings, it was for directory names that wanted to 
> end with \ and couldn't.

Can you elaborate? I find it unlikely that I would ever want a docstring 
that ends with a backslash:

def func():
r"""Documentation goes here...
more documentation...
ending with a Windows path that needs a trailing backslash
like this C:\directory\"""

That seems horribly contrived. Why use backslashes in the path when the 
strong recommendation is to use forward slashes?

And why not solve the problem by simply moving the closing quotes to the 
next line, as PEP 8 recommends?

r"""Documentation ...
C:\directory\
"""

[...]
> >Even in a raw literal, quotes can be escaped with a backslash

Indeed, they're not so much "raw" strings as only slightly blanched 
strings.

-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z354BLWONCWMUMFULE64MWUK4TA6PMK2/

[Python-Dev] Re: What to do about invalid escape sequences


On 8/9/2019 2:53 PM, Steven D'Aprano wrote:

On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:


The reason I never use raw strings is in the documentation, it is
because \ still has a special meaning, and the first several times I
felt the need for raw strings, it was for directory names that wanted to
end with \ and couldn't.

Can you elaborate? I find it unlikely that I would ever want a docstring


I didn't mention docstring.  I just wanted a string with a path name 
ending in \.



that ends with a backslash:

 def func():
 r"""Documentation goes here...
 more documentation...
 ending with a Windows path that needs a trailing backslash
 like this C:\directory\"""

That seems horribly contrived. Why use backslashes in the path when the
strong recommendation is to use forward slashes?


Windows users are used to seeing backslashes in paths, I don't care to 
be the one to explain why my program uses / and all the rest use \.



And why not solve the problem by simply moving the closing quotes to the
next line, as PEP 8 recommends?

 r"""Documentation ...
 C:\directory\
 """


This isn't my problem, I wasn't using docstrings, and including a 
newline in a path name doesn't work.  I suppose one could "solve" the 
problem by using


"c:\directory\ "[ :-1]

but that is just as annoying as

"c:\\directory\\"

and back when I discovered the problem, I was still learning Python, and 
didn't think of the above solution either.





[...]

Even in a raw literal, quotes can be escaped with a backslash

Indeed, they're not so much "raw" strings as only slightly blanched
strings.




___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EHE2CNRDGS6AF6GYO4DX7UNNE24JH6CG/

[Python-Dev] Re: What to do about invalid escape sequences

I'm not trying to be confrontational, I'm trying to understand your 
use-case(s) and see if it would be broken by the planned change to 
string escapes.

On Fri, Aug 09, 2019 at 03:18:29PM -0700, Glenn Linderman wrote:
> On 8/9/2019 2:53 PM, Steven D'Aprano wrote:
> >On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:
> >
> >>The reason I never use raw strings is in the documentation, it is
> >>because \ still has a special meaning, and the first several times I
> >>felt the need for raw strings, it was for directory names that wanted to
> >>end with \ and couldn't.
> >Can you elaborate? I find it unlikely that I would ever want a docstring
> 
> I didn't mention docstring.  I just wanted a string with a path name 
> ending in \.

You said you never used raw strings in the documentation. I read that as 
doc strings. What sort of documentation are you writing that isn't a doc 
string but is inside your .py files where the difference between raw and 
regular strings is meaningful?

> Windows users are used to seeing backslashes in paths, I don't care to 
> be the one to explain why my program uses / and all the rest use \.

If you don't use raw strings for paths, you get to explain why your 
program uses \\ and all the rest use \ *wink*

If they're Windows end users, they won't be reading your source code and 
will never know how you represent hard-coded paths in the source code.

If they're Windows developers, they ought to be aware that the Windows 
file system API allows / anywhere you can use \ and it is the 
common convention in Python to use forward slashes.

I'm also curious why the string needs to *end* with a backslash. Both of 
these are the same path:

C:\foo\bar\baz\
C:\foo\bar\baz

-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UT4WDQRJ5U5TA5YYHOM4RRDZV6KEC347/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Gregory P. Smith

On Fri, Aug 9, 2019 at 8:43 AM Guido van Rossum  wrote:

> This discussion looks like there's no end in sight. Maybe the Steering
> Council should take a vote?
>

I've merged the PR reverting the behavior in 3.8 and am doing the same in
the master branch.

The sheer volume of email this is generating shows that we're not ready to
do this to our users.

Recall the nightmare caused by md5.py and sha.py DeprecationWarning's in
2.5...  this would be similar.

We need owners of code to see the problems, not end users of other peoples
code.

FWIW, lest people think I don't like this change and just pushed the revert
buttons as a result, wrong.  I agree with the ultimate SyntaxError and
believe we should move the language there (it is better for long term code
quality).  But it needs to be done in a way that disrupts the *right*
people in the process, not disrupting an exponentially higher number of
users of other peoples code.

If the steering council does anything it should be deciding if we're still
going to do this at all and, if so, planning how we do it without repeating
past mistakes.

-gps
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/W4BUFLMDX5FAFOVLYP4C2LQ2HOTJZEZX/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread Gregory P. Smith

On Fri, Aug 9, 2019 at 11:37 AM Eric V. Smith  wrote:

> On 8/9/2019 2:28 PM, Jonathan Goble wrote:
> > On Fri, Aug 9, 2019 at 12:34 PM Nick Coghlan  wrote:
> >> I find the "Our deprecation warnings were even less visible than
> >> normal" argument for extending the deprecation period compelling.
> > Outsider's 2 cents from reading this discussion (with no personal
> > experience with this warning):
> >
> > I am perplexed at the opinion, seemingly espoused by multiple people
> > in this thread, that because a major part of the problem is that the
> > warnings were not visible enough, somehow the proposed solution is
> > making them not visible enough again? It's too late, in my
> > understanding, in the 3.8 cycle to add a new feature like a change to
> > how these warnings are produced (it seems a significant change to the
> > .pyc structure is needed to emit them at runtime), so this supposed
> > "solution" is nothing but kicking the can down the road. When 3.9
> > rolls around, public exposure to the problem of invalid escape
> > sequences will still be approximately what it is now (because if
> > nobody saw the warnings in 3.7, they certainly won't see them in 3.8
> > with this "fix"), so you'll end up with the same complaints about
> > SyntaxWarning that started this discussion, end up back on
> > DeprecationWarning for 3.9 (hopefully with support for emitting them
> > at runtime instead of just compile-time), then have to wait until
> > 3.10/4.0 for SyntaxWarning and eventually the next version to actually
> > make them errors.
>
> Yes, I think that's the idea: Deprecation warning in 3.9, but more
> visible that what 3.7 has. That is, not just at compile time but at run
> time. What's required to make that happen is an open question.
>

i've lost track of who suggested what in this thread, but yes, that concept
has been rolling over in my mind as a potentially good idea after someone
suggested it.  Compile time warnings should turn into bytecode for a
warnings.warn call in the generated pyc.  I haven't spent time trying to
reason if that actually addresses the real issues we're having moving
forward with a syntax warning change though.  A reasonable feature to ask
for as a feature in 3.9 or later perhaps.

-gps
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BR7T76SXANRAGJ3QOMWZUEGRVPVP/

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-09 Thread MRAB

On 2019-08-09 23:56, Steven D'Aprano wrote:

I'm not trying to be confrontational, I'm trying to understand your
use-case(s) and see if it would be broken by the planned change to
string escapes.

On Fri, Aug 09, 2019 at 03:18:29PM -0700, Glenn Linderman wrote:

On 8/9/2019 2:53 PM, Steven D'Aprano wrote:
>On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:
>
>>The reason I never use raw strings is in the documentation, it is
>>because \ still has a special meaning, and the first several times I
>>felt the need for raw strings, it was for directory names that wanted to
>>end with \ and couldn't.
>Can you elaborate? I find it unlikely that I would ever want a docstring

I didn't mention docstring.  I just wanted a string with a path name 
ending in \.

You said you never used raw strings in the documentation. I read that as
doc strings. What sort of documentation are you writing that isn't a doc
string but is inside your .py files where the difference between raw and
regular strings is meaningful?

Windows users are used to seeing backslashes in paths, I don't care to 
be the one to explain why my program uses / and all the rest use \.

If you don't use raw strings for paths, you get to explain why your
program uses \\ and all the rest use \ *wink*

If they're Windows end users, they won't be reading your source code and
will never know how you represent hard-coded paths in the source code.

If they're Windows developers, they ought to be aware that the Windows
file system API allows / anywhere you can use \ and it is the
common convention in Python to use forward slashes.

I'm also curious why the string needs to *end* with a backslash. Both of
these are the same path:

 C:\foo\bar\baz\
 C:\foo\bar\baz

The only time it's required is for the root directory of a drive:

C:\
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GIPRSYAINB4NE4IORCYRTYN7TZWMCZ34/

[Python-Dev] Re: What to do about invalid escape sequences

On Fri, Aug 09, 2019 at 02:28:13PM -0400, Jonathan Goble wrote:

> I am perplexed at the opinion, seemingly espoused by multiple people
> in this thread, that because a major part of the problem is that the
> warnings were not visible enough, somehow the proposed solution is
> making them not visible enough again?

Making the warnings invisible by default is only the first step, not the 
entire solution.

We don't break backwards compatibility lightly, and the current 
behaviour is not an accident, it is a documented feature which 
developers are entitled to rely on.

We are chosing to change that behaviour, breaking backwards 
compatibility, to the inconvenience of end-users, library authors, 
and developers on Mac/Unix/Linux, for two benefits:

1. To possibly allow the addition of new escape sequences such as \e 
some time in the future.

2. To strongly discourage newbie Windows developers from hard-coding 
paths using backslashes, but to use forward slashes instead.

Especially on Python-Ideas, time and time again we hear the mantra that 
we should only break backwards compatibility if the benefit strongly 
outweighs the cost of change. Raymond has given compelling (to me at 
least) testimony that right now, the cost of change is far too high for 
the two minor benefits gained.

So *right now*, it looks like we ought to be prepared to back away from 
the change altogether. We thought that the balance would be:

"it will be a little bit painful, but the benefit will outweigh the pain"

justifying breaking backwards compatibility, but we have found that the 
pain is greater than expected. If we cannot reduce the pain, and move 
the balance into the "nett positive" rather than the "nett negative" we 
have right now, we ought to cancel the deprecation.

Making the deprecation silent by default will reduce the pain. That's 
the first step. Pushing the deprecation schedule back a release or more 
will give us time to rethink the deprecation process, fix the technical 
issues we discovered about SyntaxWarnings, and give library authors time 
to eliminate the warnings from their libraries.

> It's too late, in my
> understanding, in the 3.8 cycle to add a new feature like a change to
> how these warnings are produced (it seems a significant change to the
> .pyc structure is needed to emit them at runtime), so this supposed
> "solution" is nothing but kicking the can down the road.

Is that a problem? Any deadline we have to make unrecognised backslash 
escapes an error is a self-imposed deadline. We lived with this feature 
for more than a quarter of a century, we can keep kicking the can down 
the road until the benefit outweighs the pain.

If that means "forever", then I personally will be sad, but so be it.

However, even if it is too late to add any new tools or features to 
Python 3.8 (and that's not clear: this won't be a *language* change, so 
the feature freeze may not apply) all is not lost.

We're aware of the problem, and can start pointing library authors at 
this thread, and the relevent b.p.o. ticket, and push them in the right 
direction.

Raymond mentioned two libraries by name, bottle and docutils, and Matt 
scanned the top 100 packages on PyPI. That's a good place to start for 
anyone wanting to contribute: raise bug reports on the individual 
library trackers. (If they haven't already been raised.)

https://github.com/bottlepy/bottle/issues

(I'd do that myself except I have technical problems using Github.)

I have reported it to docutils:

https://sourceforge.net/p/docutils/bugs/373/

[...]
> So put these warnings front and center now
> so package and code maintainers actually see it

The problem is that this seriously and negatively affects the experience 
for many end-users. That's what we're trying to prevent, or at least 
mitigate.

-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B7FH5IGUX24J7X7QEANAOSTIKTOHZJ5E/

[Python-Dev] Re: What is a public API?

2019-08-09 Thread Kyle Stanley

Nick Coghlan wrote:
> It's not an unwritten rule, as it already has its own subsection in
> PEP 8: 
> https://www.python.org/dev/peps/pep-0008/#public-and-internal-interfaces
> The main question in this thread is what to do about standard library
> modules that were written before those documented guidelines were put
> in place, and hence have multiple internal APIs that lack the leading
> underscore (and I don't think that's a question with a generic
> answer).
> Cheers,
> Nick.

Oh I see, thanks for the clarification. I've read over most of PEP 8 a few 
times at this point, but somehow I missed this part: "All undocumented
interfaces should be assumed to be internal". Apologies for that.

Personally, I think the stdlib modules which were written before the rule
should be gradually updated, with each one being it's own issue with
the respective experts for each of the modules carefully monitoring the changes.

It would also be appropriate to provide any user attempting to import
a module that is going to be prepended with an underscore with
warnings, and at least a couple of versions to update their code.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ICAXVGLOHF7H3GSL7OONI3Y6XDBBB54Z/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Greg Ewing


Steven D'Aprano wrote:
I find it difficult to imagine a more meaningful distinction than that 
between syntax and semantics.


That may be so, but when specifying a programming language, there is
often some flexibility as to whether language rules are considered
part of the syntax or the semantics.

An extreme example of this is the formalism used to specify Algol 68,
where, believe it or not, things such as the fact that identifiers must
be declared in an enclosing scope are expressed in the *grammar*.

Strictly speaking, anything that makes a particular sequence of symbols
not be a valid program, that could in principle be determined by static
analysis, can be considered a syntactic issue. So classifying a
TargetScopeError as a kind of SyntaxError isn't necessarily wrong.

There's actually a precedent for this in Python already:

>>> def f():
...  nonlocal x
...  x = 5
...
  File "", line 2
SyntaxError: no binding for nonlocal 'x' found

We seem to be happy to use a plain SyntaxError for that, not even
bothering with a subclass. So I'm inclined to agree that TargetScopeError
is not necessary.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PPHYLNGI3N5A5E5RQQTBLM2PUJB6DASC/

[Python-Dev] Re: What is a public API?

2019-08-09 Thread Kyle Stanley

Kyle Stanley wrote:
> It would also be appropriate to provide any user attempting to import
> a module that is going to be prepended with an underscore with
> warnings, and at least a couple of versions to update their code.

Clarification: When I mentioned prepending a module with an 
underscore, I meant for functions and classes within the module, 
not the module itself. 

It might be difficult to implement this in a way which does
not cause an excessive number of warnings, but I think it's
definitely worthwhile to aim towards having a fully consistent
standard for differentiating public and private interfaces
across all of stdlib.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2O7PZB5BGRUPPTPH4Q4MQIXUWT734BQY/

[Python-Dev] Re: What to do about invalid escape sequences

On 8/9/2019 4:07 PM, Gregory P. Smith wrote:

On Fri, Aug 9, 2019 at 11:37 AM Eric V. Smith > wrote:

On 8/9/2019 2:28 PM, Jonathan Goble wrote:
> On Fri, Aug 9, 2019 at 12:34 PM Nick Coghlan mailto:ncogh...@gmail.com>> wrote:
>> I find the "Our deprecation warnings were even less visible than
>> normal" argument for extending the deprecation period compelling.
> Outsider's 2 cents from reading this discussion (with no personal
> experience with this warning):
>
> I am perplexed at the opinion, seemingly espoused by multiple people
> in this thread, that because a major part of the problem is that the
> warnings were not visible enough, somehow the proposed solution is
> making them not visible enough again? It's too late, in my
> understanding, in the 3.8 cycle to add a new feature like a
change to
> how these warnings are produced (it seems a significant change
to the
> .pyc structure is needed to emit them at runtime), so this supposed
> "solution" is nothing but kicking the can down the road. When 3.9
> rolls around, public exposure to the problem of invalid escape
> sequences will still be approximately what it is now (because if
> nobody saw the warnings in 3.7, they certainly won't see them in 3.8
> with this "fix"), so you'll end up with the same complaints about
> SyntaxWarning that started this discussion, end up back on
> DeprecationWarning for 3.9 (hopefully with support for emitting them
> at runtime instead of just compile-time), then have to wait until
> 3.10/4.0 for SyntaxWarning and eventually the next version to
actually
> make them errors.

Yes, I think that's the idea: Deprecation warning in 3.9, but more
visible that what 3.7 has. That is, not just at compile time but
at run
time. What's required to make that happen is an open question.

i've lost track of who suggested what in this thread, but yes, that 
concept has been rolling over in my mind as a potentially good idea 
after someone suggested it.  Compile time warnings should turn into 
bytecode for a warnings.warn call in the generated pyc.  I haven't 
spent time trying to reason if that actually addresses the real issues 
we're having moving forward with a syntax warning change though. A 
reasonable feature to ask for as a feature in 3.9 or later perhaps.

The documentation actually claims it was deprecated in version 3.6. So 
it has already been 2 releases worth of deprecation, visible warning or not.

Ship it.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YYSC5CDJWOF24AUWC4IHJG45COHOTHW3/

[Python-Dev] Re: What to do about invalid escape sequences

On 8/9/2019 4:08 PM, MRAB wrote:

On 2019-08-09 23:56, Steven D'Aprano wrote:

I'm not trying to be confrontational, I'm trying to understand your
use-case(s) and see if it would be broken by the planned change to
string escapes.

On Fri, Aug 09, 2019 at 03:18:29PM -0700, Glenn Linderman wrote:

On 8/9/2019 2:53 PM, Steven D'Aprano wrote:
>On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:
>
>>The reason I never use raw strings is in the documentation, it is
>>because \ still has a special meaning, and the first several times I
>>felt the need for raw strings, it was for directory names that 
wanted to

>>end with \ and couldn't.
>Can you elaborate? I find it unlikely that I would ever want a 
docstring

I didn't mention docstring.  I just wanted a string with a path name 
ending in \.

You said you never used raw strings in the documentation. I read that as
doc strings. What sort of documentation are you writing that isn't a doc
string but is inside your .py files where the difference between raw and
regular strings is meaningful?

Windows users are used to seeing backslashes in paths, I don't care 
to be the one to explain why my program uses / and all the rest use \.

If you don't use raw strings for paths, you get to explain why your
program uses \\ and all the rest use \ *wink*

If they're Windows end users, they won't be reading your source code and
will never know how you represent hard-coded paths in the source code.

If they're Windows developers, they ought to be aware that the Windows
file system API allows / anywhere you can use \ and it is the
common convention in Python to use forward slashes.

I'm also curious why the string needs to *end* with a backslash. Both of
these are the same path:

 C:\foo\bar\baz\
 C:\foo\bar\baz

The only time it's required is for the root directory of a drive:

C:\

That's not the only time it's required, but it is a case that is far 
harder to specify in other ways.  It's required any time you  want to 
say   + filename without writing + "\\" + filename, or os.path.join( 
'C:\\", filename )
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LXM72OGMFTNJP3NQPITJWWGB6ITNRBH4/

[Python-Dev] Re: What to do about invalid escape sequences


On 8/9/2019 3:56 PM, Steven D'Aprano wrote:

I'm not trying to be confrontational, I'm trying to understand your
use-case(s) and see if it would be broken by the planned change to
string escapes.


Yeah, that's fine. Sometimes it is hard to communicate via email (versus 
saying a lot).



On Fri, Aug 09, 2019 at 03:18:29PM -0700, Glenn Linderman wrote:

On 8/9/2019 2:53 PM, Steven D'Aprano wrote:

On Fri, Aug 09, 2019 at 01:12:59PM -0700, Glenn Linderman wrote:


The reason I never use raw strings is in the documentation, it is
because \ still has a special meaning, and the first several times I
felt the need for raw strings, it was for directory names that wanted to
end with \ and couldn't.

Can you elaborate? I find it unlikely that I would ever want a docstring

I didn't mention docstring.  I just wanted a string with a path name
ending in \.

You said you never used raw strings in the documentation. I read that as
doc strings. What sort of documentation are you writing that isn't a doc
string but is inside your .py files where the difference between raw and
regular strings is meaningful?


No, what I said was that the reason is in the documentation. The reason 
that I don't use raw strings is in the Python documentation. I don't 
claim to use raw strings for documentation I write. The reason is 
because \" to end the string doesn't work, and the first good-sounding 
justification for using raw strings that I stumbled across was to avoid 
"c:\\directory\\" in favor of r"c:\directory\"  but that doesn't work, 
and neither do r"c:\directory\\". Since then, I have not found any other 
compelling need for raw strings that overcome that deficiency... the 
benefit of raw strings is that you don't have to double the \\. But the 
benefit is contradicted by not being able to use one at the end of 
sting. If you can't use it at the end of the string, the utility of not 
doubling them in the middle of the string is just too confusing to make 
it worth figuring out the workarounds when you have a string full of \ 
that happens to end in \. Just easier to remember the "always double \" 
rule, than to remember the extra "but if your string containing \ 
doesn't have one at the end you can get away with using a raw string and 
not doubling the \.



Windows users are used to seeing backslashes in paths, I don't care to
be the one to explain why my program uses / and all the rest use \.

If you don't use raw strings for paths, you get to explain why your
program uses \\ and all the rest use \ *wink*
Wrong. Users don't look at the source code. They look at the output. I 
also don't want to have to write code to convert /-laden paths to 
\-laden paths when I display them to the user.




If they're Windows end users, they won't be reading your source code and
will never know how you represent hard-coded paths in the source code.


They will if I display the path as a default value for an argument, or 
show them the path for other reasons, or if the path shows up in an 
exception message.




If they're Windows developers, they ought to be aware that the Windows
file system API allows / anywhere you can use \ and it is the
common convention in Python to use forward slashes.


This, we can agree on.


I'm also curious why the string needs to *end* with a backslash. Both of
these are the same path:

 C:\foo\bar\baz\
 C:\foo\bar\baz


Sure. But only one of them can be used successfully with   + filename  
(for example).


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OYRSO4WHUFA7Q34HJTBIMQL337JTA5RX/

[Python-Dev] Re: PEP 572 TargetScopeError

On Fri, Aug 9, 2019 at 11:06 AM Barry Warsaw  wrote:

> Nick and I are now on the same page, so I don’t think we *have* to have a
> formal SC vote.
>

OK, I agree. Let's remove TargetScopeError from the PEP and from the
implementation. Since that makes three SC members in favor the motion
passes. I hope Chris A (the third PEP author) can live with this -- in his
last post on the subject that I can find he was still in favor of a
subclass of SyntaxError (in any case he's outvoted by the other two PEP
authors :-).
-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him/his **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/D62QIKOD6TPCJK3GMOEKNTE4HRUR36NM/

[Python-Dev] Re: PEP 572 TargetScopeError

2019-08-09 Thread Chris Angelico

On Sat, Aug 10, 2019 at 12:56 PM Guido van Rossum  wrote:
>
> On Fri, Aug 9, 2019 at 11:06 AM Barry Warsaw  wrote:
>>
>> Nick and I are now on the same page, so I don’t think we *have* to have a 
>> formal SC vote.
>
>
> OK, I agree. Let's remove TargetScopeError from the PEP and from the 
> implementation. Since that makes three SC members in favor the motion passes. 
> I hope Chris A (the third PEP author) can live with this -- in his last post 
> on the subject that I can find he was still in favor of a subclass of 
> SyntaxError (in any case he's outvoted by the other two PEP authors :-).
>

I was at the time, but it was only a weak recommendation. And having
seen other examples (eg a global statement doubling a function
argument) that just use SyntaxError, I'm totally fine with just making
it SyntaxError with a useful text message.

So go for it! Remove TargetScopeError.

(That said, though: it may be helpful to have some metadata attached
to the exception, like the name that's being assigned to. Not sure how
easy/hard that'll be.)

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZFI4EL2BI2J2XN3GZLWSGRSP7HG5DCG7/

[Python-Dev] Re: PEP 572 TargetScopeError