[Python-Dev] Re: Question about bytecode stability

[Thomas Wouters]

On Mon, 6 Apr 2020, 06:30 Brandt Bucher,  wrote:

> > The best way to look at this is to consider how long a .pyc file is
> valid. They're currently named something like
> __pycache__/modulename.cpython-38.pyc which is a fairly clear indication
> that the cached compiled module should be valid for any CPython 3.8.x
> release.
>
> Perhaps an even better indicator is the “magic number” that invalidates
> the cached bytecode.


Probably the best indicator is PEP 6, which explicitly states bytecode
compatibility is a requirement:

"All .pyc and .pyo files must work (no regeneration needed) with all bugfix
releases forked off from a major release."

https://www.python.org/dev/peps/pep-0006/#prohibitions

--
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZPU4IR2O2C6GU3N3F3WKLSPWV7D7STP4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Question about bytecode stability

[Serhiy Storchaka]

06.04.20 06:48, Jonathan Goble пише:
My question is, are the opcodes guaranteed stable across the lifetime of 
a single 3.x release? In other words, are they guaranteed to not change 
values or semantics between 3.x.y and 3.x.(y+1)? Reading through the 
list of opcodes in the dis documentation, it seems that all changes have 
occurred in 3.x.0, so it seems the answer would be yes, but then the 
"CPython implementation detail" paragraph at the top doesn't specify 
that and is a little vague on whether that's true or not.


At least once it was changed in the bugfix release (3.5.2). It was minor 
change, and mostly compatible, but it changed the magic number, so all 
pyc files were needed to regenerate, and *that* caused problems for 
distributors. We no longer want to repeat this. But I do not know what 
we will do if a fatal flaw in the bytecode be found after release.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GTVKJWLCVHT3HPVZ6OAFUOSUYINOKR5W/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Any thoughts about a control flow optimizer for CPython?

[Mark Shannon]

Hi,

On 05/04/2020 12:47 pm, Ned Batchelder wrote:

On 4/3/20 11:13 AM, joannah nanjekye wrote:

Hey all,
From my CS theory, a control flow graph models a program flow and one 
of its main characteristics is it has one entry and exit point. IIRC, 
CPython’s compilation process involves generation of a control flow graph.


Contrary to peephole optimizations, optimizations on the  control flow 
graph are more global allowing us to have complex and global 
optimizations like branch and checkpoint eliminations etc.



I have seen several implementations of control flow optimizations. The 
one I am familiar with is the V8 control flow optimizer.



I tried to investigate this for one of my directed courses last fall 
but I want to know if there are people who have been thinking about 
this for CPython and what their thoughts are.




Please make it possible to disable any optimizations.  Sometimes 
programs are run to understand the program, not for speed (for example, 
during debugging, or coverage measurement.)


I have to disagree. The behavior of a program should not depend on 
whether a flag is set or not. It makes debugging harder, IMO, if the 
behavior changes depending on some flag.


For example, `pass` statements should never be executed.
This is an "optimization", but it is easier to understand if `pass` is 
never executed. It becomes confusing it "executes" some of the time, 
depending on some compiler flag.


IMO, a coverage tool should be not rely on `pass`, `while True:` and 
other similarly trivial statements existing in the bytecode.
By "trivial statements", I mean any unconditional control flow 
statements, conditional control flow statements with a constant test, 
"pass", and any expression statements that are constant and without side 
effect.


E.g. the following statements are "trivial":
try:
while False:
pass
1 + 1

On the other hand, other more powerful optimization should always 
preserve the observable behavior of the program.
Since the observable behavior is unchanged, there would be no need to 
turn them off.


Cheers,
Mark.



--Ned.


--
//Best,
Joannah Nanjekye
/"You think you know when you learn, are more sure when you can write, 
even more when you can teach, but certain when you can program."

Alan J. Perlis/

___
Python-Dev mailing list --python-dev@python.org
To unsubscribe send an email topython-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived 
athttps://mail.python.org/archives/list/python-dev@python.org/message/I376QAAWUHVLO5WGFFNOBTZAUPAVZKCB/
Code of Conduct:http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UGEVQQWB32DQHBONJKTL2Q5K2YPCPIN2/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BMYQ3TGW7YUUYGCBEPDG7E4X762ZKEX2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Any thoughts about a control flow optimizer for CPython?

[Ned Batchelder]

On 4/6/20 5:41 AM, Mark Shannon wrote:

Hi,

On 05/04/2020 12:47 pm, Ned Batchelder wrote:

On 4/3/20 11:13 AM, joannah nanjekye wrote:

Hey all,
From my CS theory, a control flow graph models a program flow and 
one of its main characteristics is it has one entry and exit point. 
IIRC, CPython’s compilation process involves generation of a control 
flow graph.


Contrary to peephole optimizations, optimizations on the control 
flow graph are more global allowing us to have complex and global 
optimizations like branch and checkpoint eliminations etc.



I have seen several implementations of control flow optimizations. 
The one I am familiar with is the V8 control flow optimizer.



I tried to investigate this for one of my directed courses last fall 
but I want to know if there are people who have been thinking about 
this for CPython and what their thoughts are.




Please make it possible to disable any optimizations.  Sometimes 
programs are run to understand the program, not for speed (for 
example, during debugging, or coverage measurement.)


I have to disagree. The behavior of a program should not depend on 
whether a flag is set or not. It makes debugging harder, IMO, if the 
behavior changes depending on some flag.
I agree.  But perhaps we need to be clearer what we mean by behavior.  I 
don't think an optimizer should change the behavior of a program, that 
is, the results it produces. If an optimizer meets that criterion, then 
enabling or disabling it doesn't affect the behavior of the program.


For example, `pass` statements should never be executed.
This is an "optimization", but it is easier to understand if `pass` is 
never executed. It becomes confusing it "executes" some of the time, 
depending on some compiler flag.


IMO, a coverage tool should be not rely on `pass`, `while True:` and 
other similarly trivial statements existing in the bytecode.
By "trivial statements", I mean any unconditional control flow 
statements, conditional control flow statements with a constant test, 
"pass", and any expression statements that are constant and without 
side effect.


The results of a coverage tool depend very much on exactly which lines 
get executed. If an optimizer decides it can skip a certain line, then 
the coverage results will show that line as unexecuted. This alarms 
users, and causes bug reports. See https://bugs.python.org/issue2506 for 
an example that has caused me headaches since 2008.


When you work in C, do you ever use the -O0 flag to disable 
optimizations? If so, why do you use it? Why does Python not have the 
same problems, and why does it not deserve a similar solution?




E.g. the following statements are "trivial":
try:
while False:
pass
1 + 1

On the other hand, other more powerful optimization should always 
preserve the observable behavior of the program.
Since the observable behavior is unchanged, there would be no need to 
turn them off.


My point is that coverage measurement is a different kind of 
observation.  Please don't overlook developers' needs in this regard.


--Ned



Cheers,
Mark.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4YWR2LHBH4OCU43QIUTYGWZRDPSGPQYD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Fabio Zadrozny]

On Thu, Apr 2, 2020 at 3:16 PM Guido van Rossum  wrote:

> Since last fall's core sprint in London, Pablo Galindo Salgado, Lysandros
> Nikolaou and myself have been working on a new parser for CPython. We are
> now far enough along that we present a PEP we've written:
>
> https://www.python.org/dev/peps/pep-0617/
>
> Hopefully the PEP speaks for itself. We are hoping for a speedy resolution
> so we can land the code we've written before 3.9 beta 1.
>
> If people insist I can post a copy of the entire PEP here on the list, but
> since a lot of it is just background information on the old LL(1) and the
> new PEG parsing algorithms, I figure I'd spare everyone the need of reading
> through that. Below is a copy of the most relevant section from the PEP.
> I'd also like to point out the section on performance (which you can find
> through the above link) -- basically performance is on a par with that of
> the old parser.
>
>
Hi Guido,

I think using a PEG parser is interesting, but I do have some questions
related to what's to expect in the future for other people which have to
follow the Python grammar, so, can you shed some light on this?

Does that mean that the grammar format currently available (which is
currently specified in https://docs.python.org/3.8/reference/grammar.html)
will no longer be updated/used?

Is it expected that other language implementations/parsers also have to
move to a PEG parser in the future? -- which would probably be the case if
the language deviates strongly off LL(1)

Thanks,

Fabio
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZETOUF7L7XBPJ2D2U7UZZBBBDTP727XZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Any thoughts about a control flow optimizer for CPython?

[Ned Batchelder]

BTW, so that we don't have to completely retrace our steps, this topic 
was also discussed in depth on Python-Ideas in May 2014: 
https://mail.python.org/archives/list/python-id...@python.org/thread/X6VCB2E4EHOLUWHO42FYUT6VDAFNUHJF/


--Ned.

On 4/6/20 6:56 AM, Ned Batchelder wrote:

On 4/6/20 5:41 AM, Mark Shannon wrote:

Hi,

On 05/04/2020 12:47 pm, Ned Batchelder wrote:

On 4/3/20 11:13 AM, joannah nanjekye wrote:

Hey all,
From my CS theory, a control flow graph models a program flow and 
one of its main characteristics is it has one entry and exit point. 
IIRC, CPython’s compilation process involves generation of a 
control flow graph.


Contrary to peephole optimizations, optimizations on the control 
flow graph are more global allowing us to have complex and global 
optimizations like branch and checkpoint eliminations etc.



I have seen several implementations of control flow optimizations. 
The one I am familiar with is the V8 control flow optimizer.



I tried to investigate this for one of my directed courses last 
fall but I want to know if there are people who have been thinking 
about this for CPython and what their thoughts are.




Please make it possible to disable any optimizations. Sometimes 
programs are run to understand the program, not for speed (for 
example, during debugging, or coverage measurement.)


I have to disagree. The behavior of a program should not depend on 
whether a flag is set or not. It makes debugging harder, IMO, if the 
behavior changes depending on some flag.
I agree.  But perhaps we need to be clearer what we mean by behavior.  
I don't think an optimizer should change the behavior of a program, 
that is, the results it produces. If an optimizer meets that 
criterion, then enabling or disabling it doesn't affect the behavior 
of the program.


For example, `pass` statements should never be executed.
This is an "optimization", but it is easier to understand if `pass` 
is never executed. It becomes confusing it "executes" some of the 
time, depending on some compiler flag.


IMO, a coverage tool should be not rely on `pass`, `while True:` and 
other similarly trivial statements existing in the bytecode.
By "trivial statements", I mean any unconditional control flow 
statements, conditional control flow statements with a constant test, 
"pass", and any expression statements that are constant and without 
side effect.


The results of a coverage tool depend very much on exactly which lines 
get executed. If an optimizer decides it can skip a certain line, then 
the coverage results will show that line as unexecuted. This alarms 
users, and causes bug reports. See https://bugs.python.org/issue2506 
for an example that has caused me headaches since 2008.


When you work in C, do you ever use the -O0 flag to disable 
optimizations? If so, why do you use it? Why does Python not have the 
same problems, and why does it not deserve a similar solution?




E.g. the following statements are "trivial":
try:
while False:
pass
1 + 1

On the other hand, other more powerful optimization should always 
preserve the observable behavior of the program.
Since the observable behavior is unchanged, there would be no need to 
turn them off.


My point is that coverage measurement is a different kind of 
observation.  Please don't overlook developers' needs in this regard.


--Ned



Cheers,
Mark.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E5GU2FPPWBWB4RPUEZVELJ75HZAYEPO3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: How to enable tracemalloc for the test suite?

[Victor Stinner]

Le dim. 5 avr. 2020 à 23:07, Skip Montanaro  a écrit :
> I'm trying it the simplest way I can think of. Registers are exactly
> like local variables, so SETLOCAL Py_XDECREFs whatever is already
> there before overwriting it with a new value. At the end of
> _PyEval_EvalFrameDefault if the code object's co_flags includes the
> (new) CO_REGISTER flag, it loops over the stack/register space calling
> Py_CLEAR. The stack/register space is also Py_CLEAR'd when the frame
> is first allocated.

Let's say that you call "func()". The stack-based bytecode "LOAD_GLOAL
0; CALL_FUNC 0" becomes register-based bytecode "LOAD_GLOBAL_REG %r1,
0; CALL_FUNC_REG %r2, %r1": the function result is stored into %r2
register.

The problem is that if the %r2 is not cleared explicitly, you change
the object lifetime and so the Python semantics. The register %r2 must
be cleared explicitly.

Same issue applies for %r1 (which stores a global variable).

The correct code should be "LOAD_GLOBAL_REG %r1, 0; CALL_FUNC_REG %r2,
%r1; CLEAR_REG %r1; CLEAR_REG %r2".

A compiler can optimize it to: "LOAD_GLOBAL_REG %r1, 0; CALL_FUNC_REG
%r1, %r1; CLEAR_REG %r1" (remove %r2: reuse %r1).

For example, I expect to get immediately a ResourceWarning when
running the following script demo.py with python3 -Wd demo.py:
---
import time

def func():
open("/etc/issue")

func()
# ResourceWarning expected here
time.sleep(60)
---

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/72Q5GNAX42TWSR4PKAVPNDQDAJHWW3KL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Jeff Allen]

The PEP gives a good exposition of the problem and proposed solution, 
thanks.


If I understand correctly, the proposal is that the PEG grammar should 
become the definitive grammar for Python at some point, probably for 
Python 3.10, so it may evolve without the LL(1) restrictions. I'd like 
to raise some points with respect to that, which perhaps the migration 
section could answer.


When definitive, the grammar would not then just be for CPython, and 
would also appear as user documentation of the language. Whether that 
change leaves Python with a more useful (readable) grammar seems an 
important test of the idea. I'm looking at 
https://github.com/we-like-parsers/cpython/blob/pegen/Grammar/python.gram 
, and assuming that is indicative of a future definitive grammar. That 
may be incorrect, as it has these issues in my view:


1. It is decorated with actions in C. If a decorated grammar is offered 
as definitive, one with Python actions (operations on the AST) is 
preferable, as implementation neutral, although still hostage to AST 
changes that are not language changes. Maybe one stripped of actions is 
best.


2. It's quite long, and not at first glance more readable than the LL(1) 
grammar. I had understood ugliness in the LL(1) grammar to result from 
skirting limitations that PEG eliminates. The PEG one is twice as long, 
but recognising about half of it is actions, let's just say that as a 
grammar it's no shorter.


3. There is some manual guidance by means of &-guards, only necessary (I 
think) as a speed-up or to force out meaningful syntax errors. That 
would be noise to the reader. (This goes away if the PEG parser 
generator generate guards from the first set at a simple "no 
backtracking" marker.)


4. In some places, expansive alternatives seem to be motivated by the 
difference between actions, for a start, wherever async pops up. Maybe 
it is also why the definition of lambda is so long. That could go away 
with different support code (e.g. is_async as an argument), but if 
improvements to the support change grammar rules, when the language has 
not changed, that's a danger sign too.


All that I think means that the "operational" grammar from which you 
build the parser is going to be quite unlike the one with which you 
communicate the language. At present ~/Grammar/Grammar both generates 
the parser (I thought) and appears as documentation. I take it to be the 
ideal that we use a single, human-readable definition. For example ANTLR 
4 has worked hard to facilitate a grammar in which actions are implicit, 
and the generation of an AST from the parse tree/events can be 
elsewhere. (I'm not plugging ANTLR specifically as a solution.)


Jeff Allen

On 02/04/2020 19:10, Guido van Rossum wrote:
Since last fall's core sprint in London, Pablo Galindo Salgado, 
Lysandros Nikolaou and myself have been working on a new parser for 
CPython. We are now far enough along that we present a PEP we've written:


https://www.python.org/dev/peps/pep-0617/

Hopefully the PEP speaks for itself. We are hoping for a speedy 
resolution so we can land the code we've written before 3.9 beta 1.


If people insist I can post a copy of the entire PEP here on the list, 
but since a lot of it is just background information on the old LL(1) 
and the new PEG parsing algorithms, I figure I'd spare everyone the 
need of reading through that. Below is a copy of the most relevant 
section from the PEP. I'd also like to point out the section on 
performance (which you can find through the above link) -- basically 
performance is on a par with that of the old parser.


==
Migration plan
==

This section describes the migration plan when porting to the new 
PEG-based parser
if this PEP is accepted. The migration will be executed in a series of 
steps that allow

initially to fallback to the previous parser if needed:

1.  Before Python 3.9 beta 1, include the new PEG-based parser 
machinery in CPython
    with a command-line flag and environment variable that allows 
switching between
    the new and the old parsers together with explicit APIs that allow 
invoking the
    new and the old parsers independently. At this step, all Python 
APIs like ``ast.parse``
    and ``compile`` will use the parser set by the flags or the 
environment variable and

    the default parser will be the current parser.

2.  After Python 3.9 Beta 1 the default parser will be the new parser.

3.  Between Python 3.9 and Python 3.10, the old parser and related 
code (like the
    "parser" module) will be kept until a new Python release happens 
(Python 3.10). In
    the meanwhile and until the old parser is removed, **no new Python 
Grammar
    addition will be added that requires the peg parser**. This means 
that the grammar

    will be kept LL(1) until the old parser is removed.

4.  In Python 3.10, remove the old parser, the command-line flag, the 
environment

    variable and the "parser" module and related code.

[Python-Dev] [RELEASE] Python 2.7.18 release candidate 1

[Benjamin Peterson]

Greetings,
2.7.18 release candidate 1, a testing release for the last release of the 
Python 2.7 series, is now available for download. The CPython core developers 
stopped applying routine bugfixes to the 2.7 branch on January 1. 2.7.18 will 
includes fixes that were made between the release of 2.7.17 and the end of 
2019. A final—very final!—release is expected in 2 weeks.

Downloads are at:

   https://www.python.org/downloads/release/python-2718rc1/

The full changelog is at

   
https://raw.githubusercontent.com/python/cpython/v2.7.18rc1/Misc/NEWS.d/2.7.18rc1.rst

Test it out, and let us know if there are any critical problems at

https://bugs.python.org/

(This is the last chance!)

All the best,
Benjamin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RAEOMDNDZKQVOUUJTG2H2U7XJCCPLJE5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Question about bytecode stability

[Jonathan Goble]

 Thanks for the feedback, everyone. I'm satisfied that the bytecodes are
stable enough to work for my purpose. Now I just have to find time to work
on it around my schoolwork.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QEC773MKZI2YWMKN75PUA3PKR2I6EPPU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Guido van Rossum]

On Mon, Apr 6, 2020 at 4:03 AM Fabio Zadrozny  wrote:

> I think using a PEG parser is interesting, but I do have some questions
> related to what's to expect in the future for other people which have to
> follow the Python grammar, so, can you shed some light on this?
>
> Does that mean that the grammar format currently available (which is
> currently specified in https://docs.python.org/3.8/reference/grammar.html)
> will no longer be updated/used?
>

The grammar format used for the PEG parser is nearly the same as the old
grammar, when you remove actions and some embellishments needed for
actions. The biggest difference is that the `|` operator is no longer
symmetrical (since if you have alternatives `A | B`, and both match at some
point in the input, PEG reports A, while the old generator would reject the
grammar as being ambiguous.


> Is it expected that other language implementations/parsers also have to
> move to a PEG parser in the future? -- which would probably be the case if
> the language deviates strongly off LL(1)
>

We don't specify how other implementations must parse the language -- in
fact I have no idea how the parsers of any of the other implementations
work. I'm sure there will be other ways to parse the same language.

But yeah, if there are implementations that currently closely follow
Python's LL(1) parser structure they may have to be changed once we start
introducing new syntax that makes use of the freedom PEG gives us. (For
example, I've been toying with the idea of introducing a "match" statement
similar to Scala's match expression by making "match" a keyword only when
followed by an expression and a colon.)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WBGUXFL54OLTYINNLRMAW5UH7KSIX7QX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Guido van Rossum]

On Mon, Apr 6, 2020 at 5:18 AM Jeff Allen  wrote:

> The PEP gives a good exposition of the problem and proposed solution,
> thanks.
>
> If I understand correctly, the proposal is that the PEG grammar should
> become the definitive grammar for Python at some point, probably for Python
> 3.10, so it may evolve without the LL(1) restrictions. I'd like to raise
> some points with respect to that, which perhaps the migration section could
> answer.
>
Thanks, you definitely have a point here.

> When definitive, the grammar would not then just be for CPython, and would
> also appear as user documentation of the language. Whether that change
> leaves Python with a more useful (readable) grammar seems an important test
> of the idea. I'm looking at
> https://github.com/we-like-parsers/cpython/blob/pegen/Grammar/python.gram
> , and assuming that is indicative of a future definitive grammar. That may
> be incorrect, as it has these issues in my view:
>
> 1. It is decorated with actions in C. If a decorated grammar is offered as
> definitive, one with Python actions (operations on the AST) is preferable,
> as implementation neutral, although still hostage to AST changes that are
> not language changes. Maybe one stripped of actions is best.
>
Yes, the plan is to strip actions and a few other embellishments (types,
names, cuts, and probably also lookaheads -- although the latter may be
significant, we only use them for optimization). The parser generator (
https://github.com/we-like-parsers/cpython/tree/pegen/Tools/peg_generator)
prints a stripped representation (though currently preserving lookaheads --
suppressing those would be a simple change to the code).

> 2. It's quite long, and not at first glance more readable than the LL(1)
> grammar. I had understood ugliness in the LL(1) grammar to result from
> skirting limitations that PEG eliminates. The PEG one is twice as long, but
> recognising about half of it is actions, let's just say that as a grammar
> it's no shorter.
>
Indeed. I believe part of this actually comes from the desire to be 100%
compatible with the old parser (an important constraint is that we don't
want to change the AST since we don't want to change the byte code
generator).

Another part of it comes from expressing in the grammar constraints that
the old parser generator cannot express. For example, the old parser
accepts `1 = x` as an assignment, and it is rejected in a later stage. The
new parser expresses this restriction in the grammar. Note that the full
grammar published in the reference manual (
https://docs.python.org/3.8/reference/grammar.html) doesn't say anything
about this; the grammar used later to describe assignment_stmt does (
https://docs.python.org/3.8/reference/simple_stmts.html#grammar-token-assignment-stmt),
but as a result it is not LL(1) -- those grammar sections sprinkled
throughout the reference manual are all written and updated by hand (and
sometimes we forget!).

> 3. There is some manual guidance by means of &-guards, only necessary (I
> think) as a speed-up or to force out meaningful syntax errors. That would
> be noise to the reader. (This goes away if the PEG parser generator
> generate guards from the first set at a simple "no backtracking" marker.)
>
Yeah, see above. We've thought of generating FIRST sets as a future
enhancement of the generator, and then they can go away. At the moment the
lookaheads we have are all carefully aimed at optimizing the time and space
requirements of the parser.

> 4. In some places, expansive alternatives seem to be motivated by the
> difference between actions, for a start, wherever async pops up. Maybe it
> is also why the definition of lambda is so long. That could go away with
> different support code (e.g. is_async as an argument), but if improvements
> to the support change grammar rules, when the language has not changed,
> that's a danger sign too.
>
Yeah, lambda is complicated by the requirement on the generated AST.
Arguably we have gone too far here (and for 'parameters', which solves
almost the same problem for regular function definitions) and we should put
some of the checks back in the support code. But I note that the old
grammar also has some warts in the area of parameter definitions (though
its lambda is definitely simpler).

> All that I think means that the "operational" grammar from which you build
> the parser is going to be quite unlike the one with which you communicate
> the language. At present ~/Grammar/Grammar both generates the parser (I
> thought) and appears as documentation. I take it to be the ideal that we
> use a single, human-readable definition. For example ANTLR 4 has worked
> hard to facilitate a grammar in which actions are implicit, and the
> generation of an AST from the parse tree/events can be elsewhere. (I'm not
> plugging ANTLR specifically as a solution.)
>
Our cheaper solution is to remove the actions from the display grammar. But
I don't think that Grammar/Grammar should be seen

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Steven D'Aprano]

On Mon, Apr 06, 2020 at 10:43:11AM -0700, Guido van Rossum wrote:

> I've been toying with the idea of introducing a "match" statement
> similar to Scala's match expression by making "match" a keyword only when
> followed by an expression and a colon.)

Didn't we conclude from `as` that having context-sensitive keywords was 
a bad idea?

Personally, I would not like to have to explain to newcomers why `match` 
is a keyword but you can still use it as a function or variable, but not 
other keywords like `raise`, `in`, `def` etc.

match expression:
match = True


-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2INKROEUGRGCSPUM7L7JY7KI4C7HUP3Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Guido van Rossum]

On Mon, Apr 6, 2020 at 11:36 AM Steven D'Aprano  wrote:

> On Mon, Apr 06, 2020 at 10:43:11AM -0700, Guido van Rossum wrote:
>
> > I've been toying with the idea of introducing a "match" statement
> > similar to Scala's match expression by making "match" a keyword only when
> > followed by an expression and a colon.)
>
> Didn't we conclude from `as` that having context-sensitive keywords was
> a bad idea?
>

I'm not sure that that was the conclusion. At the time the point was that
we *wanted* all keywords to be reserved everywhere, an `as` was an ugly
exception to that rule, which we got rid of as soon as we could -- not
because it was a bad idea but because it violated a somewhat arbitrary rule.

We went through the same thing with `async` and `await`, and the experience
there was worse: a lot of libraries in the very space `async def` was aimed
at were using `async` as a parameter name, often in APIs, and they had to
scramble to redesign their APIs and get their users to change their
programs.

In retrospect I wish we had just kept `async` as a context-sensitive
keyword, since it was totally doable.

(In an early version of the PEG parser, all keywords were
context-sensitive, and there were only very few places in the grammar where
this required us to insert negative lookaheads to make edge cases parse
correctly. The rest was taken care by careful ordering of rules, e.g. the
rule for `del_stmt` must be tried before the rule for `expression_stmt`
since `del *x` would match the latter.)


> Personally, I would not like to have to explain to newcomers why `match`
> is a keyword but you can still use it as a function or variable, but not
> other keywords like `raise`, `in`, `def` etc.
>
> match expression:
> match = True
>

What kind of newcomers do you have that they even notice that, unless you
were to draw attention to it? I'm serious -- from the kind of questions
I've seen in user forums, most newcomers are having a hard enough time
learning more fundamental concepts and abstractions than the precise rules
for reserved words.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XW6RDCITNUWIUPADTJMSIRMPBGMHFKJS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Chris Angelico]

On Tue, Apr 7, 2020 at 5:03 AM Guido van Rossum  wrote:
>
> On Mon, Apr 6, 2020 at 11:36 AM Steven D'Aprano  wrote:
>> Personally, I would not like to have to explain to newcomers why `match`
>> is a keyword but you can still use it as a function or variable, but not
>> other keywords like `raise`, `in`, `def` etc.
>>
>> match expression:
>> match = True
>
>
> What kind of newcomers do you have that they even notice that, unless you 
> were to draw attention to it? I'm serious -- from the kind of questions I've 
> seen in user forums, most newcomers are having a hard enough time learning 
> more fundamental concepts and abstractions than the precise rules for 
> reserved words.
>

>From my experience of teaching a variety of languages, including SQL,
it's usually not something people have a problem with in toy examples
- but it becomes a major nuisance when they're trying to deal with a
problem and some keyword is getting in the way. SQL is *full* of
context-sensitive keywords, and every once in a while, someone uses a
non-reserved word as a column name, and everything works until they
run into some specific context where it doesn't work. (It's a bit
messier than in Python due to multiple abstraction layers eg ORMs, and
sometimes they deal with these issues and sometimes not; but it's
still that much harder to debug specifically _because_ things aren't
always reserved.)

Ultimately it comes down to the number of edge cases that people have
to learn, and how edgy those cases are. Python already has the
possibility to override builtins, so you can say "list = []" without
an error; context-sensitive keywords sit in a space between those and
fully-reserved words. It'll come down to specific words as to whether
it's inevitably going to be a problem down the track, or almost
certainly going to be fine.

BTW, is the PEG parser going to make it easier to hack on the language
syntax? If so, it'd be that much easier to experiment with these kinds
of ideas in a separate branch/fork, and quickly find out if there's
going to be any major impact. At the moment, editing the grammar is a
bit daunting - too many easy ways to mess it up.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OAF2EVT6ZGRFOHMVLZGETZSSG52CGYQV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Problem with instantiating a C extension class from another class

[Musbur]

On Mon, 6 Apr 2020 08:32:34 +1000
Nick Coghlan  wrote:

> Hi Musbur,
> 
> While python-dev is specifically for core development, the "specific
> interest group" mailing lists are for both change proposals and
> existing usage questions.
> 
> I've cc'ed capi-sig on this reply.

Hi Nick, thanks for forwarding the post. BTW, I've posted a complete,
self-contained example on stackoverflow (which also received zero
attention so far):

https://stackoverflow.com/questions/61025268/how-to-instantiate-a-custom-object-from-within-a-c-module
 

Regards
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HO47V7CEBKICENCVRYJB44MPLE7VCUVO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Problem with instantiating a C extension class from another class

[Rhodri James]

On 06/04/2020 20:41, Musbur wrote:

Hi Nick, thanks for forwarding the post. BTW, I've posted a complete,
self-contained example on stackoverflow (which also received zero
attention so far):


After that comment I'm not particularly inclined to solve your problem 
for you, so I'll just say that you should compare your module init 
function to the example one in the Embedding and Extending docs.


--
Rhodri James *-* Kynesim Ltd
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WZAOA3AAJFAMRAPTFGYL5LQZC5IYT4DI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Steve Holden]

On Mon, Apr 6, 2020 at 8:04 PM Guido van Rossum  wrote:

> On Mon, Apr 6, 2020 at 11:36 AM Steven D'Aprano 
> wrote:
>
>>
>> Personally, I would not like to have to explain to newcomers why `match`
>> is a keyword but you can still use it as a function or variable, but not
>> other keywords like `raise`, `in`, `def` etc.
>>
>> match expression:
>> match = True
>>
>
> What kind of newcomers do you have that they even notice that, unless you
> were to draw attention to it? I'm serious -- from the kind of questions
> I've seen in user forums, most newcomers are having a hard enough time
> learning more fundamental concepts and abstractions than the precise rules
> for reserved words.
>

Absolutely. Beginners can simply be told they are keywords. If they then
come across them in other contexts, hopefully there'll be a sensible
documentation page that a web search for " keyword" would lead to
an explanation that

"Some Python keywords can only ever be used with that meaning. Others can
be used with other meanings where the context makes it clear that the
keyword interpretation does not apply. You are recommended not to use such
keywords as names in your own programs. The feature was implemented to make
porting existing code to future versions of Python simpler."

The tutorial should contain a similar passage.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S2URQJVKERGOTKGJM5ZQX3EOCA2KNNS2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Steven D'Aprano]

On Mon, Apr 06, 2020 at 11:54:54AM -0700, Guido van Rossum wrote:

> (In an early version of the PEG parser, all keywords were
> context-sensitive, and there were only very few places in the grammar where
> this required us to insert negative lookaheads to make edge cases parse
> correctly. The rest was taken care by careful ordering of rules, e.g. the
> rule for `del_stmt` must be tried before the rule for `expression_stmt`
> since `del *x` would match the latter.)

I think, on first glance, I'd rather have all keywords context-sensitive 
than just some. But I haven't put a great deal of thought into that 
aspect of it, and I reserve the right to change my mind :-)


> > Personally, I would not like to have to explain to newcomers why `match`
> > is a keyword but you can still use it as a function or variable, but not
> > other keywords like `raise`, `in`, `def` etc.
> >
> > match expression:
> > match = True
> >
> 
> What kind of newcomers do you have that they even notice that, unless you
> were to draw attention to it?

It didn't take me 25 years to try using "of" and "if" for "output file" 
and "input file", so I guess my answer to your question is ordinary 
newcomers :-)

"Newcomers" doesn't just including beginners to programming, it can 
include people experienced in one or more other language coming to 
Python for the first time.

But if we're talking about complete beginners, the concept of what is 
and isn't a keyword is not always clear. Why is the first of these legal 
but not the second? Both words are highlighted in my editor:

str = "Hello world"
class = "wizard"

People are going to learn that `match` is a keyword, and then they are 
going to come across code using it as a variable or method, and while 
the context-sensitive rule might be obvious to us, it won't be obvious 
to them precisely because they are still learning the language rules.

I think that `match` would be an especially interesting case because I 
can easily see someone starting off with a variable `match`, that they 
handle in an `if` statement, and then as the code evolves they shift it 
to a `match` statement:

match match:

and not bother to refactor the name because they are familiar enough 
with is that the meaning is obvious.

On the other hand there are definitely a few keywords that collide with 
useful names. Apart from `if`, I have wanted to use these as variables, 
parameters or functions:

class, raise, in, from, except, while, lambda

(off the top of my head, there may be others). There's at least one 
place in the random module where a parameter is misspelled "lambd" 
because lambda is a keyword. So there is certainly something to be said 
for getting rid of keywords.

On the third hand, keywords don't just make it easier for the 
interpreter, they also make it easier for the human reader. You don't 
need to care about context, `except` is `except` wherever you see it. 
That makes it a dead-simple rule for anyone to learn, because there are 
no exceptions, pun intended.

(I guess inside strings and comments are exceptions, but they are 
well-understood and *simple* exceptions.)

I just can't help feeling at this point that while there are pros and 
cons to making things a keyword, having some keywords be context 
sensitive but not others is going to combine the worst of both and end 
up be confusing and awkward.


> I'm serious -- from the kind of questions
> I've seen in user forums, most newcomers are having a hard enough time
> learning more fundamental concepts and abstractions than the precise rules
> for reserved words.

That's because the precise rules for reserved words are dead-simple to 
learn. You can't use them anywhere except in the correct context. If we 
start adding exceptions to that, that reserved words are only sometimes 
reserved, I think that will make them harder to learn. If it's only some 
reserved words but not others, that's even harder because we have three 
classes of words:

* words that are never reserved
* words that are sometimes reserved, depending on what is around them
* words that are always reserved

I had thought that "no context-sensitive keywords" was a hard rule, so I 
was surprised that you are now re-considering it.


-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NNMJB7QIRAMROQU5XRM6YV7K4NXBGXBT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Guido van Rossum]

After 30 years am I not allowed to take new information into account and
consider a change of heart? :-)

On Mon, Apr 6, 2020 at 6:21 PM Steven D'Aprano  wrote:

> On Mon, Apr 06, 2020 at 11:54:54AM -0700, Guido van Rossum wrote:
>
> > (In an early version of the PEG parser, all keywords were
> > context-sensitive, and there were only very few places in the grammar
> where
> > this required us to insert negative lookaheads to make edge cases parse
> > correctly. The rest was taken care by careful ordering of rules, e.g. the
> > rule for `del_stmt` must be tried before the rule for `expression_stmt`
> > since `del *x` would match the latter.)
>
> I think, on first glance, I'd rather have all keywords context-sensitive
> than just some. But I haven't put a great deal of thought into that
> aspect of it, and I reserve the right to change my mind :-)
>
>
> > > Personally, I would not like to have to explain to newcomers why
> `match`
> > > is a keyword but you can still use it as a function or variable, but
> not
> > > other keywords like `raise`, `in`, `def` etc.
> > >
> > > match expression:
> > > match = True
> > >
> >
> > What kind of newcomers do you have that they even notice that, unless you
> > were to draw attention to it?
>
> It didn't take me 25 years to try using "of" and "if" for "output file"
> and "input file", so I guess my answer to your question is ordinary
> newcomers :-)
>
> "Newcomers" doesn't just including beginners to programming, it can
> include people experienced in one or more other language coming to
> Python for the first time.
>
> But if we're talking about complete beginners, the concept of what is
> and isn't a keyword is not always clear. Why is the first of these legal
> but not the second? Both words are highlighted in my editor:
>
> str = "Hello world"
> class = "wizard"
>
> People are going to learn that `match` is a keyword, and then they are
> going to come across code using it as a variable or method, and while
> the context-sensitive rule might be obvious to us, it won't be obvious
> to them precisely because they are still learning the language rules.
>
> I think that `match` would be an especially interesting case because I
> can easily see someone starting off with a variable `match`, that they
> handle in an `if` statement, and then as the code evolves they shift it
> to a `match` statement:
>
> match match:
>
> and not bother to refactor the name because they are familiar enough
> with is that the meaning is obvious.
>
> On the other hand there are definitely a few keywords that collide with
> useful names. Apart from `if`, I have wanted to use these as variables,
> parameters or functions:
>
> class, raise, in, from, except, while, lambda
>
> (off the top of my head, there may be others). There's at least one
> place in the random module where a parameter is misspelled "lambd"
> because lambda is a keyword. So there is certainly something to be said
> for getting rid of keywords.
>
> On the third hand, keywords don't just make it easier for the
> interpreter, they also make it easier for the human reader. You don't
> need to care about context, `except` is `except` wherever you see it.
> That makes it a dead-simple rule for anyone to learn, because there are
> no exceptions, pun intended.
>
> (I guess inside strings and comments are exceptions, but they are
> well-understood and *simple* exceptions.)
>
> I just can't help feeling at this point that while there are pros and
> cons to making things a keyword, having some keywords be context
> sensitive but not others is going to combine the worst of both and end
> up be confusing and awkward.
>
>
> > I'm serious -- from the kind of questions
> > I've seen in user forums, most newcomers are having a hard enough time
> > learning more fundamental concepts and abstractions than the precise
> rules
> > for reserved words.
>
> That's because the precise rules for reserved words are dead-simple to
> learn. You can't use them anywhere except in the correct context. If we
> start adding exceptions to that, that reserved words are only sometimes
> reserved, I think that will make them harder to learn. If it's only some
> reserved words but not others, that's even harder because we have three
> classes of words:
>
> * words that are never reserved
> * words that are sometimes reserved, depending on what is around them
> * words that are always reserved
>
> I had thought that "no context-sensitive keywords" was a hard rule, so I
> was surprised that you are now re-considering it.
>
>
> --
> Steven
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/NNMJB7QIRAMROQU5XRM6YV7K4NXBGXBT/
> Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Guido van Rossum]

I have a large PR (https://github.com/python/cpython/pull/18239, for PEP
585) that's failing in the Azures pipeline on Win32 and Win64 only. My
trusty assistant who has a Windows laptop couldn't reproduce the failure.
Can I buy a hint from someone? Steve?

The relevant failure output is:

==
ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest)
[WinDLL('_sqlite3.dll', winmode=0)]
--
Traceback (most recent call last):
  File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass
subprocess.check_output(
  File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "d:\a\1\s\lib\subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command
'['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *;
import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit
status 1.

--

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY25VK6R2FPGJ3QKAAVIK5EZKH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Ethan Smith]

(Trusty assistant reporting in) I should also note that further up the
output there is a FileNotFoundError for sqlite3.dll, perhaps it isn't built
or is in the wrong place?

Ethan


On Mon, Apr 6, 2020, 7:19 PM Guido van Rossum  wrote:

> I have a large PR (https://github.com/python/cpython/pull/18239, for PEP
> 585) that's failing in the Azures pipeline on Win32 and Win64 only. My
> trusty assistant who has a Windows laptop couldn't reproduce the failure.
> Can I buy a hint from someone? Steve?
>
> The relevant failure output is:
>
> ==
> ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest)
> [WinDLL('_sqlite3.dll', winmode=0)]
> --
> Traceback (most recent call last):
>   File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass
> subprocess.check_output(
>   File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>   File "d:\a\1\s\lib\subprocess.py", line 524, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command
> '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *;
> import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit
> status 1.
>
> --
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY25VK6R2FPGJ3QKAAVIK5EZKH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HEESZRLMHUAORUBHSFPKMO6MNAZZMVKE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Guido van Rossum]

Hm, looking over the diff from a different perspective I think I have
changed a few `__class_getitem__` methods that I shouldn't have. I'll see
if that fixes it later tonight.

On Mon, Apr 6, 2020 at 7:23 PM Ethan Smith  wrote:

> (Trusty assistant reporting in) I should also note that further up the
> output there is a FileNotFoundError for sqlite3.dll, perhaps it isn't built
> or is in the wrong place?
>
> Ethan
>
>
> On Mon, Apr 6, 2020, 7:19 PM Guido van Rossum  wrote:
>
>> I have a large PR (https://github.com/python/cpython/pull/18239, for PEP
>> 585) that's failing in the Azures pipeline on Win32 and Win64 only. My
>> trusty assistant who has a Windows laptop couldn't reproduce the failure.
>> Can I buy a hint from someone? Steve?
>>
>> The relevant failure output is:
>>
>> ==
>> ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest)
>> [WinDLL('_sqlite3.dll', winmode=0)]
>> --
>> Traceback (most recent call last):
>>   File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in
>> should_pass
>> subprocess.check_output(
>>   File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
>> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>>   File "d:\a\1\s\lib\subprocess.py", line 524, in run
>> raise CalledProcessError(retcode, process.args,
>> subprocess.CalledProcessError: Command
>> '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *;
>> import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit
>> status 1.
>>
>> --
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> *Pronouns: he/him **(why is my pronoun here?)*
>> 
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY25VK6R2FPGJ3QKAAVIK5EZKH/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4BWW7HWMFC2GJTFXZIMH5HTW3G7PFWXT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Need help with test_ctypes failing on Windows (test_load_dll_with_flags)

[Kyle Stanley]

Looking over the commit history for the PR (
https://github.com/python/cpython/pull/18239/commits), it looks like that
specific Azure Pipelines failure did not start occurring until
upstream/master was merged into the PR branch (
https://github.com/python/cpython/pull/18239/commits/13d3742fd897e1ea77060547de6d8445877e820e).
Therefore, I suspect that the failure is very likely unrelated to the PR;
instead either an intermittent failure that was merged into master recently
or a possible issue on Azure's end. For now, I'd suggest closing and
re-opening the PR again tomorrow to see if the failure still occurs.

Note: I'm also seeing the same exact failure occur in the following
separate CPython PRs that were opened recently:

https://github.com/python/cpython/pull/19403
https://github.com/python/cpython/pull/19402
https://github.com/python/cpython/pull/19399

Seeing as it was also occurring in entirely unrelated PRs, it seems to be
unrelated to the PEP 585 PR. I'm not seeing a BPO issue for this failure,
so I'll open a new one for it.

On Mon, Apr 6, 2020 at 10:24 PM Guido van Rossum  wrote:

> I have a large PR (https://github.com/python/cpython/pull/18239, for PEP
> 585) that's failing in the Azures pipeline on Win32 and Win64 only. My
> trusty assistant who has a Windows laptop couldn't reproduce the failure.
> Can I buy a hint from someone? Steve?
>
> The relevant failure output is:
>
> ==
> ERROR: test_load_dll_with_flags (ctypes.test.test_loading.LoaderTest)
> [WinDLL('_sqlite3.dll', winmode=0)]
> --
> Traceback (most recent call last):
>   File "d:\a\1\s\lib\ctypes\test\test_loading.py", line 140, in should_pass
> subprocess.check_output(
>   File "d:\a\1\s\lib\subprocess.py", line 420, in check_output
> return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
>   File "d:\a\1\s\lib\subprocess.py", line 524, in run
> raise CalledProcessError(retcode, process.args,
> subprocess.CalledProcessError: Command
> '['d:\\a\\1\\s\\PCbuild\\win32\\python.exe', '-c', "from ctypes import *;
> import nt;WinDLL('_sqlite3.dll', winmode=0)"]' returned non-zero exit
> status 1.
>
> --
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/2IMODEGY25VK6R2FPGJ3QKAAVIK5EZKH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HDLEJCO34DGCHADVEJD3ZWUH4LYJVHES/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Greg Ewing]

On 7/04/20 5:43 am, Guido van Rossum wrote:
The biggest difference is that the `|` operator is no longer 
symmetrical (since if you have alternatives `A | B`, and both match at 
some point in the input, PEG reports A, while the old generator would 
reject the grammar as being ambiguous.


I'm still inclined to think that allowing ambiguous grammars is
more of a bug than a feature. Is there some way the generator
could be made to at least warn if the grammar is genuinely
ambiguous (as opposed to just having overlapping first sets in
alternatives)?


We don't specify how other implementations must parse the language


And this is one of the reasons. If we use a PEG grammar as the
definition of the language, and aren't careful about ambiguities
when we add new syntax, we might accidentally end up with something
that can *only* be parsed with a PEG parser or something equally
powerful.


I'm sure there will be other ways to parse the same language.


That's certainly true now, but can you be sure it will remain
true if additions are made that rely on the full power of PEG?

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HQBD73ZNDXU5OM3PGK22GMXT5GRMORCX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Greg Ewing]

On 7/04/20 6:54 am, Guido van Rossum wrote:

I'm not sure that that was the conclusion. At the time the point was 
that we *wanted* all keywords to be reserved everywhere, an `as` was an 
ugly exception to that rule, which we got rid of as soon as we could -- 
not because it was a bad idea but because it violated a somewhat 
arbitrary rule.


I don't see it as an arbitrary rule, or at least no more arbitrary
than any other language rule. Given that the rule exists, it's the
exception that seems arbitrary. There's little justification for it
other than "we only thought of using it as a keyword later".

To reduce arbitrariness, we would either have to make *all*
keywords context-sensitive, or come up with some principled way
of deciding whether a given keyword should be reserved or not.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AK5SSVAXA2EIMU6VM2AIGXLV5KTU4IRP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617: New PEG parser for CPython

[Greg Ewing]

Another point in favour of always-reserved keywords is that they
make life a lot easier for syntax highlighters.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AAQRIAXLHSWYRQIV2DYHYSISMYRASO67/
Code of Conduct: http://python.org/psf/codeofconduct/