[Python-Dev] Re: Deprecating the "u" string literal prefix

[Serhiy Storchaka]

03.12.19 19:31, Guido van Rossum пише:
I think it’s too soon to worry about this. I don’t see a reason to 
harass people who maintain code based that were just recently migrated.


Yes, I also think that it is too early to deprecate it just now. Python 
2 is not completely dead yet. But I would happy to know some estimated 
terms.


BTW, BytesWarning and the -b option also are only needed for programs 
ported from Python 2. This great feature helps to catch some porting 
errors, but makes the pure Python 3 code more cumbersome and less 
efficient. For example, you can't use just `name[0] in ('.', b'.')`, you 
have to perform the `isinstance(name, bytes)` check first, and you can't 
use the same cache for string and bytes keys (you have to use 
`(type(key), key)` as a key).

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XHNXDQYOR7C45ESI54UX46AY3M75HNSN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Serhiy Storchaka]

03.12.19 23:44, Ned Batchelder пише:

On 12/3/19 12:16 PM, Serhiy Storchaka wrote:
The 'u" string literal prefix was removed in 3.0 and reintroduced in 
3.3 to help writing the code compatible with Python 2 and 3 [1]. After 
the dead of Python 2.7 we will remove some deprecated features kept 
for compatibility with 2.7. When we are going to deprecate and remove 
the "u" prefix?


[1] https://www.python.org/dev/peps/pep-0414/


Can you clarify this?: "we will remove some deprecated features kept for 
compatibility with 2.7"  Where is the discussion of the features being 
removed?


See the thread "Should we require all deprecations to have a removal 
version that we follow through on?".


There is no sense to keep old deprecated and useless features long after 
the death of Python 2. The question is only about the terms.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/D2VZNNMBHWRHUYV7MUMSTACTUARTLCET/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Serhiy Storchaka]

04.12.19 04:41, Ned Batchelder пише:

On 12/3/19 8:13 PM, Inada Naoki wrote:

I think it is too early to determine when to remove it.
Even only talking about it causes blaming war.


Has anyone yet given a reason to remove it? It will change working code 
into broken code. Why do that?


Why the "<>" operator and the "L" suffix was removed?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VMEH2RZJJTGTTOWUSXS526XT5U2FZEXU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

[David Cuthbert via Python-Dev]

Oddly, I did not get Mark's original e-mail, but am seeing replies here. 
Piggybacking off of James' email here...

On 03/12/2019 16:15, Mark Shannon wrote:
> Hi Everyone,
> 
> I am proposing a new PEP, still in draft form, to impose a limit of one 
> million on various aspects of Python programs, such as the lines of code 
> per module.

My main concern about this PEP is it doesn't specify the behavior when a given 
limit is exceeded. Whether you choose 10 lines or 10 billion lines as the rule, 
someone annoying (like me) is going to want to know what's going to happen if I 
break the rule.

Non-exhaustively, you could:
1. Say the behavior is implementation defined
2. Physically prohibit the limit from being exceeded (limited by 
construction/physics)
3. Generate a warning
4. Raise an exception early (during parse/analysis/bytecode generation)
5. Raise an exception during runtime

The first two will keep people who hate limits happy, but essentially give the 
limit no teeth. The last three are meaningful but will upset people when a 
previously valid program breaks.

1. The C and C++ standards are littered with limits (many of which you have to 
violate to create a real-world program) that ultimately specify that the 
resulting behavior is "implementation defined." Most general-purpose compilers 
have reasonable implementations (e.g. I can actually end my file without a 
newline and not have it call abort() or execve("/usr/bin/nethack"), behaviors 
both allowed by the C99 standard). You could go this route, but the end result 
isn't much better than not having done the PEP in the first place (beyond 
having an Ivory Tower to sit upon and taunt the unwashed masses, "I told you 
so," when you do decide to break their code).

Don't go this route unless absolutely necessary. Of course, the C/C++ standard 
isn't for an implementation; this PEP has the luxury of addressing a single 
implementation (CPython).


2. Many of Java's limits are by construction. You can't exceed 2**16 bytecode 
instructions for a method because they only allocated a uint16_t (u2 in the 
classfile spec) for the program counter in various places. (Bizarrely, the size 
of the method itself is stored as a uint32_t/u4.) I believe these limits are 
less useful because you'll never hit them in a running program; you simply 
can't create an invalid program. This would be like saying the size of Python 
bytecode is limited to the number of particles in the universe (~10**80). You 
don't have to specify the consequences because physics won't let you violate 
them.

This is more useful for documenting format limits, but probably doesn't achieve 
what you're trying to achieve.

3. Realistically, this is probably what you'd have to do in the first version 
for PEP adoption to get non-readers of python-dev@ ready, but, again, it 
doesn't achieve what you're setting out to do. We'd still accept programs that 
exceed these limits, and whatever optimizations that depend on these limits 
being in place wouldn't work.

Which brings us to the real meat, 4&5.

Some limits don't really distinguish between these cases. Exceeding the total 
bytecode size for a module, for example, would have to fail at bytecode 
generation time (ignoring truly irrational behavior like silently truncating 
the bytecode). But others aren't so cut-and-dry. For example, a module that is 
compliant except for a single function that contains too many local variables. 
Whether you do 4 or 5 isn't so obvious:

Pros of choosing 4 (exception at load):
* I'm alerted of errors early, before I start a 90-hour compute job, only to 
have it crash in the write_output() function.
* Don't have to keep a poisoned function that your optimizers have to special 
case.

Pros of choosing 5 (exception at runtime):
* If I never call that function (maybe it's something in a library I don't 
use), I don't get penalized.
* In line with other Python (mis-)behaviors, e.g. raising NameError() at 
runtime if you typo a variable name.

On Tue 12/03/19, 10:05 AM, "Rhodri James"  wrote:
On 03/12/2019 16:15, Mark Shannon wrote:
> Isn't this "640K ought to be enough for anybody" again?
> ---
> 
> The infamous 640K memory limit was a limit on machine usable resources.
> The proposed one million limit is a limit on human generated code.
> 
> While it is possible that generated code could exceed the limit,
> it is easy for a code generator to modify its output to conform.
> The author has hit the 64K limit in the JVM on at least two occasions 
> when generating Java code.
> The workarounds were relatively straightforward and
> probably wouldn't have been necessary with a limit of one million 
> bytecodes or lines of code.

I can absolutely guarantee that this will come back and bite you. 
Someone out there will be doing something more complicated than you 
think is

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Anders Munch]

Chris Angelico [mailto:ros...@gmail.com] wrote:
> The first one is already the case. PEP 414 reintroduced the u"..." literal 
> form, specifically 
> as a porting tool. Given that it has absolutely zero value in pure Py3 code 
> [...]

Challenge accepted :)  Here comes my https://xkcd.com/1172/ moment.

I'm using the u prefix to tag user interface strings for translation.  u"..." 
goes through i18n, "..." doesn't.  I have tools that extract and replace texts, 
identify new strings for translation etc. based on this.  
I was very happy when 3.3 reintroduced the prefix, because it allowed me to 
upgrade without losing my very efficient workflow.  Not to mention having to 
make 10.000 code changes to replace the prefix with something or other.

If the prefix goes, I'm not going to complain, I know my setup is fairly unique 
and I can't really demand that you keep vestigial syntax around just for that.  
But I'd still be sorry to see it go.

regards, Anders
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WCZDU2M2YK6A2UDP7OSN6P3VUOPMKCYI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Victor Stinner]

Le mer. 4 déc. 2019 à 06:41, Inada Naoki  a écrit :
> Currently, u-prefix is very widely used.  It shouldn't be removed anytime 
> soon.
> And I agree that we shouldn't raise DeprecationWarning for now.
>
> But how about 5, 10, and 20 years later?  No one knows it.
> Let's stop discussing it.  It can not be productive.
>
> Instead, we can do:
>
> * Don't recommend u-prefix except in Python 2&3 code.
> * Provide a tool to remove the u-prefix.

Another approach is to use linters rather than Python to actively
"deprecate" things.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TDZB4TAVEKGD35B56RHIC36M4L7TGFL6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Ned Batchelder]

On 12/4/19 3:11 AM, Serhiy Storchaka wrote:

04.12.19 04:41, Ned Batchelder пише:

On 12/3/19 8:13 PM, Inada Naoki wrote:

I think it is too early to determine when to remove it.
Even only talking about it causes blaming war.


Has anyone yet given a reason to remove it? It will change working 
code into broken code. Why do that?


Why the "<>" operator and the "L" suffix was removed?


Is this a serious question? Many things were removed in moving from 
Python 2 to Python 3. It was explicitly decided that 2->3 would contain 
breaking changes.  If you recall, this caused a large amount of 
controversy.  Why bring that on ourselves again?


Maybe I missed something. Python used to pride itself on keeping old 
code working. When hash randomization was introduced, it was decided to 
be off by default in Python 2 because even though people shouldn't have 
counted on the order of dicts, they were counting on it, and turning on 
hash randomization would break code. So there we decided to keep things 
insecure because otherwise code would break, even "wrong" code.


Now just because "it isn't needed anymore" we're going to break working 
code? I don't understand the shift in philosophy.


--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OGM2SHWH6ZJ434YVHGX2DXTG3HHSUZ6U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Victor Stinner]

You may want to have a look at the deferred PEP 501 -- General purpose
string interpolation:
https://www.python.org/dev/peps/pep-0501/

It was proposed while PEP 498 -- Literal String Interpolation
(f-string) was being discussed, which was accepted:
https://www.python.org/dev/peps/pep-0498/

PEP 501 proposed a solution for i18n.

Victor

Le mer. 4 déc. 2019 à 10:22, Anders Munch  a écrit :
>
> Chris Angelico [mailto:ros...@gmail.com] wrote:
> > The first one is already the case. PEP 414 reintroduced the u"..." literal 
> > form, specifically
> > as a porting tool. Given that it has absolutely zero value in pure Py3 code 
> > [...]
>
> Challenge accepted :)  Here comes my https://xkcd.com/1172/ moment.
>
> I'm using the u prefix to tag user interface strings for translation.  u"..." 
> goes through i18n, "..." doesn't.  I have tools that extract and replace 
> texts, identify new strings for translation etc. based on this.
> I was very happy when 3.3 reintroduced the prefix, because it allowed me to 
> upgrade without losing my very efficient workflow.  Not to mention having to 
> make 10.000 code changes to replace the prefix with something or other.
>
> If the prefix goes, I'm not going to complain, I know my setup is fairly 
> unique and I can't really demand that you keep vestigial syntax around just 
> for that.  But I'd still be sorry to see it go.
>
> regards, Anders
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/WCZDU2M2YK6A2UDP7OSN6P3VUOPMKCYI/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CEDNHEO3U5OLTOMVMULCY7VDKVAWTOQB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Multiple reference leaks in master

[Victor Stinner]

Le mer. 4 déc. 2019 à 01:45, Charalampos Stratakis
 a écrit :
> Also noting here that the reference leaks builds are triggered once a day so 
> many times it's not possible to find the commit that triggered a leak through 
> the web ui of buildbot. In this case it's good to also check the previous 
> builds of the same worker to figure out what changed.

Once a day is a tradeoff to not stress buildbot workers too often. In
my experience, it's quite easy to look at changes made the day before
and use bisect tools to identify the leak.

I'm using different tools:

* Refleak buildbots tell you which tests leak
* ./python -m test.bisect_cmd -R 3:3 test_xxx: identify one test
method which is enough to trigger the leak in test_xxx. This speedup a
lot following steps to check if a commit has the leak or not (git
bisect), since -R 3:3 makes tests 6x slower.
* git bisect helps a lot to identify which commit introduced the leak

Overall, it's rare than it takes me longer than 30 minutes to identify
the commit and the leaking tests and test methods. Sometimes, it's
done in under 10 minutes. It's just a few simple command and wait
until the computer works for me :-) (test.bisect_cmd that I wrote
helps me a lot!)

Pablo and me are opened issues to reference leak regressions. If
possible, we comment the issue introducing the regression, or we open
new issues if it's too hard to identify the leak.

The worst is when the leak is not a recent regression, but it was
hidden for whatever reasons and can now be reproduced in Python 6
months ago, if not older. But that's the least common case.

Note: test.bisect_cmd can also be used to identify the test method
responsible for a "altered environment" (ENV_CHANGED) if you add
--fail-env-changed option.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RTXIHFKXANABGUPMGFYDXDKLDWBAC6WM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Inada Naoki]

>
> Is this a serious question? Many things were removed in moving from
> Python 2 to Python 3.

We remove garbages not only between 2 and 3.
We regularly remove garbages.

https://docs.python.org/3/whatsnew/3.8.html#api-and-feature-removals
https://docs.python.org/3/whatsnew/3.7.html#api-and-feature-removals
https://docs.python.org/3/whatsnew/3.6.html#removed
https://docs.python.org/3/whatsnew/3.5.html#removed

We need to avoid major breakage.
But we accept small breakages on every minor release.
And u-prefix is major for now.


> Maybe I missed something. Python used to pride itself on keeping old
> code working. When hash randomization was introduced, it was decided to
> be off by default in Python 2 because even though people shouldn't have
> counted on the order of dicts, they were counting on it, and turning on
> hash randomization would break code. So there we decided to keep things
> insecure because otherwise code would break, even "wrong" code.

It is not a good example because we didn't release Python 2.8.
Hash randomization might be enabled by default if we released Python 2.8.

Regards,

-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WTEWGFFMJV7A6M5XEE7DFDF2QIQNVFUR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Macros instead of inline functions?

[Skip Montanaro]

As I wander around the code base, I keep seeing macro definitions in
the C code. For example, there are four CALL* macros defined in
Python/ast_opt.c which contain not entirely trivial bits of syntax.
That code is from 2017 (as compared to, say, Modules/audioop.c, which
first saw the light of day in 1992) I see the inline keyword used
unconditionally in many places.

I don't think stable code which uses macros should be changed (though
I see the INCREF/DECREF macros just call private inline functions, so
some conversion has clearly been done). Still, in new code, shouldn't
the use of macros for more than trivial use cases (constant defs,
simple one-liners) be discouraged at this point?

Skip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AVF6W3PMCAQK73NXOXHMHNW2KP7FJOIJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Victor Stinner]

INADA-san:
> We need to avoid major breakage.
> But we accept small breakages on every minor release.
> And u-prefix is major for now.

IMHO we need a metric to measure the risk of an incompatible change:
estimate the percentage of broken Python applications. For example,
run the test suite of the PyPI top 100 projects with the incompatible
change and see how many fails. That's the root of the PEP 608 --
Coordinated Python release:
https://www.python.org/dev/peps/pep-0608/

The "PEP 606 -- Python Compatibility Version" elaborates on
incompatible changes:
https://www.python.org/dev/peps/pep-0606/

These two PEPs have been rejected since the proposed concrete changes
which were not the best options.

It seems like the common denominator is the need for a tool to run the
testsuite of popular projects on a modified Python and/or the master
of Python, to measure how many projects are broken by recent Python
changes.

It's very different if an incompatible change break 1% or 90% of
Python projects.

Right now, we take decisions on assumptions without validating them.
We push changes and hope that we don't break too many projects. It's
rare that we revert incompatible changes that break too many projects,
because too few developers test their projects on the beta versions of
Python. Most developers and users only test final Python versions.

Victor
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XJ7PF3GIA7HCENHB7CM7BSGSAZ3SWAOT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

[Oscar Benjamin]

On Wed, 4 Dec 2019 at 05:41, Chris Angelico  wrote:
>
> On Wed, Dec 4, 2019 at 3:16 PM Steven D'Aprano  wrote:
> >
> > On Wed, Dec 04, 2019 at 01:47:53PM +1100, Chris Angelico wrote:
> >
> > > Integer sizes are a classic example of this. Is it acceptable to limit
> > > your integers to 2^16? 2^32? 2^64? Python made the choice to NOT limit
> > > its integers, and I haven't heard of any non-toy examples where an
> > > attacker causes you to evaluate 2**2**100 and eats up all your RAM.
> >
> > Does self-inflicted attacks count? I've managed to bring down a
> > production machine, causing data loss, *twice* by thoughtlessly running
> > something like 10**100**100 at the interactive interpreter. (Neither
> > case was a server, just a desktop machine, but the data loss was still
> > very real.)
>
> Hmm, and you couldn't Ctrl-C it? I tried and was able to.

I don't know if this is OS-dependent but I think that maybe there has
been an improvement in recent CPython (3.8?) for using Ctrl-C in these
cases. Certainly in the past I've seen situations where creating an
absurdly large integer cannot be interrupted before it is too late and
the system needs a hard reboot.

This is actually a common source of bugs in SymPy e.g.:
https://github.com/sympy/sympy/issues/17609#issuecomment-531327039

Those bugs in SymPy can be fixed in SymPy which is uniquely in a
position to be able to represent large exponent operations without
actually evaluating them in dense integer format. I would have thought
though that on the spectrum of Python usage SymPy would be very much
at the end that really wants to use enormous integers so the fact that
it needs to be limited there makes me wonder who does really want to
evaluate them.

Note that CPython's implementation of large integers is not as
optimised as gmp so I think that if someone was using Python for
incredibly large integer calculations then they would be well advised
not to use plain int in their calculations anyway (SymPy will try to
use gmpy/gmpy2 if available).

> There ARE a few situations where I'd rather get a simple and clean
> MemoryError than have it drive my system into the swapper, but there
> are at least as many situations where you'd rather be able to use
> virtual memory instead of being forced to manually break a job up. But
> even there, you can't enshrine a limit in the language definition,
> since the actual threshold depends on the running system. (And can be
> far better enforced externally, at least on a Unix-like OS.)

Another possibility is to have a configurable limit like the recursion
limit so that users can increase it when they want to. The default
limit can be something larger than most people would ever want but
small enough that on typical hardware you can't bork the system in a
single arithmetic operation. Then the default level and
configurability of the limit can be implementation defined.

--
Oscar
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/X7XHMUDZKJTNTGADVYEFBUWNC4NQ22SL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Batuhan Taskaya]

> Instead, we can do:
>
> * Don't recommend u-prefix except in Python 2&3 code.
> * Provide a tool to remove the u-prefix.

Great idea, +1 from me.

On Wed, Dec 4, 2019, 8:39 AM Inada Naoki  wrote:

> On Wed, Dec 4, 2019 at 11:49 AM Ned Batchelder 
> wrote:
> >
> > On 12/3/19 8:13 PM, Inada Naoki wrote:
> > > I think it is too early to determine when to remove it.
> > > Even only talking about it causes blaming war.
> >
> > Has anyone yet given a reason to remove it?
>
> Note that "never" is included in ”when".
> I didn't promoting removing it.  I just said let's stop discussion about
> it.
>
>
> > It will change working code
> > into broken code.  Why do that?
>
> Of course, everyone in this thread understands it.
> No one proposes remove it now.
>
> On the other hand, we remove some parts from Python language
> and the stdlib regularly to keep Python clean.
> All removal will break some code.  That's why we have a deprecation period.
>
> Currently, u-prefix is very widely used.  It shouldn't be removed anytime
> soon.
> And I agree that we shouldn't raise DeprecationWarning for now.
>
> But how about 5, 10, and 20 years later?  No one knows it.
> Let's stop discussing it.  It can not be productive.
>
> Instead, we can do:
>
> * Don't recommend u-prefix except in Python 2&3 code.
> * Provide a tool to remove the u-prefix.
>
> Regards,
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EVKCCO5KMOGEEFMSSY2PZRVGT2LDOB5K/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5IPCQPWHUDEVAHDBZTHUMXCY4ZGPJSXS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Macros instead of inline functions?

[Serhiy Storchaka]

04.12.19 14:08, Skip Montanaro пише:

As I wander around the code base, I keep seeing macro definitions in
the C code. For example, there are four CALL* macros defined in
Python/ast_opt.c which contain not entirely trivial bits of syntax.
That code is from 2017 (as compared to, say, Modules/audioop.c, which
first saw the light of day in 1992) I see the inline keyword used
unconditionally in many places.

I don't think stable code which uses macros should be changed (though
I see the INCREF/DECREF macros just call private inline functions, so
some conversion has clearly been done). Still, in new code, shouldn't
the use of macros for more than trivial use cases (constant defs,
simple one-liners) be discouraged at this point?


You can't goto from the inline function.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XAEQBSHBCLXHBIPPHUVQWNCMDYWLE4W2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

[Joao S. O. Bueno]

* The number of classes in a running interpreter.
* The number of live coroutines in a running interpreter.

These two can (and coroutines actually are always) dynamically generated -
and it is not hard to imagine
scenarios were 1 million for these would easily be beaten.

I don't know the data structures needed for those, but it would be much
saner to keep
both limited to 2**32 (due to reasons on artificial limiting the word
length as put by
others), or at least a much higher count.



On Tue, 3 Dec 2019 at 13:22, Mark Shannon  wrote:

> Hi Everyone,
>
> I am proposing a new PEP, still in draft form, to impose a limit of one
> million on various aspects of Python programs, such as the lines of code
> per module.
>
> Any thoughts or feedback?
>
> The PEP:
> https://github.com/markshannon/peps/blob/one-million/pep-100.rst
>
> Cheers,
> Mark.
>
>
> Full text
> *
>
> PEP: 100
> Title: The one million limit
> Author: Mark Shannon 
> Status: Active
> Type: Enhancement
> Content-Type: text/x-rst
> Created: 03-Dec-2019
> Post-History:
>
>
>
> Abstract
> 
> This PR proposes a limit of one million (1 000 000) for various aspects
> of Python code and its implementation.
>
> The Python language does not specify limits for many of its features.
> Not having any limit to these values seems to enhance programmer freedom,
> at least superficially, but in practice the CPython VM and other Python
> virtual
> machines have implicit limits or are forced to assume that the limits are
> astronomical, which is expensive.
>
> This PR lists a number of features which are to have a limit of one
> million.
> If a language feature is not listed but appears unlimited and must be
> finite,
> for physical reasons if no other, then a limit of one million should be
> assumed.
>
>
> Motivation
> ==
>
> There are many values that need to be represented in a virtual machine.
> If no limit is specified for these values, then the representation must
> either be inefficient or vulnerable to overflow.
> The CPython virtual machine represents values like line numbers,
> stack offsets and instruction offsets by 32 bit values. This is
> inefficient, and potentially unsafe.
>
> It is inefficient as actual values rarely need more than a dozen or so
> bits to represent them.
>
> It is unsafe as malicious or poorly generated code could cause values to
> exceed 2\ :sup:`32`.
>
> For example, line numbers are represented by 32 bit values internally.
> This is inefficient, given that modules almost never exceed a few
> thousand lines.
> Despite being inefficent, is is still vulnerable to overflow as
> it is easy for an attacker to created a module with billions of newline
> characters.
>
> Memory access is usually a limiting factor in the performance of modern
> CPUs.
> Better packing of data structures enhances locality and reduces memory
> bandwith,
> at a modest increase in ALU usage (for shifting and masking).
> Being able to safely store important values in 20 bits would allow
> memory savings
> in several data structures including, but not limited to:
>
> * Frame objects
> * Object headers
> * Code objects
>
> There is also the potential for a more efficient instruction format,
> speeding up interpreter dispatch.
>
> Rationale
> =
>
> Imposing a limit on values such as lines of code in a module, and the
> number of local variables,
> has significant advantages for ease of implementation and efficiency of
> virtual machines.
> If the limit is sufficiently large, there is no adverse effect on users
> of the language.
>
> By selecting a fixed but large limit for these values,
> it is possible to have both safety and efficiency whilst causing no
> inconvience to human programmers
> and only very rare problems for code generators.
>
> One million
> ---
>
> The Java Virtual Machine (JVM) [1]_ specifies a limit of 2\ :sup:`16`-1
> (65535) for many program
> elements similar to those covered here.
> This limit enables limited values to fit in 16 bits, which is a very
> efficient machine representation.
> However, this limit is quite easily exceeded in practice by code
> generators and
> the author is aware of existing Python code that already exceeds 2\
> :sup:`16` lines of code.
>
> A limit of one million fits into 20 bits which, although not as
> convenient for machine representation,
> is still reasonably compact. Three signed valuses in the range -1000_000
> to +1000_000 can fit into a 64 bit word.
> A limit of one million is small enough for efficiency advantages (only
> 20 bits),
> but large enough not to impact users (no one has ever written a module
> of one million lines).
>
> The value "one million" is very easy to remember.
>
> Isn't this "640K ought to be enough for anybody" again?
> ---
>
> The infamous 640K memory limit was a limit on machine usable resources.
> The proposed one million limit is a limit on human generated code.
>
> While it is 

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Thomas Wouters]

On Wed, Dec 4, 2019 at 1:26 PM Victor Stinner  wrote:

> INADA-san:
> > We need to avoid major breakage.
> > But we accept small breakages on every minor release.
> > And u-prefix is major for now.
>
> IMHO we need a metric to measure the risk of an incompatible change:
> estimate the percentage of broken Python applications. For example,
> run the test suite of the PyPI top 100 projects with the incompatible
> change and see how many fails. That's the root of the PEP 608 --
> Coordinated Python release:
> https://www.python.org/dev/peps/pep-0608/
>
> The "PEP 606 -- Python Compatibility Version" elaborates on
> incompatible changes:
> https://www.python.org/dev/peps/pep-0606/
>
> These two PEPs have been rejected since the proposed concrete changes
> which were not the best options.
>
> It seems like the common denominator is the need for a tool to run the
> testsuite of popular projects on a modified Python and/or the master
> of Python, to measure how many projects are broken by recent Python
> changes.
>
> It's very different if an incompatible change break 1% or 90% of
> Python projects.
>

Unfortunately there is a distinctive bias if you select popular projects,
or even packages from PyPI. There is a very large body of work that never
appears there, but is nonetheless used, useful and maintained lacklusterly
enough to pose a big problem for changes like these. Tutorials, examples in
documentation, random github repos, plugins for programs that embed Python,
etc. The latter also represents an example of cases where you can't just
decide to use an older version of Python to use something that wasn't
updated yet.

It's not just about how much code it will break, but also about how easy it
is to work around the problem or fix the code for users of it. In the
specific case of u"", it seems clear to me that we should silently
deprecate it, discourage its use, provide tools to get rid of them, and
wait another five or more years before considering a noisier deprecation.


>
> Right now, we take decisions on assumptions without validating them.
> We push changes and hope that we don't break too many projects. It's
> rare that we revert incompatible changes that break too many projects,
> because too few developers test their projects on the beta versions of
> Python. Most developers and users only test final Python versions.
>
> Victor
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XJ7PF3GIA7HCENHB7CM7BSGSAZ3SWAOT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Thomas Wouters 

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I52ZGAAJJFE6MLBRUMEENKYNQMTUTUYZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Anders Munch]

Victor Stinner [mailto:vstin...@python.org] wrote:
> You may want to have a look at the deferred PEP 501 -- General purpose string 
> interpolation:
> https://www.python.org/dev/peps/pep-0501/

I'm struggling to see what i-strings would do for i18n that str.format doesn't 
do better.

regards, Anders
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E6Q2QJU656WQNBE6L6BXOFTVNBBFY7DV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Guido van Rossum]

On Wed, Dec 4, 2019 at 3:35 AM Ned Batchelder  wrote:

> On 12/4/19 3:11 AM, Serhiy Storchaka wrote:
> > Why the "<>" operator and the "L" suffix was removed?
>
> Is this a serious question? Many things were removed in moving from
> Python 2 to Python 3. It was explicitly decided that 2->3 would contain
> breaking changes.  If you recall, this caused a large amount of
> controversy.  Why bring that on ourselves again?
>

It actually has a serious answer.

It is quite easy to write straddling code that avoids using <> and
L-suffixed integers -- in 2.7, there's basically no difference between ints
and longs except that they are different types and you have to write
isinstance(x, (int, long)) in some cases.

This is not the case for u"" strings. Without them, it was found quite hard
to write straddling code that uses unicode at all; the kludges around it
were ugly and inefficient. That's why we brought them back.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QXYBCQ4UY2FLKNFBDCDFWQD35DURPWD3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

[Guido van Rossum]

I am overwhelmed by this thread (and a few other things in real life) but
here are some thoughts.

1. It seems the PEP doesn't sufficiently show that there is a problem to be
solved. There are claims of inefficiency but these aren't substantiated and
I kind of doubt that e.g. representing line numbers in 32 bits rather than
20 bits is a problem.

2. I have handled complaints in the past about existing (accidental) limits
that caused problems for generated code. People occasionally generate
*really* wacky code (IIRC the most recent case was a team that was
generating Python code from machine learning models they had developed
using other software) and as long as it works I don't want to limit such
applications.

3. Is it easy to work around a limit? Even if it is, it may be a huge pain.
I've heard of a limit of 65,000 methods in Java on Android, and my
understanding was that it was actually a huge pain for both the toolchain
maintainers and app developers (IIRC the toolchain had special tricks to
work around it, but those required app developers to change their
workflow). Yes, 65,000 is a lot smaller than a million, but in a different
context the same concern applies.

4. What does Python currently do if you approach or exceed one of these
limits? I tried a simple experiment, eval(str(list(range(200, and
this completes in a few seconds, even though the source code is a single 16
Mbyte-long line.

5. On the other hand, the current parser cannot handle more than 100 nested
parentheses, and I've not heard complaints about this. I suspect the number
of nested indent levels is similarly constrained by the parser. The default
function call recursion limit is set to 1000 and bumping it significantly
risks segfaults. So clearly some limits exist and are apparently acceptable.

6. In Linux and other UNIX-y systems, there are many per-process or
per-user limits, and they can be tuned -- the user (using sudo) can change
many of those limits, the sysadmin can change the defaults within some
range, and sometimes the kernel can be recompiled with different absolute
limits (not an option for most users or even sysadmins). These limits are
also quite varied -- the maximum number of open file descriptors is
different than the maximum pipe buffer size. This is of course as it should
be -- the limits exist to protect the OS and other users/processes from
runaway code and intentional attacks on resources. (And yet, fork bombs
exist, and it's easy to fill up a filesystem...) I take from this that
limits are useful, may have to be overridable, and should have values that
make sense given the resource they guard.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6MYFHUJ52Z3ZGBS34R32QD72XZELEUMS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Macros instead of inline functions?

[Tim Peters]

[Skip Montanaro ]
> ...
> I don't think stable code which uses macros should be changed (though
> I see the INCREF/DECREF macros just call private inline functions, so
> some conversion has clearly been done). Still, in new code, shouldn't
> the use of macros for more than trivial use cases (constant defs,
> simple one-liners) be discouraged at this point?

Within reason, I think so.  Like C's old "register" declaration,
compilers will eventually evolve to make better decisions about what
should be done than humans generally do.

But there are macros that exist just to reduce repetitive, error-prone
typing, and others that set up control flow (like the trashcan macros.
or listobject.c's IFLT).  Which is the "within reason" part above.
There are still fine uses for macros in C.

It's just that faking inline functions is no longer one of them ;-)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KGU63DELU4YJTVFALCP55SXQIJ2QN5WJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Macros instead of inline functions?

[Skip Montanaro]

> > I don't think stable code which uses macros should be changed (though
> > I see the INCREF/DECREF macros just call private inline functions, so
> > some conversion has clearly been done). Still, in new code, shouldn't
> > the use of macros for more than trivial use cases (constant defs,
> > simple one-liners) be discouraged at this point?
>
> You can't goto from the inline function.

Thanks, that is true, and if needed would add another case where
macros are preferred over inline functions (as would use of the cpp
token pasting operator, ##). I see relatively few goto-using macros
spread across a number of source files, but the examples I highlighted
in Python/ast_opt.c use return, not goto. It seems they could easily
be crafted as inline functions which return 0 (forcing early return
from enclosing function) or 1 (equivalent to current fall through).

Still, I'm not terribly worried about existing usage, especially in
stable, well-tested code. I guess I'm more wondering if a preference
for inline functions shouldn't be mentioned in PEP 7 for future
authors.

Skip


Skip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H6XQFF7RKVSLDTE64SS6D352HDLDMVCC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Macros instead of inline functions?

[Victor Stinner]

Hi,

I'm working on cleaning up the C API, long rationale:
https://pythoncapi.readthedocs.io/

Macros are causing multiple issues:

* They often leak "implementation details" and so are incompatible
with a stable ABI
* They have multiple pitfalls:
https://gcc.gnu.org/onlinedocs/cpp/Macro-Pitfalls.html
* They require hacks like "do { ... } while (0)" to behave properly as
a statement, otherwise they can introduce bugs
* They don't "check" argument types and their return type is often
unclear, or worse depends on the arguments type
* Variable scoping can be an issue. For example, Py_SETREF() macro in
Python 3.8 uses a "_py_tmp" variable name, rather than being able to
use a more common name like "op" or "obj".
* etc.

Static inline functions behave as regular functions: a function call
is an expression, parameter types and return type are well defined,
scoping is well defined by the C language, etc.

For backward compatibility, I kept implicit cast to PyObject* using a
macro. Example:

static inline void _Py_INCREF(PyObject *op)
{
_Py_INC_REFTOTAL;
op->ob_refcnt++;
}

#define Py_INCREF(op) _Py_INCREF(_PyObject_CAST(op))

Static inline functions like _Py_INCREF() are still incompatible with
a stable ABI, but it's a tradeoff between correctness and
practicability.

I wrote an article about my work on the C API in Python 3.8:
https://vstinner.github.io/split-include-directory-python38.html

--

Macros are still used for some corner cases. For example, the
following macro opens a block with { :

#define Py_BEGIN_ALLOW_THREADS { \
PyThreadState *_save; \
_save = PyEval_SaveThread();

and this one closes the block:

#define Py_END_ALLOW_THREADSPyEval_RestoreThread(_save); \
 }

Note also the "_save" variable which has a local scope ;-)

Another example of special macro, the following macro uses "return"
which cannot be used like that using a function:

#define Py_RETURN_NONE return Py_INCREF(Py_None), Py_None

By the way, I would prefer "return Py_GetNone()", but that's a
different discussion ;-)

--

Macros are sometimes overriden by #define, or even #undef followed by #define.

In my own projects, I prefer "const int my_constant = 123;" rather
than "#define constant 123". In Python, I'm using the status quo:
"#define constant 123".

Victor

Le mer. 4 déc. 2019 à 13:19, Skip Montanaro  a écrit :
>
> As I wander around the code base, I keep seeing macro definitions in
> the C code. For example, there are four CALL* macros defined in
> Python/ast_opt.c which contain not entirely trivial bits of syntax.
> That code is from 2017 (as compared to, say, Modules/audioop.c, which
> first saw the light of day in 1992) I see the inline keyword used
> unconditionally in many places.
>
> I don't think stable code which uses macros should be changed (though
> I see the INCREF/DECREF macros just call private inline functions, so
> some conversion has clearly been done). Still, in new code, shouldn't
> the use of macros for more than trivial use cases (constant defs,
> simple one-liners) be discouraged at this point?
>
> Skip
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/AVF6W3PMCAQK73NXOXHMHNW2KP7FJOIJ/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LGORMVPTX42UIPNTG72K5BMGQTACNQ6X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Victor Stinner]

Le mer. 4 déc. 2019 à 14:49, Thomas Wouters  a écrit :
>> (...)
>> It's very different if an incompatible change break 1% or 90% of
>> Python projects.
>
> Unfortunately there is a distinctive bias if you select popular projects, or 
> even packages from PyPI. There is a very large body of work that never 
> appears there, but is nonetheless used, useful and maintained lacklusterly 
> enough to pose a big problem for changes like these. Tutorials, examples in 
> documentation, random github repos, plugins for programs that embed Python, 
> etc. The latter also represents an example of cases where you can't just 
> decide to use an older version of Python to use something that wasn't updated 
> yet.

My point is that currently, we have no data to take decisions. We can
only make assumptions. Having more data than nothing should help to
take smarter decisions ;-)

I know that there is closed source and unpublished projects. But if
20% (for example) of the most popular projects on PyPI are broken by
an incompatible change, it's not hard to extrapolate that *at least*
20% of unpublished will be broken at least. Usually, closed source and
unpublished projects are getting less attention and so are less up to
date than PyPI projects.

Even if you restrict the scope to PyPI: most PyPI top 100 modules are
the most common dependencies in projects. It's easy to extrapolate
that if 20% of these PyPI top 100 modules are broken, all applications
using at least one of these broken projects will be broken as well.

Another point of the PEP 608 is that there are not often resources to
fix the most popular dependencies on PyPI, it's likely better to
*revert* the incompatible change causing the issue. Again, to be able
to revert a change, we need the information that we broke something.
If a change goes through the final release, usually we prefer to
acknoledge that the "ship has sailed" and deal with it, rather than
reverting the annoying change.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DHFWE4UNXJHS6R2NJM32WLZJJVYPLKRH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Macros instead of inline functions?

[Serhiy Storchaka]

04.12.19 18:54, Skip Montanaro пише:

I don't think stable code which uses macros should be changed (though
I see the INCREF/DECREF macros just call private inline functions, so
some conversion has clearly been done). Still, in new code, shouldn't
the use of macros for more than trivial use cases (constant defs,
simple one-liners) be discouraged at this point?


You can't goto from the inline function.


Thanks, that is true, and if needed would add another case where
macros are preferred over inline functions (as would use of the cpp
token pasting operator, ##). I see relatively few goto-using macros
spread across a number of source files, but the examples I highlighted
in Python/ast_opt.c use return, not goto. It seems they could easily
be crafted as inline functions which return 0 (forcing early return
from enclosing function) or 1 (equivalent to current fall through).


In this case it is the same as a goto. Both affect the control flow of 
the caller.


In these files (symtable.c, compile.c, ast_opt.c, etc) there are 
sequences of calls for child nodes. Every call can return an error, so 
you need to check every call and return an error immediately after the 
call. With inline functions you would need to write


if (!VISIT(...)) {
return 0;
}
if (!VISIT(...)) {
return 0;
}
if (!VISIT(...)) {
return 0;
}

instead of just

VISIT(...);
VISIT(...);
VISIT(...);

This will increase the number of lines by two or three (according to the 
current PEP 7 recommendations) times. This will add a lot of boilerplate 
which distracts the attention from the logic. It will need to pass all 
arguments explicitly (macros allow to pass some arguments implicitly). 
This will add a lot of spots for introducing errors. If you will need to 
change some details, for example decrement state->recursion_depth before 
return you will need to change handreds lines instead of few sites (and 
guess how much bugs you will introduce).


Macros exist to get rid of the boilerplate. They are used not only in 
private files, this idiom is used in the public API. See the Py_VISIT 
macro. It passes implicit arguments and returns from the caller.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I2FAPRXQMKEDLZURJIMU7P274OLLPSGT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] PEP 608 and backwards compatibility [was: Deprecating the "u" string literal prefix]

[Ethan Furman]

On 12/04/2019 04:21 AM, Victor Stinner wrote:


IMHO we need a metric to measure the risk of an incompatible change:
estimate the percentage of broken Python applications. For example,
run the test suite of the PyPI top 100 projects with the incompatible
change and see how many fails. That's the root of the PEP 608 --
Coordinated Python release:
https://www.python.org/dev/peps/pep-0608/


The problem with 608 is that it was effectively a release blocker.  If somebody 
has the resources to put the testing and measuring into place I think it would 
be valuable data -- especially if those resources could then also notify the 
responsible parties that a breaking change was coming, or mass-media it, or 
something.

But Python should not be held hostage to others' unwillingness to fix/upgrade 
their own code base, nor their inability to fix/upgrade their dependencies -- 
moving to the latest Python is not a requirement, and if a company chooses to 
maintain aging resources to support aging software, that is their prerogative.

--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YJIO67CMNQOU2WJH7EWYZLU4G5GX6P6P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Mike Miller]

On 2019-12-03 18:41, Ned Batchelder wrote:
> Has anyone yet given a reason to remove it? It will change working code into
> broken code. Why do that?

I've heard a few complaints over the years about the number of combinations of 
string prefixes being a problem and a high barrier to new ones being accepted.
Especially since they are allowed to be capital letters as well, which I've 
never actually seen used in the real world.  Some code formatters already fix these.


At the appropriate time, hopefully we can deprecate u… and the capital prefixes 
too.

-Mike
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IQET4TCCPAD4FVQBV23D2UKJ2CV7XYX3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Macros instead of inline functions?

[Skip Montanaro]

This is my last post on this, at least as far as specific usage
instances are concerned. See my question about PEP 7 below. If that is
a discussion people think worthwhile, please start a new thread.

>  if (!VISIT(...)) {
>  return 0;
>  }
>  if (!VISIT(...)) {
>  return 0;
>  }
>  if (!VISIT(...)) {
>  return 0;
>  }
>
> instead of just
>
>  VISIT(...);
>  VISIT(...);
>  VISIT(...);

That seems easily solved with the VISIT-as-macro calling
_VISIT-as-inline function. That pattern exists elsewhere in the code,
in the INCREF/DECREF stuff, for example. The advantage with inline
functions (where you can use them) is that the debugger can work with
them. They are also more readable in my mind (no protective parens
required around expressions/arguments, no do { ... } while (0) }
business, no intrusive backslashification of every line) and they
probably play nicer with editors (think Emacs speedbar or tags file -
not sure if etags groks macros).

In any case, I was just somewhat surprised to see relatively new code
using macros where it seemed inline functions would have worked as
well or better. My more general question stands. Should PEP 7 say
something about the two? (Someone mentioned constants. Should they be
preferred over macros?)

Skip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IKT2BXHOIGPUE7Y7JNYW5M7QGYMPYZQB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP proposal to limit various aspects of a Python program to one million.

[Brett Cannon]

Steven D'Aprano wrote:
> I'm going to second Chris' comment about efficiency. The purposes of 
> this PEP (as I read it) are:
> (1) Security (less chance of code intentionally or accidentally 
> exceeding low-level machine limits that allow a security exploit);
> (2) Improved memory use;
> (3) And such improved memory use will lead to faster code.
> 1 and 2 seem to be obviously true, but like Chris, I think its a bit 
> much to expect us to take 3 on faith until after the PEP is accepted:
> > Reference Implementation
> > None, as yet. This will be implemented in CPython, once the PEP has 
> > been accepted.
> > I think the change you are asking for is akin to asking us to accept the 
> GILectomy on the promise that "trust me, it will speed up CPython, no 
> reference implementation is needed". It's a big thing to ask.
> Most of us a Python programmers, not experts on the minutia of the 
> interaction between C code and CPU cache locality and prediction etc. 

While I personally am okay putting in limits where necessary for security or if 
there's a clear performance win and the margin for people is high enough, I 
agree with Steven that the PEP currently doesn't lay that out yet beyond 
conjecture that this should have some benefit.

I can also see the argument that having a language standard versus it be 
something that's per-interpreter so one can know their code will run 
everywhere, but I would also argue most auto-generated code is probably not for 
a library that's going to approach any limit being proposed and it's more going 
to be app code which is going to be more tied to a specific interpreter. And in 
that case I would rather let the interpreters manage their own limits as their 
performance characteristics will be different thus these caps may bring them no 
benefit. And so artificial constraints in the name of interpreter consistency 
goes against "practicality beats purity".
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T2E4QK3BGBHEJJILOQIPP3CE3EYAOT7Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Brett Cannon]

Greg Ewing wrote:
> On 4/12/19 8:41 am, Christian Heimes wrote:
> > I'm strongly against removing it from Python 3 or
> > even
> > raising a deprecation warning.
> > I agree. I know there is a maintenance cost to keeping
> things like this around, but in this case it's pretty
> minor. We've probably already spent more time discussing
> it than it will cost to keep it for the rest of Python's
> lifetime.

I agree. Let's document it as unnecessary for new code, linters can point it 
out to people, and we can slate removal for Py4K.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/X5SJQXIWBH3T5KGTWK4ZX74L62MGYCIH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Ned Batchelder]

On 12/3/19 1:07 PM, Ethan Furman wrote:

On 12/03/2019 09:16 AM, Serhiy Storchaka wrote:

The 'u" string literal prefix was removed in 3.0 and reintroduced in 
3.3 to help writing the code compatible with Python 2 and 3 [1]. 
After the dead of Python 2.7 we will remove some deprecated features 
kept for compatibility with 2.7. When we are going to deprecate and 
remove the "u" prefix?


Can we gather all the 2.7 compatibility shims and remove them in 3.9?  
If not, I would say let's deprecate whatever needs deprecating and 
remove them in 3.10.  I don't think the u'' prefix needs the normal 
2-cyle deprecation since it was added specifically for 2/3 
cross-version code, and, as you say, "the dead of Python 2.7" has 
occurred (at least for us).


A few people in this thread have reasoned, "since we added it just to 
help people with 2.7, we can remove it now."  But that's exactly why it 
will be a significant cost to the community: lots of people made use of 
this useful needed thing. Removing the prefix will be a speed bump for 
people's use of Python, and will slow down people who want to upgrade to 
newer versions of Python.


--Ned.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6WX5GANERILH4TAMJAB2I55HRW5VTT6G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Barry Warsaw]

On Dec 4, 2019, at 06:02, Anders Munch  wrote:
> 
> Victor Stinner [mailto:vstin...@python.org] wrote:
>> You may want to have a look at the deferred PEP 501 -- General purpose 
>> string interpolation:
>> https://www.python.org/dev/peps/pep-0501/
> 
> I'm struggling to see what i-strings would do for i18n that str.format 
> doesn't do better.

str.format() really isn’t enough to do proper i18n; it’s actually a fairly 
complex topic.

GNU Mailman 2 was one of the first big Python applications to be fully 
internationalized, with active translators, web pages that could be displayed 
in multiple languages, and email messages that could have different parts 
translated to different languages.

I wrote or co-wrote several tools that work together to provide a full i18n 
solution.  This is not just a technical problem, but a social one as well, 
since you have to understand that human translators are often not programmers.  
Take a look at these, especially the latter PyPI package:

gettext - https://docs.python.org/3/library/gettext.html
pygettext - (although now standard xgettext can grok Python)
string.Template - https://docs.python.org/3/library/string.html#template-strings
flufl.i18n - https://flufli18n.readthedocs.io/en/latest/

I’m not convinced that PEP 501 would provide much benefit on the technical side.

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LI56JQCWE46POUDQKB5CZJPRHYOQ53N6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Macros instead of inline functions?

[Serhiy Storchaka]

04.12.19 22:12, Skip Montanaro пише:

This is my last post on this, at least as far as specific usage
instances are concerned. See my question about PEP 7 below. If that is
a discussion people think worthwhile, please start a new thread.


  if (!VISIT(...)) {
  return 0;
  }
  if (!VISIT(...)) {
  return 0;
  }
  if (!VISIT(...)) {
  return 0;
  }

instead of just

  VISIT(...);
  VISIT(...);
  VISIT(...);


That seems easily solved with the VISIT-as-macro calling
_VISIT-as-inline function.


I do not understant what problem do you want to solve. VISIT() in 
symtable.c is:


#define VISIT(ST, TYPE, V) \
if (!symtable_visit_ ## TYPE((ST), (V))) \
VISIT_QUIT((ST), 0);

It literally calls other (non-inlined) function, check its result and 
returns from the caller function (in VISIT_QUIT). I do not understand 
how inline function can help here.



In any case, I was just somewhat surprised to see relatively new code
using macros where it seemed inline functions would have worked as
well or better.


In these cases macros cannot be replaced by inline functions because 
inline function do not have access to variables of the caller and cannot 
affect the control flow of the caller.


The new code follows idioms used in the old code. There are also many 
other cases where macros save as from duplicating code and cannot be 
replaced with inline functions (C++ exceptions, templates, constructors 
and constant expressions could replace macros in some cases, but CPython 
is implemented on pure C, not even using all features of the recent 
standard). Don't afraid macros, they are the part of the language and 
pretty safe if use them properly.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3Z6MO5CPOEGYEVW7YOKPP5H5BGAR5RUX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] 3.8.1rc1, 3.7.6rc1, and 3.6.10rc1 cutoffs ahead

[Ned Deily]

It's that time for another reminder: code cutoffs for the last scheduled 
releases of 2019 are coming up soon.  They are a little earlier than usual to 
try to avoid the end-of-December rush of various holidays.  Łukasz will have 
more details forthcoming about **3.8.1rc1**.  For the **3.7 branch**, the 
cutoff for **3.7.6rc1** is scheduled for this coming Monday (2019-12-09) by the 
end of day AOE.  I have also now scheduled the cutoff for the next cumulative 
security source release for the **3.6 branch**, **3.6.10rc1**, for Monday as 
well.  Please review open issues and ensure that any that you believe need to 
be addressed in any of these releases are either resolved or marked as a 
**release blocker**.  Any assistance you can provide in helping resolve issues 
will be greatly appreciated.  As usual, following the rc1 cutoff, changes 
merged to the 3.7 branch will be released in **3.7.7** three months from now 
unless you mark the issue as a release blocker prior to **3.7.6 final**, 
planned for release on **2019-12-16**, and explain why the change should be 
cherry-picked into the final release.  And as always, any proposed changes for 
the 3.6 branch need to meet the criteria for security branches and require 
release manager approval to merge.

Thanks for your continued efforts!

3.8 release PEP: https://www.python.org/dev/peps/pep-0569/
3.7 release PEP: https://www.python.org/dev/peps/pep-0537/
3.6 release PEP: https://www.python.org/dev/peps/pep-0494/
Security branch criteria: 
https://devguide.python.org/devcycle/#security-branches

--
  Ned Deily
  n...@python.org -- []
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GYGZGI5VHSRX7U5JXS7MDRQ6LEQGFTLM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Deprecating the "u" string literal prefix

[Serhiy Storchaka]

04.12.19 16:02, Anders Munch пише:

Victor Stinner [mailto:vstin...@python.org] wrote:

You may want to have a look at the deferred PEP 501 -- General purpose string 
interpolation:
https://www.python.org/dev/peps/pep-0501/


I'm struggling to see what i-strings would do for i18n that str.format doesn't 
do better.


You would not need to repeat yourself.

_('{name} costs ${price:.2f}').format(name=name, price=price)

vs

i'{name} costs ${price:.2f}'

In addition, you need to translate the format for money (in some 
cultures you need not two digits after dot), and this is hard to do with 
str.format().

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/46VZ2Q6QNT444YM7MV66YV26IYNIQEQC/
Code of Conduct: http://python.org/psf/codeofconduct/