Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Serhiy Storchaka]

30.03.18 02:16, Steven D'Aprano пише:

Is there a down-side to 2b? It sounds like something you might end up
doing at a later date regardless of what you do now.


This complicate the compiler and the eval loop, especially in the case 
of nested substitutions in formats, like


f'{value:+{width:d}.{prec:d}f}'

The speed gain can be too small. The complex implementation of the 
opcode should be tightly integrated with the ceval loop, otherwise we 
can get a slow down, as in one of my experimental implementation of 
BUILD_STRING (https://bugs.python.org/issue27078#msg270505).


The exact benefit is unknown until this feature be implemented.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How can we use 48bit pointer safely?

[Serhiy Storchaka]

30.03.18 09:28, INADA Naoki пише:

As far as I know, most amd64 and arm64 systems use only 48bit address spaces.
(except [1])

[1] 
https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf

It means there are some chance to compact some data structures.
I point two examples below.

My question is; can we use 48bit pointer safely?
It depends on CPU architecture & OS memory map.
Maybe, configure option which is available on only (amd64, amd64) *
(Linux, Windows, macOS)?


If the size be the main problem, we could use these 8 bit for encoding 
the type and the size of the value for some types, and even encode the 
value itself for some types in other 48 bits. For example 48 bit 
integers, 0-, 1- and 2-character Unicode strings, ASCII strings up to 6 
characters (and even longer if use base 64 encodings for ASCII 
identifiers), singletons like None, True, False, Ellipsis, 
NotImplementes could be encoded in 64 bit word without using additional 
memory.


But this would significantly complicate and slow down the code.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Nick Coghlan]

On 30 March 2018 at 20:05, Serhiy Storchaka  wrote:
> 30.03.18 02:16, Steven D'Aprano пише:
>>
>> Is there a down-side to 2b? It sounds like something you might end up
>> doing at a later date regardless of what you do now.
>
>
> This complicate the compiler and the eval loop, especially in the case of
> nested substitutions in formats, like
>
> f'{value:+{width:d}.{prec:d}f}'

This point reminded me that there's still
https://www.python.org/dev/peps/pep-0536/ to consider as well (that's
the PEP about migrating f-strings to a fully nested expression grammar
rather than hijacking the existing string tokenisation code).

I *think* that's an orthogonal concern (since it relates to the
initial parsing and AST compilation phase, rather then the code
generation and execution phase), but it's worth keeping in mind.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Serhiy Storchaka]

29.03.18 18:06, Terry Reedy пише:

On 3/28/2018 11:27 AM, Serhiy Storchaka wrote:
The optimizer already changes semantic. Non-optimized "if a and True:" 
would call bool(a) twice, but optimized code calls it only once.


Perhaps Ref 3.3.1 object.__bool__ entry, after " should return False or 
True.", should say something like "Should not have side-effects, as 
redundant bool calls may be optimized away (bool(bool(ob)) should have 
the same result as bool(ob))."


Do you meant that it should be idempotent operation? Because 
bool(bool(ob)) always have the same result as bool(ob)) if bool(ob) 
returns True or False.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Steven D'Aprano]

On Fri, Mar 30, 2018 at 01:29:53PM +0300, Serhiy Storchaka wrote:
> 29.03.18 18:06, Terry Reedy пише:
> >On 3/28/2018 11:27 AM, Serhiy Storchaka wrote:
> >>The optimizer already changes semantic. Non-optimized "if a and True:" 
> >>would call bool(a) twice, but optimized code calls it only once.
> >
> >Perhaps Ref 3.3.1 object.__bool__ entry, after " should return False or 
> >True.", should say something like "Should not have side-effects, as 
> >redundant bool calls may be optimized away (bool(bool(ob)) should have 
> >the same result as bool(ob))."
> 
> Do you meant that it should be idempotent operation? Because 
> bool(bool(ob)) always have the same result as bool(ob)) if bool(ob) 
> returns True or False.

Assuming that bool is the built-in, and hasn't been shadowed or 
monkey-patched.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Nathaniel Smith]

On Fri, Mar 30, 2018 at 3:29 AM, Serhiy Storchaka  wrote:
> 29.03.18 18:06, Terry Reedy пише:
>>
>> On 3/28/2018 11:27 AM, Serhiy Storchaka wrote:
>>>
>>> The optimizer already changes semantic. Non-optimized "if a and True:"
>>> would call bool(a) twice, but optimized code calls it only once.
>>
>>
>> Perhaps Ref 3.3.1 object.__bool__ entry, after " should return False or
>> True.", should say something like "Should not have side-effects, as
>> redundant bool calls may be optimized away (bool(bool(ob)) should have the
>> same result as bool(ob))."
>
>
> Do you meant that it should be idempotent operation? Because bool(bool(ob))
> always have the same result as bool(ob)) if bool(ob) returns True or False.

And bool(obj) does always return True or False; if you define a
__bool__ method that returns something else then bool rejects it and
raises TypeError. So bool(bool(obj)) is already indistinguishable from
bool(obj).

However, the naive implementation of 'if a and True:' doesn't call
bool(bool(a)), it calls bool(a) twice, and this *is* distinguishable
by user code, at least in principle.

If we want to change the language spec, I guess it would be with text
like: "if bool(obj) would be called twice in immediate succession,
with no other code in between, then the interpreter may assume that
both calls would return the same value and elide one of them".

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How can we use 48bit pointer safely?

[Antoine Pitrou]

On Fri, 30 Mar 2018 15:28:47 +0900
INADA Naoki  wrote:
> Hi,
> 
> As far as I know, most amd64 and arm64 systems use only 48bit address spaces.
> (except [1])
> 
> [1] 
> https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf
> 
> It means there are some chance to compact some data structures.

As that paper shows, effective virtual address width tends to increase
over time to accomodate growing needs.  Bigger systems like IBM POWER
sytems may already have larger virtual address spaces.  So we can't
safely assume that bits 48-63 are available for us.

Another issue is the cost of the associated bit-twiddling.  It will all
depend how often it needs to be done.  Note that pointers can be
"negative", i.e. some of them will have all 1s in their upper bits, and
you need to reproduce that when reconstituting the original pointer.

A safer alternative is to use the *lower* bits of pointers.  The bottom
3 bits are always available for storing ancillary information, since
typically all heap-allocated data will be at least 8-bytes aligned
(probably 16-bytes aligned on 64-bit processes). However, you also get
less bits :-)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Nick Coghlan]

On 30 March 2018 at 21:16, Nathaniel Smith  wrote:
> On Fri, Mar 30, 2018 at 3:29 AM, Serhiy Storchaka  wrote:
>> 29.03.18 18:06, Terry Reedy пише:
>>>
>>> On 3/28/2018 11:27 AM, Serhiy Storchaka wrote:

 The optimizer already changes semantic. Non-optimized "if a and True:"
 would call bool(a) twice, but optimized code calls it only once.
>>>
>>>
>>> Perhaps Ref 3.3.1 object.__bool__ entry, after " should return False or
>>> True.", should say something like "Should not have side-effects, as
>>> redundant bool calls may be optimized away (bool(bool(ob)) should have the
>>> same result as bool(ob))."
>>
>>
>> Do you meant that it should be idempotent operation? Because bool(bool(ob))
>> always have the same result as bool(ob)) if bool(ob) returns True or False.
>
> And bool(obj) does always return True or False; if you define a
> __bool__ method that returns something else then bool rejects it and
> raises TypeError. So bool(bool(obj)) is already indistinguishable from
> bool(obj).
>
> However, the naive implementation of 'if a and True:' doesn't call
> bool(bool(a)), it calls bool(a) twice, and this *is* distinguishable
> by user code, at least in principle.

For example:

>>> class FlipFlop:
... _state = False
... def __bool__(self):
... result = self._state
... self._state = not result
... return result
...
>>> toggle = FlipFlop()
>>> bool(toggle)
False
>>> bool(toggle)
True
>>> bool(toggle)
False
>>> bool(toggle) and bool(toggle)
False
>>> toggle and toggle
<__main__.FlipFlop object at 0x7f35293604e0>
>>> bool(toggle and toggle)
True

So the general principle is that __bool__ implementations shouldn't do
anything that will change the result of the next call to __bool__, or
else weirdness is going to result.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How can we use 48bit pointer safely?

[Ronald Oussoren]

On Mar 30, 2018, at 08:31 AM, INADA Naoki  wrote:

Hi,

As far as I know, most amd64 and arm64 systems use only 48bit address spaces.
(except [1])

[1] 
https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf

It means there are some chance to compact some data structures.
I point two examples below.

My question is; can we use 48bit pointer safely?

Not really, at least some CPUs can also address more memory than that. See 
 which talks about Linux support for 57-bit 
virtual addresses and 52-bit physical addresses. 

Ronald
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How can we use 48bit pointer safely?

[Joao S. O. Bueno]

Not only that, but afaik  Linux could simply raise that 57bit virtual
to 64bit virtual without previous
warning on any version change.

On 30 March 2018 at 08:55, Ronald Oussoren  wrote:
>
>
> On Mar 30, 2018, at 08:31 AM, INADA Naoki  wrote:
>
> Hi,
>
> As far as I know, most amd64 and arm64 systems use only 48bit address
> spaces.
> (except [1])
>
> [1]
> https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf
>
> It means there are some chance to compact some data structures.
> I point two examples below.
>
> My question is; can we use 48bit pointer safely?
>
>
> Not really, at least some CPUs can also address more memory than that. See
>  which talks about Linux support for
> 57-bit virtual addresses and 52-bit physical addresses.
>
> Ronald
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How can we use 48bit pointer safely?

[Ronald Oussoren]

On Mar 30, 2018, at 03:11 PM, "Joao S. O. Bueno"  wrote:

Not only that, but afaik Linux could simply raise that 57bit virtual
to 64bit virtual without previous
warning on any version change.

The change from 48-bit to 57-bit virtual addresses was not done without any 
warning because that would have broken too much code (IIRC due to at least some 
JS environments assuming 48bit pointers).

Ronald ___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How can we use 48bit pointer safely?

[Ronald Oussoren]

On Mar 30, 2018, at 01:40 PM, Antoine Pitrou  wrote:


A safer alternative is to use the *lower* bits of pointers. The bottom
3 bits are always available for storing ancillary information, since
typically all heap-allocated data will be at least 8-bytes aligned
(probably 16-bytes aligned on 64-bit processes). However, you also get
less bits :-)

The lower bits are more interesting to use. I'm still hoping to find some time 
to experiment with tagged pointers some day, that could be interesting w.r.t. 
performance and memory use (at the cost of being ABI incompatible). 

Ronald
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Nuking wstr [Re: How can we use 48bit pointer safely?]

[Antoine Pitrou]

On Fri, 30 Mar 2018 15:28:47 +0900
INADA Naoki  wrote:
> 
> # Possible optimizations by 48bit pointer
> 
> ## PyASCIIObject
> 
> [snip]
> unsigned int ready:1;
> /* Padding to ensure that PyUnicode_DATA() is always aligned to
>4 bytes (see issue #19537 on m68k). */
> unsigned int :24;
> } state;
> wchar_t *wstr;  /* wchar_t representation (null-terminated) */
> } PyASCIIObject;
> 
> Currently, state is 8bit + 24bit padding.  I think we can pack state and wstr
> in 64bit.

We could also simply nuke wstr.  I frankly don't think it's very
important.  It's only used when calling system functions taking a
wchar_t argument, as an « optimization ».  I'd be willing to
guess that modern workloads aren't bottlenecked by the cost overhead of
those system functions...

Of course, the question is whether all this matters.  Is it important
to save 8 bytes on each unicode object?  Only testing would tell.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Terry Reedy]

On 3/30/2018 6:29 AM, Serhiy Storchaka wrote:

29.03.18 18:06, Terry Reedy пише:

On 3/28/2018 11:27 AM, Serhiy Storchaka wrote:
The optimizer already changes semantic. Non-optimized "if a and 
True:" would call bool(a) twice, but optimized code calls it only once.


Perhaps Ref 3.3.1 object.__bool__ entry, after " should return False 
or True.", should say something like "Should not have side-effects, as 
redundant bool calls may be optimized away (bool(bool(ob)) should have 
the same result as bool(ob))."


Do you meant that it should be idempotent operation? Because 
bool(bool(ob)) always have the same result as bool(ob)) if bool(ob) 
returns True or False.


That is what the parenthetical comment says, but it is not right in the 
context and should be deleted.


For the "if a and True:" example,
'redundant bool calls may be optimized away.'
might be better written as
'duplicate implied __bool__ calls may be avoided.'

What I am trying to say is that *we* define the intended behavior of 
special methods, and we should define what an implementation may 
actually expect.  The current optimizer expects __bool__ to have no side 
effects, at least none that it need respect.


Having said what __bool__ should do, we can also say what it should not 
do to avoid possible surprises -- at least in production code, as 
opposed to 'testing' code like the examples in this thread.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2018-03-23 - 2018-03-30)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open6543 (+14)
  closed 38393 (+46)
  total  44936 (+60)

Open issues with patches: 2548 


Issues opened (42)
==

#31793: Allow to specialize smart quotes in documentation translations
https://bugs.python.org/issue31793  reopened by mdk

#33128: PathFinder is twice on sys.meta_path
https://bugs.python.org/issue33128  opened by htgoebel

#33129: Add kwarg-only option to dataclass
https://bugs.python.org/issue33129  opened by alan_du

#33130: functools.reduce signature/docstring discordance
https://bugs.python.org/issue33130  opened by vreuter

#33131: Upgrade to pip 10 for Python 3.7
https://bugs.python.org/issue33131  opened by ncoghlan

#33132: Possible refcount issues in the compiler
https://bugs.python.org/issue33132  opened by serhiy.storchaka

#33133: Don't return implicit optional types by get_type_hints
https://bugs.python.org/issue33133  opened by levkivskyi

#33135: Define field prefixes for the various config structs
https://bugs.python.org/issue33135  opened by ncoghlan

#33136: Harden ssl module against CVE-2018-8970
https://bugs.python.org/issue33136  opened by christian.heimes

#33137: line traces may be missed on backward jumps when instrumented 
https://bugs.python.org/issue33137  opened by xdegaye

#33138: Improve standard error for uncopyable types
https://bugs.python.org/issue33138  opened by serhiy.storchaka

#33139: Bdb doesn't find instruction in linecache after pdb.set_trace(
https://bugs.python.org/issue33139  opened by prounce

#33140: shutil.chown on Windows
https://bugs.python.org/issue33140  opened by eryksun

#33144: random._randbelow optimization
https://bugs.python.org/issue33144  opened by wolma

#33146: contextlib.suppress should capture exception for inspection an
https://bugs.python.org/issue33146  opened by jason.coombs

#33147: Update references for RFC 3548 to RFC 4648
https://bugs.python.org/issue33147  opened by paulehoffman

#33148: RuntimeError('Event loop is closed') after cancelling getaddri
https://bugs.python.org/issue33148  opened by vitaly.krug

#33150: Signature error for methods of class configparser.Interpolatio
https://bugs.python.org/issue33150  opened by acue

#33152: Use list comprehension in timeit module instead of loop with a
https://bugs.python.org/issue33152  opened by Windson Yang

#33153: interpreter crash when multiplying large tuples
https://bugs.python.org/issue33153  opened by imz

#33154: subprocess.Popen ResourceWarning should have activation-deacti
https://bugs.python.org/issue33154  opened by acue

#33155: Use super().method instead in Logging
https://bugs.python.org/issue33155  opened by madsjensen

#33158: Add fileobj property to csv reader and writer objects
https://bugs.python.org/issue33158  opened by samwyse

#33159: Implement PEP 473
https://bugs.python.org/issue33159  opened by skreft

#33161: Refactor of pathlib's _WindowsBehavior.gethomedir
https://bugs.python.org/issue33161  opened by onlined

#33162: TimedRotatingFileHandler in logging module
https://bugs.python.org/issue33162  opened by Nikunj jain

#33164: Blake 2 module update
https://bugs.python.org/issue33164  opened by David Carlier

#33165: Add stacklevel parameter to logging APIs
https://bugs.python.org/issue33165  opened by ncoghlan

#33166: os.cpu_count() returns wrong number of processors on specific 
https://bugs.python.org/issue33166  opened by yanirh

#33167: RFC Documentation Updates to urllib.parse.rst
https://bugs.python.org/issue33167  opened by agnosticdev

#33168: distutils build/build_ext and --debug
https://bugs.python.org/issue33168  opened by lazka

#33169: importlib.invalidate_caches() doesn't clear all caches
https://bugs.python.org/issue33169  opened by gvanrossum

#33171: multiprocessing won't utilize all of platform resources
https://bugs.python.org/issue33171  opened by yanirh

#33173: GzipFile's .seekable() returns True even if underlying buffer 
https://bugs.python.org/issue33173  opened by Walt Askew

#33174: error building the _sha3 module with Intel 2018 compilers
https://bugs.python.org/issue33174  opened by wscullin

#33176: Allow memoryview.cast(readonly=...)
https://bugs.python.org/issue33176  opened by pitrou

#33178: Add support for BigEndianUnion and LittleEndianUnion in ctypes
https://bugs.python.org/issue33178  opened by emezh

#33179: Investigate using a context variable for zero-arg super initia
https://bugs.python.org/issue33179  opened by ncoghlan

#33180: Flag for unusable sys.executable
https://bugs.python.org/issue33180  opened by steve.dower

#33181: SimpleHTTPRequestHandler shouldn't redirect to directories wit
https://bugs.python.org/issue33181  opened by oulenz

#33184: Update OpenSSL to 1.1.0h / 1.0.2o
https://bugs.python.org/issue33184  opened by ned.deily

#33185: Python 3.7.0b3 

Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Chris Jerdonek]

On Fri, Mar 30, 2018 at 4:41 AM, Nick Coghlan  wrote:
> On 30 March 2018 at 21:16, Nathaniel Smith  wrote:
>> And bool(obj) does always return True or False; if you define a
>> __bool__ method that returns something else then bool rejects it and
>> raises TypeError. So bool(bool(obj)) is already indistinguishable from
>> bool(obj).
>>
>> However, the naive implementation of 'if a and True:' doesn't call
>> bool(bool(a)), it calls bool(a) twice, and this *is* distinguishable
>> by user code, at least in principle.
>
> For example:
>
> >>> class FlipFlop:
> ... _state = False
> ... def __bool__(self):
> ... result = self._state
> ... self._state = not result
> ... return result
> ...
> >>> toggle = FlipFlop()
> >>> bool(toggle)
> False
> >>> bool(toggle)
> True
> >>> bool(toggle)
> False
> >>> bool(toggle) and bool(toggle)
> False
> >>> toggle and toggle
> <__main__.FlipFlop object at 0x7f35293604e0>
> >>> bool(toggle and toggle)
> True
>
> So the general principle is that __bool__ implementations shouldn't do
> anything that will change the result of the next call to __bool__, or
> else weirdness is going to result.

I don't think this way of stating it is general enough. For example,
you could have a nondeterministic implementation of __bool__ that
doesn't itself carry any state (e.g. flipping the result with some
probability), but the next call could nevertheless still return a
different result. So I think Nathaniel's way of stating it is probably
better:

> If we want to change the language spec, I guess it would be with text
> like: "if bool(obj) would be called twice in immediate succession,
> with no other code in between, then the interpreter may assume that
> both calls would return the same value and elide one of them".

--Chris
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Nuking wstr [Re: How can we use 48bit pointer safely?]

[Serhiy Storchaka]

30.03.18 16:54, Antoine Pitrou пише:

We could also simply nuke wstr.  I frankly don't think it's very
important.  It's only used when calling system functions taking a
wchar_t argument, as an « optimization ».  I'd be willing to
guess that modern workloads aren't bottlenecked by the cost overhead of
those system functions...


This is possible only after removing all Py_UNICODE related C API. It is 
deprecated since 3.3, but only in the documentation, and should stay to 
the EOL of 2.7. Only in 3.7 most of these functions started emitting 
deprecation warnings at compile time (GCC-only). [1] It would be good to 
make them emitted in other compilers too. In future versions we could 
make them emitting user-visible runtime deprecation warnings, and 
finally make them always failing after 2020.


[1] https://bugs.python.org/issue19569

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Subtle difference between f-strings and str.format()

[Tim Peters]

[Steven D'Aprano ]
> ...
> Is there a down-side to 2b? It sounds like something you might end up
> doing at a later date regardless of what you do now.

There are always downsides ;-)

As Serhiy noted later, the idea that "it's faster" is an educated
guess - you can't know before it's implemented.  Changes to the very
complicated eval loop often have not only surprising speed
consequences on one platform, but even consequences in opposite
directions across platforms.  Not necessarily in the part you directly
changed, either.  Optimizing C compilers just can't reliably guess
what's most important in such a massive pile of test-and-branch laden
code.  Indeed, which paths through the eval loop _are_ most important
depend on the Python program you're running at the time (which is,
e.g., why "profile-guided optimization" was invented).

So there's an ocean of potential complications there, and wading
through those has opportunity cost too:  Serhiy is a very productive
contributor, but time he spends on this is time he won't be spending
on other things of potentially greater value.  That's all up to him,
though.

I'm not keen on changing the behavior of f-strings regardless (2a or
2b).  While their implementation details aren't documented, they were
intentional, and follow the pattern increasingly large parts of the
language and std library adopted after the iterator protocol was
introduced:  compute intermediate results as they're needed, not all
in advance.  That's proved to have many advantages.

It's certainly possible to write custom purely functional (no side
effects) __format__ methods such that memory use in an f-string
remains bounded under the current implementation, but can grow without
bound if all __format__ arguments need to be evaluated before any
formatting begins.  It's akin to the difference between iterating over
range() and xrange() in Python 2.

I don't know that there's any real f-string code out there _relying_
on that - but don't know that there isn't either.  It's more plausible
to me than that there are non-functional real __format__ methods.

I'd be happiest if no behaviors changed in anything.  Then the only
downsides to optimizing are code bloat, code churn, new bugs, subtler
corner cases, less predictable behavior for end users, and increased
implementation complexity forever after ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com