Re: [Python-Dev] PEP 471 -- os.scandir() function -- a better and faster directory iterator

[MRAB]

On 2014-06-26 23:59, Ben Hoyt wrote:

Hi Python dev folks,

I've written a PEP proposing a specific os.scandir() API for a
directory iterator that returns the stat-like info from the OS, the
main advantage of which is to speed up os.walk() and similar
operations between 4-20x, depending on your OS and file system. Full
details, background info, and context links are in the PEP, which
Victor Stinner has uploaded at the following URL, and I've also
copied inline below.

http://legacy.python.org/dev/peps/pep-0471/

Would love feedback on the PEP, but also of course on the proposal
itself.


[snip]
Personally, I'd prefer the name 'iterdir' because it emphasises that
it's an iterator.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 471 -- os.scandir() function -- a better and faster directory iterator

[MRAB]

On 2014-06-27 02:37, Ben Hoyt wrote:

I don't mind iterdir() and would take it :-), but I'll just say why I
chose the name scandir() -- though it wasn't my suggestion originally:

iterdir() sounds like just an iterator version of listdir(), kinda
like keys() and iterkeys() in Python 2. Whereas in actual fact the
return values are quite different (DirEntry objects vs strings), and
so the name change reflects that difference a little.


[snip]

The re module has 'findall', which returns a list of strings, and
'finditer', which returns an iterator that yields match objects, so
there's a precedent. :-)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] LZO bug

[MRAB]

Is this something that we need to worry about?

Raising Lazarus - The 20 Year Old Bug that Went to Mars
http://blog.securitymouse.com/2014/06/raising-lazarus-20-year-old-bug-that.html

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] == on object tests identity in 3.x

[MRAB]

On 2014-07-08 17:57, Steven D'Aprano wrote:
[snip]


In particular, reflexivity for NANs was dropped for a number of reasons,
some stronger than others:

- One of the weaker reasons for NAN non-reflexivity is that it preserved
   the identity x == y <=> x - y == 0. Although that is the cornerstone
   of real arithmetic, it's violated by IEEE-754 INFs, so violating it
   for NANs is not a big deal either.

- Dropping reflexivity preserves the useful property that NANs compare
   unequal to everything.

- Practicality beats purity: dropping reflexivity allowed programmers
   to identify NANs without waiting years or decades for programming
   languages to implement isnan() functions. E.g. before Python had
   math.isnan(), I made my own:

   def isnan(x):
   return isinstance(x, float) and x != x

- Keeping reflexivity for NANs would have implied some pretty nasty
   things, e.g. if log(-3) == log(-5), then -3 == -5.


The log of a negative number is a complex number.


Basically, and I realise that many people disagree with their decision
(notably Bertrand Meyer of Eiffel fame, and our own Mark Dickenson), the
IEEE-754 committee led by William Kahan decided that the problems caused
by having NANs compare unequal to themselves were much less than the
problems that would have been caused without it.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Updates to PEP 471, the os.scandir() proposal

[MRAB]

On 2014-07-09 23:50, Ethan Furman wrote:

On 07/09/2014 02:33 PM, Ben Hoyt wrote:


On a system which did not supply is_dir automatically I would write that as:

   for entry in os.scandir(path):
   if ignore_entry(entry.name):
   continue
   if os.path.isdir(entry.full_name):
   # do something interesting

Not hard to read or understand, no time wasted in unnecessary lstat calls.


No, but how do you know whether you're on "a system which did not
supply is_dir automatically"? The above is not cross-platform, or at
least, not efficient cross-platform, which defeats the whole point of
scandir -- the above is no better than listdir().


Hit a directory with 100,000 entries and you'll change your mind.  ;)

Okay, so the issue is you /want/ to write an efficient, cross-platform 
routine...

hrmmm.

thinking

Okay, marry the two ideas together:

scandir(path, info=None, onerror=None)
"""
Return a generator that returns one directory entry at a time in a 
DirEntry object


Should that be "that yields one directory entry at a time"?


info:  None --> DirEntries will have whatever attributes the O/S 
provides
   'type'  --> DirEntries will already have at least the file/dir 
distinction
   'stat'  --> DirEntries will also already have stat information
"""

DirEntry.is_dir()
   Return True if this is a directory-type entry; may call os.lstat if the 
cache is empty.

DirEntry.is_file()
   Return True if this is a file-type entry; may call os.lstat if the cache 
is empty.

DirEntry.is_symlink()
   Return True if this is a symbolic link; may call os.lstat if the cache 
is empty.

DirEntry.stat
   Return the stat info for this link; may call os.lstat if the cache is 
empty.


Why is "is_dir", et al, functions, but "stat" not a function?



This way both paradigms are supported.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Another case for frozendict

[MRAB]

On 2014-07-16 00:48, Russell E. Owen wrote:

In article
,
  Chris Angelico  wrote:


On Mon, Jul 14, 2014 at 12:04 AM, Jason R. Coombs  wrote:
> I can achieve what I need by constructing a set on the ‘items’ of the 
dict.
>
 set(tuple(doc.items()) for doc in res)
>
> {(('n', 1), ('err', None), ('ok', 1.0))}

This is flawed; the tuple-of-tuples depends on iteration order, which
may vary. It should be a frozenset of those tuples, not a tuple. Which
strengthens your case; it's that easy to get it wrong in the absence
of an actual frozendict.


I would love to see frozendict in python.

I find myself using dicts for translation tables, usually tables that
should not be modified. Documentation usually suffices to get that idea
across, but it's not ideal.

frozendict would also be handy as a default values for function
arguments. In that case documentation isn't enough and one has to resort
to using a default value of None and then changing it in the function
body.

I like frozendict because I feel it is expressive and adds some safety.


Here's another use-case.

Using the 're' module:

>>> import re
>>> # Make a regex.
... p = re.compile(r'(?P\w+)\s+(?P\w+)')
>>>
>>> # What are the named groups?
... p.groupindex
{'first': 1, 'second': 2}
>>>
>>> # Perform a match.
... m = p.match('FIRST SECOND')
>>> m.groupdict()
{'first': 'FIRST', 'second': 'SECOND'}
>>>
>>> # Try modifying the pattern object.
... p.groupindex['JUNK'] = 'foobar'
>>>
>>> # What are the named groups now?
... p.groupindex
{'first': 1, 'second': 2, 'JUNK': 'foobar'}
>>>
>>> # And the match object?
... m.groupdict()
Traceback (most recent call last):
  File "", line 2, in 
IndexError: no such group

It can't find a named group called 'JUNK'.

And with a bit more tinkering it's possible to crash Python. (I'll
leave that as an exercise for the reader! :-))

The 'regex' module, on the other hand, rebuilds the dict each time:

>>> import regex
>>> # Make a regex.
... p = regex.compile(r'(?P\w+)\s+(?P\w+)')
>>>
>>> # What are the named groups?
... p.groupindex
{'second': 2, 'first': 1}
>>>
>>> # Perform a match.
... m = p.match('FIRST SECOND')
>>> m.groupdict()
{'second': 'SECOND', 'first': 'FIRST'}
>>>
>>> # Try modifying the regex.
... p.groupindex['JUNK'] = 'foobar'
>>>
>>> # What are the named groups now?
... p.groupindex
{'second': 2, 'first': 1}
>>>
>>> # And the match object?
... m.groupdict()
{'second': 'SECOND', 'first': 'FIRST'}

Using a frozendict instead would be a nicer solution.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] sum(...) limitation

[MRAB]

On 2014-08-02 16:27, Steven D'Aprano wrote:

On Sat, Aug 02, 2014 at 10:52:07AM -0400, Alexander Belopolsky wrote:

On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano  wrote:

> String concatenation with + is an attractive
> nuisance for many people, including some who actually know better but
> nevertheless do it. Also, for reasons I don't understand, many people
> dislike or cannot remember to use ''.join.
>

Since sum() already treats strings as a special case, why can't it simply
call (an equivalent of) ''.join itself instead of telling the user to do
it?  It does not matter why "many people dislike or cannot remember to use
''.join" - if this is a fact - it should be considered by language
implementors.


It could, of course, but there is virtue in keeping sum simple,
rather than special-casing who knows how many different types. If sum()
tries to handle strings, should it do the same for lists? bytearrays?
array.array? tuple? Where do we stop?


We could leave any special-casing to the classes themselves:

def sum(iterable, start=0):
sum_func = getattr(type(start), '__sum__')

if sum_func is None:
result = start

for item in iterable:
result = result + item
else:
result = sum_func(start, iterable)

return result


Ultimately it comes down to personal taste. Some people are going to
wish sum() tried harder to do the clever thing with more types, some
people are going to wish it was simpler and didn't try to be clever at
all.

Another argument against excessive cleverness is that it ties sum() to
one particular idiom or implementation. Today, the idiomatic and
efficient way to concatenate a lot of strings is with ''.join, but
tomorrow there might be a new str.concat() method. Who knows? sum()
shouldn't have to care about these details, since they are secondary to
sum()'s purpose, which is to add numbers. Anything else is a
bonus (or perhaps a nuisance).

So, I would argue that when faced with something that is not a number,
there are two reasonable approaches for sum() to take:

- refuse to handle the type at all; or
- fall back on simple-minded repeated addition.


By the way, I think this whole argument would have been easily
side-stepped if + was only used for addition, and & used for
concatenation. Then there would be no question about what sum() should
do for lists and tuples and strings: raise TypeError.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path related questions for Guido

[MRAB]

On 2014-08-26 03:11, Stephen J. Turnbull wrote:

Nick Coghlan writes:

  > "purge_surrogate_escapes" was the other term that occurred to me.

"purge" suggests removal, not replacement.  That may be useful too.

neutralize_surrogate_escapes(s, remove=False, replacement='\uFFFD')


How about:

replace_surrogate_escapes(s, replacement='\uFFFD')

If you want them removed, just pass an empty string as the replacement.


maybe?  (Of course the remove argument is feature creep, so I'm only
about +0.5 myself.  And the name is long, but I can't think of any
better synonyms for "make safe" in English right now).

  > Either way, my use case is to filter them out when I *don't* want to
  > pass them along to other software, but would prefer the Unicode
  > replacement character to the ASCII question mark created by using the
  > "replace" filter when encoding.

I think it would be preferable to be unicodely correct here by
default, since this is a str -> str function.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bytes path related questions for Guido

[MRAB]

On 2014-08-28 05:56, Glenn Linderman wrote:

On 8/27/2014 6:08 PM, Stephen J. Turnbull wrote:

Glenn Linderman writes:
  > On 8/26/2014 4:31 AM, MRAB wrote:
  > > On 2014-08-26 03:11, Stephen J. Turnbull wrote:
  > >> Nick Coghlan writes:

  > > How about:
  > >
  > > replace_surrogate_escapes(s, replacement='\uFFFD')
  > >
  > > If you want them removed, just pass an empty string as the
  > > replacement.

That seems better to me (I had too much C for breakfast, I think).

  > And further, replacement could be a vector of 128 characters, to do
  > immediate transcoding,

Using what encoding?


The vector would contain the transcoding. Each lone surrogate would map
to a character in the vector.


If you knew that much, why didn't you use
(write, if necessary) an appropriate codec?  I can't envision this
being useful.


If the data format describes its encoding, possibly containing data from
several encodings in various spots, then perhaps it is best read as
binary, and processed as binary until those definitions are found.

But an alternative would be to read with surrogate escapes, and then
when the encoding is determined, to transcode the data. Previously, a
proposal was made to reverse the surrogate escapes to the original
bytes, and then apply the (now known) appropriate codec. There are not
appropriate codecs that can convert directly from surrogate escapes to
the desired end result. This technique could be used instead, for
single-byte, non-escaped encodings. On the other hand, writing specialty
codecs for the purpose would be more general.


There'll be a surrogate escape if a byte couldn't be decoded, but just
because a byte could be decoded, it doesn't mean that it's correct.

If you picked the wrong encoding, the other codepoints could be wrong
too.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https://docs.python.org/3/using/index.html not linking correctly

[MRAB]

On 2014-10-21 00:09, Eli Bendersky wrote:



On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy mailto:tjre...@udel.edu>> wrote:

If I go to https://docs.python.org/3/using/index.html and click on
any of the TOC entries, I get 'connecting' indefinitely.  This
problem seems unique to this file. I tried several other index files
and clicking am entry brings up the corresponding page almost
immediately.


Works fine for me, Terry, in both Chrome and Firefox. Could be something
in your browser/caching? Try "private mode" or whatever that's called in
your browser of choice.


Just tried the first link:

https://docs.python.org/3/using/cmdline.html

I also get 'Connecting' indefinitely.

So I tested the link here:

http://www.downforeveryoneorjustme.com/

It said: "It's not just you! http://https looks down from here."

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https://docs.python.org/3/using/index.html not linking correctly

[MRAB]

On 2014-10-21 00:38, Oleg Broytman wrote:

On Tue, Oct 21, 2014 at 12:29:45AM +0100, MRAB  
wrote:

On 2014-10-21 00:09, Eli Bendersky wrote:
>
>On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <mailto:tjre...@udel.edu>> wrote:
>
>If I go to https://docs.python.org/3/using/index.html and click on
>any of the TOC entries, I get 'connecting' indefinitely.  This
>problem seems unique to this file. I tried several other index files
>and clicking am entry brings up the corresponding page almost
>immediately.
>
>Works fine for me, Terry, in both Chrome and Firefox. Could be something
>in your browser/caching? Try "private mode" or whatever that's called in
>your browser of choice.
>
Just tried the first link:

https://docs.python.org/3/using/cmdline.html

I also get 'Connecting' indefinitely.

So I tested the link here:

http://www.downforeveryoneorjustme.com/

It said: "It's not just you! http://https looks down from here."


I think this is a limitation of isup.me: it doesn't like https://
URLs -- either use http:// or no protocol prefix at all.


Well, this:

https://docs.python.org/3/using/cmdline.html

is working for me now, but this:

https://docs.python.org/3/whatsnew/index.html

isn't. It looks like some kind of intermittent glitch somewhere...

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https://docs.python.org/3/using/index.html not linking correctly

[MRAB]

On 2014-10-21 02:17, MRAB wrote:

On 2014-10-21 00:38, Oleg Broytman wrote:

On Tue, Oct 21, 2014 at 12:29:45AM +0100, MRAB  
wrote:

On 2014-10-21 00:09, Eli Bendersky wrote:
>
>On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <mailto:tjre...@udel.edu>> wrote:
>
>If I go to https://docs.python.org/3/using/index.html and click on
>any of the TOC entries, I get 'connecting' indefinitely.  This
>problem seems unique to this file. I tried several other index files
>and clicking am entry brings up the corresponding page almost
>immediately.
>
>Works fine for me, Terry, in both Chrome and Firefox. Could be something
>in your browser/caching? Try "private mode" or whatever that's called in
>your browser of choice.
>
Just tried the first link:

https://docs.python.org/3/using/cmdline.html

I also get 'Connecting' indefinitely.

So I tested the link here:

http://www.downforeveryoneorjustme.com/

It said: "It's not just you! http://https looks down from here."


I think this is a limitation of isup.me: it doesn't like https://
URLs -- either use http:// or no protocol prefix at all.


Well, this:

https://docs.python.org/3/using/cmdline.html

is working for me now, but this:

https://docs.python.org/3/whatsnew/index.html

isn't. It looks like some kind of intermittent glitch somewhere...


And now:

https://docs.python.org/3/whatsnew/index.html

is working.

All on Firefox, BTW.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] https://docs.python.org/3/using/index.html not linking correctly

[MRAB]

On 2014-10-21 01:39, Terry Reedy wrote:

On 10/20/2014 7:29 PM, MRAB wrote:

On 2014-10-21 00:09, Eli Bendersky wrote:



On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy mailto:tjre...@udel.edu>> wrote:

If I go to https://docs.python.org/3/using/index.html and click on
any of the TOC entries, I get 'connecting' indefinitely.  This
problem seems unique to this file. I tried several other index files
and clicking am entry brings up the corresponding page almost
immediately.


Works fine for me, Terry, in both Chrome and Firefox. Could be something
in your browser/caching? Try "private mode" or whatever that's called in
your browser of choice.


Just tried the first link:

https://docs.python.org/3/using/cmdline.html

I also get 'Connecting' indefinitely.

So I tested the link here:

http://www.downforeveryoneorjustme.com/

It said: "It's not just you! http://https looks down from here."


I am using Firefox, but the difference is that by happenstance, I was in
privacy mode (perhaps for the first time with python.org -- I want the
docs in the cache and history list -- because I accessed the docs to
answer a StackOverflow question.) It is not just that document but also
Execution model and Expression in Reference.  (I tried each 3 times
minutes apart.)  I have not had a similiar problem with any Arstechnica
or StackOverflow pages, or similar sites.

So the solution for me is to use normal mode, but something seems not
quite right in the interaction between privacy mode Firefox and python.org.

No problem I could find with Internet Explorer's 'In Privacy' mode.


I have/had occasional problems even though I was using normal mode.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Real-world use of Counter

[MRAB]

On 2014-11-05 16:33, Ethan Furman wrote:

I'm looking for real-world uses of collections.Counter, specifically to see if 
anyone has been surprised by, or had to
spend extra-time debugging, issues with the in-place operators.

If sufficient and/or compelling use-cases are uncovered, the behavior of 
Counter may be slightly modified.

Background:

Most Python data types will cause a TypeError to be raised if unusable types 
are passed in:

--> {'a': 0}.update(5)
TypeError: 'int' object is not iterable

--> [1, 2, 3].extend(3.14)
TypeError: 'float' object is not iterable

--> from collections import Counter
--> Counter() + [1, 2, 3]
TypeError: unsupported operand type(s) for +: 'Counter' and 'list'

Most Counter in-place methods also behave this way:

--> c /= [1, 2, 3]
TypeError: unsupported operand type(s) for /=: 'Counter' and 'list'

However, in the case of a handful of Counter in-place methods (+=, -=, &=, and 
|=), this is what happens instead:

--> c += [1, 2, 3]
AttributeError: 'list' object has no attribute 'items'

Even worse (in my opinion) is the case of an empty Counter `and`ed with an 
incompatible type:

--> c &= [1, 2, 3]
-->

No error is raised at all.

In order to avoid unnecessary code churn (the fix itself is quite simple), the 
maintainer of the collections module
wants to know if anybody has actually been affected by these inconsistencies, 
and if so, whether it was a minor
inconvenience, or a compelling use-case.

So, if this has bitten you, now is the time to speak up!  :)


Whenever I've used the Counter class, I've known it was a counter, and
not tried to do any of the above, but at least, if I did, I'd get an
exception (or hope I do).

The final example, however, is odd. I think that one should be fixed.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Static checker for common Python programming errors

[MRAB]

On 2014-11-18 01:21, Terry Reedy wrote:

On 11/17/2014 9:49 AM, Stefan Bucur wrote:

I'm developing a Python static analysis tool that flags common
programming errors in Python programs. The tool is meant to complement
other tools like Pylint (which perform checks at lexical and syntactic
level) by going deeper with the code analysis and keeping track of the
possible control flow paths in the program (path-sensitive analysis).

For instance, a path-sensitive analysis detects that the following
snippet of code would raise an AttributeError exception:

if object is None: # If the True branch is taken, we know the object is None
   object.doSomething() # ... so this statement would always fail

I'm writing first to the Python developers themselves to ask, in their
experience, what common pitfalls in the language & its standard library
such a static checker should look for. For instance, here [1] is a list
of static checks for the C++ language, as part of the Clang static
analyzer project.


You could also a) ask on python-list (new thread), or scan python
questions on StackOverflow.  Todays's example: "Why does my function
return None?"  Because there is no return statement.  Perhaps current
checkers can note that, but what about if some branches have a return
and others do not?  That is a likely bug.


Mutable default parameters comes up occasionally.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators

[MRAB]

On 2014-11-19 20:10, Guido van Rossum wrote:

There's a new PEP proposing to change how to treat StopIteration
bubbling up out of a generator frame (not caused by a return from
the frame). The proposal is to replace such a StopIteration with a
RuntimeError (chained to the original StopIteration), so that only
*returning* from a generator (or falling off the end) causes the
iteration to terminate.


The PEP says """any generator that depends on an implicitly-raised
StopIteration to terminate it will have to be rewritten to either catch
that exception or use a for-loop"""

Shouldn't that be "... explicitly-raised ...", because returning raises
StopIteration implicitly? ("raise StopIteration" is explicit)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] datetime nanosecond support (ctd?)

[MRAB]

On 2014-12-11 18:33, Skip Montanaro wrote:

On Thu, Dec 11, 2014 at 11:58 AM, Matthieu Bec  wrote:

...or keep using "%f" if acceptable...


That might be a problem. While it will probably work most of the time,
there are likely to be situations where the caller assumes it
generates a six-digit string. I did a little poking around. It seems
like "%N" isn't used.


Could the number of digits be specified? You could have "%9f" for
nanoseconds, "%3f" for milliseconds, etc. The default would be 6
microseconds for backwards compatibility.

Maybe, also, strptime could support "%*f" to gobble as many digits as
are available.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Any grammar experts?

[MRAB]

On 2015-01-26 19:39, Petr Viktorin wrote:

On Mon, Jan 26, 2015 at 8:29 PM, Ethan Furman  wrote:

On 01/26/2015 11:24 AM, Barry Warsaw wrote:

On Jan 26, 2015, at 10:55 AM, Ethan Furman wrote:


In the your example

 from_env = {'a': 12}
 from_config = {'a': 13}

 f(**from_env, **from_config)

I would think 'a' should be 13, as from_config is processed /after/ from_env.

So which is it?


In the face of ambiguity, refuse the temptation to guess.


Lots of things are ambiguous until one learns the rules.  ;)


I don't see why `f(**{'a': 12}, **{'a': 13})` should not be equivalent
to `f(a=12, **{'a':13})` – iow, raise TypeError.


One the one hand we have:

>>> foo(a=12, **{'a': 13})
Traceback (most recent call last):
  File "", line 1, in 
TypeError: foo() got multiple values for keyword argument 'a'

and on the other hand we have:

>>> foo(a=12, a=13)
  File "", line 1
SyntaxError: keyword argument repeated

(Should this be a SyntaxError?)

But we also have:

>>> {'a': 12, 'a': 13}
{'a': 13}

So, what should:

>>> f(**from_env, **from_config)

do if there are common keys?

Raise an exception? Behave like dict.update? Behave like ChainMap?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] also

[MRAB]

On 2015-01-28 14:39, Alan Armour wrote:

if you can do this

a chemical physics and element physics like everything from melting
points to how much heat you need to add two chemicals together

and physics like aerodynamics, space dynamics, and hydrodynamics
etcetera for propellers and motors and stuff.

just having this in a main language seems to make a shit ton of sense.


Just like all the physics you can think of from electrical equipment to
building microchips to oscillators and resistors and stuff like that.

thanks


You should be looking at Wolfram Alpha instead...
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing builtin data structures

[MRAB]

On 2015-02-13 00:55, Guido van Rossum wrote:

On Thu, Feb 12, 2015 at 4:36 PM, Ethan Furman mailto:et...@stoneleaf.us>> wrote:

I suspect the last big hurdle to making built-in data structures
nicely subclassable is the insistence of such types to
return new instances as the base class instead of the derived class.

In case that wasn't clear ;)

--> class MyInt(int):
...   def __repr__(self):
... return 'MyInt(%d)' % self
...
--> m = MyInt(42)
--> m
MyInt(42)
--> m + 1
43
--> type(m+1)


Besides the work it would take to rectify this, I imagine the
biggest hurdle would be the performance hit in always
looking up the type of self.  Has anyone done any preliminary
benchmarking?  Are there other concerns?


Actually, the problem is that the base class (e.g. int) doesn't know how
to construct an instance of the subclass -- there is no reason (in
general) why the signature of a subclass constructor should match the
base class constructor, and it often doesn't.


> So this is pretty much a no-go. It's not unique to Python -- it's a
> basic issue with OO.
>
Really?


>>> class BaseInt:
... def __init__(self, value):
... self._value = value
... def __add__(self, other):
... return type(self)(self._value + other)
... def __repr__(self):
... return '%s(%s)' % (type(self), self._value)
...
>>> class MyInt(BaseInt):
... pass
...
>>>
>>> m = BaseInt(42)
>>> m
(42)
>>> m + 1
(43)
>>> type(m + 1)

>>>
>>> m = MyInt(42)
>>> m
(42)
>>> m + 1
(43)
>>> type(m + 1)

>>>

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] TypeError messages

[MRAB]

On 2015-02-19 22:50, Serhiy Storchaka wrote:> Different patterns for
TypeError messages are used in the stdlib:
>
>   expected X, Y found
>   expected X, found Y
>   expected X, but Y found
>   expected X instance, Y found
>   X expected, not Y
>   expect X, not Y
>   need X, Y found
>   X is required, not Y
>   Z must be X, not Y
>   Z should be X, not Y
>
> and more.
>
> What the pattern is most preferable?
>
Stylistically, if the first part is in the active voice, then the
second part should also be in the active voice:

expected X, but found Y

The active voice tends to be simpler and easier to parse than the
passive voice.

I think that the word "but" adds clarity here.

Strictly speaking, that message is OK only if it's expecting X itself;
if, in fact, it's expecting an instance of X, then you should really be
saying something along the lines of:

expected X instance, but found Y instance

or:

expected instance of X, but found instance of Y

> Some messages use the article before X or Y. Should the article be
> used or omitted?
>
Messages tend not to be complete sentences anyway, so I think that it
would be fitting to omit articles.

> Some messages (only in C) truncate actual type name (%.50s, %.80s,
> %.200s, %.500s). Should type name be truncated at all and for how
> limit? Type names newer truncated in TypeError messages raised in
> Python code.
>
Truncating type names is probably not a good idea.

> Some messages enclose actual type name with single quotes ('%s',
> '%.200s'). Should type name be quoted? It is uncommon if type name
> contains spaces.
>
I think that it should be quoted only if it's expecting those
characters, e.g. if it's expecting a closing parenthesis, then it
should say ')'. If, on the other hand, it's expecting a certain type,
then it should give that type unquoted.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] TypeError messages

[MRAB]

On 2015-02-21 17:14, Antoine Pitrou wrote:

On Fri, 20 Feb 2015 14:05:11 +
Brett Cannon  wrote:

On Thu Feb 19 2015 at 5:52:07 PM Serhiy Storchaka 
wrote:

> Different patterns for TypeError messages are used in the stdlib:
>
>  expected X, Y found
>  expected X, found Y
>  expected X, but Y found
>  expected X instance, Y found
>  X expected, not Y
>  expect X, not Y
>  need X, Y found
>  X is required, not Y
>  Z must be X, not Y
>  Z should be X, not Y
>
> and more.
>
> What the pattern is most preferable?
>

My preference is for "expected X, but found Y".


If we are busy nitpicking, why are we saying "found Y"? Nothing was
*found* by the callee, it just *got* an argument.


Well, it depends on the reason for the message.

If you're passing an argument, then 'found' is the wrong word, but if
you're parsing, say, a regex, then 'got' is the wrong word.


So it should be "expected X, but got Y".

Personally, I think the "but" is superfluous: the contradiction is
already implied, so "expected X, got Y" is terser and conveys the
meaning just as well.


If you wanted a message to cover both argument-passing and parsing,
then "expected Y, not Y" would do.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] some minor questions about pep8

[MRAB]

On 2015-03-21 21:00, Donald Stufft wrote:



On Mar 21, 2015, at 4:53 PM, Barry Warsaw  wrote:

On Mar 20, 2015, at 08:53 PM, Guido van Rossum wrote:


FWIW, I think __version__, __author__ etc. were bad ideas. Almost nobody
manages these correctly. Note that the PEP 8 section starts with less than
an endorsement: "If you *have* to have Subversion, CVS, or RCS crud in your
source file, do it as follows."


I tend to agree.  Individual module version numbers are mostly useless,
especially with today's hash-based vcses.

That's different than package versions, and for which I really need to
resurrect and update PEP 396.


I sort of think (though I haven’t completely convinced myself) that adding
something like __version__ to a package is a work around to the fact that
we don’t have an API that lets someone ask “what is the distribution and
version that this module/import-package came from”.

Something like:

 >>> import theorticalapi
 >>> import mymodule
 >>>
 >>> d = theorticalapi.module_dist(mymodule)
 >>> print(d.name)
 >>> mycustommodules
 >>> print(d.version)
 >>> 1.0

Since we (theoretically) should have all of this information already inside
of the packaging metadata, we just don’t have a way to say “given a particular
import, give me the information I want).

A bit OT (and should probably be in python-ideas), but I've come to 
think that it might be more helpful to have double version numbers.


What do I mean by that?

Well, an "addition" version that's increased when something is added, 
and a "deletion" version that's increased when something is removed.


Suppose there's a new release of a module.

If it has a higher "addition version", then something has been added, 
which won't affect existing code.


If it has a higher "deletion version", then something has been removed, 
which _might_ affect existing code, so you'll need to check. If the code 
doesn't use what was removed, then you can accept the higher deletion 
version.





That said, if an official answer is required, common sense would suggest
that __version__ should go before the imports. (I would put it before the
docstring too, except then the docstring wouldn't be a docstring any more.
Go figure.)


And after __future__ imports too, right?



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Aware datetime from naive local time Was: Status on PEP-431 Timezones

[MRAB]

On 2015-04-13 20:14, Alexander Belopolsky wrote:


On Mon, Apr 13, 2015 at 2:05 PM, Chris Barker mailto:chris.bar...@noaa.gov>> wrote:

However, different UTC times may map to the same wall time and
some expressible wall times are not results of a map of any UTC
time.


got it. I suggest you perhaps word it something like:

wall_time = f( location, utc_time)

and

utc_time = f( location, utc_time )

These are two different problems, and one is much harder than the
other! (though both are ugly!)


You probably meant "utc_time = f( location, wall_time)" in the last
equation, but that would still be wrong.
A somewhat more correct equation would be

utc_time = f^(-1)( location, wall_time)

where f^(-1) is the inverse function of f, but since f in not monotonic,
no such inverse exists.


Don't you mean "f⁻¹"? :-)


Finding the inverse of f is the same as solving the equation f(x) = y
for any given y.  If f is such that this
equation has only one solution for all possible values of y then an
inverse exists, but this
is not so in our case.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] async/await in Python; v2

[MRAB]

On 2015-04-23 18:51, Barry Warsaw wrote:

On Apr 21, 2015, at 01:26 PM, Yury Selivanov wrote:


The updated version of the PEP should be available shortly at
https://www.python.org/dev/peps/pep-0492 and is also pasted in this email.


There's a lot to like about PEP 492.  I only want to mildly bikeshed a bit on
the proposed new syntax.

Why "async def" and not "def async"?

My concern is about existing tools that already know that "def" as the first
non-whitespace on the line starts a function/method definition.  Think of a
regexp in an IDE that searches backwards from the current line to find the
function its defined on.  Sure, tools can be updated but it is it *necessary*
to choose a syntax that breaks tools?

 def async useful():

seems okay to me.

Probably the biggest impact on the PEP would be symmetry with asynchronous
with and for.  What about:

 with async lock:

and

 for async data in cursor:

That would also preserve at least some behavior of existing tools.

Anyway, since the PEP doesn't explicitly describe this as an alternative, I
want to bring it up.

(I have mild concerns about __a*__ magic methods, since I think they'll be
harder to visually separate, but here the PEP does describe the __async_*__
alternatives.)


On the other hand, existing tools might be expecting "def" and "for" to
be followed by a name.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 559 - built-in noop()

[MRAB]

On 2017-09-09 19:46, Barry Warsaw wrote:

I couldn’t resist one more PEP from the Core sprint.  I won’t reveal where or 
how this one came to me.

-Barry

PEP: 559
Title: Built-in noop()
Author: Barry Warsaw 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2017-09-08
Python-Version: 3.7
Post-History: 2017-09-09


Abstract


This PEP proposes adding a new built-in function called ``noop()`` which does
nothing but return ``None``.


[snip]

I'd prefer the more tradition "nop". :-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 554 v3 (new interpreters module)

[MRAB]

On 2017-09-23 10:45, Antoine Pitrou wrote:


Hi Eric,

On Fri, 22 Sep 2017 19:09:01 -0600
Eric Snow  wrote:


Please elaborate.  I'm interested in understanding what you mean here.
Do you have some subinterpreter-based concurrency improvements in
mind?  What aspect of CSP is the PEP following too faithfully?


See below the discussion of blocking send()s :-)


As to "running_interpreters()" and "idle_interpreters()", I'm not sure
what the benefit would be.  You can compose either list manually with
a simple comprehension:

[interp for interp in interpreters.list_all() if interp.is_running()]
[interp for interp in interpreters.list_all() if not interp.is_running()]


There is a inherit race condition in doing that, at least if
interpreters are running in multiple threads (which I assume is going
to be the overly dominant usage model).  That is why I'm proposing all
three variants.


An alternative to 3 variants would be:

interpreters.list_all(running=True)

interpreters.list_all(running=False)

interpreters.list_all(running=None)

[snip]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] how/where is open() implemented ?

[MRAB]

On 2017-10-06 03:19, Yubin Ruan wrote:

Hi,
I am looking for the implementation of open() in the src, but so far I
am not able to do this.


From my observation, the implementation of open() in python2/3 does

not employ the open(2) system call. However without open(2) how can
one possibly obtain a file descriptor?


I think it's somewhere in here:

https://github.com/python/cpython/blob/master/Modules/_io/fileio.c
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

[MRAB]

On 2017-10-28 22:05, Guido van Rossum wrote:
On Sat, Oct 28, 2017 at 12:09 AM, Nick Coghlan > wrote:


On 28 October 2017 at 01:57, Guido van Rossum mailto:gu...@python.org>> wrote:

Oh. Yes, that is being discussed about once a year two. It
seems Matthew isn't very interested in helping out with the
port, and there are some concerns about backwards
compatibility with the `re` module. I think it needs a champion!


Matthew's been amenable to the idea when it comes up, and he
explicitly wrote the module to be usable as a drop-in replacement
for "re" (hence the re-compatible v0 behaviour still being the
default).

The resistance has more been from our side, since this is a case
where existing regex module users are clearly better off if it
remains a separate project, as that keeps upgrades independent of
the relatively slow standard library release cycle (and allows it
to be used on Python 2.7 as well as in 3.x). By contrast, the
potential benefits of standard library inclusion accrue primarily
to Python newcomers and folks writing scripts without the benefit
of package management tools, since they'll have a more capable
regex engine available as part of the assumed language baseline.

That means that if we add regex to the standard library in the
regular way, there's a more than fair chance that we'll end up
with an outcome like the json vs simplejson split, where we have
one variant in the standard library, and another variant on PyPI,
and the variants may drift apart over time if their maintenance is
being handled by different people. (Note: one may argue that we
already have this split in the form of re vs regex. So if regex
was brought in specifically to replace _sre as the re module
implementation, rather than as a new public API, then we at least
wouldn't be making anything *worse* from a future behavioural
consistency perspective, but we'd be risking a compatibility break
for anyone depending on _sre and other internal implementation
details of the re module).

One potential alternative approach that is then brought up (often
by me) is to suggest instead *bundling* the regex module with
CPython, without actually bringing it fully within the regular
standard library maintenance process. The idea there would be to
both make the module available by default in python.org
 downloads, *and* make it clear to
redistributors that the module is part of the expected baseline of
Python functionality, but otherwise keep it entirely in its
current independently upgradable form.

That would still be hard (since it would involve establishing new
maintenance policy precedents that go beyond the current
special-casing of `pip` in order to bootstrap PyPI access), but
would have the additional benefit of paving the way for doing
similar things with other modules where we'd like them to be part
of the assumed baseline for end users, but also have reasons for
wanting to avoid tightly coupling them to the standard libary's
regular maintenance policy (most notably, requests).

And that's where discussions tend to fizzle out:

* outright replacement of the current re module implementation
with a private copy of the regex module introduces compatibility
risks that would need a fiat decision from you as BDFL to say
"Let's do it anyway, make sure the test suite still works, and
then figure out how to cope with any other consequences as they arise"
* going down the bundling path requires making some explicit
community management decisions around what we actually want the
standard library to *be* (and whether or not there's a difference
between "the standard library" and "the assumed available package
set" for Python installations that are expected to run arbitrary
third party scripts rather than specific applications)
* having both the current re API and implementation *and* a new
regex based API and implementation in the standard library
indefinitely seems like it would be a maintainability nightmare
that delivered the worst of all possible outcomes for everyone
involved (CPython maintainers, regex maintainers, Python end users)


Maybe it would be easier if Matthew were amenable to maintaining the 
stdlib version and only add new features to the PyPI version when 
they've also been added to the stdlib version. IOW if he were 
committed to *not* letting the [simple]json thing happen.


I don't condone having two different regex implementations/APIs 
bundled in any form, even if one were to be deprecated -- we'd never 
get rid of the deprecated one until 4.0. (FWIW I don't condone this 
pattern for other packages/modules either.) Note that even if we 
outright switched there would *still* be two versions, becaus

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

[MRAB]

On 2017-10-29 00:48, Steven D'Aprano wrote:

On Sun, Oct 29, 2017 at 12:31:01AM +0100, MRAB wrote:

Not that I'm planning on making any further additions, just bug fixes 
and updates to follow the Unicode updates. I think I've crammed enough 
into it already. There's only so much you can do with the regex syntax 
with its handful of metacharacters and possible escape sequences...


What do you think of the Perl 6 regex syntax?

https://en.wikipedia.org/wiki/Perl_6_rules#Changes_from_Perl_5

I think I prefer something that's more like PEG, with quoted literals, 
perhaps because it looks more like a programming language, but also 
because it's clearer than saying "these characters are literal, but 
those aren't". That webpage says "Literals: word characters (letters, 
numbers and underscore) matched literally", but is that all letters? And 
what about diacritics, and combining characters?


I'm not keen on  and , I like & and ! better, 
but then how would you write a lookbehind?


Named rules are good, better than regex's use of named capture groups, 
and if you quote literal, then you wouldn't need to wrap rule call in 
<...>, as Perl 6 does.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] \G (match last position) regex operator non-existant in python?

[MRAB]

On 2017-10-29 12:27, Serhiy Storchaka wrote:

27.10.17 18:35, Guido van Rossum пише:
The "why" question is not very interesting -- it probably wasn't in PCRE 
and nobody was familiar with it when we moved off PCRE (maybe it wasn't 
even in Perl at the time -- it was ~15 years ago).


I didn't understand your description of \G so I googled it and found a 
helpful StackOverflow article: 
https://stackoverflow.com/questions/21971701/when-is-g-useful-application-in-a-regex. 
 From this I understand that when using e.g. findall() it forces 
successive matches to be adjacent.


This looks too Perlish to me. In Perl regular expressions are the part
of language syntax, they can contain even Perl expressions. Arguments to
them are passed implicitly (as well as to Perl's analogs of str.strip()
and str.split()) and results are saved in global special variables.
Loops also can be implicit.

It seems to me that \G makes sense only to re.findall() and
re.finditer(), not to re.match(), re.search() or re.split().

In Python all this is explicit. Compiled regular expressions are
objects, and you can pass start and end positions to Pattern.match().
The Python equivalent of \G looks to me like:

p = re.compile(...)
i = 0
while True:
  m = p.match(s, i)
  if not m: break
  ...
  i = m.end()


You're correct. \G matches at the start position, so .search(r\G\w+') 
behaves the same as .match(r'\w+').


findall and finditer perform a series of searches, but with \G at the 
start they'll perform a series of matches, each anchored at where the 
previous one ended.



The one also can use the undocumented Pattern.scanner() method. Actually
Pattern.finditer() is implemented as iter(Pattern.scanner().search).
iter(Pattern.scanner().match) would return an iterator of adjacent matches.

I think it would be more Pythonic (and much easier) to add a boolean
parameter to finditer() and findall() than introduce a \G operator.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default

[MRAB]

On 2017-11-07 14:17, Philipp A. wrote:
Nick Coghlan mailto:ncogh...@gmail.com>> schrieb am 
Di., 7. Nov. 2017 um 14:57 Uhr:


Users of applications written in Python are not python-dev's users:
they're the users of those applications, and hence the quality of
that experience is up to the developers of those applications. […]


Thank you, that’s exactly what I’m talking about. Besides: Nobody is 
“hosed”… There will be one occurrence of every DeprecationWarning in the 
stderr of the application. Hardly the end of the world for CLI 
applications and even invisible for GUI applications.


If the devs care about the user not seeing any warnings in their CLI 
application, they’ll have a test set up for that, which will tell them 
that the newest python-dev would raise a new warning, once they turn on 
testing for that release. That’s completely fine!


Explicit is better than implicit! If I know lib X raises 
DeprecationWarnings I don’t care about, I want to explicitly silence 
them, instead of missing out on all the valuable information in other 
DeprecationWarnings.


Also, errors should never pass silently. Deprecation warnings are future 
errors.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tricky way of of creating a generator via a comprehension expression

[MRAB]

On 2017-11-25 02:21, Chris Jerdonek wrote:

On Fri, Nov 24, 2017 at 5:06 PM, Nathaniel Smith  wrote:

On Fri, Nov 24, 2017 at 4:22 PM, Guido van Rossum  wrote:

The more I hear about this topic, the more I think that `await`, `yield` and
`yield from` should all be banned from occurring in all comprehensions and
generator expressions. That's not much different from disallowing `return`
or `break`.


I would say that banning `yield` and `yield from` is like banning
`return` and `break`, but banning `await` is like banning function
calls.


I agree. I was going to make the point earlier in the thread that
using "await" can mostly just be thought of as a delayed function
call, but it didn't seem at risk of getting banned until Guido's
comment so I didn't say anything (and there were too many comments
anyways).

I think it's in a different category for that reason. It's much easier
to reason about than, say, "yield" and "yield from".


+1

Seeing "await" there didn't/doesn't confuse me; seeing "yield" or "yield 
from" does.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Regular expressions: splitting on zero-width patterns

[MRAB]

On 2017-11-28 20:04, Serhiy Storchaka wrote:

The two largest problems in the re module are splitting on zero-width
patterns and complete and correct support of the Unicode standard. These
problems are solved in regex. regex has many other features, but they
are less important.

I want to tell the problem of splitting on zero-width patterns. It
already was discussed on Python-Dev 13 years ago [3] and maybe later.
See also issues: [4], [5], [6], [7], [8].

In short it doesn't work. Splitting on the pattern r'\b' doesn't split
the text at boundaries of words, and splitting on the pattern
r'\s+|(?<=-)' will split the text on whitespaces, but will not split
words with hypens as expected.

In Python 3.4 and earlier:

  >>> re.split(r'\b', 'Self-Defence Class')
['Self-Defence Class']
  >>> re.split(r'\s+|(?<=-)', 'Self-Defence Class')
['Self-Defence', 'Class']
  >>> re.split(r'\s*', 'Self-Defence Class')
['Self-Defence', 'Class']

Note that splitting on r'\s*' (0 or more whitespaces) actually split on
r'\s+' (1 or more whitespaces). Splitting on patterns that only can
match the empty string (like r'\b' or r'(?<=-)') never worked, while
splitting

Starting since Python 3.5 splitting on a pattern that only can match the
empty string raises a ValueError (this never worked), and splitting a
pattern that can match the empty string but not only emits a
FutureWarning. This taken developers a time for replacing their patterns
r'\s*' to r'\s+' as they should be.

Now I have created a final patch [9] that makes re.split() splitting on
zero-width patterns.

  >>> re.split(r'\b', 'Self-Defence Class')
['', 'Self', '-', 'Defence', ' ', 'Class', '']
  >>> re.split(r'\s+|(?<=-)', 'Self-Defence Class')
['Self-', 'Defence', 'Class']
  >>> re.split(r'\s*', 'Self-Defence Class')
['', 'S', 'e', 'l', 'f', '-', 'D', 'e', 'f', 'e', 'n', 'c', 'e', 'C',
'l', 'a', 's', 's', '']

The latter case the result is differ too much from the previous result,
and this likely not what the author wanted to get. But users had two
Python releases for fixing their code. FutureWarning is not silent by
default.

Because these patterns produced errors or warnings in the recent two
releases, we don't need an additional parameter for compatibility.

But the problem was not just with re.split(). Other functions also
worked not good with patterns that can match the empty string.

  >>> re.findall(r'^|\w+', 'Self-Defence Class')
['', 'elf', 'Defence', 'Class']
  >>> list(re.finditer(r'^|\w+', 'Self-Defence Class'))
[, , ,
]
  >>> re.sub(r'(^|\w+)', r'<\1>', 'Self-Defence Class')
'<>S- '

After matching the empty string the following character will be skipped
and will be not included in the next match. My patch fixes these
functions too.

  >>> re.findall(r'^|\w+', 'Self-Defence Class')
['', 'Self', 'Defence', 'Class']
  >>> list(re.finditer(r'^|\w+', 'Self-Defence Class'))
[, , ,
]
  >>> re.sub(r'(^|\w+)', r'<\1>', 'Self-Defence Class')
'<>- '

I think this change don't need preliminary warnings, because it change
the behavior of more rarely used patterns. No re tests have been broken.
I was needed to add new tests for detecting the behavior change.

But there is one spoonful of tar in a barrel of honey. I didn't expect
this, but this change have broken a pattern used with re.sub() in the
doctest module: r'(?m)^\s*?$'. This was fixed by replacing it with
r'(?m)^[^\S\n]+?$'). I hope that such cases are pretty rare and think
this is an avoidable breakage.

The new behavior of re.split() matches the behavior of regex.split()
with the VERSION1 flag, the new behavior of re.findall() and
re.finditer() matches the behavior of corresponding functions in the
regex module (independently from the version flag). But the new behavior
of re.sub() doesn't match exactly the behavior of regex.sub() with any
version flag. It differs from the old behavior as you can see in the
example above, but is closer to it that to regex.sub() with VERSION1.
This allowed to avoid braking existing tests for re.sub().

  >>> regex.sub(r'(\W+|(?<=-))', r':', 'Self-Defence Class')
  
  


'Self:Defence:Class'
  
  


  >>> regex.sub(r'(?V1)(\W+|(?<=-))', r':', 'Self-Defence Class')
  
  


'Self::Defence:Class'
  >>> re.sub(r'(\W+|(?<=-))', r':', 'Self-Defence Class')
'Self:Defence:Class'

As re.split() it never matches the empty string adjacent to the previous
match. re.findall() and re.finditer() only don't match the empty string
adjacent to the previous empty string match. In the regex module
regex.sub() is mutually consistent with regex.findall() and
regex.finditer() (with the VERSION1 flag), but regex.split() is not
consistent with them. In the re module re.split() and re.sub() will be
mutually consistent, as well as re.findall() and re.finditer(). This is
more backward compatible. And I don't know reasons for preferring the
behavior of re.findall() and re.finditer() over the behavior of
re.split() in this corner case.

FTR, you could make an argument for either behaviour. For regex, I wen

Re: [Python-Dev] Regular expressions: splitting on zero-width patterns

[MRAB]

On 2017-11-28 20:04, Serhiy Storchaka wrote:

The two largest problems in the re module are splitting on zero-width
patterns and complete and correct support of the Unicode standard. These
problems are solved in regex. regex has many other features, but they
are less important.

I want to tell the problem of splitting on zero-width patterns. It
already was discussed on Python-Dev 13 years ago [3] and maybe later.
See also issues: [4], [5], [6], [7], [8].


[snip]
After some thought, I've decided that if this happens in the re module 
in Python 3.7, then, for the sake of compatibility (and because the edge 
cases are debatable anyway), I'll have the regex module do the same when 
used on Python 3.7.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Regular expressions: splitting on zero-width patterns

[MRAB]

On 2017-11-28 22:27, Guido van Rossum wrote:
On Tue, Nov 28, 2017 at 2:23 PM, MRAB <mailto:pyt...@mrabarnett.plus.com>> wrote:


On 2017-11-28 20:04, Serhiy Storchaka wrote:

The two largest problems in the re module are splitting on
zero-width
patterns and complete and correct support of the Unicode
standard. These
problems are solved in regex. regex has many other features,
but they
are less important.

I want to tell the problem of splitting on zero-width patterns. It
already was discussed on Python-Dev 13 years ago [3] and maybe
later.
See also issues: [4], [5], [6], [7], [8].

[snip]
After some thought, I've decided that if this happens in the re
module in Python 3.7, then, for the sake of compatibility (and
because the edge cases are debatable anyway), I'll have the regex
module do the same when used on Python 3.7.


Maybe it should also be selectable with a version flag?

Well, when anyone who uses re updates to Python 3.7, they'll be faced 
with the change anyway.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[MRAB]

On 2017-12-01 13:24, Random832 wrote:

On Fri, Dec 1, 2017, at 05:31, Steven D'Aprano wrote:
I'm more confused than ever. You seem to be arguing that Python 
functions CAN short-circuit their arguments and avoid evaluating them. 
Is that the case?


If this is merely about when the name "function" is looked up, then I 
don't see why that's relevant to the PEP.


What am I missing?


You're completely missing the context of the discussion, which was the
supposed reason that a *new* function call operator, with the proposed
syntax function?(args), that would short-circuit (based on the
'function' being None) could not be implemented. The whole thing doesn't
make sense to me anyway, since a new operator could have its own
sequence different from the existing one if necessary.


The code:

function?(args)

would be equivalent to:

None if function is None else function(args)

where 'function' would be evaluated once.

If function is None, the arguments would not be evaluated.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Zero-width matching in regexes

[MRAB]

I've finally come to a conclusion as to what the "correct" behaviour of 
zero-width matches should be: """always return the first match, but 
never a zero-width match that is joined to a previous zero-width match""".


If it's about to return a zero-width match that's joined to a previous 
zero-width match, then backtrack and keep on looking for a match.


Example:

>>> print([m.span() for m in re.finditer(r'|.', 'a')])
[(0, 0), (0, 1), (1, 1)]

re.findall, re.split and re.sub should work accordingly.

If re.finditer finds n matches, then re.split should return a list of 
n+1 strings and re.sub should make n replacements (excepting maxsplit, 
etc.).

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Zero-width matching in regexes

[MRAB]

On 2017-12-05 20:26, Terry Reedy wrote:

On 12/4/2017 6:21 PM, MRAB wrote:
I've finally come to a conclusion as to what the "correct" behaviour of 
zero-width matches should be: """always return the first match, but 
never a zero-width match that is joined to a previous zero-width match""".


Is this different from current re or regex?


Sometimes yes.

It's difficult to know how a zero-width match should be handled.

The normal way that, say, findall works is that it searches for a match 
and then continues from where it left off.


If at any point it matched an empty string, it would stall because the 
end of the match is also the start of the match.


How should that be handled?

The old buggy behaviour of the re module was to just advance by one 
character after a zero-width match, which can result in a character 
being skipped and going missing.


A solution is to prohibit a zero-width match that's joined to the 
previous match, but I'm not convinced that that's correct.


If it's about to return a zero-width match that's joined to a previous 
zero-width match, then backtrack and keep on looking for a match.


Example:

 >>> print([m.span() for m in re.finditer(r'|.', 'a')])
[(0, 0), (0, 1), (1, 1)]

re.findall, re.split and re.sub should work accordingly.

If re.finditer finds n matches, then re.split should return a list of 
n+1 strings and re.sub should make n replacements (excepting maxsplit, 
etc.).



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Accepting PEP 560 -- Core support for typing module and generic types

[MRAB]

On 2017-12-15 16:36, Antoine Pitrou wrote:

On Fri, 15 Dec 2017 14:05:46 +0200
Serhiy Storchaka  wrote:

15.12.17 04:00, Guido van Rossum пише:
> In the light of Antoine's and Stephan's feedback I think this can be 
> reconsidered -- while I want to take a cautious stance about resource 
> consumption I don't want to stand in the way of progress.  

I don't see any problems with implementing this on types defined in C. 
This isn't harder than implementing __sizeof__ or pickling support, and 
NumPy classes already have implemented both. Maybe Yury forgot about 
METH_STATIC and METH_CLASS?


The cost of adding new slots:

1. Increased memory consumption. This increases the size of *every* 
class, even if they don't implement this feature.


2. Increased class initialization time. For every class for every slot 
we need to look up corresponding methods in dictionaries of the class 
itself and all its parents (caching doesn't work fine at this stage). 
Significant part of class initialization time is spent on initializing 
slots. This will increase the startup time and the time of creating 
local classes. The relative overhead is more significant in Cython.


3. We need to add a new type feature flag Py_TPFLAGS_HAVE_*. The number 
of possible flags is limited, and most bits already are used. We can add 
the limited number of new slots, and should not spent this resource 
without large need.


4. Increased complexity. Currently the code related to PEP 560 is 
located in few places. With supporting new slots we will need to touch 
more delicate code not related directly to PEP 560. It is hard to review 
and to test such kind of changes. I can't guarantee the correctness.


These are all very good points (except #1 which I think is a red
herring, see my posted example).  Do you have any general idea how to
speed up class creation?

Re the flags, could a flag be used to indicate that there are additional 
flags?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[MRAB]

On 2017-12-21 22:45, Chris Barker wrote:
On Thu, Dec 21, 2017 at 11:55 AM, Terry Reedy > wrote:


I think the understanding problem with this feature arises from two
factors: using annotations to define possibly un-initialized slots
is non-obvious; a new use of annotations for something other than
static typing is a bit of a reversal of the recent pronouncement
'annotations should only be used for static typing'. 



you know, that may be where part of my confusion came from -- all the 
talk lately has been about "type hints" and "type annotations" -- the 
idea of "arbitrary annotations" has been lost.


Therefore, getting the permanent doc 'right' is important.


yup.

@dataclass
class C:
     x
     y = 0

I think the doc should explicitly say that uninitialized fields
require annotation with something (anything, not necessarily a type)
simply to avoid NameError during class creation. 



would this be possible?

@dataclass
class C:
     x:
     y: = 0

That is -- the colon indicates an annotation, but in this case, it's a 
"nothing" annotation.




"..." or "pass", perhaps?

@dataclass
class C:
 x: ...
 y: ... = 0

or:

@dataclass
class C:
 x: pass
 y: pass = 0

It's a syntax error now, but would it be possible to change that? Or 
would the parsing be ambiguous? particularly in other contexts.


of course, then we'd need something to store in as a "nothing" 
annotation -- empty string? None? (but None might mean something) create 
yet anther type for "nothing_annotation"


Hmm, I may have talked myself out of it


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] is typing optional in dataclasses?

[MRAB]

On 2017-12-22 00:19, Gregory P. Smith wrote:

(subject for this sub-thread updated)

On Thu, Dec 21, 2017 at 4:08 PM Chris Barker > wrote:


On Thu, Dec 21, 2017 at 3:36 PM, Gregory P. Smith mailto:g...@krypto.org>> wrote:

 But we already have ... which does - so I'd suggest that for
people who are averse to importing anything from typing and
using the also quite readable Any.  (ie: document this as the
expected practice with both having the same meaning)


I don't think they do, actually - I haven't been following the
typing discussions, but someone in this thread said that ... means
"use the type of teh default" or something like that.


indeed, they may not.  though if that is the definition is it 
reasonable to say that type analyzers recognize the potential 
recursive meaning when the _default_ is ... and treat that as Any?


another option that crossed my mind was "a: 10" without using =.  but 
that really abuses __attributes__ by sticking the default value in 
there which the @dataclass decorator would presumably immediately need 
to undo and fix up before returning the class.  but I don't find 
assigning a value without an = sign to be pythonic so please lets not 
do that! :)


If you allowed "a: 10" (an int value), then you might also allow "a: 
'foo'" (a string value), but wouldn't that be interpreted as a type 
called "foo"?


If you can't have a string value, then you shouldn't have an int value 
either.


[snip]

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[MRAB]

On 2017-12-22 21:02, Mike Miller wrote:


On 2017-12-22 12:15, Chris Barker wrote:

Would it be crazy to bring typing.Any into the builtin namespace?

@dataclass:
     a: Any
     b: Any = 34
     c: int = 0

That reads pretty well to me

  > And having Any available in the built in namespace may help in other cases 
where

There is already an "any" function in the builtins.  It looks fine but not sure
how it will interact with type checkers.

The "dataclass.Data" idea mentioned in a sibling thread is good alternative,
though just wordy enough to make ... a shortcut.

The function is "any", the type is "Any", and "any" != "Any", although I 
wonder how many people will be caught out by that...

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] (no subject)

[MRAB]

On 2017-12-26 07:01, Yogev Hendel wrote:


I don't know if this is the right place to put this,
but I've found the following lines of code results in an incredibly long 
processing time.

Perhaps it might be of use to someone.

/import re/
/pat = re.compile('^/?(?:\\w+)/(?:[%\\w-]+/?)+/?$')/
/pat.match('/t/a-aa-aa-a-aa-aa-aa-aa-aa-aa./')/

The pattern has a repeated repeat, which results in catastrophic 
backtracking.


As an example, think about how the pattern (?:a+)+b would try to match 
the string 'aaac'.


Match 'aaa', but not 'c'.

Match 'aa' and 'a', but not 'c'.

Match 'a' and 'aa', but not 'c'.

Match 'a' and 'a' and 'a', but not 'c'.

That's 4 failed attempts.

Now try match the string 'c'.

Match '', but not 'c'.

Match 'aaa' and 'a', but not 'c'.

Match 'aa' and 'aa', but not 'c'.

Match 'aa' and 'a a', but not 'c'.

Match 'a' and 'aaa', but not 'c'.

Match 'a' and 'aa' and 'a', but not 'c'.

Match 'a' and 'a aa', but not 'c'.

Match 'a' and 'a a' and 'a', but not 'c'.

That's 8 failed attempts.

Each additional 'a' in the string to match will double the number of 
attempts.


Your pattern has (?:[%\w-]+/?)+, and the '/' is optional. The string has 
a '.', which the pattern can't match, but it'll keep trying until it 
finally fails.


If you add just 1 more 'a' or '-' to the string, it'll take twice as 
long as it does now.


You need to think more carefully about how the pattern matches and what 
it'll do when it doesn't match.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bug report in audioop module.

[MRAB]

On 2018-01-10 20:37, Moses Egypt wrote:
I was told to post this here when I asked what to do on the python 
reddit. This is the issue:

https://bugs.python.org/issue32004

It has received no response since I posted it two months ago, so I 
figured I didn't fill something out correctly to get it put on someone's 
tracker. This is my first bug report, so please let me know if there is 
anything I need to do get it appointed to the correct person next time. 
Thanks.


This is open source; nothing gets done unless someone chooses to do it, 
usually because it "scratches their itch" or they have a problem and 
no-one else is interested in doing it.

If you don't get any response, you could always try fixing it yourself.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why is Python for Windows compiled with MSVC?

[MRAB]

On 2018-01-31 19:07, Ray Donnelly wrote:

On Wed, Jan 31, 2018 at 3:04 PM, Oleg Sivokon  wrote:

Hello list.

I'll give some background before asking my question in more detail.


[snip]


Now all I had to do was to re-create my success on Windows (most of the 
employees in my company use Windows).  At first I thought that I'd 
cross-compile on Linux using MinGW.  I compiled Go shared library into a DLL, 
then tried to compile my Python extension and... it didn't work.  I downloaded 
VirtualBox and some Windows images, etc... tried to compile on Windows.  It 
didn't work.  I started asking around, and was told that even though for some 
earlier versions of Python this was kind of possible, for Python 3.5, 3.6 it is 
not.  You must use MSVC to compile Python extensions.  No way around it.
Now, since Go won't compile with MSVC, I'll have to scrap my project and spend 
many weeks re-implementing kubectl.

Here's my question: Why?

Why did you choose to use non-free compiler, which also makes cross-compilation 
impossible?  There wasn't really a reason not to choose MinGW as a way to 
compile extensions on Windows (Ruby does that, Go uses MinGW, perhaps some 
others too).  It would've made things like CI and packaging so much easier...  
What do Python users / developers get from using MSVC instead?


You can compile extension modules with mingw-w64 just fine (modulus a
few gotchas). The Anaconda Distribution we do this for a few packages,
for example rpy2. You can see the build script used here:
https://github.com/AnacondaRecipes/rpy2-feedstock/blob/master/recipe/bld.bat
(disclaimer: I work for Anaconda Inc on this stuff).

MSYS2 also have mingw-w64 builds of Python that might meet your needs.
It is pretty popular in some parts of the open source on Windows world
(disclaimer: I did a lot of the work for this stuff on MSYS2).
I build the wheels (binaries for Windows) for the regex module using 

mingw-w64 and they also work just fine.

I can also build using Microsoft Visual Studio Community 2017 (which is 
free) and they work.


[snip]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pycapsule:PyObject * is NULL pointer

[MRAB]

On 2018-02-21 02:28, 赵亚 wrote:

This "python-dev" list is for development of the Python language itself. 
I think that this message should instead have been sent to "python-list".



i have question:call,c-->python-->c.
1.the c pointer void* abc="123" by pycapsule in the c code.
..
void* lpContext = "abc";
PyObject * lpPyContext = PyCapsule_New(lpContext, "Context",NULL);
..


Is this C code calling the function 'test' in the Python code?

If yes, are you sure that lpPyContext should be in pArgs at index 1 and 
not at index 0?


The function 'test' expects argument 0 => lpContext, argument 1 => 
lpRequest, argument 2 => lpAnswer.



PyTuple_SetItem(pArgs, 1, lpPyContext);
PyObject* pyResult = PyObject_CallObject(pFunc, pArgs);

2.c call python:

in the python code:
import ctypes
pycapsule = ctypes.windll.LoadLibrary("C:\Users\zhaoya16975\Documents\Visual Studio 
2017\Projects\cpython\Release\pycapsule.dll")
def test( lpContext,lpRequest,lpAnswer):
 print lpContext
 pycapsule.hello()
 pycapsule.GetContext(lpContext)
.
the lpContest is ""
but,i can't lpContext in the pycapsule.GetContest:
the capsule is null poniter,the GetContext no execute!!

void* FUNCTION_CALL_MODE GetContext(PyObject  *capsule) {
printf(" GetContext..\n");
//if (!PyCapsule_IsValid((PyObject *)capsule, "Context")) {
//  printf(" the Capsule Context is no Valid\n");
//  return NULL;
//}
//return PyCapsule_GetPointer((PyObject *)capsule, "Context");
return NULL;

}

FUNCTION_CALL_MODE is __stdcall i think no problem.because, if GetContext(PyObject 
 *capsule) -->GetContext() is ok. PyObject  *capsule from python to c unknown 
error occurred.

i hope c call python  pass void* ,and python call c pass void* ,but the capsule 
call no success?


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] ttk.Treeview.insert() does not allow to insert item with iid=0

[MRAB]

On 2018-03-16 20:22, Terry Reedy wrote:

On 3/16/2018 6:54 AM, Игорь Яковченко wrote:

[snip]

There is no item with such iid. This item has autogenerated iid just 
like it's not specified.
I investigated problem and found that in ttk.py, Treeview.insert(... 
iid=None, ...) in method's body has a check:

         if iid:
             res = self.tk.call(self._w, "insert", parent, index,
                 "-id", iid, *opts)
         else:
             res = self.tk.call(self._w, "insert", parent, index, *opts)
It means that if iid is "True" then use it else autogenerate it.
Maybe there should be "if iid is not None", not "if iid"? Or there are 
some reasons to do check this way?


It might be that accepting '' as an iid would be a problem.
Our current tkinter expert, Serhiy Storchaka, should know.  If so, "if
iid in (None, '')" would be needed.


The root of the tree has the iid ''.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] IDLE colorizer

[MRAB]

A thread on python-ideas is talking about the prefixes of string 
literals, and the regex used in IDLE.


Line 25 of Lib\idlelib\colorizer.py is:

stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?"

which looks slightly wrong to me.

The \b will apply only to the first choice.

Shouldn't it be more like:

stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?"

?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] IDLE colorizer

[MRAB]

On 2018-04-02 05:43, Guido van Rossum wrote:
My question for you: how on earth did you find this?! Speaking of a 
needle in a haystack. Did you run some kind of analysis program that 
looks for regexprs? (We've received some good reports from someone who 
did that looking for possible DoS attacks.)



The thread was about string prefixes.

Terry Reedy wrote "IDLE's colorizer does its parsing with a giant regex."

I wondered: "How bad could it be?" (It's smaller now that the IGNORECASE 
flag can have a local scope.)


It wasn't hard to find because it was in a file called "colorizer.py" in 
a folder called "idlelib".


On Sun, Apr 1, 2018 at 6:49 PM, MRAB <mailto:pyt...@mrabarnett.plus.com>> wrote:


A thread on python-ideas is talking about the prefixes of string
literals, and the regex used in IDLE.

Line 25 of Lib\idlelib\colorizer.py is:

    stringprefix = r"(?i:\br|u|f|fr|rf|b|br|rb)?"

which looks slightly wrong to me.

The \b will apply only to the first choice.

Shouldn't it be more like:

    stringprefix = r"(?:\b(?i:r|u|f|fr|rf|b|br|rb))?"

?



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python version numbers

[MRAB]

On 2018-04-03 18:09, Paul G wrote:



On 04/03/2018 12:36 PM, Brett Cannon wrote:

On Tue, 3 Apr 2018 at 07:39 Paul G  wrote:



Paul's point is that he knows e.g. code working in 3.6.0 will work when he
upgrades to 3.6.5, and if his code is warning-free  and works with all
__future__ statements in 3.6 that it will work fine in 3.7. With CalVer you
could make a similar promise if you keep the major version the year of
release and then keep our feature/bugfix number promise like we already
have, but at that point who cares about the year?


I think it is reasonable to use a scheme like this:

YY.MM.patch


Surely that should be:

.MM.patch

[snip]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] in

[MRAB]

On 2018-04-04 00:34, Ethan Furman wrote:

This behavior was recently brought to my attention [1]:

--> 1 in 'hello'
Traceback (most recent call last):
File "", line 1, in 
TypeError: 'in ' requires string as left operand, not int

However, in any other collection (set, dict, list, tuple, etc), the answer 
would be False.

Does anyone remember the reason why an exception is raised in the string 
instance instead of returning False?

Well, strings aren't really a collection like set, etc, which can 
contain various types, even a mixture of types.

A string can contain only strings (including codepoints).

A bytestring can contain only bytestrings and ints (and there's been 
debate on whether the latter was a good idea!).

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572: Assignment Expressions

[MRAB]

On 2018-04-17 22:53, Terry Reedy wrote:

On 4/17/2018 3:46 AM, Chris Angelico wrote:

[snip]



Augmented assignment is not supported in expression form::

 >>> x +:= 1
   File "", line 1
 x +:= 1
 ^
 SyntaxError: invalid syntax


I would have expected :+=, but agree with the omission.

x = x op 1 is abbreviated to x op= 1, so x := x op 1 would be 
abbreviated to x op:= 1. That's what's used in the Icon language.


[snip]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572: Assignment Expressions

[MRAB]

On 2018-04-17 22:44, Greg Ewing wrote:

Paul Moore wrote:

the next question will likely be "so why does = exist at all?"


And if we decide to make ':=' the official assigment operator and
deprectate '=', the next question will be "Why do we have '=='
instead of '='?"

Some languages use '=' for assignment, others for equality, but do you 
know of a language that uses ':=' for equality' or '==' for assignment?


If Python saw '=' it could ask "Do you mean assignment ':=' or equality 
'=='?".

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572: Assignment Expressions

[MRAB]

On 2018-04-20 22:33, Tim Peters wrote:
[snip]


And I'll take this opportunity to repeat the key point for me:  I
tried hard, but never found a single case based on staring at real
code where allowing _fancier_ (than "plain name") targets would be a
real improvement.  In every case I thought it _might_ help, it turned
out that it really didn't unless Python _also_ grew an analog to C's
"comma operator" (take only the last result from a sequence of
expressions).  I'll also note that I asked if anyone else had a
real-life example, and got no responses.

Could a semicolon in a parenthesised expression be an equivalent to C's 
"comma operator"?


[snip]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 572: Assignment Expressions

[MRAB]

On 2018-04-21 03:15, Tim Peters wrote:

[Tim]
>> And I'll take this opportunity to repeat the key point for me:  I
>> tried hard, but never found a single case based on staring at real
>> code where allowing _fancier_ (than "plain name") targets would be a
>> real improvement.  In every case I thought it _might_ help, it turned
>> out that it really didn't unless Python _also_ grew an analog to C's
>> "comma operator" (take only the last result from a sequence of
>> expressions).  I'll also note that I asked if anyone else had a
>> real-life example, and got no responses.

[MRAB ]
> Could a semicolon in a parenthesised expression be an equivalent to C's
> "comma operator"?

I expect it could, but I it's been many years since I tried hacking
Python's grammar, and I wouldn't want a comma operator anyway ;-)

[snip]
Just reading this:

https://www.bfilipek.com/2018/04/refactoring-with-c17-stdoptional.html

about C++17, and what did I see? An example with a semicolon in parentheses!

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] assignment expressions: an alternative proposal

[MRAB]

On 2018-04-24 23:32, Cameron Simpson wrote:

On 24Apr2018 08:51, Ethan Furman  wrote:

When I compare to variables from outer scopes they *usually* are on
the *right* side of '=='.


You mean something like

 if 2 == x:

?  I never write code like that, and I haven't seen it, either.


Just to this, I also never write code like that but I've certainly seen it
advocated.

I think the rationale was that it places the comparison value foremost in one's
mind, versus the name being tested. I'm not persuaded, but it is another
subjective situation.

It's sometimes advocated in C/C++ code to help catch the inadvertent use 
of = instead of ==, but that's not a problem in Python.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Drop/deprecate Tkinter?

[MRAB]

On 2018-05-02 22:56, Josh Stephens wrote:

Hello list,

   If I may voice my opinion I would like to say that I just built an
application using Tkinter using python3. I used it because it was
included in python by default and I didn't have to using something
like PyQT or any other framework that was heavy. While I agree that
the docs can sometimes be confusing, I am not sure that it warrants
tossing it out. I am not even sure that my opinion gives much weight
but I figured I would just toss in a quick here is my vote and my
story about using Tkinter with SqlAlchemy and Py2App to build a native
Mac OS X app as of last month.

I have a few applications that use tkiner. Whilst it has its 
limitations, it's too useful to throw out.


It's a battery, not a power station, but batteries have their uses.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Drop/deprecate Tkinter?

[MRAB]

On 2018-05-03 13:24, Steve Holden wrote:
On Thu, May 3, 2018 at 12:12 AM, Ivan Pozdeev via Python-Dev 
mailto:python-dev@python.org>> wrote:


On 03.05.2018 1:01, Antoine Pitrou wrote:

On Wed, 2 May 2018 22:54:04 +0100
Paul Moore mailto:p.f.mo...@gmail.com>> wrote:

On 2 May 2018 at 22:37, Antoine Pitrou mailto:solip...@pitrou.net>> wrote:

To elaborate a bit: the OP, while angry, produced both a
detailed
analysis *and* a PR.  It's normal to be angry when an
advertised
feature doesn't work and it makes you lose hours of work
(or, even,
forces you to a wholesale redesign). Producing a
detailed analysis and a
PR is more than most people will ever do.

His *other* email seems reasonable, and warrants a response,
yes. But
are we to take the suggestion made here (to drop tkinter)
seriously,
based on the fact that there's a (rare - at least it appears
that the
many IDLE users haven't hit it yet) race condition that
causes a crash
in Python 2.7? (It appears that the problem doesn't happen
in the
python.org  3.x builds, if I understand
the description of the issue).

In 3.x, Tkinter+threads is broken too, albeit in a different way --
see https://bugs.python.org/issue33412
 (this should've been the 2nd
link in the initial message, sorry for the mix-up).


​The observation in t​hat issue that tkinter and threads should be 
handled in specific ways is certainly a given for old hands, who have 
long put the GUI code in one thread with one or more concurrent worker 
threads typically communicating through queues. But I haven't built 
anything like that recently, so I couldn't say how helpful the current 
documenation might be.


Interacting with the GUI only in the main thread is something that I've 
had to do in other languages (it is/was the recommended practice), so I 
naturally do the same with Python and tkinter. It's also easier to 
reason about because you don't get elements of the GUI changing 
unexpectedly.


[snip]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Slow down...

[MRAB]

On 2018-05-07 19:49, Craig Rodrigues wrote:



[snip]


Would it be reasonable to request a 10 year moratorium on making changes 
to the core Python language,
and for the next 10 years only focus on things that do not require core 
language changes,
such as improving/bugfixing existing libraries, writing new libraries, 
improving tooling, improving infrastructure (PyPI),

improving performance, etc., etc.?

There are still many companies still stuck on Python 2, so giving 10 
years of breathing room
for these companies to catch up to Python 3 core language, even past 
2020 would be very helpful.



[snip]

I don't see how a 10 year moratorium will help those still on Python 2, 
given that Python 2.7 has been around for almost 8 years already, by 
2020 it will have been 10 years, and it was made clear that it would be 
the last in the Python 2 line.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Unicode 11.0.0 released

[MRAB]

Unicode 11.0.0 has been released. Will Python 3.7 be updated to it, or 
is it too late?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode 11.0.0 released

[MRAB]

On 2018-06-07 08:40, Ned Deily wrote:

On Jun 6, 2018, at 00:22, Benjamin Peterson  wrote:
> On Tue, Jun 5, 2018, at 12:17, MRAB wrote:
>> Unicode 11.0.0 has been released. Will Python 3.7 be updated to it, or 
>> is it too late?
> 
> https://github.com/python/cpython/pull/7439 will update 3.8. Generally, we've considered updating the Unicode database to be a feature and not backported updates to bugfix branches. Thus, tradition would seem to exclude Unicode 11.0.0 from landing so late in 3.7's release cycle. That said, the update is highly unlikely to break anything. It's up to Ned.


I'd hate for 3.7 to fall behind in the emoji race :)
The Python community _is_ meant to be inclusive, and we should support 
the addition of ginger emoijs. :-)

   Seriously, there are some important additions that will, no doubt, appear in 
platforms over the lifetime of 3.7 and the risk is very low.  Thanks for 
pinging about it.

https://github.com/python/cpython/pull/7470


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] segfaults due to hash randomization in C OrderedDict

[MRAB]

On 2015-05-21 15:55, Eric Snow wrote:

(see http://bugs.python.org/issue16991)

I an working on resolving an intermittent segfault that my C
OrderedDict patch introduces.  The failure happens in
test_configparser (RawConfigParser uses OrderedDict internally), but
only sporadically.  However, Ned pointed out to me that it appears to
be related to hash randomization, which I have verified.  I'm looking
into it.

In the meantime, here's a specific question.  What would lead to the
pattern of failures I'm seeing?  I've verified that the segfault
happens consistently for certain hash randomization seeds and never
for the rest.  I don't immediately recognize the pattern but expect
that it would shed some light on where the problem lies.  I ran the
following command with the OrderedDict patch applied:

   for i in `seq 1 100`; do echo $i; PYTHONHASHSEED=$i ./python -m
test.regrtest -m test_basic test_configparser ; done

Through 100 I get segfaults with seeds of 7, 15, 35, 37, 39, 40, 42,
47, 50, 66, 67, 85, 87, 88, and 92.  I expect the distribution across
all seeds is uniform, but I haven't verified that.

Thoughts?


In "_odict_get_index", for example (there are others), you're caching
"ma_keys":

PyDictKeysObject *keys = ((PyDictObject *)od)->ma_keys;

If it resizes, you go back to the label "start", which is after that
line, but could "ma_keys" change when it's resized?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] segfaults due to hash randomization in C OrderedDict

[MRAB]

On 2015-05-21 22:52, Eric Snow wrote:
> Good catch.  Unfortunately, sticking "keys = ((PyDictObject
> *)od)->ma_keys;" right after "hash = ..." did not make a difference.
> I still get the same segfault.

So, does it change sometimes?

>
> On Thu, May 21, 2015 at 11:17 AM, MRAB  
wrote:

> > On 2015-05-21 15:55, Eric Snow wrote:
> >>
> >> (see http://bugs.python.org/issue16991)
> >>
> >> I an working on resolving an intermittent segfault that my C
> >> OrderedDict patch introduces.  The failure happens in
> >> test_configparser (RawConfigParser uses OrderedDict internally), but
> >> only sporadically.  However, Ned pointed out to me that it appears to
> >> be related to hash randomization, which I have verified.  I'm looking
> >> into it.
> >>
> >> In the meantime, here's a specific question.  What would lead to the
> >> pattern of failures I'm seeing?  I've verified that the segfault
> >> happens consistently for certain hash randomization seeds and never
> >> for the rest.  I don't immediately recognize the pattern but expect
> >> that it would shed some light on where the problem lies.  I ran the
> >> following command with the OrderedDict patch applied:
> >>
> >>for i in `seq 1 100`; do echo $i; PYTHONHASHSEED=$i ./python -m
> >> test.regrtest -m test_basic test_configparser ; done
> >>
> >> Through 100 I get segfaults with seeds of 7, 15, 35, 37, 39, 40, 42,
> >> 47, 50, 66, 67, 85, 87, 88, and 92.  I expect the distribution across
> >> all seeds is uniform, but I haven't verified that.
> >>
> >> Thoughts?
> >>
> > In "_odict_get_index", for example (there are others), you're caching
> > "ma_keys":
> >
> > PyDictKeysObject *keys = ((PyDictObject *)od)->ma_keys;
> >
> > If it resizes, you go back to the label "start", which is after that
> > line, but could "ma_keys" change when it's resized?
> >

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] segfaults due to hash randomization in C OrderedDict

[MRAB]

On 2015-05-21 23:17, Eric Snow wrote:
> On Thu, May 21, 2015 at 4:06 PM, MRAB  wrote:
> > On 2015-05-21 22:52, Eric Snow wrote:
> >> Good catch.  Unfortunately, sticking "keys = ((PyDictObject
> >> *)od)->ma_keys;" right after "hash = ..." did not make a difference.
> >> I still get the same segfault.
> >
> > So, does it change sometimes?
>
> The segfault is consistent if I use the same seed (e.g. 7):
>
>   PYTHONHASHSEED=7 ./python -m test.regrtest -m test_basic 
test_configparser

>
> Some seeds always segfault and some seeds never segfault.
>
OK, another thought.

In "_odict_get_index" again, you say that if the hash has changed, the 
dict might've

been resized, but could the dict be resized _without_ the hash changing?

Could the value of "keys" still become invalid even if the hash is the same?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] segfaults due to hash randomization in C OrderedDict

[MRAB]

On 2015-05-22 00:22, Eric Snow wrote:

On Thu, May 21, 2015 at 4:41 PM, MRAB  wrote:
> On 2015-05-21 23:17, Eric Snow wrote:
>> The segfault is consistent if I use the same seed (e.g. 7):
>>
>>   PYTHONHASHSEED=7 ./python -m test.regrtest -m test_basic
>> test_configparser
>>
>> Some seeds always segfault and some seeds never segfault.
>>
> OK, another thought.
>
> In "_odict_get_index" again, you say that if the hash has changed, the dict
> might've
> been resized, but could the dict be resized _without_ the hash changing?
>
> Could the value of "keys" still become invalid even if the hash is the same?

Good question.  The only way I can see here that the dict would resize
is during re-entrance to the interpreter eval loop via Python code
potentially triggered through the PyObject_Hash call.

Also, there's no check for a changed hash.  The code compares the size
of ma_keys (effectively the dict keys hash table) against the size of
of the odict "fast nodes" table.

Ah, OK.

I'm not looking at the use of "PyTuple_Pack". As I understand it, 
"PyTuple_Pack" borrows the
references of the objects passed, and when the tuple itself is DECREFed, 
those objects will be

DECREFed

"odict_reduce" calls "PyTuple_Pack", passing 1 or 2 references to 
Py_None which aren't INCREFed
first, so could there be a bug there? (There might be similar issues in 
other functions.)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] segfaults due to hash randomization in C OrderedDict

[MRAB]

On 2015-05-22 01:12, Eric Snow wrote:

On Thu, May 21, 2015 at 5:55 PM, MRAB  wrote:
> I'm not looking at the use of "PyTuple_Pack". As I understand it,
> "PyTuple_Pack" borrows the
> references of the objects passed, and when the tuple itself is DECREFed,
> those objects will be
> DECREFed

>From the docs [1] it seems that PyTuple_Pack does not steal any
references and it returns a new reference.  Perhaps you were thinking
of PyTuple_SetItem (and PyTuple_SET_ITEM)?

[1] https://docs.python.org/3.5//c-api/tuple.html

>
> "odict_reduce" calls "PyTuple_Pack", passing 1 or 2 references to Py_None
> which aren't INCREFed
> first, so could there be a bug there? (There might be similar issues in
> other functions.)

Alas, I don't think it is. :(

I'd come to the same conclusion.

Oh, well, I'll keep looking...

I'll point out that the configparser test in question does a lot of
resizes.  It may be that the problem only surfaces after many resizes
and apparently only for certain hash randomization seeds.  At the
moment I'm looking at how hash randomization impacts resizing.  I'm
certainly seeing that the resizes happen at different item counts
depending on the seed.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Unable to build regex module against Python 3.5 32-bit

[MRAB]

As the subject says, I've been unable to build the regex module against
Python 3.5b1 for 32-bit. MingGW says:

skipping incompatible .../libpython35.a when searching for -lpython35

It builds without a problem against Python 3.5 for 64-bit.

Any ideas? Should I just wait until beta 2?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

[MRAB]

On 2015-05-25 21:09, Ryan Gonzalez wrote:
> Try building the module with -m32. The error message basically means: 
"../libpython35.a is 32-bit, but what you're building is 64-bit." Gotta 
love ld!

>
Unless I've missing something, I'm already passing it to gcc.

All of the other versions build without a problem: Python 2.5-2.7 and 
Python 3.1-3.4, both 32-bit and 64-bit, and now Python 3.5 64-bit.


That's 15 building and 1 failing!

> On May 25, 2015 3:06:01 PM CDT, MRAB  wrote:
>
> As the subject says, I've been unable to build the regex module 
against

> Python 3.5b1 for 32-bit. MingGW says:
>
>  skipping incompatible .../libpython35.a when searching for 
-lpython35

>
> It builds without a problem against Python 3.5 for 64-bit.
>
> Any ideas? Should I just wait until beta 2?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

[MRAB]

On 2015-05-25 22:59, Paul Moore wrote:
> On 25 May 2015 at 21:06, MRAB  wrote:
> > As the subject says, I've been unable to build the regex module against
> > Python 3.5b1 for 32-bit. MingGW says:
> >
> > skipping incompatible .../libpython35.a when searching for 
-lpython35

> >
> > It builds without a problem against Python 3.5 for 64-bit.
> >
> > Any ideas? Should I just wait until beta 2?
>
> MinGW is (and always has been) only marginally supported,
> unfortunately. I'd rather it didn't break totally for 3.5, but I am
> anticipating some difficulties (there have been a lot of
> compiler-related changes with 3.5).
>
> Could you raise a bug, including details of precisely how you tried to
> build the module (presumably
> https://pypi.python.org/pypi/regex/2015.05.10) and assign it to me?
> I'll try to take a look and reproduce the issue. With luck, it may be
> as simple as the wrong version of libpython35.a being picked up
> somewhere.
>
> Just to check the obvious - you *are* using 32-bit Python 3.5b1 and a
> 32-bit Mingw to build the 32-bit version, and 64-bit Python 3.5b1 and
> a 64-bit Mingw to build the 64-bit one? (I.e., two installations of
> Python and two of Mingw)
>
I'm not sure what happened, but I'm now getting this for Python 3.5 
(32-bit):


C:\Python35(32-bit)\libs/libpython35.a(dsxbs01290.o):(.idata$7+0x0): 
undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a'
C:\Python35(32-bit)\libs/libpython35.a(dsxbs00283.o):(.idata$7+0x0): 
undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a'
C:\Python35(32-bit)\libs/libpython35.a(dsxbs00291.o):(.idata$7+0x0): 
undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a'
C:\Python35(32-bit)\libs/libpython35.a(dsxbs00273.o):(.idata$7+0x0): 
undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a'
C:\Python35(32-bit)\libs/libpython35.a(dsxbs00255.o):(.idata$7+0x0): 
undefined reference to `_head_C__build_cpython_PCBuild_win32_libpython35_a'
C:\Python35(32-bit)\libs/libpython35.a(dsxbs01280.o):(.idata$7+0x0): 
more undefined references to 
`_head_C__build_cpython_PCBuild_win32_libpython35_a' follow

collect2: ld returned 1 exit status


All other builds, from Python 2.5 to Python 3.4, both 32-bit and 64-bit, 
and also Python 3.5 (64-bit), work.


The 32-bit Python says it's 32-bit and the 64-bit Python says it's 64-bit.

---8<---

C:

rem Compile for Python 3.5 (64-bit) [works]
cd C:\MinGW64\bin
"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35\include" -c 
"D:\projects\mrab-regex\regex_3\regex\_regex_unicode.c" -o 
"D:\projects\mrab-regex\regex_3\Release(3.5)\_regex_unicode.o"
"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35\include" -c 
"D:\projects\mrab-regex\regex_3\regex\_regex.c" -o 
"D:\projects\mrab-regex\regex_3\Release(3.5)\_regex.o"
"C:\MinGW64\bin\gcc.exe" -m64 -shared -s 
"D:\projects\mrab-regex\regex_3\Release(3.5)\_regex_unicode.o" 
"D:\projects\mrab-regex\regex_3\Release(3.5)\_regex.o" 
-L"C:\Python35\libs" -lpython35 -o 
"D:\projects\mrab-regex\regex_3\Release(3.5)\_regex.pyd"


rem Compile for Python 3.5 (32-bit) [fails]
cd C:\MinGW\bin
"C:\MinGW\bin\gcc.exe" -mdll -m32  -O2 -Wall -Wsign-compare -Wconversion 
-I"C:\Python35(32-bit)\include" -c 
"D:\projects\mrab-regex\regex_3\regex\_regex_unicode.c" -o 
"D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)\_regex_unicode.o"
"C:\MinGW\bin\gcc.exe" -mdll -m32  -O2 -Wall -Wsign-compare -Wconversion 
-I"C:\Python35(32-bit)\include" -c 
"D:\projects\mrab-regex\regex_3\regex\_regex.c" -o 
"D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)\_regex.o"
"C:\MinGW\bin\gcc.exe" -m32 -shared -s 
"D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)\_regex_unicode.o" 
"D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)\_regex.o" 
-L"C:\Python35(32-bit)\libs" -lpython35 -o 
"D:\projects\mrab-regex\regex_3\Release(3.5)(32-bit)\_regex.pyd"


---8<---


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

[MRAB]

On 2015-05-26 14:24, Paul Moore wrote:
> On 26 May 2015 at 13:55, Steve Dower  wrote:
> > The builds I am responsible for include it because someone reported 
an issue

> > and was persistent and helpful enough that I fixed it for them.
> >
> > That said, until MinGW agrees on a stable branch/version/fork, 
there seems
> > to be a good chance that the shipped lib won't work for some 
people. If this
> > is what's happened here, I see it as a good enough reason to stop 
shipping
> > the lib and to add instructions on generating it instead (the 
gendef/dlltool

> > dance).
>
> Agreed. If shipping it helps, then great. If it's going to cause bug
> reports, let's go back to the status quo of not having it. The
> instructions for generating it were in the old distutils docs, now
> removed in the cleanup / redirection to packaging.python.org. I'm
> inclined to just leave it undocumented - the people who need it know
> how to do it or can find it, whereas documenting the process implies a
> level of support that we're not yet really able to provide.
>
> Let's wait to see what the OP comes back with before making a final
> decision. I see a few questions:
>
> 1. Does using distutils work (as opposed to the quoted manual compile 
steps)?

> 2. Does using whatever process he used in the past to generate
> libpythonXY.a result in a working version?
>
> https://docs.python.org/2.7/install/index.html#gnu-c-cygwin-mingw
> suggests using pexports rather than gendef. I don't know if that could
> produce a different result, but I can't see how... It also implicitly
> assumes you're using dlltool from the toolchain you'll be building
> with rather than using -m. Again, I can't see why that would affect
> the results.
>
It's been so long since the last Python release that I didn't remember 
that I had to make the lib file.


I made libpython35.a like I did for the others and it's all working now. 
All the tests pass. :-)


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

[MRAB]

On 2015-05-26 18:27, Paul Moore wrote:
> On 26 May 2015 at 18:23, MRAB  wrote:
> > I made libpython35.a like I did for the others and it's all working 
now. All

> > the tests pass. :-)
>
> Cool. How did you make libpython35.a? Sounds like there is some
> difference in the process being used for the shipped version.
>
For making the .def files, I used:

C:\MinGW64\x86_64-w64-mingw32\bin\gendef.exe

MinGW didn't contain gendef.exe!

For making the .a files, I used:

C:\MinGW64\bin\dlltool.exe

for the 64-bit builds and:

C:\MinGW\bin\dlltool.exe

for the 32-bit builds.

They all worked.

When I tried:

C:\MinGW64\bin\dlltool.exe

or:

C:\MinGW64\x86_64-w64-mingw32\bin\dlltool.exe

for the 32-bit builds, they wouldn't link.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

[MRAB]

On 2015-05-27 09:25, Paul Moore wrote:

On 27 May 2015 at 09:10, Nick Coghlan  wrote:

The old distutils docs aren't gone, the top level links just moved to the
distutils package docs: https://docs.python.org/3/library/distutils.html

I kept them (with the same deep link URLs) because I know there's stuff in
there that isn't currently documented anywhere else. I moved them to a more
obscure location because there's also stuff in there that's thoroughly
outdated, and it's a non-trivial task to figure out which is which and move
the still useful stuff to a more appropriate home :)


Thanks.

Your plan worked perfectly, because I never knew they were there :-)

https://docs.python.org/3/install/index.html#older-versions-of-python-and-mingw
implies that the libpythonXY.a files are only needed in older versions
of Python/mingw. I don't know how true that is, although I do know
that mingw should be able to link directly to a DLL without needing a
lib file.

It would be interesting to know if MRAB's build process can use the
DLL, rather than requiring a lib file (or for that matter if distutils
works without the lib file!)


Steve Dower's post has prompted me to look again at building the regex
module for Python 3.5, 32-bit and 64-bit, using just Mingw64 and
linking against python32.dll. It works!

Earlier versions of Python, however, including Python 2.7, still seem
to want libpython??.a.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unable to build regex module against Python 3.5 32-bit

[MRAB]

On 2015-06-05 01:37, MRAB wrote:

On 2015-05-27 09:25, Paul Moore wrote:

On 27 May 2015 at 09:10, Nick Coghlan  wrote:

The old distutils docs aren't gone, the top level links just moved to the
distutils package docs: https://docs.python.org/3/library/distutils.html

I kept them (with the same deep link URLs) because I know there's stuff in
there that isn't currently documented anywhere else. I moved them to a more
obscure location because there's also stuff in there that's thoroughly
outdated, and it's a non-trivial task to figure out which is which and move
the still useful stuff to a more appropriate home :)


Thanks.

Your plan worked perfectly, because I never knew they were there :-)

https://docs.python.org/3/install/index.html#older-versions-of-python-and-mingw
implies that the libpythonXY.a files are only needed in older versions
of Python/mingw. I don't know how true that is, although I do know
that mingw should be able to link directly to a DLL without needing a
lib file.

It would be interesting to know if MRAB's build process can use the
DLL, rather than requiring a lib file (or for that matter if distutils
works without the lib file!)


Steve Dower's post has prompted me to look again at building the regex
module for Python 3.5, 32-bit and 64-bit, using just Mingw64 and
linking against python32.dll. It works!

Earlier versions of Python, however, including Python 2.7, still seem
to want libpython??.a.


For reference, here's how I can build the regex module on Windows 8.1,
64-bit, using only MinGW64.

For Python 3.5, I can link against "python35.dll", but for earlier
versions, including Python 2.7, I need "libpython.a".

I have built regex module for all of the 16 supported versions of
Python (2.5-2.7, 3.1-3.5, 64-bit and 32-bit) and they have all passed
the tests.


rem For Python 3.5, 64-bit.
rem Can link against the Python DLL.

rem Compile
"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35-64\include" -c 
"D:\mrab-regex\source\_regex_unicode.c" -o 
"D:\mrab-regex\release\3.5-64\_regex_unicode.o"


"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35-64\include" -c 
"D:\mrab-regex\source\_regex.c" -o "D:\mrab-regex\release\3.5-64\_regex.o"


rem Link
"C:\MinGW64\bin\gcc.exe" -m64 -shared -s 
"D:\mrab-regex\release\3.5-64\_regex_unicode.o" 
"D:\mrab-regex\release\3.5-64\_regex.o" -L"C:\Python35" -lpython35 -o 
"D:\mrab-regex\release\3.5-64\_regex.pyd"



rem For Python 3.5, 32-bit.
rem Can link against the Python DLL.

rem Compile
"C:\MinGW64\bin\gcc.exe" -mdll -m32  -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35-32\include" -c 
"D:\mrab-regex\source\_regex_unicode.c" -o 
"D:\mrab-regex\release\3.5-32\_regex_unicode.o"


"C:\MinGW64\bin\gcc.exe" -mdll -m32  -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python35-32\include" -c 
"D:\mrab-regex\source\_regex.c" -o "D:\mrab-regex\release\3.5-32\_regex.o"


rem Link
"C:\MinGW64\bin\gcc.exe" -m32 -shared -s 
"D:\mrab-regex\release\3.5-32\_regex_unicode.o" 
"D:\mrab-regex\release\3.5-32\_regex.o" -L"C:\Python35-32" -lpython35 -o 
"D:\mrab-regex\release\3.5-32\_regex.pyd"



rem For Python 3.4, 64-bit.
rem Need to link against the Python .a file.

rem Make libpython34.a
"C:\MinGW64\x86_64-w64-mingw32\bin\gendef.exe" - 
"C:\Windows\System32\python34.dll" >"C:\Python34-64\libs\libpython34.def"


"C:\MinGW64\bin\dlltool.exe" --dllname python34.dll --def 
"C:\Python34-64\libs\libpython34.def" --output-lib 
"C:\Python34-64\libs\libpython34.a"


rem Compile
"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python34-64\include" -c 
"D:\mrab-regex\source\_regex_unicode.c" -o 
"D:\mrab-regex\release\3.4-64\_regex_unicode.o"


rem Link
"C:\MinGW64\bin\gcc.exe" -mdll -m64 -DMS_WIN64 -O2 -Wall -Wsign-compare 
-Wconversion -I"C:\Python34-64\include" -c 
"D:\mrab-regex\source\_regex.c" -o "D:\mrab-regex\release\3.4-64\_regex.o"


"C:\MinGW64\bin\gcc.exe" -m64 -shared -s 
"D:\mrab-regex\release\3.4-64\_regex_unicode.o" 
"D:\mrab-regex\release\3.4-64\_regex.o" -L"C:\Python34-64\libs" 
-lpython34 -o "D:\mrab-regex\release\3.4-64\_regex.pyd"



rem For Python 3.4, 32-bit.
rem Need to link against the Python .a file.

rem Make libpython34.a
"C:\MinGW64\x86_64-w64-mingw32\bin\gendef.exe" - 
"C:\Windows\SysWOW64\python34.dll" >"C:\Python34-32\libs\libpython34.def"


"

Re: [Python-Dev] Obtaining stack-frames from co-routine objects

[MRAB]

On 2015-06-13 11:38, jaivish kothari wrote:

Hi ,

I had a Question,i hope i'll find the solution here.

Say i have a Queue.
  >>> h = Queue.Queue(maxsize=0)
 >>> h.put(1)
 >>> h.put(2)
 >>> h.empty()
False
 >>> h.join()
 >>> h.empty()
False
 >>> h.get()
1
 >>> h.get()
2
 >>> h.get()
Blocked...

My Question is :
In a single threaded environment why does the get() gets blocked ,
instead of raising an exception.On interpreter i have no way to resume
working.

And my second question is :
Why doe we have to explicitly call task_done after get(). why doesn't
get() implicitly call task_done().
as for put() entry for unfinished_task is automatically added .why not
get deleted in get() then.


The way it's used is to get a item, process it, and then signal that it
has finished with it. There's going to be a period of time when an item
is not in the queue, but is still being processed.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode 8.0 and 3.5

[MRAB]

On 2015-06-18 19:33, Larry Hastings wrote:

On 06/18/2015 11:27 AM, Terry Reedy wrote:

Unicode 8.0 was just released.  Can we have unicodedata updated to
match in 3.5?



What does this entail?  Data changes, code changes, both?


It looks like just data changes.

There are additional codepoints and a renamed property (which the
standard library doesn't support anyway).

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Unicode 8.0 and 3.5

[MRAB]

On 2015-06-19 00:56, Steven D'Aprano wrote:

On Thu, Jun 18, 2015 at 08:34:14PM +0100, MRAB wrote:

On 2015-06-18 19:33, Larry Hastings wrote:
>On 06/18/2015 11:27 AM, Terry Reedy wrote:
>>Unicode 8.0 was just released.  Can we have unicodedata updated to
>>match in 3.5?
>>
>
>What does this entail?  Data changes, code changes, both?
>
It looks like just data changes.


At the very least, there is a change to the casefolding algorithm.
Cherokee was classified as unicameral but is now considered bicameral
(two cases, like English). Unusually, case-folding Cherokee maps to
uppercase rather than lowercase.


Doesn't the case-folding just depend on the data and the algorithm
remains the same?


The full set of changes is listed here:

http://unicode.org/versions/Unicode8.0.0/

Apart from the addition of 7716 characters and changes to
str.casefold(), I don't think any of the changes will make a big
difference to Python's implementation. But it would be good to support
Unicode 8 (to the degree that Python actually does support Unicode,
rather than just that character set part of it).



There are additional codepoints and a renamed property (which the
standard library doesn't support anyway).


Which one are you referring to, Indic_Matra_Category renamed to
Indic_Positional_Category?


Yes.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How far to go with user-friendliness

[MRAB]

On 2015-07-14 14:09, Nick Coghlan wrote:

On 14 July 2015 at 22:53, Xavier Morel  wrote:


On 2015-07-14, at 14:39 , Nick Coghlan  wrote:


On 14 July 2015 at 22:06, Dima Tisnek  wrote:

Thus the question, how far should Python go to detect possible
erroneous user behaviour?

Granted it is in tests only, but why not detect assrte, sasert, saster
and assrat?


Because "r" and "e" are right next to each other on a QWERTY keyboard
(so it's an easy typo to make), and transposing them doesn't change
the overall shape of the word (so it's a hard typo to detect).



If you get the "a" or "t" out of position you change the shape of the
word so typos involving those are easier to detect visually, while "s"
and "e" are on different keyboard rows so typos at that point in the
word are less likely in the first place.


"sasert" fits these rules though.


That changes one of the end letters, and human word recognition is
generally pretty good at picking up changes to the first and last
letter. It's the ones in the middle that can be a problem, especially
when they're all the same height like "sser". And that's with typical
word recognition capabilities - my understanding is that for folks
that actually have dyslexia, discrepancies in letter order are even
harder to detect.

Dima's right that the main defence against this kind of error is
actually linters and IDEs, but detecting this particular one at
runtime is harmless, so there's no particular reason *not* to do it
when it's possible to construct a reasonable rationale for "Why this
particular typo?" and not all the other possible ways of transposing
adjacent letters in "assert".


What about "aasert"?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] How far to go with user-friendliness

[MRAB]

On 2015-07-14 23:05, Ethan Furman wrote:

On 07/14/2015 02:53 PM, Robert Collins wrote:

On 15 July 2015 at 09:41, A.M. Kuchling  wrote:

On Tue, Jul 14, 2015 at 09:53:33AM -0700, Ethan Furman wrote:

Part of writing tests is making sure they fail (and for the right reason) -- 
proper testing of the tests would reveal such a typo.


And there are other failure modes for writing tests that succeed but
are not testing what you think.  For example, you might re-use the
same method name:

def test_connection(self):
# Never executed
...

... 200 lines and 10 other test methods later ...

def test_connection(self):
...

Or misuse assertRaises:

with self.assertRaises(TypeError):
1 + "a"
# Second statement never reached
[] + 'b'

I don't think unittest can protect its users from such things.


It can't, but there is a sliding scale of API usability, and we should
try to be up the good end of that :).


I hope you're not suggesting that supporting misspellings, and thereby ruling 
out the proper use of an otherwise fine variable name, is at the good end of 
that scale?


Somewhat OT, but did you know that the Unicode "Line_Break" property
has "Inseparable" as one of its possible values, and that "Inseperable"
is a permitted alias of it? 

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status on PEP-431 Timezones

[MRAB]

On 2015-07-27 15:46, Lennart Regebro wrote:

On Mon, Jul 27, 2015 at 4:13 PM, Steven D'Aprano  wrote:

To me, Paul's example is a datetime operation: you start with a datetime
(7am today), perform arithmetic on it by adding a period of time (one
day), and get a datetime as the result (7am tomorrow).


Well, OK, let's propose these wordings: It looks like a date
operation, ie, add one to the date, but in reality it's a time
operation, ie add 86400 seconds to the time. These things sound
similar but are very different.

I called it a "calendar" operation, because these operation include
such things as "add one year", where you expect to get the 27th of
July 2016, but you will get the 26th if you use a timedelta, because
2016 is a leap year. So we need to separate date (or calendar)
operations from time operations. The same thing goes with months, add
30 days, and you'll sometimes get the same result as if you add one
month and sometimes not.


Also, if you "add one year" to 29 February 2016, what date do you get?


timedelta adds time, not days, month or years. Except when you cross a
DST border where it suddenly, surprisingly and intentionally may add
more or less time than you told it to.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status on PEP-431 Timezones

[MRAB]

On 2015-07-27 15:59, Paul Moore wrote:

On 27 July 2015 at 14:59, R. David Murray  wrote:

I have a feeling that I'm completely misunderstanding things, since
tzinfo is still a bit of a mystery to me.


You're not the only one :-)

I think the following statements are true. If they aren't, I'd
appreciate clarification. I'm going to completely ignore leap seconds
in the following - I hope that's OK, I don't understand leap seconds
*at all* and I don't work in any application areas where they are
relevant (to my knowledge) so I feel that for my situation, ignoring
them (and being able to) is reasonable.

Note that I'm not talking about internal representations - this is
purely about user-visible semantics.


Would it help if it was explicit and we had LocalDateTime and
UTCDateTime?


1. "Naive" datetime arithmetic means treating a day as 24 hours, an
hour as 60 minutes, etc. Basically base-24/60/60 arithmetic.
2. If you're only working in a single timezone that's defined as UTC
or a fixed offset from UTC, naive arithmetic is basically all there
is.
3. Converting between (fixed offset) timezones is a separate issue
from calculation - but it's nothing more than applying the relevant
offsets.
4. Calculations involving 2 different timezones (fixed-offset ones as
above) is like any other exercise involving values on different
scales. Convert both values to a common scale (in this case, a common
timezone) and do the calculation there. Simple enough.
5. The problems all arise *only* with timezones whose UTC offset
varies depending on the actual time (e.g., timezones that include the
transition to DST and back).

Are we OK to this point? This much comprises what I would class as a
"naive" (i.e. 99% of the population ;-)) understanding of datetimes.

The stdlib datetime module handles naive datetime values, and
fixed-offset timezones, fine, as far as I can see. (I'm not sure that
the original implementation included fixed-offset tzinfo objects, but
the 3.4 docs say they are there now, so that's fine).

Looking at the complicated cases, the only ones I'm actually aware of
in practice are the ones that switch to DST and back, so typically
have two offsets that differ by an hour, switching between the two at
some essentially arbitrary points. If there are other more complex
forms of timezone, I'd like to never need to know about them, please
;-)

The timezones we're talking about here are things like
"Europe/London", not "GMT" or "BST" (the latter two are fixed-offset).

There are two independent issues with complex timezones:

1. Converting to and from them. That's messy because the conversion to
UTC needs more information than just the date & time (typically, for
example, there is a day when 01:45:00 maps to 2 distinct UTC times).
This is basically the "is_dst" bit that Tim discussed in an earlier
post. The semantic issue here is that users typically say "01:45" and
it never occurs to them to even think about *which* 01:45 they mean.
So recovering that extra information is hard (it's like dealing with
byte streams where the user didn't provide details of the text
encoding used). Once we have the extra information, though, doing
conversions is just a matter of applying a set of rules.

2. Arithmetic within a complex timezone. Theoretically, this is simple
enough (convert to UTC, do the calculation naively, and convert back).
But in practice, that approach doesn't always match user expectations.
So you have 2 mutually incompatible semantic options - 1 day after 4pm
is 3pm the following day, or adding 1 day adds 25 hours - either is a
viable choice, and either will confuse *some* set of users. This, I
think, is the one where all the debate is occurring, and the one that
makes my head explode.

It seems to me that the problem is that for this latter issue, it's
the *timedelta* object that's not rich enough. You can't say "add 1
day, and by 1 day I mean keep the same time tomorrow" as opposed to
"add 1 day, and by that I mean 24 hours"[1]. In some ways, it's
actually no different from the issue of adding 1 month to a date
(which is equally ill-defined, but people "know what they mean" to
just as great an extent). Python bypasses the latter by not having a
timedelta for "a month". C (and the time module) bypasses the former
by limiting all time offsets to numbers of seconds - datetime gave us
a richer timedelta object and hence has extra problems.

I don't have any solutions to this final issue. But hopefully the
above analysis (assuming it's accurate!) helps clarify what the actual
debate is about, for those bystanders like me who are interested in
following the discussion. With luck, maybe it also gives the experts
an alternative perspective from which to think about the problem - who
knows?

Paul

[1] Well, you can, actually - you say that a timedelta of "1 day"
means "the same time tomorrow" and if you want 24 hours, you say "24
hours" not "1 day". So timedelta(days=1) != timedelta(hours=24) even
though they give the s

Re: [Python-Dev] PEP-498: Literal String Formatting

[MRAB]

On 2015-08-10 01:24, Peter Ludemann via Python-Dev wrote:

What if logging understood lambda? (By testing for types.FunctionType).

[snip]

Why not use 'callable'?

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP-498: Literal String Formatting

[MRAB]

On 2015-08-10 20:23, Guido van Rossum wrote:

On Mon, Aug 10, 2015 at 8:49 PM, Eric V. Smith mailto:e...@trueblade.com>> wrote:

On 08/10/2015 02:44 PM, Yury Selivanov wrote:
> On 2015-08-10 2:37 PM, Eric V. Smith wrote:
>> This is why I think PEP-498 isn't the solution for i18n. I'd really like
>> to be able to say, in a debugging context:
>>
>> print('a:{self.a} b:{self.b} c:{self.c} d:{self.d}')
>>
>> without having to create locals to hold these 4 values.
>
> Why can't we restrict expressions in f-strings to
> attribute/item getters?
>
> I.e. allow f'{foo.bar.baz}' and f'{self.foo["bar"]}' but
> disallow f'{foo.bar(baz=something)}'

It's possible. But my point is that Barry doesn't even want
attribute/item getters for an i18n solution, and I'm not willing to
restrict it that much.


I also don't want to tie this closely to i18n. That is (still) very much
a wold of its own.

What I want with f-strings (by any name) is a way to generalize from
print() calls with multiple arguments. We can write

   print('Todo:', len(self.todo), '; busy:', len(self.busy))

but the same thing is more awkward when you have to pass it as a single
string to a function that just sends one string somewhere. And note that
the above example inserts a space before the ';' which I don't really
like. So it would be nice if instead we could write

   print(f'Todo: {len(self.todo)}; busy: {len(self.busy)}')

which IMO is just as readable as the multi-arg print() call[1], and
generalizes to other functions besides print().

In fact, the latter form has less punctuation noise than the former --
every time you insert an expression in a print() call, you have a
quote+comma before it and a comma+quote after it, compared to a brace
before and one after in the new form. (Note that this is an argument for
using f'{...}' rather than '\{...}' -- for a single interpolation it's
the same amount of typing, but for multiple interpolations,
f'{...}{...}' is actually shorter than '\{...}\{...}', and also the \{
part is ugly.)

Anyway, this generalization from print() is why I want arbitrary
expressions. Wouldn't it be silly if we introduced print() today and
said "we don't really like to encourage printing complicated
expressions, but maybe we can introduce them in a future version"... :-)

Continuing the print()-generalization theme, if things become too long
to fit on a line we can write

   print('Todo:', len(self.todo),
 '; busy:', len(self.busy))

Can we allow the same in f-strings? E.g.

   print(f'Todo: {len(self.todo)
 }; busy: {len(self.busy)
 }')

or is that too ugly? It could also be solved using implicit
concatenation, e.g.

   print(f'Todo: {len(self.todo)}; '
 f'busy: {len(self.busy)}')

[1] Assuming syntax colorizers catch on.


I'd expect f'...' to follow similar rules to '...'.

You could escape it:

print(f'Todo: {len(self.todo)\
}; busy: {len(self.busy)\
}')

which would be equivalent to:

print(f'Todo: {len(self.todo)}; busy: {len(self.busy)}')

or use triple-quoted a f-string:

print(f'''Todo: {len(self.todo)
}; busy: {len(self.busy)
}''')

which would be equivalent to:

print(f'Todo: {len(self.todo)\n}; busy: {len(self.busy)\n}')

(I think it might be OK to have a newline in the expression because
it's wrapped in {...}.)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

[MRAB]

On 2015-08-10 23:54, Victor Stinner wrote:


PEP 498:

"""


Leading whitespace in expressions is skipped


Because expressions may begin with a left brace ('{'), there is a
problem when parsing such expressions. For example:


f'{{k:v for k, v in [(1, 2), (3, 4)]}}'  '{k:v for k, v in [(1, 2), (3, 4)]}'


"""

For me, this example is crazy. You should not add a special case (ignore
spaces) just to support a corner case.


Is it a special case? Don't we already ignore leading spaces in
bracketed expressions?


This example can easily be rewritten using a temporary variable and it
makes the code simpler.

items={k:v for k, v in [(1, 2), (3, 4)]; f'{items}'

Seriously, a dict-comprehension inside a f-string should be considered
as an abuse of the feature. Don't you think so?


True. Or we can wrap it in parentheses. :-)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

[MRAB]

On 2015-08-11 00:26, Victor Stinner wrote:

Le mardi 11 août 2015, Eric V. Smith mailto:e...@trueblade.com>> a écrit :

It sounds like you want to disallow leading spaces just to
disallow this one type of expression.


  I would like to reduce the number of subtle differences between
f-string and str.format().


I'm a little bit surprised at seeing this:

>>> '{0}'.format('foo')
'foo'
>>> '{ 0}'.format('foo')
Traceback (most recent call last):
  File "", line 1, in 
KeyError: ' 0'
>>> '{a}'.format(a='foo')
'foo'
>>> '{ a}'.format(a='foo')
Traceback (most recent call last):
  File "", line 1, in 
KeyError: ' a'

In some other cases, leading and trailing spaces are ignored:

>>> int(' 0 ')
0

Outside string literals, they're also ignored.

But, then:

>>> '{-1}'.format('foo')
Traceback (most recent call last):
  File "", line 1, in 
KeyError: '-1'

It's a string key, even though it looks like an int position.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Can't import tkinter in Python 3.5.0rc1

[MRAB]

As the subject says, I'm unable to import tkinter in Python 3.5.0rc1.

The console says:

C:\Python35>python
Python 3.5.0rc1 (v3.5.0rc1:1a58b1227501, Aug 10 2015, 05:18:45) [MSC 
v.1900 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python35\lib\tkinter\__init__.py", line 35, in 
import _tkinter # If this fails your Python may not be configured 
for Tk

ImportError: DLL load failed: The specified module could not be found.


Is this a known problem?

I'm on Windows 10 Home (64-bit).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Can't import tkinter in Python 3.5.0rc1

[MRAB]

On 2015-08-12 02:05, Steve Dower wrote:
We saw and fixed it before RC 1. I'll check whether that fix didn't 
stick, but go ahead, open an issue and assign me.


It's issue 24847.



From: MRAB <mailto:pyt...@mrabarnett.plus.com>
Sent: ‎8/‎11/‎2015 17:25
To: Python-Dev <mailto:python-dev@python.org>
Subject: [Python-Dev] Can't import tkinter in Python 3.5.0rc1

As the subject says, I'm unable to import tkinter in Python 3.5.0rc1.

The console says:

C:\Python35>python
Python 3.5.0rc1 (v3.5.0rc1:1a58b1227501, Aug 10 2015, 05:18:45) [MSC
v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import tkinter
Traceback (most recent call last):
   File "", line 1, in 
   File "C:\Python35\lib\tkinter\__init__.py", line 35, in 
 import _tkinter # If this fails your Python may not be configured
for Tk
ImportError: DLL load failed: The specified module could not be found.


Is this a known problem?

I'm on Windows 10 Home (64-bit).


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP-498: Literal String Formatting

[MRAB]

On 2015-08-17 23:06, Steve Dower wrote:

On 17Aug2015 0813, Barry Warsaw wrote:

On Aug 18, 2015, at 12:58 AM, Chris Angelico wrote:


The linters could tell you that you have no 'end' or 'start' just as
easily when it's in that form as when it's written out in full.
Certainly the mismatched brackets could easily be caught by any sort
of syntax highlighter. The rules for f-strings are much simpler than,
say, the PHP rules and the differences between ${...} and {$...},
which I've seen editors get wrong.


I'm really asking whether it's technically feasible and realistically possible
for them to do so.  I'd love to hear from the maintainers of pyflakes, pylint,
Emacs, vim, and other editors, linters, and other static analyzers on a rough
technical assessment of whether they can support this and how much work it
would be.


With the current format string proposals (allowing arbitrary
expressions) I think I'd implement it in our parser with a
FORMAT_STRING_TOKEN, a FORMAT_STRING_JOIN_OPERATOR and a
FORMAT_STRING_FORMAT_OPERATOR.

A FORMAT_STRING_TOKEN would be started by f('|"|'''|""") and ended by
matching quotes or before an open brace (that is not escaped).

A FORMAT_STRING_JOIN_OPERATOR would be inserted as the '{', which we'd
either colour as part of the string or the regular brace colour. This
also enables a parsing context where a colon becomes the
FORMAT_STRING_FORMAT_OPERATOR and the right-hand side of this binary
operator would be FORMAT_STRING_TOKEN. The final close brace becomes
another FORMAT_STRING_JOIN_OPERATOR and the rest of the string is
FORMAT_STRING_TOKEN.

So it'd translate something like this:

f"This {text} is my {string:>{length+3}}"

FORMAT_STRING_TOKEN[f"This ]
FORMAT_STRING_JOIN_OPERATOR[{]
IDENTIFIER[text]
FORMAT_STRING_JOIN_OPERATOR[}]
FORMAT_STRING_TOKEN[ is my ]
FORMAT_STRING_JOIN_OPERATOR[{]
IDENTIFIER[string]
FORMAT_STRING_FORMAT_OPERATOR[:]
FORMAT_STRING_TOKEN[>]
FORMAT_STRING_JOIN_OPERATOR[{]
IDENTIFIER[length]
OPERATOR[+]
NUMBER[3]
FORMAT_STRING_JOIN_OPERATOR[}]
FORMAT_STRING_TOKEN[]
FORMAT_STRING_JOIN_OPERATOR[}]
FORMAT_STRING_TOKEN["]


I'm not sure about that. I think it might work better with, say,
FORMAT_OPEN for '{' and FORMAT_CLOSE for '}':

FORMAT_STRING_TOKEN[f"This ]
FORMAT_OPEN
IDENTIFIER[text]
FORMAT_CLOSE
FORMAT_STRING_TOKEN[ is my ]
FORMAT_OPEN
IDENTIFIER[string]
FORMAT_STRING_FORMAT_OPERATOR[:]
FORMAT_STRING_TOKEN[>]
FORMAT_OPEN
IDENTIFIER[length]
OPERATOR[+]
NUMBER[3]
FORMAT_CLOSE
FORMAT_CLOSE
FORMAT_STRING_TOKEN["]


I *believe* (without having tried it) that this would let us produce a
valid tokenisation (in our model) without too much difficulty, and
highlight/analyse correctly, including validating matching braces.
Getting the precedence correct on the operators might be more difficult,
but we may also just produce an AST that looks like a function call,
since that will give us "good enough" handling once we're past tokenisation.

A simpler tokenisation that would probably be sufficient for many
editors would be to treat the first and last segments ([f"This {] and
[}"]) as groupings and each section of text as separators, giving this:

OPEN_GROUPING[f"This {]
EXPRESSION[text]
COMMA[} is my {]
EXPRESSION[string]
COMMA[:>{]
EXPRESSION[length+3]
COMMA[}}]
CLOSE_GROUPING["]

Initial parsing may be a little harder, but it should mean less trouble
when expressions spread across multiple lines, since that is already
handled for other types of groupings. And if any code analysis is
occurring, it should be happening for dict/list/etc. contents already,
and so format strings will get it too.

So I'm confident we can support it, and I expect either of these two
approaches will work for most tools without too much trouble. (There's
also a middle ground where you create new tokens for format string
components, but combine them like the second example.)

Cheers,
Steve


Cheers,
-Barry



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/python%40mrabarnett.plus.com




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Can't install Python 3.5rc3

[MRAB]

I've been unable to install Python 3.5rc3.

It progresses to installing Tcl/tk, then reports that a newer version
of 3.5 is already installed, then it rolls back (slowly).

I don't know where it's finding this "newer version"; I've uninstalled
the previous releases of Python 3.5.

I'm on Windows 10.


Here's the contents of the log file:

[0A40:1CC8][2015-09-08T04:29:45]i001: Burn v3.10.0.1823, Windows v10.0 
(Build 10240: Service Pack 0), path: 
C:\Users\MRAB\Downloads\Python\python-3.5.0rc3.exe
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'ActionLikeInstalling' to value 'Installing'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'ActionLikeInstallation' to value 'Setup'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'ShortVersion' to value '3.5'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'ShortVersionNoDot' to value '35'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'InstallAllUsers' to value '0'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'InstallLauncherAllUsers' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'TargetDir' to value ''
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'DefaultAllUsersTargetDir' to value '[ProgramFilesFolder]Python 
[ShortVersion]'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'TargetPlatform' to value 'x86'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'DefaultJustForMeTargetDir' to value 
'[LocalAppDataFolder]Programs\Python\Python[ShortVersionNoDot]-32'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'OptionalFeaturesRegistryKey' to value 
'Software\Python\PythonCore\[ShortVersion]-32\InstalledFeatures'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'TargetDirRegistryKey' to value 
'Software\Python\PythonCore\[ShortVersion]-32\InstallPath'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'DefaultCustomTargetDir' to value ''
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'InstallAllUsersState' to value 'enabled'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'InstallLauncherAllUsersState' to value 'enabled'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'CustomInstallLauncherAllUsersState' to value 
'[InstallLauncherAllUsersState]'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'TargetDirState' to value 'enabled'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing string variable 
'CustomBrowseButtonState' to value 'enabled'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_core' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_exe' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_dev' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_lib' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_test' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_doc' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_tools' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_tcltk' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_pip' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_launcher' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_symbols' to value '0'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Include_debug' to value '0'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'LauncherOnly' to value '0'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'AssociateFiles' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'Shortcuts' to value '1'
[0A40:1CC8][2015-09-08T04:29:45]i000: Initializing numeric variable 
'PrependPath' to value '0'
[0A40:1CC8][2015

Re: [Python-Dev] Can't install Python 3.5rc3

[MRAB]

On 2015-09-08 14:22, Steve Dower wrote:
> It thinks you have version 3.5.5339.0 installed, which IIRC was a 
test build I made for some tcl changes. It's probably listed under 
xPython - can you see it?

>
> As I said at the time, they should be totally independent, but I 
guess not. Thanks for the heads up.

>
I purged the xPython bits, but it still wouldn't install.

I then purged the Python3.5dev... bits.

Success!

Rather a messy fix though...

>
> From: MRAB
> Sent: ‎9/‎7/‎2015 20:41
> To: Python-Dev
> Subject: [Python-Dev] Can't install Python 3.5rc3
>
> I've been unable to install Python 3.5rc3.
>
> It progresses to installing Tcl/tk, then reports that a newer version
> of 3.5 is already installed, then it rolls back (slowly).
>
> I don't know where it's finding this "newer version"; I've uninstalled
> the previous releases of Python 3.5.
>
> I'm on Windows 10.
>
>
> Here's the contents of the log file:

[snip]

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 495 Was: PEP 498: Literal String Interpolation is ready for pronouncement

[MRAB]

On 2015-09-12 02:23, Alexander Belopolsky wrote:


On Fri, Sep 11, 2015 at 8:56 PM, Random832 mailto:random...@fastmail.com>> wrote:

Alexander Belopolsky mailto:alexander.belopol...@gmail.com>> writes:
> There is no "earlier" or "later". There are "lesser" and "greater"
> which are already defined for all pairs of aware datetimes. PEP 495
> doubles the set of possible datetimes

That depends on what you mean by "possible".

What exactly depends on the meaning of  "possible"?  In this context
"possible" means "can appear in a Python program."

> and they don't fit in one
> straight line anymore. The whole point of PEP 495 is to introduce a
> "fold" in the timeline.

That doesn't make sense. Within a given timezone, any given moment of
UTC time (which is a straight line [shut up, no leap seconds here]) maps
to only one local time. The point of PEP 495 seems to be to eliminate
the cases where two UTC moments map to the same aware local time.

Yes, but it does that at the cost of introducing the second local
"01:30" which is "later" than the first "01:40" while "obviously" (and
according to the current datetime rules)  "01:30" < "01:40".

Out of curiosity, can "fold" ever be any value other than 0 or 1?

Thankfully, no.


[snip]
What would happen if it's decided to stay on DST and then, later on, to
reintroduce DST?

Or what would happen in the case of "British Double Summer Time" (go
forward twice in the spring and backward twice in the autumn)?

https://en.wikipedia.org/wiki/British_Summer_Time

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[MRAB]

On 2015-09-28 19:00, Erik Bray wrote:

On Fri, Sep 25, 2015 at 6:27 PM, Chris Barker - NOAA Federal
 wrote:


You can use "Windows SDK for Windows 7 and .NET Framework 4".

http://www.microsoft.com/en-us/download/details.aspx?id=8279


Thanks. Last time I tried that route, it was for 64 bit py2.7. And it
required some kludging of environment variables, and registry acces I don't
have permission for.

But it still may be the best bet. I'll give it a try when I have a chance.

And this should be in the "official" docs...


For what it's worth, I've had good luck compiling *most* extension
modules on Windows using the gcc from MinGW-w64.

The big notable exception was that last time I tried compiling Numpy
with it I got a segfaulting Numpy.  But I never had a chance to
investigate why or if it's fixable.  My own extension modules work
fine on Windows when compiled in MinGW though.

Erik B.


Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
that wasn't due to a bug in the source code.



On Sat, Sep 26, 2015 at 12:24 AM, Chris Barker - NOAA Federal
 wrote:


As I understand it, the MS VS2010 compiler is required (or at least
best practice) for compiling Python extensions for the python.org
Windows builds of py 3.4 and ?[1]

However, MS now makes it very hard (impossible?) to download VS2010
Express ( or Community, or whatever the free as in beer version is
called).

I realize that this is not python-dev's responsibility, but if there
is any way to either document where it can be found, or put a bit of
pressure on MS to make it available, as they have for VS2008 and
py2.7, that would be great.

Sorry to bug this list, I didn't know where else to reach out to.

-Chris

[1] it's actually prefer hard to find out which compiler version is
used for which python version. And has been for years. Would a patch
to the docs, probably here:

https://docs.python.org/3.4/using/windows.html#compiling-python-on-windows

Be considered?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com





--
INADA Naoki  


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/erik.m.bray%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/python%40mrabarnett.plus.com




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] VS 2010 compiler

[MRAB]

On 2015-09-28 21:38, Paul Moore wrote:

On 28 September 2015 at 21:18, MRAB  wrote:
> Same here. I compile the regex module for Python 2.5-2.7 and 3.1-3.5,
> both 32-bit and 64-bit, using MinGW-w64, and I haven't had a problem yet
> that wasn't due to a bug in the source code.

Interesting. With Python 3.5, what CRT does the module use? With
Python 3.4 and earlier, distutils arranged for mingw builds to link to
the CRT that matched the CPython build, but I don't think that
happened for 3.5 (because mingw doesn't support ucrt yet). So you'd
have a CRT that doesn't match the core. Obviously this isn't causing
you issues, but I'm not sure if it could (CRT problems seem to be
mostly about "might cause issues" type problems...)


I can't remember the details off-hand, but here's the issue:

http://bugs.python.org/issue24385

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Should PEP 498 specify if rf'...' is valid?

[MRAB]

On 2015-10-26 18:45, Sven R. Kunze wrote:

On 26.10.2015 16:22, Ethan Furman wrote:

On 10/23/2015 08:20 AM, Nick Coghlan wrote:

My own objection isn't to allowing "fR" or "fbR", it's to
allowing the uppercase "F".

I also don't understand why we can't say "if 'f' is part of a
string prefix, it must be first".


Sometimes order matters, and sometimes it does not.  If the order
does not have an impact on the final code, it does not matter, and
making us have to remember an order that does not matter is a
waste.


Order that matters? You must be kidding. That would turn different
types of string extremely hard to understand because semantics
differ.

That is, btw., one reason, why I favor a fixed order (alphabetically
or something). Easy to remember and no way to misinterpret it.


In Python 2, how often have you seen prefix "ur" rather than "ru"?

I always used "ur".

How often is alphabetical order used in the prefixes?

If the order isn't alphabetical, then it's going to be some order
that's harder to remember, so I agree with Ethan here.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reading Python source file

[MRAB]

On 2015-11-17 01:53, Serhiy Storchaka wrote:

I'm working on rewriting Python tokenizer (in particular the part that
reads and decodes Python source file). The code is complicated. For now
there are such cases:

* Reading from the string in memory.
* Interactive reading from the file.
* Reading from the file:
- Raw reading ignoring encoding in parser generator.
- Raw reading UTF-8 encoded file.
- Reading and recoding to UTF-8.

The file is read by the line. It makes hard to check correctness of the
first line if the encoding is specified in the second line. And it makes
very hard problems with null bytes and with desynchronizing buffered C
and Python files. All this problems can be easily solved if read all
Python source file in memory and then parse it as string. This would
allow to drop a large complex and buggy part of code.

Are there disadvantages in this solution? As for memory consumption, the
source text itself will consume only small part of the memory consumed
by AST tree and other structures. As for performance, reading and
decoding all file can be faster then by the line.

[1] http://bugs.python.org/issue25643

As I understand it, *nix expects the shebang to be b'#!', which means 
that the
first line should be ASCII-compatible (it's possible that the UTF-8 BOM 
might

be present). This kind of suggests that encodings like UTF-16 would cause a
problem on such systems.

The encoding line also needs to be ASCII-compatible.

I believe that the recent thread "Support of UTF-16 and UTF-32 source
encodings" also concluded that UTF-16 and UTF-32 shouldn't be supported.

This means that you could treat the first 2 lines as though they were some
kind of extended ASCII (Latin-1?), the line ending being '\n' or '\r' or
'\r\n'.

Once you'd identify the encoding, you could decode everything (including the
shebang line) using that encoding.

(What should happen if the encoding line then decoded differently, i.e.
encoding_line.decode(encoding) != encoding_line.decode('latin-1')?)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Reference has no mention of list comÃprehensions

[MRAB]

On 2015-12-03 15:09, Random832 wrote:

On 2015-12-03, Laura Creighton  wrote:

Who came up with the word 'display' and what does it have going for
it that I have missed?  Right now I think its chief virtue is that
it is a meaningless noun.  (But not meaningless enough, as I
associate displays with output, not construction).


In a recent discussion it seemed like people mainly use it
because they don't like using "literal" for things other than
single token constants.  In most other languages' contexts the
equivalent thing would be called a literal.


"Literals" also tend to be constants, or be constructed out of
constants.

A list comprehension can contain functions, etc.


I think that

6.2.4 Constructing lists, sets and dictionaries

would be a much more useful title, and

6.2.4 Constructing lists, sets and dictionaries -- explicitly or through 
the use of comprehensions


I don't like the idea of calling it "explicit construction".
Explicit construction to me means the actual use of a call to the
constructor function.



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python Language Reference has no mention of list comÃprehensions

[MRAB]

On 2015-12-04 01:25, Steven D'Aprano wrote:

On Thu, Dec 03, 2015 at 09:25:53AM -0800, Andrew Barnert via Python-Dev wrote:

> On Dec 3, 2015, at 08:15, MRAB  wrote:
>
>>> On 2015-12-03 15:09, Random832 wrote:
>>> On 2015-12-03, Laura Creighton  wrote:
>>> Who came up with the word 'display' and what does it have going for
>>> it that I have missed?  Right now I think its chief virtue is that
>>> it is a meaningless noun.  (But not meaningless enough, as I
>>> associate displays with output, not construction).


I completely agree with Laura here -- to me "display" means output, not
construction, no matter what the functional programming community says
:-) but I suppose the connection is that you can construct a list using
the same syntax used to display that list: [1, 2, 3] say.

I don't think the term "display" will ever feel natural to me, but I
have got used to it.


Random832 wrote:


>> In a recent discussion it seemed like people mainly use it
>> because they don't like using "literal" for things other than
>> single token constants.  In most other languages' contexts the
>> equivalent thing would be called a literal.


I'm not sure where you get "most" other languages from. At the very
least, I'd want to see a language survey. I did a *very* fast one (an
entire three languages *wink*) and found these results:

The equivalent of a list [1, a, func(), x+y] is called:

"display" (Python)

"literal" (Ruby)

"constructor" (Lua)

http://ruby-doc.org/core-2.1.1/doc/syntax/literals_rdoc.html#label-Arrays
http://www.lua.org/manual/5.1/manual.html

Of the three, I think Lua's terminology is least worst.


MRAB:

> "Literals" also tend to be constants, or be constructed out of
> constants.


Andrew:

I've seen people saying that before, but I don't know where they get
that. It's certainly not the way, say, C++ or JavaScript use the term.


I wouldn't take either of those two languages as examples of best
practices in language design :-)

"Literal" in computing has usually meant something like MRAB's sense
for close on 20 years, at least. This definition is from FOLDOC (Free
On-Line Dictionary Of Computing), dated 1996-01-23:

literal

 A constant made available to a process, by
inclusion in the executable text.  Most modern systems do not
allow texts to modify themselves during execution, so literals
are indeed constant; their value is written at compile-time
and is read-only at run time.

In contrast, values placed in variables or files and accessed
by the process via a symbolic name, can be changed during
execution.  This may be an asset.  For example, messages can
be given in a choice of languages by placing the translation
in a file.

Literals are used when such modification is not desired.  The
name of the file mentioned above (not its content), or a
physical constant such as 3.14159, might be coded as a
literal.  Literals can be accessed quickly, a potential
advantage of their use.


I think that an important factor is that "literal" is a description of
something in source code, like "expression" and "declaration". We surely
don't want a distinction between the *values* x and y below:

x = 3.14159

y = 4.0 - len("a")
y += 0.14159

but we might want to distinguish between the way they are constructed: x
is constructed from a literal, y is not.


[...]

> A list comprehension can contain functions, etc.

A non-comprehension display can include function calls, lambdas, or
any other kind of expression, just as easily as a comprehension can.
Is [1, x, f(y), lambda z: w+z] a literal? If so, why isn't [i*x for i
in y] a literal?


I wouldn't call either a literal.

I often find myself (mis)using the term "literal" to describe
constructing a list using a display where each item is itself a literal:

x = [1, 2, 3]


Where there can be a grey area is in those languages where strings are
mutable: you can assign a string literal to a variable and then mutate
it.

In that case, there's copying going on, either on the assignment or on
the modification (copy on write, more efficient than always copying
when, say, passing it into a function).


(or at least something which *could* have been a literal, if Python's
parsing rules were just a tiny bit different, like -1 or 2+3j) but I
accept that's an abuse of the term. But I certainly wouldn't use the
term to describe a list constructed from non-literal parts:

x = [a, b**2, func() or None]

and absolutely not for a list comprehension.



The problem is that we need a word that distinguishes the former;
trying to press "literal" into service to help the distinction does

Re: [Python-Dev] Python Language Reference has no mention of list comÃprehensions

[MRAB]

On 2015-12-04 01:56, Chris Angelico wrote:

On Fri, Dec 4, 2015 at 12:25 PM, Steven D'Aprano  wrote:

I don't see any good reason for maintaining that there's just one
syntax, "display", which comes in two forms: a comma-separated set of
values, or a for-loop. The only thing they have in common (syntax-wise)
is that they both use [ ] as delimiters. They look different, they
behave differently, and only one matches what the list actually displays
as. Why use one term for what is clearly two distinct (if related)
syntaxes?


You come across something syntactic that begins by opening a square
bracket, and you know that its semantics are: "construct a new list".
That's what's common here.

What goes *inside* those brackets can be one of two things:

1) A (possibly empty) comma-separated sequence of expressions

2) One or more nested 'for' loops, possibly guarded by 'if's, and a
single expression

So we have two subforms of the same basic syntax. The first one
corresponds better to the output format, in the same way that a string
literal might correspond to its repr under specific circumstances.
Neither is a literal. Neither is a call to a constructor function
(contrast "list()" or "list.__new__(list)", which do call a
constructor). So what is this shared syntax? Whatever word is used,
it's going to be a bit wrong. I'd be happy with either "constructor"
or "display", myself.


The problem with "constructor" is that it's already used for the "__new__"
class method.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third milestone of FAT Python

[MRAB]

On 2015-12-04 12:49, Victor Stinner wrote:
[snip]


Constant folding


This optimization propagates constant values of variables. Example:

 def func()
 x = 1
 y = x
 return y

Constant folding:

 def func()
 x = 1
 y = 1
 return 1


[snip]

I don't think that's constant folding, but constant _propagation_.

Constant folding is when, say, "1 + 2" replaced by "2".

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third milestone of FAT Python

[MRAB]

On 2015-12-04 19:22, Isaac Morland wrote:

On Fri, 4 Dec 2015, MRAB wrote:

> Constant folding is when, say, "1 + 2" replaced by "2".

Isn't that called backspacing?  ;-)


Oops! I meant "1 + 1", of course. Or "3". Either would work. :-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com