Re: [Python-Dev] PEP 435 - ref impl disc 2

[Glenn Linderman]

On 5/10/2013 11:02 PM, Ethan Furman wrote:

On 05/10/2013 10:15 PM, Glenn Linderman wrote:


But the last few lines of demo1 demonstrate that NIE doesn't like, 
somehow, remember that its values, deep down under
the covers, are really int.  And doesn't even like them when they are 
wrapped into IntET objects.  This may or may not

be a bug in the current Enum implementation.


You're right, sort of.  ;)

If you do

  print( repr( NIE1.x.value ))

you'll see

  ('NIE1.x', 1)

In other words, the value of NEI.x is `('NEI1.x', 1)` and that is what 
you would have to pass back into NEI to get the enum member.


Ah! But the value of NIE.x should be IntET('NIE.x', 1), no? So Enum is 
presently saving the constructor parameters as the value, rather than 
the constructed object? So for Enums of builtin types, there is little 
difference, but for complex types (as opposed to complex numbers), there 
is a difference, and I guess I ran into the consequences of that difference.


As an aside, I suspect you are doing this the hard way.  Perhaps 
writing your own __new__ in NIE will have better results


NIE.__new__ wouldn't have the name available, unless it is passed in. So 
it seems to me that Enum (or Enum+) has to pass in the parameter... in 
which case NIE.__new__ can be pretty ordinary.


Other implementation strategies that occurred to me... maybe I'll try 
them all, if I have time... but time looks to get scarcer soon...


* I'm playing with adding another keyword parameter to Enum, but it is 
presently giving me an error about unknown keyword parameter passed to 
__prepare__ even though I added **kwds to the list of its parameters. 
I'll learn something by doing this.


* Implement a subclass of Enum that has a bunch of operator tracking 
methods like IntET. However, the results of arithmetic operations 
couldn't add a new enumeration member (legally), so some other type 
would still have to exist... and it would also have to have those 
operation tracking methods.


* Implement a subclass of EnumMeta that installs all the operator 
tracking methods. Same legal issue.


* Do one of the above, but allow new calculated members to exist, 
although perhaps not in the initial, iterable set. Might have to call it 
something besides Enum to obey the proclamations :) FlagBits, maybe.  
But it could use much the same technology as Enum.


(I'd try, but I gotta get some sleep! ;) . 


Thanks for the response... it cleared up the demo1 mystery, anyway.


Oh, newest code posted.


That'll give me some practice pulling from your repository into mine :)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

[Nick Coghlan]

On Sat, May 11, 2013 at 2:24 PM, Ben Hoyt  wrote:
> In all the *practical* examples I've seen (and written myself), I
> iterate over a directory and I just need to know whether it's a file
> or directory (or maybe a link). Occassionally you need the size as
> well, but that would just mean a similar check "if st.st_size is None:
> st = os.stat(...)", which on Linux/OS X would call stat(), but it'd
> still be free and fast on Windows.

Here's the full set of fields on a current stat object:

st_atime
st_atime_ns
st_blksize
st_blocks
st_ctime
st_ctime_ns
st_dev
st_gid
st_ino
st_mode
st_mtime
st_mtime_ns
st_nlink
st_rdev
st_size
st_uid

Do we really want to publish an object with all of those as attributes
potentially set to None, when the abstraction we're trying to present
is intended primarily for the benefit of os.walk?

And if we're creating a custom object instead, why return a 2-tuple
rather than making the entry's name an attribute of the custom object?

To me, that suggests a more reasonable API for os.scandir() might be
for it to be an iterator over "dir_entry" objects:

name (as a string)
is_file()
is_dir()
is_link()
stat()
cached_stat (None or a stat object)

On all platforms, the query methods would not require a separate
stat() call. On Windows, cached_stat would be populated with a full
stat object when scandir builds the entry. On non-Windows platforms,
cached_stat would initially be None, and you would have to call stat()
to populate it.

If we find other details that we can reliably provide cross-platform
from the dir information, then we can add more query methods or
attributes to the dir_entry object.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

[Christian Heimes]

Am 11.05.2013 16:34, schrieb Nick Coghlan:
> Here's the full set of fields on a current stat object:
> 
> st_atime
> st_atime_ns
> st_blksize
> st_blocks
> st_ctime
> st_ctime_ns
> st_dev
> st_gid
> st_ino
> st_mode
> st_mtime
> st_mtime_ns
> st_nlink
> st_rdev
> st_size
> st_uid

And there are more fields on some platforms, e.g. st_birthtime.

> To me, that suggests a more reasonable API for os.scandir() might be
> for it to be an iterator over "dir_entry" objects:
> 
> name (as a string)
> is_file()
> is_dir()
> is_link()
> stat()
> cached_stat (None or a stat object)

I suggest that we call it .lstat() and .cached_lstat to make clear that
we are talking about no-follow stat() here. On platforms that support
fstatat() it should use fstatat(dir_fd, name, &buf, AT_SYMLINK_NOFOLLOW)
where dir_fd is the fd from dirfd() of opendir()'s return value.

> On all platforms, the query methods would not require a separate
> stat() call. On Windows, cached_stat would be populated with a full
> stat object when scandir builds the entry. On non-Windows platforms,
> cached_stat would initially be None, and you would have to call stat()
> to populate it.

+1

> If we find other details that we can reliably provide cross-platform
> from the dir information, then we can add more query methods orst
> attributes to the dir_entry object.

I'd like to see d_type and d_ino, too. d_type should default to
DT_UNKNOWN, d_ino to None.

Christian


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

[Nick Coghlan]

On Sun, May 12, 2013 at 1:42 AM, Christian Heimes  wrote:
> I suggest that we call it .lstat() and .cached_lstat to make clear that
> we are talking about no-follow stat() here.

Fair point.

> On platforms that support
> fstatat() it should use fstatat(dir_fd, name, &buf, AT_SYMLINK_NOFOLLOW)
> where dir_fd is the fd from dirfd() of opendir()'s return value.

It may actually make sense to expose the dir_fd as another attribute
of the dir_entry object.

>> If we find other details that we can reliably provide cross-platform
>> from the dir information, then we can add more query methods orst
>> attributes to the dir_entry object.
>
> I'd like to see d_type and d_ino, too. d_type should default to
> DT_UNKNOWN, d_ino to None.

I'd prefer to see a more minimal set to start with - just the features
needed to implement os.walk and os.fwalk more efficiently, and provide
ready access to the full stat result.

Once that core functionality is in place, *then* start debating what
other use cases to optimise based on which platforms would support
those optimisations and which would require dropping back to the full
stat implementation anyway.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

[Nick Coghlan]

On Sun, May 12, 2013 at 2:30 AM, Nick Coghlan  wrote:
> Once that core functionality is in place, *then* start debating what
> other use cases to optimise based on which platforms would support
> those optimisations and which would require dropping back to the full
> stat implementation anyway.

Alternatively, we could simply have a full "dirent" attribute that is
None on Windows. That would actually make sense at an implementation
level anyway - is_file() etc would check self.cached_lstat first, and
if that was None they would check self.dirent, and if that was also
None they would raise an error.

Construction of a dir_entry would require either a stat object or a
dirent object, but complain if it received both.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 435 - ref impl disc 2

[Glenn Linderman]

On 5/11/2013 12:11 AM, Glenn Linderman wrote:
* I'm playing with adding another keyword parameter to Enum, but it is 
presently giving me an error about unknown keyword parameter passed to 
__prepare__ even though I added **kwds to the list of its parameters. 
I'll learn something by doing this.


OK, I figured out the error was because __prepare__ didn't have a **kwds 
parameters, but then got a similar one regarding __init__ — but EnumMeta 
doesn't even have an __init__ so I guess it was using type.__init__, but 
then I wondered if type.__init__ even does anything, because when I 
added __init__ to (my modified ref435a) EnumMeta, it didn't seem to 
matter if my __init__ did nothing, called super().__init__, or called 
type.__init__.  Anyway, defining one seems to get past the errors, and 
then the keyword can work.


So compare your ref435.py and my ref435a.py at 
 to see the code required to 
support a keyword parameter that would expect a base type containing a 
name parameter to its __new__ and __init__, by providing the 
module-qualified name as that parameter.


Would this be a controversial enhancement to EnumMeta?  Together with my 
flags.py at the same link, it would enable definitions of enumeration 
values which have names, and which names could be reported in 
exceptions see demo3.py at the same link.


I suppose it might also be good to validate that no unexpected keyword 
parameters are passed in, rather than just ignoring them, as my code 
presently does? Not sure what the general thinking is regarding such 
parameters in such usages; it could be that some mixin class might want 
to define some keyword parameters too, so ignoring them seems a more 
flexible option.





___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 435 - ref impl disc 2

[Glenn Linderman]

On 5/11/2013 12:11 AM, Glenn Linderman wrote:

Oh, newest code posted.


That'll give me some practice pulling from your repository into mine :)


Well, I had to bring your changes to my local repository, and then push 
them up to my bitbucket repo... not sure if there is a way to merge from 
your bitbucket repo to mine directly... I couldn't find it, if there is.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] 2.7.5 baking

[Benjamin Peterson]

The long anticipated "emergency" 2.7.5 release has now been tagged. It
will be publicly announced as binaries arrive.

Originally, I was just going to cherrypick regression fixes onto the
2.7.4 release and release those as 2.7.5. I started to this but ran
into some conflicts. Since we don't have buildbot testing of release
branches, I decided it would be best to just cut from the maintenance
branch.

--
Regards,
Benjamin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com