[Python-Dev] PEP 471: scandir(fd) and pathlib.Path(name, dir_fd=None)

2014-07-01 Thread Victor Stinner
Hi, IMO we must decide if scandir() must support or not file descriptor. It's an important decision which has an important impact on the API. To support scandir(fd), the minimum is to store dir_fd in DirEntry: dir_fd would be None for scandir(str). scandir(fd) must not close the file descripto

[Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Victor Stinner
Hi, @Ben: it's time to update your PEP to complete it with this discussion! IMO DirEntry must be as simple as possible and portable: - os.scandir(str) - DirEntry.lstat_result object only available on Windows, same result than os.lstat() - DirEntry.fullname(): os.path.join(directory, DirEntry.name

Re: [Python-Dev] PEP 471: scandir(fd) and pathlib.Path(name, dir_fd=None)

2014-07-01 Thread Ben Hoyt
Thanks, Victor. I don't have any experience with dir_fd handling, so unfortunately can't really comment here. What advantages does it bring? I notice that even os.listdir() on Python 3.4 doesn't have anything related to file descriptors, so I'd be in favour of not including support. We can always

Re: [Python-Dev] PEP 471: scandir(fd) and pathlib.Path(name, dir_fd=None)

2014-07-01 Thread Victor Stinner
2014-07-01 14:26 GMT+02:00 Ben Hoyt : > Thanks, Victor. > > I don't have any experience with dir_fd handling, so unfortunately > can't really comment here. > > What advantages does it bring? I notice that even os.listdir() on > Python 3.4 doesn't have anything related to file descriptors, so I'd >

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ben Hoyt
Thanks for spinning this off to (hopefully) finished the discussion. I agree it's nearly time to update the PEP. > @Ben: it's time to update your PEP to complete it with this > discussion! IMO DirEntry must be as simple as possible and portable: > > - os.scandir(str) > - DirEntry.lstat_result obje

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Victor Stinner
2014-07-01 15:00 GMT+02:00 Ben Hoyt : > (a) it doesn't call stat for you (on POSIX), so you have to check an > attribute and call scandir manually if you need it, Yes, and that's something common when you use the os module. For example, don't try to call os.fork(), os.getgid() or os.fchmod() on Wi

[Python-Dev] Excess help() output

2014-07-01 Thread anatoly techtonik
Hi, The help() output is confusing for beginners: >>> class B(object): ... pass ... >>> help(B) Help on class B in module __main__: class B(__builtin__.object) | Data descriptors defined here: | | __dict__ | dictionary for instance variables (if defined) |

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Jonas Wielicki
On 01.07.2014 15:00, Ben Hoyt wrote: > I'm leaning towards preferring #2 (Nick's proposal) because it solves > or gets around the caching issue. My one concern is error handling. Is > it an issue if scandir's __next__ can raise an OSError either from the > readdir() call or the call to stat()? My t

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ben Hoyt
> No need for a microsecond-timed deletion -- a directory with +r but > without +x will allow you to list the entries, but stat calls on the > files will fail with EPERM: Ah -- very good to know, thanks. This definitely points me in the direction of wanting better control over error handling. Spe

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Nick Coghlan
On 1 Jul 2014 07:31, "Victor Stinner" wrote: > > 2014-07-01 15:00 GMT+02:00 Ben Hoyt : > > 2) Nick Coghlan's proposal on the previous thread > > (https://mail.python.org/pipermail/python-dev/2014-June/135261.html) > > suggesting an ensure_lstat keyword param to scandir if you need the > > lstat_r

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ben Hoyt
> We need per-iteration error handling for the readdir call anyway, so I think > an onerror callback is a better option than dropping the ability to easily > obtain full stat information as part of the iteration. I don't mind the idea of an "onerror" callback, but it's adding complexity. Putting a

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ethan Furman
On 07/01/2014 07:59 AM, Jonas Wielicki wrote: I had the idea to treat a failing lstat() inside scandir() as if the entry wasn’t found at all, but in this context, this seems wrong too. Well, os.walk supports passing in an error handler -- perhaps scandir should as well. -- ~Ethan~ __

Re: [Python-Dev] PEP 471 -- os.scandir() function -- a better and faster directory iterator

2014-07-01 Thread Janzert
On 6/26/2014 6:59 PM, Ben Hoyt wrote: Rationale = Python's built-in ``os.walk()`` is significantly slower than it needs to be, because -- in addition to calling ``os.listdir()`` on each directory -- it executes the system call ``os.stat()`` or ``GetFileAttributes()`` on each file to dete

Re: [Python-Dev] PEP 471: scandir(fd) and pathlib.Path(name, dir_fd=None)

2014-07-01 Thread Akira Li
Ben Hoyt writes: > Thanks, Victor. > > I don't have any experience with dir_fd handling, so unfortunately > can't really comment here. > > What advantages does it bring? I notice that even os.listdir() on > Python 3.4 doesn't have anything related to file descriptors, so I'd > be in favour of not

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Nick Coghlan
On 1 July 2014 08:42, Ben Hoyt wrote: >> We need per-iteration error handling for the readdir call anyway, so I think >> an onerror callback is a better option than dropping the ability to easily >> obtain full stat information as part of the iteration. > > I don't mind the idea of an "onerror" ca

[Python-Dev] Network Security Backport Status

2014-07-01 Thread Alex Gaynor
Hi all, I wanted to bring everyone up to speed on the status of PEP 466, what's been completed, and what's left to do. First the completed stuff: * hmac.compare_digest * hashlib.pbkdf2_hmac Are both backported, and I've added support to use them in Django, so users should start seeing these ben

Re: [Python-Dev] Network Security Backport Status

2014-07-01 Thread Nick Coghlan
On 1 Jul 2014 11:28, "Alex Gaynor" wrote: > > I've come up with a new approach, which I believe is most likely to be > successful, but I'll need help to implement it. > > The idea is to find the most recent commit which is a parent of both the > ``2.7`` and ``default`` branches. Then take every si

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Jonas Wielicki
On 01.07.2014 17:30, Ben Hoyt wrote: >> No need for a microsecond-timed deletion -- a directory with +r but >> without +x will allow you to list the entries, but stat calls on the >> files will fail with EPERM: > > Ah -- very good to know, thanks. This definitely points me in the > direction of wa

Re: [Python-Dev] Network Security Backport Status

2014-07-01 Thread Antoine Pitrou
Le 01/07/2014 14:26, Alex Gaynor a écrit : I can do all the work of reviewing each commit, but I need some help from a mercurial expert to automate the cherry-picking/rebasing of every single commit. What do folks think? Does this approach make sense? Anyone willing to help with the mercurial s

Re: [Python-Dev] Network Security Backport Status

2014-07-01 Thread Guido van Rossum
I have to agree with Antoine -- I don't think there's a shortcut that avoids *someone* actually having to understand the code to the point of being able to recreate the same behavior in the different context (pun not intended) of Python 2. On Tue, Jul 1, 2014 at 1:54 PM, Antoine Pitrou wrote: >

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Paul Moore
On 1 July 2014 14:00, Ben Hoyt wrote: > 2) Nick Coghlan's proposal on the previous thread > (https://mail.python.org/pipermail/python-dev/2014-June/135261.html) > suggesting an ensure_lstat keyword param to scandir if you need the > lstat_result value > > I would make one small tweak to Nick Coghl

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Glenn Linderman
On 7/1/2014 2:20 PM, Paul Moore wrote: Please, let's stick to a low-level wrapper round the OS API for the first iteration of this feature. Enhancements can be added later, when real-world usage has proved their value. I almost wrote this whole message this morning, but didn't have time. Than

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ethan Furman
On 07/01/2014 02:20 PM, Paul Moore wrote: Please, let's stick to a low-level wrapper round the OS API for the first iteration of this feature. Enhancements can be added later, when real-world usage has proved their value. +1 ___ Python-Dev mailing li

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Chris Angelico
On Wed, Jul 2, 2014 at 7:20 AM, Paul Moore wrote: > I think that thin wrapper is needed - even > if the various bells and whistles are useful, they can be built on top > of a low-level version (whereas the converse is not the case). +1. Make everything as simple as possible (but no simpler). Chr

[Python-Dev] [RELEASE] Python 2.7.8

2014-07-01 Thread Benjamin Peterson
Greetings, I have the distinct privilege of informing you that the latest release of the Python 2.7 series, 2.7.8, has been released and is available for download. 2.7.8 contains several important regression fixes and security changes: - The openssl version bundled in the Windows installer has be

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Nick Coghlan
On 1 July 2014 14:20, Paul Moore wrote: > On 1 July 2014 14:00, Ben Hoyt wrote: >> 2) Nick Coghlan's proposal on the previous thread >> (https://mail.python.org/pipermail/python-dev/2014-June/135261.html) >> suggesting an ensure_lstat keyword param to scandir if you need the >> lstat_result value