Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-02 Thread Nikolaus Rath
Ben Hoyt writes: > Thanks for the clarifications and support. > >> Ah, the wording in the PEP says "Linux, Windows, OS X". Superficially, >> that said "everywhere" to me. It might be worth calling out >> specifically some examples where it's not available without an extra >> system call, just to m

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-02 Thread Ben Hoyt
Thanks for the clarifications and support. > Ah, the wording in the PEP says "Linux, Windows, OS X". Superficially, > that said "everywhere" to me. It might be worth calling out > specifically some examples where it's not available without an extra > system call, just to make the point explicit.

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-02 Thread Paul Moore
tl;dr - I agree with your points and think that the original PEP 471 proposal is fine. The details here are just clarification of why my proposal wasn't just "use PEP 471 as written" in the first place... On 2 July 2014 13:41, Ben Hoyt wrote: > 1) It's a nasty API to actually write code with. If

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-02 Thread Ben Hoyt
Thanks for the effort in your response, Paul. I'm all for KISS, but let's just slow down a bit here. > I think that thin wrapper is needed - even > if the various bells and whistles are useful, they can be built on top > of a low-level version (whereas the converse is not the case). Yes, but API

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-02 Thread Jonas Wielicki
On 01.07.2014 23:20, Paul Moore wrote: > [snip] > Please, let's stick to a low-level wrapper round the OS API for the > first iteration of this feature. Enhancements can be added later, when > real-world usage has proved their value. > > Paul +1 to the whole thing. That’s an ingeniously simple so

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Nick Coghlan
On 1 July 2014 14:20, Paul Moore wrote: > On 1 July 2014 14:00, Ben Hoyt wrote: >> 2) Nick Coghlan's proposal on the previous thread >> (https://mail.python.org/pipermail/python-dev/2014-June/135261.html) >> suggesting an ensure_lstat keyword param to scandir if you need the >> lstat_result value

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Chris Angelico
On Wed, Jul 2, 2014 at 7:20 AM, Paul Moore wrote: > I think that thin wrapper is needed - even > if the various bells and whistles are useful, they can be built on top > of a low-level version (whereas the converse is not the case). +1. Make everything as simple as possible (but no simpler). Chr

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ethan Furman
On 07/01/2014 02:20 PM, Paul Moore wrote: Please, let's stick to a low-level wrapper round the OS API for the first iteration of this feature. Enhancements can be added later, when real-world usage has proved their value. +1 ___ Python-Dev mailing li

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Glenn Linderman
On 7/1/2014 2:20 PM, Paul Moore wrote: Please, let's stick to a low-level wrapper round the OS API for the first iteration of this feature. Enhancements can be added later, when real-world usage has proved their value. I almost wrote this whole message this morning, but didn't have time. Than

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Paul Moore
On 1 July 2014 14:00, Ben Hoyt wrote: > 2) Nick Coghlan's proposal on the previous thread > (https://mail.python.org/pipermail/python-dev/2014-June/135261.html) > suggesting an ensure_lstat keyword param to scandir if you need the > lstat_result value > > I would make one small tweak to Nick Coghl

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Jonas Wielicki
On 01.07.2014 17:30, Ben Hoyt wrote: >> No need for a microsecond-timed deletion -- a directory with +r but >> without +x will allow you to list the entries, but stat calls on the >> files will fail with EPERM: > > Ah -- very good to know, thanks. This definitely points me in the > direction of wa

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Nick Coghlan
On 1 July 2014 08:42, Ben Hoyt wrote: >> We need per-iteration error handling for the readdir call anyway, so I think >> an onerror callback is a better option than dropping the ability to easily >> obtain full stat information as part of the iteration. > > I don't mind the idea of an "onerror" ca

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ethan Furman
On 07/01/2014 07:59 AM, Jonas Wielicki wrote: I had the idea to treat a failing lstat() inside scandir() as if the entry wasn’t found at all, but in this context, this seems wrong too. Well, os.walk supports passing in an error handler -- perhaps scandir should as well. -- ~Ethan~ __

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ben Hoyt
> We need per-iteration error handling for the readdir call anyway, so I think > an onerror callback is a better option than dropping the ability to easily > obtain full stat information as part of the iteration. I don't mind the idea of an "onerror" callback, but it's adding complexity. Putting a

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Nick Coghlan
On 1 Jul 2014 07:31, "Victor Stinner" wrote: > > 2014-07-01 15:00 GMT+02:00 Ben Hoyt : > > 2) Nick Coghlan's proposal on the previous thread > > (https://mail.python.org/pipermail/python-dev/2014-June/135261.html) > > suggesting an ensure_lstat keyword param to scandir if you need the > > lstat_r

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ben Hoyt
> No need for a microsecond-timed deletion -- a directory with +r but > without +x will allow you to list the entries, but stat calls on the > files will fail with EPERM: Ah -- very good to know, thanks. This definitely points me in the direction of wanting better control over error handling. Spe

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Jonas Wielicki
On 01.07.2014 15:00, Ben Hoyt wrote: > I'm leaning towards preferring #2 (Nick's proposal) because it solves > or gets around the caching issue. My one concern is error handling. Is > it an issue if scandir's __next__ can raise an OSError either from the > readdir() call or the call to stat()? My t

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Victor Stinner
2014-07-01 15:00 GMT+02:00 Ben Hoyt : > (a) it doesn't call stat for you (on POSIX), so you have to check an > attribute and call scandir manually if you need it, Yes, and that's something common when you use the os module. For example, don't try to call os.fork(), os.getgid() or os.fchmod() on Wi

Re: [Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Ben Hoyt
Thanks for spinning this off to (hopefully) finished the discussion. I agree it's nearly time to update the PEP. > @Ben: it's time to update your PEP to complete it with this > discussion! IMO DirEntry must be as simple as possible and portable: > > - os.scandir(str) > - DirEntry.lstat_result obje

[Python-Dev] My summary of the scandir (PEP 471)

2014-07-01 Thread Victor Stinner
Hi, @Ben: it's time to update your PEP to complete it with this discussion! IMO DirEntry must be as simple as possible and portable: - os.scandir(str) - DirEntry.lstat_result object only available on Windows, same result than os.lstat() - DirEntry.fullname(): os.path.join(directory, DirEntry.name