[issue14702] os.makedirs breaks under autofs directories

Antoine Pitrou added the comment: +1 for closing. -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue14702] os.makedirs breaks under autofs directories

Charles-François Natali added the comment: I still don't like the idea of adding such a kludge to work around OS bugs, so I'd suggest closing as won't fix. -- ___ Python tracker ___

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: > Maybe I'm confused, but the presence of "/net/prodigy" is *not* > the issue here, and what gets mounted is *not* "/net/prodigy", > but "/net/prodigy/tmp" (do "mount" to confirm or dispute). No, /net/prodigy is the mountpoint in this case: amcnabb@sage:~ :) mo

[issue14702] os.makedirs breaks under autofs directories

Martin v. Löwis added the comment: > Of course, I would love to see clear documentation of how the kernel > is defined to behave in this situation. It would certainly be > intuitive for mkdir("/net/prodigy/tmp") to force a mount of > "/net/prodigy", but defined behavior isn't always intuit

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: > Which one-line fix do you propose? Doing a stat("/net/prodigy/tmp") before mkdir("/net/prodigy/tmp") is an extremely simple workaround. Of course, I would love to see clear documentation of how the kernel is defined to behave in this situation. It would cer

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: >>> """ >>> stat("/net/prodigy", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 >>> mkdir("/net/prodigy/tmp", 0777) = -1 EACCES (Permission denied) >>> """ >>> >>> As you can see, a stat() is already done on /net/prodigy. >> >> To be fair, that shouldn’

[issue14702] os.makedirs breaks under autofs directories

Martin v. Löwis added the comment: >> """ >> stat("/net/prodigy", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 >> mkdir("/net/prodigy/tmp", 0777) = -1 EACCES (Permission denied) >> """ >> >> As you can see, a stat() is already done on /net/prodigy. > > To be fair, that shouldn’t trigge

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: > """ > stat("/net/prodigy", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 > mkdir("/net/prodigy/tmp", 0777) = -1 EACCES (Permission denied) > """ > > As you can see, a stat() is already done on /net/prodigy. To be fair, that shouldn’t trigger a moun

[issue14702] os.makedirs breaks under autofs directories

Charles-François Natali added the comment: > I see no evidence that this is a bug in Linux, """ stat("/net/prodigy", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 mkdir("/net/prodigy/tmp", 0777) = -1 EACCES (Permission denied) """ As you can see, a stat() is already done on /net/prodi

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: I posted a bug report with the kernel here: https://bugzilla.kernel.org/show_bug.cgi?id=43262 -- ___ Python tracker ___ ___

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: I see no evidence that this is a bug in Linux, and I think it's ridiculous to close it when a trivial one-line fix is available. I won't reopen it because it's obvious no one wants to address this. :( -- ___ Python

[issue14702] os.makedirs breaks under autofs directories

Charles-François Natali added the comment: Alright, closing for good then. Andrew, if you want to get this fixed, you should report this to the autofs folks, because it's definitely not a Python bug. -- stage: -> committed/rejected status: open -> closed _

[issue14702] os.makedirs breaks under autofs directories

Martin v. Löwis added the comment: I agree that the proposed solutions are hacks, and still propose to close this as "won't fix". I think this is a bug in Linux. mkdir("/net/prodigy/tmp") ought to trigger the mount, then fail with EEXIST. -- nosy: +loewis ___

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: I sense a very easy fix: just do a stat() on the final path before we do anything (and quickly return if present). Moving current makedirs into a private _makedirs that gets called if the stat fails. I'm still feeling uneasy about that though. It _is_ a klud

[issue14702] os.makedirs breaks under autofs directories

Charles-François Natali added the comment: Yes, creating the directories in a bottom-up way (i.e. '/net', '/net/prodigy', '/net/prodigy/foo') could maybe avoid this problem. But this is definietely an autofs bug, and there are probably many other places where such code might break. --

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: By the way, if my hunch about the difference in stat of '/net/prodigy' vs. '/net/prodigy/tmp' is correct, then this would explain why makedirs on deeper directories work. Specifically, one of the shallower stat calls would force the mount to complete before pr

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: Hmm. Maybe there's a difference between doing stat('/net/prodigy') vs. stat('/net/prodigy/tmp'). Just a guess, but maybe the former can succeed before the mount completes, but the latter has to wait for the mount to complete. -- __

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: My suspicion re the race condition is that a chdir waits for autofs to mount but mkdir doesn't. You could check that yourself if a chdir/sleep would help. -- ___ Python tracker

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: If I read it correctly, we _do_ a stat: stat("/net/prodigy", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 mkdir("/net/prodigy/tmp", 0777) = -1 EACCES (Permission denied) ISTM that this is a different problem altogether, or am I missing something? I

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: This isn't fixed. All of the examples I've given were with a 3.3.0 kernel. Doing a stat would be a fix. -- status: closed -> open ___ Python tracker __

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: I agree, closing. Thank you for your valuable assistance Andrew! -- resolution: -> wont fix stage: needs patch -> status: open -> closed ___ Python tracker ___

[issue14702] os.makedirs breaks under autofs directories

Charles-François Natali added the comment: > I guess this is the magic in mkdir -p: > > mkdir("expert", 0755)                   = -1 EACCES (Permission denied) > chdir("expert")                         = 0 > mkdir("tmp", 0755)                      = -1 EEXIST (File exists) > > I'm not sure how I

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: I guess this is the magic in mkdir -p: mkdir("expert", 0755) = -1 EACCES (Permission denied) chdir("expert") = 0 mkdir("tmp", 0755) = -1 EEXIST (File exists) I'm not sure how I feel about that. I

[issue14702] os.makedirs breaks under autofs directories

Changes by Andrew McNabb : Added file: http://bugs.python.org/file25612/makedirs.out ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue14702] os.makedirs breaks under autofs directories

Changes by Andrew McNabb : Added file: http://bugs.python.org/file25611/mkdir-p.out ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: Some interesting information. If I do `os.mkdir('/net/prodigy/tmp')`, it gives "OSError: [Errno 13] Permission denied: '/net/prodigy/tmp'". However, if I instead do `os.mkdir('/net/prodigy/tmp/hi')`, it succeeds. (Note that I'm being careful to start from a fr

[issue14702] os.makedirs breaks under autofs directories

Andrew McNabb added the comment: > Andrew, are you still with us? I'm here, but it's been a busy few weeks. I'll see if I can spend some time on this today. -- ___ Python tracker

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: Andrew, are you still with us? -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubsc

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: >> Charles, I don't think you can blame autofs here. The problem at hand is >> that makedirs() never checks whether the directory exists (that would >> trigger the mount too I presume). > > Yes, it does. Have a look at line 148: > """ > if head and tail

[issue14702] os.makedirs breaks under autofs directories

R. David Murray added the comment: Hynek: you said "just like EEXIST", which doesn't check to see if the directory exists before continuing, thus my confusion. If mkdir -p does a stat first, then changing the makedirs algorithm to match is probably not a bad idea for OS compatibility reasons,

[issue14702] os.makedirs breaks under autofs directories

Charles-François Natali added the comment: > Charles, I don't think you can blame autofs here. The problem at hand is that > makedirs() never checks whether the directory exists (that would trigger the > mount too I presume). Yes, it does. Have a look at line 148: """ if head and tail an

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: Charles, I don't think you can blame autofs here. The problem at hand is that makedirs() never checks whether the directory exists (that would trigger the mount too I presume). Instead, it tries a mkdir and looks if it gets an EEXIST. If you try that approac

[issue14702] os.makedirs breaks under autofs directories

Charles-François Natali added the comment: To me, this doesn't look like a os.makedirs() bug, but rather an autofs one (didn't know it was still in use). I don't like the idea of adding such a kludge (i.e. catching EPERM and checking whether the directory appeared in between). Could you pleas

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: David: What do you mean? I'd just catch EPERM and look whether the directory "appeared" in the meantime. If it exists, continue. If not: re-raise. Am I missing something? I may have a look at GNU mkdir if it doesn't help. --

[issue14702] os.makedirs breaks under autofs directories

R. David Murray added the comment: Hynek: it doesn't seem like that would work, since legitimate EPERM errors are much more likely. -- nosy: +r.david.murray ___ Python tracker

[issue14702] os.makedirs breaks under autofs directories

Changes by Hynek Schlawack : -- keywords: +easy ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue14702] os.makedirs breaks under autofs directories

Hynek Schlawack added the comment: As makedirs in 3.x doesn’t handle EPERM and is otherwise the same, I presume the error is there as well. I also presume, that after the failed makedirs(), the directory is mounted? I'd just handle the error just we handle EEXIST in 3.x now. -- nosy:

[issue14702] os.makedirs breaks under autofs directories

New submission from Andrew McNabb : When a os.makedirs is used under an autofs directory, it crashes. For example, on my machine, `os.makedirs('/net/prodigy/tmp')` crashes with the following traceback: Traceback (most recent call last): ... File "/usr/lib64/python2.7/os.py", line 157, in m