Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Antoine Pitrou
Le Fri, 2 Aug 2013 02:27:43 +0200, Victor Stinner a écrit : > 2013/7/28 Antoine Pitrou : > >> (A) How should we support support where os.set_inheritable() is not > >> supported? Can we announce that os.set_inheritable() is always > >> available or not? Does such platform exist? > > > > FD_CLOEXEC

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Antoine Pitrou
Le Fri, 2 Aug 2013 13:30:56 +0200, Victor Stinner a écrit : > Is it possible to implement atfork on Windows? > > A Python lock would be ignored by other C threads. It is unsafe if > Python is embedded. It is unsafe if Python is embedded *and* the embedding application uses fork() + exec(). Rega

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Richard Oudkerk
On 02/08/2013 12:30pm, Victor Stinner wrote: Is it possible to implement atfork on Windows? On Windows the patch does expose atfork.getlock() and uses it in subprocess. (It should also modify os.spawn?(), os.startfile() etc.) But atfork.atfork() is Unix only. A Python lock would be ignore

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Victor Stinner
Is it possible to implement atfork on Windows? A Python lock would be ignored by other C threads. It is unsafe if Python is embedded. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Victor Stinner
http://support.microsoft.com/kb/315939/en-us Ah yes, we may implement pass_handles on Windows using a critical section to inherit *handles*. File descriptors cannot be inherited using CreateProcess(), only using spawn(). Or can we rely on the undocumented fields used by spawn()? Victor _

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Victor Stinner
Le 2 août 2013 08:32, "Charles-François Natali" a écrit : > > 2013/8/2 Victor Stinner : > > 2013/7/28 Antoine Pitrou : > >>> (A) How should we support support where os.set_inheritable() is not > >>> supported? Can we announce that os.set_inheritable() is always > >>> available or not? Does such pl

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Richard Oudkerk
On 02/08/2013 1:21am, Victor Stinner wrote: 2013/7/30 Victor Stinner : I would be nice to have a "pass_handles" on Windows. I'm not sure that it's possible to implement this atomically. It's probably better to leave the application to choose how the inheritance is defined. Example: for handl

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Richard Oudkerk
On 02/08/2013 12:59am, Victor Stinner wrote: On Windows, a file cannot be removed if at least one process opened it. If you create a temporary file, run a program, and delete the temporary file: the deletion fails if the program inherited the file and the program is not done before the deletion.

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-02 Thread Richard Oudkerk
On 02/08/2013 7:44am, Charles-François Natali wrote: Then how about changing the default to creating file descriptors unheritable on Windows (which is apparently the default)? Then you can implement keep_fds by setting them inheritable right before creation, and resetting them right after: sure t

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-01 Thread Charles-François Natali
2013/8/2 Victor Stinner : > On Windows, inheritable handles (including open files) are still > inherited when a standard stream is overriden in the subprocess module > (default value of close_fds is set to False in this case). This issue > cannot be solved (at least, I don't see how): it is a limit

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-01 Thread Charles-François Natali
2013/8/2 Victor Stinner : > 2013/7/28 Antoine Pitrou : >>> (A) How should we support support where os.set_inheritable() is not >>> supported? Can we announce that os.set_inheritable() is always >>> available or not? Does such platform exist? >> >> FD_CLOEXEC is POSIX: >> http://pubs.opengroup.org/o

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-01 Thread Victor Stinner
2013/7/30 Antoine Pitrou : > Le Tue, 30 Jul 2013 09:09:38 +0200, > Charles-François Natali a écrit : >> This looks more and more like PEP 433 :-) >> >> And honestly, when I think about it, I think that this whole mess is a >> solution looking for a problem. >> If we don't want to inherit file desc

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-01 Thread Victor Stinner
2013/7/28 Antoine Pitrou : >> (A) How should we support support where os.set_inheritable() is not >> supported? Can we announce that os.set_inheritable() is always >> available or not? Does such platform exist? > > FD_CLOEXEC is POSIX: > http://pubs.opengroup.org/onlinepubs/9699919799/functions/fcn

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-01 Thread Victor Stinner
2013/7/30 Victor Stinner : > I would be nice to have a "pass_handles" on Windows. I'm not sure that it's possible to implement this atomically. It's probably better to leave the application to choose how the inheritance is defined. Example: for handle in handles: os.set_inheritable(handle, T

Re: [Python-Dev] PEP 446: Open issues/questions

2013-08-01 Thread Victor Stinner
2013/7/30 Richard Oudkerk : > The documentation for STARTUPINFO says this about STARTF_USESTDHANDLES: > > If this flag is specified when calling one of the process creation > functions, the handles must be inheritable and the function's > bInheritHandles parameter must be set to TRUE. > >

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Richard Oudkerk
On 30/07/2013 11:52am, Victor Stinner wrote: > You can redirect standard streams (stdin, stdout, stderr) using the > startup info structure: > > startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES > startupinfo.hStdInput = p2cread > startupinfo.hStdOutput = c2pwrite > startupinfo.hStdError = errwr

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Victor Stinner
2013/7/30 Richard Oudkerk : > Note that on Windows subprocess has no equivalent of a passfds argument, and > if you redirect the standard streams then you are forced to inherit all > inheritable handles. You can redirect standard streams (stdin, stdout, stderr) using the startup info structure: s

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Richard Oudkerk
On 30/07/2013 8:09am, Charles-François Natali wrote: If we don't want to inherit file descriptors in child processes, the answer is simple: the subprocess module (this fact is not even mentioned in the PEP). Note that on Windows subprocess has no equivalent of a passfds argument, and if you re

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Victor Stinner
Le 30 juil. 2013 09:11, "Charles-François Natali" a écrit : > > Perhaps this advocates for a global flag, e.g. > > sys.set_default_fd_inheritance(), with False (non-inheritable) being > > the default for sanity and security. > > This looks more and more like PEP 433 :-) I don't like the global mo

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Antoine Pitrou
Le Tue, 30 Jul 2013 09:09:38 +0200, Charles-François Natali a écrit : > > > > Perhaps this advocates for a global flag, e.g. > > sys.set_default_fd_inheritance(), with False (non-inheritable) being > > the default for sanity and security. > > This looks more and more like PEP 433 :-) > > And hon

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Victor Stinner
2013/7/28 Charles-François Natali : > Also, it'll be puzzling to have syscalls automatically set the cloexec > flag. I guess a lot of people doing system programming with Python > will get bitten, but that's a discussion we already had months ago... The inheritance of file descriptors (and Windows

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-30 Thread Charles-François Natali
>> Having stdin/stdout/stderr cloexec (e.g. after a dup() to redirect to >> a log file, a socket...) will likely break a lot of code, e.g. code >> using os.system(), or code calling exec manually (and I'm sure there's >> a bunch of it). > > Hmm. os.exec*() could easily make standard streams non-CLO

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-29 Thread Victor Stinner
2013/7/28 Antoine Pitrou : >> (B) Should subprocess make the file descriptors of pass_fds >> inheritable? If yes, should it be done before or after the fork? If it >> is done after the fork and before exec, it only affects the child >> process, at least on Linux (the file descriptor is still >> non

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-29 Thread Antoine Pitrou
On Mon, 29 Jul 2013 23:42:36 +0200 Victor Stinner wrote: > > >> So perhaps only the *original* standard streams should be left > >> inheritable? > > I plan to only change functions *creating* (and replacing) new file > descriptors. Existing file descriptors (like 0, 1, 2) are unchanged. You can

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-29 Thread Victor Stinner
2013/7/28 Charles-François Natali : > 2013/7/28 Antoine Pitrou : >>> (C) Should we handle standard streams (0: stdin, 1: stdout, 2: stderr) >>> differently? For example, os.dup2(fd, 0) should make the file >>> descriptor 0 (stdin) inheritable or non-inheritable? On Windows, >>> os.set_inheritable(f

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-29 Thread Antoine Pitrou
Le Sun, 28 Jul 2013 12:18:55 +0200, Charles-François Natali a écrit : > 2013/7/28 Antoine Pitrou : > >> (C) Should we handle standard streams (0: stdin, 1: stdout, 2: > >> stderr) differently? For example, os.dup2(fd, 0) should make the > >> file descriptor 0 (stdin) inheritable or non-inheritable

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-28 Thread Charles-François Natali
2013/7/28 Antoine Pitrou : >> (C) Should we handle standard streams (0: stdin, 1: stdout, 2: stderr) >> differently? For example, os.dup2(fd, 0) should make the file >> descriptor 0 (stdin) inheritable or non-inheritable? On Windows, >> os.set_inheritable(fd, False) fails (error 87, invalid argumen

Re: [Python-Dev] PEP 446: Open issues/questions

2013-07-28 Thread Antoine Pitrou
On Sun, 28 Jul 2013 02:39:19 +0200 Victor Stinner wrote: > Hi, > > I have a few more questions on the PEP 446: > > (A) How should we support support where os.set_inheritable() is not > supported? Can we announce that os.set_inheritable() is always > available or not? Does such platform exist? F

[Python-Dev] PEP 446: Open issues/questions

2013-07-27 Thread Victor Stinner
Hi, I have a few more questions on the PEP 446: (A) How should we support support where os.set_inheritable() is not supported? Can we announce that os.set_inheritable() is always available or not? Does such platform exist? (B) Should subprocess make the file descriptors of pass_fds inheritable?