Re: [Python-Dev] Issue 13524: subprocess on Windows

2012-03-21 Thread Brad Allen
I tripped over this one trying to make one of our Python at work
Windows compatible. We had no idea that a magic 'SystemRoot'
environment variable would be required, and it was causing issues for
pyzmq.

It might be nice to reflect the findings of this email thread on the
subprocess documentation page:

http://docs.python.org/library/subprocess.html

Currently the docs mention this:

"Note If specified, env must provide any variables required for the
program to execute. On Windows, in order to run a side-by-side
assembly the specified env must include a valid SystemRoot."

How about rewording that to:

"Note If specified, env must provide any variables required for the
program to execute. On Windows, a valid SystemRoot environment
variable is required for some Python libraries such as the 'random'
module. Also, in order to run a side-by-side assembly the specified
env must include a valid SystemRoot."
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Issue 13524: subprocess on Windows

2012-03-23 Thread Brad Allen
On Thu, Mar 22, 2012 at 2:35 PM, Glyph Lefkowitz
 wrote:

> Also, in order to execute in any installation environment where libraries are 
> found in non-default locations, you will need to set LD_LIBRARY_PATH.  Oh, 
> and you will also need to set $PATH on UNIX so that libraries can find their 
> helper programs and %PATH% on Windows so that any compiled 
> dynamically-loadable modules and/or DLLs can be loaded.  And by the way you 
> will also need to relay DYLD_LIBRARY_PATH if you did a UNIX-style build on OS 
> X, not LD_LIBRARY_PATH.  Don't forget that you probably also need PYTHONPATH 
> to make sure any subprocess environments can import the same modules as their 
> parent.  Not to mention SSH_AUTH_SOCK if your application requires access to 
> _remote_ process spawning, rather than just local.  Oh and DISPLAY in case 
> your subprocesses need GUI support from an X11 program (which sometimes you 
> need just to initialize certain libraries which don't actually do anything 
> with a GUI).  Oh and __CF_USER_TEXT_ENCODING is important sometimes too, 
> don't forget that.  And if your subprocess is in Perl or Ruby or Java you may 
> need a couple dozen other variables which your deployment environment has set 
> for you too.  Did I mention CFLAGS or LC_ALL yet?  Let me tell you a story 
> about this one HP/UX machine...
>
> Ahem.
>
> Bottom line: it seems like screwing with the process spawning environment to 
> make it minimal is a good idea for simplicity, for security, and for 
> modularity.  But take it from me, it isn't.  I guarantee you that you don't 
> actually know what is in your operating system's environment, and 
> initializing it is a complicated many-step dance which some vendor or 
> sysadmin or product integrator figured out how to do much better than your 
> hapless Python program can.
>
> %SystemRoot% is just the tip of a very big, very nasty iceberg.  Better not 
> to keep refining why exactly it's required, or someone will eventually be 
> adding a new variable (starting with %APPDATA% and %HOMEPATH%) that can 
> magically cause your subprocess not to spawn properly to this page every six 
> months for eternity.  If you're spawning processes as a regular user, you 
> should just take the environment you're given, perhaps with a few specific 
> light additions whose meaning you understand.  If you're spawning a process 
> as an administrator or root, you should probably initialize the environment 
> for the user you want to spawn that process as using an OS-specific mechanism 
> like login(1).  (Sorry that I don't know the Windows equivalent.)


Thanks, Glyph. In that case maybe the Python subprocess docs need not
single out SystemRoot, but instead plaster a big warning around the
use of the 'env' parameter.:

Here is what the docs currently state for the Popen constructor 'env' parameter:

>If env is not None, it must be a mapping that defines the environment 
>variables for the new process; these are used instead of inheriting the 
>current process’ environment, which is the default behavior.

> Note: If specified, env must provide any variables required for the program 
> to execute. On Windows, in order to run a side-by-side assembly the specified 
> env must include a valid SystemRoot.

The "Note" section could instead state something like: "In most cases,
the child process will need many of the same environment variables as
the current process. Usually the safest course of action is to build
the env dict to contain all the same keys and values from os.environ.
For example... "
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Issue 13524: subprocess on Windows

2012-03-23 Thread Brad Allen
On Fri, Mar 23, 2012 at 3:46 PM, Glyph  wrote:
> On Mar 23, 2012, at 1:26 PM, Brad Allen wrote:
>
> Thanks, Glyph. In that case maybe the Python subprocess docs need not
> single out SystemRoot, but instead plaster a big warning around the
> use of the 'env' parameter.
>
>
> I agree.  I'm glad that my bitter experience here might be useful to someone
> in the future - all those late nights trying desperately to get my unit
> tests to run on some newly configured, slightly weird buildbot didn't go to
> waste :).

Ok, I'll open a ticket on the bugtracker for this over the weekend.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com