subprocess call is not waiting.

[paulstaten]

I have a subprocess.call which tries to download a data from a remote server 
using HTAR. I put the call in a while loop, which tests to see if the download 
was successful, and if not, loops back around up to five times, just in case my 
internet connection has a hiccup.

Subprocess.call is supposed to wait.

But it doesn't work as intended. The loop quickly runs 5 times, starting a new 
htar command each time. After five times around, my program tells me my 
download failed, because the target file doesn't yet exist. But it turns out 
that the download is still happening---five times.

When I run htar from the shell, I don't get a shell prompt again until after 
the download is complete. How come control is returned to python before the 
htar command is through?

I've tried using Popen with wait and/or communicate, but no waiting ever 
happens. This is troublesome not only because I don't get to post process my 
data, but because when I run this script for multiple datasets (checking to see 
whether I have local copies), I quickly get a "Too many open files" error. (I 
began working on that by trying to use Popopen with fds_close, etc.)

Should I just go back to os.system?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subprocess call is not waiting.

[paulstaten]

Thanks, guys.
MRAB-RedHat 6 64-bit, Python 2.6.5
JM-Here's the relevant stuff from my last try. I've also tried with 
subprocess.call. Just now I tried shell=True, but it made no difference.

sticking a print(out) in there just prints a blank line in between each 
iteration. It's not until the 5 trials are finished that I am told: download 
failed, etc.

from os.path import exists
from subprocess import call
from subprocess import Popen
from shlex import split
from time import sleep

while (exists(file)==0) and (nTries < 5):
   a = Popen(split('htar -xvf ' + htarArgs), stdout=PIPE, stderr=PIPE)
   (out,err) = a.communicate()
   if exists(file)==0:
  nTries += 1
  sleep(0.5)

if exists(file)==0: # now that the file should be moved
   print('download failed: ' + file)
   return 1

I've also tried using shell=True with popopen.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subprocess call is not waiting.

[paulstaten]

os.system worked fine, and I found something in another section of code that 
was causing the "Too many open errors." (I was fooled, because output from 
subprocess call didn't seem to be coming out until the open files error.

I'll go back and play with subprocess.call more, since os.system works. That's 
interesting about using shlex at run time. Is that just for the sake of 
computational cost?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: subprocess call is not waiting.

[paulstaten]

That's a habit I'll make sure to avoid, then.
Thanks, Chris!
-- 
http://mail.python.org/mailman/listinfo/python-list