> > I had to download some files. > > > > Using the code below I could download almost all I had to download, but I > > have a problem > > : > > r = urllib.urlopen(url) > > : > > And the problem is that the script sleeps for some hours. And thats > > all - no data is read. > > > is there a reason why you are using urlopen() instead of > urlretrieve(url, localFile)? it will download the file for you so you > don't have to do it chunks at a time. is it because you're working > with an unreliable server where a timeout is required? otherwise yes, > you will have to set a socket timeout to give up on a partial download > if the other side is hanging.
I don't use urlretrieve because I have to get more info about all the files I'm downloading (I need regular downloading of file so if I want to check the changes in file (reading headers) and after analyzing changes I'll decide do I have to download that file again or not). And I need timeout because some of the pages I'am checking often are inaccessible. > > you may also want to consider multithreading this application (using > the threading module). if the individual downloads are not dependent > on each other. I/O-bound applications are a great use case for Python > MT and will help improve performance. it will allow for multiple > "simultaneous" downloads in parallel and allow it to complete sooner. Thanks for advice, multithreading will be in next version of my program :) _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
