Manlio Perillo ha scritto:
> [...]
> Here is a quick example, ABSOLUTELY NOT TESTED:
>
> class DownloadQueue(object):
> SIZE = 50
>
> def init(self):
> self.requests = [] # queued requests
> self.deferreds = [] # waiting requests
>
> def addRequest(self, url, timeout
rzimerman ha scritto:
> I'm hoping to write a program that will read any number of urls from
> stdin (1 per line), download them, and process them. So far my script
> (below) works well for small numbers of urls. However, it does not
> scale to more than 200 urls or so, because it issues HTTP reque
"rzimerman" wrote:
> Is Twisted the best library for me to be using? I do like Twisted, but
> it seems more suited to batch mode operations. Is there some way that I
> could continue registering url requests while the reactor is running?
> Is there a way to specify a time out per page request, rat
rzimerman wrote:
> I'm hoping to write a program that will read any number of urls from
> stdin (1 per line), download them, and process them. So far my script
> (below) works well for small numbers of urls. However, it does not
> scale to more than 200 urls or so, because it issues HTTP requests f
