Hi everyone,
I was wondering how to pause and resume crawls when using CrawlerProcess
(instead of the command line). There's unfortunately not much info on this
in the doc http://doc.scrapy.org/en/latest/topics/jobs.html
Would the following suffice?
settings.set('JOBDIR', 'some/path/')
process = CrawlerProcess(settings = settings)
process.crawl(SomeSpider, arg1 = arg1, arg2 =arg2, ...)
process.start()
Regards,
Dibya
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
