On May 11, 2007, at 7:32 AM, David Xiao wrote:
Hello,
I am using crawler to index and search some intranet webpages which
need authorization. I wrote my own crawler for this kind of needs.
But with the requirement is evolving, I need another crawler for
external webpages (on internet) too,
Hello,
I am using crawler to index and search some intranet webpages which need
authorization. I wrote my own crawler for this kind of needs. But with the
requirement is evolving, I need another crawler for external webpages (on
internet) too, so I am looking for a generic crawler that can
