On Wed, 2007-02-07 at 18:03 +0200, Sami Siren wrote: > rubdabadub wrote: > > Hi: > > > > Are there relatively stand-alone crawler that are > > suitable/customizable for Solr? has anyone done any trials.. I have > > seen some discussion about coocon crawler.. was that successfull? > > There's also integration path available for Nutch[1] that i plan to > integrate after 0.9.0 is out.
sounds very nice, I just finished to read. Thanks. Today a submitted a proposal for an Apache Labs project called Apache Druids. http://mail-archives.apache.org/mod_mbox/labs-labs/200702.mbox/browser Basic idea is to create a flexible crawler framework. The core should be a simple crawler which could be easily expended by plugins. So if a project/app needs special processing for a crawled url one could write a plugin to implement the functionality. salu2 > > -- > Sami Siren > > [1]http://blog.foofactory.fi/2007/02/online-indexing-integrating-nutch-with.html -- Thorsten Scherler thorsten.at.apache.org Open Source Java & XML consulting, training and solutions
