: with Solr. What crawler do you guys use? or you coded one by yourself? I
neither -- i've never indexed "crawled" data With Solr, i only ever index structured data in one form or another. (the closest i've ever come to using a crawler with Solr is some ant tasks i whiped up one day to recursively apply an XSLT to some XML data and then index the content because i needed a quick hack to build the index .... before we had DIH) If i needed to build an index of crawled data, i'd use Nutch. -Hoss
