Well, it's not that Nutch doesn't support it. Solr itself doesn't support it. Indexing applications need to know which shard they want to send documents to. This may be a good case for a new wish issue in Solr JIRA?
Otis ---- Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ ----- Original Message ---- > From: Markus Jelsma <markus.jel...@buyways.nl> > To: solr-user@lucene.apache.org > Sent: Wed, June 16, 2010 3:31:49 PM > Subject: RE: Re: Solr and Nutch/Droids - to use or not to use? > > Nutch does not, at this moment, support some form of consistent hashing to > select an appropriate shard. It would be nice if someone could file an issue > in > Nutch' Jira to add sharding support to it, perhaps someone with a better > understanding and more experience with Solr's distributed search than i have > at > the moment. I can't point Nutch' developers to the right piece of > documentation > on this one ;) -----Original message----- From: Otis Gospodnetic > < > href="mailto:otis_gospodne...@yahoo.com";>otis_gospodne...@yahoo.com> Sent: > Wed 16-06-2010 21:03 To: > href="mailto:solr-user@lucene.apache.org";>solr-user@lucene.apache.org; > Subject: Re: Solr and Nutch/Droids - to use or not to use? Hi > Mitch, Solr can do distributed search, so it can definitely handle > indices that can't fit on a single server without sharding. What I think > *might* be the case that the Nutch indexer that sends docs to Solr might not > be > capable of sending documents to multiple Solr cores/shards. If that is the > case, I think you need to move this to the Nutch user/dev list and see how to > feed multiple Solr indices/cores/shards with Nutch > data. Otis ---- Sematext :: > target=_blank >http://sematext.com/ :: Solr - Lucene - Nutch Lucene > ecosystem search :: http://search-lucene.com/ > ----- Original Message ---- > From: MitchK < > ymailto="mailto:mitc...@web.de"; > href="mailto:mitc...@web.de";>mitc...@web.de> > To: > ymailto="mailto:solr-user@lucene.apache.org"; > href="mailto:solr-user@lucene.apache.org";>solr-user@lucene.apache.org > > Sent: Wed, June 16, 2010 2:27:16 PM > Subject: Re: Solr and Nutch/Droids - > to use or not to use? > > Thanks, that really helps to find the > right beginning for such a journey. > :-) > * Use Solr, > not Nutch's search webapp > As > far as I have read, Solr > can't scale, if the index gets too large for > > one Server > The setup explained here has one significant > > caveat you also need to keep > in mind: scale. You cannot use > this kind of > setup with vertical scale > (collection size) that > goes beyond one Solr > box. The horizontal scaling > (query > throughput) is still possible with > the standard Solr > replication > tools. > ...from > > Lucidimagination.com Is this still the case? Furthermore, as far as I > > have understood this blogpost: > href=" > href="http://www.lucidimagination.com/blog/2009/03/09/nutch-solr/"; > target=_blank > >http://www.lucidimagination.com/blog/2009/03/09/nutch-solr/"; target=_blank > > > > href="http://www.lucidimagination.com/blog/2009/03/09/nutch-solr/ > " target=_blank >http://www.lucidimagination.com/blog/2009/03/09/nutch-solr/ > Lucidimagination.com > : Nutch and Solr , they index the whole > stuff with nutch and reindex it to > Solr - sounds like a lot of > redundant work. Lucid, Sematext and the > Nutch-wiki are the only > information-sources where I can find talks about > Nutch and Solr, but > no one seems to talk about these facts - except this one > > blogpost. If you say this is wrong or contingent on the shown setup, can > > you tell me how to avoid these problems? A lot of questions, > but it's > such an exciting topic... Hopefully you can answer some > of > them. Again, thank you for the feedback, Otis. - > Mitch -- > View this message in context: > href=" > href="http://lucene.472066.n3.nabble.com/Solr-and-Nutch-Droids-to-use-or-not-to-use-tp900069p900604.html"; > > target=_blank > >http://lucene.472066.n3.nabble.com/Solr-and-Nutch-Droids-to-use-or-not-to-use-tp900069p900604.html"; > > > > target=_blank > > > href="http://lucene.472066.n3.nabble.com/Solr-and-Nutch-Droids-to-use-or-not-to-use-tp900069p900604.html > " target=_blank > >http://lucene.472066.n3.nabble.com/Solr-and-Nutch-Droids-to-use-or-not-to-use-tp900069p900604.html > Sent > from the Solr - User mailing list archive at > Nabble.com.
