your technique seems to work best (modify the sql) because DIH resides in a solr core , distributing is the load is not easy
There are plans to make DIH work as a library where it will post docs using SolrJ. May be it is possible to include this feature there On Thu, Jul 2, 2009 at 3:34 AM, Jay Hill<jayallenh...@gmail.com> wrote: > I'm using the DIH to index records from a relational database. No problems, > everything works great. But now, due to the size of index (70GB w/ 25M+ > docs) I need to shard and want the DIH to distribute documents evenly > between two shards. Current approach is to modify the sql query in the > config file to get only even numbered ids on one host and odd numbered ids > on the other host. Is there are more elegant way to distribute the > documents? Has anyone else come up with a better way to approach this? > > Thanks, > -Jay > -- ----------------------------------------------------- Noble Paul | Principal Engineer| AOL | http://aol.com
