Solr is multi threaded, so you are free to send as many parallel update requests needed to utilize your HW. Each request will get its own thread. Simply configure StreamingUpdateSolrServer from your client.
If there is some lengthy work to be done, it needs to be done in SOME thread, and I guess you just have to choose where :) A JMSUpdateHandler sounds heavy weight, but does not need to be, and might be the logically best place for such a feature imo. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 14. feb. 2011, at 17.42, Rich Cariens wrote: > Thanks Jan, > > I don't think I want to tie up a thread on two boxes waiting for an > UpdateRequestProcessor to finish. I'd prefer to offload it all to the target > shards. And a special JMSUpdateHandler feels like overkill. I *think* I'm > really just looking for a simple API that allows me to add a > SolrInputDocument to the index in-process. > > Perhaps I just need to use the EmbeddedSolrServer in the Solrj packages? I'm > worried that this will break all the nice stuff one gets with the standard > SOLR webapp (stats, admin, etc). > > Best, > Rich > > > On Mon, Feb 14, 2011 at 11:18 AM, Jan Høydahl <jan....@cominvent.com> wrote: > >> Hi, >> >> One option would be to keep the JMS listener as today but move the >> downloading >> and transforming part to a SolrUpdateRequestProcessor on each shard. The >> benefit >> is that you ship only a tiny little SolrInputDocument over the wire with a >> reference to the doc to download, and do the heavy lifting on Solr side. >> >> If each JMS topic/channel corresponds to a particular shard, you could >> move the whole thing to Solr. If so, a new JMSUpdateHandler could perhaps >> be a way to go? >> >> -- >> Jan Høydahl, search solution architect >> Cominvent AS - www.cominvent.com >> >> On 14. feb. 2011, at 16.53, Rich Cariens wrote: >> >>> Hello, >>> >>> I've built a system that receives JMS events containing links to docs >> that I >>> must download and index. Right now the JMS receiving, downloading, and >>> transformation into SolrInputDoc's happens in a separate JVM that then >> uses >>> Solrj javabin HTTP POSTs to distribute these docs across many index >> shards. >>> >>> For various reasons I won't go into here, I'd like to relocate/deploy >> most >>> of my processing (JMS receiving, downloading, and transformation into >>> SolrInputDoc's) into the SOLR webapps running on each distributed shard >>> host. I might be wrong, but I don't think the request-driven idiom of the >>> DataImportHandler is not a good fit for me as I'm not kicking off full or >>> delta imports. If that's true, what's the correct way to hook my >> components >>> into SOLR's update facilities? Should I try to get a reference a >> configured >>> DirectUpdateHandler? >>> >>> I don't know if this information helps, but I'll put it out there >> anyways: >>> I'm using Spring 3 components to receive JMS events, wired up via webapp >>> context hooks. My plan would be to add all that to my SOLR shard webapp. >>> >>> Best, >>> Rich >> >>
