Hi, One option would be to keep the JMS listener as today but move the downloading and transforming part to a SolrUpdateRequestProcessor on each shard. The benefit is that you ship only a tiny little SolrInputDocument over the wire with a reference to the doc to download, and do the heavy lifting on Solr side.
If each JMS topic/channel corresponds to a particular shard, you could move the whole thing to Solr. If so, a new JMSUpdateHandler could perhaps be a way to go? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 14. feb. 2011, at 16.53, Rich Cariens wrote: > Hello, > > I've built a system that receives JMS events containing links to docs that I > must download and index. Right now the JMS receiving, downloading, and > transformation into SolrInputDoc's happens in a separate JVM that then uses > Solrj javabin HTTP POSTs to distribute these docs across many index shards. > > For various reasons I won't go into here, I'd like to relocate/deploy most > of my processing (JMS receiving, downloading, and transformation into > SolrInputDoc's) into the SOLR webapps running on each distributed shard > host. I might be wrong, but I don't think the request-driven idiom of the > DataImportHandler is not a good fit for me as I'm not kicking off full or > delta imports. If that's true, what's the correct way to hook my components > into SOLR's update facilities? Should I try to get a reference a configured > DirectUpdateHandler? > > I don't know if this information helps, but I'll put it out there anyways: > I'm using Spring 3 components to receive JMS events, wired up via webapp > context hooks. My plan would be to add all that to my SOLR shard webapp. > > Best, > Rich
