Re: Guidance for event-driven indexing

2011-02-15 Thread Rich Cariens
Thanks Jan, I was referring to a custom UpdateHandler, not a RequestHandler. You know, the one that the Solr wiki discourages :). Best, Rich On Tue, Feb 15, 2011 at 8:37 AM, Jan Høydahl wrote: > Hi, > > You would wire your JMSUpdateRequestHandler into solrconfig.xml as normal, > and if you wan

Re: Guidance for event-driven indexing

2011-02-15 Thread Jan Høydahl
Hi, You would wire your JMSUpdateRequestHandler into solrconfig.xml as normal, and if you want to apply an UpdateChain, that would look like this: myPipeline See http://wiki.apache.org/solr/SolrRequestHandler for details -- Jan Høydahl, search solution architect Cominve

Re: Guidance for event-driven indexing

2011-02-15 Thread Rich Cariens
Thanks Jan. For the JMSUpdateHandler option, how does one plugin a custom UpdateHandler? I want to make sure I'm not missing any important pieces of Solr processing pipeline. Best, Rich On Tue, Feb 15, 2011 at 4:36 AM, Jan Høydahl wrote: > Solr is multi threaded, so you are free to send as man

Re: Guidance for event-driven indexing

2011-02-15 Thread Jan Høydahl
Solr is multi threaded, so you are free to send as many parallel update requests needed to utilize your HW. Each request will get its own thread. Simply configure StreamingUpdateSolrServer from your client. If there is some lengthy work to be done, it needs to be done in SOME thread, and I gues

Re: Guidance for event-driven indexing

2011-02-14 Thread Rich Cariens
Thanks Jan, I don't think I want to tie up a thread on two boxes waiting for an UpdateRequestProcessor to finish. I'd prefer to offload it all to the target shards. And a special JMSUpdateHandler feels like overkill. I *think* I'm really just looking for a simple API that allows me to add a SolrIn

Re: Guidance for event-driven indexing

2011-02-14 Thread Jan Høydahl
Hi, One option would be to keep the JMS listener as today but move the downloading and transforming part to a SolrUpdateRequestProcessor on each shard. The benefit is that you ship only a tiny little SolrInputDocument over the wire with a reference to the doc to download, and do the heavy lifting

Guidance for event-driven indexing

2011-02-14 Thread Rich Cariens
Hello, I've built a system that receives JMS events containing links to docs that I must download and index. Right now the JMS receiving, downloading, and transformation into SolrInputDoc's happens in a separate JVM that then uses Solrj javabin HTTP POSTs to distribute these docs across many index