Re: Solr Map Reduce Indexer Tool GoLive to SolrCloud with index on local file system

2014-09-29 Thread Nirmal
Tom Chen gmail.com> writes: > > Hi, > > In the GoLive stage, the MRIT sends the MERGEINDEXES requests to Solr > instances. The request has a indexDir parameter with a hdfs path to the > index generated on HDFS, as shown in the MRIT log: > > 2014-07-02 15:03:55,123 DEBUG > org.apache.http.impl.

Re: Solr Map Reduce Indexer Tool GoLive to SolrCloud with index on local file system

2014-07-05 Thread Erick Erickson
Ok, I asked some folks who know and the response is that "that should work, but it's not supported/tested". IOW, you're into somewhat uncharted territory. The people who wrote the code don't have this use-case in their priority list and probably won't be expending energy in this direction any time

Re: Solr Map Reduce Indexer Tool GoLive to SolrCloud with index on local file system

2014-07-03 Thread Erick Erickson
Hmmm, interesting, I actually hadn't thought of doing it that way. I don't know the internals well enough to comment on it but I do know someone who does. I'll check with them Erick On Thu, Jul 3, 2014 at 9:18 AM, Tom Chen wrote: > Hi, > > In the GoLive stage, the MRIT sends the MERGEINDEXES

Re: Solr Map Reduce Indexer Tool GoLive to SolrCloud with index on local file system

2014-07-03 Thread Tom Chen
Hi, In the GoLive stage, the MRIT sends the MERGEINDEXES requests to Solr instances. The request has a indexDir parameter with a hdfs path to the index generated on HDFS, as shown in the MRIT log: 2014-07-02 15:03:55,123 DEBUG org.apache.http.impl.conn.DefaultClientConnection: Sending request: GE

Re: Solr Map Reduce Indexer Tool GoLive to SolrCloud with index on local file system

2014-07-02 Thread Erick Erickson
How would the MapReduceIndexerTool (MRIT for short) find the local disk to write from HDFS to for each shard? All it has is the information in the Solr configs, which are usually relative paths on the local Solr machines, relative to SOLR_HOME. Which could be different on each node (that would be s