Re: Solr and hadoop

2014-09-25 Thread Tom Chen
Science of Influence Marketing” > > 18 East 41st Street > > New York, NY 10017 > > t: @appinions <https://twitter.com/Appinions> | g+: > plus.google.com/appinions > < > https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts > > &g

Solr and hadoop

2014-09-25 Thread Tom Chen
I wonder if Solr has InputFormat and OutputFormat like the EsInputFormat and EsOutputFormat that are provided by Elasticserach for Hadoop (es-hadoop). Is it possible for Solr to provide such integration with Hadoop? Best, Tom

Re: MRIT's morphline mapper doesn't co-locate with data

2014-09-25 Thread Tom Chen
ottleneck remains the same. Having said that, we have an > ingestion tool in the works that will take advantage of data locality for > splitable files as well. > > Wolfgang. > > On Sep 24, 2014, at 9:38 AM, Tom Chen wrote: > > > Hi, > > > > The MRIT (MapReduceInde

MRIT's morphline mapper doesn't co-locate with data

2014-09-24 Thread Tom Chen
Hi, The MRIT (MapReduceIndexerTool) uses NLineInputFormat for the morphline mapper. The mapper doesn't co-locate with the input data that it process. Isn't this a performance hit? Ideally, morphline mapper should be run on those hosts that contain most data blocks for the input files it process.

Re: Solr Map Reduce Indexer Tool GoLive to SolrCloud with index on local file system

2014-07-03 Thread Tom Chen
xTool, but there's also > the Core Admin command. > > The sub-indexes are in a partition in HDFS and numbered > sequentially. > > Best, > Erick > > On Wed, Jul 2, 2014 at 3:23 PM, Tom Chen wrote: > > Hi, > > > > > > When we run Solr Map Reduce Ind

Solr Map Reduce Indexer Tool GoLive to SolrCloud with index on local file system

2014-07-02 Thread Tom Chen
Hi, When we run Solr Map Reduce Indexer Tool ( https://github.com/markrmiller/solr-map-reduce-example), it generates indexes on HDFS The last stage is Go Live to merge the generated index to live SolrCloud index. If the live SolrCloud write index to local file system (rather than HDFS), the Go

Solr and Kerberos

2014-06-06 Thread Tom Chen
Hi, I wonder how to secure Solr with Kerberos. We can Kerberos secure Solr by configuring the AuthenticationFilter from the hadoop-auth.jar that is packaged in solr.war. But after we do that, 1) How does a SolrJ client connect to the secured Solr server? 2) In SolrCloud environment, how one Sol