Re: mapreduce job using soirj 5

2015-06-17 Thread Shenghua(Daniel) Wan
Check mapreduce.task.classpath.user.precedence and its equivalent property in different hadoop version. HADOOP_OPTS needs to work with this property being set to true. I met problem like yours. And playing with these parameters solved my problem. On Wed, Jun 17, 2015 at 12:28 AM, adfel70 wrote

Re: mapreduce job using soirj 5

2015-06-17 Thread Erick Erickson
For sure there are a few rough edges here On Wed, Jun 17, 2015 at 12:28 AM, adfel70 wrote: > We cannot downgrade httpclient in solrj5 because its using new features and > we dont want to start altering solr code, anyway we thought about upgrading > httpclient in hadoop but as Erick said its s

Re: mapreduce job using soirj 5

2015-06-17 Thread Mark Miller
I think there is some better classpath isolation options in the works for Hadoop. As it is, there is some harmonization that has to be done depending on versions used, and it can get tricky. - Mark On Wed, Jun 17, 2015 at 9:52 AM Erick Erickson wrote: > For sure there are a few rough edges here

Re: mapreduce job using soirj 5

2015-06-17 Thread adfel70
We cannot downgrade httpclient in solrj5 because its using new features and we dont want to start altering solr code, anyway we thought about upgrading httpclient in hadoop but as Erick said its sounds more work than just put the jar in the data nodes. About that flag we tried it, hadoop even has

Re: mapreduce job using soirj 5

2015-06-16 Thread Shenghua(Daniel) Wan
Hadoop has a switch that lets you use your jar rather than the one hadoop carries. google for HADOOP_OPTS good luck. On Tue, Jun 16, 2015 at 7:23 AM, adfel70 wrote: > Hi, > > We recently started testing solr 5, our indexer creates mapreduce job that > uses solrj5 to index documents to our SolrCl

Re: mapreduce job using soirj 5

2015-06-16 Thread Shawn Heisey
On 6/16/2015 9:24 AM, Erick Erickson wrote: > Sounds like a question better asked in one of the Cloudera support > forums, 'cause all I can do is guess ;). > > I suppose, theoretically, that you could check out the Solr5 > code and substitute the httpclient-4.2.5.jar in the build system, > recompil

Re: mapreduce job using soirj 5

2015-06-16 Thread Erick Erickson
Sounds like a question better asked in one of the Cloudera support forums, 'cause all I can do is guess ;). I suppose, theoretically, that you could check out the Solr5 code and substitute the httpclient-4.2.5.jar in the build system, recompile and go, but that's totally a guess based on zero know