Tim,
Here is the complete content of the appConfig-default.json file. I haven’t
worked with Slider so far, so not very sure if some mistake has crept into this
file while modifying it as per the changes mentioned by Lucidworks on the
Github page. However, I tried to simulate the steps mentioned on the Github
Solr-Slider page. As you can see from below file, I am using Java 7. I have
installed HDP 2.3 GA version which has Slider version 0.80 included in it.
{
"schema": "http://example.org/specification/v2.0.0";,
"metadata": {
},
"global": {
"application.def": "/home/hdfs/solr-slider/solr-on-yarn.zip",
"java_home": "/usr/lib/jvm/jre-1.7.0-openjdk.x86_64",
"site.global.app_root":
"${AGENT_WORK_ROOT}/app/install/solr-5.2.1-SNAPSHOT",
"site.global.zk_host": "myhdpcluster.com:2181",
"site.global.solr_host": "${SOLR_HOST}",
"site.global.listen_port": "${SOLR.ALLOCATED_PORT}",
"site.global.xmx_val": "1g",
"site.global.xms_val": "1g",
"site.global.gc_tune": "-XX:NewRatio=3 -XX:SurvivorRatio=4
-XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC
-XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4
-XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m
-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50
-XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled
-XX:+ParallelRefProcEnabled -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution
-XX:+PrintGCApplicationStoppedTime",
"site.global.zk_timeout": "15000",
"site.global.server_module": "--module=http",
"site.global.stop_key": "solrrocks",
"site.global.solr_opts": ""
},
"components": {
"slider-appmaster": {
"jvm.heapsize": "512M"
},
"SOLR": {
}
}
}
Thanks & Regards
Vijay
-----Original Message-----
From: Timothy Potter [mailto:thelabd...@gmail.com]
Sent: 27 August 2015 21:44
To: solr-user@lucene.apache.org
Subject: Re: FW: Issue while setting Solr on Slider / YARN
Hi Vijay,
I'm not sure what's wrong here ... have you posted to the Slider mailing list?
Also, which version of Java are you using when interacting with Slider? I know
it had some issues with Java 8 at one point. Which version of Slider so I can
try to reproduce ...
Cheers,
Tim
On Thu, Aug 27, 2015 at 11:05 AM, Vijay Bhoomireddy
<vijaya.bhoomire...@whishworks.com> wrote:
>
>
> Hi Tim,
>
>
>
> For some reason, I was not receiving messages from Solr mailing list,
> though I could post it to the list. Now I got that sorted. For my
> below query, I saw your response on the mailing list. Below is the
> snippet of your
> response:
>
>
>
> Hi Vijay,
>
>
>
> Verify the ResourceManager URL and try passing the --manager param to
>
> explicitly set the ResourceManager URL during the create step.
>
>
>
> Cheers,
>
> Tim
>
>
>
>
>
> I verified the Resource Manager URL and its pointing correctly. In my
> case, it's myhdpcluster.com:8032 I even tried passing --manager param
> to the slider create solr command, but without any luck. So not sure
> where it's getting wrong. Can you please help me in understanding if I
> need to modify something else to get this working? I am just wondering
> whether "${AGENT_WORK_ROOT} in step below has any impact? I haven't
> changed this line in the json file. Should this be modified?
>
>
>
> I am able to login to Ambari console and see all the services running
> fine , including YARN related and ZooKeeper. Also, I could login to
> Resource Manager's web UI as well and its working fine. Can you please
> let me know what / where it could have gone wrong?
>
>
>
>
>
> Thanks & Regards
>
> Vijay
>
>
>
> From: Vijay Bhoomireddy [mailto:vijaya.bhoomire...@whishworks.com]
> Sent: 17 August 2015 11:37
> To: solr-user@lucene.apache.org <mailto:solr-user@lucene.apache.org>
> Subject: Issue while setting Solr on Slider / YARN
>
>
>
> Hi,
>
>
>
> Any help on this please?
>
>
>
> Thanks & Regards
>
> Vijay
>
>
>
> From: Vijay Bhoomireddy [mailto:vijaya.bhoomire...@whishworks.com]
> Sent: 14 August 2015 18:03
> To: solr-user@lucene.apache.org <mailto:solr-user@lucene.apache.org>
> Subject: Issue while setting Solr on Slider / YARN
>
>
>
> Hi,
>
>
>
> We have a requirement of setting up of Solr Cloud to work along with Hadoop.
> Earlier, I could setup a SolrCloud cluster separately alongside the
> Hadoop cluster i.e. it looks like two logical clusters sitting next
> to each other, both relying on HDFS.
>
>
>
> However, the experiment now I am trying to do is to install SolrCloud
> on YARN using Apache Slider. I am following LucidWorks blog at
> https://github.com/LucidWorks/solr-slider for the same. I already have
> a Hortonworks HDP cluster. When I try to setup Solr on my HDP cluster
> using Slider, I am facing some issues.
>
>
>
> As per the blog, I have performed the below steps:
>
>
>
> 1. I have setup a single node HDP cluster for which the hostname is
> myhdpcluster.com with all the essential services including ZooKeeper
> and Slider running on it.
>
> 2. Updated the resource manager address and port in slider-client.xml
> present under /usr/hdp/current/slider-client/conf
>
> <property>
>
> <name>yarn.resourcemanager.address</name>
>
> <value> myhdpcluster.com:8032</value>
>
> </property>
>
> 3. Cloned the LucidWorks git and moved it under /home/hdfs/solr-slider
>
> 4. Downloaded solr latest stable distribution and renamed it as
> solr.tgz and placed it under
> /home/hdfs/solr-slider/package/files/solr.tgz
>
> 5. Next ran the following command from within the
> /home/hdfs/solr-slider folder
>
> zip -r solr-on-yarn.zip metainfo.xml package/
>
> 6. Next ran the following command as hdfs user
>
> slider install-package --replacepkg --name solr --package
> /home/hdfs/solr-slider/solr-on-yarn.zip
>
> 7. Modified the following settings in the
> /home/hdfs/solr-slider/appConfig-default.json file
>
> "java_home": MY_JAVA_HOME_LOCATION
>
> "site.global.app_root":
> "${AGENT_WORK_ROOT}/app/install/solr-5.2.1",
> (Should this be changed to any other value?)
>
> "site.global.zk_host": " myhdpcluster.com:2181",
>
> 8. Set yarn.component.instances to 1 in resources-default.json file
>
> 9. Next ran the following command
>
> slider create solr --template
> /home/hdfs/solr-slider/appConfig-default.json
> --resources /home/hdfs/solr-slider/resources-default.json
>
>
>
> During this step, I am seeing an message INFO client.RMProxy -
> Connecting to ResourceManager at myhdpcluster.com/10.0.2.15:8032
>
>
> INFO ipc.Client - Retrying connect to server:
> myhdpcluster.com/10.0.2.15:8032. Already tried 0 time(s);
>
>
>
> This message keeps repeating for 50 times and then pauses for a couple
> of seconds and then prints the same message in a loop eternally. Not
> sure on where the problem is.
>
>
>
> Can anyone please help me out to get away from this issue and help me
> setup Solr on Slider/YARN?
>
>
>
> Thanks & Regards
>
> Vijay
>
>
> --
> The contents of this e-mail are confidential and for the exclusive use
> of the intended recipient. If you receive this e-mail in error please
> delete it from your system immediately and notify us either by e-mail
> or telephone. You should not copy, forward or otherwise disclose the
> content of the e-mail. The views expressed in this communication may
> not necessarily be the view held by WHISHWORKS.
--
The contents of this e-mail are confidential and for the exclusive use of
the intended recipient. If you receive this e-mail in error please delete
it from your system immediately and notify us either by e-mail or
telephone. You should not copy, forward or otherwise disclose the content
of the e-mail. The views expressed in this communication may not
necessarily be the view held by WHISHWORKS.
