Testing/packaging question
Hi, I'm pretty much of a Solr newbie currently packaging solrpy for Debian; see http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ In order to run solrpy's supplied tests at build time, I'd need Solr to know about the schema.xml that comes with the tests. Can anyone tell me how do that properly? I'd basically need Solr to temporarily recognize that schema.xml without permanently installing it -- is there any way to do this, eg via environment variables? TIA Bernhard Reiter
Re: Testing/packaging question
Hi, I'm now trying to export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml" and restarting tomcat (v6 package from ubuntu maverick) via sudo /etc/init.d/tomcat6 restart but solr still doesn't seem to find that schema.xml, as it complains about unknown fields when running the tests that require that schema.xml Can someone please tell me what I'm doing wrong -- and what I should be doing? TIA again, Bernhard Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: > Hi, > > I'm pretty much of a Solr newbie currently packaging solrpy for Debian; > see > http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ > > In order to run solrpy's supplied tests at build time, I'd need Solr to > know about the schema.xml that comes with the tests. > Can anyone tell me how do that properly? I'd basically need Solr to > temporarily recognize that schema.xml without permanently installing it > -- is there any way to do this, eg via environment variables? > > TIA > Bernhard Reiter
RE: Testing/packaging question
The thing is, I only have a schema.xml -- no data, no lib directories. See the tests subdirectory in the solrpy package: http://pypi.python.org/packages/source/s/solrpy/solrpy-0.9.3.tar.gz Bernhard Am Donnerstag, den 04.11.2010, 15:59 -0500 schrieb Olson, Ron: > I believe it should point to the directory above, where conf and lib are > located (though I have a multi-core setup). > > Mine is set to: > > /usr/local/jboss-5.1.0.GA/server/solr/solr_data/ > > And in solr_data the solr.xml defines the two cores, but in each core > directory, is a conf, data, and lib directory, which contains the schema.xml. > > > > -Original Message- > From: Bernhard Reiter [mailto:ock...@raz.or.at] > Sent: Thursday, November 04, 2010 3:49 PM > To: solr-user@lucene.apache.org > Subject: Re: Testing/packaging question > > Hi, > > I'm now trying to > > export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml" > > and restarting tomcat (v6 package from ubuntu maverick) via > > sudo /etc/init.d/tomcat6 restart > > but solr still doesn't seem to find that schema.xml, as it complains > about unknown fields when running the tests that require that schema.xml > > Can someone please tell me what I'm doing wrong -- and what I should be > doing? > > TIA again, > Bernhard > > Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: > > Hi, > > > > I'm pretty much of a Solr newbie currently packaging solrpy for Debian; > > see > > http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ > > > > In order to run solrpy's supplied tests at build time, I'd need Solr to > > know about the schema.xml that comes with the tests. > > Can anyone tell me how do that properly? I'd basically need Solr to > > temporarily recognize that schema.xml without permanently installing it > > -- is there any way to do this, eg via environment variables? > > > > TIA > > Bernhard Reiter > > > > > DISCLAIMER: This electronic message, including any attachments, files or > documents, is intended only for the addressee and may contain CONFIDENTIAL, > PROPRIETARY or LEGALLY PRIVILEGED information. If you are not the intended > recipient, you are hereby notified that any use, disclosure, copying or > distribution of this message or any of the information included in or with it > is unauthorized and strictly prohibited. If you have received this message > in error, please notify the sender immediately by reply e-mail and > permanently delete and destroy this message and its attachments, along with > any copies thereof. This message does not create any contractual obligation > on behalf of the sender or Law Bulletin Publishing Company. > Thank you.
RE: Testing/packaging question
Thanks for your instructions. Unfortunately, I need to do all that as part of my package's (python-solrpy) build procedure, so I can't change any global configuration, such as in the catalina subdirectories. I've already sensed that restarting tomcat is also just too system-invasive and would include changing its (system-wide) configuration. Are there any other ways to use solr for running the tests from http://pypi.python.org/packages/source/s/solrpy/solrpy-0.9.3.tar.gz without having to change any system configuration? Maybe via a user Tomcat instance such as provided by the tomcat6-user debian package? Thanks for your help! Bernhard Am Donnerstag, den 04.11.2010, 16:15 -0500 schrieb Turner, Robbin J: > You need to either add that to catalina.sh or create a setenv.sh in the > CATALINA_HOME/bin directory. Then you can restart tomcat. > > So, setenv.sh would contain the following: > >export JAVA_HOME="/path/to/jre" >export JAVA_OPTS="="$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml" > > If you were setting the export in your own environment and then issuing the > restart, tomcat was not picking up your local environment because it's > running as root. You don't want to change root's environment. > > You could also, create a context.xml in you > CATALINA_HOME/conf/CATALINA/localhost. You should be able to find those > instruction on/through the Solr FAQ. > > Hope this helps. > > From: Bernhard Reiter [ock...@raz.or.at] > Sent: Thursday, November 04, 2010 4:49 PM > To: solr-user@lucene.apache.org > Subject: Re: Testing/packaging question > > Hi, > > I'm now trying to > > export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml" > > and restarting tomcat (v6 package from ubuntu maverick) via > > sudo /etc/init.d/tomcat6 restart > > but solr still doesn't seem to find that schema.xml, as it complains > about unknown fields when running the tests that require that schema.xml > > Can someone please tell me what I'm doing wrong -- and what I should be > doing? > > TIA again, > Bernhard > > Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: > > Hi, > > > > I'm pretty much of a Solr newbie currently packaging solrpy for Debian; > > see > > http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ > > > > In order to run solrpy's supplied tests at build time, I'd need Solr to > > know about the schema.xml that comes with the tests. > > Can anyone tell me how do that properly? I'd basically need Solr to > > temporarily recognize that schema.xml without permanently installing it > > -- is there any way to do this, eg via environment variables? > > > > TIA > > Bernhard Reiter
Re: Testing/packaging question
Hi,
and thanks for your hints. I've done some additional research and found
that there doesn't really seem to be any possibility of an embedded solr
server in solrpy.
Jetty, then. It'd all be probably kinda easy if it weren't for the way
things are unbundled in debian. I've recently posted to the debian-java
ML, but received no replay -- maybe because it's so solr-specific (so,
sorry for cross-posting here). It's kinda long as I've tried to capture
the issues I've run into so far. I'd really appreciate any help -- maybe
there are some solr ninjas here that know how to work around the
problems I'm experiencing.
> [...]
> The tests are basically a python file and a corresponding schema.xml.
> Normally, it's suggested to just use solr's example directory (with
> integrated jetty) and replace its schema.xml with the one provided by
> solrpy, and then run it via java -jar start.jar
>
>
>
> On debian however, solr and jetty are unbundled (for good reason), and
i
> can't of course just replace solr's example schema.xml (which gets
> installed to /etc/solr/conf).
>
> I've started by looking for ways to run solr on jetty as non-root by
> basically running
>
> java -jar /usr/share/jetty/start.jar
>
> But that is highly non-trivial, as jetty's configuration files seem to
> point to solr's in a pretty hardcoded way.
>
> I was able to change jetty's logs and home location by
>
> java -jar -Djetty.home=/usr/share/jetty
>
-Djetty.logs=/home/me/solr/logs /usr/share/jetty/start.jar /etc/jetty/jetty.xml
>
> I can run jetty as a non-root user by issuing that command from
> within /usr/share/jetty, but it fails to start solr, as I haven't yet
> found a way to make solr use another datadir than /var/lib/solr/data
--
> which is hardcoded in /etc/solr/conf/solrconfig.xml.
>
> I think I might be able to change the latter by having the solr-common
> package change its /etc/solr/conf/solrconfig.xml to read
>
> ${solr.data.dir}
>
> instead of
>
> /var/lib/solr/data
>
> and put a solrcore.properties into solr.home
>
> #solrcore.properties
> data.dir=/data/solrindex
>
> ( as seen over at
>
http://wiki.apache.org/solr/SolrConfigXml#System_property_substitution )
> -- or just pass -Dsolr.data.dir=/data/dir from within some startup
> script.
>
> Would such a modification of solr-common have any chance to be
accepted
> by the java team?
>
> ---
>
> With schema.xml, I'm rather clueless. -Dsolr.solr.home=/home/me/solr/
> doesn't do the trick as this stuff is also hardcoded
> (in /usr/share/solr/WEB-INF/jetty-web.xml -- which is part of
> solr-jetty). Actually, that's the only purpose of jetty-web.xml, which
> is evaluated because of jetty's context settings (I think). Moreover,
> I'm trying not to duplicate the other contents of that directory, so
> setting solr.solr.home to a different location might not be what I
want
> at all.
>
> I think my most relevant questions are:
> 1. Is anyone able to successfully run jetty with solr
> via /usr/share/jetty/start.jar as a non-root user (ideally from within
> any directory, not just from within /usr/share/jetty)? If yes: how?
> 2. How can I make solr use my schema.xml instead of the one installed?
> 3. I haven't tried the whole thing in tomcat yet, so does anyone know
if
> it would be easier there -- maybe because configuration files aren't
> that much entangled and hardcoded there? (In the end, I might have to
do
> the whole thing also for tomcat anyway as I'm build-depending on
> solr-jetty | solr-tomcat which conflict each other).
>
> I'm kinda desperate with the load of configuration files, so I'd
really
> appreciate any help that gets me closer to get this thing done.
> Kind regards
> Bernhard
Am Donnerstag, den 04.11.2010, 23:57 +0100 schrieb Peter Karich:
> Hi,
>
> don't know if the python package provides one but solrj offers to start
> solr embedded (|EmbeddedSolrServer|) and
> setting up different schema + config is possible. for this see:
> https://karussell.wordpress.com/2010/06/10/how-to-test-apache-solrj/
>
> if you need an 'external solr' (via jetty and java -jar start.jar) while
> tests running see this:
> http://java.dzone.com/articles/getting-know-solr
>
> Regards,
> Peter.
>
>
> > Hi,
> >
> > I'm pretty much of a Solr newbie currently packaging solrpy for Debian;
> > see
> > http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/
> >
> > In order to run solrpy's supplied tests at build time, I'd need Solr to
> > know about the schema.xml that comes with the tests.
> > Can anyone tell me how do that properly? I'd basically need Solr to
> > temporarily recognize that schema.xml without permanently installing it
> > -- is there any way to do this, eg via environment variables?
> >
> > TIA
> > Bernhard Reiter
> >
>
RE: Testing/packaging question
Hi, I have up to now focussed on Jetty as it's already bundled with solr. The main issue there seems to be the way it's unbundled by Debian; I figure things might be similar with Tomcat, depending on how entangled configuration is there. Before I dig deeper into the Tomcat option: would you mind sending me the script you mention? Maybe that could clear up things a bit. Regards Bernhard Am Donnerstag, den 04.11.2010, 17:16 -0500 schrieb Turner, Robbin J: > You can setup your own tomcat instance which would contain just > configurations you need. You won't even have to recreate all the tomcat > configuration and binaries, just the ones that were not defaults. So, if you > lookup multiple tomcat configuration instance (google it), and then you'll > have a set of directories. You'll need to have your own startup script that > points to your configurations. You can use the current startup script as a > model, then in your build procedures (I've done all this with a script) have > this added to the system so you can preform restart. You'd have to have a > couple of other environment variables set: > > export CATALINA_BASE=/path/to/your/tomcat/instance/conf/files > export CATALINA_HOME=/path/to/default/installation/bin/files > export SOLR_HOME=/path/to/solr/dataNconf > > Good luck > > > From: Bernhard Reiter [ock...@raz.or.at] > Sent: Thursday, November 04, 2010 5:49 PM > To: solr-user@lucene.apache.org > Subject: RE: Testing/packaging question > > Thanks for your instructions. Unfortunately, I need to do all that as > part of my package's (python-solrpy) build procedure, so I can't change > any global configuration, such as in the catalina subdirectories. > > I've already sensed that restarting tomcat is also just too > system-invasive and would include changing its (system-wide) > configuration. > > Are there any other ways to use solr for running the tests from > http://pypi.python.org/packages/source/s/solrpy/solrpy-0.9.3.tar.gz > without having to change any system configuration? Maybe via a user > Tomcat instance such as provided by the tomcat6-user debian package? > > Thanks for your help! > Bernhard > > Am Donnerstag, den 04.11.2010, 16:15 -0500 schrieb Turner, Robbin J: > > You need to either add that to catalina.sh or create a setenv.sh in the > > CATALINA_HOME/bin directory. Then you can restart tomcat. > > > > So, setenv.sh would contain the following: > > > >export JAVA_HOME="/path/to/jre" > >export JAVA_OPTS="="$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml" > > > > If you were setting the export in your own environment and then issuing the > > restart, tomcat was not picking up your local environment because it's > > running as root. You don't want to change root's environment. > > > > You could also, create a context.xml in you > > CATALINA_HOME/conf/CATALINA/localhost. You should be able to find those > > instruction on/through the Solr FAQ. > > > > Hope this helps. > > > > From: Bernhard Reiter [ock...@raz.or.at] > > Sent: Thursday, November 04, 2010 4:49 PM > > To: solr-user@lucene.apache.org > > Subject: Re: Testing/packaging question > > > > Hi, > > > > I'm now trying to > > > > export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=/path/to/my/schema.xml" > > > > and restarting tomcat (v6 package from ubuntu maverick) via > > > > sudo /etc/init.d/tomcat6 restart > > > > but solr still doesn't seem to find that schema.xml, as it complains > > about unknown fields when running the tests that require that schema.xml > > > > Can someone please tell me what I'm doing wrong -- and what I should be > > doing? > > > > TIA again, > > Bernhard > > > > Am Montag, den 01.11.2010, 19:01 +0100 schrieb Bernhard Reiter: > > > Hi, > > > > > > I'm pretty much of a Solr newbie currently packaging solrpy for Debian; > > > see > > > http://svn.debian.org/viewsvn/python-modules/packages/python-solrpy/trunk/ > > > > > > In order to run solrpy's supplied tests at build time, I'd need Solr to > > > know about the schema.xml that comes with the tests. > > > Can anyone tell me how do that properly? I'd basically need Solr to > > > temporarily recognize that schema.xml without permanently installing it > > > -- is there any way to do this, eg via environment variables? > > > > > > TIA > > > Bernhard Reiter
