Hi Dmitry,
So it seems solrjmeter should not assume the adminPath - and perhaps needs
to be passed as an argument. When you set the adminPath, are you able to
access localhost:8983/solr/statements/admin/cores ?
roman
On Wed, Aug 21, 2013 at 7:36 AM, Dmitry Kan <solrexp...@gmail.com> wrote:
> Hi Roman,
>
> I have noticed a difference with different solr.xml config contents. It is
> probably legit, but thought to let you know (tests run on fresh checkout as
> of today).
>
> As mentioned before, I have two cores configured in solr.xml. If the file
> is:
>
> [code]
> <solr persistent="false">
>
> <!--
> adminPath: RequestHandler path to manage cores.
> If 'null' (or absent), cores will not be manageable via request handler
> -->
> <cores adminPath="/admin/cores" host="${host:}"
> hostPort="${jetty.port:8983}" hostContext="${hostContext:solr}">
> <core name="metadata" instanceDir="metadata" />
> <core name="statements" instanceDir="statements" />
> </cores>
> </solr>
> [/code]
>
> then the instruction:
>
> python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
> ./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60 -R
> cms -t /solr/statements -e statements -U 100
>
> works just fine. If however the solr.xml has adminPath set to "/admin"
> solrjmeter produces an error:
>
> [error]
> **ERROR**
> File "solrjmeter.py", line 1386, in <module>
> main(sys.argv)
> File "solrjmeter.py", line 1278, in main
> check_prerequisities(options)
> File "solrjmeter.py", line 375, in check_prerequisities
> error('Cannot find admin pages: %s, please report a bug' % apath)
> File "solrjmeter.py", line 66, in error
> traceback.print_stack()
> Cannot find admin pages: http://localhost:8983/solr/admin, please report a
> bug
> [/error]
>
> With both solr.xml configs the following url returns just fine:
>
> http://localhost:8983/solr/statements/admin/system?wt=json
>
> Regards,
>
> Dmitry
>
>
>
> On Wed, Aug 14, 2013 at 2:03 PM, Dmitry Kan <solrexp...@gmail.com> wrote:
>
> > Hi Roman,
> >
> > This looks much better, thanks! The ordinary non-comarison mode works.
> > I'll post here, if there are other findings.
> >
> > Thanks for quick turnarounds,
> >
> > Dmitry
> >
> >
> > On Wed, Aug 14, 2013 at 1:32 AM, Roman Chyla <roman.ch...@gmail.com
> >wrote:
> >
> >> Hi Dmitry, oh yes, late night fixes... :) The latest commit should make
> it
> >> work for you.
> >> Thanks!
> >>
> >> roman
> >>
> >>
> >> On Tue, Aug 13, 2013 at 3:37 AM, Dmitry Kan <solrexp...@gmail.com>
> wrote:
> >>
> >> > Hi Roman,
> >> >
> >> > Something bad happened in fresh checkout:
> >> >
> >> > python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
> >> > ./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs
> 60
> >> -R
> >> > cms -t /solr/statements -e statements -U 100
> >> >
> >> > Traceback (most recent call last):
> >> > File "solrjmeter.py", line 1392, in <module>
> >> > main(sys.argv)
> >> > File "solrjmeter.py", line 1347, in main
> >> > save_into_file('before-test.json', simplejson.dumps(before_test))
> >> > File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line
> >> 286,
> >> > in dumps
> >> > return _default_encoder.encode(obj)
> >> > File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line
> >> 226,
> >> > in encode
> >> > chunks = self.iterencode(o, _one_shot=True)
> >> > File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line
> >> 296,
> >> > in iterencode
> >> > return _iterencode(o, 0)
> >> > File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line
> >> 202,
> >> > in default
> >> > raise TypeError(repr(o) + " is not JSON serializable")
> >> > TypeError: <__main__.ForgivingValue object at 0x7fc6d4040fd0> is not
> >> JSON
> >> > serializable
> >> >
> >> >
> >> > Regards,
> >> >
> >> > D.
> >> >
> >> >
> >> > On Tue, Aug 13, 2013 at 8:10 AM, Roman Chyla <roman.ch...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi Dmitry,
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Aug 12, 2013 at 9:36 AM, Dmitry Kan <solrexp...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Hi Roman,
> >> > > >
> >> > > > Good point. I managed to run the command with -C and double
> quotes:
> >> > > >
> >> > > > python solrjmeter.py -a -C "g1,cms" -c hour -x
> >> ./jmx/SolrQueryTest.jmx
> >> > > >
> >> > > > As a result got several files (html, css, js, csv) in the running
> >> > > directory
> >> > > > (any way to specify where the output should be stored in this
> case?)
> >> > > >
> >> > >
> >> > > i know it is confusing, i plan to change it - but later, now it is
> too
> >> > busy
> >> > > here...
> >> > >
> >> > >
> >> > > >
> >> > > > When I look onto the comparison dashboard, I see this:
> >> > > >
> >> > > > http://pbrd.co/17IRI0b
> >> > > >
> >> > >
> >> > > two things: the tests probably took more than one hour to finish, so
> >> they
> >> > > are not aligned - try generating the comparison with '-c 14400'
> (ie.
> >> > > 4x3600 secs)
> >> > >
> >> > > the other thing: if you have only two datapoints, the dygraph will
> not
> >> > show
> >> > > anything - there must be more datapoints/measurements
> >> > >
> >> > >
> >> > >
> >> > > >
> >> > > > One more thing: all the previous tests were run with softCommit
> >> > disabled.
> >> > > > After enabling it, the tests started to fail:
> >> > > >
> >> > > > $ python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
> >> > > > ./queries/demo/demo.queries -s localhost -p 8983 -a
> >> --durationInSecs 60
> >> > > -R
> >> > > > g1 -t /solr/statements -e statements -U 100
> >> > > > $ cd g1
> >> > > > Reading results of the previous test
> >> > > > $ cd 2013.08.12.16.32.48
> >> > > > $ cd /home/dmitry/projects/lab/solrjmeter4/solrjmeter/g1
> >> > > > $ mkdir 2013.08.12.16.33.02
> >> > > > $ cd 2013.08.12.16.33.02
> >> > > > $ cd /home/dmitry/projects/lab/solrjmeter4/solrjmeter/g1
> >> > > > $ cd /home/dmitry/projects/lab/solrjmeter4/solrjmeter
> >> > > > $ cd /home/dmitry/projects/lab/solrjmeter4/solrjmeter
> >> > > > Traceback (most recent call last):
> >> > > > File "solrjmeter.py", line 1427, in <module>
> >> > > > main(sys.argv)
> >> > > > File "solrjmeter.py", line 1381, in main
> >> > > > before_test = harvest_details_about_montysolr(options)
> >> > > > File "solrjmeter.py", line 562, in
> harvest_details_about_montysolr
> >> > > > indexLstModified =
> >> > cores_data['status'][cn]['index']['lastModified'],
> >> > > > KeyError: 'lastModified'
> >> > > >
> >> > >
> >> > > Thanks for letting me know, that info is probably not available in
> >> this
> >> > > situation - i've cooked st quick to fix it, please try the latest
> >> commit
> >> > > (hope it doesn't do more harm, i should get some sleep ..;))
> >> > >
> >> > > roman
> >> > >
> >> > >
> >> > > >
> >> > > > In case it matters: Python 2.7.3, ubuntu, solr 4.3.1.
> >> > > >
> >> > > > Thanks,
> >> > > >
> >> > > > Dmitry
> >> > > >
> >> > > >
> >> > > > On Thu, Aug 8, 2013 at 2:22 AM, Roman Chyla <
> roman.ch...@gmail.com>
> >> > > wrote:
> >> > > >
> >> > > > > Hi Dmitry,
> >> > > > > The command seems good. Are you sure your shell is not doing
> >> > something
> >> > > > > funny with the params? You could try:
> >> > > > >
> >> > > > > python solrjmeter.py -C "g1,foo" -c hour -x
> >> ./jmx/SolrQueryTest.jmx
> >> > -a
> >> > > > >
> >> > > > > where g1 and foo are results of the individual runs, ie.
> something
> >> > that
> >> > > > was
> >> > > > > started and saved with '-R g1' and '-R foo' respectively
> >> > > > >
> >> > > > > so, for example, i have these comparisons inside
> >> > > > > '/var/lib/montysolr/different-java-settings/solrjmeter', so I am
> >> > > > generating
> >> > > > > the comparison by:
> >> > > > >
> >> > > > > export
> >> > > > >
> >> SOLRJMETER_HOME=/var/lib/montysolr/different-java-settings/solrjmeter
> >> > > > > python solrjmeter.py -C "g1,foo" -c hour -x
> >> ./jmx/SolrQueryTest.jmx
> >> > -a
> >> > > > >
> >> > > > >
> >> > > > > roman
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Aug 7, 2013 at 10:03 AM, Dmitry Kan <
> solrexp...@gmail.com
> >> >
> >> > > > wrote:
> >> > > > >
> >> > > > > > Hi Roman,
> >> > > > > >
> >> > > > > > One more question. I tried to compare different runs (g1 vs
> cms)
> >> > > using
> >> > > > > the
> >> > > > > > command below, but get an error. Should I attach some other
> >> > param(s)?
> >> > > > > >
> >> > > > > >
> >> > > > > > python solrjmeter.py -C g1,foo -c hour -x
> >> ./jmx/SolrQueryTest.jmx
> >> > > > > > **ERROR**
> >> > > > > > File "solrjmeter.py", line 1427, in <module>
> >> > > > > > main(sys.argv)
> >> > > > > > File "solrjmeter.py", line 1303, in main
> >> > > > > > check_options(options, args)
> >> > > > > > File "solrjmeter.py", line 185, in check_options
> >> > > > > > error("The folder '%s' does not exist" % rf)
> >> > > > > > File "solrjmeter.py", line 66, in error
> >> > > > > > traceback.print_stack()
> >> > > > > > The folder '0' does not exist
> >> > > > > >
> >> > > > > > Dmitry
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, Aug 7, 2013 at 4:13 PM, Dmitry Kan <
> >> solrexp...@gmail.com>
> >> > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Roman,
> >> > > > > > >
> >> > > > > > > Finally, this has worked! Thanks for quick support.
> >> > > > > > >
> >> > > > > > > The graphs look awesome. At least on the index sample :) It
> is
> >> > > quite
> >> > > > > easy
> >> > > > > > > to setup and run + possible to run directly on the shard
> >> server
> >> > in
> >> > > > > > > background mode.
> >> > > > > > >
> >> > > > > > > my test run was:
> >> > > > > > >
> >> > > > > > > python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
> >> > > > > > > ./queries/demo/demo.queries -s localhost -p 8983 -a
> >> > > --durationInSecs
> >> > > > 60
> >> > > > > > -R
> >> > > > > > > foo -t /solr/statements -e statements
> >> > > > > > >
> >> > > > > > > Thanks!
> >> > > > > > >
> >> > > > > > > Dmitry
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Wed, Aug 7, 2013 at 6:54 AM, Roman Chyla <
> >> > roman.ch...@gmail.com
> >> > > >
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > >> Hi Dmitry,
> >> > > > > > >>
> >> > > > > > >> I've modified the solrjmeter to retrieve data from under
> the
> >> > core
> >> > > > (the
> >> > > > > > -t
> >> > > > > > >> parameter) and the rest from the /solr/admin - I could test
> >> it
> >> > > only
> >> > > > > > >> against
> >> > > > > > >> 4.0, but it is there the same as 4.3 - it seems...so you
> can
> >> try
> >> > > the
> >> > > > > > fresh
> >> > > > > > >> checkout
> >> > > > > > >>
> >> > > > > > >> my test was: python solrjmeter.py -a -x
> >> ./jmx/SolrQueryTest.jmx
> >> > -t
> >> > > > > > >> /solr/collection1 -R foo -q ./queries/demo/* -p 9002 -s
> >> adsate
> >> > > > > > >>
> >> > > > > > >> Thanks!
> >> > > > > > >>
> >> > > > > > >> roman
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >> On Tue, Aug 6, 2013 at 9:46 AM, Dmitry Kan <
> >> > solrexp...@gmail.com>
> >> > > > > > wrote:
> >> > > > > > >>
> >> > > > > > >> > Hi,
> >> > > > > > >> >
> >> > > > > > >> > Thanks for the clarification, Shawn!
> >> > > > > > >> >
> >> > > > > > >> > So with this in mind, the following work:
> >> > > > > > >> >
> >> > > > > > >> >
> http://localhost:8983/solr/statements/admin/system?wt=json
> >> > > > > > >> >
> http://localhost:8983/solr/statements/admin/mbeans?wt=json
> >> > > > > > >> >
> >> > > > > > >> > not copying their output to save space.
> >> > > > > > >> >
> >> > > > > > >> > Roman:
> >> > > > > > >> >
> >> > > > > > >> > is this something that should be set via -t parameter as
> >> well?
> >> > > > > > >> >
> >> > > > > > >> > Dmitry
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> >
> >> > > > > > >> > On Tue, Aug 6, 2013 at 4:34 PM, Shawn Heisey <
> >> > s...@elyograg.org
> >> > > >
> >> > > > > > wrote:
> >> > > > > > >> >
> >> > > > > > >> > > On 8/6/2013 6:17 AM, Dmitry Kan wrote:
> >> > > > > > >> > > > Of three URLs you asked for, only the 3rd one gave
> >> > response:
> >> > > > > > >> > > <snip>
> >> > > > > > >> > > > The rest report 404.
> >> > > > > > >> > > >
> >> > > > > > >> > > > On Mon, Aug 5, 2013 at 8:38 PM, Roman Chyla <
> >> > > > > > roman.ch...@gmail.com>
> >> > > > > > >> > > wrote:
> >> > > > > > >> > > >
> >> > > > > > >> > > >> Hi Dmitry,
> >> > > > > > >> > > >> So I think the admin pages are different on your
> >> version
> >> > of
> >> > > > > solr,
> >> > > > > > >> what
> >> > > > > > >> > > do
> >> > > > > > >> > > >> you see when you request... ?
> >> > > > > > >> > > >>
> >> > > > > > >> > > >> http://localhost:8983/solr/admin/system?wt=json
> >> > > > > > >> > > >> http://localhost:8983/solr/admin/mbeans?wt=json
> >> > > > > > >> > > >> http://localhost:8983/solr/admin/cores?wt=json
> >> > > > > > >> > >
> >> > > > > > >> > > Unless you have a valid defaultCoreName set in your
> >> > > (old-style)
> >> > > > > > >> > > solr.xml, the first two URLs won't work, as you've
> >> > discovered.
> >> > > > > > >> Without
> >> > > > > > >> > > that valid defaultCoreName (or if you wanted info from
> a
> >> > > > different
> >> > > > > > >> > > core), you'd need to add a core name to the URL for
> them
> >> to
> >> > > > work.
> >> > > > > > >> > >
> >> > > > > > >> > > The third one, which works for you, is a global handler
> >> for
> >> > > > > > >> manipulating
> >> > > > > > >> > > cores, so naturally it doesn't need a core name to
> >> function.
> >> > > > The
> >> > > > > > URL
> >> > > > > > >> > > path for this handler is defined by solr.xml.
> >> > > > > > >> > >
> >> > > > > > >> > > Thanks,
> >> > > > > > >> > > Shawn
> >> > > > > > >> > >
> >> > > > > > >> > >
> >> > > > > > >> >
> >> > > > > > >>
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
