Hi Roman,
I have noticed a difference with different solr.xml config contents. It is
probably legit, but thought to let you know (tests run on fresh checkout as
of today).
As mentioned before, I have two cores configured in solr.xml. If the file
is:
[code]
<solr persistent="false">
<!--
adminPath: RequestHandler path to manage cores.
If 'null' (or absent), cores will not be manageable via request handler
-->
<cores adminPath="/admin/cores" host="${host:}"
hostPort="${jetty.port:8983}" hostContext="${hostContext:solr}">
<core name="metadata" instanceDir="metadata" />
<core name="statements" instanceDir="statements" />
</cores>
</solr>
[/code]
then the instruction:
python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60 -R
cms -t /solr/statements -e statements -U 100
works just fine. If however the solr.xml has adminPath set to "/admin"
solrjmeter produces an error:
[error]
**ERROR**
File "solrjmeter.py", line 1386, in <module>
main(sys.argv)
File "solrjmeter.py", line 1278, in main
check_prerequisities(options)
File "solrjmeter.py", line 375, in check_prerequisities
error('Cannot find admin pages: %s, please report a bug' % apath)
File "solrjmeter.py", line 66, in error
traceback.print_stack()
Cannot find admin pages: http://localhost:8983/solr/admin, please report a
bug
[/error]
With both solr.xml configs the following url returns just fine:
http://localhost:8983/solr/statements/admin/system?wt=json
Regards,
Dmitry
On Wed, Aug 14, 2013 at 2:03 PM, Dmitry Kan <solrexp...@gmail.com> wrote:
> Hi Roman,
>
> This looks much better, thanks! The ordinary non-comarison mode works.
> I'll post here, if there are other findings.
>
> Thanks for quick turnarounds,
>
> Dmitry
>
>
> On Wed, Aug 14, 2013 at 1:32 AM, Roman Chyla <roman.ch...@gmail.com>wrote:
>
>> Hi Dmitry, oh yes, late night fixes... :) The latest commit should make it
>> work for you.
>> Thanks!
>>
>> roman
>>
>>
>> On Tue, Aug 13, 2013 at 3:37 AM, Dmitry Kan <solrexp...@gmail.com> wrote:
>>
>> > Hi Roman,
>> >
>> > Something bad happened in fresh checkout:
>> >
>> > python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
>> > ./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60
>> -R
>> > cms -t /solr/statements -e statements -U 100
>> >
>> > Traceback (most recent call last):
>> > File "solrjmeter.py", line 1392, in <module>
>> > main(sys.argv)
>> > File "solrjmeter.py", line 1347, in main
>> > save_into_file('before-test.json', simplejson.dumps(before_test))
>> > File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line
>> 286,
>> > in dumps
>> > return _default_encoder.encode(obj)
>> > File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line
>> 226,
>> > in encode
>> > chunks = self.iterencode(o, _one_shot=True)
>> > File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line
>> 296,
>> > in iterencode
>> > return _iterencode(o, 0)
>> > File "/usr/lib/python2.7/dist-packages/simplejson/encoder.py", line
>> 202,
>> > in default
>> > raise TypeError(repr(o) + " is not JSON serializable")
>> > TypeError: <__main__.ForgivingValue object at 0x7fc6d4040fd0> is not
>> JSON
>> > serializable
>> >
>> >
>> > Regards,
>> >
>> > D.
>> >
>> >
>> > On Tue, Aug 13, 2013 at 8:10 AM, Roman Chyla <roman.ch...@gmail.com>
>> > wrote:
>> >
>> > > Hi Dmitry,
>> > >
>> > >
>> > >
>> > > On Mon, Aug 12, 2013 at 9:36 AM, Dmitry Kan <solrexp...@gmail.com>
>> > wrote:
>> > >
>> > > > Hi Roman,
>> > > >
>> > > > Good point. I managed to run the command with -C and double quotes:
>> > > >
>> > > > python solrjmeter.py -a -C "g1,cms" -c hour -x
>> ./jmx/SolrQueryTest.jmx
>> > > >
>> > > > As a result got several files (html, css, js, csv) in the running
>> > > directory
>> > > > (any way to specify where the output should be stored in this case?)
>> > > >
>> > >
>> > > i know it is confusing, i plan to change it - but later, now it is too
>> > busy
>> > > here...
>> > >
>> > >
>> > > >
>> > > > When I look onto the comparison dashboard, I see this:
>> > > >
>> > > > http://pbrd.co/17IRI0b
>> > > >
>> > >
>> > > two things: the tests probably took more than one hour to finish, so
>> they
>> > > are not aligned - try generating the comparison with '-c 14400' (ie.
>> > > 4x3600 secs)
>> > >
>> > > the other thing: if you have only two datapoints, the dygraph will not
>> > show
>> > > anything - there must be more datapoints/measurements
>> > >
>> > >
>> > >
>> > > >
>> > > > One more thing: all the previous tests were run with softCommit
>> > disabled.
>> > > > After enabling it, the tests started to fail:
>> > > >
>> > > > $ python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
>> > > > ./queries/demo/demo.queries -s localhost -p 8983 -a
>> --durationInSecs 60
>> > > -R
>> > > > g1 -t /solr/statements -e statements -U 100
>> > > > $ cd g1
>> > > > Reading results of the previous test
>> > > > $ cd 2013.08.12.16.32.48
>> > > > $ cd /home/dmitry/projects/lab/solrjmeter4/solrjmeter/g1
>> > > > $ mkdir 2013.08.12.16.33.02
>> > > > $ cd 2013.08.12.16.33.02
>> > > > $ cd /home/dmitry/projects/lab/solrjmeter4/solrjmeter/g1
>> > > > $ cd /home/dmitry/projects/lab/solrjmeter4/solrjmeter
>> > > > $ cd /home/dmitry/projects/lab/solrjmeter4/solrjmeter
>> > > > Traceback (most recent call last):
>> > > > File "solrjmeter.py", line 1427, in <module>
>> > > > main(sys.argv)
>> > > > File "solrjmeter.py", line 1381, in main
>> > > > before_test = harvest_details_about_montysolr(options)
>> > > > File "solrjmeter.py", line 562, in harvest_details_about_montysolr
>> > > > indexLstModified =
>> > cores_data['status'][cn]['index']['lastModified'],
>> > > > KeyError: 'lastModified'
>> > > >
>> > >
>> > > Thanks for letting me know, that info is probably not available in
>> this
>> > > situation - i've cooked st quick to fix it, please try the latest
>> commit
>> > > (hope it doesn't do more harm, i should get some sleep ..;))
>> > >
>> > > roman
>> > >
>> > >
>> > > >
>> > > > In case it matters: Python 2.7.3, ubuntu, solr 4.3.1.
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Dmitry
>> > > >
>> > > >
>> > > > On Thu, Aug 8, 2013 at 2:22 AM, Roman Chyla <roman.ch...@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Hi Dmitry,
>> > > > > The command seems good. Are you sure your shell is not doing
>> > something
>> > > > > funny with the params? You could try:
>> > > > >
>> > > > > python solrjmeter.py -C "g1,foo" -c hour -x
>> ./jmx/SolrQueryTest.jmx
>> > -a
>> > > > >
>> > > > > where g1 and foo are results of the individual runs, ie. something
>> > that
>> > > > was
>> > > > > started and saved with '-R g1' and '-R foo' respectively
>> > > > >
>> > > > > so, for example, i have these comparisons inside
>> > > > > '/var/lib/montysolr/different-java-settings/solrjmeter', so I am
>> > > > generating
>> > > > > the comparison by:
>> > > > >
>> > > > > export
>> > > > >
>> SOLRJMETER_HOME=/var/lib/montysolr/different-java-settings/solrjmeter
>> > > > > python solrjmeter.py -C "g1,foo" -c hour -x
>> ./jmx/SolrQueryTest.jmx
>> > -a
>> > > > >
>> > > > >
>> > > > > roman
>> > > > >
>> > > > >
>> > > > > On Wed, Aug 7, 2013 at 10:03 AM, Dmitry Kan <solrexp...@gmail.com
>> >
>> > > > wrote:
>> > > > >
>> > > > > > Hi Roman,
>> > > > > >
>> > > > > > One more question. I tried to compare different runs (g1 vs cms)
>> > > using
>> > > > > the
>> > > > > > command below, but get an error. Should I attach some other
>> > param(s)?
>> > > > > >
>> > > > > >
>> > > > > > python solrjmeter.py -C g1,foo -c hour -x
>> ./jmx/SolrQueryTest.jmx
>> > > > > > **ERROR**
>> > > > > > File "solrjmeter.py", line 1427, in <module>
>> > > > > > main(sys.argv)
>> > > > > > File "solrjmeter.py", line 1303, in main
>> > > > > > check_options(options, args)
>> > > > > > File "solrjmeter.py", line 185, in check_options
>> > > > > > error("The folder '%s' does not exist" % rf)
>> > > > > > File "solrjmeter.py", line 66, in error
>> > > > > > traceback.print_stack()
>> > > > > > The folder '0' does not exist
>> > > > > >
>> > > > > > Dmitry
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Aug 7, 2013 at 4:13 PM, Dmitry Kan <
>> solrexp...@gmail.com>
>> > > > wrote:
>> > > > > >
>> > > > > > > Hi Roman,
>> > > > > > >
>> > > > > > > Finally, this has worked! Thanks for quick support.
>> > > > > > >
>> > > > > > > The graphs look awesome. At least on the index sample :) It is
>> > > quite
>> > > > > easy
>> > > > > > > to setup and run + possible to run directly on the shard
>> server
>> > in
>> > > > > > > background mode.
>> > > > > > >
>> > > > > > > my test run was:
>> > > > > > >
>> > > > > > > python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q
>> > > > > > > ./queries/demo/demo.queries -s localhost -p 8983 -a
>> > > --durationInSecs
>> > > > 60
>> > > > > > -R
>> > > > > > > foo -t /solr/statements -e statements
>> > > > > > >
>> > > > > > > Thanks!
>> > > > > > >
>> > > > > > > Dmitry
>> > > > > > >
>> > > > > > >
>> > > > > > > On Wed, Aug 7, 2013 at 6:54 AM, Roman Chyla <
>> > roman.ch...@gmail.com
>> > > >
>> > > > > > wrote:
>> > > > > > >
>> > > > > > >> Hi Dmitry,
>> > > > > > >>
>> > > > > > >> I've modified the solrjmeter to retrieve data from under the
>> > core
>> > > > (the
>> > > > > > -t
>> > > > > > >> parameter) and the rest from the /solr/admin - I could test
>> it
>> > > only
>> > > > > > >> against
>> > > > > > >> 4.0, but it is there the same as 4.3 - it seems...so you can
>> try
>> > > the
>> > > > > > fresh
>> > > > > > >> checkout
>> > > > > > >>
>> > > > > > >> my test was: python solrjmeter.py -a -x
>> ./jmx/SolrQueryTest.jmx
>> > -t
>> > > > > > >> /solr/collection1 -R foo -q ./queries/demo/* -p 9002 -s
>> adsate
>> > > > > > >>
>> > > > > > >> Thanks!
>> > > > > > >>
>> > > > > > >> roman
>> > > > > > >>
>> > > > > > >>
>> > > > > > >> On Tue, Aug 6, 2013 at 9:46 AM, Dmitry Kan <
>> > solrexp...@gmail.com>
>> > > > > > wrote:
>> > > > > > >>
>> > > > > > >> > Hi,
>> > > > > > >> >
>> > > > > > >> > Thanks for the clarification, Shawn!
>> > > > > > >> >
>> > > > > > >> > So with this in mind, the following work:
>> > > > > > >> >
>> > > > > > >> > http://localhost:8983/solr/statements/admin/system?wt=json
>> > > > > > >> > http://localhost:8983/solr/statements/admin/mbeans?wt=json
>> > > > > > >> >
>> > > > > > >> > not copying their output to save space.
>> > > > > > >> >
>> > > > > > >> > Roman:
>> > > > > > >> >
>> > > > > > >> > is this something that should be set via -t parameter as
>> well?
>> > > > > > >> >
>> > > > > > >> > Dmitry
>> > > > > > >> >
>> > > > > > >> >
>> > > > > > >> >
>> > > > > > >> > On Tue, Aug 6, 2013 at 4:34 PM, Shawn Heisey <
>> > s...@elyograg.org
>> > > >
>> > > > > > wrote:
>> > > > > > >> >
>> > > > > > >> > > On 8/6/2013 6:17 AM, Dmitry Kan wrote:
>> > > > > > >> > > > Of three URLs you asked for, only the 3rd one gave
>> > response:
>> > > > > > >> > > <snip>
>> > > > > > >> > > > The rest report 404.
>> > > > > > >> > > >
>> > > > > > >> > > > On Mon, Aug 5, 2013 at 8:38 PM, Roman Chyla <
>> > > > > > roman.ch...@gmail.com>
>> > > > > > >> > > wrote:
>> > > > > > >> > > >
>> > > > > > >> > > >> Hi Dmitry,
>> > > > > > >> > > >> So I think the admin pages are different on your
>> version
>> > of
>> > > > > solr,
>> > > > > > >> what
>> > > > > > >> > > do
>> > > > > > >> > > >> you see when you request... ?
>> > > > > > >> > > >>
>> > > > > > >> > > >> http://localhost:8983/solr/admin/system?wt=json
>> > > > > > >> > > >> http://localhost:8983/solr/admin/mbeans?wt=json
>> > > > > > >> > > >> http://localhost:8983/solr/admin/cores?wt=json
>> > > > > > >> > >
>> > > > > > >> > > Unless you have a valid defaultCoreName set in your
>> > > (old-style)
>> > > > > > >> > > solr.xml, the first two URLs won't work, as you've
>> > discovered.
>> > > > > > >> Without
>> > > > > > >> > > that valid defaultCoreName (or if you wanted info from a
>> > > > different
>> > > > > > >> > > core), you'd need to add a core name to the URL for them
>> to
>> > > > work.
>> > > > > > >> > >
>> > > > > > >> > > The third one, which works for you, is a global handler
>> for
>> > > > > > >> manipulating
>> > > > > > >> > > cores, so naturally it doesn't need a core name to
>> function.
>> > > > The
>> > > > > > URL
>> > > > > > >> > > path for this handler is defined by solr.xml.
>> > > > > > >> > >
>> > > > > > >> > > Thanks,
>> > > > > > >> > > Shawn
>> > > > > > >> > >
>> > > > > > >> > >
>> > > > > > >> >
>> > > > > > >>
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
