Re: Measuring SOLR performance

2013-09-04 Thread Dmitry Kan
Hi Roman, Ok, I will. Thanks! Cheers, Dmitry On Tue, Sep 3, 2013 at 4:46 PM, Roman Chyla wrote: > Hi Dmitry, > > Thanks for the feedback. Yes, it is indeed jmeter issue (or rather, the > issue of the plugin we use to generate charts). You may want to use the > github for whatever comes next >

Re: Measuring SOLR performance

2013-09-03 Thread Roman Chyla
Hi Dmitry, Thanks for the feedback. Yes, it is indeed jmeter issue (or rather, the issue of the plugin we use to generate charts). You may want to use the github for whatever comes next https://github.com/romanchyla/solrjmeter/issues Cheers, roman On Tue, Sep 3, 2013 at 7:54 AM, Dmitry Kan

Re: Measuring SOLR performance

2013-09-03 Thread Dmitry Kan
Hi Roman, Thanks, the --additionalSolrParams was just what I wanted and works fine. BTW, if you have some special "bug tracking forum" for the tool, I'm happy to submit questions / bug reports there. Otherwise, this email list is ok (for me at least). One other thing I have noticed in the err lo

Re: Measuring SOLR performance

2013-09-02 Thread Roman Chyla
Hi Dmitry, If it is something you want to pass with every request (which is my use case), you can pass it as additional solr params, eg. python solrjmeter --additionalSolrParams="fq=other_field:bar+facet=true+facet.field=facet_field_name" the string should be url encoded. If it is somethin

Re: Measuring SOLR performance

2013-09-02 Thread Dmitry Kan
Hi Erick, Agree, this is perfectly fine to mix them in solr. But my question is about solrjmeter input query format. Just couldn't find a suitable example on the solrjmeter's github. Dmitry On Mon, Sep 2, 2013 at 5:40 PM, Erick Erickson wrote: > filter and facet queries can be freely intermix

Re: Measuring SOLR performance

2013-09-02 Thread Erick Erickson
filter and facet queries can be freely intermixed, it's not a problem. What problem are you seeing when you try this? Best, Erick On Mon, Sep 2, 2013 at 7:46 AM, Dmitry Kan wrote: > Hi Roman, > > What's the format for running the facet+filter queries? > > Would something like this work: > > fi

Re: Measuring SOLR performance

2013-09-02 Thread Dmitry Kan
Hi Roman, What's the format for running the facet+filter queries? Would something like this work: field:foo >=50 fq=other_field:bar facet=true facet.field=facet_field_name Thanks, Dmitry On Fri, Aug 23, 2013 at 2:34 PM, Dmitry Kan wrote: > Hi Roman, > > With adminPath="/admin" or adminP

Re: Measuring SOLR performance

2013-08-23 Thread Dmitry Kan
Hi Roman, With adminPath="/admin" or adminPath="/admin/cores", no. Interestingly enough, though, I can access http://localhost:8983/solr/statements/admin/system But I can access http://localhost:8983/solr/admin/cores, only when with adminPath="/admin/cores" (which suggests that this is the right

Re: Measuring SOLR performance

2013-08-22 Thread Roman Chyla
Hi Dmitry, So it seems solrjmeter should not assume the adminPath - and perhaps needs to be passed as an argument. When you set the adminPath, are you able to access localhost:8983/solr/statements/admin/cores ? roman On Wed, Aug 21, 2013 at 7:36 AM, Dmitry Kan wrote: > Hi Roman, > > I have not

Re: Measuring SOLR performance

2013-08-21 Thread Dmitry Kan
Hi Roman, I have noticed a difference with different solr.xml config contents. It is probably legit, but thought to let you know (tests run on fresh checkout as of today). As mentioned before, I have two cores configured in solr.xml. If the file is: [code] [/code] then the

Re: Measuring SOLR performance

2013-08-14 Thread Dmitry Kan
Hi Roman, This looks much better, thanks! The ordinary non-comarison mode works. I'll post here, if there are other findings. Thanks for quick turnarounds, Dmitry On Wed, Aug 14, 2013 at 1:32 AM, Roman Chyla wrote: > Hi Dmitry, oh yes, late night fixes... :) The latest commit should make it

Re: Measuring SOLR performance

2013-08-13 Thread Roman Chyla
Hi Dmitry, oh yes, late night fixes... :) The latest commit should make it work for you. Thanks! roman On Tue, Aug 13, 2013 at 3:37 AM, Dmitry Kan wrote: > Hi Roman, > > Something bad happened in fresh checkout: > > python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q > ./queries/demo/demo.qu

Re: Measuring SOLR performance

2013-08-13 Thread Dmitry Kan
Hi Roman, Something bad happened in fresh checkout: python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q ./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60 -R cms -t /solr/statements -e statements -U 100 Traceback (most recent call last): File "solrjmeter.py", line 1392,

Re: Measuring SOLR performance

2013-08-12 Thread Roman Chyla
Hi Dmitry, On Mon, Aug 12, 2013 at 9:36 AM, Dmitry Kan wrote: > Hi Roman, > > Good point. I managed to run the command with -C and double quotes: > > python solrjmeter.py -a -C "g1,cms" -c hour -x ./jmx/SolrQueryTest.jmx > > As a result got several files (html, css, js, csv) in the running dir

Re: Measuring SOLR performance

2013-08-12 Thread Dmitry Kan
Hi Roman, Good point. I managed to run the command with -C and double quotes: python solrjmeter.py -a -C "g1,cms" -c hour -x ./jmx/SolrQueryTest.jmx As a result got several files (html, css, js, csv) in the running directory (any way to specify where the output should be stored in this case?) W

Re: Measuring SOLR performance

2013-08-07 Thread Roman Chyla
Hi Dmitry, The command seems good. Are you sure your shell is not doing something funny with the params? You could try: python solrjmeter.py -C "g1,foo" -c hour -x ./jmx/SolrQueryTest.jmx -a where g1 and foo are results of the individual runs, ie. something that was started and saved with '-R g1'

Re: Measuring SOLR performance

2013-08-07 Thread Dmitry Kan
Hi Roman, One more question. I tried to compare different runs (g1 vs cms) using the command below, but get an error. Should I attach some other param(s)? python solrjmeter.py -C g1,foo -c hour -x ./jmx/SolrQueryTest.jmx **ERROR** File "solrjmeter.py", line 1427, in main(sys.argv) File

Re: Measuring SOLR performance

2013-08-07 Thread Dmitry Kan
Hi Roman, Finally, this has worked! Thanks for quick support. The graphs look awesome. At least on the index sample :) It is quite easy to setup and run + possible to run directly on the shard server in background mode. my test run was: python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q ./qu

Re: Measuring SOLR performance

2013-08-06 Thread Roman Chyla
Hi Dmitry, I've modified the solrjmeter to retrieve data from under the core (the -t parameter) and the rest from the /solr/admin - I could test it only against 4.0, but it is there the same as 4.3 - it seems...so you can try the fresh checkout my test was: python solrjmeter.py -a -x ./jmx/SolrQu

Re: Measuring SOLR performance

2013-08-06 Thread Dmitry Kan
Hi, Thanks for the clarification, Shawn! So with this in mind, the following work: http://localhost:8983/solr/statements/admin/system?wt=json http://localhost:8983/solr/statements/admin/mbeans?wt=json not copying their output to save space. Roman: is this something that should be set via -t p

Re: Measuring SOLR performance

2013-08-06 Thread Shawn Heisey
On 8/6/2013 6:17 AM, Dmitry Kan wrote: > Of three URLs you asked for, only the 3rd one gave response: > The rest report 404. > > On Mon, Aug 5, 2013 at 8:38 PM, Roman Chyla wrote: > >> Hi Dmitry, >> So I think the admin pages are different on your version of solr, what do >> you see when you re

Re: Measuring SOLR performance

2013-08-06 Thread Dmitry Kan
Hi Roman, With fresh checkout, the reported admin_endpoint is: http://localhost:8983/solr/admin. This url redirects to http://localhost:8983/solr/#/ . I'm using solr 4.3.1. Is your tool supporting this version? Of three URLs you asked for, only the 3rd one gave response: {"responseHeader":{"stat

Re: Measuring SOLR performance

2013-08-05 Thread Roman Chyla
Hi Dmitry, So I think the admin pages are different on your version of solr, what do you see when you request... ? http://localhost:8983/solr/admin/system?wt=json http://localhost:8983/solr/admin/mbeans?wt=json http://localhost:8983/solr/admin/cores?wt=json If your core -t was '/solr/statements',

Re: Measuring SOLR performance

2013-08-05 Thread Dmitry Kan
Hi Roman, No problem. Still trying to launch the thing.. The query with the added -t parameter generated an error: 1. python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q ./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60 -R test -t /solr/statements [passed relative path

Re: Measuring SOLR performance

2013-08-02 Thread Roman Chyla
Hi Dmitry, Thanks, It was a toothing problem, fixed now, please try the fresh checkout AND add the following to your arguments: -t /solr/core1 that sets the path under which solr should be contacted, the handler is set in the jmeter configuration, so if you were using different query handlers tha

Re: Measuring SOLR performance

2013-08-02 Thread Dmitry Kan
Hi Roman, Sure: python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60 -R test This is vanilla install (git clone) except for one change that I had to do related to solr cores: > git diff d

Re: Measuring SOLR performance

2013-08-02 Thread Erick Erickson
8 PM, Roman Chyla wrote: > > > >> Hi, here is a short post describing the results of the yesterday run > with > >> added parameters as per Shawn's recommendation, have fun getting > confused > >> ;) > >> > >> > http://29min.wordpress.co

Re: Measuring SOLR performance

2013-08-01 Thread Roman Chyla
gt;> http://29min.wordpress.com/**2013/08/01/measuring-solr-**performance-ii/<http://29min.wordpress.com/2013/08/01/measuring-solr-performance-ii/> >> > > I am having a very difficult time with the graphs. I have no idea what > I'm looking at. The graphs are probably

Re: Measuring SOLR performance

2013-08-01 Thread Shawn Heisey
On 8/1/2013 2:08 PM, Roman Chyla wrote: Hi, here is a short post describing the results of the yesterday run with added parameters as per Shawn's recommendation, have fun getting confused ;) http://29min.wordpress.com/2013/08/01/measuring-solr-performance-ii/ I am having a very difficult

Re: Measuring SOLR performance

2013-08-01 Thread Roman Chyla
Hi, here is a short post describing the results of the yesterday run with added parameters as per Shawn's recommendation, have fun getting confused ;) http://29min.wordpress.com/2013/08/01/measuring-solr-performance-ii/ roman On Wed, Jul 31, 2013 at 12:32 PM, Roman Chyla wrote: > I&#

Re: Measuring SOLR performance

2013-08-01 Thread Roman Chyla
Hi Bernd, On Thu, Aug 1, 2013 at 4:07 AM, Bernd Fehling < bernd.fehl...@uni-bielefeld.de> wrote: > Yes, UseNuma is only for Parallel Scavenger garbage collector and only > for Solaris 9 and higher and Linux kernel 2.6.19 and glibc 2.6.1. > And it performs with 64-bit better than 32-bit. > So no

Re: Measuring SOLR performance

2013-08-01 Thread Roman Chyla
Dmitry, Can you post the entire invocation line? roman On Thu, Aug 1, 2013 at 7:46 AM, Dmitry Kan wrote: > Hi Roman, > > When I try to run with -q > /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries > > here what is reported: > Traceback (most recent call last): > File "solrjmete

Re: Measuring SOLR performance

2013-08-01 Thread Dmitry Kan
Hi Roman, When I try to run with -q /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries here what is reported: Traceback (most recent call last): File "solrjmeter.py", line 1390, in main(sys.argv) File "solrjmeter.py", line 1309, in main tests = find_tests(options) File

Re: Measuring SOLR performance

2013-08-01 Thread Bernd Fehling
Yes, UseNuma is only for Parallel Scavenger garbage collector and only for Solaris 9 and higher and Linux kernel 2.6.19 and glibc 2.6.1. And it performs with 64-bit better than 32-bit. So no effects for G1. With standard applications CMS is very slightly better than G1 but when it comes to huge he

Re: Measuring SOLR performance

2013-07-31 Thread Roman Chyla
gt; Load is obviously higher with G1GC. > > > -Original message- > > From:Roman Chyla > > Sent: Wednesday 31st July 2013 18:32 > > To: solr-user@lucene.apache.org > > Subject: Re: Measuring SOLR performance > > > > I'll try to run it with the new paramete

RE: Measuring SOLR performance

2013-07-31 Thread Markus Jelsma
1GC! Load is obviously higher with G1GC. -Original message- > From:Roman Chyla > Sent: Wednesday 31st July 2013 18:32 > To: solr-user@lucene.apache.org > Subject: Re: Measuring SOLR performance > > I'll try to run it with the new parameters and let you know how

Re: Measuring SOLR performance

2013-07-31 Thread Roman Chyla
I'll try to run it with the new parameters and let you know how it goes. I've rechecked details for the G1 (default) garbage collector run and I can confirm that 2 out of 3 runs were showing high max response times, in some cases even 10secs, but the customized G1 never - so definitely the paramete

Re: Measuring SOLR performance

2013-07-31 Thread Roman Chyla
Hi Dmitry, probably mistake in the readme, try calling it with -q /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries as for the base_url, i was testing it on solr4.0, where it tries contactin /solr/admin/system - is it different for 4.3? I guess I should make it configurable (it alread

Re: Measuring SOLR performance

2013-07-31 Thread Shawn Heisey
On 7/31/2013 12:24 AM, William Bell wrote: > But that link does not tell me which on you are using? > > You are listing like 4 versions on your site. > > Also, what did it fix? Pause times? > > > Any other words of wisdom ? I'm not sure whether that was directed at me or Roman, but here's my a

Re: Measuring SOLR performance

2013-07-31 Thread Dmitry Kan
Ok, got the error fixed by modifying the base solr ulr in solrjmeter.py (added core name after /solr part). Next error is: WARNING: no test name(s) supplied nor found in: ['/home/dmitry/projects/lab/solrjmeter/demo/queries/demo.queries'] It is a 'slow start with new tool' symptom I guess.. :) O

Re: Measuring SOLR performance

2013-07-31 Thread Dmitry Kan
Hi Roman, What version and config of SOLR does the tool expect? Tried to run, but got: **ERROR** File "solrjmeter.py", line 1390, in main(sys.argv) File "solrjmeter.py", line 1296, in main check_prerequisities(options) File "solrjmeter.py", line 351, in check_prerequisities e

Re: Measuring SOLR performance

2013-07-30 Thread William Bell
But that link does not tell me which on you are using? You are listing like 4 versions on your site. Also, what did it fix? Pause times? Any other words of wisdom ? On Tue, Jul 30, 2013 at 9:01 PM, Shawn Heisey wrote: > On 7/30/2013 6:59 PM, Roman Chyla wrote: > > I have been wanting some

Re: Measuring SOLR performance

2013-07-30 Thread Shawn Heisey
On 7/30/2013 6:59 PM, Roman Chyla wrote: > I have been wanting some tools for measuring performance of SOLR, similar > to Mike McCandles' lucene benchmark. > > so yet another monitor was born, is described here: > http://29min.wordpress.com/2013/07/31/measuring-solr-query-performance/ > > I teste

Measuring SOLR performance

2013-07-30 Thread Roman Chyla
Hello, I have been wanting some tools for measuring performance of SOLR, similar to Mike McCandles' lucene benchmark. so yet another monitor was born, is described here: http://29min.wordpress.com/2013/07/31/measuring-solr-query-performance/ I tested it on the problem of garbage collectors (see