Hi Roman, I read with interest your thread about relevance testing a couple of weeks ago and yes, I noticed it was related somehow. But what you were proposing there is a different approach I think.
In my tool, you have some baseline setting (it might be good or bad), and using a single query, you can visually see how documents rank differently when changing parameters. But the way to see the difference is the user's eye, and I am not using any statistical measurement to compare both settings. So it is a bit limited. In your approach I understand that you have some data that allows you to measure how well queries do (like clicks etc). So I think your approach is more useful, but probably harder to achieve. Not sure how I could merge that into my tool. In order to test, myself, I am working on some Geb tests ( http://www.gebish.org/testing) to check things like Steffen proposes on your thread. regards xavier On Mon, Feb 25, 2013 at 8:23 PM, Roman Chyla <roman.ch...@gmail.com> wrote: > Oh, wonderful! Thank you :) I was hacking some simple python/R scripts that > can do a similar job for qf... the idea was to let the algorithm create > possible combinations of params and compare that against the baseline. > > Would it be possible/easy to instruct the tool to harvest results for > different combinations and export it? I would like to make plots similar to > those: > > > https://github.com/romanchyla/r-ranking-fun/blob/master/plots/raw/test-plot-showing-factors.pdf?raw=true > > roman > > On Sat, Feb 23, 2013 at 9:12 AM, jmlucjav <jmluc...@gmail.com> wrote: > > > Hi, > > > > I have built a small tool to help me tweak some params in Solr (typically > > qf, bf in edismax). As maybe others find it useful, I am open sourcing it > > on github: https://github.com/jmlucjav/vifun > > > > Check github for some more info and screenshots. I include part of the > > github page below. > > regards > > > > Description > > > > Did you ever spend lots of time trying to tweak all numbers in a > *edismax* > > handler *qf*, *bf*, etc params so docs get scored to your liking? > Imagine > > you have the params below, is 20 the right boosting for *name* or is it > too > > much? Is *population* being boosted too much versus distance? What about > > new documents? > > > > <!-- fields, boost some --> > > <str name="qf">name^20 textsuggest^10 edge^5 ngram^2 > > phonetic^1</str> > > <str name="mm">33%</str> > > <!-- boost closest hits --> > > <str name="bf">recip(geodist(),1,500,0)</str> > > <!-- boost by population --> > > <str name="bf">product(log(sum(population,1)),100)</str> > > <!-- boost newest docs --> > > <str name="bf">recip(rord(moddate),1,1000,1000)</str> > > > > This tool was developed in order to help me tweak the values of boosting > > functions etc in Solr, typically when using edismax handler. If you are > fed > > up of: change a number a bit, restart Solr, run the same query to see how > > documents are scored now...then this tool is for you. > > <https://github.com/jmlucjav/vifun#features>Features > > > > - Can tweak numeric values in the following params: *qf, pf, bf, bq, > > boost, mm* (others can be easily added) even in *<appends> or > > <invariants>* > > - View side by side a Baseline query result and how it changes when > you > > gradually change each value in the params > > - Colorized values, color depends on how the document does related to > > baseline query > > - Tooltips give you Explain info > > - Works on remote Solr installations > > - Tested with Solr 3.6, 4.0 and 4.1 (other versions would work too, as > > long as wt=javabin format is compatible) > > - Developed using Groovy/Griffon > > > > <https://github.com/jmlucjav/vifun#requirements>Requirements > > > > - */select* handler should be available, and not have any *<appends> > or > > <invariants>*, as it could interfere with how vifun works. > > - Java6 is needed (maybe it runs on Java5 too). A JRE should be > enough. > > > > <https://github.com/jmlucjav/vifun#getting-started>Getting started > > < > > > https://github.com/jmlucjav/vifun#click-here-to-download-latest-version-and-unzip > > >Click > > here to download latest > > version< > http://code.google.com/p/vifun/downloads/detail?name=vifun-0.4.zip > > > > > and > > unzip > > > > - Run vifun-0.4\bin\vifun.bat or vifun-04\bin\vifun if on linux/OSX > > - Edit *Solr URL* to match yours (in Sol4.1 default is > > http://localhost:8983/solr/collection1 for example) [image: hander > > selection]< > > https://github.com/jmlucjav/vifun/raw/master/img/screenshot-handlers.jpg > > > > - *Show Handerls*, and select the handler you wish to tweak from * > > Handerls* dropdown. The text area below shows the parameters of the > > handler. > > - Modify the values to run a baseline query: > > - *q*: query string you want to use > > - *rows*: as in Solr, don't choose a number too small, so you can > see > > more documents, I typically use 500 > > - *fl*: comma separated list of fields you want to show for each > doc, > > keep it short (other fields needed will be added, like the id, > score) > > - *rest*: in case you need to add more params, for example: sfield, > > fq etc) [image: query > > params]< > > https://github.com/jmlucjav/vifun/raw/master/img/screenshot-qparams.jpg> > > - *Run Query*. The two panels on the right will show the same result, > > sorted by score.[image: > > results]< > > https://github.com/jmlucjav/vifun/raw/master/img/screenshot-results.jpg> > > - Use the mouse to select the number you want to tweak in Score params > > (select all the digits). Note the label of the field is highlighted > with > > current value. [image: target > > selection]< > > > https://github.com/jmlucjav/vifun/raw/master/img/screenshot-selecttarget.jpg > > > > > - Move the slider, release and see how a new query is run, and you can > > compare how result changes with the current value. In the Current > table, > > you can see current position/score and also delta relative to the > > baseline. > > The colour of the row reflects how much the doc gained/lost. [image: > > tweaking a value]< > > https://github.com/jmlucjav/vifun/raw/master/img/screenshot-baseline.jpg > > > > - You can increase the limits of the slider, and if you are satisfied > > with a value, set it, so it will be set to current value. > > - Tweak another number... > > - Tooltips exist in Current (Explain info) and Baseline (pos/score of > > that doc in Current, and Explain in Score column) > > >
