What do you want to achieve with these tests? Is it meant as a regression, to make sure that only the queries/boosts you changed are affected? Then you will have to implement tests that cover your specific schema/boosts. I'm not aware of any frameworks that do this - we're using Java based tests that retrieve documents from solr, map them to our domain model (objects representing a document) and do assertions on "debug values" (e.g. score)
Or is it more about "whats more relevant for the user?" Then you will need some kind of user tracking, as Markus described already. BR On 12 February 2013 23:16, Markus Jelsma <markus.jel...@openindex.io> wrote: > Roman, > > Logging clicks and their position in the result list is one useful method > to measure the relevance. Using the position you can calculate the mean > reciprocal rank, a value near 1.0 is very good so over time you can clearly > see whether changes actually improve user experience/expectations. Keep in > mind that there is some noise because users tend to click one or more of > the first few results anyway. > > You may also be interested in A/B testing. > > http://en.wikipedia.org/wiki/Mean_reciprocal_rank > http://en.wikipedia.org/wiki/A/B_testing > > Cheers > Markus > > > -----Original message----- > > From:Roman Chyla <roman.ch...@gmail.com> > > Sent: Tue 12-Feb-2013 23:04 > > To: solr-user@lucene.apache.org > > Subject: what do you use for testing relevance? > > > > Hi, > > I do realize this is a very broad question, but still I need to ask it. > > Suppose you make a change into the scoring formula. How do you > > test/know/see what impact it had? Any framework out there? > > > > It seems like people are writing their own tools to measure relevancy. > > > > Thanks for any pointers, > > > > roman > > >
