Roman, Logging clicks and their position in the result list is one useful method to measure the relevance. Using the position you can calculate the mean reciprocal rank, a value near 1.0 is very good so over time you can clearly see whether changes actually improve user experience/expectations. Keep in mind that there is some noise because users tend to click one or more of the first few results anyway.
You may also be interested in A/B testing. http://en.wikipedia.org/wiki/Mean_reciprocal_rank http://en.wikipedia.org/wiki/A/B_testing Cheers Markus -----Original message----- > From:Roman Chyla <roman.ch...@gmail.com> > Sent: Tue 12-Feb-2013 23:04 > To: solr-user@lucene.apache.org > Subject: what do you use for testing relevance? > > Hi, > I do realize this is a very broad question, but still I need to ask it. > Suppose you make a change into the scoring formula. How do you > test/know/see what impact it had? Any framework out there? > > It seems like people are writing their own tools to measure relevancy. > > Thanks for any pointers, > > roman >
