Hi, Testing quality requires "right answers" (query relevance judgments), which is expensive to create. Once you have qrels, you can evaluate effectiveness of your system with metrics (MAP, ERR@20, NDCG@20, etc)
Here is a presentation you might find relevant. http://opensourceconnections.com/blog/2014/07/24/using-quepid-to-improve-relevancy-of-advance-auto-intranet-search/ Ahmet On Wednesday, September 30, 2015 4:58 PM, marotosg <marot...@gmail.com> wrote: Hi, I have some doubts about how to define a process to evaluate the quality of search results. I have a solr collection with 4M documents with information about people. I search across several fields like first name ,second name, email, address, phone etc. There is plenty of logic in the query. Some fields ranks higher, exact match ranks higher than trail search etc. I was thinking on an approach where I can create some automated tests. Based on some searches if the results are good enough or the ones which come first are actually better than the other ones. I would like to be able to identify how new functionality affects previous results. I thought on this creating two type of tests. a) Functional tests which will test if the functionality is correct. They will be based on a subset of records which are static. In this case, I know what order results should come back from the query. b) Based on full data. I would like to run queries and see if the results are good enough. That's the part I am not sure if makes sense or how to do it. I am not sure if that's not correct or if there is a any standad to follow. Any help would be much appreciate. -- View this message in context: http://lucene.472066.n3.nabble.com/Regression-tests-and-evaluate-quality-of-results-tp4232047.html Sent from the Solr - User mailing list archive at Nabble.com.
