Re: Relevancy and random sorting

2012-01-12 Thread Chris Hostetter
: We have a listing aggregator that gets product listings from a lot of : different sites and since they are added in batches, sometimes you see a : lot of pages from the same source (site). We are working on some changes to : shift things around and reduce this "blocking" effect, so we can presen

Re: Relevancy and random sorting

2012-01-12 Thread Ahmet Arslan
> This document already has a field that indicates the source > (site). > The issue we are trying to solve is when we list all > documents without any > specific criteria. Since we bring the most recent ones and > the ones that > contains images, we end up having a lot of listings from a > single s

Re: Relevancy and random sorting

2012-01-12 Thread Alexandre Rocco
Michael, We are using the random sorting in combination with date and other fields but I am trying to change this to affect the ranking instead of sorting directly. That way we can also use other useful tweaks on the rank itself. Alexandre On Thu, Jan 12, 2012 at 11:46 AM, Michael Kuhlmann wrot

Re: Relevancy and random sorting

2012-01-12 Thread Michael Kuhlmann
Does the random sort function help you here? http://lucene.apache.org/solr/api/org/apache/solr/schema/RandomSortField.html However, you will get some very old listings then, if it's okay for you. -Kuli Am 12.01.2012 14:38, schrieb Alexandre Rocco: Erick, This document already has a field tha

Re: Relevancy and random sorting

2012-01-12 Thread Alexandre Rocco
Erick, This document already has a field that indicates the source (site). The issue we are trying to solve is when we list all documents without any specific criteria. Since we bring the most recent ones and the ones that contains images, we end up having a lot of listings from a single site, sin

Re: Relevancy and random sorting

2012-01-11 Thread Ted Dunning
I think the OP meant to use random order in the case of score ties. On Wed, Jan 11, 2012 at 9:31 PM, Erick Erickson wrote: > Alexandre: > > Have you thought about grouping? If you can analyze the incoming > documents and include a field such that "similar" documents map > to the same value, than

Re: Relevancy and random sorting

2012-01-11 Thread Erick Erickson
Alexandre: Have you thought about grouping? If you can analyze the incoming documents and include a field such that "similar" documents map to the same value, than group on that value you'll get output that isn't dominated by repeated copies of the "similar" documents. It depends, though, on being

Re: Relevancy and random sorting

2012-01-11 Thread Alexandre Rocco
Erick, Probably I really written something silly. You are right on either sorting by field or ranking. I just need to change the ranking to shift things around as you said. To clarify the use case: We have a listing aggregator that gets product listings from a lot of different sites and since the

Re: Relevancy and random sorting

2012-01-11 Thread Erick Erickson
I really don't understand what this means: "random sorting for the records but also preserving the ranking" Either you're sorting on rank or you're not. If you mean you're trying to shift things around just a little bit, *mostly* respecting relevance then I guess you can do what you're thinking.