gt;>>>
>>>> a question for better understanding scoring of a shard in a cloud.
>>>>
>>>> I see different scores from different replicas of the same shard.
>>>> Is this normal and if yes, why?
>>>>
>>>> My understanding until now was that replicas are always the same within
>> a
>>>> shard
>>>> and the same query to each replica within a shard gives always the same
>>>> score.
>>>>
>>>> Can someone help me to understand this?
>>>>
>>>> Regards
>>>> Bernd
>>>>
>>>
>>
7;s a tradeoff between consistency and performance.
Solr's choice is to be eventually consistent, this "lag" is the price to
pay for scalability and fault tolerance.
TLOG are not as realtime as NRT (obviously) so it could be a problem too,
the replicas lag behind the leader (for score stats
tanding scoring of a shard in a cloud.
> >>
> >> I see different scores from different replicas of the same shard.
> >> Is this normal and if yes, why?
> >>
> >> My understanding until now was that replicas are always the same within
> a
> >> shard
> >> and the same query to each replica within a shard gives always the same
> >> score.
> >>
> >> Can someone help me to understand this?
> >>
> >> Regards
> >> Bernd
> >>
> >
>
me within a
shard
and the same query to each replica within a shard gives always the same
score.
Can someone help me to understand this?
Regards
Bernd
lica within a shard gives always the same
> score.
>
> Can someone help me to understand this?
>
> Regards
> Bernd
>
within a shard gives always the same score.
Can someone help me to understand this?
Regards
Bernd
cludes a filterQuery, `QueryUtils.combineQueryAndFilter` will
build A new BooleanQuery and copy it to the Query object。
so `query instanceof RankQuery` is false, This causes the score to be lost in
the RankQuery phase.
Can you change this to determine if the original query is RankQuery:
`cmd.g
This was my mistake.
Thank you.
Taisuke
2020年10月23日(金) 15:02 Taisuke Miyazaki :
> Thanks.
>
> I analyzed it as explain=true and this is what I found.
> Why does this behave this way?
>
> fq=foo:1
> bq=foo:(1)^1
> bf=sum(200)
>
> If you do this, the score will b
Thanks.
I analyzed it as explain=true and this is what I found.
Why does this behave this way?
fq=foo:1
bq=foo:(1)^1
bf=sum(200)
If you do this, the score will be boosted by bq.
However, if you remove fq, the score will not be boosted by bq.
However, if you change the boost value of bq to 2
You’d get a much better idea of what goes on
if you added &explain=true and analyzed the
output. That’d show you exactly what is
calculated when.
Best,
Erick
> On Oct 22, 2020, at 4:05 AM, Taisuke Miyazaki
> wrote:
>
> Hi,
>
> If you use a high value for the score, the
Hi,
If you use a high value for the score, the values on the smaller scale are
ignored.
Example :
bq = foo:(1.0)^1.0
bf = sum(200)
When I do this, the additional score for "foo" at 1.0 does not affect the
sort order.
I'm assuming this is an issue with the precision of the
wrote:
> Hi,
>
> I'm working on a Solr core where we don't want to use TF-IDF (BM25).
> We rank documents with boost based on popularity, exact match, phrase
> match,
> etc.
>
> To bypass TF-IDF, we use constant score like this "q=harry^=0.5
> potter^=0.
Hi,
I'm working on a Solr core where we don't want to use TF-IDF (BM25).
We rank documents with boost based on popularity, exact match, phrase match,
etc.
To bypass TF-IDF, we use constant score like this "q=harry^=0.5 potter^=0.5"
(score is always 1 before boost)
We have ju
Hi
Is it possible to normalize the per field score before applying the boosts?
let say 2 documents match my search criteria on the query fields *title* and
*description* using Dismax Parser with individual boosts.
q=cookie&qf = text^2 description^1
let's say below are the TF-IDF score
Hi,
I want to return more fields in moreLikeThis response, how should I reach it?
Currently the main doc returns all fields, but morelikethis result only has I’d
and score, please help
Thanks
GetDocListNC method in the SolrIndexSearcher class, Query is converted to
BooleanQuery when fq queries are available.
Has a call query instanceof RankQuery below, which will never be true.
The score of rerank is not correct.
Do you need to add a loop to the query type?
ProcessedFilter pf
I check the frange on another field and not on query($bq) for some reason on
schema field it's filter in the right way and I get all the relevant values,
But when I do the filter on the return score from query($bq) the upper and the
lower bound behave different and in addition I didn'
filter range on score
I don't know if this inclusive works though I know that incl is for including
the lower bound and incu is for including the upper bound.
On Mon, 9 Dec 2019 at 16:49, Raboah, Avi wrote:
> Thanks for your fast response!
>
> Without the frange I get all the docu
I don't know if this inclusive works though I know that incl is for
including the lower bound and incu is for including the upper bound.
On Mon, 9 Dec 2019 at 16:49, Raboah, Avi wrote:
> Thanks for your fast response!
>
> Without the frange I get all the documents with the score
Thanks for your fast response!
Without the frange I get all the documents with the score field from 1.0
(default score) to max score after boosting.
When I add the frange for example
bq=text:"Phrase"^3&defType=edismax&fl=*,score&fq={!frange l=0 u=3
inclusive=true}query($
I was just going to suggest you frange. You're already using it.
Please post the whole query. Have you confirmed that by removing the
frange, you are able to see the documents with score=1.0.
On Mon, 9 Dec 2019 at 14:21, Raboah, Avi wrote:
> That's right,
>
> I check som
That's right,
I check something like this fq={!frange l=0 u=5}query($bq)
And it's partially work but it's not return the documents with score = 1.0
Do you know why?
Thanks.
-Original Message-
From: Paras Lehana [mailto:paras.leh...@indiamart.com]
Sent: Monday, Decembe
Hi Raboah,
What do you mean by filter range? Please post expected result. Do you want
to put an fq on the score?
On Sun, 8 Dec 2019 at 17:54, Raboah, Avi wrote:
> Hi,
>
> In order to use solr boost mechanism for specific text I use the bq field
> under deftype=edisMax.
>
>
Hi,
In order to use solr boost mechanism for specific text I use the bq field under
deftype=edisMax.
For example -
q=*:*&deftype=edisMax&bq=text:"phrase"^3&fl=*,score
after I do this query I get the relevant documents boosted with the solr
calculation score.
Now my ques
, "QTime":6, "params":{ "hl":"on", "pt":
> > "54.6973867999,25.22481530046", "fl":"score,*,store:[subquery
> > fromIndex=stores]", "store.rows":"1", "fq":&qu
us":400, "QTime":6, "params":{ "hl":"on", "pt":
> "54.6973867999,25.22481530046", "fl":"score,*,store:[subquery
> fromIndex=stores]", "store.rows":"1", "fq":"{!edismax qf=\"title
&
I tried and this way also, but here is another error.
{ "responseHeader":{ "status":400, "QTime":6, "params":{ "hl":"on", "pt":
"54.6973867999,25.22481530046", "fl":"score,*,store:[subquery
fr
Space matters. Check my snippet once again please.
On Thu, Nov 14, 2019 at 9:56 PM Vasily Ogar wrote:
> I tried today with plus but always got same error.
> { "responseHeader":{ "status":400, "QTime":2, "params":{ "hl":"on", &
I tried today with plus but always got same error.
{ "responseHeader":{ "status":400, "QTime":2, "params":{ "hl":"on", "pt":
"54.6973867999,25.22481530046", "fl":"score,*,store:[subquery
fromInd
It should be like
"q":"{!join from=site_id to=site_id fromIndex=stores
score=max}+{!geofilt} {!func}geodist() ",
post debugQuery
On Thu, Nov 14, 2019 at 4:44 PM Vasily Ogar wrote:
> I was glad too early because it can only sort or only filter, but can't do
> toget
!join from=site_id to=site_id fromIndex=stores}*:*",
> "fq":"{!edismax qf=\"title description\" v=\"iphone xr 64gb\" mm=3<90%}",
> "fl":"score,*,store:[subquery fromIndex=stores]",
> "store.q":"{!terms f=
Hello,
I fixed it. If I need to sort by price:
"q":"{!join from=site_id to=site_id fromIndex=stores}*:*",
"fq":"{!edismax qf=\"title description\" v=\"iphone xr 64gb\" mm=3<90%}", "
fl":"score,*,store:[subquery from
Ogar wrote:
>
> > First of all, thank you for your help.
> > Now it doesn't show any errors, but somehow score is based on the title
> and
> > description but not on the geodist.
> > "params":{ "hl":"on", "pt":"5
tlrd;
I noticed func under fq that make no sense. Only q or sort yield scores.
On Tue, Nov 12, 2019 at 6:43 PM Vasily Ogar wrote:
> First of all, thank you for your help.
> Now it doesn't show any errors, but somehow score is based on the title and
> description but not
First of all, thank you for your help.
Now it doesn't show any errors, but somehow score is based on the title and
description but not on the geodist.
"params":{ "hl":"on", "pt":"54.6973867999,25.22481530046",
"fl":"scor
Hello,
It seems like I breached the limit on unconscious replies in mailing list
I'd rather start with this:
q={!join from=site_id to=site_id fromIndex=stores
score=max}+{!geofilt}
{!func}geodist()&sfield=coordinates&pt=54.6973867999,25.22481530046&d=10
On Mon, Nov 11,
pache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"], "msg":"A
> ValueSource isn't directly available from this field. Instead try a query
> using the distance as the score.",
>
> On Mon, Nov 11, 2
t enough parameters:[]",
If I set parameters then I got another error
"metadata":[ "error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.common.SolrException"], "msg":"A
ValueSource isn't di
Hello, Vasily.
Why not? What have you got in debugQuery=true?
On Mon, Nov 11, 2019 at 1:19 PM Vasily Ogar wrote:
> Hello,
> Is it possible to sort by score in join by geodist()? For instance,
> something like this
> q={!join from=site_id to=site_id fromIndex=stores score=max}
>
Hello,
Is it possible to sort by score in join by geodist()? For instance,
something like this
q={!join from=site_id to=site_id fromIndex=stores score=max}
+{!func}gedist() +{!geofilt sfield=coordinates
pt=54.6973867999,25.22481530046 d=10}
sort=score desc
Thank you
un 3 in a separate thread to 4 & 5, to ensure Solr
communication need not block fetching entity data / writing. We could do
more optimisation around these tasks, but I'm sure you've already
understood.
Many thanks for your input.
Best,
Edd
On Thu, 3 Oct 2019 at 19:13, Chris Hostetter
: We show a table of search results ordered by score (relevancy) that was
: obtained from sending a query to the standard /select handler. We're
: working in the life-sciences domain and it is common for our result sets to
: contain many millions of results (unfortunately). After users b
docs you’re exporting into a
> huge list, which may perform worse than CursorMark even if it doesn’t blow
> up memory.
> >
> > The root of this problem is that export can sort in batches since the
> values it’s sorting on are contained in each document, so it can iterate in
>
rk even if it doesn’t blow up
>> memory.
>>
>> The root of this problem is that export can sort in batches since the
>> values it’s sorting on are contained in each document, so it can iterate in
>> batches, send them out, then iterate again on the remaining docume
ry.
>
> The root of this problem is that export can sort in batches since the
> values it’s sorting on are contained in each document, so it can iterate in
> batches, send them out, then iterate again on the remaining documents.
>
> Score, since it’s dynamic, can’t do that. Solr has to sco
ort in batches since the values
> it’s sorting on are contained in each document, so it can iterate in batches,
> send them out, then iterate again on the remaining documents.
>
> Score, since it’s dynamic, can’t do that. Solr has to score _all_ the docs to
> know where a doc lands
documents.
Score, since it’s dynamic, can’t do that. Solr has to score _all_ the docs to
know where a doc lands in the final set relative to any other doc, so if it
were going to work it’d have to have enough memory to hold the scores of all
the docs in an ordered list, which is very expensive
Hi all,
As far as I understand, SolrCloud currently does not allow the use of
sorting by the pseudofield, score in the /export request handler (i.e., get
the results in relevancy order). If we do attempt this, we get an
exception, "org.apache.solr.search.SyntaxError: Scoring is not curr
Hello Experts,
I have a following query
product:TV or os:Android or size:(55 60 65) or brand:samsung or issmart:yes
or ram:[4 TO *] or rate:[10 TO *] or bezel : no or sound:dolby
In Total there are 9 conditions.
Now I need the document with best match should return top. Best match I mean
w
Found the issue.I am using solr 5.4.0 version. in ExpandComponent - the
TermFieldCollector is created with the TotalDefaultScores and
TotalDefaultMaxScores were passed as false. This limits any field sorting
(other than score) makes the score to be returned as NaN.
By passing either or both of
All,
I am using the collapse plugin with the following query params
text_field:( apple OR apples) ) )
Fq : {!collapse field=string_field sort='numeric_field asc , score desc '
nullPolicy=expand}
Raw params : expand=true&expand.rows=600&expand.sort=numeric_field asc
Dear reader, I've found an different solution for my problem
and don't need a depth dependent score anymore.
Kind regards, Jochen
Am 19.02.19 um 14:42 schrieb Jochen Barth:
Dear reader,
I'll have a hierarchical graph "like a book":
{ id:solr_doc1; title:book }
{ id
I could do:
+_query_:"{!graph from=parent_ids to=id}title:book"
+_query_:"{!graph from=parent_ids to=id}title:chapter",
The result would be solr_doc2 and solr_doc3;
but is there a way to "boost" or "put a higher score" on solr_doc2 than
on solr_doc3 b
Hi Baloo,
> Is there and solution/workaround available for this issue?
> or going back to Solr 7.2.1 is the only solution - As per comments in above
> issue (https://issues.apache.org/jira/browse/LUCENE-8099 these changes are
> not there in 7.2.1)
The only workaround, I know of, is building SOLR
Hi Erick,
> You are saying that "X" doesn't work, in this case the scores are different
> in the debug section. But this implies that there is a problem "Y" that
> you're having.
The issue is not that they don't match, the issue is that the calcu
have Solr cloud architecture(3
> shards
> > > and 4 replicas) and the whole size of index is of around 2.5 GB.
> > >
> > > We use this collection to fetch the most clicked products against a
> query
> > > and boost in search results. Boost score is the query score
Thanks Erick,
We will stick to Solr 7.2.1 which works fine with multiple boost queries.
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
queries did not match
> >> In 3rd document boost query matched and total score of 3rd document
> >> became 400.04 which is higher than 1st and 2nd document.
>
> So I am wondering why Solr ignored score which is coming from boost query
> section?
>
>
g
> > total of around 8 million records. We have Solr cloud architecture(3 shards
> > and 4 replicas) and the whole size of index is of around 2.5 GB.
> >
> > We use this collection to fetch the most clicked products against a query
> > and boost in search results. B
8 million records. We have Solr cloud architecture(3 shards
> and 4 replicas) and the whole size of index is of around 2.5 GB.
>
> We use this collection to fetch the most clicked products against a query
> and boost in search results. Boost score is the query score on aggregation
> c
I will try to provide more data about this issue,
If you see attached query response , It shows
>> In 1st, 2nd document only query matched and boost queries did not match
>> In 3rd document boost query matched and total score of 3rd document
>> became 400.04 which is highe
Thanks Erick to answer your question "What is "Y"?"
Score that we see in debug section actually looks correct and if we order
documents by that score we can get similar ranking of results that we were
getting for solr 6.4.2.
But With the score field that we get with each
Thanks.I Agree.
Regards
Ashish
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
grating Solr 6.4.2 to Solr 7.4. We pass multiple boost
> queries (multiplicative boost queries, Solr's 'boost' parameter) to Solr
> with each query. We have migrated all our custom components and solr
> configurations to Solr 7.4.2 but during verification we have seen differ
on we have seen different
score in debug section which is not matching with the score that we see when
we return score field using fl parameter.
After spending half of my day analysing issue I found similar issue is
already open. Below issue is exactly same as what I am also observing with
Sol
GB.
We use this collection to fetch the most clicked products against a query
and boost in search results. Boost score is the query score on aggregation
collection.
But when the query goes to different replica we get different boost score
for some of the keywords, hence on page refresh results
Optimization is safe. The large segment is irrelevant, you'll
lose a little parallelization, but on an index with this few
documents I doubt you'll notice.
As of Solr 5, optimize will respect the max segment size
which defaults to 5G, but you're well under that limit.
Best,
Erick
On Sun, Feb 3,
ecause the
term _frequencies_ would still include counts from deleted docs.
99% of the time in my experience, worrying excessively about
this kind of detail about score calculations is wasted effort,
but it's hard to recommend one way or the other without knowing
what "Y" is above.
Best,
Hi,
Currently score is calculated based on "Max Doc" instead of "Num Docs".Is
it possible to change it to "Num Docs"(i.e without deleted docs).Will it
require a code change or some config change.
Regards
Ashish
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Thanks Erick and everyone.We are checking on stats cache.
I noticed stats skew again and optimized the index to correct the same.As
per the documents.
https://lucidworks.com/2017/10/13/segment-merging-deleted-documents-optimize-may-bad/
and
https://lucidworks.com/2018/06/20/solr-and-optimizing-y
...@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Jan 29, 2019, at 10:38 AM, David Hastings
> wrote:
>
> Maybe instead of using the solr score in your metrics, find a way to use
> the documents location in the results? you can never trust the score to
>
Maybe instead of using the solr score in your metrics, find a way to use
the documents location in the results? you can never trust the score to
be consistent, its constantly changing as the indexes changes
On Tue, Jan 29, 2019 at 1:29 PM Ashish Bisht
wrote:
> Hi Erick,
>
> Our
Hi Erick,
Our business wanted score not to be totally based on default relevancy algo.
Instead a mix of solr relevancy+usermetrics(80%+20%).
Each result doc is calculated against max score as a fraction of
80.Remaining 20 is from user metrics.
Finally sort happens on new score.
But say we
No, this is not a bug but a consequence of the design. ExactStats can help,
but there is no guarantee that different replicas will compute the exact same
score. Scores should be very close however.
You haven't explained why you need the scores to match. 99% of the time,
worrying about scor
l the docs
present on one replica will be present and searchable on
the other. However, they will be in different segments so the
"stats skew" will remain. *
We need these score,so as a temporary solution if we monitor these metrics
for any issues and take action (either optimiz
On 04.01.19, 09:11, "Thomas Aglassinger" wrote:
> When debugging a query using multiplicative boost based on the product()
> function I noticed that the score computed in the explain section is correct
> while the score in the actual result is wrong.
We digged into th
or different
> replicas for the same query with most replication schemes. This is mildly
> annoying when the score is displayed to the user, although I have found
> most end users do not pay that much attention to the floating point score.
> Testers do. On a small index with high wr
ildly
annoying when the score is displayed to the user, although I have found
most end users do not pay that much attention to the floating point score.
Testers do. On a small index with high write/delete traffic and homogenous
docs, I've seen it cause document re-orderings when the same query i
Hi Erick,
Your statement "*At best, I've seen UIs where they display, say, 1 to 5
stars that are just showing the percentile that the particular doc had
_relative to the max score*" is something we are trying to achieve,but we
are dealing in percentages rather stars(ratings)
Cha
ll remain.
But displaying the scores isn't a good reason to worry about
this. Frankly, that's almost always a mistake. Scores are
meaningless outside of ranking the docs _in a single
query_. Because a doc in one query got a score of 10 but
some other doc in some other query scored 5 do
f and no manual activity required.
For us score matters as we are using it to display some scenarios on search
and it gave changing values.As of now we are dependent of single
shard-replica but in future we might need more replicas
Will planning indexing and updates outside peak query hour help?
I
0:25 PM Ashish Bisht wrote:
>
> Hi Erick,
>
> Thank you for the details,but doesn't look like a time difference in
> autocommit caused this issue.As I said if I do retrieve all query/keyword
> query on both server,they returned correct number of docs,its just relevancy
> score
Hi Erick,
Thank you for the details,but doesn't look like a time difference in
autocommit caused this issue.As I said if I do retrieve all query/keyword
query on both server,they returned correct number of docs,its just relevancy
score is taking diff values.
I waited for brief period,
9 at 11:12 AM Ashish Bisht wrote:
>
> Hi Erick,
>
> I have updated that I am not facing this problem in a new collection.
>
> As per 3) I can try deleting a replica and adding it again, but the
> confusion is which one out of two should I delete.(wondering which replica
>
Hi Erick,
I have updated that I am not facing this problem in a new collection.
As per 3) I can try deleting a replica and adding it again, but the
confusion is which one out of two should I delete.(wondering which replica
is giving correct score for query)
Both replicas give same number of
:
> Version Solr 7.4.0 zookeeper 3.4.11 Achitecture Two boxes
> Machine-1,Machine-2
> holding single instances of solr
>
> We are having a collection which was single shard and single replica i.e
> s=1
> and rf=1
>
> Few days back we tried to add replica to it.But the score
=score desc, solrId asc to the query.
That article was written before TLOG and PULL replicas came into the
picture. Since those replica types all have the
exact same index structure you shouldn't have this problem in that case.
Best,
Erick
On Fri, Jan 4, 2019 at 3:40 AM AshB wrote:
>
> V
Version Solr 7.4.0 zookeeper 3.4.11 Achitecture Two boxes Machine-1,Machine-2
holding single instances of solr
We are having a collection which was single shard and single replica i.e s=1
and rf=1
Few days back we tried to add replica to it.But the score for same query is
coming different from
Hi!
When debugging a query using multiplicative boost based on the product()
function I noticed that the score computed in the explain section is correct
while the score in the actual result is wrong.
As an example here’s a simple query that boosts a field name_text_de
(containing German
e, Email etc). I have an indexed all the fields and the
> > combined copy field as well.
> > In the query that i have which is a combination of all the fields as
> input
> > (Title + Description+Phone+email).
> > There are some samples where if the Email/Phone has a matc
t of this in Solr relevance training
>> https://opensourceconnections.com/events/training/)
>>
>> Hope that helps
>> -Doug
>>
>>
>> On Wed, Nov 28, 2018 at 3:21 PM Tanya Bompi
>> wrote:
>>
>> > Hi,
>> > I have an index that is
> In the query that i have which is a combination of all the fields as input
> (Title + Description+Phone+email).
> There are some samples where if the Email/Phone has a match the resulting
> Solr score is lower still. I have tried boosting the fields say Email^2 but
> that results in
some samples where if the Email/Phone has a match the resulting
Solr score is lower still. I have tried boosting the fields say Email^2 but
that results in any token in the input query being matched against the
email which results in erroneous results.
How can i formulate a query that I can boost for
Use a bf of 10 * RANK. That will give the same ordering as dividing the score
by 10 and adding RANK.
There are problems with additive boosts, so I strongly recommend looking at the
“boost” parameter, which is a multiplicative boost. That is more stable over a
wide range of score values
>
> On Thu, Oct 25, 2018 at 11:58 AM Amjad Khan wrote:
>
>> Hi
>>
>> Is there a way to achieve the following -
>>
>> We have a RANK field in each document, and essentially, I would like my
>> score to be influenced by this RANK as follows -
&g
h document, and essentially, I would like my
> score to be influenced by this RANK as follows -
>
> score = score*0.1 + RANK
>
> How can I achieve this with function queries?
>
> Thanks!
Hi
Is there a way to achieve the following -
We have a RANK field in each document, and essentially, I would like my score
to be influenced by this RANK as follows -
score = score*0.1 + RANK
How can I achieve this with function queries?
Thanks!
On 10/17/2018 5:06 PM, Vincenzo D'Amore wrote:
I tried to use constant score into qf parameter but I had an exception.
Is this normal? The qf parameter actually is something like this:
field1^3 field2^4 field3^5... etc.
You didn't actually say, but it sounds like you're trying
Hi all,
I tried to use constant score into qf parameter but I had an exception.
Is this normal? The qf parameter actually is something like this:
field1^3 field2^4 field3^5... etc.
Because it's a query for an ecommerce website I don't want that the results
are influenced by tf/idf.
Hi there!
I have the following use case I'd like to implement in Solr and hence are
seeking for information onto the internals of Solr and the performance of
such an approach.
I would like to score documents matched by a query using a custom scoring
function. Generally, up to my knowledge,
1 - 100 of 1115 matches
Mail list logo
