Re: How different is solr 4.7 from latest version.
Thanks Joe and Shawn.
This is very useful information.
@Joe, I need to upgrade and use synonym support of support. So I think I
should better start with latest version.
.Best,
Srini Sampath.
.Best,
Srini Sampath
On Fri, Jan 12, 2018 at 8:42 PM, Joe Heasly wrote:
> Srini,
>
> We upgraded from Solr 4.6 to 6.4 last summer. There are fundamental
> differences between those versions in the way the default Boolean operator
> and 'minimum should match' functions interact. Here's an excellent
> discussion of the change here (Jason Hellman does it more justice than I
> could hope to):
>
> http://blog.innoventsolutions.com/innovent-solutions-blog/
> 2017/02/solr-edismax-boolean-query.html
>
> If you're just starting out and you're starting with a recent version,
> this won't matter. But if you're upgrading, it's critical to be aware.
>
> Regards,
> Joe
>
> { Joe Heasly | L.L.Bean, Inc. | [O] 207 552-2254 [M] 207 756-9250 }
>
> -Original Message-
> From: Shawn Heisey [mailto:apa...@elyograg.org]
> Sent: Friday, January 12, 2018 10:02 AM
> To: solr-user@lucene.apache.org
> Subject: Re: How different is solr 4.7 from latest version.
>
> On 1/12/2018 5:58 AM, srini sampath wrote:
> > I am reading a book (Solr in action
> > > lr-2DAction-2DTrey-2DGrainger_dp_1617291021&d=DwICaQ&c=uC6H3HqR7J0hkle
> > XqZF0oA&r=LGfOV9gkzZFmyXgI5jYqo5FeO_fORxZZyF8winHfJ8s&m=xoJLS_ZP0u4l7P
> > AGZslEYaLCEBqnoJfoXeneaibCb-8&s=rPpw1EkXBvtuGGJXE7VSTHcgViWw_6X0Au7zZW
> > Af4iw&e=>) to understand how to work with different features in solr. It
> uses solr 4.7 to explain features. But I don't find any better material
> (IMHO, documentation has many looped references which makes it too
> difficult to understand for a newbie).
> >
> > Does it cover all the features related to new version (like important
> > features) or is it better to follow some other resource?
>
> The latest version is 7.2, and the 7.2.1 release is being finalized now.
>
> That's three major versions newer. Most of the info in that book will
> still be relevant, but there is quite a bit of new functionality.
>
> Here's the reference guide that is published as official documentation.
> You can download this as a PDF using the "Other Formats" link at the top
> of the page:
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lucene.
> apache.org_solr_guide_7-5F2_&d=DwICaQ&c=uC6H3HqR7J0hkleXqZF0oA&r=
> LGfOV9gkzZFmyXgI5jYqo5FeO_fORxZZyF8winHfJ8s&m=xoJLS_
> ZP0u4l7PAGZslEYaLCEBqnoJfoXeneaibCb-8&s=ISiRaa2CCBrg1Y3ElGGY8_
> I0fCvSGGKiJGjU8TcsAas&e=
>
> Full disclosure: There isn't very much available for extreme beginners.
> This lack is something the project is aware of, but writing documentation
> for the uninitiated is a difficult task. The reference guide isn't awful,
> but it could be a lot better.
>
> For differences between versions, there is the CHANGES.txt file included
> in every download. The reference guide also has a section about big
> differences from the previous major version.
>
> Thanks,
> Shawn
>
Re: LTR and working with feature stores
Hi Dariusz,
On Jan 12, 2018 14:40, "Dariusz Wojtas" wrote:
Hi,
I am working with the LTR rescoring.
Works beautifully, but I am curious about something.
How do I specify the feature store in a way different than using the
[features] syntax?
[features store=yourFeatureStore]
What is the problem with this syntax? If the problem is the name of the
field, you can also call it by doing fl=title,authors,myfield=[features
store=yourFeatureStore]
I can't think of alternative ways..
I have a range of models in my custom feature store, with plenty of
features implemented.
I have found that when I call LTR with model using only two features, Solr
still executes them all.
My setup in solrconfig.xml
-
id,score,why_score:[explain style=nl],[features
store=store_incidentDB]
{!ltr reRankDocs=$reRankDocs model=simpleModelA}
--
simpleModel above only uses LinearModel with 2 features.
What do I see in results?
In response I can see it has executed ALL features (there are values
calculated) in section:
1) -> response -> result -> doc -> HERE
In addition, there is my model executed and only TWO features of the
executed model are presented in:
It is intended, the reason is that usually you want to execute your model
and at the same time log a *superset* of the features to train the next
model. If you want to compute only the features of the model you can define
a featureStore that matches exactly the features that you have in the model.
2) -> response -> debug -> explain
Why do I see all features being executed, if the specified model only
contains two features?
I tried to reduce 'fl' to:
id,score,why_score:[explain style=nl]
and id works as expected then:
1. additional features are not executed (correct)
2. my model works, only two features of the selected model (correct)
And the final questions for this long email are:
1. why does it execute all features when i specify 'store'?
2. how do I specify the 'store', if I have more stores, but do not want to
execute all their features?
Just define a feature store that matches the features that you have in the
model. Please note that the featureStore that you specify in fl= [features]
field **will not** affect the reranking (the model will compute only the
features that are specified in the model json file), you should ask for the
[features] only if you want to log them.
Please do not hesitate to ask if something is not clear ;)
Cheers,
Diego
Best regards,
Dariusz Wojtas
Re: LTR and working with feature stores
Hi,
Thanks for the response, I understand that all features from the given
store are calculated, no matter if they are used or not.
OK, spread features across different models.
But what if different models share some features?
Creating copies of feature definitions in different stores, one per model,
is erroneous ...
Having several models in one store, some of them use only part of these
features - that seems 'expensive' ;)
Simple syntax evolution would be very helpful, to give {!ltr} optional
'store' parameter. It could override the current features store, is
specified.
{!ltr reRankDocs=25 store=storeA model=simpleModelA}
And {!ltr} executes 'model based calculation', not 'store based
calculation'. Model knows what featues are required.
Why are all features executed?
Best regards,
Dariusz Wojtas
On Sat, Jan 13, 2018 at 4:03 PM, Diego Ceccarelli <
diego.ceccare...@gmail.com> wrote:
> Hi Dariusz,
>
> On Jan 12, 2018 14:40, "Dariusz Wojtas" wrote:
>
> Hi,
>
> I am working with the LTR rescoring.
> Works beautifully, but I am curious about something.
> How do I specify the feature store in a way different than using the
> [features] syntax?
> [features store=yourFeatureStore]
>
>
>
> What is the problem with this syntax? If the problem is the name of the
> field, you can also call it by doing fl=title,authors,myfield=[features
> store=yourFeatureStore]
> I can't think of alternative ways..
>
>
>
> I have a range of models in my custom feature store, with plenty of
> features implemented.
> I have found that when I call LTR with model using only two features, Solr
> still executes them all.
>
> My setup in solrconfig.xml
> -
> id,score,why_score:[explain style=nl],[features
> store=store_incidentDB]
> {!ltr reRankDocs=$reRankDocs model=simpleModelA}
> --
>
> simpleModel above only uses LinearModel with 2 features.
>
>
> What do I see in results?
> In response I can see it has executed ALL features (there are values
> calculated) in section:
> 1) -> response -> result -> doc -> HERE
>
> In addition, there is my model executed and only TWO features of the
> executed model are presented in:
>
>
> It is intended, the reason is that usually you want to execute your model
> and at the same time log a *superset* of the features to train the next
> model. If you want to compute only the features of the model you can define
> a featureStore that matches exactly the features that you have in the
> model.
>
> 2) -> response -> debug -> explain
>
> Why do I see all features being executed, if the specified model only
> contains two features?
>
> I tried to reduce 'fl' to:
> id,score,why_score:[explain style=nl]
> and id works as expected then:
> 1. additional features are not executed (correct)
> 2. my model works, only two features of the selected model (correct)
>
> And the final questions for this long email are:
> 1. why does it execute all features when i specify 'store'?
> 2. how do I specify the 'store', if I have more stores, but do not want to
> execute all their features?
>
>
> Just define a feature store that matches the features that you have in the
> model. Please note that the featureStore that you specify in fl= [features]
> field **will not** affect the reranking (the model will compute only the
> features that are specified in the model json file), you should ask for the
> [features] only if you want to log them.
> Please do not hesitate to ask if something is not clear ;)
>
> Cheers,
> Diego
>
>
>
> Best regards,
> Dariusz Wojtas
>
Re: LTR and working with feature stores
Ups,
Diego, I have just read your answer again.
Now I see that it is [features] element that triggers calculation of all
store features.
That gives hope model only executed the features it needs ;)
Best regards,
Dariusz Wojtas
On Sat, Jan 13, 2018 at 11:12 PM, Dariusz Wojtas wrote:
> Hi,
>
> Thanks for the response, I understand that all features from the given
> store are calculated, no matter if they are used or not.
> OK, spread features across different models.
> But what if different models share some features?
> Creating copies of feature definitions in different stores, one per model,
> is erroneous ...
> Having several models in one store, some of them use only part of these
> features - that seems 'expensive' ;)
>
> Simple syntax evolution would be very helpful, to give {!ltr} optional
> 'store' parameter. It could override the current features store, is
> specified.
> {!ltr reRankDocs=25 store=storeA model=simpleModelA}
>
> And {!ltr} executes 'model based calculation', not 'store based
> calculation'. Model knows what featues are required.
> Why are all features executed?
>
> Best regards,
> Dariusz Wojtas
>
>
> On Sat, Jan 13, 2018 at 4:03 PM, Diego Ceccarelli <
> diego.ceccare...@gmail.com> wrote:
>
>> Hi Dariusz,
>>
>> On Jan 12, 2018 14:40, "Dariusz Wojtas" wrote:
>>
>> Hi,
>>
>> I am working with the LTR rescoring.
>> Works beautifully, but I am curious about something.
>> How do I specify the feature store in a way different than using the
>> [features] syntax?
>> [features store=yourFeatureStore]
>>
>>
>>
>> What is the problem with this syntax? If the problem is the name of the
>> field, you can also call it by doing fl=title,authors,myfield=[features
>> store=yourFeatureStore]
>> I can't think of alternative ways..
>>
>>
>>
>> I have a range of models in my custom feature store, with plenty of
>> features implemented.
>> I have found that when I call LTR with model using only two features, Solr
>> still executes them all.
>>
>> My setup in solrconfig.xml
>> -
>> id,score,why_score:[explain style=nl],[features
>> store=store_incidentDB]
>> {!ltr reRankDocs=$reRankDocs model=simpleModelA}
>> --
>>
>> simpleModel above only uses LinearModel with 2 features.
>>
>>
>> What do I see in results?
>> In response I can see it has executed ALL features (there are values
>> calculated) in section:
>> 1) -> response -> result -> doc -> HERE
>>
>> In addition, there is my model executed and only TWO features of the
>> executed model are presented in:
>>
>>
>> It is intended, the reason is that usually you want to execute your model
>> and at the same time log a *superset* of the features to train the next
>> model. If you want to compute only the features of the model you can
>> define
>> a featureStore that matches exactly the features that you have in the
>> model.
>>
>> 2) -> response -> debug -> explain
>>
>> Why do I see all features being executed, if the specified model only
>> contains two features?
>>
>> I tried to reduce 'fl' to:
>> id,score,why_score:[explain style=nl]
>> and id works as expected then:
>> 1. additional features are not executed (correct)
>> 2. my model works, only two features of the selected model (correct)
>>
>> And the final questions for this long email are:
>> 1. why does it execute all features when i specify 'store'?
>> 2. how do I specify the 'store', if I have more stores, but do not want to
>> execute all their features?
>>
>>
>> Just define a feature store that matches the features that you have in the
>> model. Please note that the featureStore that you specify in fl=
>> [features]
>> field **will not** affect the reranking (the model will compute only the
>> features that are specified in the model json file), you should ask for
>> the
>> [features] only if you want to log them.
>> Please do not hesitate to ask if something is not clear ;)
>>
>> Cheers,
>> Diego
>>
>>
>>
>> Best regards,
>> Dariusz Wojtas
>>
>
>
