Hi Emir,
The solution we wanted to implement is to show top 100 best match technology
names from the list of technology names we have. Whatever technology names user
has typed will first reach SQL Server and exact match will be done if
possible[name==name] , only those do not exactly match[spelling mistakes,
jumbled words] will be searched in SOLR.
With the below setup if I query title:(Microsoft Ofice 365) I get the below
result [note:scores are same?]
{
"title":"Lync - Microsoft Office 365",
"score":7.7472024
},
{
"title":"Microsoft Office 365",
"score":7.7472024
},
When I query title:(Microsoft Ofice 365) OR title_ws:(Microsoft Ofice 365) {
"title":"RIM BlackBerry Enterprise Server (BES) for Microsoft Office 365
1.0",
"title_ws":"RIM BlackBerry Enterprise Server (BES) for Microsoft Office 365
1.0",
"score":3.9297152
},
{
"title":"Microsoft Office 365",
"title_ws":"Microsoft Office 365",
"score":3.1437721
}
When I query title:(Microsoft Ofice 365) OR title_ws:(Microsoft Ofice 365)
qf=title_ws^1
I don’t get any results
The expected result is
{
"title":"Microsoft Office 365",
"title_ws":"Microsoft Office 365",
},
{
"title":"Microsoft Office 365 1.0",
"title_ws":"Microsoft Office 365 1.0",
},
{
"title":"Microsoft Office 365 14.0",
"title_ws":"Microsoft Office 365 14.0",
},
{
"title":"Microsoft Office 365 14.3",
"title_ws":"Microsoft Office 365 14.3",
},
{
"title":"Microsoft Office 365 14.4",
"title_ws":"Microsoft Office 365 14.4",
},
<fieldType name="txt_token_ng" class="solr.TextField" positionIncrementGap="0"
omitNorms="false">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2"
maxGramSize="2"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2"
maxGramSize="2"/>
</analyzer>
</fieldType>
<field name="id" type="int" indexed="true" stored="true" required="true"
multiValued="false" />
<field name="title" type="txt_token_ng" indexed="true" stored="true"
multiValued="false"/>
<field name="manufacturername" type="txt_token_ng" indexed="true" stored="true"
multiValued="false"/>
<field name="productname" type="txt_token_ng" indexed="true" stored="true"
multiValued="false"/>
<field name="version" type="txt_token_ng" indexed="true" stored="true"
multiValued="false"/>
<fieldType name="txt_token_ws" class="solr.TextField" positionIncrementGap="0"
omitNorms="false">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
<field name="title_ws" type="txt_token_ws" indexed="true" stored="true"
multiValued="false"/>
<field name="manufacturername_ws" type="txt_token_ws" indexed="true" stored="true"
multiValued="false"/>
<field name="productname_ws" type="txt_token_ws" indexed="true" stored="true"
multiValued="false"/>
<field name="version_ws" type="txt_token_ws" indexed="true" stored="true"
multiValued="false"/>
<copyField source="title" dest="title_ws"/>
<copyField source="manufacturername" dest="manufacturername_ws"/>
<copyField source="productname" dest="productname_ws"/>
<copyField source="version" dest="version_ws"/>
Thanks
Rajesh
Corporate Executive Board India Private Limited. Registration No:
U741040HR2004PTC035324. Registered office: 6th Floor, Tower B, DLF Building
No.10 DLF Cyber City, Gurgaon, Haryana-122002, India.
This e-mail and/or its attachments are intended only for the use of the
addressee(s) and may contain confidential and legally privileged information
belonging to CEB and/or its subsidiaries, including CEB subsidiaries that offer
SHL Talent Measurement products and services. If you have received this e-mail
in error, please notify the sender and immediately, destroy all copies of this
email and its attachments. The publication, copying, in whole or in part, or
use or dissemination in any other way of this e-mail and attachments by anyone
other than the intended person(s) is prohibited.
-----Original Message-----
From: Emir Arnautovic [mailto:emir.arnauto...@sematext.com]
Sent: Monday, March 7, 2016 8:16 PM
To: solr-user@lucene.apache.org
Subject: Re: Text search NGram
Not sure I understood question. What I meant is you to try setting
omitNorms="false" to your txt_token field type if you want to stick with ngram
only solution:
<fieldType name="txt_token" class="solr.TextField" positionIncrementGap="0"
omitNorms="false">
<analyzer type="index">
<charFilter class="solr.PatternReplaceCharFilterFactory"
pattern="\s+" replacement=" "/>
<tokenizer
class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2"
maxGramSize="800"/>
</analyzer>
<analyzer type="query">
<charFilter class="solr.PatternReplaceCharFilterFactory"
pattern="\s+" replacement=" "/>
<tokenizer
class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2"
maxGramSize="800"/>
</analyzer>
</fieldType>
and to add new field type and field to keep nonngram version of field.
Something like:
<fieldType name="txt_token_simple" class="solr.TextField" positionIncrementGap="0"
>
<analyzer type="index">
<charFilter class="solr.PatternReplaceCharFilterFactory"
pattern="\s+" replacement=" "/>
<tokenizer
class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<charFilter class="solr.PatternReplaceCharFilterFactory"
pattern="\s+" replacement=" "/>
<tokenizer
class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
and use copyField to copy to both fields and query title:test OR
title_simple:test.
Emir
On 07.03.2016 15:31, G, Rajesh wrote:
Hi Emir,
I have already applied
<tokenizer class="solr.WhitespaceTokenizerFactory"/> and then I have applied <filter
class="solr.NGramFilterFactory" minGramSize="2" maxGramSize="800"/>. Is this what you wanted me to
have in my config?
Thanks
Rajesh
Corporate Executive Board India Private Limited. Registration No:
U741040HR2004PTC035324. Registered office: 6th Floor, Tower B, DLF Building
No.10 DLF Cyber City, Gurgaon, Haryana-122002, India.
This e-mail and/or its attachments are intended only for the use of the
addressee(s) and may contain confidential and legally privileged information
belonging to CEB and/or its subsidiaries, including CEB subsidiaries that offer
SHL Talent Measurement products and services. If you have received this e-mail
in error, please notify the sender and immediately, destroy all copies of this
email and its attachments. The publication, copying, in whole or in part, or
use or dissemination in any other way of this e-mail and attachments by anyone
other than the intended person(s) is prohibited.
-----Original Message-----
From: G, Rajesh [mailto:r...@cebglobal.com]
Sent: Monday, March 7, 2016 7:50 PM
To: solr-user@lucene.apache.org
Subject: RE: Text search NGram
Hi Emir,
Thanks for you email. Can you please help me to understand what do you mean by "e.g.
boost if matching tokenized fileds to make sure exact matches are ordered first"
Corporate Executive Board India Private Limited. Registration No:
U741040HR2004PTC035324. Registered office: 6th Floor, Tower B, DLF Building
No.10 DLF Cyber City, Gurgaon, Haryana-122002, India.
This e-mail and/or its attachments are intended only for the use of the
addressee(s) and may contain confidential and legally privileged information
belonging to CEB and/or its subsidiaries, including CEB subsidiaries that offer
SHL Talent Measurement products and services. If you have received this e-mail
in error, please notify the sender and immediately, destroy all copies of this
email and its attachments. The publication, copying, in whole or in part, or
use or dissemination in any other way of this e-mail and attachments by anyone
other than the intended person(s) is prohibited.
-----Original Message-----
From: Emir Arnautovic [mailto:emir.arnauto...@sematext.com]
Sent: Monday, March 7, 2016 7:36 PM
To: solr-user@lucene.apache.org
Subject: Re: Text search NGram
Hi Rajesh,
It is most likely related to norms - you can try setting omitNorms="true" and
reindexing content. Anyway, it is not common to use just ngrams for matching content - in
such case you can expect more unexpected ordering/results. You should combine ngrams
fields with normally tokenized fields (e.g. boost if matching tokenized fileds to make
sure exact matches are ordered first).
Regards,
Emir
On 07.03.2016 11:44, G, Rajesh wrote:
Hi Team,
We have the blow type and we have indexed the value "title": "Microsoft Visual Studio 2006" and
"title": "Microsoft Visual Studio 8.0.61205.56 (2005)"
When I search for title:(Microsoft Visual AND Studio AND 2005) I get Microsoft
Visual Studio 8.0.61205.56 (2005) as the second record and Microsoft Visual
Studio 2006 as first record. I wanted to have Microsoft Visual Studio
8.0.61205.56 (2005) listed first since the user has searched for Microsoft
Visual Studio 2005. Can you please help?.
We are using NGram so it takes care of misspelled or jumbled words[it
works as expected] e.g.
searching Micrs Visual Studio will gets Microsoft Visual Studio
searching Visual Microsoft Studio will gets Microsoft Visual Studio
<fieldType name="txt_token" class="solr.TextField" positionIncrementGap="0"
>
<analyzer type="index">
<charFilter class="solr.PatternReplaceCharFilterFactory"
pattern="\s+" replacement=" "/>
<tokenizer
class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2"
maxGramSize="800"/>
</analyzer>
<analyzer type="query">
<charFilter class="solr.PatternReplaceCharFilterFactory"
pattern="\s+" replacement=" "/>
<tokenizer
class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.NGramFilterFactory" minGramSize="2"
maxGramSize="800"/>
</analyzer>
</fieldType>
Corporate Executive Board India Private Limited. Registration No:
U741040HR2004PTC035324. Registered office: 6th Floor, Tower B, DLF Building
No.10 DLF Cyber City, Gurgaon, Haryana-122002, India..
This e-mail and/or its attachments are intended only for the use of the
addressee(s) and may contain confidential and legally privileged information
belonging to CEB and/or its subsidiaries, including CEB subsidiaries that offer
SHL Talent Measurement products and services. If you have received this e-mail
in error, please notify the sender and immediately, destroy all copies of this
email and its attachments. The publication, copying, in whole or in part, or
use or dissemination in any other way of this e-mail and attachments by anyone
other than the intended person(s) is prohibited.
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr &
Elasticsearch Support * http://sematext.com/
