Hi Shamik,
How about using ShingleFilter which constructs token n-grams from a token
stream?
http://lucene.apache.org/core/6_6_0/analyzers-common/org/apache/lucene/analysis/shingle/ShingleFilter.html
As for "about dynamic block", ShingleFilter produces "about dynamic" and "dynamic
block".
Thanks,
Koji
On 2017/07/20 5:54, Shamik Bandopadhyay wrote:
Hi,
I'm trying to show titles with exact query phrase match at the top of the
result. That includes supporting stop words as part of the phrase. For e.g.
if I'm using "about dynamic "block" , I expect the title with "About
Dynamic Blocks" to appear at the top. Since the title field uses
stoprword filter factory as part of its analysis chain, I decided to create
a copyfield of title and use that in search with a higher boost. That
didn't seem to work either. Although it brought back the expected document
at the top, it excluded documents with title "Dynamic Block Grip
Reference", to be precise content which doesn't have "about" in title or
subject. Even setting the default operator to OR didn't make any
difference. Here's the entry from config.
<field name="title" type="adsktext" indexed="true" stored="true"
multiValued="true"/>
<field name="titleExact" type="textExact" indexed="true" stored="true" />
<field name="subject" type="adsktext" indexed="true" stored="true"/>
<field name="description" type="adsktext" indexed="true" stored="true"/>
<copyField source="title" dest="titleExact" />
<fieldType name="adsktext" class="solr.TextField"
positionIncrementGap="100" autoGeneratePhraseQueries="false">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymGraphFilterFactory"
synonyms="synonyms/synonyms.txt" ignoreCase="true" expand="true"/>
<!--<filter class="solr.KeywordRepeatFilterFactory"/>-->
<!--<filter class="solr.CommonGramsFilterFactory" words="stopwords.txt"
ignoreCase="true"/>-->
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1"
generateNumberParts="1" catenateWords="0" catenateNumbers="1"
catenateAll="0" splitOnCaseChange="0"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" />
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<!--<filter class="solr.PorterStemFilterFactory"/>-->
<filter class="com.nicholasding.search.solr.LemmatizerFilterFactory"/>
<!--<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>-->
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymGraphFilterFactory"
synonyms="synonyms/synonyms.txt" ignoreCase="true" expand="true"/>
<!--<filter class="solr.KeywordRepeatFilterFactory"/>-->
<!--<filter class="solr.CommonGramsFilterFactory" words="stopwords.txt"
ignoreCase="true"/>-->
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1"
generateNumberParts="1" catenateWords="0" catenateNumbers="1"
catenateAll="0" splitOnCaseChange="0"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt" />
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
<!--<filter class="solr.PorterStemFilterFactory"/>-->
<filter class="com.nicholasding.search.solr.LemmatizerFilterFactory"/>
<!--<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>-->
</analyzer>
</fieldType>
<fieldType name="textExact" class="solr.TextField"
positionIncrementGap="100" >
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LimitTokenCountFilterFactory" maxTokenCount="20"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LimitTokenCountFilterFactory" maxTokenCount="20"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
Request handler:
<requestHandler name="/browse" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<str name="wt">velocity</str>
<str name="v.template">browse</str>
<str name="v.layout">layout</str>
<str name="title">Solritas</str>
<str name="q.op">AND</str>
<str name="defType">edismax</str>
<str name="qf">title^5 titleExact^15 subject^3 description^2</str>
<!--<str name="pf">title^10 titleExact^100</str>-->
<str name="mm">100%</str>
<str name="q.alt">*:*</str>
<str name="rows">10</str>
<str name="fl">*,score</str>
</lst>
</requestHandler>
Sample data:
<add>
<doc>
<field name="id">SOLR1000</field>
<field name="title">About Dynamic Blocks</field>
<field name="subject">Dynamic blocks contain rules, or parameters, for how
to change the appearance of the block reference when it is inserted in the
drawing. With dynamic blocks you can insert one block that can change
shape, size, or configuration instead of inserting one of many static block
definitions. For example, instead of creating multiple interior door blocks
of different sizes, you can create one resizable door block. You author
dynamic blocks with either constraint parameters or action parameters.
Note: Using both constraint parameters and action parameters in the same
block definition is not recommended. Add Constraints In a block definition,
constraint parameters Associate objects with one another Restrict geometry
or dimensions The following example shows a block reference with a
constraint (in gray) and a constraint parameter (blue with grip). Once the
block is inserted into the drawing, the constraint parameters can be edited
as properties by using the Properties palette. Add Actions and Parameters
In a block definition, actions and parameters provide rules for the
behavior of a block once it is inserted into the drawing. Depending on the
specified block geometry or parameter, you can associate an action to that
parameter. The parameter is represented as a grip in the drawing. When you
edit the grip, the associated action determines what will change in the
block reference. Like constraint parameters, action parameters can be
changed using the Properties palette.</field>
<field name="description">Dynamic blocks contain rules, or parameters, for
how to change the appearance of the block reference when it is inserted in
the drawing.</field>
</doc>
<doc>
<field name="id">SOLR1001</field>
<field name="title">About Creating Dynamic Blocks</field>
<field name="subject">This table gives an overview of the steps required
add behaviors that make blocks dynamic. Plan the block content. Know how
the block should change or move, and what parts will depend on the others.
Example: The block will be resizable, and after it is resized, additional
geometry is displayed. Draw the geometry. Draw the block geometry in the
drawing area or the Block Editor. Note: If you will use visibility states
to change how geometry is displayed, you may not want to include all the
geometry at this point. Add parameters. Add either individual parameters or
parameter sets to define geometry that will be affected by an action or
manipulation. Keep in mind the objects that will be dependent on one
another. Add actions. If you are working with action parameters, if
necessary, add actions to define what will happen to the geometry when it
is manipulated. Define custom properties. Add properties that determine how
the block is displayed in the drawing area. Custom properties affect grips,
labels, and preset values for block geometry. Test the block. On the
ribbon, in the Block Editor contextual tab, Open/Save panel, click Test
Block to test the block before you save it.</field>
<field name="description">This table gives an overview of the steps
required add behaviors that make blocks dynamic.</field>
</doc>
<doc>
<field name="id">SOLR1002</field>
<field name="title">About Modifying Dynamic Block Definitions</field>
<field name="subject">Use the Block Editor to edit, correct, and save a
block definition. Correct Errors in Action Parameters A yellow alert icon (
) is displayed when A parameter is not associated with an action An action
is not associated with a parameter or selection set To correct these
errors, hover over the yellow alert icon until the tooltip displays a
description of the problem. Then double-click the constraint and follow the
prompts. Save Dynamic Blocks When you save a block definition, the current
values of the geometry and parameters in the block become the default
values for the block reference. The default visibility state for the block
reference is the visibility state at the top of the list in the Manage
Visibility States dialog box. Note: If you click File menu Save while you
are in the Block Editor, you will save the drawing but not the block
definition. You must specifically save the block definition while you are
in the Block Editor.</field>
<field name="description">Use the Block Editor to edit, correct, and save a
block definition.</field>
</doc>
<doc>
<field name="id">SOLR1003</field>
<field name="title">About Adding Parameters to Dynamic Blocks</field>
<field name="subject">Parameters determine the geometry that will be
affected by an action when you manipulate a block reference. When you add a
parameter to a dynamic block definition, grips are displayed at key points
of the parameter. Key points are the parts of a parameter that you use to
manipulate the block reference. For example, a linear parameter has key
points at its base point and end point. You can manipulate the parameter
distance from either key point. You can specify grip size and color for
display in the Block Editor. This setting does not affect the size and
color of the grips in a block reference.</field>
<field name="description">Parameters determine the geometry that will be
affected by an action when you manipulate a block reference.</field>
</doc>
<doc>
<field name="id">SOLR1004</field>
<field name="title">About Adding Actions to Dynamic Blocks</field>
<field name="subject">Actions define how the geometry of a dynamic block
reference will move or change when its grips are manipulated. In general,
you associate an action with a parameter and the following: Key point . The
point on a parameter that drives the action. Selection set . The geometry
that will be affected by the action. When you move the grip in the example
above, only the geometry in the selection set is stretched. Specify
Distance and Angle Override Values Distance multiplier and angle offset
override properties allow you to specify a factor by which a parameter
value is increased or decreased. Action overrides are properties of actions
that have no effect on the block reference until it is manipulated in a
drawing. Use distance multiplier overrides with the following actions: Move
action Stretch action Polar Stretch action You can specify these action
override properties on the command line when you add an action to a dynamic
block definition. You can also specify these properties in the Properties
palette when you select an action in the Block Editor.</field>
<field name="description">Actions define how the geometry of a dynamic
block reference will move or change when its grips are manipulated.</field>
</doc>
<doc>
<field name="id">SOLR1005</field>
<field name="title">Dynamic Block Grip Reference</field>
<field name="subject">This table describes the grips and how they're used.
Grip Type Grip Movement or Result Parameters: Associated Actions Standard
Within a plane in any direction Base: None Point: Move, Stretch, Polar:
Move, Scale, Stretch, Polar Stretch, Array XY: Move, Scale, Stretch, Array
Linear Back and forth in a defined direction or along an axis Linear: Move,
Scale, Stretch, Array Rotation Around an axis Rotation: Rotate Flip
Switches to a mirror image of the block geometry Flip: Flip Alignment
Within a plane in any direction; when moved over an object, triggers the
block reference to align with the object Alignment: None (action is
implied) Lookup Displays a list of values Visibility: None (action is
implied) Lookup: Lookup</field>
<field name="description">This table describes the grips and how they're
used.</field>
</doc>
<doc>
<field name="id">SOLR1006</field>
<field name="title">To Open a Drawing Saved as a Dynamic Block (Block
Editor)</field>
<field name="subject">Click the Application button Open Drawing. Open the
drawing file that is saved as a block. An alert states that the drawing
contains authoring elements. In the alert dialog box, click Yes to open the
drawing in the Block Editor.</field>
<field name="description">None</field>
</doc>
</add>
Here's my query:
http://localhost:8983/solr/techproducts/browse?q=About%20dynamic%20blocks&wt=xml&fl=title,subject,score&facet=off&hl=off&spellcheck=off&qf=title
^5%20titleExact^1%20subject^1%20description^1&debugQuery=true&rows=100&q.op=OR
It skips SOLR1004 and SOLR1005 since they don't have the term "about".
Adding q.op=OR didn't make any difference in result. Here's the debug query
for both q.op=AND and OR
AND:
<str name="parsedquery">(+(+DisjunctionMaxQuery((titleExact:about))
+DisjunctionMaxQuery((titleExact:dynamic | description:dynamic |
(title:dynamic)^5.0 | subject:dynamic))
+DisjunctionMaxQuery((titleExact:blocks | description:block |
(title:block)^5.0 | subject:block))))/no_coord</str>
<str name="parsedquery_toString">+(+(titleExact:about) +(titleExact:dynamic
| description:dynamic | (title:dynamic)^5.0 | subject:dynamic)
+(titleExact:blocks | description:block | (title:block)^5.0 |
subject:block))</str>
OR:
<str name="parsedquery">(+(DisjunctionMaxQuery((titleExact:about))
DisjunctionMaxQuery((titleExact:dynamic | description:dynamic |
(title:dynamic)^5.0 | subject:dynamic))
DisjunctionMaxQuery((titleExact:blocks | description:block |
(title:block)^5.0 | subject:block)))~3)/no_coord</str>
<str name="parsedquery_toString">+(((titleExact:about) (titleExact:dynamic
| description:dynamic | (title:dynamic)^5.0 | subject:dynamic)
(titleExact:blocks | description:block | (title:block)^5.0 |
subject:block))~3)</str>
Apologies for the long thread, but not sure what I'm doing wrong. I'll
appreciate if you someone can provide pointers. If there's a different
approach to solving this issue, please let me know.
Thanks,
Shamik
