Hi,
I have added a PayloadTermQueryPlugin after reading
https://issues.apache.org/jira/browse/SOLR-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
my class is :
*/
*import org.apache.solr.common.params.SolrParams;*
*import org.apache.solr.common.util.NamedList;*
*import org.apache.solr.common.SolrException;*
*import org.apache.solr.request.SolrQueryRequest;*
*import org.apache.lucene.search.Query;*
*import org.apache.lucene.search.payloads.*;*
*import org.apache.lucene.queryParser.ParseException;*
*import org.apache.lucene.index.Term;*
*import org.apache.solr.search.QParser;*
*import org.apache.solr.search.QParserPlugin;*
*import org.apache.solr.search.QueryParsing;*
*
*
*public class PayloadTermQueryPlugin extends QParserPlugin {*
* private MinPayloadFunction payloadFunc;*
* @Override*
* public void init(NamedList args) {*
* this.payloadFunc=new MinPayloadFunction();*
* }*
*
*
* @Override*
* public QParser createParser(String qstr, SolrParams localParams,
SolrParams params, SolrQueryRequest req) {*
* return new QParser(qstr, localParams, params, req) {*
* @Override*
* public Query parse() throws ParseException {*
*
*
* Term term = new Term(localParams.get(QueryParsing.F),
localParams.get(QueryParsing.V));*
* return new PayloadTermQuery(term,payloadFunc, false);*
* }*
* };*
* }*
I tested it using Solrj
* @Override*
* protected void setUp() throws Exception {*
* super.setUp();*
* System.setProperty("solr.solr.home", "C:\\temp\\solr_home1.4");*
* CoreContainer.Initializer initializer = new
CoreContainer.Initializer();*
*
*
* try {*
* coreContainer = initializer.initialize();*
* } catch (IOException ex) {*
*
Logger.getLogger(BoostingSymilarityTest.class.getName()).log(Level.SEVERE,
null, ex);*
* } catch (ParserConfigurationException ex) {*
*
Logger.getLogger(BoostingSymilarityTest.class.getName()).log(Level.SEVERE,
null, ex);*
* } catch (SAXException ex) {*
*
Logger.getLogger(BoostingSymilarityTest.class.getName()).log(Level.SEVERE,
null, ex);*
* }*
* server = new EmbeddedSolrServer(coreContainer, "");*
* }*
* *
* public void testSeacrhAndBoost() {*
* SolrQuery query = new SolrQuery();*
* query.setQuery("PFirstName:steve");*
*query.setParam("hl.fl", "PFirstName");*
* query.setParam("defType", "payload");*
* query.setIncludeScore(true);*
*
*
* query.setRows(10);*
* query.setFacet(false);*
*
*
* try {*
* QueryResponse qr = server.query(query);*
* *
* List<PersonDoc> l = qr.getBeans(PersonDoc.class);*
* for (PersonDoc personDoc : l) {*
* System.out.println(personDoc);*
* }*
*
*
* } catch (SolrServerException ex) {*
*
Logger.getLogger(BoostingSymilarityTest.class.getName()).log(Level.SEVERE,
null, ex);*
*
*
* }*
* }*
*}*
I get an NPE trying to access localParams in the *public QParser
createParser(String qstr, SolrParams localParams, SolrParams params,
SolrQueryRequest req)* method
The NPE is actually in the *public Query parse() throws ParseException*method
I could not find documentation about the parse method, How can I pass
the localParams?
What is the difference between the localParams and params?
I would be happy to write the a case study on the wiki but, I'm not sure
exactly what you mean- The resolution i will eventually come to or the
process of finding it?
I'm still trying to figure out what exactly to do. I have purchased the
Solr 1.4 book , but it doesn't seem to have much information about my needs.
On Tue, Nov 10, 2009 at 10:09, David Ginzburg <da...@digitaltrowel.com>wrote:
> I would be happy to.
> I'm not sure exactly what you mean- The resolution i will eventually come
> to or the process of finding it?
> I'm still trying to figure out what exactly to do. I have purchased the
> Solr 1.4 book , but it doesn't seem to have much information about my needs.
>
>
> ---------- Forwarded message ----------
> From: Lance Norskog <goks...@gmail.com>
> Date: Tue, Nov 10, 2009 at 04:11
> Subject: Re: synonym payload boosting
> To: solr-user@lucene.apache.org
>
>
> David, when you get this working would you consider writing a case
> study on the wiki? Nothing complex, just something that describes how
> you did several customizations to create a new feature.
>
> On Mon, Nov 9, 2009 at 4:10 AM, Grant Ingersoll <gsing...@apache.org>
> wrote:
> >
> > On Nov 9, 2009, at 4:41 AM, David Ginzburg wrote:
> >
> >> I have found this
> >>
> >>
> https://issues.apache.org/jira/browse/SOLR-1485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> >> patch
> >> But i don't want to use any function, just the normal scoring and the
> >> similarity class I have written.
> >> Can you point me to modifications I need (if any) ?
> >>
> >>
> >
> > Amhet's point is that you need some query that will actually invoke the
> > payload in scoring. PayloadTermQuery and PayloadNearQuery are the two
> that
> > do this in Lucene. You can certainly write your own, as well.
> >
> > -Grant
> >
> >>
> >> On Sun, Nov 8, 2009 at 16:33, AHMET ARSLAN <iori...@yahoo.com> wrote:
> >>
> >>> Additionaly you need to modify your queryparser to return
> >>> BoostingTermQuery, PayloadTermQuery, PayloadNearQuery etc.
> >>>
> >>> With these types of Queries scorePayload method invoked.
> >>>
> >>> Hope this helps.
> >>>
> >>> --- On Sun, 11/8/09, David Ginzburg <da...@digitaltrowel.com> wrote:
> >>>
> >>>> From: David Ginzburg <da...@digitaltrowel.com>
> >>>> Subject: synonym payload boosting
> >>>> To: solr-user@lucene.apache.org
> >>>> Date: Sunday, November 8, 2009, 4:06 PM
> >>>> Hi,
> >>>> I have a field and a wighted synonym map.
> >>>> I have indexed the synonyms with the weight as payload.
> >>>> my code snippet from my filter
> >>>>
> >>>> *public Token next(final Token reusableToken) throws
> >>>> IOException *
> >>>> * . *
> >>>> * . *
> >>>> * .*
> >>>> * Payload boostPayload;*
> >>>> *
> >>>> *
> >>>> * for (Synonym synonym : syns)
> >>>> {*
> >>>> * *
> >>>> * Token newTok =
> >>>> new Token(nToken.startOffset(),
> >>>> nToken.endOffset(), "SYNONYM");*
> >>>> *
> >>>> newTok.setTermBuffer(synonym.getToken().toCharArray(), 0,
> >>>> synonym.getToken().length());*
> >>>> * // set the
> >>>> position increment to zero*
> >>>> * // this tells
> >>>> lucene the synonym is*
> >>>> * // in the exact
> >>>> same location as the originating word*
> >>>> *
> >>>> newTok.setPositionIncrement(0);*
> >>>> * boostPayload =
> >>>> new
> >>>> Payload(PayloadHelper.encodeFloat(synonym.getWieght()));*
> >>>> *
> >>>> newTok.setPayload(boostPayload);*
> >>>> *
> >>>> *
> >>>> I have put it in the index time analyzer : this is my field
> >>>> definition:
> >>>>
> >>>> *
> >>>> <fieldType name="PersonName" class="solr.TextField"
> >>>> positionIncrementGap="100" >
> >>>> <analyzer type="index">
> >>>> <tokenizer
> >>>> class="solr.WhitespaceTokenizerFactory"/>
> >>>> <filter
> >>>> class="solr.StopFilterFactory" ignoreCase="true"
> >>>> words="stopwords.txt"/>
> >>>> <filter
> >>>> class="solr.LowerCaseFilterFactory"/>
> >>>> <filter
> >>>> class="com.digitaltrowel.solr.DTSynonymFactory"
> >>>> FreskoFunction="names_with_scoresPipe23Columns.txt"
> >>>> ignoreCase="true"
> >>>> expand="false"/>
> >>>>
> >>>> <!--<filter
> >>>> class="solr.EnglishPorterFilterFactory"
> >>>> protected="protwords.txt"/>-->
> >>>> <!--<filter
> >>>> class="solr.RemoveDuplicatesTokenFilterFactory"/>-->
> >>>> </analyzer>
> >>>> <analyzer type="query">
> >>>> <tokenizer
> >>>> class="solr.WhitespaceTokenizerFactory"/>
> >>>> <filter
> >>>> class="solr.LowerCaseFilterFactory"/>
> >>>> <!--<filter
> >>>> class="com.digitaltrowel.solr.DTSynonymFactory"
> >>>> synonyms="synonyms.txt" ignoreCase="true"
> >>>> expand="false"/>-->
> >>>> <filter
> >>>> class="solr.StopFilterFactory" ignoreCase="true"
> >>>> words="stopwords.txt"/>
> >>>> <!--<filter
> >>>> class="solr.EnglishPorterFilterFactory"
> >>>> protected="protwords.txt"/>-->
> >>>> <!--<filter
> >>>> class="solr.RemoveDuplicatesTokenFilterFactory"/
> >>>>>
> >>>>> -->
> >>>>
> >>>> </analyzer>
> >>>> </fieldType>
> >>>>
> >>>>
> >>>> my similarity class is
> >>>> public class BoostingSymilarity extends DefaultSimilarity
> >>>> {
> >>>>
> >>>>
> >>>> public BoostingSymilarity(){
> >>>> super();
> >>>>
> >>>> }
> >>>> @Override
> >>>> public float scorePayload(String field,
> >>>> byte [] payload, int offset,
> >>>> int length)
> >>>> {
> >>>> double weight = PayloadHelper.decodeFloat(payload, 0);
> >>>> return (float)weight;
> >>>> }
> >>>>
> >>>> @Override public float coord(int overlap, int maxoverlap)
> >>>> {
> >>>> return 1.0f;
> >>>> }
> >>>>
> >>>> @Override public float idf(int docFreq, int numDocs)
> >>>> {
> >>>> return 1.0f;
> >>>> }
> >>>>
> >>>> @Override public float lengthNorm(String fieldName, int
> >>>> numTerms)
> >>>> {
> >>>> return 1.0f;
> >>>> }
> >>>>
> >>>> @Override public float tf(float freq)
> >>>> {
> >>>> return 1.0f;
> >>>> }
> >>>> }
> >>>>
> >>>> My problem is that scorePayload method does not get called
> >>>> at search time
> >>>> like the other methods in my similarity class.
> >>>> I tested and verified it with break points.
> >>>> What am I doing wrong?
> >>>> I used solr 1.3 and thinking of the payload boos support in
> >>>> solr 1.4.
> >>>>
> >>>>
> >>>> *
> >>>>
> >>>
> >>> __________________________________________________
> >>> Do You Yahoo!?
> >>> Tired of spam? Yahoo! Mail has the best spam protection around
> >>> http://mail.yahoo.com
> >>>
> >>
> >>
> >>
> >> --
> >> Regards
> >>
> >> _____________________
> >> David Ginzburg
> >> Developer, Digital Trowel
> >> 1 Hayarden St., Airport City
> >> [POB 169, NATBAG]
> >> Lod, 70151, Israel
> >> http://www.digitaltrowel.com/
> >> Office: +972 73 240 522
> >> Mobile: +972 50 496 0595
> >>
> >> CHECK OUT OUR NEW TEXT MINING BLOG:
> >> http://mineyourbusiness.wordpress.com/
> >
> > --------------------------
> > Grant Ingersoll
> > http://www.lucidimagination.com/
> >
> > Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
> > Solr/Lucene:
> > http://www.lucidimagination.com/search
> >
> >
>
>
>
> --
> Lance Norskog
> goks...@gmail.com
>
>
>
> --
> Regards
>
> _____________________
> David Ginzburg
> Developer, Digital Trowel
> 1 Hayarden St., Airport City
> [POB 169, NATBAG]
> Lod, 70151, Israel
> http://www.digitaltrowel.com/
> Office: +972 73 240 522
> Mobile: +972 50 496 0595
>
> CHECK OUT OUR NEW TEXT MINING BLOG:
> http://mineyourbusiness.wordpress.com/
>
--
Regards
_____________________
David Ginzburg
Developer, Digital Trowel
1 Hayarden St., Airport City
[POB 169, NATBAG]
Lod, 70151, Israel
http://www.digitaltrowel.com/
Office: +972 73 240 522
Mobile: +972 50 496 0595
CHECK OUT OUR NEW TEXT MINING BLOG:
http://mineyourbusiness.wordpress.com/
