docFreq disable / disable end of word letter removal

2006-07-12 Thread Tom Weber

Hello,

  for my specific project, I would like to ask if the following  
settings can be made on the solr system:


  - Currently, I see that the docFreq is also playing in the  
scoring. Is is possible to disable this feature so that this is not  
calculated in the score ?
  - I see that solr is stripping some characters at the end of the  
search words. This is okay, but i try, for example, a search on  
"comed", and he is searching for "come". Can I select when the system  
will remove which letters and when ? Or where can I disable this  
system ? The removal of the trailing "s" is great, but for some  
circumstances, the "d" removal of "comed" is not the ideal way.


  Thanks for the help

  And thanks for the really great tool solr is

  Tom


how to use FunctionQuery with lucene?

2006-07-12 Thread Brent Verner
Hi,

  FunctionQuery looks _exactly_ like what I want.  I want to apply a rank/sort
kind of thing based on the value of a Document field value.  Specifically, my
users want more "recent" documents to have a higher rank than older documents
and I have a date field with values like 20060707.  It looks like I can do 
what I want with a LinearFloatFunction :-)

  I'm using lucene's query parser to get a Query.  How can I use FunctionQuery
in conjunction with my existing query?  I'm gonna _try_ Query.combine(Query[]),
but I'm not sure, from reading the lucene docs, that it will do what I want...

Many thanks!
  Brent



Re: docFreq disable / disable end of word letter removal

2006-07-12 Thread Chris Hostetter
:- Currently, I see that the docFreq is also playing in the
: scoring. Is is possible to disable this feature so that this is not
: calculated in the score ?

this is a fairly core aspect of the Lucene scoring calculation, but it can
be changed with a small bit of java coding.  If you write your own
subclass of "Similarity" you can override the "idf" function to return a
constanct value regardless of the docFreq.  You can then specify your new
Similarity class by name in your schema.xml and Solr will use it instead
of the default...

http://lucene.apache.org/java/docs/api/org/apache/lucene/search/Similarity.html

:- I see that solr is stripping some characters at the end of the
: search words. This is okay, but i try, for example, a search on
: "comed", and he is searching for "come". Can I select when the system
: will remove which letters and when ? Or where can I disable this
: system ? The removal of the trailing "s" is great, but for some
: circumstances, the "d" removal of "comed" is not the ideal way.

This is all determined by the Analyzer used for each field (or more
generally: field type) ... this is also configured via the schema.xml.  As
with SImilarity, you can write your own java subclass to use if you want
extremely customized behavior, or you can use any of the Analyzers that
come with lucene (by name) or you can build up an Analyzer in your
schema.xml using solr TokenizerFactories and TokenFilterFactories.
Docs on all of the Solr Factories can be found in the wiki...

http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
http://lucene.apache.org/java/docs/api/org/apache/lucene/analysis/Analyzer.html



-Hoss



Re: how to use FunctionQuery with lucene?

2006-07-12 Thread Chris Hostetter

:   FunctionQuery looks _exactly_ like what I want.  I want to apply a rank/sort
: kind of thing based on the value of a Document field value.  Specifically, my
: users want more "recent" documents to have a higher rank than older documents
: and I have a date field with values like 20060707.  It looks like I can do
: what I want with a LinearFloatFunction :-)

pretty much ... but RecipricalFloatFunction is pretty great for dates too.

:   I'm using lucene's query parser to get a Query.  How can I use
: FunctionQuery in conjunction with my existing query?  I'm gonna _try_
: Query.combine(Query[]), but I'm not sure, from reading the lucene docs,
: that it will do what I want...

that's not really what that method is for ... just construct a
BooleanQuery and add your FunctionQuery and the query you got from the
QUeryParser to it as sub-clauses.  You'll probably want the clause from
QUeryParser to be mandatory, and the FunctionQUery to be optional.




-Hoss



Re: how to use FunctionQuery with lucene?

2006-07-12 Thread Brent Verner
[2006-07-12 11:55] Chris Hostetter said:
 
| pretty much ... but RecipricalFloatFunction is pretty great for dates too.

  Will have to check it out :-)

| : Query.combine(Query[]), but I'm not sure, from reading the lucene docs,
| : that it will do what I want...
| 
| that's not really what that method is for ... just construct a
| BooleanQuery and add your FunctionQuery and the query you got from the

  Yeah, I dug through query parser and saw that BooleanQuery was used to
combine queries.

  BTW, _very_ nice work on solr.  I'm only using a little bit of it in my
current project, but I suspect I'll take advantage of some of the other 
bits in the future!

cheers!
  Brent