Re: Got it working! And some questions

2006-09-13 Thread Erik Hatcher


On Sep 12, 2006, at 4:47 PM, Chris Hostetter wrote:

: I've implemented the ability to override the default operator with
: q.op=AND|OR.  The patch is pasted below for your review.

if i'm reading that right, one subtlety is that "new
SolrQueryParser(schema,field)" no longer pas attention to
schema.getQueryParserDefaultOperator() -- that only only becomes
applicable when using QueryParsing.parseQuery

...i am very okay with this change, i wasn't really a fan of the  
fact that

the SolrQueryParser pulled that info out of the IndexSchema in it's
constructor previously, i just wanted to point out that this patch  
would

change that.

Perhaps the constructor for SolrQueryParser shouldn't be aware of  
the op
at all (either from the schema or from the SolrParams) -- and  
setting it
should be left to QueryParsing.parseQuery (or some other utility in  
the
QueryParsing class) ... personally i'm a fan of leaving  
SolrQueryParser as
much like QueryParser as possible -- with the only real change  
being the

knowledege of hte individual field formats.


I've reworked it based on your feedback.  The patch is pasted below.

SolrQueryParser now knows nothing about the default operator, it is  
set from QueryParsing.parseQuery() when passed a SolrParams.


QueryParsing.parseQuery() methods could be simplified, perhaps even  
into a single method, that took a query expression and a  
SolrQueryRequest, where it can get the SolrParams and  IndexSchema.   
It could even get the "q" parameter from there, but there is code  
that passes expressions that don't come from "q".  Maybe we could  
have two parseQuery() methods:  parseQuery(String expression,  
SolrQueryRequest req) and parseQuery(SolrQueryRequest req), and for  
the latter the "q" parameter is pulled from the request and used as  
the expression.


As it is, the patch below works fine and I'm happy to commit it, but  
am happy to rework this sort of thing to get it as clean as others like.


Erik


Index: src/java/org/apache/solr/search/SolrQueryParser.java
===
--- src/java/org/apache/solr/search/SolrQueryParser.java	(revision  
442772)

+++ src/java/org/apache/solr/search/SolrQueryParser.java(working copy)
@@ -37,7 +37,6 @@
 super(defaultField == null ? schema.getDefaultSearchFieldName 
() : defaultField, schema.getQueryAnalyzer());

 this.schema = schema;
 setLowercaseExpandedTerms(false);
-setDefaultOperator("AND".equals 
(schema.getQueryParserDefaultOperator()) ? QueryParser.Operator.AND :  
QueryParser.Operator.OR);

   }
   protected Query getFieldQuery(String field, String queryText)  
throws ParseException {

Index: src/java/org/apache/solr/search/QueryParsing.java
===
--- src/java/org/apache/solr/search/QueryParsing.java   (revision 442772)
+++ src/java/org/apache/solr/search/QueryParsing.java   (working copy)
@@ -19,6 +19,7 @@
import org.apache.lucene.search.*;
import org.apache.solr.search.function.*;
import org.apache.lucene.queryParser.ParseException;
+import org.apache.lucene.queryParser.QueryParser;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.Term;
import org.apache.solr.core.SolrCore;
@@ -26,6 +27,7 @@
import org.apache.solr.schema.IndexSchema;
import org.apache.solr.schema.SchemaField;
import org.apache.solr.schema.FieldType;
+import org.apache.solr.request.SolrParams;
import java.util.ArrayList;
import java.util.regex.Pattern;
@@ -37,6 +39,7 @@
  * @version $Id$
  */
public class QueryParsing {
+  public static final String OP = "q.op";
   public static Query parseQuery(String qs, IndexSchema schema) {
 return parseQuery(qs, null, schema);
@@ -58,8 +61,26 @@
 }
   }
+  public static Query parseQuery(String qs, String defaultField,  
SolrParams params, IndexSchema schema) {

+try {
+  String opParam = params.get(OP,  
schema.getQueryParserDefaultOperator());
+  QueryParser.Operator defaultOperator = "AND".equals(opParam) ?  
QueryParser.Operator.AND : QueryParser.Operator.OR;
+  SolrQueryParser parser = new SolrQueryParser(schema,  
defaultField);

+  parser.setDefaultOperator(defaultOperator);
+  Query query = parser.parse(qs);
+  if (SolrCore.log.isLoggable(Level.FINEST)) {
+SolrCore.log.finest("After QueryParser:" + query);
+  }
+  return query;
+
+} catch (ParseException e) {
+  SolrCore.log(e);
+  throw new SolrException(400,"Error parsing Lucene query",e);
+}
+  }
+
   /***
* SortSpec encapsulates a Lucene Sort and a count of the number  
of documents

* to return.
Index: src/java/org/apache/solr/request/StandardRequestHandler.java
===
--- src/java/org/apache/solr/request/StandardRequestHandler.java	 
(revision 442772)
+++ src/java/org/apache/solr/request/StandardRequestHandler.java	 
(working copy)

@@ -105,7 +105

Re: Mobile phone shop + Solr

2006-09-13 Thread climbingrose

I probably need to visualise my models:

MobileInfo (1)(1...*) SellingItem

MobileInfo has many fields to describe the characteristics of a mobile phone
model (color, size..). SellingItem is an "instance" of MobileInfo that is
currently sold by a user. So in the ERD terms, SellingItem will probably
have foreign key call MobileInfoId that references the primary key of
MobileInfo. Now obviously, I need to index MobileInfo to support faceted
browsing. How should I index SellingItem? The simplest way probably is to
combile mobile phone specs in MobileInfo and and fields in SellingItem, and
then index all of them. In this case, if I have 1000 SellingItems
referencing a particular MobileInfo, I have to repeat the fields in
MobileInfo a thousand times.

On 9/13/06, Chris Hostetter <[EMAIL PROTECTED]> wrote:



: Because the mobile phone info has many fields (>40), I don't want to
: repeatedly submit it to Solr.

i'm not really sure what you mean by "repeatedly submit to Solr" or how it
relates to haveing more then 40 fields.  40 fields really isn't that many.

To give you a basis of comparison: the last Solr index i built from
scratch had 47  declarations, and 4  declarations
...those 4 dynamic fields result in approximately 1200 'fields' in the
index -- not every document has a value for every field, but the average
is above 200 fields per document.



-Hoss





--
Regards,

Cuong Hoang


Re: Got it working! And some questions

2006-09-13 Thread Chris Hostetter

: SolrQueryParser now knows nothing about the default operator, it is
: set from QueryParsing.parseQuery() when passed a SolrParams.

i didn't test it, but it looks clean to me.

the only other thing i would do is beaf up the javadocs for
SolrQueryParser (to clarify that IndexSchema is only used for determining
field format) and QueryParsing.parseQuery (to clarify that it *does* use
IndexSearcher to get extra parsing options).

: QueryParsing.parseQuery() methods could be simplified, perhaps even
...
: It could even get the "q" parameter from there, but there is code
: that passes expressions that don't come from "q".  Maybe we could

...yeha, it's utility for simple queries regardless of the "primary"
language of a request handler is key.

: have two parseQuery() methods:  parseQuery(String expression,
: SolrQueryRequest req) and parseQuery(SolrQueryRequest req), and for
: the latter the "q" parameter is pulled from the request and used as
: the expression.

That sounds good to me ... but it doesn't seem critical ... clean house as
much as you want, but i don't think anybody else will mind a bit of dust
on the window sills.



-Hoss



Error in faceted browsing

2006-09-13 Thread Jeff Rodenburg

I just pulled down the nightly solr build from 9/12 and have it up and
running.  I copied an index created in a solr version that's about 3 months
old.

I have a query formulated like this:
http://solrbox:8080/solr/select?q=description:dell&rows=0&facet=true&facet.limit=-1&facet.field=merchant_name

The  definition from schema.xml:

  
  
  
  
  

The result:

 
 0
 2


 
 
 
java.util.NoSuchElementException
   at java.util.TreeMap.key(TreeMap.java:433)
   at java.util.TreeMap.lastKey(TreeMap.java:297)
   at java.util.TreeSet.last(TreeSet.java:417)
   at org.apache.solr.util.BoundedTreeSet.adjust(BoundedTreeSet.java:54)
   at org.apache.solr.util.BoundedTreeSet.setMaxSize(BoundedTreeSet.java
:50)
   at org.apache.solr.util.BoundedTreeSet.(BoundedTreeSet.java:31)
   at org.apache.solr.request.SimpleFacets.getFacetTermEnumCounts(
SimpleFacets.java:187)
   at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(
SimpleFacets.java:137)
   at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java
:84)
   at org.apache.solr.request.StandardRequestHandler.getFacetInfo(
StandardRequestHandler.java:180)
   at org.apache.solr.request.StandardRequestHandler.handleRequest(
StandardRequestHandler.java:120)
   at org.apache.solr.core.SolrCore.execute(SolrCore.java:586)
   at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:91)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
   at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:252)
   at org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:173)
   at org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:213)
   at org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:178)
   at org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:126)
   at org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:105)
   at org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:107)
   at org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:148)
   at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java
:869)
   at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
(Http11BaseProtocol.java:664)
   at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(
PoolTcpEndpoint.java:527)
   at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(
LeaderFollowerWorkerThread.java:80)
   at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java:684)
   at java.lang.Thread.run(Thread.java:595)





What am I missing?

-- j


Re: Error in faceted browsing

2006-09-13 Thread Chris Hostetter

: I just pulled down the nightly solr build from 9/12 and have it up and
: running.  I copied an index created in a solr version that's about 3 months
: old.

it looks like my changes to have a sensible default (which is when
facet.limit=-1 became legal) didn't make it into solr-2006-09-12.zip, but
it is in solr-2006-09-13.zip.

with the version you are using leaving out the facet.limit should achieve
what you want ... but based on your schema, using merchant_name as a facet
field may not work like you expect -- you'll probably want an exact String
version of the merchant_name field (or just use merchant_id and lookup the
name in a handy Map)

:
: I have a query formulated like this:
: 
http://solrbox:8080/solr/select?q=description:dell&rows=0&facet=true&facet.limit=-1&facet.field=merchant_name
:
: The  definition from schema.xml:
:
:
:
:
:
:
:
: The result:
: 
:   
:   0
:   2
: 
: 
:   
:   
:   
: java.util.NoSuchElementException
: at java.util.TreeMap.key(TreeMap.java:433)
: at java.util.TreeMap.lastKey(TreeMap.java:297)
: at java.util.TreeSet.last(TreeSet.java:417)
: at org.apache.solr.util.BoundedTreeSet.adjust(BoundedTreeSet.java:54)
: at org.apache.solr.util.BoundedTreeSet.setMaxSize(BoundedTreeSet.java
: :50)
: at org.apache.solr.util.BoundedTreeSet.(BoundedTreeSet.java:31)
: at org.apache.solr.request.SimpleFacets.getFacetTermEnumCounts(
: SimpleFacets.java:187)
: at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(
: SimpleFacets.java:137)
: at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java
: :84)
: at org.apache.solr.request.StandardRequestHandler.getFacetInfo(
: StandardRequestHandler.java:180)
: at org.apache.solr.request.StandardRequestHandler.handleRequest(
: StandardRequestHandler.java:120)
: at org.apache.solr.core.SolrCore.execute(SolrCore.java:586)
: at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:91)
: at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
: at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
: ApplicationFilterChain.java:252)
: at org.apache.catalina.core.ApplicationFilterChain.doFilter(
: ApplicationFilterChain.java:173)
: at org.apache.catalina.core.StandardWrapperValve.invoke(
: StandardWrapperValve.java:213)
: at org.apache.catalina.core.StandardContextValve.invoke(
: StandardContextValve.java:178)
: at org.apache.catalina.core.StandardHostValve.invoke(
: StandardHostValve.java:126)
: at org.apache.catalina.valves.ErrorReportValve.invoke(
: ErrorReportValve.java:105)
: at org.apache.catalina.core.StandardEngineValve.invoke(
: StandardEngineValve.java:107)
: at org.apache.catalina.connector.CoyoteAdapter.service(
: CoyoteAdapter.java:148)
: at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java
: :869)
: at
: 
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
: (Http11BaseProtocol.java:664)
: at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(
: PoolTcpEndpoint.java:527)
: at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(
: LeaderFollowerWorkerThread.java:80)
: at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
: ThreadPool.java:684)
: at java.lang.Thread.run(Thread.java:595)
: 
: 
: 
:
:
: What am I missing?
:
: -- j
:



-Hoss



Re: Error in faceted browsing

2006-09-13 Thread Jeff Rodenburg

Thanks Chris.

I bumped the facet.limit to 10 and it works like a charm.

Thanks for the heads up on the merchant_name.  I would probably just keep a
dictionary in memory, but if I wanted to pull the stored merchant_name back,
how would/can I do that?

thanks,
j

On 9/13/06, Chris Hostetter <[EMAIL PROTECTED]> wrote:



: I just pulled down the nightly solr build from 9/12 and have it up and
: running.  I copied an index created in a solr version that's about 3
months
: old.

it looks like my changes to have a sensible default (which is when
facet.limit=-1 became legal) didn't make it into solr-2006-09-12.zip, but
it is in solr-2006-09-13.zip.

with the version you are using leaving out the facet.limit should achieve
what you want ... but based on your schema, using merchant_name as a facet
field may not work like you expect -- you'll probably want an exact String
version of the merchant_name field (or just use merchant_id and lookup the
name in a handy Map)

:
: I have a query formulated like this:
:
http://solrbox:8080/solr/select?q=description:dell&rows=0&facet=true&facet.limit=-1&facet.field=merchant_name
:
: The  definition from schema.xml:
:
:
:
:
:
:
:
: The result:
: 
:   
:   0
:   2
: 
: 
:   
:   
:   
: java.util.NoSuchElementException
: at java.util.TreeMap.key(TreeMap.java:433)
: at java.util.TreeMap.lastKey(TreeMap.java:297)
: at java.util.TreeSet.last(TreeSet.java:417)
: at org.apache.solr.util.BoundedTreeSet.adjust(BoundedTreeSet.java
:54)
: at org.apache.solr.util.BoundedTreeSet.setMaxSize(
BoundedTreeSet.java
: :50)
: at org.apache.solr.util.BoundedTreeSet.(BoundedTreeSet.java
:31)
: at org.apache.solr.request.SimpleFacets.getFacetTermEnumCounts(
: SimpleFacets.java:187)
: at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(
: SimpleFacets.java:137)
: at org.apache.solr.request.SimpleFacets.getFacetCounts(
SimpleFacets.java
: :84)
: at org.apache.solr.request.StandardRequestHandler.getFacetInfo(
: StandardRequestHandler.java:180)
: at org.apache.solr.request.StandardRequestHandler.handleRequest(
: StandardRequestHandler.java:120)
: at org.apache.solr.core.SolrCore.execute(SolrCore.java:586)
: at org.apache.solr.servlet.SolrServlet.doGet(SolrServlet.java:91)
: at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
: at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
: at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
: ApplicationFilterChain.java:252)
: at org.apache.catalina.core.ApplicationFilterChain.doFilter(
: ApplicationFilterChain.java:173)
: at org.apache.catalina.core.StandardWrapperValve.invoke(
: StandardWrapperValve.java:213)
: at org.apache.catalina.core.StandardContextValve.invoke(
: StandardContextValve.java:178)
: at org.apache.catalina.core.StandardHostValve.invoke(
: StandardHostValve.java:126)
: at org.apache.catalina.valves.ErrorReportValve.invoke(
: ErrorReportValve.java:105)
: at org.apache.catalina.core.StandardEngineValve.invoke(
: StandardEngineValve.java:107)
: at org.apache.catalina.connector.CoyoteAdapter.service(
: CoyoteAdapter.java:148)
: at org.apache.coyote.http11.Http11Processor.process(
Http11Processor.java
: :869)
: at
:
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
: (Http11BaseProtocol.java:664)
: at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(
: PoolTcpEndpoint.java:527)
: at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(
: LeaderFollowerWorkerThread.java:80)
: at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
: ThreadPool.java:684)
: at java.lang.Thread.run(Thread.java:595)
: 
: 
: 
:
:
: What am I missing?
:
: -- j
:



-Hoss




Re: Error in faceted browsing

2006-09-13 Thread Yonik Seeley

On 9/13/06, Jeff Rodenburg <[EMAIL PROTECTED]> wrote:

Thanks for the heads up on the merchant_name.  I would probably just keep a
dictionary in memory, but if I wanted to pull the stored merchant_name back,
how would/can I do that?


If you don't want merchant_name tokenized at all, just change the type
to string.
If you want an additional field for faceting on with merchant_name
untokenized, then use copyField in schema.xml to copy merchant_name to
merchant_name_exact
and define
 

-Yonik


Re: Error in faceted browsing

2006-09-13 Thread Jeff Rodenburg

Outstanding, thanks.

- j

On 9/13/06, Yonik Seeley <[EMAIL PROTECTED]> wrote:


On 9/13/06, Jeff Rodenburg <[EMAIL PROTECTED]> wrote:
> Thanks for the heads up on the merchant_name.  I would probably just
keep a
> dictionary in memory, but if I wanted to pull the stored merchant_name
back,
> how would/can I do that?

If you don't want merchant_name tokenized at all, just change the type
to string.
If you want an additional field for faceting on with merchant_name
untokenized, then use copyField in schema.xml to copy merchant_name to
merchant_name_exact
and define
  

-Yonik



Faceted Searching problems

2006-09-13 Thread Andre Basse
Hi all,
 
I just installed the nightly build to try the Faceted Searching . After some 
testing I discovered that some characters are missing in the result XML and 
that fields with "/" chars are sometimes split into two entries.
 
Example:
1 should be France
1 should be Culture/Festivals

Please find details below.
 
Original XML
=
 
Metro
 

Culture/Film
Culture/Festivals



France
Sydney

 
 
 
SOLR response for the query 
=
(http://192.168.157.128:8983/solr/select/?q=Bellucci&rows=0&facet=true&facet.limit=5&facet.field=section&facet.field=geoloc&facet.field=classification)
 

−
 
0
518


−
 

−
 
−
 
2
0
0
0
0

−
 
1
1
0
0
0

−
 
1
1
1
1
1




 
 
Any help is much appreciated!
 
 
Thanks,
 
Andre
 
 
 


*
The information contained in this e-mail message and any accompanying files is 
or may be confidential.  If you are not the intended recipient, any use, 
dissemination, reliance, forwarding, printing or copying of this e-mail or any 
attached files is unauthorised. This e-mail is subject to copyright. No part of 
it should be reproduced, adapted or communicated without the written consent of 
the copyright owner. If you have received this e-mail in error, please advise 
the sender immediately by return e-mail, or telephone and delete all copies. 
Fairfax does not guarantee the accuracy or completeness of any information 
contained in this e-mail or attached files. Internet communications are not 
secure, therefore Fairfax does not accept legal responsibility for the contents 
of this message or attached files.
*



Re: Faceted Searching problems

2006-09-13 Thread Chris Hostetter

: I just installed the nightly build to try the Faceted Searching . After
: some testing I discovered that some characters are missing in the result
: XML and that fields with "/" chars are sometimes split into two entries.

I believe what you are encountering is an issue of tokenization (or
analysis) ... you didn't post your schema.xml, but i'm guessing these two
fields have a datatype that is analyzed right?  Take a look at the
followup posts in this recent thread...

http://www.nabble.com/Error-in-faceted-browsing-tf2267819.html

...i'll try to update the docs for facet.field to make this more obvious.


:
: Example:
: 1 should be France
: 1 should be Culture/Festivals
:
: Please find details below.
:
: Original XML
: =
:
: Metro
:
: 
: Culture/Film
: Culture/Festivals
: 
:
: 
: France
: Sydney
: 
:
:
:
: SOLR response for the query
: =
: 
(http://192.168.157.128:8983/solr/select/?q=Bellucci&rows=0&facet=true&facet.limit=5&facet.field=section&facet.field=geoloc&facet.field=classification)
:
: 
: −
:  
: 0
: 518
: 
: 
: −
:  
: 
: −
:  
: −
:  
: 2
: 0
: 0
: 0
: 0
: 
: −
:  
: 1
: 1
: 0
: 0
: 0
: 
: −
:  
: 1
: 1
: 1
: 1
: 1
: 
: 
: 
: 
:
:
: Any help is much appreciated!
:
:
: Thanks,
:
: Andre
:
:
:
:
:
: 
*
: The information contained in this e-mail message and any accompanying files 
is or may be confidential.  If you are not the intended recipient, any use, 
dissemination, reliance, forwarding, printing or copying of this e-mail or any 
attached files is unauthorised. This e-mail is subject to copyright. No part of 
it should be reproduced, adapted or communicated without the written consent of 
the copyright owner. If you have received this e-mail in error, please advise 
the sender immediately by return e-mail, or telephone and delete all copies. 
Fairfax does not guarantee the accuracy or completeness of any information 
contained in this e-mail or attached files. Internet communications are not 
secure, therefore Fairfax does not accept legal responsibility for the contents 
of this message or attached files.
: 
*
:
:



-Hoss



Re: Faceted Searching problems

2006-09-13 Thread Yonik Seeley

On 9/13/06, Andre Basse <[EMAIL PROTECTED]> wrote:

Example:
1 should be France
1 should be Culture/Festivals


Hi Andre,

Field faceting works over the indexed terms... so you get back what
was indexed (word splitting, lowercasing, stemming, etc...  the
process is not generally reversible).

Perhaps you "classification" field should be of type "string" which is
indexed by not analyzed at all.  If you need some analysis (like if
you also want a query of "Festival" to match against
"Culture/Festivals", then you should index the field again as a
non-tokenized (non analyzed) "string" type.  This can be easily done
with an extra field definition and an a copyField statement in the
schema.xml

-Yonik


Please find details below.

Original XML
=

Metro


Culture/Film
Culture/Festivals



France
Sydney




SOLR response for the query
=
(http://192.168.157.128:8983/solr/select/?q=Bellucci&rows=0&facet=true&facet.limit=5&facet.field=section&facet.field=geoloc&facet.field=classification)


−
 
0
518


−
 

−
 
−
 
2
0
0
0
0

−
 
1
1
0
0
0

−
 
1
1
1
1
1






Any help is much appreciated!


Thanks,

Andre


RE: Faceted Searching problems

2006-09-13 Thread Andre Basse
Sorry, please ignore that email. Problem solved (I should read more mails...)

Thanks to Jeff.










Hi all,
 
I just installed the nightly build to try the Faceted Searching . After some 
testing I discovered that some characters are missing in the result XML and 
that fields with "/" chars are sometimes split into two entries.
 
Example:
1 should be France 1 
should be Culture/Festivals

Please find details below.
 
Original XML
=
 
Metro
 

Culture/Film
Culture/Festivals



France
Sydney

 
 
 
SOLR response for the query
=
(http://192.168.157.128:8983/solr/select/?q=Bellucci&rows=0&facet=true&facet.limit=5&facet.field=section&facet.field=geoloc&facet.field=classification)
 

−
 
0
518


−
 

−
 
−
 
2
0
0
0
0

−
 
1
1
0
0
0

−
 
1
1
1
1
1




 
 
Any help is much appreciated!
 
 
Thanks,
 
Andre
 
 
 


*
The information contained in this e-mail message and any accompanying files is 
or may be confidential.  If you are not the intended recipient, any use, 
dissemination, reliance, forwarding, printing or copying of this e-mail or any 
attached files is unauthorised. This e-mail is subject to copyright. No part of 
it should be reproduced, adapted or communicated without the written consent of 
the copyright owner. If you have received this e-mail in error, please advise 
the sender immediately by return e-mail, or telephone and delete all copies. 
Fairfax does not guarantee the accuracy or completeness of any information 
contained in this e-mail or attached files. Internet communications are not 
secure, therefore Fairfax does not accept legal responsibility for the contents 
of this message or attached files.
*



Re: Faceted Searching problems

2006-09-13 Thread Erik Hatcher
You need to use an untokenized field for facets.  I can see we're  
going to get this question frequently now - it was mentioned earlier  
today in fact.  You can use a  that is untokenized such  
that you can use one field for searching, and one for facets.


You are obviously using a stemming analyzer, and that is why France  
became franc, etc - just to explain why you are seeing those terms  
listed.


Erik


On Sep 13, 2006, at 9:19 PM, Andre Basse wrote:


Hi all,

I just installed the nightly build to try the Faceted Searching .  
After some testing I discovered that some characters are missing in  
the result XML and that fields with "/" chars are sometimes split  
into two entries.


Example:
1 should be France
1 should be Culture/Festivals

Please find details below.

Original XML
=

Metro


Culture/Film
Culture/Festivals



France
Sydney




SOLR response for the query
=
(http://192.168.157.128:8983/solr/select/? 
q=Bellucci&rows=0&facet=true&facet.limit=5&facet.field=section&facet.f 
ield=geoloc&facet.field=classification)



−
 
0
518


−
 

−
 
−
 
2
0
0
0
0

−
 
1
1
0
0
0

−
 
1
1
1
1
1






Any help is much appreciated!


Thanks,

Andre





** 
***
The information contained in this e-mail message and any  
accompanying files is or may be confidential.  If you are not the  
intended recipient, any use, dissemination, reliance, forwarding,  
printing or copying of this e-mail or any attached files is  
unauthorised. This e-mail is subject to copyright. No part of it  
should be reproduced, adapted or communicated without the written  
consent of the copyright owner. If you have received this e-mail in  
error, please advise the sender immediately by return e-mail, or  
telephone and delete all copies. Fairfax does not guarantee the  
accuracy or completeness of any information contained in this e- 
mail or attached files. Internet communications are not secure,  
therefore Fairfax does not accept legal responsibility for the  
contents of this message or attached files.
** 
***






Re: Faceted Searching problems

2006-09-13 Thread Erik Hatcher


On Sep 13, 2006, at 9:37 PM, Chris Hostetter wrote:

http://www.nabble.com/Error-in-faceted-browsing-tf2267819.html

...i'll try to update the docs for facet.field to make this more  
obvious.


Would it ever make sense to generate facets on a tokenized field?   
Maybe the facet implementation could throw an error if the field name  
specified is tokenized?


Erik



Re: Faceted Searching problems

2006-09-13 Thread Yonik Seeley

On 9/13/06, Erik Hatcher <[EMAIL PROTECTED]> wrote:

Would it ever make sense to generate facets on a tokenized field?
Maybe the facet implementation could throw an error if the field name
specified is tokenized?


I think it probably can make sense...
- finding top terms in a full-text field that match a query could be useful
- the analysis could just be for normalization - trimming whitespace
or normalization
- it allows more flexibility on how to represent tags... one may
already have tags in a whitespace delimited field rather than separate
values in a multi-valued field.

-Yonik


Re: Faceted Searching problems

2006-09-13 Thread Yonik Seeley

On 9/13/06, Erik Hatcher <[EMAIL PROTECTED]> wrote:

You need to use an untokenized field for facets.


At least 3 answers in 5 minutes... we should try synchronized swimming ;-)

-Yonik


Re: MoreLikeThis class in Lucene within Solr?

2006-09-13 Thread Michael Imbeault
Thanks for the answer; and try to enjoy your vacation / travel! Can't 
wait to be able to interface with MoreLikeThis within Solr!


Michael Imbeault
CHUL Research Center (CHUQ)
2705 boul. Laurier
Ste-Foy, QC, Canada, G1V 4G2
Tel: (418) 654-2705, Fax: (418) 654-2212



Erik Hatcher wrote:


On Sep 12, 2006, at 3:41 PM, Michael Imbeault wrote:
I haven't looked at the specifics of how MoreLikeThis determine which 
items are similar; I'm mainly wondering about performance here. 
Yesterday I tried to code myself a poor man's similarity class (which 
was nothing more than doing a search with OR between words and 
sorting by score), and the performance was abysmal (well, I kinda 
expected it. 1000+ words queries on a 15 millions docs collection, 
you don't expect miracles). At first glance I think it searches for 
the most 'relevant' words, I'm I right? What kind of performance are 
you getting with it?


Performance with MoreLikeThis is not an issue.  It has many parameters 
to tune how many terms are used in the query it builds, and it pulls 
these terms in an extremely efficient manner from the Lucene index.


I'm doing some traveling soon, which is always a good time to hack on 
something tractable like adding MoreLikeThis to Solr.  So your wish 
may be granted in a week :)


Erik



duplicating all records added to index

2006-09-13 Thread Tim Archambault

My index seems to be duplicating all records on insert even though I have my
add statements set to not allow duplicates.

I've provided a samle xml file of add docs. Anyone experienced this?





  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16140

  20001204
  
  Obituaries
  BDN Classifieds
  BURGESS, Fidalis 'Dale' J., 82
  HERMON - Fidalis "Dale" J. Burgess, 82,
husband of the late Lottie (Glidden) Burgess, passed away unexpectedly Dec.
1, 2000, at his residence. He was born Jan. 21, 1918, in Bangor, the son of
Elias and Margaret (Cheverie) Burgess.  Dale lived in Hermon   
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16141

  20001204
  
  Obituaries
  BDN Classifieds
  CLARKE, Paul H., 79
  BANGOR- Paul H. Clarke, 79, died Dec. 1, 2000, at
his residence. He was born Jan. 14, 1921, in Saco the son of Charles and
Jennie (Larson) Clarke.  Paul was a life long member of Elks Lodge 244,
Bangor.  He worked most of his life as a meat cutter for
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16142

  20001204
  
  Obituaries
  BDN Classifieds
  COMEAU, Janice Mary, 63
  WALTHAM - Janice Mary Comeau, 63, died Dec. 1,
2000, at her home in Waltham. She was born Oct. 22, 1937, in Hartford,
Conn., the daughter of Joseph Edmund and Helen (LeBel) Comeau.  Janice
served her country in the U.S. Army as a nurse. She graduated 
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16143

  20001204
  
  Obituaries
  BDN Classifieds
  CONNERS, Lois Marie
  HERMON AND BANGOR - Funeral services for Lois Marie
Conners will be held 9:30 a.m. Monday at Brookings-Smith, 133 Center St.,
Bangor with the Rev. Robert T. Carlson, pastor of the East Orrington
Congregational Church, officiating.  Interment will be in   
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16144

  20001204
  
  Obituaries
  BDN Classifieds
  CORBETT, Linda L., 53
  CARIBOU - Linda L. Corbett, 53, wife of Nathan
Corbett, died Dec. 1, 2000, at Bangor. She was born at Caribou, March 7,
1947, the daughter of Jerry and Luella (Clark) Hewitt.  She was a graduate
of the Caribou High School and was a loving and devoted 
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16145

  20001204
  
  Obituaries
  BDN Classifieds
  EDWARDS, James E.
  BANGOR - Mr. James E. Edwards died Dec. 2, 2000, at
his residence after a long illness. He was born in Hartford, Conn., May 27,
1929, the son of James V. and Cecelia (Fury) Edwards.  James served in the
U.S. Navy in Guam attaining the rank of fireman. He 
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16146

  20001204
  
  Obituaries
  BDN Classifieds
  FOLSOM, Robert E., 64
  MILLINOCKET - Robert E. Folsom, 64, died at a local
hospital, Dec. 1, 2000, after a brief illness. He was born in Millinocket,
the son of Lee and Ada (Hall) Folsom.  Bob retired from Great Northern Paper
Co. after many years. He was a member of BPO Elks  
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16147

  20001204
  
  Obituaries
  BDN Classifieds
  FORTIN, Ludger Joseph, 62
  CARIBOU - Ludger J. Fortin, 62, died Dec. 2, 2000,
at Van Buren, after a long illness. He was born at St. Anne, N.B., Canada,
Sept. 28, 1938, the son of Edmond and Azilda (Burby) Fortin.  In 1960, he
married Shirley (Stubbs) Fortin. Ludger was employed   
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16148

  20001204
  
  Obituaries
  BDN Classifieds
  FOX, Gertrude G.
  ORRINGTON - A Mass of Christian Burial for Gertrude
G. Fox will be celebrated 10 a.m. Monday at St. Teresa Catholic Church,
Brewer with the Rev. Joel R. Cyr, pastor, celebrant.  Burial will be 12 noon
Tuesday at the Maine Veterans Memorial Cemetery,  
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16149

  20001204
  
  Obituaries
  BDN Classifieds
  GRENDELL, Edith, 83
  MAPLETON - Edith Grendell, 83, wife of the late
Ashley A. Grendell, died Dec. 1, 2000, at a Presque Isle health-care
facility. She was born in Somerville, Mass., Jan. 9, 1917, the daughter of
Albert B. and Ella (Morrow) MacLaughlin.  She was a member of  
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16150

  20001204
  
  Obituaries
  BDN Classifieds
  HOWARD, Theron G., 53
  BUCKSPORT - On Sunday, Dec. 3, 2000, Theron G.
Howard, 53, rode the spirit horse home to once again be with his mother and
father, Phyllis (Robertson) and Urban Coombs, his brother, Steven Howard;
and father-in-law, Francios Desaulniers.  Theron was born 
  




  obituaries_
  
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16151

  20001204
  
  Obituaries
  BDN Classifieds
  H

Re: duplicating all records added to index

2006-09-13 Thread Tim Archambault

In the example I sent the "id" field is not unique, but I've long since
corrected that and still getting duplication. FYI

On 9/14/06, Tim Archambault <[EMAIL PROTECTED]> wrote:


 My index seems to be duplicating all records on insert even though I have
my add statements set to not allow duplicates.

I've provided a samle xml file of add docs. Anyone experienced this?




 
   obituaries_
   
http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16140

   20001204
   
   Obituaries
   BDN Classifieds
   BURGESS, Fidalis 'Dale' J., 82

   HERMON - Fidalis "Dale" J. Burgess, 82,
husband of the late Lottie (Glidden) Burgess, passed away unexpectedly Dec.
1, 2000, at his residence. He was born Jan. 21, 1918, in Bangor, the son of
Elias and Margaret (Cheverie) Burgess.  Dale lived in Hermon   
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16141

   20001204
   
   Obituaries
   BDN Classifieds
   CLARKE, Paul H., 79
   BANGOR- Paul H. Clarke, 79, died Dec. 1, 2000, at
his residence. He was born Jan. 14, 1921, in Saco the son of Charles and
Jennie (Larson) Clarke.  Paul was a life long member of Elks Lodge 244,
Bangor.  He worked most of his life as a meat cutter for
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16142

   20001204
   
   Obituaries
   BDN Classifieds
   COMEAU, Janice Mary, 63
   WALTHAM - Janice Mary Comeau, 63, died Dec. 1,
2000, at her home in Waltham. She was born Oct. 22, 1937, in Hartford,
Conn., the daughter of Joseph Edmund and Helen (LeBel) Comeau.  Janice
served her country in the U.S. Army as a nurse. She graduated 
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16143

   20001204
   
   Obituaries
   BDN Classifieds
   CONNERS, Lois Marie
   HERMON AND BANGOR - Funeral services for Lois
Marie Conners will be held 9:30 a.m. Monday at Brookings-Smith, 133 Center
St., Bangor with the Rev. Robert T. Carlson, pastor of the East Orrington
Congregational Church, officiating.  Interment will be in   
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16144

   20001204
   
   Obituaries
   BDN Classifieds
   CORBETT, Linda L., 53
   CARIBOU - Linda L. Corbett, 53, wife of Nathan
Corbett, died Dec. 1, 2000, at Bangor. She was born at Caribou, March 7,
1947, the daughter of Jerry and Luella (Clark) Hewitt.  She was a graduate
of the Caribou High School and was a loving and devoted 
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16145

   20001204
   
   Obituaries
   BDN Classifieds
   EDWARDS, James E.
   BANGOR - Mr. James E. Edwards died Dec. 2, 2000,
at his residence after a long illness. He was born in Hartford, Conn., May
27, 1929, the son of James V. and Cecelia (Fury) Edwards.  James served in
the U.S. Navy in Guam attaining the rank of fireman. He 
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16146

   20001204
   
   Obituaries
   BDN Classifieds
   FOLSOM, Robert E., 64
   MILLINOCKET - Robert E. Folsom, 64, died at a
local hospital, Dec. 1, 2000, after a brief illness. He was born in
Millinocket, the son of Lee and Ada (Hall) Folsom.  Bob retired from Great
Northern Paper Co. after many years. He was a member of BPO Elks  
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16147

   20001204
   
   Obituaries
   BDN Classifieds
   FORTIN, Ludger Joseph, 62
   CARIBOU - Ludger J. Fortin, 62, died Dec. 2,
2000, at Van Buren, after a long illness. He was born at St. Anne, N.B.,
Canada, Sept. 28, 1938, the son of Edmond and Azilda (Burby) Fortin.  In
1960, he married Shirley (Stubbs) Fortin. Ludger was employed   
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16148

   20001204
   
   Obituaries
   BDN Classifieds
   FOX, Gertrude G.
   ORRINGTON - A Mass of Christian Burial for
Gertrude G. Fox will be celebrated 10 a.m. Monday at St. Teresa Catholic
Church, Brewer with the Rev. Joel R. Cyr, pastor, celebrant.  Burial will be
12 noon Tuesday at the Maine Veterans Memorial Cemetery,  
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16149

   20001204
   
   Obituaries
   BDN Classifieds
   GRENDELL, Edith, 83
   MAPLETON - Edith Grendell, 83, wife of the late
Ashley A. Grendell, died Dec. 1, 2000, at a Presque Isle health-care
facility. She was born in Somerville, Mass., Jan. 9, 1917, the daughter of
Albert B. and Ella (Morrow) MacLaughlin.  She was a member of  
   
 


 
   obituaries_
   http://www.bangordailynews.com/a/class/obituaries/obituary.cfm?id=16150

   20001204
   
   Obituaries
   BDN Classifieds
   HOWARD, Theron G., 53
   BUCKSPORT - On Sunday, Dec. 3, 2000, Theron G.
Howard, 53, rode the spirit horse home to once again be with hi