Re: Field Collapsing SOLR-236

2010-06-16 Thread Eric Caron 
I've had the best luck checking out the newest Solr/Lucene (so the 1.5-line)
from SVN, then just doing "patch -p0 < SOLR-236-trunk.patch" from inside the
trunk directory. I just did it against the newest checkout and it works fine
still.

On Wed, Jun 16, 2010 at 11:35 AM, Moazzam Khan  wrote:

> Actually I take that back. I am just as lost as you. I wish there was
> a tutorial on how to do this (although I get the feeling that once I
> know how to do it I will go "ohh... I can't believe I couldn't figure
> that out")
>
> - Moazzam
>
> On Wed, Jun 16, 2010 at 8:25 AM, Moazzam Khan  wrote:
> > Hi Rakhi,
> >
> > You are supposed to get the code for solr 1.4 from SVN here:
> > http:/svn.apache.org/repos/asf/lucene/solr/tags/
> >
> > Then apply the path to it and comppile. It should work.
> >
> >
> > However, you will probably get an error at run time saying some java
> > class is missing. I haven't been able to figure out what to do after
> > that.
> >
> >
> >
> > - moazzam
> > http://moazzam-khan.com
> >
> > On Wed, Jun 16, 2010 at 3:37 AM, Rakhi Khatwani 
> wrote:
> >> Hi,
> >> I wanted to try out field collapsing for a requirement. i went
> through
> >> the wiki and solr-236. but there are lot of patch files. and the
> comments
> >> below left me confused.
> >>
> >> i tried applyin the patch file on 1.4.0 release but ended up with many
> >> compile errors.
> >> i even downloaded the latest code from the repository and applied the
> >> patch(solr-trunk-236 dtd 16th May 2010). but ended up with build errors.
> >>
> >> Can someone tell me which patch file to apply on which build? so that i
> can
> >> get collapsing working?
> >>
> >> Regards,
> >> Raakhi.
> >>
> >> On Thu, Mar 25, 2010 at 11:15 PM, Rob Z  wrote:
> >>
> >>>
> >>> What do you mean you had to revert to Trunk 1.5. Do you mean upgrade?
> Which
> >>> version were you using before hand?
> >>>
> >>> Can you please list the exact version of 1.5 and the patch # you used.
> I
> >>> downloaded the latest nightly build and tried patching using the 2/1
> patch.
> >>> Everything went ok but I am getting 1 failing test.
> >>>
> >>> Would you recommend using the latest nightly 1.5 build or 1.4 for
> >>> production use? I really need this feature so I don't think I have much
> of a
> >>> choice. Can you also explain the performance implications you are
> seeing AND
> >>> what configuration tweaks you've used that helped.
> >>>
> >>> Thanks!
> >>>
> >>> > From: mark.robe...@red-gate.com
> >>> > To: solr-user@lucene.apache.org
> >>> > Date: Thu, 25 Mar 2010 15:21:54 +
> >>> > Subject: RE: Field Collapsing SOLR-236
> >>> >
> >>> > Yeah got it working fine - but I needed to revert to Trunk (1.5) to
> get
> >>> the patch to apply.
> >>> >
> >>> > It does certainly have some performance implications, but tweaking
> >>> configuration can help here.
> >>> >
> >>> > Overall the benefits very much outweigh the costs for us :)
> >>> >
> >>> > Mark.
> >>> >
> >>> >
> >>> > -Original Message-
> >>> > From: Dennis Gearon [mailto:gear...@sbcglobal.net]
> >>> > Sent: 25 March 2010 00:49
> >>> > To: solr-user@lucene.apache.org
> >>> > Subject: Re: Field Collapsing SOLR-236
> >>> >
> >>> > Boy, I hope that field collapsing works! I'm planning on using it
> >>> heavily.
> >>> > Dennis Gearon
> >>> >
> >>> > Signature Warning
> >>> > 
> >>> > EARTH has a Right To Life,
> >>> >   otherwise we all die.
> >>> >
> >>> > Read 'Hot, Flat, and Crowded'
> >>> > Laugh at http://www.yert.com/film.php
> >>> >
> >>> >
> >>> > --- On Wed, 3/24/10, blargy  wrote:
> >>> >
> >>> > > From: blargy 
> >>> > > Subject: Field Collapsing SOLR-236
> >>> > > To: solr-user@lucene.apache.org
> >>> > > Date: Wednesday, March 24, 2010, 12:17 PM
> >>> > >
> >>> > > Has anyone had any luck with the field collapsing patch
> >>> > > (SOLR-236) with Solr
> >>> > > 1.4? I tried patching my version of 1.4 with no such luck.
> >>> > >
> >>> > > Thanks
> >>> > > --
> >>> > > View this message in context:
> >>>
> http://old.nabble.com/Field-Collapsing-SOLR-236-tp28019949p28019949.html
> >>> > > Sent from the Solr - User mailing list archive at
> >>> > > Nabble.com.
> >>> > >
> >>> > >
> >>>
> >>> _
> >>> Hotmail: Trusted email with Microsoft’s powerful SPAM protection.
> >>> http://clk.atdmt.com/GBL/go/210850552/direct/01/
> >>>
> >>
> >
>

Using WhitespaceTokenizer but still wanting to match when all fields are concatenated

2010-11-22 Thread Eric Caron 
Problem:
Indexed phrase: JetBlue Airlines
Ideal matching queries: jetblue, "jet blue" "jetblue airway", "jetblue
company"

I'd like to be able to use synonyms (to convert airway to airline),
stopwords (to drop "company"), strip periods and use ASCII folding, and
split on case.

I'm close with the following:
***







***
Except the problem that I can't do synonyms or stopwords because of the
non-tokenizing tokenizer. There's also the problem that a wildcard at the
end of the exact-match returns nothing.

Does anyone have suggestions on how this could be accomplished? The dataset
is under 100k entries and none of the docs are more than 200 characters.

Strange NPE with SOLR-236 (Field collapsing)

2010-05-10 Thread Eric Caron 
Using the latest from trunk as of 2010-04-29, and the
SOLR-236-trunk.patch
from 2010-03-29 05:08, I get a nullpointerexception whenever I use
collapse.field and a fq.

Works: /solr/select/?q=sales&fq=country%3A1
Works: /solr/select/?q=sales&collapse.field=company
Doesn't work: /solr/select/?q=sales&collapse.field=company&fq=country%3A1

The top of the trace is:
java.lang.NullPointerException
at
org.apache.solr.search.fieldcollapse.NonAdjacentDocumentCollapser$FloatValueFieldComparator.compare(NonAdjacentDocumentCollapser.java:450)
at
org.apache.solr.search.fieldcollapse.NonAdjacentDocumentCollapser$DocumentComparator.compare(NonAdjacentDocumentCollapser.java:262)
at
org.apache.solr.search.fieldcollapse.NonAdjacentDocumentCollapser$DocumentPriorityQueue.lessThan(NonAdjacentDocumentCollapser.java:196)
at
org.apache.lucene.util.PriorityQueue.insertWithOverflow(PriorityQueue.java:148)
at
org.apache.solr.search.fieldcollapse.NonAdjacentDocumentCollapser.doCollapsing(NonAdjacentDocumentCollapser.java:113)
at
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.executeCollapse(AbstractDocumentCollapser.java:259)
at
org.apache.solr.search.fieldcollapse.AbstractDocumentCollapser.collapse(AbstractDocumentCollapser.java:179)
at
org.apache.solr.handler.component.CollapseComponent.doProcess(CollapseComponent.java:173)
at
org.apache.solr.handler.component.CollapseComponent.process(CollapseComponent.java:127)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)

What's oddest is that the search works on the first load, and on a F5 in the
browser, but a hard-refresh (control-F5) forces it to crash. The only
problems are with FieldCollapse, everything else works great. I'd love to
help more with debugging, but I'm not Java-savvy enough (yet).

Appreciative of any help available,
@ecaron

(This is also posted on
https://issues.apache.org/jira/browse/SOLR-236#comment_12862327_header_ecaron
)