Re: PatternReplaceFilterFactory creating empty string as a term

2010-10-08 Thread Shawn Heisey
On 10/5/2010 10:38 PM, Shawn Heisey wrote: That fixed it. Thank you. If I have time, I'll peek at the patternfilter source code and see if I can figure out how to make it optionally remove empty terms. For me, it's not terribly critical, because my database is the bottleneck in my indexing p

Re: PatternReplaceFilterFactory creating empty string as a term

2010-10-05 Thread Shawn Heisey
On 10/5/2010 6:28 PM, Markus Jelsma wrote: I'm not sure if this is the best approach but a LengthFilter will stop blank terms. http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LengthFilterFactory Two people with the answer I needed. Thank you! Shawn

Re: PatternReplaceFilterFactory creating empty string as a term

2010-10-05 Thread Shawn Heisey
On 10/5/2010 6:34 PM, Ken Krugler wrote: Is there any existing way to remove empty terms during analysis? I tried TrimFilterFactory but that made no difference. You could use LengthFilterFactory to restrict terms to being at least one character long. Is this a bug in PatternReplaceFilter

Re: PatternReplaceFilterFactory creating empty string as a term

2010-10-05 Thread Robert Muir
CPU cycles and > some memory. > > -Original message- > From: Markus Jelsma > Sent: Wed 06-10-2010 00:29 > To: solr-user@lucene.apache.org; > Subject: RE: PatternReplaceFilterFactory creating empty string as a term > > I'm not sure if this is the best approac

RE: PatternReplaceFilterFactory creating empty string as a term

2010-10-05 Thread Markus Jelsma
and some memory.   -Original message- From: Markus Jelsma Sent: Wed 06-10-2010 00:29 To: solr-user@lucene.apache.org; Subject: RE: PatternReplaceFilterFactory creating empty string as a term I'm not sure if this is the best approach but a LengthFilter will stop blank terms. http://

Re: PatternReplaceFilterFactory creating empty string as a term

2010-10-05 Thread Ken Krugler
On Oct 5, 2010, at 6:24pm, Shawn Heisey wrote: I am developing a new schema. It has a pattern filter that trims leading and trailing punctuation from terms. It is resulting in empty terms, because there are situations in the analyzer stream where a term happens to be composed of nothing

RE: PatternReplaceFilterFactory creating empty string as a term

2010-10-05 Thread Markus Jelsma
I'm not sure if this is the best approach but a LengthFilter will stop blank terms. http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.LengthFilterFactory   -Original message- From: Shawn Heisey Sent: Wed 06-10-2010 00:25 To: solr-user@lucene.apache.org; Subject: PatternRe