Tokenizing and searching named character entity references

2008-07-24 Thread F Knudson
Greetings: I am working with many different data sources - some source employ "entity references" ; others do not. My goal is to make the searching across sources as consistent as possible. Example text - Source1: weakening Hδ absorption Source1: zero-field gap ω Source2: weakening H de

Letter-number transitions - can this be turned off

2007-09-30 Thread F Knudson
Is there a flag to disable the letter-number transition in the solr.WordDelimiterFilterFactory? We are indexing category codes, thesaurus codes for which this letter number transition makes no sense. It is bloating the indexing (which is already large). Thanks F Knudson -- View this message

Re: Letter-number transitions - can this be turned off

2007-10-02 Thread F Knudson
Thanks for your helpful suggestions. I have considered other analyzers but WDF has great strengths. I will experiment with maintaining transitions and then consider modifying the code. F. Knudson Mike Klaas wrote: > > On 30-Sep-07, at 12:47 PM, F Knudson wrote: > >> >&g

Optimization taking days/weeks

2008-02-28 Thread F Knudson
Optimization time on solr index has turned into days/weeks. We are using solr 1.2. We use one box to build/optimize indexes. This index is copied to another box for searching purposes. We welcome suggestions/comments, etc. We are a bit stumped on this. Details are below. Box details Proc: 8 Dual

RE: Optimization taking days/weeks

2008-02-29 Thread F Knudson
We will review the java settings. The current settings are a bit low - but the indexed typically does not reach even 50% of the allocated 1024MB Max Heap. Yes the index is large - only 3 fields are stored - and I have set the positionIncrementGap to 50 (down from 100) in an attempt to reduce ind

Re: Optimization taking days/weeks

2008-02-29 Thread F Knudson
e of IO? Is there a reason for such a small heap on a > machine with 32GB of RAM? > > Otis > > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > ----- Original Message >> From: F Knudson <[EMAIL PROTECTED]> >> To: solr-user@luc

Re: Optimization taking days/weeks

2008-02-29 Thread F Knudson
ot of CPU. > If you are using a Java6 JVM, it should have management enabled by > default and you should be able to connect to it via jconsole and > check. > > -Yonik > > On Thu, Feb 28, 2008 at 9:54 AM, F Knudson <[EMAIL PROTECTED]> wrote: >> >> Optimizati