Brian, This is not really a job for n-grams. It sounds like you'll want to write a custom Tokenizer that has knowledge about this particular pattern, knows how to split input like the one in your example, and produce multiple tokens out of it. For the natural language part you can probably get away with one of the existing tokenizers/analyzers/factories. For the first part you'll likely want to extract (W+)0+ -- 1 or morel etters followed by 1 or more zeros as one token, and then 0+(D+) -- 1 or more zeros followed by 1 or more digits.
Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- From: Brian Carmalt <[EMAIL PROTECTED]> To: solr-user@lucene.apache.org Sent: Tuesday, December 11, 2007 9:17:32 AM Subject: How to effectively search inside fields that should be indexed with changing them. Hello all, The titles of our docs have the form "ABC0001231-This is an important doc.pdf". I would like to be able to search for 'important', or '1231', or 'ABC000*', or 'This is an important doc' in the title field. I looked a the NGramTokenizer and tried to use it. In the index it doesn't seem to work, I cannot get any hits. The analysis tool on the admin pages shows me that the ngram tokenizing works by highlighting the matches between the indexed value and a query. I have set the min and max ngram size to 2 and 6, with side equal to left. Can anyone recommend a procedure that will allow me to search as stated above? I would also like to find out more about how to use the NgramTokenizer, but have found little in the form of documentation. Anyone know about any good sources? Thanks, Brian
