Whole unfiltered content in response document field

[solrfan]

Hi, I have a question to the content of the document fields. My configuration
is ok so far, I index a database with DIH and have configured a index
analyser as folow:








... 

 
 
   
 

On the analysis view, my filters work poperly. On the end of the filter
chain I have only interest tokens. But when I search with Solr, I become as
a response the whole content of the indexed databse field. The field
contains stopwords, whitespaces, upercases and so on. I search for
stopwords, and I can find them. I would expect, I find in the response
document only the filtered content in the field and not the original raw
content that I would to index. 

Is this a normal behaviour? Do I understand Solr right? 

Many thanks! 

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Whole-unfiltered-content-in-response-document-field-tp2911588p2911588.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Whole unfiltered content in response document field

[solrfan]

I understand now. I become the raw content of the field because is "stored".
The filtered content is in the response not visible. I can only see this in
the analysis view. Ok now :)

I will try to move the StopFilter under the WordDelimeter.


Thanks!

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Whole-unfiltered-content-in-response-document-field-tp2911588p2918316.html
Sent from the Solr - User mailing list archive at Nabble.com.

Custom filter development

[solrfan]

Hi, I would like to write my own filter. I try to use the following class:

public class MyFilter extends TokenFilter {

private String myField

public SemanticQueryExpansionFilter(TokenStream input, myFiled) 
{
super(input);
this.myField = myField;
}

@SuppressWarnings("deprecation")
public Token next() throws IOException
{
return parseToken(this.input.next());
}
 
@SuppressWarnings("deprecation")
public Token next(Token result) throws IOException
{
return parseToken(this.input.next());
}
 
protected Token parseToken(Token input)
{
/* do magic stuff with in.termBuffer() here (a char[] which can be
manipulated) */
/* set the changed length of the new term with in.setTermLength();
before returning it */
}
}

The factory and deploying is no problem, but I have a different question.

I want to trigger my filter at the last position after I have a clear set of
Tokens. This I can configure in my analyser XML-configuration.

My object from type MyFilter becomes in the constructor a input TokenStream.
I assume that this is a "list" of Tokens. The methods "next" use the
"parseToken" method. This is ok, the next Token from the input will be get
an a modified Token will be returned.

But this is a problem for me. The one-to-one mapping. I want to map a given
Token, for example "a" to three Tokens "a1", "a2", "a3". I want to do a
one-to-one mapping to "b" -> "c" too, and I want to have the possibility to
remove a Token "d" -> "".

How can I do this, when the "next" methods returns only one Token, not a
collection?


Thanks!

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Custom-filter-development-tp2918459p2918459.html
Sent from the Solr - User mailing list archive at Nabble.com.