Hi Ed, Thanks for the help, but i have some queries, i understand that we need to have a stopwords_french.txt and protwords_french.txt files say for french in solr/conf directory. Is it like we need to write the classes like FrenchStopFilterFactory, FrenchPorterFilterFactory for each language or do we have these classes in built in solr? I didnt find them in SOLR/Lucene APIs. I found some classes like org.apache.lucene.analysis.fr.FrenchAnalyzer etc., in lucene-analyzers.jar. Any idea what is this class used for?
Thanks in advance, Regards Dilip -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Ed Summers Sent: Monday, November 12, 2007 7:00 PM To: solr-user@lucene.apache.org; [EMAIL PROTECTED] Subject: Re: I18N with SOLR? I'd say yes. Solr supports Unicode and ships with language specific analyzers, and allows you to provide your own custom analyzers if you need them. This allows you to create different <fieldType> definitions for the languages you want to support. For example here is an example field type for French text which uses a French stopword list and French stemming. <fieldType name="text_french" class="solr.TextField" > <analyzer> <tokenizer class="solr.WhitespaceTokenizerFactory" /> <filter class="solr.FrenchStopFilterFactory" ignoreCase="true" words="stopwords_french.txt" /> <filter class="solr.FrenchPorterFilterFactory" protected="protwords_french.txt" /> <filter class="solr.RemoveDuplicatesTokenFilterFactory" /> </analyzer> </fieldType> Then you can create a <dynamicField> definitions that allow you to index and query your documents using the correct field type: <dynamicField name="*_french" type="text_french" indexed="true" stored="true"/> This means that when you index you need to know what language your data is in so that you know what field names to use in your document (e.g. title_french). And at search time you need to know what language you are in so you know which fields to search. Most user interfaces are in a single language context so from the query perspective you'll most likely know the language they want to search in. If you don't know the language context in either case you could try to guess using something like org.apache.nutch.analysis.lang.LanguageIdentifier. I hope this helps. We used this technique (without the guessing) quite effectively at the Library of Congress recently for a prototype application that needed to provide search functionality in 7 different languages. //Ed On Nov 12, 2007 1:56 AM, Dilip.TS <[EMAIL PROTECTED]> wrote: > Hello, > > Does SOLR supports I18N (with multiple language support) ? > Thanks in advance. > > Regards, > Dilip TS > >
