Re: Language detection for multivalued field

2014-09-09 Thread lsanchez
Hi all, I don't know if this can help somebody, I've changed the method process of the class LanguageIdentifierUpdateProcessor in order to support of multivalued fields and it works pretty well protected SolrInputDocument process(SolrInputDocument doc) { String docLang = null; HashSet doc

Re: Language detection for multivalued field

2013-10-30 Thread vatuska
And if I use dynamic fields to split multivalued field on different fields, can I use this dynamic field in *updateRequestProcessorChain* ? I've tried this, but seems dynamic values doesn't supported in langid.map.individual.fl -- View this message in context: http://lucene.472066.n3.nabble.c

Re: Language detection for multivalued field

2013-10-30 Thread Jan Høydahl
Hi, First, the feature will only detect ONE language per field, even if it is a multi-valued field. In your case there is VERY little text for the detector, so do not expect great detection quality. But I believe the detector chose ES as language and mapped the whole field as tag_es. The reason

Re: Language detection for multivalued field

2013-10-22 Thread vatuska
*Can you elaborate on your comment "There isn't tag indexed". Are you saying that your multiValued "tag" field is not indexed at all, gone, missing? * There aren't any tag_... field despite of indexed=true stored=true for dynamicField I found the reason, but I don't understand why If I specify en

Re: Language detection for multivalued field

2013-10-22 Thread Jan Høydahl
Hi, The feature is designed to detect exactly one language per field. In case of multValued, it will concatenate all values before detection. Can you elaborate on your comment "There isn't tag indexed". Are you saying that your multiValued "tag" field is not indexed at all, gone, missing? If yo