On 1/3/07, Chris Hostetter <[EMAIL PROTECTED]> wrote:
: I've written an unpolished custom fragmenter for highlighting which is : more expensive than the BasicFragmenter that ships with lucene, but : generates more natural candidate fragments (it will tend to produce : beginning/ends of sentences). : : Would there be interest in the community in releasing it and/or : including it in Solr? Mike: I don't really follow the highlighting/fragmenting buzz, but it seems like it might make sense to contribute this directly to Lucene-Java ... of course, if you want to go ahead and commit it to Solr, it can allways be "promoted" up to Lucene-Java later (like i suspect FunctioQuery will be just as soon as someone gets an itch to move it)
Yeah, I thought about that. There's a few reasons I wouldn't want to contribute it there immediately: - ease of maintenance - Highlighting is a contrib module in lucene, and there are various aspects of it that I don't really like. I see it more as a means of implementing Solr's highlighting. What I'd like to do is improve the end-user's experience with highlighting in Solr. If as a result a high-quality component for lucene Highlighter is fleshed out, that can always be contributed to Lucene later. Generally, we should strive for a high-quality out-of-the-box highlighting in Solr. That might involve making things like better fragmenters and a few other tricks(*) the default setup, and providing a "quick & dirty" setting for speed demons. (*) Doing some basic cleaning of the generated fragments works wonders. -Mike
