Re: Extending Solr Highlighter to pull information from external source

I haven't seen any interest in this, but for anyone following, I updated the alternateField logic to support pulling from the external field if available. Would be useful to know how to get solr to use this external field provider in general so we wouldn't have to modify the highlighter at all, ju

Re: Extending Solr Highlighter to pull information from external source

I tried the patch at SOLR-1397 but it didn't work as I'd expect. Test subject message 0 29 The start position is right, but the end position seems to be the length of the field. On Fri, Jul 15, 2011 at 4:25 PM, Jamie Johnson wrote: > I a

Re: Extending Solr Highlighter to pull information from external source

I added the highlighting code I am using to this JIRA (https://issues.apache.org/jira/browse/SOLR-1397). Afterwards I noticed this JIRA (https://issues.apache.org/jira/browse/SOLR-1954) which talks about another solution. I think David's patch would have worked equally well for my problem, just w

Re: Extending Solr Highlighter to pull information from external source

Boy it's been a long time since I first wrote this, sorry for the delay I think I have this working as I expect with a test implementation. I created the following interface public interface SolrExternalFieldProvider extends NamedListInitializedPlugin { public String[] getFieldConten

Re: Extending Solr Highlighter to pull information from external source

I found https://issues.apache.org/jira/browse/SOLR-1397 but there is not much going on there LUCENE-1522 has a lot of fascinating discussion on this topic though There is a couple of long lived issues in jira for this (I'd like to try to se

Re: Extending Solr Highlighter to pull information from external source

(11/06/20 21:41), Jamie Johnson wrote: I am trying to index data where I'm concerned that storing the contents of a specific field will be a bit of a hog so we are planning to retrieve this information as needed for highlighting from an external source. I am looking to extend the default solr hi

Re: Extending Solr Highlighter to pull information from external source

Yes that sounds about right. I also have in mind an optimization for highlighting so it doesn't need to pull the whole field value. The fast vector highlighter is working with offsets into the field, and should work better w/random access into the field value(s). But that should come as a la

Re: Extending Solr Highlighter to pull information from external source

perhaps it should be an array that gets returned to be consistent with getValues(fieldName); On Mon, Jun 20, 2011 at 9:59 AM, Jamie Johnson wrote: > Yes, in that case the code becomes > > if(!schemaField.stored()){ > > > SchemaField keyField = schema.getUniqueKeyField(); >

Re: Extending Solr Highlighter to pull information from external source

Yes, in that case the code becomes if(!schemaField.stored()){ SchemaField keyField = schema.getUniqueKeyField(); String key = doc.getValues(keyField.getName())[0]; docTexts = doc.getValues(fieldName); if(key != null && key.length() > 0){

Re: Extending Solr Highlighter to pull information from external source

Another option for determining whether to go to external storage would be to examine the SchemaField, see if it is stored, and if not, try to fetch from a file or whatever. That way you won't have to configure anything. -Mike On 06/20/2011 09:46 AM, Jamie Johnson wrote: In my case chucking t

Re: Extending Solr Highlighter to pull information from external source

In my case chucking the external storage is simply not an option. I'll definitely share anything I find, the following is a very simple example of adding text to the default solr highlighter (had to copy a large portion of the class since the method that actually does the highlighting is private

Re: Extending Solr Highlighter to pull information from external source

Mike I would be very interested in the answer to that question too. My hunch is that the answer is no too. I have a few text databases that range from 200MB to about 60GB with which I could run some tests. I will have some downtime in early July and will post results. From what I can tell the

Re: Extending Solr Highlighter to pull information from external source

I'd be very interested in this, as well, if you do it before me and are willing to share... A related question I have tried to ask on this list, and have never really gotten a good answer to, is whether it makes sense to just chuck the external storage and treat the lucene index as the primary

Extending Solr Highlighter to pull information from external source

I am trying to index data where I'm concerned that storing the contents of a specific field will be a bit of a hog so we are planning to retrieve this information as needed for highlighting from an external source. I am looking to extend the default solr highlighting capability to work with inform