Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-09-01 Thread Jamie Johnson
Tracking not teaching... Auto complete is fun... On Tue, Sep 1, 2015, 6:34 AM Jamie Johnson wrote: > No worries, thanks again I'll begin teaching this > > On Mon, Aug 31, 2015, 5:16 PM Tomás Fernández Löbbe > wrote: > >> Sorry Jamie, I totally missed this email. There was no Jira that I could >

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-09-01 Thread Jamie Johnson
No worries, thanks again I'll begin teaching this On Mon, Aug 31, 2015, 5:16 PM Tomás Fernández Löbbe wrote: > Sorry Jamie, I totally missed this email. There was no Jira that I could > find. I created SOLR-7996 > > On Sat, Aug 29, 2015 at 5:26 AM, Jamie Johnson wrote: > > > This sounds like a

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-31 Thread Tomás Fernández Löbbe
Sorry Jamie, I totally missed this email. There was no Jira that I could find. I created SOLR-7996 On Sat, Aug 29, 2015 at 5:26 AM, Jamie Johnson wrote: > This sounds like a good idea, I'm assuming I'd need to make my own > UnInvertingReader (or subclass) to do this right? Is there a way to do

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-29 Thread Jamie Johnson
Also since DocValues seems to be the future of faceting, is there another mechanism that I should be looking at to do authorization based filtering like this? I know that I can do this filtering at a document level and get the desired result, but am wondering about at the Term level. As always th

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-29 Thread Jamie Johnson
This sounds like a good idea, I'm assuming I'd need to make my own UnInvertingReader (or subclass) to do this right? Is there a way to do this on the 5.x codebase or would I still need the solrindexer factory work that Tomás mentioned previously? Tomás, is there a ticket for the SolrIndexer facto

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Yonik Seeley
UnInvertingReader makes indexed fields look like docvalues fields. The caching itself is still done in FieldCache/FieldCacheImpl but you could perhaps wrap what is cached there to either screen out stuff or construct a new entry based on the user. -Yonik On Thu, Aug 27, 2015 at 12:55 PM, Jamie J

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Jamie Johnson
I think a custom UnInvertingReader would work as I could skip the process of putting things in the cache. Right now in Solr 4.x though I am caching based but including the users authorities in the key of the cache so we're not rebuilding the UnivertedField on every request. Where in 5.x is the ob

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Yonik Seeley
On Thu, Aug 27, 2015 at 11:59 AM, Tomás Fernández Löbbe wrote: > I don't think there is a way to do this now. Maybe we should separate the > logic of creating the SolrIndexSearcher to a factory. That should probably be extended down to where lucene creates searchers as well (delete-by-query). Rig

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Yonik Seeley
The FieldCache has become implementation rather than interface, so I don't think you're going to see plugins at that level (it's all package protected now). One could either subclass or re-implement UnInvertingReader though. -Yonik On Thu, Aug 27, 2015 at 12:09 PM, Jamie Johnson wrote: > Also

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Jamie Johnson
Also in this vein I think that Lucene should support factories for the cache creation as described @ https://issues.apache.org/jira/browse/LUCENE-2394. I'm not endorsing the patch that is provided (I haven't even looked at it) just the concept in general. On Thu, Aug 27, 2015 at 12:01 PM, Jamie J

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Jamie Johnson
That makes sense, then I could extend the SolrIndexSearcher by creating a different factory class that did whatever magic I needed. If you create a Jira ticket for this please link it here so I can track it! Again thanks On Thu, Aug 27, 2015 at 11:59 AM, Tomás Fernández Löbbe < tomasflo...@gmail

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-27 Thread Tomás Fernández Löbbe
I don't think there is a way to do this now. Maybe we should separate the logic of creating the SolrIndexSearcher to a factory. Moving this logic away from SolrCore is already a win, plus it will make it easier to unit test and extend for advanced use cases. Tomás On Wed, Aug 26, 2015 at 8:10 PM,

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-26 Thread Jamie Johnson
Sorry to poke this again but I'm not following the last comment of how I could go about extending the solr index searcher and have the extension used. Is there an example of this? Again thanks Jamie On Aug 25, 2015 7:18 AM, "Jamie Johnson" wrote: > I had seen this as well, if I over wrote this

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-25 Thread Jamie Johnson
I had seen this as well, if I over wrote this by extending SolrIndexSearcher how do I have my extension used? I didn't see a way that could be plugged in. On Aug 25, 2015 7:15 AM, "Mikhail Khludnev" wrote: > On Tue, Aug 25, 2015 at 2:03 PM, Jamie Johnson wrote: > > > Thanks Mikhail. If I'm rea

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-25 Thread Mikhail Khludnev
On Tue, Aug 25, 2015 at 2:03 PM, Jamie Johnson wrote: > Thanks Mikhail. If I'm reading the SimpleFacets class correctly, out > delegates to DocValuesFacets when facet method is FC, what used to be > FieldCache I believe. DocValuesFacets either uses DocValues or builds then > using the Uninverti

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-25 Thread Jamie Johnson
Thanks Mikhail. If I'm reading the SimpleFacets class correctly, out delegates to DocValuesFacets when facet method is FC, what used to be FieldCache I believe. DocValuesFacets either uses DocValues or builds then using the UninvertingReader. I am not seeing a clean extension point to add a cust

Re: Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-24 Thread Mikhail Khludnev
Hello Jamie, I don't understand how it could choose DocValuesFacets (it occurs on docValues=true) field, but then switches to UninvertingReader/FieldCache which means docValues=false. If you can provide more details it would be great. Beside of that, I suppose you can only implement and inject your

Lucene/Solr 5.0 and custom FieldCahe implementation

2015-08-24 Thread Jamie Johnson
as mentioned in a previous email I have a need to provide security controls at the term level. I know that Lucene/Solr doesn't support this so I had baked something onto a 4.x baseline that was sufficient for my use cases. I am now looking to move that implementation to 5.x and am running into an