Re: Solr as am html cache

2016-11-22 Thread Erick Erickson
bq: This seems like it might even be a good approach for creating additional cores primarily for the purpose of caching I think you're making it too complex, especially for such a small data set ;) 1> All the data is memory mapped anyway, so what's not in the JVM will be in the OS's memory eventu

Re: Solr as am html cache

2016-11-21 Thread Aristedes Maniatis
Thanks Erick Very helpful indeed. Your guesses on data size are about right. There might only be 50,000 items in the whole index. And typically we'd fetch a batch of 10. Disk is cheap and this really isn't taking much room anyway. For such a tiny data set, it seems like this approach will wor

Re: Solr as am html cache

2016-11-21 Thread Erick Erickson
Searching isn't really going to be impacted much, if at all. You're essentially talking about setting some field with store="true" and stuffing the HTML into that, right? It will probably have indexed="false" and docValues="false". So.. what that means is that very early in the indexing process, t

Solr as am html cache

2016-11-21 Thread Aristedes Maniatis
I'm familiar enough with 7-8 years of Solr usage in how it performs as a full text search index, including spatial coordinates and much more. But for the most part, we've been returning database ids from Solr rather than a full record ready to display. We then grab the data and related records f