Doesn't it make sense to have a Indexer, Query Engine setup? Indexer = Solr instance with replication configured as Master Query Engine = One or more Solr instances with replication configured as Slave
So that, you can do batch indexing on the Indexer, perform threshold checks if needed by disabling the replication. If threshold checks are passed, enable the replication. This way you can configure the caches, other settings that you need for indexing and configure something else on your Query Engine. Thanks, Kranti K. Parisa http://www.linkedin.com/in/krantiparisa On Fri, Jan 17, 2014 at 11:31 AM, P Williams <williams.tricia.l...@gmail.com > wrote: > You're both completely right. There isn't any issue with indexing with > large cache settings. > > I ran the same indexing job five times, twice with large cache and twice > with the default values. I threw out the first job because no matter if > it's cached or uncached it runs ~2x slower. This must have been the > observation I based my incorrect caching notion on. > > I unloaded with delete of the data directory and reloaded the core each > time. I'm using DIH with the FileEntityProcessor and > PlainTextEnityProcessor to index ~11000 fulltext books. > > w/ cache > 0:13:14.823 > 0:12:33.910 > > w/o cache > 0:12:13.186 > 0:15:56.566 > > There is variation, but not anything that could be explained by the cache > settings. Doh! > > Thanks, > Tricia > > > On Mon, Jan 13, 2014 at 6:08 PM, Shawn Heisey <s...@elyograg.org> wrote: > > > On 1/13/2014 4:44 PM, Erick Erickson wrote: > > > >> On the face of it, it's somewhat unusual to have the cache settings > >> affect indexing performance. What are you seeing and how are you > indexing? > >> > > > > I think this is probably an indirect problem. Cache settings don't > > directly affect indexing speed, but when autoWarm values are high and NRT > > indexing is happening, new searchers are requested frequently and the > > autoWarm makes that happen slowly with a lot of resources consumed. > > > > Thanks, > > Shawn > > > > >
