You're both completely right. There isn't any issue with indexing with large cache settings.
I ran the same indexing job five times, twice with large cache and twice with the default values. I threw out the first job because no matter if it's cached or uncached it runs ~2x slower. This must have been the observation I based my incorrect caching notion on. I unloaded with delete of the data directory and reloaded the core each time. I'm using DIH with the FileEntityProcessor and PlainTextEnityProcessor to index ~11000 fulltext books. w/ cache 0:13:14.823 0:12:33.910 w/o cache 0:12:13.186 0:15:56.566 There is variation, but not anything that could be explained by the cache settings. Doh! Thanks, Tricia On Mon, Jan 13, 2014 at 6:08 PM, Shawn Heisey <s...@elyograg.org> wrote: > On 1/13/2014 4:44 PM, Erick Erickson wrote: > >> On the face of it, it's somewhat unusual to have the cache settings >> affect indexing performance. What are you seeing and how are you indexing? >> > > I think this is probably an indirect problem. Cache settings don't > directly affect indexing speed, but when autoWarm values are high and NRT > indexing is happening, new searchers are requested frequently and the > autoWarm makes that happen slowly with a lot of resources consumed. > > Thanks, > Shawn > >
