Thanks Toke. Your answer did help me a lot. But one part about your answer is something that has always been confusing to be me.
> The JVM heap is not used for caching the index data directly (although it holds derived data). What you need is free memory on your machine for OS disk-caching. > The ideal JVM size is extremely dependent on how you index, query and adjust the filter-cache (secondarily the other caches, but the filter-cache tends to be the large one). A heap of 10GB might very well be fine for handling your whole 50GB index. If that is on a 64GB machine, the remaining 54GB of RAM (minus the other stuff that is running) ought to ensure a fully cached index. How can 50GB index be handled by a 10GB heap? I am a developer myself and would love to know as many details as possible. So a long answer would be much appreciated. Also, on a related note - Are stored fields or doc values part of index too? Are they stored in JVM or OS-cache? (I would guess latter, but does that mean JVM is just not required for those or a small percentage?) Thanks SG On Thu, May 11, 2017 at 7:33 AM, Shawn Heisey <apa...@elyograg.org> wrote: > On 5/10/2017 11:52 AM, S G wrote: > > Is there a recommendation on the size of index that one should host > > per core? > > No, there really isn't. > > I can list off a bunch of recommendations, but a whole bunch of things > that I don't know about your install could make those recommendations > completely wrong. An index size that works really well for one person > might have terrible performance for another. > > If you haven't already built it, then there are possibly even things > that YOU don't know about your install yet that can affect what what you > need. > > https://lucidworks.com/sizing-hardware-in-the-abstract-why- > we-dont-have-a-definitive-answer/ > > The only general advice I can give you is this: You're probably going > to need more RAM. > > Thanks, > Shawn > >
