bq: disk space is three times True, I keep forgetting about compound since I never use it...
On Wed, Apr 10, 2013 at 11:05 AM, Walter Underwood <wun...@wunderwood.org> wrote: > Correct, except the worst case maximum for disk space is three times. --wunder > > On Apr 10, 2013, at 6:04 AM, Erick Erickson wrote: > >> You're mixing up disk and RAM requirements when you talk >> about having twice the disk size. Solr does _NOT_ require >> twice the index size of RAM to optimize, it requires twice >> the size on _DISK_. >> >> In terms of RAM requirements, you need to create an index, >> run realistic queries at the installation and measure. >> >> Best >> Erick >> >> On Tue, Apr 9, 2013 at 10:32 PM, bigjust <bigj...@lambdaphil.es> wrote: >>> >>> >>> >>>>> On 4/9/2013 7:03 PM, Furkan KAMACI wrote: >>>>>> These are really good metrics for me: >>>>>> You say that RAM size should be at least index size, and it is >>>>>> better to have a RAM size twice the index size (because of worst >>>>>> case scenario). >>>>>> On the other hand let's assume that I have a RAM size that is >>>>>> bigger than twice of indexes at machine. Can Solr use that extra >>>>>> RAM or is it a approximately maximum limit (to have twice size of >>>>>> indexes at machine)? >>>>> What we have been discussing is the OS cache, which is memory that >>>>> is not used by programs. The OS uses that memory to make everything >>>>> run faster. The OS will instantly give that memory up if a program >>>>> requests it. >>>>> Solr is a java program, and java uses memory a little differently, >>>>> so Solr most likely will NOT use more memory when it is available. >>>>> In a "normal" directly executable program, memory can be allocated >>>>> at any time, and given back to the system at any time. >>>>> With Java, you tell it the maximum amount of memory the program is >>>>> ever allowed to use. Because of how memory is used inside Java, >>>>> most long-running Java programs (like Solr) will allocate up to the >>>>> configured maximum even if they don't really need that much memory. >>>>> Most Java virtual machines will never give the memory back to the >>>>> system even if it is not required. >>>>> Thanks, Shawn >>>>> >>>>> >>> Furkan KAMACI <furkankam...@gmail.com> writes: >>> >>>> I am sorry but you said: >>>> >>>> *you need enough free RAM for the OS to cache the maximum amount of >>>> disk space all your indexes will ever use* >>>> >>>> I have made an assumption my indexes at my machine. Let's assume that >>>> it is 5 GB. So it is better to have at least 5 GB RAM? OK, Solr will >>>> use RAM up to how much I define it as a Java processes. When we think >>>> about the indexes at storage and caching them at RAM by OS, is that >>>> what you talk about: having more than 5 GB - or - 10 GB RAM for my >>>> machine? >>>> >>>> 2013/4/10 Shawn Heisey <s...@elyograg.org> >>>> >>> >>> 10 GB. Because when Solr shuffles the data around, it could use up to >>> twice the size of the index in order to optimize the index on disk. >>> >>> -- Justin > > -- > Walter Underwood > wun...@wunderwood.org > > >
