Hello,
----- Original Message ---- > From: bharath venkatesh <bharathv6.proj...@gmail.com> > To: solr-user@lucene.apache.org > Sent: Tue, November 10, 2009 8:07:59 AM > Subject: Re: tracking solr response time > > Otis, > > This means we have to leave enough space for os cache to cache the whole > index . so In case of 16 GB index ., if I am not wrong at least 16 GB > memory must not be allocated to any application for os cache to utilize > the memory . No, on the contrary - you want to give the process (the JVM in this case) enough so it works comfortably, but not too much, since it is not the process itself that will load and cache your data/index, but the OS. Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR > >> The operating systems are very good at maintaining this cache. It > > > usually better to give the Solr JVM enough memory to run comfortably > > > and rely on the OS cache to optimize disk I/O, instead of giving it > > > all available ram. > > how much ram would be good enough for the Solr JVM to run comfortably. > > > thanks, > Bharath > > > On Tue, Nov 10, 2009 at 3:59 AM, Otis Gospodnetic < > otis_gospodne...@yahoo.com> wrote: > > > Bharat, > > > > No, you should not give the JVM so much memory. Give it enough to avoid > > overly frequent GC, but don't steal memory from the OS cache. > > > > Otis > > -- > > Sematext is hiring -- http://sematext.com/about/jobs.html?mls > > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR > > > > > > > > ----- Original Message ---- > > > From: bharath venkatesh > > > To: solr-user@lucene.apache.org > > > Sent: Sun, November 8, 2009 2:15:00 PM > > > Subject: Re: tracking solr response time > > > > > > Thanks Lance for the clear explanation .. are you saying we should give > > > solr JVM enough memory so that os cache can optimize disk I/O efficiently > > .. > > > that means in our case we have 16 GB index so would it be enough to > > > allocated solr JVM 20GB memory and rely on the OS cache to optimize disk > > I/O > > > i .e cache the index in memory ?? > > > > > > > > > below is stats related to cache > > > > > > > > > *name: * queryResultCache *class: * org.apache.solr.search.LRUCache * > > > version: * 1.0 *description: * LRU Cache(maxSize=512, initialSize=512, > > > autowarmCount=256, > > > regenerator=org.apache.solr.search.solrindexsearche...@67e112b3) > > > *stats: *lookups > > > : 0 > > > hits : 0 > > > hitratio : 0.00 > > > inserts : 8 > > > evictions : 0 > > > size : 8 > > > cumulative_lookups : 15 > > > cumulative_hits : 7 > > > cumulative_hitratio : 0.46 > > > cumulative_inserts : 8 > > > cumulative_evictions : 0 > > > > > > > > > *name: * documentCache *class: * org.apache.solr.search.LRUCache * > > > version: * 1.0 *description: * LRU Cache(maxSize=512, initialSize=512) > > * > > > stats: *lookups : 0 > > > hits : 0 > > > hitratio : 0.00 > > > inserts : 0 > > > evictions : 0 > > > size : 0 > > > cumulative_lookups : 744 > > > cumulative_hits : 639 > > > cumulative_hitratio : 0.85 > > > cumulative_inserts : 105 > > > cumulative_evictions : 0 > > > > > > > > > *name: * filterCache *class: * org.apache.solr.search.LRUCache > > > *version: *1.0 > > > *description: * LRU Cache(maxSize=512, initialSize=512, > > autowarmCount=256, > > > regenerator=org.apache.solr.search.solrindexsearche...@1e3dbf67) > > > *stats: *lookups > > > : 0 > > > hits : 0 > > > hitratio : 0.00 > > > inserts : 20 > > > evictions : 0 > > > size : 12 > > > cumulative_lookups : 64 > > > cumulative_hits : 60 > > > cumulative_hitratio : 0.93 > > > cumulative_inserts : 12 > > > cumulative_evictions : 0 > > > > > > > > > hits and hit ratio are zero for ducment cache , filter cache and query > > > cache .. only commulative hits and hitratio has a non zero numbers .. > > is > > > this how it is supposed to be .. or do we to configure it properly ? > > > > > > Thanks, > > > Bharath > > > > > > > > > > > > > > > > > > On Sat, Nov 7, 2009 at 5:47 AM, Lance Norskog wrote: > > > > > > > The OS cache is the memory used by the operating system (Linux or > > > > Windows) to store a cache of the data stored on the disk. The cache is > > > > usually by block numbers and are not correlated to files. Disk blocks > > > > that are not used by programs are slowly pruned from the cache. > > > > > > > > The operating systems are very good at maintaining this cache. It > > > > usually better to give the Solr JVM enough memory to run comfortably > > > > and rely on the OS cache to optimize disk I/O, instead of giving it > > > > all available ram. > > > > > > > > Solr has its own caches for certain data structures, and there are no > > > > solid guidelines for tuning those. The solr/admin/stats.jsp page shows > > > > the number of hits & deletes for the caches and most people just > > > > reload that over & over. > > > > > > > > On Fri, Nov 6, 2009 at 3:09 AM, bharath venkatesh > > > > wrote: > > > > >>I have to state the obvious: you may really want to upgrade to 1.4 > > when > > > > > it's out > > > > > > > > > > when would solr 1.4 be released .. is there any beta version > > available ? > > > > > > > > > >>We don't have the details, but a machine with 32 GB RAM and 16 GB > > index > > > > > should have the whole index cached by >the OS > > > > > > > > > > do we have to configure solr for the index to be cached by OS in a > > > > > optimised way . how does this caching of index in memory happens ? > > r > > > > > there any docs or link which gives details regarding the same > > > > > > > > > >>unless something else is consuming the memory or unless something is > > > > > constantly throwing data out of the OS >cache (e.g. frequent index > > > > > optimization). > > > > > > > > > > what are the factors which would cause constantly throwing data out > > of > > > > the > > > > > OS cache (we are doing index optimization only once in a day during > > > > > midnight ) > > > > > > > > > > > > > > > Thanks, > > > > > Bharath > > > > > > > > > > > > > > > > > > > > > -- > > > > Lance Norskog > > > > goks...@gmail.com > > > > > > > >
