On 9/3/2010 12:37 PM, Jonathan Rochkind wrote:
Is the OS disk cache something you configure, or something the OS just does automatically based on available free RAM? Or does it depend on the exact OS? Thinking about the OS disk cache is new to me. Thanks for any tips.
Depends on what you want to configure. If you're referring to the size, I haven't seen any way to change it, but I haven't really tried to look. It's pretty much automatic, with the OS using all free RAM as necessary. If the memory is suddenly required for an application, it just discards the least used entries from the cache.
Most operating systems will let you tune how likely it is that the OS will swap application memory out to disk so it can have more RAM for disk cache. Depending on how that's set, you might actually find your applications being swapped out to make room for the cache. If that's happening, things will REALLY slow down.
Linux distributions usually have a default /proc/sys/vm/swappiness setting of 60%, which despite being pretty high, is perfectly fine for most workloads when you've got enough memory. If you are not using up most of your system memory with applications, it'll never touch the swap. If you're running very tight, you can to adjust that down, but in the end you just need more memory.
When it comes to BSD or other genetic UNIX systems, I'm in unknown territory, but as far as I know, they all work about the same.
On Windows, the server products come with the memory settings optimized for file serving (disk cache), which is awesome for Solr, unless you don't have enough RAM. The desktop products are optimized for applications.
On most of the free OSes like Linux and the BSDs, the output of the 'free' command will tell you lots of things. Important numbers are "cached" and the amount of swap currently used. You can also watch 'vmstat 5' and see these values (and a few others) change over time. If you're on Windows, open the task manager performance tab, resize it up a ways, and run it for a while.
Thanks, Shawn
