On Fri, Jul 21, 2006 at 11:39:23PM -0400, [EMAIL PROTECTED] wrote: > > yeesh. for what it's worth, I sat in on an OLS BOF today which > discussed this issue (using big pages in linux). from what I gather, > kernel authorities don't want to complicate the VM by mixing normal > and big pages.... .... > (falling back from large to small pages and replacing contiguous ranges > of small pages with larger ones), perhaps purely at user-level. > > it would be incredibly useful if the HPC community could come up with > some actual numbers on how important bigpages are. I'm ashamed to say > that I haven't made any effort to measure this on any of my clusters. > oprofile appears to make it pretty easy to capture real data on how > often TLB's are a problem. > > have any of you tried to get a handle on this stuff?
It might pay to look at papers by Kazushige Goto. http://www.tacc.utexas.edu/general/news/archive/20040809_02.php http://www.tacc.utexas.edu/general/staff/goto/ http://www.tacc.utexas.edu/resources/software/#blas His blas libraries are tuned in a number of ways including page size and the expense of TLB hits. I mention this because numerous HPC benchmark numbers are built to use his libraries. As such these HPC numbers confuse numbers obtained from less optimum libs and any TLB related values. i.e. compare HPC numbers where the only thing that changes is the blas lib. See #9: http://www.cs.utexas.edu/users/flame/Publications/ "Kazushige Goto and Robert van de Geijn. On Reducing TLB Misses in Matrix Multiplication. FLAME Working Note #9, The University of Texas at Austin, Department of Computer Sciences. Technical Report TR-2002-55. Nov. 2002." http://www.cs.utexas.edu/users/flame/pubs/FLAWN9.ps.gz In this paper are a number of tables that give a clue of the baseline expense of TLB hits as well as resulting improvements when this expense taken into account in the design of the library. My thought right now is that very large pages for data would be nifty to have if you knew you could get them. If you cannot guarantee them then use libs like the Goto-blas library so it matters less. Important, http://www.tacc.utexas.edu/resources/software/gotoblasfaq.php "GOTO is my last name and does not mean "go to." (Nor is it pronounced "go to"; the correct pronunciation of my name is more like "goat-toe.")" -- T o m M i t c h e l l Found me a new place to hang my hat :-) Now it got bought. _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
