NFS or iscsi? The defaults usually work pretty good for nfs especially if using log devices.
On Jan 17, 2012, at 11:08, Grant Albitz <[email protected]> wrote: > i wanted to make a correct, sync writes do utilize the zil, but based on what > i am reading, large sync writes do not. > > ________________________________________ > From: Grant Albitz [[email protected]] > Sent: Tuesday, January 17, 2012 11:01 AM > To: Discussion list for OpenIndiana > Subject: Re: [OpenIndiana-discuss] Is there anything I can do to tune caching > > my environment consists of 2 esxi hosts each connected to the san over a > dedicated 10gb link. i ran the benchmark from inside one of my guest vms. > > the performance is definately good enough for the environment. I understand > that by default zfs may not cache synchronous writes. I guess my question is > can that be changed? i would rather have those writes goto ram then reserve > ram for read items that may or may not be requested... Is it possible to > create a ram disk in solaris and then use that as the zil? then again i read > the zil is generally not used for synchronous writes so i might be back to > the same point even if i implement somethign like that. > > i understand the risk of writes going to ram but that can be addressed in > other ways, i am just wondering if the synchronous caching can be > enabled/tweaked. > > also my l2arch devices remain fairly empty and not utilized, is there anyway > to push zfs to cache more items or do i just need to let this build over > time? is anyone aware of a time to live for items in the zfs cache? Given my > relatively light workload i would prefer that items put in cache werent > removed unless its running low on space. these may just turn out to be nice > to haves that arent possible, i just figured they might be =) > > > > ________________________________________ > From: Grant Albitz [[email protected]] > Sent: Tuesday, January 17, 2012 10:05 AM > To: Discussion list for OpenIndiana > Subject: Re: [OpenIndiana-discuss] Is there anything I can do to tune caching > > i installed solaris on p0 of one of the ssds, performed a software raid 1 to > the other ssd for boot, and then used thh remaining partition on the ssds as > the cache device. this is why you see the cache drives as c2t12d0p2 and > c2t13d0p2 > > > ________________________________________ > From: Andy Lubel [[email protected]] > Sent: Tuesday, January 17, 2012 9:33 AM > To: Discussion list for OpenIndiana > Cc: Discussion list for OpenIndiana > Subject: Re: [OpenIndiana-discuss] Is there anything I can do to tune caching > > I'm curious, where is the OS installed? Also did you compile crystaldisk for > Solaris or run it over network from a windows box? Perhaps looking into file > bench would give you more numbers to look at and depending on what you plan > on doing with the nas, help decide if it is tuned correctly. > > > > On Jan 16, 2012, at 21:33, Grant Albitz <[email protected]> wrote: > >> Sorry my second table formatting was lost. >> >> The ssds each have ~24gigs used with ~190g free. >> >> -----Original Message----- >> From: Grant Albitz [mailto:[email protected]] >> Sent: Monday, January 16, 2012 9:30 PM >> To: Discussion list for OpenIndiana ([email protected]) >> Subject: [OpenIndiana-discuss] Is there anything I can do to tune caching >> >> I just completed my server and I am generally happy with it. I am running >> openindiana on a dell r510 with 64gb of memory and 12 2tb 7200rpm sas >> drives. I have 2 256g Samsung 830s as l2arche. I have noticed a few things. >> First the l2arche is basically not being populated at all (only 20g or so). >> Also when running a crystaldisk benchmark against the lun on the zfs server, >> small writes seem to goto ram. If I perform a test with 100m I get about >> 800MB/s writes. If I increase that to 500m I get only about 80MB/s. It seems >> that the larger writes go directly to the disk. I have disabled zil with the >> expectation that all writes would goto ram but this may not be the case. >> Below are some numbers that I pulled. Is there anyway to increase l2arche >> usage and also ram usage? I understand that the l2arch may not be populated >> if most of the activity can fit in the ram (and it probably can, my total >> used space is only 300gigs and right now I am testing this with single user >> access). But if > > > t he reason l2arch isn't being populated is due to ram availability, then > why are some of my writes skipping ram and going directly to the disk? >> >> ARC syncronous write cache >> System Memory: >> Physical RAM: 65515 MB >> Free Memory : 10187 MB >> LotsFree: 1023 MB >> >> ZFS Tunables (/etc/system): >> >> ARC Size: >> Current Size: 48921 MB (arcsize) >> Target Size (Adaptive): 48921 MB (c) >> Min Size (Hard Limit): 8061 MB (zfs_arc_min) >> Max Size (Hard Limit): 64491 MB (zfs_arc_max) >> >> ARC Size Breakdown: >> Most Recently Used Cache Size: 93% 45557 MB (p) >> Most Frequently Used Cache Size: 6% 3364 MB (c-p) >> >> ARC Efficency: >> Cache Access Total: 72661651 >> Cache Hit Ratio: 84% 61140347 [Defined State for buffer] >> Cache Miss Ratio: 15% 11521304 [Undefined State for >> Buffer] >> REAL Hit Ratio: 76% 55338630 [MRU/MFU Hits Only] >> >> Data Demand Efficiency: 97% >> Data Prefetch Efficiency: 35% >> >> CACHE HITS BY CACHE LIST: >> Anon: 8% 5398251 [ New >> Customer, First Cache Hit ] >> Most Recently Used: 44% 27083442 (mru) [ Return >> Customer ] >> Most Frequently Used: 46% 28255188 (mfu) [ >> Frequent Customer ] >> Most Recently Used Ghost: 0% 141042 (mru_ghost) [ Return >> Customer Evicted, Now Back ] >> Most Frequently Used Ghost: 0% 262424 (mfu_ghost) [ >> Frequent Customer Evicted, Now Back ] >> CACHE HITS BY DATA TYPE: >> Demand Data: 65% 40237050 >> Prefetch Data: 9% 5777420 >> Demand Metadata: 24% 15098015 >> Prefetch Metadata: 0% 27862 >> CACHE MISSES BY DATA TYPE: >> Demand Data: 7% 864227 >> Prefetch Data: 91% 10540593 >> Demand Metadata: 0% 112421 >> Prefetch Metadata: 0% 4063 >> --------------------------------------------- >> >> >> pool >> >> alloc >> >> free >> >> read >> >> write >> >> read >> >> write >> >> ------------- >> >> ----- >> >> ----- >> >> ----- >> >> ----- >> >> ----- >> >> ----- >> >> PSC.Net >> >> 442G >> >> 10.4T >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> mirror >> >> 73.6G >> >> 1.74T >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t0d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t1d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> mirror >> >> 73.8G >> >> 1.74T >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t2d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t3d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> mirror >> >> 73.8G >> >> 1.74T >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t4d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t5d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> mirror >> >> 73.8G >> >> 1.74T >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t6d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t7d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> mirror >> >> 73.7G >> >> 1.74T >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t8d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t9d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> mirror >> >> 73.8G >> >> 1.74T >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t10d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t11d0 >> >> - >> >> - >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> cache >> >> - >> >> - >> >> - >> >> - >> >> - >> >> - >> >> c2t12d0p2 >> >> 24.8G >> >> 194G >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> c2t13d0p2 >> >> 24.4G >> >> 194G >> >> 0 >> >> 0 >> >> 0 >> >> 0 >> >> >> >> _______________________________________________ >> OpenIndiana-discuss mailing list >> [email protected] >> http://openindiana.org/mailman/listinfo/openindiana-discuss >> >> _______________________________________________ >> OpenIndiana-discuss mailing list >> [email protected] >> http://openindiana.org/mailman/listinfo/openindiana-discuss > > _______________________________________________ > OpenIndiana-discuss mailing list > [email protected] > http://openindiana.org/mailman/listinfo/openindiana-discuss > _______________________________________________ > OpenIndiana-discuss mailing list > [email protected] > http://openindiana.org/mailman/listinfo/openindiana-discuss > _______________________________________________ > OpenIndiana-discuss mailing list > [email protected] > http://openindiana.org/mailman/listinfo/openindiana-discuss > _______________________________________________ > OpenIndiana-discuss mailing list > [email protected] > http://openindiana.org/mailman/listinfo/openindiana-discuss _______________________________________________ OpenIndiana-discuss mailing list [email protected] http://openindiana.org/mailman/listinfo/openindiana-discuss
