On Wed, Mar 04, 2009 at 06:30:47PM -0500, Denis Hainsworth wrote:
> Thanks for your response. Sorry I left out any details in my intial
> email. Also wanted to thank you Marco for all the work you've done for
> openbsd.
>
>
> We are running a fairly basic bonnie++ test like so
> bonnie++ -d /path -s 12000 -m machinename -u 0:0
>
> Here are the full details:
> Version 1.03 ------Sequential Output------ --Sequential Input-
> --Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
> --Seeks--
> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> %CP
> machine1 12000M 6131 3 6470 1 6220 1 72017 51 103703 19 361.3
> 0
> ------Sequential Create------ --------Random
> Create--------
> -Create-- --Read--- -Delete-- -Create-- --Read---
> -Delete--
> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
> %CP
> 16 1423 93 +++++ +++ 8049 27 1434 93 1689 99 4005
> 98
> machine1,12000M,6131,3,6470,1,6220,1,72017,51,103703,19,361.3,0,16,1423,93,++++
>
> +,+++,8049,27,1434,93,1689,99,4005,98
>
>
> This is on a 388GB partition that is made up out of 4disks in a RAID5
> config. Here are the mount options we have been using:
> type ffs (local, noatime, softdep) .. we've tried turing off softdep,
> but it did not help. This filesystem was initialized using 4k inodes.
> Increasing the inode size also did not seem to help.
Ehhm ffs has a fixed inode size. What paramters dud you use for newfs exactly?
-Otto
>
> Due to the fact that we have not gotten the megacli working, we
> have not made any changes to the raid options. I can at least read the
> options sorta using LSI's builtin Megacli in their newer firmware. It
> shows that the cache policy is set to:
>
> WriteThrough, ReadAheadNone, Direct, NoWrite if Bad BBU (I confirmed the
> BBU shows as good)
> Write Policy: WT (writethrough I presume)
> Read Policy : None
>
> We threw openbsd 4.4 on a spare machine we dug up. This machine is
> phsically identical to the ones we are having issues with.
>
> We did not see the same write problems .. here is a similar bonnie run,
> against a standard openbsd filesystem mounted as type ffs (local,
> nodev) this files system was initialized using 8k inodes.
>
> Version 1.03 ------Sequential Output------ --Sequential Input-
> --Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> %CP
> machine2 12000M 31174 13 40551 5 10003 1 60582 35 82658 7 381.1 0
> ------Sequential Create------ --------Random Create--------
> -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
> 16 850 1 +++++ +++ 1848 1 868 1 +++++ +++ 344 0
> machine2,12000M,31174,13,40551,5,10003,1,60582,35,82658,7,381.1,0,16,850,1,+++
> +++,+++,1848,1,868,1,+++++,+++,344,0
>
>
> Additionally as I stated before our previous version of hardware which is
> nearly identical and running the same openbsd 4.2 setup also performs fine.
> It has the same 4k inode filesystem mounted with (local, noatime, softdep)
> options.
>
> Version 1.03 ------Sequential Output------ --Sequential Input-
> --Random-
> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
> --Seeks--
> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
> %CP
> machine3 12000M 23656 20 30849 20 10298 5 29102 37 36291 12 225.0
> 1
> ------Sequential Create------ --------Random
> Create--------
> -Create-- --Read--- -Delete-- -Create-- --Read---
> -Delete--
> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
> %CP
> 16 1206 92 +++++ +++ 16083 45 1070 81 1417 97 2991
> 88
> machine3,12000M,23656,20,30849,20,10298,5,29102,37,36291,12,225.0,1,16,1206,92,+
>
> ++++,+++,16083,45,1070,81,1417,97,2991,88
>
>
>
> So since machine1 and machine3 are running the same exact software it would
> seem to indicate a hardware issue. Yet the same hardware running openbsd
> 4.4 also seems ok which tends to indicate some odd interaction between the
> hardware and openbsd 4.2.
>
> So if anyone has any thoughts they would be appreciated as I have been
> going around in circles on this issue for a while now.
>
> Lastly Marco mentioned I might have interrupt issues. How would I check
> for that?
>
> -denis
>
> Os Tue, Mar 03, 2009 at 10:21:51PM -0600, Marco Peereboom wrote:
> > I think I have to conclude that you (or your hardware) are doing it
> > wrong. Either you have interrupt issues or your raid card setup wrong.
> >
> > I also have no idea how you measured these numbers so I can't deduce
> > any validity from them.
> >
> > On Tue, Mar 03, 2009 at 05:45:02PM -0500, Denis Hainsworth wrote:
> > > Hello,
> > >
> > > Some what of a novice openbsd user and hope I didnt miss something
> > > obvious. I
> > > have a box running openbsd 4.2 which I cannot upgrade for various
> > > reasons. The machine has the Symbios Logic MegaRAID SAS 1064R
> > > controller and currently has a 4 disk Raid5 set up in bios with Seagate
> > > ST9146802SS disks. We also run the same machines with Linux and
> > > openbsd 4.2 with older hardware using an Ultra Scsi controller.
> > >
> > > Anyway we have some basic tests we run and noticed that a basic Bonnie++
> > > test on all 3 machines gave the following performance numbers.
> > >
> > > Openbsd 4.2 and Symbios SAS
> > > 6073 K/sec Block Seq Output and 83086 K/sec Block Seq Input
> > > Linux RH 7 and Symbios SAS
> > > 65096 K/sec Block Seq Output and 165194 K/sec Block Seq Input
> > > Openbsd 4.2 and Adaptec SCSI
> > > 30849 K/sec Block Seq Output and 36291 K/sec Block Seq Input
> > >
> > > Now I wouldnt expect the numbers to match exactly given a number
> > > parameters but the 6073 K/sec Block Seq Output is somewhat of a
> > > concern. Since it doesnt seem to affect Linux using the exact same
> > > hardware I think its most likly a driver and or configuration issue.
> > > So does anyone know if there are issues with the mfi driver or any
> > > configuration bits within openbsd I might be missing with regaurds to
> > > disk performance?
> > >
> > > Here's the exact line from syslog if thats of any help:
> > > Feb 18 19:45:33 /bsd: mfi0 at pci4 dev 14 function 0 "Symbios Logic
> > > MegaRAID SAS 1064R" rev 0x00: apic 8 int 18 (irq 5)
> > > Feb 18 19:45:33 /bsd: mfi0: logical drives 1, version 5.1.1-0038, 128MB
> > > RAM
> > >
> > > I will note that we needed to apply a driver patch to recognise this
> > > card in 4.2 but that was simply to add the pci device type to the list
> > > of other supported types. I am unaware of any other changes.
> > >
> > > thank you for your time
> > > -denis
> > >
> > > --
> > > __________________________
> > > Denis Alan Hainsworth
> > > [email protected]
>
> --
> __________________________
> Denis Alan Hainsworth
> [email protected]
