On Thu, 11 Jul 2013, Stan Hoeppner wrote: > On 7/11/2013 11:25 AM, Henrique de Moraes Holschuh wrote: > > On Thu, 11 Jul 2013, Stan Hoeppner wrote: > >>> At some point in the future, I may buy one or two additional HDD and use > >>> hardware RAID. > >> > >> It's best to use identical drives with identical firmware, which means > >> buying all your drives up front from the same lot. You also need to > > > > That is a fast way to data loss. > > This is FUD, plain and simple.
Well, I've seen it happen at a small government datacenter where we did _not_ use FC storages for everything. Most of the data was stored in direct-attach JBODs hooked to U320 SCSI RAID cards (decomissioned five years ago), and later in internal hotswap bays hooked to SATA HBAs (I managed to end that practice a few years ago, mostly to get rid of SATA HDDs which are indeed all crap) or to SAS HW RAID cards. Disks would sometimes fail within one month of each other, and we did have a couple episodes of discs in the same RAID set failing a few days of each other. And every time that kind of time-linked failure happened, the drives *were* either from the same manufacturing lot, or from the same factory and close manufacturing date, and in the same disk pool (thus subject to almost the same usage load). After two episotes of that sort, we started mixing discs on anything that wasn't a FC storage box. It is worth noticing these failures happened to devices that where "somewhat old", with more than 3 years of 24/7 service. > > You do not want the drives to fail close > > to each other, so get them from different manufacturing lots, preferably > > with different manufacturing dates separated by at least 6 months. > > You're out of your element Henrique. Some who use strictly software md > RAID prefer this strategy, and conceptually it makes sense. However in > the real world, when you order drives, the vendor doesn't allow you to > pick and choose lot numbers, firmware revs, and manufacturing dates. No, they don't... except when they do. Anyway, FWIW, I indeed have done a few datacenter purchase contracts while I worked at IMA. The HP, IBM and Dell distributors we dealt with were quite willing to sell us mixed HDD manufacturing lots most of the time. These HDDs were *NOT* installed on storages, but directly into the server hotswap bays, attached to the internal RAID cards. For the few FC storage boxes we had, we didn't care as they all had 4-hour MTTR 24/7/365 on-site repair contracts, *and* always had at least two hotspares per group. I am very willing to believe these required all discs to be from the same manufacturer and firmware level. That said, exactly what demography are we addressing here? Last time I checked, debian-user is for your regular and power user. This is not debian-isp, the closest list we have to a debian-datacenter... and this is a thread about SATA3 HBAs. Not SAS-6G HBAs, not SAS-6G HW RAID, and certainly not extremely picky SAN storage systems. So, we are addressing *exactly* the crowd who will benefit more from md software raid, *and* of mixing HDDs. > With respect to hardware RAID controllers, the current discussion you > responded to, having a bunch of drives from different vendors with > different firmware is a recipe for disaster. md RAID can tolerate just > about anything. The firmware on most hardware RAID cards cannot, > because they are designed for speed, not compatibility with any/every > drive combination. Meh, I would not call the firmware on most of the low and medium-end HW RAID cards I dealt with (from Intel, LSI, IBM and HP) "engineered for speed". I sure hope they were engineered for reliability instead... But they did make our life a lot easier due to the SAFTE bay integration, error reporting, auto-scrubbing, the non-volatile cache, and easier setup. We didn't buy them for the speed. > Back to the point, over the years every time I had an issue and called > support at any of the RAID vendors, the first question they asked was > "which RAID card and firmware version?" The seconds question was "how > many drives, which vendor, does the firmware rev on all drives match? > If not, what is the firmware rev on each drive?" [...] > So please, do tell people that they should use drives from different > vendors with mismatched firmware on a real RAID controller, and that if > everything matches they'll lose data. Fifteen+ years of experience > proves the exact opposite of what you say. In reality, drives and > firmware need to match, or be certified by the vendor. If not you'll > drop drives constantly at best, and lose your array at worst once you > put some load on it in production. Anyway, I will make a summary that actually separates the apples from the oranges, because we were clearly talking about different things: 1. If you're buying enterprise RAID arrays, you should have matching disks and firmware, and they must be in the vendor approved list. Best to get them all from the same vendor. 2. If you're buying HBAs and disks to build software RAID arrays, a mixed disk farm is a good idea. PS: I'd appreciate if you could list some of the HW RAID cards that are doing disk blacklisting, that's something nice to know so that I avoid buying one of them used for the home. I've seen HW RAID cards mark the HDD as "failed" in the on-disk metadata and you would have to take special action to be able to add them back into an array... but actually blacklisting failed disks in NVRAM is news to me. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130712011734.ga7...@khazad-dum.debian.net
