Hi Ray, I’ll try to answer your questions…
On Sep 7, 2015, at 4:36 PM, ray <r...@aarden.us> wrote: > Rick, > > Thank you for responding and providing all the info. > > On Monday, September 7, 2015 at 6:20:07 AM UTC-5, Rick Thomas wrote: >> On Sep 5, 2015, at 7:24 PM, ray wrote: >> >>> I would like to configure LVMs for everything including boot. >> >> Is it "just for fun" or do you have a real-world reason for wanting >> everything, including boot, to be on LVM? > Lack of knowledge. I was expecting it to be cleaner. But it no longer looks > that way. >> >> I'll describe my own typical setup (special purpose systems may have >> different setups to meet special purpose needs). For purposes of >> illustration, I'll describe a system with two identical disks. The >> principles should be clear as they apply to systems with larger or more >> varied configurations. If you have only a single disk, you can skip the >> RAID parts in this and go straight to LVM. > > I have 3 pairs of SSDs, each pair in a RAID0. I would use RAID1 on each pair (3 SSDs worth of “usable” space), or RAID5 or 6 on a larger aggregate. E.g. RAID6 on all 6 drives (gives 4 SSDs worth of “usable” space), or RAID5 on 6 drives (gives 5 SSDs worth of “usable” space). Each of those configurations can survive a loss of one SSD (or two, in the case of RAID6) without data loss. Your choice of RAID0 in pairs gives the full 6 SSDs worth of “usable” space, but has zero redundancy. If that works for you, that’s great. I’ve got enough experience (40 years) as a sysadmin to have seen users tearing hair over lost data (I always had backups — often tape in those days — so the only thing really lost was uptime, but you get the point…) In case it’s not clear, “usable” means space left over after subtracting out all the redundant data in the array. > >> >> I configure a small (<1GB) "/boot" partition as a primary partition (e.g. >> /dev/sda1) on one of the disks, with the same space on the other disk >> unused. [1] I make another primary partition (e.g. /dev/sd[ab]2), on each >> of the disks, sized to be one half of the size I want for my swap. The rest >> of the space on each disk goes into a single, large, logical partition (e.g. >> /dev/sd[ab]5). >> > This is similar to my setup. I have the swaps on a separate RAID0. Should work fine. Of course you should read Pascal’s post for a different point of view. > >> The two swap partitions I set up as a RAID0 (e.g. /dev/md0). This will be >> my system swap. [2] >> >> The two large logical partitions, I configure into a RAID1 (e.g. /dev/md1). >> [3] >> > I made my large partition a primary. Could this be problematic? It would only be a problem if you need another primary partition for something. You only get 3 primary partitions on a disk, so I like to leave one free “just incase”. > >> I configure the RAID1 as the physical volume for a single volume group which >> I partition using LVM into a root that's big enough to be about 50% full >> when fully installed, and /home that's as big as I think I'll need for my >> users. The remainder of the VG I leave unconfigured, so I can grow into it >> as needs become apparent over time. >> >> If I have enough RAM to make it useful, I'll put /tmp on a tmpfs. I size it >> at about 50% of my swap space. With a smaller RAM, I make a separate >> logical volume for /tmp. >> > I would like to know more about the purpose of these criteria - please. Putting /tmp on tmpfs is for speed. If your /tmp usage fits in RAM (after allowing for apps, data, and disk buffering) you get RAM speed access to your temporary data. If it overflows, then the excess goes into swap, so you’re no worse off than if you had /tmp on hard disk in the first place. This can make large compilations (as an example) run *much* faster. On the other hand, if you are tight on RAM, putting /tmp on disk doesn’t hurt, and eliminates a source of contention for RAM which is assumed to be a scarce resource. The 50% figure is just a rule of thumb I picked up over the years. There’s nothing magic about it. And, as always, YMMV. It’s highly application dependent. > >> [1] I know there are ways to make grub work with RAID1, but it's too >> complicated for me to get it right. Instead, I just make regular backups of >> the /boot partition. If the disk with /boot on it develops a bad spot in an >> inconvenient place, I simply boot from a CD in rescue mode and restore the >> contents of /boot from a backup into the unused space that I reserved on the >> other disk. > > Yes, that was my plan also. But it was also my plan to backup the rest of > the system to HDD. As such, I planned for no redundancy in drive > configuration and only stripe for speed. It might make sense to create a logical volume with some of the unused space in the volume group called (for lack of a better name) /backup. It’s “reliable” in the sense that it resides on a RAID1/5/6 array, so it’s a good place to put your backups of things like /boot. For “ultimate” backup, I usually use huge (e.g. 4TB or larger) external USB disk drives. I don’t RAID them, instead I have two or three and rotate amongst them using each one for a week, then swapping it out for the next one. The currently offline drive(s) I keep in a fire-proof safe, preferably in a separate building from the server… > >> >> [2] There's no particular point in putting swap on a redundant RAID. If >> your swap develops a bad spot, you probably want to boot from a CD into >> rescue mode ASAP so you can take necessary measures to fix the problem. >> Using RAID1 for swap would just mask the problem -- possibly until it's too >> late. >> >> [3] Conversely, everything else on the system wants to be redundantly >> protected. If I have three or more disks, I'll use RAID5; with four or more >> I might opt for RAID6. >> >> Here's an example: >> >> rbthomas@monk:~$ lsblk >> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT >> sda 8:0 0 465.8G 0 disk >> |-sda1 8:1 0 953M 0 part /boot >> |-sda2 8:2 0 18.6G 0 part >> | `-md0 9:0 0 37.3G 0 raid0 [SWAP] >> |-sda3 8:3 0 1K 0 part >> `-sda5 8:5 0 446.2G 0 part >> `-md1 9:1 0 446.1G 0 raid1 >> |-root-root 253:0 0 18.6G 0 lvm / >> `-root-home 253:3 0 210G 0 lvm /home >> sdb 8:16 0 465.8G 0 disk >> |-sdb1 8:17 0 953M 0 part >> |-sdb2 8:18 0 18.6G 0 part >> | `-md0 9:0 0 37.3G 0 raid0 [SWAP] >> |-sdb3 8:19 0 1K 0 part >> `-sdb5 8:21 0 446.2G 0 part >> `-md1 9:1 0 446.1G 0 raid1 >> |-root-root 253:0 0 18.6G 0 lvm / >> `-root-home 253:3 0 210G 0 lvm /home >> sr0 11:0 1 1024M 0 rom >> >> >> Enjoy! >> Rick > >
