Hey folks, Andy -- thanks for the pointer; bcachefs continue to be on my list of cool things to understand better, but I'm now aware using it in Debian stable has some overhead.
Eben -- I raised a somewhat similar point to yours on one of these lists a few months ago. My requirements were that in exchange for my showing up with a bunch of disks, the system could tolerate any of them failing with minimal overhead on my part. The solution I settled on was guaranteeing more than three disks, and putting root on btrfs in RAID1c3. This way, any one disk could fail with no data loss. This leaves you with the problem of being able to boot the machine given any one disk failing; I addressed this by putting GRUB on *all* the disks, and manually (or with grub-hooks), reinstalling grub to everything after each update. Each new disk thus has ~100 Mb of bootable-flagged (in gparted) space up front, with the rest allocated to the btrfs pool. This seems to have worked over two disk failures over four machines in the last six months; but I would be very glad to hear about any loopholes I've missed.... Cheers! On Fri, Mar 21, 2025 at 6:26 PM Andy Smith <a...@strugglers.net> wrote: > > Hi, > > On Fri, Mar 21, 2025 at 03:32:39PM -0400, Eben King wrote: > > I need to do something that ends up with a larger array using 3, maybe > > 4 disks. It'd be nice if it supports disks of disparate sizes (and > > actually uses the extra space), so I can upgrade by attrition. > > It depends on how wedded you are to the idea of making use of more of > the caoacity of mismatched (capacity) devices. > > md and zfs are both fine choices on Linux but in the case of mirrors > will only give you mirrors of the smallest device up until you have > both sides of the mirror with the larger device, then you can grow it > in place to the full capacity. > > So for example, with your existing 2x1TB if you add 2x8TB or something, > you could: > > 1. Break your 1TB mirror > 2. Add an 8TB drive as the other half of your mirror > 3. Resync your mirror > 4. Use removed 1TB drive and remaining 8TB drive to make another (1TB) mirror > 5. LVM the two mirrors together to get 2TB usable space > 6. At later date replace 1TB drives with more 8TB ones and grow it all > in place to two mirrors of 8TB usable each. > > Or start again with ZFS and make the equivalent stripe of two mirror > vdevs that again is only two sets of 1TB until the small drives are > replaced and it can be grown in place. > > Now, if you immediately want to obtain more capacity you could do it by > making separate arrays, i.e. keep the 1TB mirror and make an 8TB mirror > for a total 9TB usable. Obviously it's a pain having to manually decide > where to put data. > > If you absolutely want the capacity and you absolutely must have it in > one array, you could do btrfs. It is great for making redundant use of > mixed size drives. With a "raid1" profile and the drive sizes described > you would get 9TB usable. The performance won't be predictable though > because you won't be able to control where the extents for the files go. > > You would also be able to replace the smaller drives later on to get > higher usable capacity. > > Here is a capacity calculator: > > > https://carfax.org.uk/btrfs-usage/?c=2&slo=1&shi=1&p=0&dg=1&d=8000&d=8000&d=1000&d=1000 > > > On Fri, Mar 21, 2025 at 05:11:19PM -0400, Boyan Penkov wrote: > > If someone who has a firm grasp of the current state of btrfs vs bcachefs > > for "power users who still need to get stuff other than 'administrativia' > > done" would speak up, I'd be glad for some perspective as well... > > bcachefs is not ready for production use and its upstream author > specifically has asked Debian users not to use it as they want only > people who use the latest userland tools to test it out and report > bugs. > > > https://lore.kernel.org/all/36xhap5tafvm4boiy3acu5kxlhkvnp32wp3oknbfbkxbdkeq7r@galecvidi3bn/ > https://jonathancarter.org/2024/08/29/orphaning-bcachefs-tools-in-debian/ > > Thanks, > Andy > > -- > https://bitfolk.com/ -- No-nonsense VPS hosting > -- Boyan Penkov
