Flexible fileserver on Debian Stretch
Finally getting around to reinstall my fileserver, I wanted to make it flexible with data integrity.
ZFS wasn’t flexible enough for my tastes, and LVM has no integrity check, so I had to create something with btrfs.
Early tests with using btrfs on LVM for root and /boot made the virtual machine unbootable, but using btrfs directly was no problem.
This meant I still had to use separate partitions for /boot, root, and swap (on mdraid10), and then use the rest for LVM.
It seems the installation of Stretch has trouble when adding raid or lvm afterwards, so I partitioned the installation this way:
vda1, 512M, bootable, btrfs, /boot
vda2, 4096M, btrfs, /
vda3, 512M, device for raid (to be made swap after creating raid10 after installation)
vg0: vda4, vdb4, vdc4
tmp_0, 512M, btrfs, /tmp
home_0, 1024M, btrfs, /home
var_0, 1024M, btrfs, /var
After installation I created the same partitions for vdb and vdc, then created the mdraid10 for swap:
mdadm –create –run /dev/md99 –level=10 –layout=n2 raid-devices=3 /dev/vda3 /dev/vdb3 /dev/vdc3
With that done I exchanged all /dev/mapper devices for the UUID’s of the btrfs devices in /etc/fstab and did a reboot as test.
With everything running I created logical volumes tmp_1, tmp_2, home_1, home_2, etc. by specifying the different physical devices to be put on:
lvcreate -L 512M -n tmp_1 vg0 /dev/vdb4
lvcreate -L 512M -n tmp_2 vg0 /dev/vdc4
Adding the LV’s to btrfs:
btrfs device add -f /dev/mapper/vg0-tmp_1 /dev/mapper/vg0-tmp_2 /tmp
btrfs device add -f /dev/mapper/vg0-home_1 /dev/mapper/vg0-home_2 /home
btrfs balance start -dconvert=raid1 -mconvert=raid1 /tmp
btrfs balance start -dconvert=raid1 -mconvert=raid1 /home
For unimportant data I made a logical volume striped across the three disks:
lvcreate -L xxG -n roraid -i 3 vg0
mkfs.ext4 /dev/mapper/vg0-noraid (could also be btrfs but the data integrity isn’t important here.
..and added its UUID to fstab with mountpoint /data/noraid
For the data protected with checksum integrity:
create three LV’s raid1_0, 1, 2 on the three PV’s like /tmp, /home, etc. above and make another btrfs raid of these.
Last, don’t forget to grub-install /dev/vdb and /dev/vdc.
The /data/raid1 mount point caused an error after first reboot, but later no more. Maybe a glitch in the matrix..
With this setup I can expand /tmp, /home, /var, /data/noraid and /data/raid1 as needed since I don’t know if I get more data for raid or not.
With two fileservers I can keep a backup of the raid1 data of one server on the noraid volume of the other. The btrfs volumes only protect from bit rot after all.
And if it really fills up I can start exchanging the disks for bigger ones.