ITjes en datjes

Dingen uit dagelijks IT werk

Archive for the ‘storage’ Category

Flexible fileserver on Debian Stretch

Finally getting around to reinstall my fileserver, I wanted to make it flexible with data integrity.
ZFS wasn’t flexible enough for my tastes, and LVM has no integrity check, so I had to create something with btrfs.

Early tests with using btrfs on LVM for root and /boot made the virtual machine unbootable, but using btrfs directly was no problem.
This meant I still had to use separate partitions for /boot, root, and swap (on mdraid10), and then use the rest for LVM.

It seems the installation of Stretch has trouble when adding raid or lvm afterwards, so I partitioned the installation this way:

vda1, 512M, bootable, btrfs, /boot
vda2, 4096M, btrfs, /
vda3, 512M, device for raid (to be made swap after creating raid10 after installation)
vda4, LVM
vg0: vda4, vdb4, vdc4
tmp_0, 512M, btrfs, /tmp
home_0, 1024M, btrfs, /home
var_0, 1024M, btrfs, /var

After installation I created the same partitions for vdb and vdc, then created the mdraid10 for swap:
mdadm –create –run /dev/md99 –level=10 –layout=n2 raid-devices=3 /dev/vda3 /dev/vdb3 /dev/vdc3
mkswap /dev/md99

With that done I exchanged all /dev/mapper devices for the UUID’s of the btrfs devices in /etc/fstab and did a reboot as test.

With everything running I created logical volumes tmp_1, tmp_2, home_1, home_2, etc. by specifying the different physical devices to be put on:
lvcreate -L 512M -n tmp_1 vg0 /dev/vdb4
lvcreate -L 512M -n tmp_2 vg0 /dev/vdc4

Adding the LV’s to btrfs:
btrfs device add -f /dev/mapper/vg0-tmp_1 /dev/mapper/vg0-tmp_2 /tmp
btrfs device add -f /dev/mapper/vg0-home_1 /dev/mapper/vg0-home_2 /home

btrfs balance start -dconvert=raid1 -mconvert=raid1 /tmp
btrfs balance start -dconvert=raid1 -mconvert=raid1 /home

For unimportant data I made a logical volume striped across the three disks:
lvcreate -L xxG -n roraid -i 3 vg0
mkfs.ext4 /dev/mapper/vg0-noraid (could also be btrfs but the data integrity isn’t important here.
..and added its UUID to fstab with mountpoint /data/noraid

For the data protected with checksum integrity:
create three LV’s raid1_0, 1, 2 on the three PV’s like /tmp, /home, etc. above and make another btrfs raid of these.

Last, don’t forget to grub-install /dev/vdb and /dev/vdc.

The /data/raid1 mount point caused an error after first reboot, but later no more. Maybe a glitch in the matrix..

With this setup I can expand /tmp, /home, /var, /data/noraid and /data/raid1 as needed since I don’t know if I get more data for raid or not.
With two fileservers I can keep a backup of the raid1 data of one server on the noraid volume of the other. The btrfs volumes only protect from bit rot after all.

And if it really fills up I can start exchanging the disks for bigger ones.



Written by mnystrom

2016/04/04 at 22:20

Thoughts about file location and performance

MyDefrag system scripts put the MFT$ and directories at one third of the size of the disk for better access performance.
Creating a zone of frequently accessed files together also serves as a performance enhancer so the disk takes up less time to get from one file to the other.

Looking at recently accessed files on my laptop show mostly browser cache in /home and system logfiles in /var. The OS files would only be used once at startup, binaries only when starting a program. For a single user system it seems there’s no benefit to optimising the location of files on disk beyond keeping /var and /home close together on disk. (done by using separate partitions for these and using a separate partition for data to keep that out of /home)

On a file server it might benefit to create a data partition of about a third the size of the disk, then a partition for /var, then another partition for data (combining both data partitions through LVM). Then the disk arm would be closer to the files on average, reducing wear and tear.

On a multi user server create the /home partition after / and /var and link the user’s data directories in the profile to a separate data partition.
I’ve done this with Windows terminal servers where I used a separate profiles partition (can be done by modifying the system and moving files in save mode, then creating links of the original directories) after the OS, swap and program files partitions and using policies to link document directories to the the file server (had to be done anyway, otherwise it would have been a larger data partition).
One of the biggest benefits was preventing the OS partition of filling up by users (even local system accounts).
Too bad the Windows profiles directory can’t be linked to a network location like on a *nix machine, creating a NFS link to /home on another server.

Written by mnystrom

2015/06/06 at 18:10

Geplaatst in linux, storage

Revive old home server

After replacing the failing home server with a decent Synology NAS I got my hands on a HP Storageworks Data Vault X510.

Being curious I wanted to see if I could get it to run with Debian so I could do some tests with mdadm.

Turns out it’s not too difficult.

-place the harddisk to install it on in a standard PC and use that to install the i386 version of Debian (in this case it was Jessie)
-add a fixed IP address for eth0 in /etc/networking/interfaces (the Realtek r8168 (seen as r8169) couldn’t get a DHCP address for some reason)
-place the disk in the X510 and boot
-login with SSH (you did install it, didn’t you?)
-have fun with it


Written by mnystrom

2015/06/03 at 20:41

Geplaatst in Debian, EN, hardware, installation, linux, server, storage

Tagged with , ,

ff onthouden; diverse dingetjes

Telnet test mail;

telnet 25
Trying ip adress…
Connected to
Escape character is ‘^]’.
220 ESMTP provider mailservice ready
501 Syntax: HELO hostname
helo svmbitz01001.mbitz01.local
mail from: user@svmbitz01001.mbitz01.local
250 2.1.0 Ok
250 2.1.5 Ok
354 End data with <CR><LF>.<CR><LF>
subject:test delivery mail
insert body text here
250 2.0.0 Ok: queued as 96582184093
221 2.0.0 Bye
Connection closed by foreign host.

ATF software;

Fout xxxx.xx is not a floating point number: landinstellingen aanpassen en de punt als decimaal teken zetten.

Brother printers;

Drum reset als dit na vervanging niet gedaan is: druk 3 + 9 tegelijk in, bevestig in het menu de drum reset.

Drakon: visual programming language, flowchart programmeering, handig voor opzet van processen en basis voor software schrijven

Snapraid: semi raid en backup optie voor opslag (*nix, Windows, OSX)


Written by mnystrom

2012/05/22 at 20:18

Windows Home server toegang zonder wachtwoord

Iemand dacht dat het een goed idee was om een HP Data Vault X510 als een backup apparaat te gebruiken voor een Windows 2003 SBS server. Nou, niet echt het ding wat ik zou kiezen maar aangezien het er al stond ging ik het toch maar gebruiken.

Eerste probleem was niet weten wat het Administrator wachtwoord was, en natuurlijk was er geen documentatie over wat dat ding deed. :-/
Tweede probleem was de vermiste recovery disc. Geen disc, geen optie om de server te herstellen en nieuw wachtwoord toe te wijzen. Tijd voor wat ouderwets hakwerk.
Als eerste de Hirens boot cd gebruikt om het administrator wachtwoord te verwijderen maar dat is niet voldoende aangezien Windows geen externe toegang verleent zonder een wachtwoord ingesteld te hebben. Ik trok de eerste schijf eruit, sloot deze aan op een andere Windows machine en gebruikte regedit om de SOFTWARE hive te laden uit (sysdrive):\windows\system32\config. Daar voegde ik de DWORD waarde IsRemoteAccessUser toe aan (hivenaam)\Microsoft\Windows\Windows Home Server\User Manager\Users\Administrator en wijzigde deze in 1. Op deze manier kon ik Externe Bureaublad gebruiken om een verbinding the maken met de Data Vault.
Om werkelijk in te kunnen loggen moest ik ook de SYSTEM hive laden en LimitBlankPasswordUse wijzigen in 0 onder (hivenaam)\ControlSet001\Control\Lsa en ControlSet003 (eventueel 002).

Na de schijf terug gestoken te hebben en de server gestart had kon ik inloggen en de boel recht zetten.


Written by mnystrom

2012/04/25 at 21:56

Geplaatst in hardware, NL, server, storage

Tagged with , ,

Windows Home server access without a password

Someone thought it was a good idea to use an HP Data Vault X510 as a backup device for a Windows 2003 SBS server. Well, not really the thing I’d choose but as it was already there I was going to use it.

First problem was not knowing the Administrator password, and of course no documentation on anything that thing does. :-/
Second problem was the missing recovery disc. No disc, no restoring the server and assigning a new password. Time for some oldfashioned hacking.
First off used the Hirens boot cd to blank the administrator password but that wasn’t enough to gain access as Windows doesn’t allow remote access without a password set. I pulled the first disk, hooked it up to another Windows machine and used regedit to load the SOFTWARE hive from (sysdrive):\windows\system32\config. There I added the DWORD value IsRemoteAccessUser to (hivename)\Microsoft\Windows Home Server\User Manager\Users\Administrator and set it to 1. This way I would be able to use Remote Desktop to connect to the Data Vault.
To actually be able to login I also needed to load the SYSTEM hive and change LimitBlankPasswordUse to 0 in (hivename)\ControlSet001\Control\Lsa and ControlSet003 (or 002).

After plugging the disk back in and booting up I was able to log into the server and set things right.

Written by mnystrom

2012/04/25 at 21:37

Geplaatst in EN, hardware, server, storage

Tagged with , ,