Skip to content

Convert a running Linux system to Software RAID

Last amended: 2024-09-21

Introduction

My main workstation is running Linux and is hosting valuable and partly sensitive data. In order to improve data security I want to turn my one disk Debian Jessie system into a two disk RAID 1 machine.

I have gained very good experience of my FreeBSD machines (native FreeBSD and FreeNAS) all running mirrored ZFS pools which consist of two disks each (see this link for further details).

However, under Linux both ZFS and BTRFS are not yet stable enough to host important data. In my opinion this is particularly true when using RAID and encryption. Therefore I decided to use mature and reliable techniques like Linux software RAID (aka MD RAID), dm-crypt/LUKS and ext4.

Initially my system used GPT partion layout with separate home and swap partitions. Both home and swap are encrypted. After the conversion I will have three RAID 1 arrays:

  • /dev/md0 for the root file system
  • /dev/md1 for an encrypted home partition
  • /dev/md2 for encrypted swap

Although I am going to use mature techniques only, it has been hard to find exactly matching information. Existing documents are outdated, because they refer to Grub1 (aka Grub Legacy) instead to Grub2 and because they do use MBR partition layout instead of GPT. Furthermore, existing information suggests using LVM2 with RAID, and describes full disk encryption instead of partial encryption. This is the reason why I have decided to write this blog article. Nevertheless, the following three references have been helpful to achieve my RAID 1 conversion:

In the following detailed description we assume that we have a running Debian Jessie system booted from /dev/sda and a second empty disk /dev/sdb with a capacity greater or equal of /dev/sda. The system is booted in Legacy BIOS mode with GPT partiton layout on /dev/sda.

Create a full backup of your system

The whole process described below works without losing any data. But it is good practice to have a full backup available just in case. I did the full backup but fortunately I have not needed it.

Install needed software

On my system everything execpt mdadm has already been installed. Use defaults for mdadm:

apt-get install mdadm initramfs-tools cryptsetup

The defaults can easily be changed on demand in the configuration file /etc/default/mdadm. Load required kernel modules to avoid a reboot:

modprobe raid1

Prepare the new disk

The partition layouts of the new and the existing disk have to match exactly. The easiest way to achieve this is to use sgdisk to copy the partition layout and subsequently adapt GPT names and partition types:

sgdisk -R=/dev/sdb /dev/sda
sgdisk -G /dev/sdb
sgdisk --change-name=1:gptboot1 /dev/sdb
sgdisk --change-name=2:root1 /dev/sdb
sgdisk --change-name=3:home1 /dev/sdb
sgdisk --change-name=4:swap1 /dev/sdb
sgdisk --type-code=2:FD00 /dev/sdb
sgdisk --type-code=3:FD00 /dev/sdb
sgdisk --type-code=4:FD00 /dev/sdb

Current partiton layout of the new disk:

sgdisk -p /dev/sdb
...
Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           22527   10.0 MiB    EF02  gptboot1
   2           22528      1363175423   650.0 GiB   FD00  root1
   3      1363175424      1916823551   264.0 GiB   FD00  home1
   4      1916823552      1952475135   17.0 GiB    FD00  swap1

Create three degraded RAID 1 arrays on the new disk

"Degraded" means that each of the RAID 1 arrays consists of one partition only but is prepared to get added the second partition later:

mdadm --zero-superblock /dev/sdb2
mdadm --zero-superblock /dev/sdb3
mdadm --zero-superblock /dev/sdb4
mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/sdb2
mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb3
mdadm --create /dev/md2 --level=1 --raid-disks=2 missing /dev/sdb4

You may ignore any warning about metadata versions. In spite of intensive search I have not been able to find any information about what version is required by Grub2. I finally decided to continue with the default metadata version 1.2 which works well with Grub2.

Check the current state of the arrays:

cat /proc/mdstat

In the output of the above command the "_U" indicates the degraded state of the three arrays. Finally, create a configuration file for mdadm:

mdadm --examine --scan >> /etc/mdadm/mdadm.conf

Linux software RAID reserves some space (called superblock) on every component of a RAID array. This space contains metadata about the RAID array and allows correct assembly of the array. You can check e.g. the metadata version of the components /dev/sda2 and /dev/sdb2 of the RAID array /dev/md0:

mdadm --examine /dev/sda2
mdadm --examine /dev/sdb2

Currently, however, RAID array /dev/md0 is degraded and therefore the first command will fail. You can find further details about the RAID superblock and metadata versions here and here.

Mount the degraded RAID array for the root file system

mkdir /mnt/md0
mount /dev/md0 /mnt/md0

Install Grub2 to the new disk

grub-install --target=i386-pc --debug /dev/sdb

Update initial RAM file system

We need a new initial RAM file system containing the modules raid1 and md_mod in order to make /dev/md0 a bootable root file system:

update-initramfs -uv

Copy the root file system to the degraded RAID array

The root file system will now be copied to the RAID array /dev/md0 mounted under /mnt/md0:

cd /
umount /home
cp --preserve=xattr -dpRxv . /mnt/md0/

Debian Jessie uses extended attributes to the root file system. These are preserved with --preserve=xattr. This could have been achieved with rsync and corresponding options as well. But be careful: Unfortunately, GNU tar silently drops extended attributes even if you explicitely ask tar to preserve them. The ping binary e.g. has extended attributes and you can easily check them on the old and the new root file system:

getcap /bin/ping
getcap /mnt/md0/bin/ping

Finally adapt /mnt/md0/etc/fstab to use /dev/md0 as the new root file system:

/dev/md0  /  ext4 errors=remount-ro  0  1

Take care that you make this change on the root RAID array /dev/md0 and not on the original root file system /dev/sda2.

First boot from the degraded RAID array

Reboot the system

shutdown -r now

and interactively edit the Grub2 boot menu to change the disk name with the root file system and the partition of the root file system:

set root='hd1,gpt2'
linux /boot/... root=/dev/md0 ...

Proceed to boot with these changes.

During my tests in a VirtualBox virtual machine Grub2 sometimes did not find the root file system. In this case you are dropped into initramfs prompt with an error message pointing to some possible timing problem. In this case add rootdelay=3 to the kernel boot parameters. With my real hardware I have kept this setting to be on the safe side. I have not yet investigated, whether it is a VirtualBox specific issue. Unfortunately, I have lost a lot of time, because as a typical timing problem this error occurs only occasionally.

Check, that you have currently mounted the root file system on /dev/md0:

mount |grep md0

If everything has worked properly so far, you are on the new root file system while having the original root file system fully working in place.

Encrypted swap on the degraded RAID array

I am using a partly encrypted system where an encrypted swap partition enhances data security. There are generally two ways to combine MD RAID and dm-crypt/LUKS:

  • Encrypt the RAID array
  • Build the RAID array on encrypted devices

The pros and cons are discussed here and I have decided to use the first option.

Encrypt /dev/md2 using an arbitrary initial passphrase:

cryptsetup luksFormat /dev/md2

On every reboot the swap will be encrypted with a random passphrase.

Open the encrypted RAID array and create the swap file system:

cryptsetup luksOpen /dev/md2 crypt.md2.luks
mkswap /dev/mapper/crypt.md2.luks

Adapt /etc/crypttab:

crypt.md2.luks  /dev/md2  /dev/urandom  swap,cipher=aes-cbc-essiv:sha256,size=256

Now change the swap entry in /etc/fstab:

/dev/mapper/crypt.md2.luks  none  swap  sw  0  0

Enable and check encrypted swap:

swapoff -a
swapon -a
swapon -s

The first command turns off the swap usage on the first disk, e.g. the usage of /dev/sda4 or - like on my system - its encrypted counterpart.

Encrypt degraded RAID array for the new home partition

Encrypt the RAID array /dev/md1 for new /home:

cryptsetup -v --cipher aes-xts-plain64:sha256 --key-size 512 luksFormat /dev/md1
cryptsetup luksDump /dev/md1

Open the encrypted LUKS device /dev/md1 and create a file system on it: 

cryptsetup luksOpen /dev/md1 crypt.md1.luks
mke2fs -t ext4 /dev/mapper/crypt.md1.luks

Mount encrypted and degraded RAID array for the new home partition

mkdir /mnt/md1
mount /dev/mapper/crypt.md1.luks /mnt/md1

Copy the home partiton to the encrypted and degraded RAID array

The home partition will now be copied from /home to the the RAID array /dev/md1 mounted under /mnt/md1:

mount /home
cd /home
cp --preserve=xattr -dpRxv . /mnt/md1/
umount /home

I do manually mount my encrypted home partition. Accordingly, I do not need any entry for it in /etc/crypttab. I adapt /etc/fstab:

/dev/mapper/crypt.md1.luks  /home  ext4  noauto  0  0

Mount the home partition on the encrypted and degraded RAID array according to the new /etc/fstab entry:

umount /mnt/md1
mount /home

Add the partitions of the old disk to the degraded RAID arrays

Before you proceed you should have a short break. Until now you have had your whole initial system still on disk /dev/sda. Please make sure, if you have completely copied all data to the three degraded RAID arrays so far. The next steps will delete all data of you initial system.

Adapt the GPT names and the partition types on the old disk /dev/sda:

sgdisk --change-name=1:gptboot0 /dev/sda
sgdisk --change-name=2:root0 /dev/sda
sgdisk --change-name=3:home0 /dev/sda
sgdisk --change-name=4:swap0 /dev/sda
sgdisk --type-code=2:FD00 /dev/sda
sgdisk --type-code=3:FD00 /dev/sda
sgdisk --type-code=4:FD00 /dev/sda

Now add the partitons of the old disk /dev/sda to the three degraded RAID arrays:

mdadm --add /dev/md0 /dev/sda2
mdadm --add /dev/md1 /dev/sda3
mdadm --add /dev/md2 /dev/sda4

Observe the resync aka resilver process:

cat /proc/mdstat

Show details of the three RAID 1 arrays:

mdadm --detail /dev/md0
mdadm --detail /dev/md1
mdadm --detail /dev/md2

Install Grub2 to the old disk

It is not neccessary but safe to wait until the resync process has finished. Now install Grub2 on the old disk:

grub-install --target=i386-pc --debug /dev/sda

Create a new Grub2 configuration file

If you have not yet added the rootdelay=3 parameter to /etc/default/grub this is the right time to do so. Furthermore, a new Grub2 configuration file is needed to set the root file system correctly:

update-grub

You can play with the rootdelay parameter to see if it is needed at all and what delay is right for your hardware. I will do the same and I will update the blog entry with my results.

The entire system is converted to RAID 1 now. You should try to reboot now to see if everything is working as expected.

Conclusion and feedback

The whole process described is very straightforeward. It has taken me one afternoon of reading, making a plan and creating a full system backup.

And a second afternoon was required to carry out the conversion to RAID 1. The most of the time turned out to be the resync process. Here are some details:

  • Root file system on /dev/md0: 1 hour for 300 GiB of unencrypted data
  • Home on /dev/md1: 40 minutes for 100 GiB of dm-crypt/LUKS encrypted data

These times strongly depend on the specific hard disk used. In my case these times are achieved with 2 disks of Seagate Constellation ES.3 each having 1 TiB of capacity.

As of this writing my system is running on RAID 1 for about one month without any issues. I have done my best to describe the process of RAID 1 conversion very carefully according to my experience but I cannot assume liability for the described process.

I appreciate any feedback on the blog entry. Feel free to contact me and please let me know, whether you would like to be mentioned by name and/or email in this blog with your proposed changes or additions.