Increase mdadm raid rebuild speed

mdadm can take quite a while when rebuilding arrays and the more disks there are, the longer it will take. As an example, it took 5 x 2TB WD green disks 3 days to complete.

There are two main ways to increase the rebuild speed of a mdadm array which are detailed below. Remember that it doesn’t matter what options you use here – you can still be limited by your CPU, memory and hard disk channel bandwidth.

Remove the mdadm rebuild speed restriction

mdadm has a parameter which restricts the speed of the rebuild process on each array. The idea behind this is to stop mdadm from making the rest of your system unusable by guaranteeing that it doesn’t consume all the available resource and IO bandwidth. The attributes can be increased to raise the speed limit on mdadm and reduce the time it takes to complete the rebuild process.

You will be editing your sysctl.conf file which holds many attributes for the Linux system. As always, and before editing any configuration file, you should take a backup.

cp /etc/sysctl.conf /etc/sysctl.conf_ORIG

Open the sysctl.conf file with a text editor.

vi /etc/sysctl.conf

Find or add the below parameter and set the value to the speed in KB/s which you would like to use. The below example sets the speed limit to 50 MB/s.

dev.raid.speed_limit_max = 51200

You will then need to load the settings using the sysctl command.

/sbin/sysctl -p

Add bitmap indexes to mdadm

Adding a bitmap index to a mdadm before rebuilding the array can dramatically speed up the rebuild process.

Use the below command to add a bitmap index to an array. The example assumes your array is found at /dev/md0.

mdadm --grow --bitmap=internal /dev/md0

Once the process has completed, use the below command to remove the mdadm bitmap index. Again, this example assumes your array is found at /dev/md0.

mdadm --grow --bitmap=none /dev/md0

How to grow a mdadm RAID 5 array

mdadm --add /dev/md1 /dev/sdf1

mdadm --grow /dev/md1 --size=max

mdadm --grow --bitmap=none /dev/md1

watch cat /proc/mdstat

e2fsck -f /dev/md1
resize2fs /dev/md1
e2fsck -f /dev/md1

mount -a