0

I just got a CENTOS new machine setup. It has 2x SSD drives for linux, and 4x 12TB SAS drives for extra storage.

I setup a software RAID10 array on the 4x 12TB drives, file system ext4. Obviously it had to resync the drives, as it's a new RAID array. Eventhough the drives are totally empty.

It's been resync'ing for about 15 hours now, and it's only done 14%.

    Every 2.0s: cat /proc/mdstat                                                                                                                                                                                         Tue Oct  8 05:28:01 2019

Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath] [faulty]
md4 : active raid10 sdf1[3] sde1[2] sdd1[1] sdc1[0]
      23437503488 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      [==>..................]  resync = 14.3% (3369612096/23437503488) finish=12927.6min speed=25872K/sec
      bitmap: 151/175 pages [604KB], 65536KB chunk

md2 : active raid1 sda2[0] sdb2[1]
      20478912 blocks [2/2] [UU]

md3 : active raid1 sdb3[1] sda3[0]
      447318976 blocks [2/2] [UU]
      bitmap: 0/4 pages [0KB], 65536KB chunk

unused devices: <none>

The machine is not being used for anything else at the moment. The resync speed has gone down a lot. It started around 100,000k/s, but it's currently sitting around 20,000k/s.

It will take me like a week to get these drives ready at this speed, if it doesn't get even slower.

Is there anything I can do to speed things up?

CPU is hardly being used, and plenty of RAM available too.

Improve this question

1 Answer 1

Reset to default
2

You will be editing your sysctl.conf file which holds many attributes for the Linux system. As always, and before editing any configuration file, you should take a backup.

cp /etc/sysctl.conf /etc/sysctl.conf_ORIG

Open the sysctl.conf file with a text editor.

nano /etc/sysctl.conf

Find or add the below parameter and set the value to the speed in KB/s which you would like to use. The below example sets the speed limit to 50 MB/s.

dev.raid.speed_limit_max = 51200

You will then need to load the settings using the sysctl command.

/sbin/sysctl -p

Add bitmap indexes to mdadm Adding a bitmap index to a mdadm before rebuilding the array can dramatically speed up the rebuild process.

Use the below command to add a bitmap index to an array. The example assumes your array is found at /dev/md0.

mdadm --grow --bitmap=internal /dev/md0

Once the process has completed, use the below command to remove the mdadm bitmap index. Again, this example assumes your array is found at /dev/md0.

mdadm --grow --bitmap=none /dev/md0
Improve this answer
4
  • 2
    It seems you messed things. Bitmap lets you to track on each disk separately which blocks are changed and weren't replicated to other disks. This is useful if you remove disk, then reinsert it after some time, because it will only sync that changed blocks. But maintaining a bitmap by itself slows disk, because it causes heads to jump between data track and bitmap track. This is why it is recommended to do initial sync and full disk rebuilds WITHOUT any bitmap, and add if afterwards, when sync is completed. Alternatively bitmap could be placed on external fast device, for example SSD.
    – Nikita Kipriyanov
    Oct 8, 2019 at 5:57
  • Thanks guys. So should I add the bitmap index to an array like djdomi suggested or not?
    – Mr.Boon
    Oct 8, 2019 at 6:30
  • 1
    i added the part of Bitmap due to the rebuild, sorry if it was not clean written. ususally the initiativ build should be done without as i do ususally
    – djdomi
    Oct 8, 2019 at 17:34
  • @mr.boon if it helped you i would be glad if you accept this answer
    – djdomi
    Oct 10, 2019 at 8:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .