I have a RAID5 array that I tried to add a disk on to grow the array and it appears to be stuck. I have found all kind of threads on similar issues, but can't seem to locate a clear solution on what to do.
Basically, I did this:
mdadm --grow --raid-devices=6 --backup-file=/root/grow_md0.bak /dev/md0
Now, I get:
[root@server ~]# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdg1[7] sde1[3] sdd1[5] sdc1[6] sda1[2] sdb1[0]
15627540480 blocks super 1.2 level 5, 512k chunk, algorithm 2 [6/6] [UUUUUU]
[>....................] reshape = 0.0% (4096/3906885120) finish=362200428.2min speed=0K/sec
Notice the 0k/sec speed. Now, the 4096 was originally 1024. I tried this based on some suggestions:
mdadm --grow --continue --backup-file=/root/grow_md0.bak /dev/md0
Which bumped the 1024 up to 2048, but did nothing else.
I also noticed at this point that /root/grow_md0.bak doesn't even exist. A couple other "continue" attempts got me up to 4096 but nothing more.
Other info:
[root@server ~]# cat /sys/block/md0/md/stripe_cache_size
643
[root@server ~]# cat /sys/block/md0/md/sync_min
0
[root@server ~]# cat /sys/block/md0/md/sync_max
8192
[root@server ~]# cat /sys/block/md0/md/sync_speed
0
[root@server ~]# cat /proc/sys/dev/raid/speed_limit_min
50000
[root@server ~]# cat /proc/sys/dev/raid/speed_limit_max
200000
[root@server ~]# dmesg |grep md
[ 2734.805157] md: md_do_sync() got signal ... exiting
[ 2735.071850] md: reshape of RAID array md0
[ 2735.071854] md: minimum _guaranteed_ speed: 50000 KB/sec/disk.
[ 2735.071855] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reshape.
[ 2735.071860] md: using 128k window, over a total of 3906885120k.
[ 2766.527554] md: md_do_sync() got signal ... exiting
[ 2766.578718] md: reshape of RAID array md0
[ 2766.578722] md: minimum _guaranteed_ speed: 50000 KB/sec/disk.
[ 2766.578723] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reshape.
[ 2766.578728] md: using 128k window, over a total of 3906885120k.
[ 2950.824300] md: md_do_sync() got signal ... exiting
[ 2950.874411] md: reshape of RAID array md0
[ 2950.874418] md: minimum _guaranteed_ speed: 50000 KB/sec/disk.
[ 2950.874420] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reshape.
[ 2950.874427] md: using 128k window, over a total of 3906885120k.
[ 2974.287731] md: md_do_sync() got signal ... exiting
[ 2974.365615] md: reshape of RAID array md0
[ 2974.365620] md: minimum _guaranteed_ speed: 50000 KB/sec/disk.
[ 2974.365621] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reshape.
[ 2974.365625] md: using 128k window, over a total of 3906885120k.
Would changing something here help? I bumped up speed_limit_min to 50000, but that didn't seem to get me anywhere as it's stuck, not slow.
EDIT
I have tried updating the stripe_cache_size to 16384 to no avail.
