I run a CentOS system with 2x SSD in RAID1 and 4X SAS drives in RAID10. Linux is installed on the RAID1 array.
This morning I noticed that my RAID10 array (md4) had gone missing, and that my RAID1 array (m2/m3) was broken. I ran a smartctl on all the drives, and they all report to be OK.
I checked the raid array:
[root@m1 etc]# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] [multipath] [faulty]
md4 : inactive sdh1[2](S) sdg1[3](S) sdf1[1](S) sde1[0](S)
46875008958 blocks super 1.2
md3 : active raid1 sdb3[1]
447318976 blocks [2/1] [_U]
bitmap: 2/4 pages [8KB], 65536KB chunk
md2 : active raid1 sdb2[1]
20478912 blocks [2/1] [_U]
unused devices: <none>
Both md2 and md3 are not in good shape it seems, and my md127 is now called m4 somehow, and is set to inactive.
Many folders on the filesystem I cannot access anymore, such as /home and /etc, but there are many more. When reading out such a directory I get this error:
[root@m1 etc]# dir
-bash: /usr/bin/dir: Input/output error
Inspecting md3 shows:
[root@m1 etc]# mdadm --detail /dev/md3
/dev/md3:
Version : 0.90
Creation Time : Mon Oct 7 10:46:06 2019
Raid Level : raid1
Array Size : 447318976 (426.60 GiB 458.05 GB)
Used Dev Size : 447318976 (426.60 GiB 458.05 GB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 3
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Mon Oct 21 01:59:57 2019
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0
Consistency Policy : bitmap
Number Major Minor RaidDevice State
- 0 0 0 removed
1 8 19 1 active sync
But using examine shows:
[root@m1 etc]# mdadm --examine /dev/md3
mdadm: No md superblock detected on /dev/md3.
Because /etc for example is not accessible, many important system files cannot be accessed, which is maybe why my md127 array is also gone missing. I think the first step should be fixing md2/md3. As it's RAID1, data should be in intact on at least 1 of the 2 drives I think.
How can I restore those md2 and md3 raid arrays so I get my data back?
