I have an ext4 file system on a raid5 array, /dev/md0 built from mdadm.
I found it wasn't mounted after I reboot once. And I ran a fcsk on it but get a superblock error.
So I was going to run mke2fs -n /dev/md0 to get the information of backup superblock. But I ran mke2fs /dev/md0 by mistake. I pressed Ctrl-C while the first time I was asked for inputing something about journal. But I still remember it was said that inode table was written.
After that, I ran mke2fs -S /dev/md0 to generate the superblock only again since I couldn't recover it from backups.
Now, the superblock problem disappeared, but fsck want to fix tons of inode problems. I answered yes for some of them and pressed Ctrl-C when I realized that it seems trying to fix the inode to a fresh new inode table.
Is it still possible to get all my data back? If yes, how?
Here is the raid information:
$ sudo mdadm -D /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Mon Jun 27 18:02:54 2022
Raid Level : raid5
Array Size : 7813707776 (7451.73 GiB 8001.24 GB)
Used Dev Size : 3906853888 (3725.87 GiB 4000.62 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Mon Jun 27 20:20:25 2022
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1
Layout : left-symmetric
Chunk Size : 64K
Consistency Policy : bitmap
Rebuild Status : 2% complete
Name : plq-pi:0 (local to host plq-pi)
UUID : a2250592:81ebb835:4646cf4f:64feea42
Events : 854
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 17 1 active sync /dev/sdb1
3 254 0 2 spare rebuilding /dev/dm-0
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 dm-0[3] sdb1[1] sdc1[0]
7813707776 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/2] [UU_]
[>....................] recovery = 2.0% (79757168/3906853888) finish=6664.4min speed=9570K/sec
bitmap: 0/30 pages [0KB], 65536KB chunk
unused devices: <none>
And the /dev/dm-0 is a lvm constructed by two disks.
The commands I executed:
$ sudo fsck /dev/md0
fsck from util-linux 2.33.1
e2fsck 1.44.5 (15-Dec-2018)
ext2fs_open2: Bad magic number in super-block
fsck.ext4: Superblock invalid, trying backup blocks...
fsck.ext4: Bad magic number in super-block while trying to open /dev/md0
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
$ sudo mkfs.ext4 /dev/md0
mke2fs 1.44.5 (15-Dec-2018)
Creating filesystem with 1835008000 4k blocks and 229376000 inodes
Filesystem UUID: cca2e089-18bf-4d19-a5c2-077b47304fd2
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544
Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): ^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C
$ sudo mkfs.ext4 -n /dev/md0
mke2fs 1.44.5 (15-Dec-2018)
Creating filesystem with 1835008000 4k blocks and 229376000 inodes
Filesystem UUID: 722ca4ca-1cf7-4860-baff-91a019258b3f
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000, 214990848, 512000000, 550731776, 644972544
$ sudo e2fsck -b 1934917632 /dev/md0
e2fsck 1.44.5 (15-Dec-2018)
e2fsck: Invalid argument while trying to open /dev/md0
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
$ sudo e2fsck /dev/md0
e2fsck 1.44.5 (15-Dec-2018)
/dev/md0 contains a file system with errors, check forced.
Resize inode not valid. Recreate<y>? yes
Pass 1: Checking inodes, blocks, and sizes
Inode 1 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Root inode is not a directory. Clear<y>? yes
Inode 17 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 33 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 49 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 65 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 81 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 97 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 113 has EXTENTS_FL flag set on filesystem without extents support.
Clear ('a' enables 'yes' to all) <y>? yes
yyyyyyInode 129 has EXTENTS_FL flag set on filesystem without extents support.
Clear ('a' enables 'yes' to all) <y>? yes
Inode 145 has EXTENTS_FL flag set on filesystem without extents support.
Clear ('a' enables 'yes' to all) <y>? yes
Inode 161 has EXTENTS_FL flag set on filesystem without extents support.
Clear ('a' enables 'yes' to all) <y>? yes
Inode 177 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 193 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 209 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes
Inode 225 has EXTENTS_FL flag set on filesystem without extents support.
Clear<y>? yes to all
Inode 497 has EXTENTS_FL flag set on filesystem without extents support.
Clear? yes
... more these ...
Inode 4840 is in use, but has dtime set. Fix? yes
Inode 4840 has a extra size (6436) which is invalid
Fix? yes
Timestamp(s) on inode 4840 beyond 2310-04-04 are likely pre-1970.
Fix? yes
^CInode 4840 has a bad extended attribute block 1879399463. Clear? yes
/dev/md0: e2fsck canceled.
/dev/md0: ***** FILE SYSTEM WAS MODIFIED *****
/dev/md0: ********** WARNING: Filesystem still has errors **********
$ sudo e2fsck -f /dev/md0
e2fsck 1.44.5 (15-Dec-2018)
Resize inode not valid. Recreate<y>? yes
Pass 1: Checking inodes, blocks, and sizes
Root inode is not a directory. Clear<y>?
/dev/md0: e2fsck canceled.
/dev/md0: ***** FILE SYSTEM WAS MODIFIED *****
/dev/md0: ********** WARNING: Filesystem still has errors **********
