I had a disk die in my level 5 RAID and before I could replace it, my RAID crapped out while I was copying some data onto it.
Reading the RAID kernel wiki on RAID recovery I've tried forcing an assemble with no success:
# mdadm --assemble --force /dev/md1 /dev/sda1 /dev/sdc1
mdadm: /dev/md1 assembled from 1 drive - not enough to start the array.
The results from mdadm --examine are:
/dev/sda1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 2f3274a9:e3975c44:43e0066f:3c4e819d
Name : Jupiter:1
Creation Time : Sun Aug 26 17:41:46 2012
Raid Level : raid5
Raid Devices : 3
Avail Dev Size : 3907025072 (1863.01 GiB 2000.40 GB)
Array Size : 3907023872 (3726.03 GiB 4000.79 GB)
Used Dev Size : 3907023872 (1863.01 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 6c242f92:9b39c56a:c238964e:82ed1fdd
Internal Bitmap : 8 sectors from superblock
Update Time : Sat Sep 21 11:20:47 2013
Checksum : e38c9e3b - correct
Events : 13880
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : .A. ('A' == active, '.' == missing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x3
Array UUID : 2f3274a9:e3975c44:43e0066f:3c4e819d
Name : Jupiter:1
Creation Time : Sun Aug 26 17:41:46 2012
Raid Level : raid5
Raid Devices : 3
Avail Dev Size : 3907025072 (1863.01 GiB 2000.40 GB)
Array Size : 3907023872 (3726.03 GiB 4000.79 GB)
Used Dev Size : 3907023872 (1863.01 GiB 2000.40 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Recovery Offset : 395033224 sectors
State : clean
Device UUID : 0aa93873:826723b0:5e73d226:dd8a8071
Internal Bitmap : 8 sectors from superblock
Update Time : Mon Aug 5 05:11:05 2013
Checksum : 8ddf93da - correct
Events : 13298
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAA ('A' == active, '.' == missing)
My initial thoughts from reading the wiki is that is because the Event count is so different mdadm can't reassemble the array.
I'm confused because I've been copying data over to the array successfully until now, so I don't know why the Event count would have drifted so far apart.
I desperately don't want to lose the data, but I need to obviously get the RAID going to then add a new third disk.
