I have encountered unpleasant situation. Given mdadm RAID1 of 2 HDD on server. Yesterday one of HDDs completely dies, other shows bad SMART status but was in accessible state. Mdadm showed "clean, degraded".
I've replaced completely failed disk and rebuilt array with new one. After replication completed I'd replaced other disk and again replicated.
All this things have been done from rescue system as GRUB did not work since hdd crash. So after restoring first of all I've mounted md root partition manually to some folder. The good news was that files in root system was there. The bad thing was that there were only old ones (latest is about year ago).
How could this happen and what to do? Any ideas?