FSCK run and unmount issue

Question

A partition /dev/sdb in web my server has failed FSCK check but I can't seem to figure out a way to repair it. OS is CentOS 6.8.

In /etc/fstab, I have commented out this partition.

I used the command umount /dev/sdb and it returned saying it is not mounted.

Also, I tried

root@server3 [/]# umount /dev/sdb1
umount: /dev/sdb1: not mounted

root@server3 [/]# fsck -A /dev/sdb1
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
/dev/sda5 is mounted.
e2fsck: Cannot continue, aborting.

File system is currently set as:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda5       221G  156G   55G  75% /
tmpfs            16G     0   16G   0% /dev/shm
/dev/sda1       504M   66M  413M  14% /boot
/dev/sda3       4.0G  139M  3.7G   4% /tmp
/dev/sdb2       219G   66G  142G  32% /wjgbk

FSTAB file looks like this:

#
# /etc/fstab
# Created by anaconda on Fri Oct 11 07:42:05 2013
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=8fec8cf7-4999-489c-a344-8b16ab7402e2       /       ext4    usrjquota=quota.user,jqfmt=vfsv0        1       1
UUID=7f2583f8-7d07-43a9-9f81-0f24a449e97a /boot                   ext4    defaults        1 2
#UUID=793f6375-bd81-4392-9cf7-adff4149b72c      /home2  ext4    usrjquota=quota.user,jqfmt=vfsv0        1       2
UUID=2e71e9e4-a05a-4282-9e48-a82a364ef3e8 /tmp                    ext4    defaults        1 2
UUID=7e4202fe-c53a-4475-bfb8-66271e08137f       /wjgbk  ext4    defaults        1       2
UUID=67f4cccf-fca2-46b8-8f4c-53af3512b7af swap                    swap    defaults        0 0
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/tmp             /var/tmp                    ext3    defaults,bind,noauto        0 0

And the UID's refer to the following:

root@server3 [/dev]# blkid
/dev/sdb1: UUID="793f6375-bd81-4392-9cf7-adff4149b72c" TYPE="ext4"
/dev/sdb2: UUID="7e4202fe-c53a-4475-bfb8-66271e08137f" TYPE="ext4"
/dev/sda1: UUID="7f2583f8-7d07-43a9-9f81-0f24a449e97a" TYPE="ext4"
/dev/sda2: UUID="67f4cccf-fca2-46b8-8f4c-53af3512b7af" TYPE="swap"
/dev/sda3: UUID="2e71e9e4-a05a-4282-9e48-a82a364ef3e8" TYPE="ext4"
/dev/sda5: UUID="8fec8cf7-4999-489c-a344-8b16ab7402e2" TYPE="ext4"

Any ideas on how to proceed with the FSCK check? I need to recover the data from this partition and move it all to a new server asap.

EDIT 1 Changed the fsck -A command from /dev/sdb to /dev/sdb1

EDIT 2 Changed the fsck -A command to fsck and here are the results:

    root@server3 [/]# fsck /dev/sdb1
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
/dev/sdb1 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Error reading block 59770103 (Attempt to read block from filesystem resulted in short read) while getting next inode from scan.  Ignore error<y>? yes

Force rewrite<y>? no

Pass 2: Checking directory structure
Entry '383294_502fe9d985d3b76501e4272957465134f1b4c4d4' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14394754.  Clear<y>? yes

Entry '383304_41397ee07938e4643f13371c7fe0dbbf830739b7' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14394764.  Clear<y>? yes

Entry '383293_f885e3b62c361401c18e54b626553ee3818a43f4' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14394753.  Clear<y>? yes

I realised this may be deleting the files so I said N eventually which resulted in:

    Entry '429064_247dc988326a61ac34725f538d91836904e39ae1' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14962043.  Clear<y>? no

Error reading block 59770103 (Attempt to read block from filesystem resulted in short read).  Ignore error<y>? no

ext2fs_read_inode: Attempt to read block from filesystem resulted in short read while reading inode 14962043 in check_filetype
e2fsck: aborted

There are 10000's of files in the attachments folder. What would be the best way to tackle this?

EDIT 3 Right, I tried to brave this without enough knowledge but here is the outcome of entire run:

root@server3 [/]# fsck /dev/sdb1
fsck from util-linux-ng 2.17.2
e2fsck 1.41.12 (17-May-2010)
/dev/sdb1 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Error reading block 59770103 (Attempt to read block from filesystem resulted in short read) while getting next inode from scan.  Ignore error<y>? yes

Force rewrite<y>? yes

Pass 2: Checking directory structure
Entry '383297_262f4d70abcf3d2c216151dbe949014711c272af' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14394757.  Clear<y>? no

Entry '429064_247dc988326a61ac34725f538d91836904e39ae1' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14962043.  Clear<y>? no

Entry '429064_247dc988326a61ac34725f538d91836904e39ae1' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Entry '383308_b1543d401a14ea6b49c0858186c1035dff384865' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14394768.  Clear<y>? no

Entry '383308_b1543d401a14ea6b49c0858186c1035dff384865' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Entry '429056_2d9fef7cf6e345d84f61fda8b304fab3314d98d7' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14962036.  Clear<y>? no

Entry '429056_2d9fef7cf6e345d84f61fda8b304fab3314d98d7' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Entry '429067_2dc02dc3352898752f4f6925aee9fd2b999e8a60' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14962046.  Clear<y>? no

Entry '429067_2dc02dc3352898752f4f6925aee9fd2b999e8a60' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Entry '429055_695706ed8849d0c7d9fb22e7031c8ced335e1d6d' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14962035.  Clear<y>? no

Entry '429055_695706ed8849d0c7d9fb22e7031c8ced335e1d6d' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Entry '429069_3a8ae4186c516271931537ed190379cadb18723d' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14962048.  Clear<y>? no

Entry '429069_3a8ae4186c516271931537ed190379cadb18723d' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Entry '383307_633c8342ad75635917fe2334efaca3e646f2fcae' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14394767.  Clear<y>? no

Entry '383307_633c8342ad75635917fe2334efaca3e646f2fcae' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Entry '383306_e9d1b3d89e582d3ab4e9f4e1d7f16c3eea095d75' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14394766.  Clear<y>? no

Entry '383306_e9d1b3d89e582d3ab4e9f4e1d7f16c3eea095d75' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Entry '383302_468fc12b1f4f6d739b28bbf450aea564c0376044' in /user1/public_html/boards/attachments (13631562) has deleted/unused inode 14394762.  Clear<y>? no

Entry '383302_468fc12b1f4f6d739b28bbf450aea564c0376044' in /user1/public_html/boards/attachments (13631562) has an incorrect filetype (was 1, should be 0).
Fix<y>? yes

Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences:  -(59274765--59275233) -(59277488--59277567) -(60222133--60222135) -(60223991--60224205) -(60224208--60224390)
Fix<y>? no

Inode bitmap differences:  -(14394753--14394768) -(14962033--14962048)
Fix<y>? no


/dev/sdb1: ***** FILE SYSTEM WAS MODIFIED *****

/dev/sdb1: ********** WARNING: Filesystem still has errors **********

/dev/sdb1: 1071256/16007168 files (0.4% non-contiguous), 52105940/64000000 blocks

Any ideas on which changes I should accept and what I shouldn't?

Remove the -A from the fsck command. That parameter pulls the disks to check from /etc/fstab which is not what you want. — davidgo, Apr 18 '20 at 09:25
Done that and got some results. Is there a way to get a report of all the issues and then I can tackle specific issues one after another? — fountain-radar, Apr 18 '20 at 09:41
@davidgo, Please have a look at Edit 3 in the post. Any recommendations? — fountain-radar, Apr 18 '20 at 10:33
I'm unwilling to comment further as I could be putting your data at risk, and I don't know is nature or value. — davidgo, Apr 18 '20 at 11:20
@davidgo Fair enough. If you are able to help me understand what these things mean, I can make a decision and that way your conscience is clear? — fountain-radar, Apr 18 '20 at 11:24
Why not do a bitcopy of the block device and then fsck again, using -y to answer yes to all the questions? — davidgo, Apr 18 '20 at 19:07
Considering the options, I took the brave step of accepting the deletion of those files. I said no to rest of the options and then tried to remount the partition. Lucky for me, the remount worked, then CPU froze due to bad sectors. Eventually, I managed to get the drive remounted again on reboot and started migrating all the data across to the new server. All the data except for about 20 of the files that I accepted for deletion seems to have transferred over. I will know for sure in the next few days. Thank you for your help! — fountain-radar, Apr 19 '20 at 11:50
I'm pleased you got most of your data off. I did not realise you had a hardware issue. If that happens again, do a bitcopy of the disk as the first step (is using ddrescue) as any operations you do on the disk can make the problem worse. — davidgo, Apr 19 '20 at 19:08

score -1 · Answer 1 · answered Apr 18 '20 at 08:43

-1

/dev/sdb is your entire disk, not a partition on your disk. You need to fsck on the partition, e.g. /dev/sdb1 for /home2.

answered Apr 18 '20 at 08:43

JeLuF

11
1

Clearly the fscj was not done on /dev/SDB as per details in question. The problem is the syntax used for fstab, not the partition names. – davidgo Apr 18 '20 at 09:27
No no, JeLuF was correct. I edited the partition name and was about to make the comment that it didn't work. I'll run it without the -A option now. – fountain-radar Apr 18 '20 at 09:35

FSCK run and unmount issue

1 Answers1