4

We've been deploying fanless PCs for a research study. Some of these are having HD issues as their OS runs from an SD card. I'm seeing cases where I need to fsck a folder or two.

I've tried using shutdown -rF now but it doesn't seem to be doing the trick. There are notes in the syslog that say stuff was fixed but it doesn't appear to be. Also the order of events in the syslog makes it sound like the OS was fully up when the fsck was done (IE fs was mounted). Certainly not a good thing.

Any suggestions on other ways to fix this without having to do service calls and replace the units?

OS: debian 6.x

Note: I did see this. Gave me the notion for the shutdown command but it doesn't seem to be working properly (or I'm not using it properly).

Community
  • 1
ethrbunny
  • 2,369
  • 4
  • 41
  • 75

1 Answers1

5

Also the order of events in the syslog makes it sound like the OS was fully up when the fsck was done

The root filesystem usually needs to be mounted to run fsck. The rootfs is usually mounted read-only, and then re-mounted read-write after the fsck is complete. Of course the automatic fsck can't always fix all the problems. Sometimes you need to manually run it to fix things.

Any suggestions on other ways to fix this without having to do service calls and replace the units?

Assuming you have SSH acccess to this box, you should be able to stop all services that write to the filesystems, and then remount any filesystems as read-only, while leaving your SSH server running(1). If you can get everything remounted read-only then should be able to safely run a manual fsck.

  • (1) sshd doesn't normally write to the filesystem directly. It writes via syslog, and syslog can be stopped while a system is running.

And the standard disclaimers, test stopping services and remounting on a box you have physical access to first. If possibly make sure you have someone in place you can call and have power-cycle the box just in case you stop the wrong services and kill your remote access.

Zoredache
  • 130,897
  • 41
  • 276
  • 420
  • Getting everything to let go of /var is proving to be tough. Even when I have it unmounted I see lots of stuff in `fuser -ivm /var`. – ethrbunny Feb 10 '14 at 20:42
  • 2
    Not sure what to tell you here. If you can't stop the programs with those files open then you may need to look into building an initrd image that permits you to SSH to the system in remotely before the system is booted. Something like what people do to permit remotely [entering a luks key](http://blog.neutrino.es/2011/unlocking-a-luks-encrypted-root-partition-remotely-via-ssh/). – Zoredache Feb 10 '14 at 21:43
  • If everything in `fuser` is marked 'rc' or 'rce' is it safe to try an fsck? – ethrbunny Feb 11 '14 at 12:11