4

For debugging purposes, I want to open a file on a specific predefined block. For instance, if I suspect a specific block is damaged, I want to write and read from it, and I'd rather do that in user mode, while the partition is mounted.

Is there a way to tell Linux, "hey! open this new file on block 4579 if it's free".

Yes, I can edit the block device directly, but that would likely to trash the filesystem if the drive is mounted.

Generic answers are welcomed, but even answer for the ext filesystems families is good enough.

mikebloch
  • 1,577
  • 11
  • 21
  • This would be highly dependent on the specific filesystem in use. Some filesystems don't even have the concept of blocks. But you've tagged the question ext4. I highly doubt that there is such a feature for ext4 or any other similar filesystem. – Celada Jul 04 '12 at 16:38
  • @Celada they have to have the concept of block, since this is the basic unit they write to the physical device, isn't it? – mikebloch Jul 05 '12 at 07:16
  • I said *some* filesystems don't have the concept of a block. nfs, tmpfs, virtual filesystems, most fuse filesystems, jffs2 (though that one does erase by the block), etc... But I just said that by the way, since I gather you are asking about ext4 specifically. – Celada Jul 05 '12 at 13:26
  • @Celada, can you please explain (I'm new to block devices), won't jffs2 eventually write 4K to the second block in the underlying flash? (or whatever). Maybe it'll allocate a certain block to many files, maybe it doesn't have concept of files at all, but at the end of the day, it'll write 4K (or whatever the block size is) of data to the underlying block device (which might be a file pretending to be a block device, but it'll still think it's a block device). Am I wrong? – mikebloch Jul 05 '12 at 15:08
  • Not when using NOR flash. NOR flash is read and written one bit at a time, not one block at a time (cheaper NAND flash does have "pages", but they still don't quite work the same way as a block device's block). There is no "underlying block device": jffs2's underlying device is a MTD device. – Celada Jul 05 '12 at 17:47
  • 1
    This question reminds me of people who want to access a specific physical memory address from user space. Sigh... – paulsm4 Jul 05 '12 at 23:43
  • @Celada of course you're right at the HW level, but don't the linux driver infrastructure force you to use block device wrapper? What will the `FS_IOC_FIEMAP` return when running on `jffs2`? – mikebloch Jul 08 '12 at 05:51

1 Answers1

2

For instance, if I suspect a specific block is damaged, I want to write and read from it ... Is there a way to tell Linux, "hey! open this new file on block 4579 if it's free".

Yes, you can use the underlying block device as a file and seek to that location on the file. This seems appropriate given your "debugging" use case. Note that writes to this block may destroy the integrity of the filesystem above (even writing back the contents just read).

Otherwise, no, the filesystems are designed to hide often mask the real geometry underlying device's layout and as such there is no such mechanism to give hints on where the file should be created.

Brian Cain
  • 14,403
  • 3
  • 50
  • 88
  • Even the drives don't expose their real geometry. All flash devices except raw NAND chips do wear leveling automatically, of course, so there simply is no notion of a specific location in storage. And magnetic devices can detect errors via ECC mechanisms and transparently remap the sectors without any visibility at the OS level. – Andy Ross Jul 04 '12 at 17:58
  • @AndyRoss I don't care about it in that level, I just want to get the same block level access that the OS uses. – mikebloch Jul 05 '12 at 07:15
  • @Brian I know that but, I don't want to trash my file system. – mikebloch Jul 05 '12 at 07:15
  • @Brian, your answer is basically "no!". Can you add a reference to that claim (eg, LKML post saying that by design you can't control it), it looks false, since there are ioctl calls that tells you which blocks does each file reside in, and which blocks are free on the block device, so generally speaking your premise is incorrect. (-1 till you'll fix that). – mikebloch Jul 05 '12 at 13:02
  • 2
    @mikebloch I don't think the onus is on Brian to show an LKML reference saying the feature was considered and rejected. It would be a very odd thing for a filesystem to allow external input into its block allocation decision strategy, so it's more than likely that it's never even been discussed before. The ioctls you mention to find the location of files and free space after the fact were specifically added to solve concrete use cases (like the needs of a bootloader. Even then, it's fragile to expose this: what does it even mean when RAID parity is involved? Or file data stored in the inode? – Celada Jul 05 '12 at 13:35
  • @Celada, well, it can be useful for instance to implement a fine grain defrag software. Currently, if you want to defrag a file, you must move all its blocks to a new area, if you know where the file is stored, you can move just a little bit of it. Actually, from what I can see, the EXT4_IOC_MOVE_EXT ioctl, is used to move file blocks around, and you can have more fine-grained defragger if you could control which blocks are moved. Windows allows moving a file to a certain block, as you mentioned, there's the `FS_IOC_FIEMAP`, I don't think moving a file to a specific block is so far fetched. – mikebloch Jul 05 '12 at 15:02
  • @Celada, oh, and if the file is inlined in an inode - simply give back the block where the inode resides in. RAID has blocks as well, they don't map to blocks on an actual disk, but the RAID controller pretends he's a disk, if it stores 4K on block 7821, you should report that the file is on block 7821, regardless to what the RAID controller does afterwards. – mikebloch Jul 05 '12 at 15:04
  • 2
    @mikebloch defrag software, like fsck, would be considered part of the implementation of the filesystem and is tightly associated with a particular filesystem type and probably developed by the same team, too. It would use private interfaces to the filesystem that aren't designed for public use, and could not really be extended to be generic enough anyway. Think, for example, of zfs: it's a copy-on-write filesystem so a file moves around to a new block every time it's modified. Are you expecting that? Anyway, for zfs, you have to deal with much more than just blocks: you have storage pools – Celada Jul 05 '12 at 17:40
  • @mikebloch re: RAID. That depends on your use case! When obtaining the block map for a file, a bootloader would need to resolve the file's location through the RAID layer all the way down to the physical disk, so no, you would NOT want to treat the RAID array as an opaque virtual device. You see how these are murky waters? – Celada Jul 05 '12 at 17:42
  • @AndyRoss, yes, "real geometry" indeed -- the block device could yet be SCSI-over-IP-over-SCSI-over-IP-over avian carrier. I don't think this thread is totally relevant. I've removed the term "geometry" from the answer since it seems to distract from the question at hand. – Brian Cain Jul 06 '12 at 02:01