4

I've implemented a basic filesystem using FUSE, with all foreseeable POSIX functionality implemented [naturally I haven't even profiled yet ;)]. Currently I'm able to run the filesystem on a regular file (st_mode & S_IFREG), but the next step in development is to host it on an actual block device. Running my code as is, immediately fails on reading st_size after calling fstat on the device. Of course I don't expect the problems to stop there so:

  • What changes are required to operate on block devices as opposed to regular files?
  • What are some special considerations I need to make with regard to performance, limitations, special features and the like?
  • Are there any tutorials and references with dealing with block special files? Googling has turned up very little useful; I only have background knowledge (ironically from MSDN in my dark past) and some scanty information in the manpages.

Update0

  • I've pointed out what I mean by "regular file".
  • I don't want to concentrate on getting the device size, I want general guidelines for differences between regular files and device files with respect to performance and usage.
Matt Joiner
  • 112,946
  • 110
  • 377
  • 526
  • I can't think of any reason that reading the st_size member of the structure whose address you passed to fstat should crash the program no matter what, unless the kernel has a bug and isn't checking the access rights on that memory correctly in the system call but is in the application when the MMU catches it. – nategoose Sep 02 '10 at 22:20
  • This might only indirectly be related to the block device issue - just because it works with a regular file doesn't mean it doesn't have subtle errors. Please post the code that allocates your stat struct, calls fstat(), and reads st_size. – Chris Stratton Dec 20 '10 at 18:03

2 Answers2

4

Currently I'm able to run the filesystem on a regularly file, but the next step in development is to host it on an actual block device

I don't completely understand what you mean - I assume you are saying that "you currently save your filesystem data to a plain file on a normally mounted filesystem - but now wish to use a raw block device for your data storage".

If so - having done this a few times - I'd advise the following:

  • Never use an "actual" block device for you filesystem. Always use a partition. There are several rarely-used partition-types that you can use to denote that such a filesystem may be your filesystem type, and that your filesystem can check and mount it if it is such. Thus, you will never be running on something like "/dev/sdb", but rather you will store you data on one such as /dev/sdb1, and assign it some partition type. This has obvious advantages, like allowing your filesystem to be co-resident on a single phyiscal disk as another, etc.
  • If you are implementing any caching in your filesystem (like Linux does with the Page Cache), do all I/Os to the block devices with O_DIRECT. This requires you to pass page-alligned memory to do all I/O, and requires that the requests be sector/block aligned - but will remove a data copy which would otherwise be required when data is moved from the block device to the page cache, then from the page-cache to your user-space [filesystem] reader.

What do you mean that the fstat "fails"? This is an fstat trying to determing the length of the block device? Do you receive an error? What is it?

Brad
  • 11,262
  • 8
  • 55
  • 74
2

block devices behave very much like files - tools like dd can operate on them without any special handling. fstat, though, returns information about the special-file node, not the blockdev it refers to. you probably want to use the BLKGETSIZE64 ioctl to read the size.

there's no particular reason to use a partition over a raw device, though - a blockdev is a blockdev. O_DIRECT is good, as well, assuming your workload won't generate repeated accesses. don't confuse it with a real protocol for ensuring the permanence and atomicity of your filesystem, though (fsync, barriers, etc).

markhahn
  • 565
  • 4
  • 6