4

I am using BTRFS to create a RAID0 volume from 2 disks. Both are on an AWS EC2 instance with 30 GB space available each. The command I used to create the filesystem is: btrfs -d raid0 -L MYLABEL /dev/xvdf /dev/xvdg.

Now Ubuntu tells me the disk space is full, just as BTRFS:

root@host:/var/docker-apps/prometheus# btrfs balance start -dconvert=raid0 .
ERROR: error during balancing '.': No space left on device
There may be more info in syslog - try dmesg | tail
root@host:/var/docker-apps/prometheus#

The fun thing is, when I look at btrfs filesystem usage I get this:

root@host:/var/docker-apps/prometheus# btrfs filesystem usage .
Overall:
    Device size:          60.00GiB
    Device allocated:     60.00GiB
    Device unallocated:   2.00MiB
    Device missing:       0.00B
    Used:                 25.26GiB
    Free (estimated):     34.22GiB    (min: 34.22GiB)
    Data ratio:           1.00
    Metadata ratio:       2.00
    Global reserve:       256.00MiB    (used: 0.00B)

Data,RAID0: Size:57.98GiB, Used:23.76GiB
   /dev/xvdf      28.99GiB
   /dev/xvdg      28.99GiB

Metadata,RAID1: Size:1.00GiB, Used:766.11MiB
   /dev/xvdf       1.00GiB
   /dev/xvdg       1.00GiB

System,RAID1: Size:8.00MiB, Used:16.00KiB
   /dev/xvdf       8.00MiB
   /dev/xvdg       8.00MiB

Unallocated:
   /dev/xvdf       1.00MiB
   /dev/xvdg       1.00MiB
root@host:/var/docker-apps/prometheus#

Now, for me that reads like "I have 60 GB of RAID0 space available, and I currently use about 25 of them (Used: entry in the top block, together with Free: entry in the same block)". I think this is pretty clear.

But still: No space left on device.

So - am I wrong? How do I have to read this? Why can't I create files in the BTRFS partition any more? How do I have to read this output? What's going on here?

flypenguin
  • 203
  • 1
  • 2
  • 12

1 Answers1

7

Read this: http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html

It seems that you are suffering the "metadata is full case" of the article, because Btrfs reserved 0.5 GB for itself, meaning it is full and therefore preventing any new writes. The article suggests that you should do a full rebalance on it.

Keep also on mind that normal df lies, it is never going to tell you the real btrfs usage stats.

Marc Stürmer
  • 1,904
  • 13
  • 15
  • As far as I remember the metadata seemed fine - although I don't have the screenshots available any more. The balance did not work, I tried that - it aborted with "no space left on device", which is kinda annoying since this was what I wanted to fix ... :) – flypenguin Jul 27 '16 at 22:38
  • 2
    The metadata is _not_ fine, just take a look at the btrfs fileusage dump above: "Metadata,RAID1: Size:1.00GiB, Used:766.11MiB" This means: 234 MB of Metadata is free. Btrfs reserves 500 MB for itself as I stated above, meaning: it's full. This is your problem. – Marc Stürmer Jul 29 '16 at 05:08