2

I am facing No space left on BTRFS.

What I already tried myself:

  • After remount RW: btrfs filesystem resize max /mnt/hdd
  • I ran balance (for weeks ;-) ) with different parameters btrfs balance start /mnt/hdd -dlimit=<param>

Nothing did help. What can I do more?

So what i am facing is (Ubuntu 18.04.1 LTS):

BTRFS: error (device sda) in __btrfs_inc_extent_ref:2268: errno=-28 No space left
BTRFS info (device sda): forced readonly

However there is plenty of space:

$ sudo btrfs fi show /mnt/hdd
Label: none  uuid: 262a8d86-279a-4f6b-8968-32e200c32255
    Total devices 1 FS bytes used 1.68TiB
    devid    1 size 7.28TiB used 1.70TiB path /dev/sda

However, this command is reporting the Data part is running out of space?

$ btrfs fi df /mnt/hdd
Data, single: total=1.47TiB, used=1.46TiB
System, single: total=4.00MiB, used=224.00KiB
Metadata, single: total=236.01GiB, used=222.76GiB
GlobalReserve, single: total=512.00MiB, used=495.75MiB

Also: 

$ sudo btrfs fi usage /mnt/hdd
Overall:
    Device size:           7.28TiB
    Device allocated:          1.70TiB
    Device unallocated:        5.58TiB
    Device missing:          0.00B
    Used:              1.68TiB
    Free (estimated):          5.58TiB  (min: 5.58TiB)
    Data ratio:               1.00
    Metadata ratio:           1.00
    Global reserve:      512.00MiB  (used: 495.75MiB)

Data,single: Size:1.47TiB, Used:1.46TiB
   /dev/sda    1.47TiB

Metadata,single: Size:236.01GiB, Used:222.76GiB
   /dev/sda  236.01GiB

System,single: Size:4.00MiB, Used:224.00KiB
   /dev/sda    4.00MiB

Unallocated:
   /dev/sda    5.58TiB
robert
  • 163
  • 7
  • `Nothing did help. What can I do more?` Getting rid of btrfs? Take a hint from Redhat - they actually abandoned btrfs and will not support it any longer in RHEL8... – Sven Dec 05 '18 at 13:28
  • That may be so, but they also "abandoned" DRBD and plenty of other packages because "they're not made there". So you might take RH dropping support for a particular product with a grain of salt. – Spooler Dec 05 '18 at 15:24
  • @Spooler BTRFS really is a steaming pile of garbage. Even Red Hat couldn't make it better, and not for any lack of trying! – Michael Hampton Dec 05 '18 at 17:09
  • [BTRFS: “no space left on device”, but btrfs command says otherwise](https://serverfault.com/q/790464/343888) – phuclv Jan 30 '20 at 10:55

1 Answers1

1

Your device is mostly unallocated, which can be corrected with a resize/rebalance. But that already didn't work, so there's "something weird" in the way. An output of BTRFS related messages from dmesg might help to identify why.

Did you try turning it off and turning it back on again? Seriously though, remounting fixes most BTRFS weirdness. Back up your data before unmounting if you haven't already.

Aside from that, you can perform a check. This is different than a scrub, and is done against the backing block device(s) while BTRFS is offline. This will consume lots of RAM, and will take a while to complete. This validates structures, and will not repair things on its own unless told to (with the --repair option). Don't tell it to repair things until you have an output of what's wrong and understand what will correct that (for example if you have an undetected physical drive failure, doing a repair will almost certainly trash the filesystem).

An offline check can be issued thusly:

btrfs check -p /dev/sda

You also should check the backing device (read its SMART data, do a long test, etc) if things come out weird from the BTRFS check. This can be done with smartctl -a /dev/sda and smartctl -t long /dev/sda respectively. This is assuming that this is a baremetal node which has real drives in it. If it's a VM, start checking from the bottom up if things are looking damaged in BTRFS.

Spooler
  • 7,046
  • 18
  • 29