LVM: moving physical volumes

Question

In a virtual server (Debian GNU/Linux 8 amd64) I have the following disks and filesystems:

# pvscan 
  PV /dev/vda1   VG vg0   lvm2 [100.00 GiB / 0    free]
  PV /dev/vdb1   VG vg0   lvm2 [46.56 GiB / 0    free]
  PV /dev/vda2   VG vg0   lvm2 [100.00 GiB / 0    free]
  PV /dev/vdc1   VG vg0   lvm2 [60.00 GiB / 60.00 GiB free]
  Total: 4 [306.55 GiB] / in use: 4 [306.55 GiB] / in no VG: 0 [0   ]

# lvdisplay 
  --- Logical volume ---
  LV Path                /dev/vg0/root
  LV Name                root
  VG Name                vg0
  LV UUID                qpeei3-v1nW-pYVR-lK7Y-4wwy-Y4Y4-c9yQEl
  LV Write Access        read/write
  LV Creation host, time nomos, 2015-03-17 16:34:05 +0100
  LV Status              available
  # open                 1
  LV Size                246.56 GiB
  Current LE             63119
  Segments               3
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:0

# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/dm-0       243G  135G   98G  59% /
[... no other on-disk filesystems]

The root filesystem is ext4.

Since /dev/vda has enough room for all my data, I want to remove /dev/vdb and /dev/vdc. The latter is there as a temporary space only to provide room for the needed maneuvers. The data on /dev/vdb1 should be moved to /dev/vda* before removing /dev/vdb1. However the root filesystem currently spans all three partitions /dev/vda1, /dev/vda2 and /dev/vdb1.

The problem is I can take the virtual machine down only for very short time periods (a few seconds, just reboots), so I can't reduce the size of the root filesystem, because that requires to unmount it and to keep the server down for quite a long time. I can add other disks though, up to 500GB, if needed.

I've run pvmove to move data off the drive:

# pvmove /dev/vdb1 
  Detected pvmove in progress for /dev/vdb1
  /dev/vdb1: Moved: 4.0%

and waited until it reached 100%. However pvmove moved the data to /dev/vdc1.

# pvs
  PV         VG   Fmt  Attr PSize   PFree 
  /dev/vda1  vg0  lvm2 a--  100.00g     0 
  /dev/vda2  vg0  lvm2 a--  100.00g     0 
  /dev/vdb1  vg0  lvm2 a--   46.56g 46.56g
  /dev/vdc1  vg0  lvm2 a--   60.00g 13.43g

And now? I could probably remove /dev/vdb1, but I'm stuck with my data on /dev/vdc1. What I actually need is to move the allocated inodes of the root filesystem off /dev/vdb1 to the free filesystem space in /dev/vda*. Then I dream I can move /dev/vdb1 out of the way because the filesystem moved to /dev/vda*. I realize it does not work that way automatically, but I'm not able to imagine a migration strategy that lets me do that even manually, without shrinking the root filesystem.

Can you help, please?

Ahhh....according to the man page for pvmove, the command does allow you to specify BOTH source and destination disks. — mdpc, Oct 06 '16 at 23:40

Lucio Crusca · Answer 1 · 2016-10-07T09:25:37.867

The following procedure is only an idea, never tried, but I think it can work:

Consider Btrfs pros and cons, and the fact it isn't really production ready yet (well, as of 2015, but I couldn't find anything more recent on the subject). Let's assume you want it anyway.
Backup backup backup
Convert the ext4 filesystem to Btrfs, because Btrfs supports online migration from ext4
Shrink the new root filesystem, now Btrfs, because Btrfs supports online shrinking
Shrink the LVM contaner volume (vg0/root)
pvmove data off /dev/vdb1 (now vdc1 if we consider I've already moved it to /dev/vdc1)
remove /dev/vdb1 (or /dev/vdc1)
say goodbye to ext4 forever

Assuming no other better answers are posted, I will try this and report back here with the details.

Looks good! Please comment in a month or so to let us know about btrfs stability. — kubanczyk, Oct 07 '16 at 09:14

score 0 · Answer 2 · answered Oct 06 '16 at 17:37

0

Ok, you evaquated your data from vdb1. Now:

remove /dev/vdb1 using vgreduce vg0 /dev/vdb1
use pvmove to remove data from/dev/vdc1
issue vgreduce vg0 /dev/vdc1 to remove it from the volume group

At this point, your data should be only on vda*

Be very carefull before removing physical volumes, as you should be sure your data were evaquated from the to-be-removed device. And take backups before doing anything

answered Oct 06 '16 at 17:37

shodanshok

47,711
7
111
180

what should be the exact `pvmove` command in order to force it move the data to `/dev/vda*` instead of back to `/dev/vdb1`? – Lucio Crusca Oct 06 '16 at 17:46
You had to `vgremove` vdb1 first. Please follow the above instructions. – shodanshok Oct 06 '16 at 17:51
Yes, I've already removed vdb1, but: `# pvmove /dev/vdc1 No extents available for allocation` – Lucio Crusca Oct 06 '16 at 18:25
This is because you have no free space on vda, as show by `pvs` – shodanshok Oct 06 '16 at 18:35
Yes that's exactly my question in the first place. I actually have only 135GB worth of data and 200GB of capacity on vda. I want to use only vda but I can't shrink the filesystem. – Lucio Crusca Oct 07 '16 at 07:16
As your root filesystem is ext4, you should be able to resize it using `resize2fs`. Have you tried it? Anyway, shrinking a filesystem is a complex and potentially dangerous operation, so be sure to have valid backups. – shodanshok Oct 07 '16 at 08:47

score 0 · Answer 3 · edited Apr 13 '17 at 12:37

Your problem is to resize2fs a live root filesystem and shrink it to 200 GB. One possible approach is well outlined in the Unix.SE How to shrink root filesystem question. It uses pivot_root twice and requires a manual restart of most of the OS processes and services; it leads to a loss of recent data, so you cannot use it if you have a living database; and it is fearsome, to put it mildly. But still, it's an alternative that people tried with some success.

LVM: moving physical volumes

3 Answers3