LVM made easy

Imagine you had a magic knife that would allow you to slice a loaf of bread and spread some butter on the slices. There's nothing magical about that, is there? Now imagine this knife will let you join the bread slices together again or make them thinner without affecting the butter. This is exactly what a technology called Logical Volume Manger (LVM) enables you to do. The loaf is your disk, the slices are the disk partitions, the butter is any filesystem and LVM is the magic knife.

If you enjoyed our articles Virtualisation Made Easy, MythTV Made Easy and Find Files the Easy Way, then read on to take your skills a bit further with LVM!

LVM adds a logical layer between the physical disk (this can be a real disk, RAID array, or something else) and the filesystem. This logical layer (or a volume) then allows you to dynamically resize the logical volume (if supported by the filesystem), transparently move the filesystems to a different physical location, create read/write snapshots of a filesystem and spread your data over multiple physical locations.

If we were to map how filesystems, logical volumes and physical volumes all came together into one system, it would look something like this:

I have used a virtual machine, but you can use any physical system with some spare disk partitions (at least 100MB). LVM's equivalent of disk partitions are called logical volumes, and filesystems are normally created on them. Logical volumes are also grouped together into volume groups. To create a volume group you need to have a physical volume (typically created on a disk partition). Right, that's the theory – now we can have a look at how to create and use volumes.

Part One: Basic operations

First we need to identify and set up the available partitions. It is important that the partition type is set to 8E (LVM). Please be careful when playing with partitions and volumes, as you can destroy your data (all commands have to given as root).

#fdisk /dev/sdb
[... Several superfluous lines removed ...]
Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-65270, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-65270, default 65270):
Using default value 65270
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

Let's create LVM physical volume now:

#pvcreate /dev/sdb1
  Physical volume “/dev/sdb1” successfully created

Now we can create a volume group and assign the physical volume to it. You can choose any meaningful name for the volume group, but it's a good habit to use 'vg' at the end of the volume group name. This will help you to identify which filesystems are under LVM management.

#vgcreate testvg /dev/sdb1
  Volume group “testvg” successfully created

To be able to create a filesystem we need a logical volume first. Again, any name can be used, but I'd recommend that you add 'lv' to the end.

#lvcreate -L 20M -n testlv testvg 
  Logical volume “testlv” created

Now we have a logical volume called testlv that has 20MB allocated (parameter -L). It is accessible via /dev/testvg/testlv (See why we have used 'vg' at the end of the volume group name?). Finally we can create a filesystem. Any type of filesystem can be used, but please bear in mind that not all filesystems support extending or shrinking, so we'll stick with ext3.

#mkfs.ext3 /dev/testvg/testlv 
 [... loads of superfluous text removed ...]

In order to use the filesystem, we have to mount it somewhere. We'll use mount point /mnt/test.

#mkdir /mnt/test
#mount /dev/testvg/testlv /mnt/test

The filesystem is mounted only temporarily. If you reboot, it will disappear and you would have to mount it again. If you want to keep it mounted permanently, add it to /etc/fstab.

Finding information

Before we test what have we done, we can recap and try a couple of commands that will provide details of all LVM components. First, we've created partitions and physical volumes. pvdisplay provides information about the physical volumes. As with all LVM commands, if called without any parameter, it will show information about all physical volumes in the system. Volume group was second. vgdisplay allows you to see the attributes of a particular volume group (or all volume groups if no parameter is specified).

Probably the most interesting pieces of information provided by vgdisplay are volume group size, PE size (size of an extent), total, allocated and free number of extents/MB (Total PE, Alloc PE and Free PE). The last LVM object was a logical volume. The command to display details of a logical volume is lvdisplay. Similarly, the most interesting information seems to be the size of the logical volume (LV size) which is a value that defines the size of a filesystem. You can try these commands one by one:

#pvdisplay /dev/sdb1
#vgdisplay testvg
#lvdisplay /dev/testvg/testlv

What about the filesystem?

#df -h /mnt/test
Filesystem                 Size  Used Avail Use% Mounted on
/dev/mapper/testvg-testlv   20M  1.2M   18M   7% /mnt/test

Have you noticed what's wrong? We have created and mounted the filesystem from /dev/testvg/testlv and suddenly it is mounted from /dev/mapper/testvg-testlv. Nothing is wrong, really. This is normal with LVM2 as it is using device-mapper to interface with the kernel. It is also another way of recognising that the device you are using is actually an LVM volume (the first way is the naming convention we've used when creating volume groups and volumes).

Testing and resizing

Now we've created them, let's test the volumes to find out if we can actually copy anything onto the filesystem.

#cp /usr/bin/* /mnt/test/

Oops! If you're following this tutorial to the letter you should see an output like this:

cp: cannot create regular file `/mnt/test/zjsdecode': No space left on device
cp: cannot create regular file `/mnt/test/zsoelim': No space left on device

What has happened? Let's check the free space on the filesystem:

#df  -h /mnt/test
Filesystem                 Size  Used Avail Use% Mounted on
/dev/mapper/testvg-testlv   20M   20M     0 100% /mnt/test

As you can see, we have used up all the available space on the filesystem. If this filesystem had been created on a disk partition, the only way forward would be either to delete some data or to repartition the disk, but LVM can extend the logical volume and the filesystem. We have to start with the logical volume:

#lvdisplay /dev/testvg/testlv | grep Size
  LV Size                20.00 MB
#lvextend -L100M /dev/testvg/testlv 
  Extending logical volume testlv to 100.00 MB
  Logical volume testlv successfully resized
#lvdisplay /dev/testvg/testlv | grep Size
  LV Size                100.00 MB

If you're using ext3 you can extend the filesystem online without any outage.

#resize2fs /dev/testvg/testlv 
resize2fs 1.40.8 (13-Mar-2008)
Filesystem at /dev/testvg/testlv is mounted on /mnt/test; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/testvg/testlv to 102400 (1k) blocks.
The filesystem on /dev/testvg/testlv is now 102400 blocks long.
#df -h /mnt/test
Filesystem                 Size  Used Avail Use% Mounted on
/dev/mapper/testvg-testlv   98M   20M   74M  22% /mnt/test

Looking at the df output above, I've decided that the filesystem is too big and that I want reduce it a bit. To do this, you need to know that LVM is allocating space in 'extents'. By default one extent has 4MB. You can see the extent size in the output of vgdisplay under the 'PE size' field. When you're reducing a logical volume, it will be reduced in multiples of 4MB. The first step is to reduce the size of the filesystem. Unfortunately ext3 doesn't support online reduction, so we have to unmount the filesystem first and run a filesystem check (fsck).

#umount /mnt/test
#e2fsck -f /dev/testvg/testlv
[... several superfluous lines removed ...]
/dev/testvg/testlv: 426/22256 files (1.2% non-contiguous), 22964/102400 blocks

The check makes sure that the filesystem is consistent and allows us to reduce its size with a minimal risk.

#resize2fs /dev/testvg/testlv 50M
resize2fs 1.40.8 (13-Mar-2008)
Resizing the filesystem on /dev/testvg/testlv to 51200 (1k) blocks.
The filesystem on /dev/testvg/testlv is now 51200 blocks long.

Here is the maths bit. Fifty can't be divided by four without a remainder. If we try to reduce the size of the logical volume to 50MB it can be either 48MB or 52MB. Fortunately, lvreduce is clever enough to use the higher value:

#lvreduce -L 50M /dev/testvg/testlv 
  Rounding up size to full physical extent 52.00 MB
  WARNING: Reducing active logical volume to 52.00 MB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce testlv? [y/n]: y
  Reducing logical volume testlv to 52.00 MB
  Logical volume testlv successfully resized

The last step is to resize the filesystem again to use all the available space.

#resize2fs /dev/testvg/testlv
resize2fs 1.40.8 (13-Mar-2008)
Resizing the filesystem on /dev/testvg/testlv to 53248 (1k) blocks.
The filesystem on /dev/testvg/testlv is now 53248 blocks long.

We can now remount the filesystem and check that it is the correct size.

#mount /dev/testvg/testlv /mnt/test/
#df -h /mnt/test
Filesystem                 Size  Used Avail Use% Mounted on
/dev/mapper/testvg-testlv   51M   20M   30M  40% /mnt/test

Please keep the extent size in mind when resizing logical volumes and make sure that the new volume size is always the same or bigger than the filesystem size.

This is system-config-lvm. Not the sexiest name, but it does what it says on the tin and does it well.

Part Two: More advanced operations

LVM allows you to create a copy of a logical volume at a certain point in time (this is called a snapshot). “What's the use in that?”, I hear you cry. Well, imagine you have a database (or any other application) that you want to back up. To create a consistent backup you have to shut it down, or at least put it to sleep. However, backups can take a long time, which would mean a long outage of your system. With LVM you can stop your database, take a snapshot of its filesystem and start it again, which would take only a couple of seconds. You can then back up the snapshot.

Another example would be virtualisation. Imagine that you want to test something on your virtual machine, but you're worried that something will go wrong and you'll destroy it. The solution is simple: create a snapshot of the filesystem holding your virtual machine disks. You can now start your virtual machine from this snapshot, perform your test and if anything goes wrong, you simply remove the snapshot. Your original filesystem still contains all the data at the point when you started. Ingenious! So, how do we do this? First, we have to make sure that there is enough space in the volume group (see vgdisplay above). If yes, we simply create a new logical volume with the -s parameter (snapshot) and mount it as if were an ordinary filesystem.

#lvcreate  -s -L50M -n testlvsnap /dev/testvg/testlv 
  Rounding up size to full physical extent 52.00 MB
  Logi	cal volume “testlvsnap” created
#mkdir /mnt/snap
#mount /dev/testvg/testlvsnap /mnt/snap
#df -h /mnt/test /mnt/snap
Filesystem                     Size  Used Avail Use% Mounted on
/dev/mapper/testvg-testlv       51M   20M   30M  40% /mnt/test
/dev/mapper/testvg-testlvsnap   51M   20M   30M  40% /mnt/snap

They both look the same. You can double-check by running the following commands (we removed the output, as it was too long):

#diff /mnt/test /mnt/snap/
#ls -l /mnt/test
#ls -l /mnt/snap

We can now test if LVM really works by removing and creating some files on our snapshot.

#rm  -rf /mnt/snap/*    (BE CAREFUL WITH THIS COMMAND!)
#touch /mnt/snap/SNAP_TEST
#touch /mnt/test/ORIG_TEST
#ls -l /mnt/test
#ls -l /mnt/snap

If we don't need the snapshot any longer, it can be removed in the same way as any logical volume.

#umount /mnt/snap
#lvremove /dev/testvg/testlvsnap 
Do you really want to remove active logical volume “testlvsnap”? [y/n]: y
  Logical volume “testlvsnap” successfully removed

In a snap

If you are wondering how the snapshots work, it's quite simple. When a snapshot is created, LVM doesn't have to copy anything on the filesystem. It allocates required space in the volume group and starts keeping a track of changes. Both the original and the snapshot initially point to the same data blocks. If any of them are changed, LVM will create a copy of the changed data and will keep them separate. This technique is called copy-on-write, and is the reason why the snapshots are created instantly whatever the size of the filesystem.

Another advanced feature of LVM is transparent migration of data between physical volumes. This comes into its own if you come to replace your hard disk without undue hassle. Without LVM you would have to boot from a CD and use it to copy filesystems – with LVM you can simply add the new disk (or its partition) to the existing volume group and ask LVM to move everything on to it. To be able to follow the following exercise you need to have another partition available (doesn't matter if it's on a different or the same disk). Create the partition and set its type to 8E as we did earlier.

#pvcreate /dev/sdc1
  Physical volume “/dev/sdc1” successfully created
#vgextend testvg /dev/sdc1
  Volume group “testvg” successfully extended

Our volume group now has two physical volumes assigned (sdb1 and sdc1). This is how you can extend an existing volume group (and its volumes). This is also a way to create filesystems that are bigger than one disk. We need to do the following to move the data to the new volume and to remove the old disk.

#pvmove /dev/sdb1 /dev/sdc1
  /dev/sdb1: Moved: 100.0%
#vgreduce testvg /dev/sdb1
  Removed “/dev/sdb1” from volume group “testvg”

Pvmove has to physically move all the data from one volume to another. On our tiny filesystem it will be done in a few seconds, but it can take many hours on a big filesystem (especially if it is in a constant use).

Part Three: Installing or converting to LVM

Have I convinced you that LVM is the best thing since sliced bread? Do you want to install your system on LVM managed volumes? Don't worry, it's fairly straightforward. Fedora and CentOS are practically identical and both will use LVM if you select the default layout. OpenSUSE will let you choose between partition and LVM based installations, and Ubuntu Server has a guided installation using LVM. The only distribution I've tested that doesn't support installation on LVM is Ubuntu Desktop. Fortunately there is a workaround. You can find the details here: www.debuntu.org/how-to-install-ubuntu-over-lvm-filesystem.

What if you want to use LVM on your existing filesystems, but don't want to reinstall your system? Unfortunately there isn't a magical tool which could convert partitions to logical volumes. You can do it, but the filesystem has to have at least 50% free space. (The alternative is to use either a spare disk or another partition with enough disk space.) Please make sure that you have at least some backup of your precious data before modifying partitions and filesystems.

First, use the available disk space to create a new partition. GParted is the best tool for this. Now, set the partition type to LVM (using fdisk or cfdisk), create a physical volume, a volume group and a logical volume, then create a filesystem. Use something sensible like dd, cpio, dump/restore or tar to copy the data to the new filesystem. When done, change the original partition type to LVM, create a physical volume on it and add it to the volume group. You can now extend the logical volume to a desired size.

If you want to convert your root filesystem to LVM you have to boot from any live CD/DVD and do it from there. You also need to know that the /boot filesystem must not be placed on a logical volume or you won't be able to start your system any more. The reason is that boot loaders (such as Grub) don't understand LVM and can't find the kernel to boot from it.

Even more

So far we've covered some basic and slightly advanced features of Logical Volume Manager. LVM, however, offers more. Especially if you have more than one disk. For example, if you assign two physical volumes to a volume group, LVM will use all the available space on the first volume, then move to the second one. This may not be what you want though, it might be faster to read and write to two disks in parallel. So, you can tell LVM to stripe the data across multiple physical volumes in a fashion similar to RAID0. LVM does not provide any data redundancy like RAID1 or 5, but physical volumes can be created on top of Linux Software RAID devices (md). This means that you can create several md devices and combine them into one or more logical volumes.

I've missed out a few more features like clustering support and some commands such as vgscan, vgchange and vgs. If you want to know more about it, have a look at the LVM HOWTO document at www.tldp.org/HOWTO/LVM-HOWTO. Although it's slightly outdated in places, it's still an excellent source of information.

First published in Linux Format magazine

You should follow us on Identi.ca or Twitter