LVM Caching

Getting Started

We use LVM caching when we have a server with both spinning disks and SSD disks and the spinning disks are experience disk i/o slow downs.

Before you can create a cache device, you have to add the following setting to /etc/lvm/lvmlocal.conf so that logical volumes can be available as physical volumes:

devices {
  scan_lvs = 1
}

In addition, ensure that thin-provisioning-tools is installed.

The commands

For example, on a server in which the spinning disks are in vg0 and the SSDs are in vg1, we can add a cache called cache001 to the logical volume weborigin002 with the following commands:

# Create /dev/mapper/vg1-cache001 (on SSD volume group)
lvcreate --size 25gb --name cache001 vg1
# Add it as a physical volume to vg0
vgextend vg0 /dev/mapper/vg1-cache001
# Use it as a cache for weborigin002
lvcreate --type cache --cachemode writeback -l 100%FREE --name vg0_cachepool vg0/weborigin002 /dev/mapper/vg1-cache001

To examine the cache:

lvs vg0/weborigin002

To see all parts of the cache:

lvs -a

To “uncache”:

lvconvert --uncache vg0/weborigin002

More information

There is full documentation.

LVM cache logical volume types

LVM caching uses the following LVM logical volume types. All of these associated logical volumes must be in the same volume group.

  • Origin logical volume: the large, slow logical volume
  • Cache pool logical volume: the small, fast logical volume, which is composed of two devices: the cache data logical volume, and the cache metadata logical volume
    • Cache data logical volume: the logical volume containing the data blocks for the cache pool logical volume
    • Cache metadata logical volume: the logical volume containing the metadata for the cache pool logical volume, which holds the accounting information that specifies where data blocks are stored (for example, on the origin logical volume or the cache data logical volume).
  • Cache logical volume: the logical volume containing the origin logical volume and the cache pool logical volume. This is the resultant usable device which encapsulates the various cache volume components.
  • Spare metadata logical volume: This is related to a [meta data failure recovery feature](https://manpages.debian.org/stretch/lvm2/lvmthin.7.en.html#Spare_metadata_LV metadata failure-recovery feature). “If thin pool metadata is damaged, it may be repairable. Checking and repairing thin pool metadata is analagous to running fsck on a file system.” Another explanation

Cachemode

When creating the cache cachemode has two possible options:

  • writethrough ensures that any data written will be stored both in the cache pool LV and on the origin LV. The loss of a device associated with the cache pool LV in this case would not mean the loss of any data;
  • writeback ensures better performance, but at the cost of a higher risk of data loss in case the drive used for cache fails.

In the examples above we’ve opted for writeback because the logical volume created for the cache on the physical host is backed by a RAID1 array created from two SSD drives so we already have a lower level of protection against drive failure in place. Choose wisely.

Do not exhaust space on “metadadata” logical volume ¶

We should not allow the “metadadata” volume of an lvm cachepool to exhaust its available space. In the man pages there is documentation about what happens to lvm thinpool when it reaches metadata space exhaustion and it sounds all bad.