Chapter 20. Overcommitting with KVM
The KVM hypervisor supports overcommitting CPUs and overcommitting memory. Overcommitting is allocating more virtualized CPUs or memory than there are physical resources on the system. With CPU overcommit, under-utilized virtualized servers or desktops can run on fewer servers which saves power and money.
Most operating systems and applications do not use 100% of the available RAM all the time. This behavior can be exploited with KVM. KVM can allocate more memory for virtualized guests than the host has physically available.
With KVM, virtual machines are Linux processes. Guests on the KVM hypervisor do not have dedicated blocks of physical RAM assigned to them, instead guests function as Linux processes. The Linux kernel allocates each process memory when the process requests more memory. KVM guests are allocated memory when requested by the guest operating system. The guest only requires slightly more physical memory than the virtualized operating system reports as used. The Linux kernel swaps infrequently used memory out of physical memory and into virtual memory. Swapping decreases the amount of memory required by virtualized guests.
When physical memory is completely used or a process is inactive for some time, Linux moves the process's memory to swap. Swap is usually a partition on a hard disk drive or solid state drive which Linux uses to extend virtual memory. Swap is significantly slower than RAM due to the throughput and response times of hard drives and solid state drives.
As KVM virtual machines are Linux processes, underused or idle memory of virtualized guests is moved by default to swap. The total memory used by guests can be overcommitted, which is to use more than the physically available host memory. Overcommitting requires sufficient swap space for all guests and all host processes.
Without sufficient swap space for all processes in virtual memory the
pdflush process, the cleanup process, starts. The
pdflush process kills processes to free memory so the system does not crash.
pdflush may destroy virtualized guests or other system processes which may cause file system errors and may leave virtualized guests unbootable.This can cause issues if virtualized guests use their total RAM.
If sufficient swap is not available guest operating systems will be forcibly shut down. This may leave guests inoperable. Avoid this by never overcommitting more memory than there is swap available.
If KSM is used ensure the swap size is sufficient for the overcommitted RAM. KSM reduces the RAM usage of identical or similar guests. Overcommitting guests with KSM without sufficient swap may be possible but is not recommended. For more information on KSM and overcommitting, refer to Chapter 21, KSM
Configuring swap for overcommitting memory
The swap partition is used for swapping underused memory to the hard drive to speed up memory performance. The default size of the swap partition is calculated from the physical RAM of the host.
Red Hat Knowledgebase
has an article on safely and efficiently determining the size of the swap partition.
The swap partition must be large enough to provide virtual memory for all guests and the host system.
The example below is provided as a guide for configuring swap only. The settings listed may not be appropriate for your environment.
Example 20.1. Memory overcommit example
ExampleServer1 has 32GB of RAM. The system is being configured to run 56 guests with 1GB of virtualized memory. The host system rarely uses more than 4GB of memory for system processes, drivers and storage caching.
32GB minus 4GB for the host leaves 28GB of physical RAM for virtualized guests. Each guest uses 1GB of RAM, a total of 56GB of virtual RAM is required for the guests.
The Red Hat Knowledgebase recommends 8GB of swap for a system with 32GB of RAM. To safely overcommit memory there must be sufficient virtual memory for all guests and the host. The host has 28GB of RAM for guests (which need 56GB of RAM). Therefore, the system needs at least 28GB of swap for the guests.
ExampleServer1 requires at least 36GB (8GB for the host and 28GB for the guests) of swap to safely overcommit for all 56 guests.
It is possible to overcommit memory over ten times the amount of physical RAM in the system. This only works with certain types of guest, for example, desktop virtualization with minimal intensive usage or running several identical guests with KSM. Configuring swap and memory overcommit is not a formula, each environment and setup is different. Your environment must be tested and customized to ensure stability and performance.
Overcommitting virtualized CPUs
The KVM hypervisor supports overcommitting virtualized CPUs. Virtualized CPUs can be overcommitted as far as load limits of virtualized guests allow. Use caution when overcommitting VCPUs as loads near 100% may cause dropped requests or unusable response times.
Virtualized CPUs are overcommitted best when each virtualized guest only has a single VCPU. The Linux scheduler is very efficient with this type of load. KVM should safely support guests with loads under 100% at a ratio of five VCPUs. Overcommitting single VCPU virtualized guests is not an issue.
You cannot overcommit symmetric multiprocessing guests on more than the physical number of processing cores. For example a guest with four VCPUs should not be run on a host with a dual core processor. Overcommitting symmetric multiprocessing guests in over the physical number of processing cores will cause significant performance degradation.
Assigning guests VCPUs up to the number of physical cores is appropriate and works as expected. For example, running virtualized guests with four VCPUs on a quad core host. Guests with less than 100% loads should function effectively in this setup.
Do not overcommit memory or CPUs in a production environment without extensive testing. Applications which use 100% of memory or processing resources may become unstable in overcommitted environments. Test before deploying.