The balloon driver, vmmemctl, is unaware of pinned pages

Details

The balloon driver shipped with VMware Tools is not aware of a guest operating system's pinned pages (sometimes called locked pages). If the maximum balloon size is not adjusted to account for pinned pages, the balloon might over-inflate, causing an out-of-memory condition within the guest operating system.

For example, in ESX Server 3.0.1 the default maximum balloon size is 65% of the guest memory. If, due to pinned pages, less than 65% of the guest memory is available to be reclaimed, the guest might experience out-of-memory errors if the balloon driver attempts to reclaim more memory than is available.

How do pages become pinned?

Certain applications benefit from having their memory pinned so that the operating system does not attempt to swap this memory to disk. The administrator of a Windows system can allow pages to be pinned using the Lock pages in memorypermission. Windows applications can then use the VirtualLock() function. Linux applications can use mlock(2) or theMAP_LOCKED flag with mmap(2) to pin pages. Other operating systems have similar functionality. Large (2MB or 4MB) pages are typically pinned by default.

You should consider implementing the solution described below if both of these conditions are true:

• you overcommit memory on your host
• your guest environment pins memory

Solution

The administrator of a system using pinned pages should specify the maximum balloon size (in megabytes) using thesched.mem.maxmemctl parameter in the virtual machine's .vmx configuration file. This overrides the system-wide default.

For example, consider a virtual machine with:

• 4GB guest memory
• 2GB pinned for a high-performance user application

In this example, you can specify sched.mem.maxmemctl = "1330" to limit the memory that the balloon driver will attempt to swap to the guest swap space. In this case, a value of 1330MB is used because it is about 65% of the 2GB that can safely be swapped to disk (this leaves some headroom for pinned pages belonging to the guest operating system).

This recommendation assumes you have followed the instructions in the Resource Management Guide for ESX Server. Specifically, "...be sure your guest operating systems have sufficient swap space." This swap space must be greater than or equal to:

(the virtual machine's configured memory size - its Reservation) + (the space required to accommodate the workload of the virtual machine)

Furthermore, the maximum balloon size set with sched.mem.maxmemctl must be less than or equal to the configured swap space in the guest operating system. See the Swap Space and Guest Operating System section in the Resource Management Guide for ESX Server for additional information.