Total Pageviews

My YouTube Channel

Wednesday, 13 February 2013

Using Storage vMotion to migrate a virtual machine with many disks timeout

Default Value is 100 Seconds


You may experience these symptoms:
  • Storage vMotion fails.
  • The Storage vMotion operation fails with a timeout between 5-10% or 90-95% complete.
  • On ESX 4.1 you may see the errors:

    In hostd.log

    v ix: [7196 foundryVM.c:10177]: Error VIX_E_INVALID_ARG in VixVM_CancelOps(): One of the parameters was invalid 'vm:/vmfs/volumes/4e417019-4a3c4130-ed96-a4badb51cd0a/Mail02/Mail02.vmx' opID=9BED9F06-000002BE-9d] Failed to unset VM medatadata: FileIO error: Could not find file : /vmfs/volumes/4e417019-4a3c4130-ed96-a4badb51cd0a/Mail02/Mail02-aux.xml.tmp.
     In vmware.log

    vmkernel: 114:03:25:51.489 cpu0:4100)WARNING: FSR: 690: 1313159068180024 S: Maximum switchover time (100 seconds) reached. Failing migration; VM should resume on source.
    vmkernel: 114:03:25:51.489 cpu2:10561)WARNING: FSR: 3281: 1313159068180024 D: The migration exceeded the maximum switchover time of 100 second(s). ESX has preemptively failed the migration to allow the VM to continue running on the source host.
    vmkernel: 114:03:25:51.489 cpu2:10561)WARNING: Migrate: 296: 1313159068180024 D: Failed: Maximum switchover time for migration exceeded(0xbad0109) @0x41800f61cee2
  • vCenter Server logs contain entries similar to:

    [yyyy-mm-dd hh:mm:ss.nnn tttt error 'App'] [MIGRATE] (migrateidentifier) vMotion failed: vmodl.fault.SystemError
    [yyyy-mm-dd hh:mm:ss.nnn tttt verbose 'App'] [VpxVmomi] Throw vmodl.fault.SystemError with:
    (vmodl.fault.SystemError) {
    dynamicType = <unset>,
    reason = "Source detected that destination failed to resume.",
    msg = "A general system error occurred: Source detected that destination failed to resume.


Note: A virtual machine with many virtual disks might be unable to complete a migration with Storage vMotion. The Storage vMotion process requires time to open, close, and process disks during the final copy phase. Storage vMotion migration of virtual machines with many disks might timeout because of this per-disk overhead.
This timeout occurs when the maximum amount of time for switchover to the destination is exceeded. This may occur if there are a large number of provisioning, migration, or power operations occurring on the same datastore as the Storage vMotion. The virtual machine's disk files are reopened during this time, so disk performance issues or large numbers of disks may lead to timeouts.
The default timeout is 100 seconds, and can be modified by changing the fsr.maxSwitchoverSeconds option in the virtual machine configuration to a larger value.
Note: Ensure this change is performed when the virtual machine is powered down.
To modify the fsr.maxSwitchoverSeconds option using the vSphere Client:
  1. Open vSphere Client and connect to the ESX/ESXi host or to vCenter Server.
  2. Locate the virtual machine in the inventory.
  3. Power off the virtual machine.
  4. Right-click the virtual machine and click Edit Settings.
  5. Click the Options tab.
  6. Select the Advanced: General section.
  7. Click the Configuration Parameters button.

    Note: The Configuration Parameters button is disabled when the virtual machine is powered on.
  8. From the Configuration Parameters window, click Add Row.
  9. In the Name field, enter the parameter name:

  10. In the Value field, enter the new timeout value in seconds (for example: 150).
  11. Click the OK buttons twice to save the configuration change.
  12. Power on the virtual machine.
To modify the fsr.maxSwitchoverSeconds option by editing the .vmx file manually:
The virtual machine's .vmx configuration file can be manually edited to add or modify the option. Add the optionfsr.maxSwitchoverSeconds = " <new value>" on its own line.
For more information, see Tips for editing a .vmx file (1714).

Note: To edit a virtual machines configuration file, you need to power off the virtual machine, remove it from Inventory, make the changes to the vmx file, add the virtual machine back to inventory, and then power on the virtual machine again.