Understanding virtual machine snapshots in VMware ESXi and ESX

Symptoms

This article may be helpful when you encounter these issues:

• Virtual machines are not responding or cannot start due to broken parent and child virtual disk dependencies.
• Virtual machines are not responding or do not start due to redo logs residing on datastores that do not have free space.
• Snapshot creation takes too long when specifying the memory snapshot option.
• Snapshot delete or remove operations result in the vSphere or VMware Infrastructure (VI) Client timing out.
• Backups fail while quiescing during a snapshot operation.

Purpose

This article provides information about virtual machine snapshots.

Resolution

  

What is a snapshot?

A snapshot preserves the state and data of a virtual machine at a specific point in time.

• The state includes the virtual machine’s power state (for example, powered-on, powered-off, suspended).
• The data includes all of the files that make up the virtual machine. This includes disks, memory, and other devices, such as virtual network interface cards.

A virtual machine provides several operations for creating and managing snapshots and snapshot chains. These operations let you create snapshots, revert to any snapshot in the chain, and remove snapshots. You can create extensive snapshot trees.

In VMware Infrastructure 3 and vSphere 4.x, the virtual machine snapshot delete operation combines the consolidation of the data and the deletion of the file. This caused issues when the snapshot files are removed from the Snapshot Manager, but the consolidation failed. This left the VM still running on snapshots, and the user may not notice until the datastore is full.

In vSphere 4.x, an alarm can be created to indicate if a virtual machine was running in snapshot mode. For more information, seeConfiguring VMware vCenter Server to send alarms when virtual machines are running from snapshots (1018029).

In vSphere 5.0, enhancements have been made to the snapshot removal. In vSphere 5.0, you are informed via the UI if the consolidation part of a RemoveSnapshot or RemoveAllSnapshots operation has failed. A new option, Consolidate, is available via the Snapshot menu to restart the consolidation.

Creating a snapshot

When creating a snapshot, there are several options you can specify:

• Name: This is used to identify the snapshot.
• Description: This is used to describe the snapshot.
• Memory: If the <memory> flag is 1 or true, a dump of the internal state of the virtual machine is included in the snapshot. Memory snapshots take longer to create.
• Quiesce: If the <quiesce> flag is 1 or true, and the virtual machine is powered on when the snapshot is taken, VMware Tools is used to quiesce the file system in the virtual machine. Quiescing a file system is a process of bringing the on-disk data of a physical or virtual computer into a state suitable for backups. This process might include such operations as flushing dirty buffers from the operating system's in-memory cache to disk, or other higher-level application-specific tasks. 
  
  Note: Quiescing indicates pausing or altering the state of running processes on a computer, particularly those that might modify information stored on disk during a backup, to guarantee a consistent and usable backup.

When a snapshot is created, it is comprised of these files:

• <vm>-<number>.vmdk and <vm>-<number>-delta.vmdk
  A collection of .vmdk and -delta.vmdk files for each virtual disk is connected to the virtual machine at the time of the snapshot. These files can be referred to as child disks, redo logs, or delta links. These child disks can later be considered parent disks for future child disks. From the original parent disk, each child constitutes a redo log pointing back from the present state of the virtual disk, one step at a time, to the original.
  
  Note: The <number> value may not be consistent across all child disks from the same snapshot. The file names are chosen based on filename availability.
  
• <vm>.vmsd
  The .vmsd file is a database of the virtual machine's snapshot information and the primary source of information for the snapshot manager. The file contains line entries which define the relationships between snapshots as well as the child disks for each snapshot.
  
• <vm>Snapshot<number>.vmsn
These files are the memory state at the time of the snapshot.

Note: The above files will be placed in the working directory by default in ESX/ESX 3.x and 4.x. This behavior can be changed if desired. For more information on creating snapshots in another directory, see Creating snapshots in a different location than default virtual machine directory (1002929). In ESXi 5.x and later snapshots descriptor and delta VMDK files will be stored in the same location as the virtual disks (which can be in a different directory to the working directory). To change this behavior, seeChanging the location of snapshot delta files for virtual machines in ESXi 5.0 (2007563)

What products use the snapshot feature?

In addition to being able to use snapshot manager to create snapshots, snapshots are used by many VMware and third-party products and features. Some VMware products that use snapshots extensively are:

• VMware Data Recovery
• VMware Lab Manager
• VMware vCenter and the VMware Infrastructure Client (Snapshot Manager, Storage vMotion)

Note: This is not an exhaustive list.

How do snapshots work?

Our VMware API allows VMware and third-party products to perform operations with virtual machines and their snapshots. This is a list of common operations that can be performed on virtual machines and snapshots using our API:

• CreateSnapshot: Creates a new snapshot of a virtual machine. As a side effect, this updates the current snapshot.
• RemoveSnapshot: Removes a snapshot and deletes any associated storage.
• RemoveAllSnapshots: Remove all snapshots associated with a virtual machine. If a virtual machine does not have any snapshots, then this operation simply returns successfully.
• RevertToSnapshot: Changes the execution state of a virtual machine to the state of this snapshot.
• (vSphere 5.0 only) Consolidate: Merges the hierarchy of redo logs.

This is a high-level overview of how to create, remove, or revert snapshot requests that are processed within the VMware environment:

1. A request to create, remove, or revert a snapshot for a virtual machine is sent from the client to the server using the VMware API. 
2. The request is forwarded to the VMware ESX host that is currently hosting the virtual machine in question.
   
   Note: This only occurs if the original request was sent to a different server, such as vCenter, which is managing the ESX host.
   
3. If the snapshot includes the memory option, the ESX host writes the memory of the virtual machine to disk.
   
   Note: The virtual machine is stunned throughout the duration of time the memory is being written. The length of time of the stun cannot be pre-calculated, and is dependent on the performance of the disk in question and the amount of memory being written. ESX/ESXi 4.x and later have shorter stun times when memory is being written. For more information, seeTaking a snapshot with virtual machine memory stuns the virtual machine while the memory is written to disk (1013163).
   
4. If the snapshot includes the quiesce option, the ESX host requests the guest operating system to quiesce the disks via VMware Tools.
   
   Note: Depending on the guest operating system, the quiescing operation can be done by the sync driver, the vmsync module, or Microsoft's Volume Shadow Copy (VSS) service. For more information on quiescing, see Troubleshooting Volume Shadow Copy (VSS) quiesce related issues (1007696) for VSS or A virtual machine can freeze under load when you take quiesced snapshots or use custom quiescing scripts (5962168) for the SYNC driver.
   
5. The ESX host makes the appropriate changes to the virtual machine's snapshot database (.vmsd file) and the changes are reflected in the snapshot manager of the virtual machine.
   
   Note: When removing a snapshot, the snapshot entity in the snapshot manager is removed before the changes are made to the child disks. The snapshot manager does not contain any snapshot entries while the virtual machine continues to run from the child disk. For more information, see Committing snapshots when there are no snapshot entries in the snapshot manager (1002310).
   
6. The ESX host calls a function similar to the Virtual Disk API functions to make changes to the child disks (-delta.vmdkand .vmdk files) and the disk chain.
   
   Note: During a snapshot removal, if the child disks are large in size, the operation may take a long time. This can result in a timeout error message from either VirtualCenter or the VMware Infrastructure Client. For more information about timeout error messages, see vCenter operation times out with the error: Operation failed since another task is in progress (1004790).

The child disk

The child disk, which is created with a snapshot, is a sparse disk. Sparse disks employ the copy-on-write (COW) mechanism, in which the virtual disk contains no data in places, until copied there by a write. This optimization saves storage space. The grain is the unit of measure in which the sparse disk uses the copy-on-write mechanism. Each grain is a block of sectors containing virtual disk data. The default size is 128 sectors or 64KB.

Child disks and disk usage

It is important to note these points regarding the space utilization of child disks:

• If a virtual machine is running off of a snapshot, it is making changes to a child or sparse disk. The more write operations made to this disk, the larger it grows.
• The space requirements of the child disk are in addition to the parent disk on which it depends. If a virtual machine has a 10 GB disk with a child disk, the space used will be 10 GB + the child disk size.
• Child disks have been known to grow large enough to fill an entire datastore.
• The speed at which child disks grow is directly dependent on the amount of I/O being done to the disk.
• The size of the child disk has a direct impact on the length of time it takes to delete the snapshot associated to the child disk.

These Knowledge Base articles touch on the topic of child disks and disk usage:

• No more space for the redo log error when attempting to start a virtual machine (1002103)
• Why snapshot removal can stop a virtual machine for long time (1002836)
• Troubleshooting a datastore or VMFS volume that is full or near capacity (1003412)

The disk chain

Generally, when you create a snapshot for the first time, the first child disk is created from the parent disk. Successive snapshots generate new child disks from the last child disk on the chain. The relationship can change if you have multiple branches in the snapshot chain.

This diagram is an example of a snapshot chain. Each square represents a block of data or a grain as described above:

 

Caution: Manually manipulating the individual child disks or any of the snapshot configuration files may compromise the disk chain. VMware does not recommend manually modifying the disk chain as it may result in data loss. For more information, seeConsolidating snapshots (1007849).

Additional Information

• To determine if a virtual machine is running on snapshots, see Determining if a virtual machine is using snapshots (1004343). 
  
• There are specific considerations when hosting a Microsoft Active Directory controller in a virtual environment. For a full list of considerations, see Microsoft Knowledge Base article 888794.
  
  Note: The preceding link was valid as of August 1, 2012. If you find the link to be broken, provide feedback on the article and a VMware employee will update the article as necessary.
  
• Time-sensitive applications may be impacted by reverting to a previous snapshot. Reverting the snapshot will revert the virtual machine to the point in time when the snapshot was created. This includes any operations conducted by the time-sensitive service or application in the guest operating system. 
  
• Reverting virtual machines to a snapshot causes all settings configured in the guest operating system since that snapshot to be reverted. The configuration which is reverted includes, but is not limited to, previous IP addresses, DNS names, UUIDs, guest OS patch versions, etc.

• A snapshot operation should not be performed on a virtual machine which uses third-party iSCSI software initiators and is running in VMware Infrastructure 3. You can perform a snapshot operation in a vSphere environment, but it requires additional steps. For more information, see Using a Microsoft or 3rd party software iSCSI Initiator in a VMware environment (1010547) or Running a Third-Party iSCSI initiator in the Virtual Machine section of: 
  
  
  • VMware Infrastucture 3: SAN System Design and Deployment Guide
  • VMware vSphere 4: SAN System Design and Deployment Guide
    
• There have been changes to the Delete All snapshot process in the patch releases for ESX/ESXi 3.5 and 4.0. For more information, see: 
  
  
  • VMware ESX 3.5, Patch ESX350-201006401-SG: Updates VMkernel, VMware Tools, hostd, VMX, VMnix (1022899)
  • VMware ESXi 3.5, Patch ESXe350-201006401-I-SG: Updates firmware (1020052)
  • VMware ESX 4.0, Patch ESX400-201006201-UG: Updates the VMware ESX 4.0 Core and CIM components (1017721)
  • VMware ESXi 4.0, Patch ESXi400-201006201-UG: Updates Firmware (1017739)
    
• For versions prior to VMware ESX 4.0 Update-2, the task of consolidating all snapshots (Remove All Snapshots task) caused unique changes stored only in the second snapshot delta disk to be copied upward through the snapshot chain and into the first snapshot, or its "parent". This effect is recursive for each preceding parent file.
  
  Example: You have a base disk of size 8 GB and 2 levels of snapshots, each of 4 GB each. During a Remove All Snapshot Tasks, the first snapshot delta disk file can grow, worse-case scenario, to 8 GB, as all new blocks from the second snapshot are written. Any common changes stored in both snapshot levels do not require additional space.
  
• From ESX4.0 Update 2 onwards, the snapshot mechanism has changed. VMware ESX now incorporates improved consolidation procedures which lessen the demand of free space. You are able to consolidate virtual machine delta disks even while minimal free space on your datastore is available.

See Also

• No more space for the redo log error when attempting to start a virtual machine
• Committing snapshots when there are no snapshot entries in the Snapshot Manager
• Why snapshot removal can stop a virtual machine for long time
• Creating snapshots in a different location than default virtual machine directory
• Troubleshooting a datastore or VMFS volume that is full or near capacity
• Determining if a virtual machine is using snapshots
• vCenter operation times out with the error: Operation failed since another task is in progress
• Troubleshooting Volume Shadow Copy (VSS) quiesce related issues
• Consolidating snapshots
• Using a Microsoft or 3rd party software iSCSI Initiator in a VMware environment
• Taking a snapshot with virtual machine memory stuns the virtual machine while the memory is written to disk
• VMware ESX 4.0, Patch ESX400-201006201-UG: Updates the VMware ESX 4.0 Core and CIM components
• VMware ESXi 4.0, Patch ESXi400-201006201-UG: Updates Firmware
• Configuring VMware vCenter Server to send alarms when virtual machines are running from snapshots
• VMware ESXi 3.5, Patch ESXe350-201006401-I-SG: Updates firmware
• VMware ESX 3.5, Patch ESX350-201006401-SG: Updates VMkernel, VMware Tools, hostd, VMX, VMnix
• VMware ESX の仮想マシンのスナップショットについて
• Troubleshooting VMware Data Recovery snapshot and VSS errors
• Entendiendo los snapshots o imágenes instantáneas de una máquina virtual en VMware ESX
• Changing the location of snapshot delta files for virtual machines in ESXi 5.0
• Compreendendo instantâneos de máquinas virtuais no VMware ESX
• A virtual machine can freeze under load when you take quiesced snapshots or use custom quiescing scripts

Source:-
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1015180

Growing a local datastore from the command line in vSphere ESXi 4.x and 5.0

Purpose

VMFS Datastores in vSphere 4.x and 5.0 can be increased in size by adding a new extent on a different storage device (spanning), or by increasing the size of the existing storage device and then growing the existing datastore extent to fill that available adjacent capacity.

VMFS Datastore extents may be contained within Primary or Logical partitions, following the MBR/EBR partitioning scheme. VMFS Datastores on the ESX boot device are contained within a Logical partition, and those on an ESXi boot device are contained within a Primary partition.

• Datastore extents within a Primary partition and on a non-Local storage device can be grown into adjacent space using the vSphere Client. For more information, see the Changing VMFS Datastore Properties section of the ESX/ESXi Server Configuration Guide for your version of vSphere.
  
• Datastore extents within Extended and Logical partitions on a Local or Boot storage device cannot be grown into adjacent space using the vSphere Client. This is the default layout for an ESX 4.x installation. For more information, see Growing a local datastore from the command line in vSphere ESX 4.x (1009125).
  
• Datastore extents within Primary partitions on a Local or Boot storage device cannot be grown into adjacent space using the vSphere Client. This is the default layout for an ESXi 4.x and 5.0 installation. This article provides steps for growing an existing VMFS Datastore in a Primary partition to fill available adjacent space on the local boot device.
  

Notes:

1. This article assumes that the underlying storage volume has already had its capacity increased from the hardware perspective, possibly by adding additional disk to a RAID set. For more information, engage your hardware vendor.
   
2. A Datastore on a LUN detected as a snapshot cannot be grown. For more information, see vSphere handling of LUNs detected as snapshot (1011387).
   
3. A Datastore's partitions can only be grown into contiguous adjacent space on the disk. Ensure that the partitions in question are at the end of the disk.

Warning: Be very careful to not overlap the any Primary and Logical partitions. This could result in data loss.

Resolution

To increase the size of a Datastore on a local boot storage device, recreate the partition layout to accommodate the larger filesystem, and then grow the Datastore to fill the larger partition.

1. Use the boot device hardware's management tools to add additional disk capacity to the device. For more information, engage your hardware vendor.
   
2. Open a console to the ESXi host. For more information, see   Using Tech Support Mode in ESXi 4.1 and ESXi 5.0 (1017910)
   
3. Obtain the device identifier for the Datastore to be modified (eg: naa, mpx, eui, vml, etc). For more information, seeIdentifying disks when working with VMware ESX (1014953).
   
   # vmkfstools -P "/vmfs/volumes/DatastoreName
   
   Example output:
   
   VMFS-3.33 file system spanning 1 partitions.
File system label (if any): DatastoreName
Mode: public
Capacity 145223581696 (138496 file blocks * 1048576), 43937431552 (41902 blocks) avail
UUID: 4a14d968-88bf7161-700f-00145ef48f76
Partitions spanned (on "lvm"):
mpx.vmhba0:C0:T0:L0:3
   
4. Record the amount of free disk space on the Datastore. For more information, see Investigating disk space on an ESX or ESXi host (1003564).
   
5. Equipped with the device identifier, identify the existing partitions on the device using the partedUtil command. For more information, see Using the partedUtil command line utility on ESX and ESXi (1036609).
   
   # partedUtil get "/vmfs/devices/disks/mpx.vmhba0:C0:T0:L0"
   
   For example, a disk containing an ESX 4.x installation with 8 existing partitions:
   
15360 64 32 40000001 - Geometry of the disk. Disk size in sectors is 40000001.
4 32 8191 4 128 - Primary #4, Type 4=Fat16<32MB, Bootable, Sectors 32-8191
1 8192 1843199 5 0 - Primary #1, Type 5=Extended, Sectors 8192-1843199
5 8223 520191 6 0 - Logical #5, Type 6=Fat16, Sectors 8223-520191
6 520224 1032191 6 0 - Logical #6, Type 6=Fat16, Sectors 520224-10321917 1032224 1257471 252 0 - Logical #7, Type 252=0xFC=VMKcore, Sectors 1032224-1257471
8 1257504 1843119 6 0 - Logical #8, Type 6=Fat16, Sectors 1257504-1843119
2 1843200 10229759 6 0 - Primary #2, Type 6=Fat16, Sectors 1843200-10229759
3 10229760 31457279 251 0 - Primary #3, Type 251=0xFB=VMFS, Sectors 10229760-31457279
| |        |        |   |
| |        |        |   \--- attribute
| |        |        \------- type
| |        \---------------- ending sector
| \------------------------- starting sector
\--------------------------- partition number
   
6. Identify the partitions which need to be resized, and the size of the space to be used. From the example in step 5, Primary Partition 3 is the last partition on the disk, and there is empty free space between this partition and the end of the disk. For example:
   
   

   		8223	520191	520224	1032191	1032224	1257471	1257504	1843119
   		Logical #5 Type 6 (Fat16)		Logical #6 Type 6 (Fat16)		Logical #7 Type 252 (VMKcore)		Logical #8 Type 6 (Fat16)				Datastore
   63	2249099	2474010						520191		1843200	10229759	10229760	31457279	31457280	40000000
   Primary Partition 4 (Bootable)		Partition 1 Type 5 (Extended)								Primary #2 Type 6 (Fat16)		Primary #3 Type 251 (VMFS)		Empty Space To Be Used

7. Identify the desired ending sector number for the target VMFS Datastore's partitions. To use all space out to the end of the disk, subtract 1 from the disk size in sectors as reported in step 5 to obtain the last usable sector.
   
   For example, Disk sector count 40000001 - 1 = 40000000 as the last usable sector.
   
   Note: With ESXi 5.0 we can use the partedUtil getUsableSectors option to get the last usable sector.
   
   # partedUtil getUsableSectors "/vmfs/devices/disks/mpx.vmhba0:C0:T0:L0"
   
8. Resize the partition containing the target VMFS Datastore using the partedUtil command, specifying the existing starting sector of the partition and the desired ending sector:
   
   # partedUtil resize "/vmfs/devices/disks/Device" PartitionNumber NewStartingSector NewEndingSector
   
   For example, to resize the Primary partition 3 from the example in step 5:
   
   # partedUtil resize "/vmfs/devices/disks/mpx.vmhba0:C0:T0:L0" 3 10229760 40000000
   
9. During step 8, the partedUtil command may report the warning:
   
   The kernel was unable to re-read the partition table on /dev/Device (Device or resource busy).
   
   If you receive this warning, reboot the host before proceeding with the next step. For more information, see Rebooting an ESX Server host (1003530).
   
10. The partition tables have been adjusted, but the VMFS Datastore within the partition is still the same size. There is now empty space within the partition in which the VMFS Datastore can be grown. For example:
    
    

    		8223	520191	520224	1032191	1032224	1257471	1257504	1843119
    		Logical #5 Type 6 (Fat16)		Logical #6 Type 6 (Fat16)		Logical #7 Type 252 (VMKcore)		Logical #8 Type 6 (Fat16)				Datastore	Empty Space
    63	2249099	2474010						520191		1843200	10229759	10229760	40000000
    Primary Partition 4 (Bootable)		Partition 1 Type 5 (Extended)								Primary #2 Type 6 (Fat16)		Primary #3 Type 251 (VMFS)

11. Grow the VMFS Datastore in to the new space using the vmkfstools --growfs command, specifying the partition containing the target VMFS Datastore twice.
    
    # vmkfstools --growfs "/vmfs/devices/disks/device:partition" "/vmfs/devices/disks/device:partition"
    
    For example:
    
    # vmkfstools --growfs "/vmfs/devices/disks/mpx.vmhba0:C0:T0:L0:3" "/vmfs/devices/disks/mpx.vmhba0:C0:T0:L0:3"
    
12. Validate that the size of the VMFS Datastore has increased. For more information, see Investigating disk space on an ESX or ESXi host (1003564).
    
    Note: Click the Refresh button in the vSphere Client to update the Datastore capacity and usage.

See Also

• Rebooting an ESX Server host
• Investigating disk space on an ESX or ESXi host
• Growing a local datastore from the command line in vSphere ESX 4.x
• vSphere handling of LUNs detected as snapshot
• Identifying disks when working with VMware ESX
• Using Tech Support Mode in ESXi 4.1 and ESXi 5.0
• Using the partedUtil command line utility on ESX and ESXi

Source:-
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2002461

VMFS Volume is Locked

Details

The event indicates that a VMFS volume on the ESX/ESXi host is currently locked due to an I/O error.

For example, if naa.60060160b3c018009bd1e02f725fdd11:1 represents one of the partitions used in a VMFS volume, then the following is displayed if the VMFS volume is inaccessible:

Volume on device naa.60060160b3c018009bd1e02f725fdd11:1 locked, possibly because remote host 10.17.211.73 encountered an error during a volume operation and couldn’t recover.

If this occurs, the VMFS volume (and the virtual machines residing on the affected volume) are unavailable to the ESX/ESXi host

Solution

To correct this issue you will need to login to the ESX/ESXi console:

• For information on how to login to ESXi 4.1 and 5.0 hosts refer to Using Tech Support Mode in ESXi 4.1 and ESXi 5.0 (1017910).
• For information on how to login to ESXi 4.0 refer to Tech Support Mode for Emergency Support (1003677)

Log in to the terminal of the VMware ESX or ESXi host and run the following commands:

1. Break the existing LVM lock on the datastore:
   
   # vmkfstools –B <vmfs device>
   
   Note: You can also use the parameter --breaklock instead of -B with the vmkfstools command.
   From the error message above, the following command is used:
   
   
   # vmkfstools -B /vmfs/devices/disks/naa.60060160b3c018009bd1e02f725fdd11:1
   
   The following output will be displayed:
   
   
   VMware ESX Question:
   LVM lock on device /vmfs/devices/disks/naa.60060160b3c018009bd1e02f725fdd11:1 will be forcibly broken. Please consult vmkfstools or ESX documentation to understand the consequences of this.
   
   Please ensure that multiple servers aren't accessing this device.
   
   Continue to break lock?
   0) Yes
   1) No
   
   Please choose a number [0-1]:
   
   
2. Enter 0 to break the lock.
   
   
3. Re-read and reload VMFS datastore metadata to memory:
   
   # vmkfstools –V
   
4. From the vSphere UI, Refresh the Storage Datastores View under Configuration tab.

Source:-

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009570

vSphere Storage Appliance (VSA) networking best practices

Purpose

This article provides explanations and guidelines on resolving networking issues you may encounter during a vSphere Storage Appliance (VSA)  installation or during its network configuration.

Resolution

Ensure the vSphere vCenter Server has only a single IP. This is a requirement until the VSA Cluster is deployed. For more info see the VMware vSphere Storage Appliance 1.0 Release Notes.

 

Prior to VSA Manager installation, you must verify that vCenter Server:

• Is using static IP.
• If the static IP is given by DHCP, the hostname must resolve to the IP and the IP to the hostname.
• Is using IPv4 only. VSA 1.0 does not support IPv6.

 

Prior to the VSA Cluster deployment, you must verify verify the vSphere vCenter Server and the ESXi 5.0 hosts which are designated as the VSA nodes:

• Are in the same subnet.
• Have static IP addresses set.
• Have the same Default Gateway configured.

 

Using VLANs:

 

• vSphere vCenter and ESXi management vmkernel port must be in the front-end VLAN before start deploying VSA Cluster. You can put the ESXi management vmkernel port on a VLAN using the DCUI (Direct Console User Interface). On DCUI Select Customize System > Configure Management Network > VLAN (optional). Set the VLAN Id.
• Using VLAN = 0 is the same as having no VLAN.
• All the ESXi 5.0 NICs must be connected to trunking ports on the physical Switch as the ESXi 5.0 host may send packets with different VLANs through its NICs.
• The VLAN protocol on the physical switch must be IEEE 802.1Q and NOT ISL (Inter-Switch Link, Cisco proprietary). ISL is used between Cisco switches.

 

 

If a previous VSA installation has failed, ensure to:

• Remove possible existing Alerts about network redundancy on every ESXi host. Unable to select ESXi host during VSA Cluster Installation (2005092).
• Remove any non-default networking created by the failed installation (remove vSwitch1, front-end and back-end portgroups) on every host.

 

Networking best practices:

• All the ESXi 5.0 NICs must be connected to trunking ports on the physical Switch, because the ESXi 5.0 host may send packets with different VLANs through its NICs.
• The VLAN protocol on the physical switch must be IEEE 802.1Q and NOT ISL (Inter-Switch Link, Cisco proprietary). ISL is used between Cisco switches.
• If during the VSA installation you select DHCP for Feature IP (VMotion), a DHCP server must be running on the front-end network, and the IPs given for VMotion must be static. It is recommended to set up these IPs manually on the VSA installation wizard.

 

Frequently Asked Questions:

 

Q: Can I install VSA Cluster in hosts with more than 1 NIC on vSwitch0 (default setup)?

A: Assuming the host has 4 NICs, if vSwitch0 has 1 or 2 NICs it will work, if it has 3 or 4 NICs the installation will fail. However, it doesn't matter which NICs are in vSwitch0 at the beginning, as the VSA installer may rearrange them differently to the initial configuration.

 

Q: If the ESXi host has more than 4 NICs and some of them are not suitable for VSA (not connected to trunking port or other reason), how does VSA decide which NICs it should take for VSA?

A: VSA does not detect the non suitable NICs and may use them to configure the networking. This means the Cluster may not work if such a NIC gets chosen. All the NICs (present during VSA installation) on the host should have similar configuration.

 

The Selection of nics is as follows:
The 1st nic is assigned to vSwitch0, which is already connected.

 

For the 2nd nic, here is the selection order:
 - a live nic port on a different nic card + connected to a different physical switch; or
 - a live nic port on a different nic card; or
 - a live nic port connected to a different physical switch; or
 - next available nic

 

Note: we can only differentiate between switches if they are cisco switches.

If the incorrect nics are being selected, and you are unable to change the network configuration, easiest solution is to assign the 2 appropriate nics to vSwitch0 before starting the VSA Cluster deployment. The other vSwitch will be assigned the remaining 2 nics. Thus, as long as your selection for vSwitch0 was correct, networking should be all set.

 

Q: If the NICs used for VSA are not selected by the user but automatically picked by the installer, how does it ensure a proper distribution of NICs among the 2 physical switches to ensure networking redundancy (only one physical switch is supported, but does not provide redundancy)?

A: The VSA installer utilizes the Cisco Discovery Protocol (CDP) for Cisco switches and the Link Layer Discovery Protocol (LLDP) for non-Cisco switches to try to determine the optimally redundant network configuration when matching uplinks to vSwitches. Failing to get this information will result in a random matching of uplink to vSwitches.

 

Q: Does the VSA appliance support DNS?

A: No, all data for installation must be supplied as an IP address.

 

 

 

 

Q: How should I do testing/pinging to ensure correct connectivity?

A:

Frontend network: this should be accessible from VC. Thus, simply do a ping from VC to ensure connectivity & no conflicts. 
There could be a scenario where management PortGroup is on a different VLAN than VSA-Front End & VSA-Vmotion. In this case, create the vSwitch1 and VSA-Vmotion PortGroup manually, and ensure that the different ESXi hosts can vmkping each other. This will guarantee that all ESXi hosts have access to frontend VLAN.

 

Backend network: VC can't access backend. Thus, easiest thing to do here (to ensure all hosts have backend connectivity) is to create vSwtich1 and VSA-Vmotion PortGroup manually. However, instead of assigning that PortGroup the frontend VLAN ID, assign it the backend VLAN ID. Once done, ensure all ESXi hosts can vmkping each other. This will guarantee that all ESXi hosts have access to the backend VLAN.

Note: In both cases, it's best to use one NIC at a time, and verify that all NICs that could at any point use either of the VLANs can indeed access them. 

 

For more information, see VMware vSphere Storage Appliance FAQ (2001389)

 

 

---

SCENARIOS

 

Problem A

 

Issue

 

VSA Installation fails with the following error: Failed to ping VSA service on VM 10.10.10.202.

 

Reason

 

vCenter cannot reach the VSA appliances.

 

Resolution

 

It may have many different reasons, but one common is that vCenter should be in the same VLAN as the front-end network. If the vCenter Server is a VM, connect it to a PortGroup with the front-end VLAN If the vCenter Server is a physical host, connect it to a physical port associated with the front-end VLAN.

 

 

Problem B

 

Issue

VSA Installation fails with the following error: CreateClusterEvent event: Failed to complete network configuration for specifiedhost.localhost, hence reverted all other hosts.

 

Reason

At least one of the ESXi hosts networking is not set up correctly. You can find which host from the Datacenter Tasks tab.

 

Resolution

See the suggestions about removing non-default networking after a failed installation. Make sure that the first 4 NICs (as they appear in Host > Configuration > Network Adapters) are suitable for VSA use.

 

 

Problem C

 

Issue

 

• VSA Cluster creation fails with the following error: 
  
  Unable to reconfigure back end interface 
• On VSAManager.log you may see something similar to:

2011-11-02 11:39:38,003 319 [ClusterService] [Thread-2251] INFO - Connect to VSA: 10.1.11.24

2011-11-02 11:39:38,066 325 [ClusterService] [Thread-2251] INFO - Successfully login to VSA: 10.1.11.24

2011-11-02 11:39:38,066 132 [SvaMessagingService] [Thread-2251] INFO - createStorageCluster()

2011-11-02 11:39:38,207 166 [SvaMessagingService] [Thread-2251] ERROR - createCluster() failed.Unable to reconfigure back end interface.

2011-11-02 11:39:38,222 61 [PersistenceService] [Thread-2251] INFO - clear Data - Key: createClusterTask

2011-11-02 11:39:38,269 169 [BaseEventListener] [Thread-2251] INFO - Stopping the JMS listener connection on 10.1.11.24

2011-11-02 11:39:38,269 169 [BaseEventListener] [Thread-2251] INFO - Stopping the JMS listener connection on 10.1.11.22

2011-11-02 11:39:38,269 169 [BaseEventListener] [Thread-2251] INFO - Stopping the JMS listener connection on 10.1.11.24

2011-11-02 11:39:38,269 169 [BaseEventListener] [Thread-2251] INFO - Stopping the JMS listener connection on 10.1.11.22

2011-11-02 11:39:38,269 188 [CreateClusterThread] [Thread-2251] ERROR - Create cluster failed: java.lang.Exception: createCluster() failed.Unable to reconfigure back end interface.

 

Reason

 

The IPs of the back-end interface (192.168.0.1, 192.168.0.2, ...) are already in use in the network (2 or 3 IPs, depending if the VSA cluster has 2 or 3 nodes).

 

Resolution

 

Make sure that these IPs are not assigned or in use by any OS on the back end network. To verify whether the IPs are in use, ping them from any OS with access to the back-end network (if using VLANs), or from vCenter Server (if not using VLANs) to ensure no one responds.

Additional Information

For more information, see the VSA Networking map.

See Also

• VMware vSphere Storage Appliance FAQ
• Cannot select ESXi hosts during VSA cluster installation

Source:-
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2007363